Image segmentation method, device, equipment and medium

Abstract

The embodiments of the present application provide an image segmentation method, apparatus, device and medium. In the embodiments of the present application, the adjusted second pixel value corresponding to each first pixel point in the image to be segmented in the preset channel is determined based on the first difference between the first pixel value corresponding to each first pixel point in the image to be segmented in the preset channel and the minimum value of the first pixel value in the image to be segmented, and a preset ratio value greater than 1. Compared with the first pixel value, the difference between the second pixel values ​​corresponding to pixels with different pixel values ​​is larger, which can effectively alleviate the problem of serious dependence of the image segmentation scheme in the prior art on the lighting and color of the image background, thereby improving the accuracy of image segmentation.

Description

Technical Field

[0001] The present invention relates to the field of image segmentation technology, and in particular to an image segmentation method, device, equipment and medium. Background Art

[0002] With the rapid development of image segmentation technology, image segmentation technology has been widely used in the television and film industry, video live broadcast industry, etc. Image segmentation refers to the process of extracting the foreground image of interest from a complete image.

[0003] In related technologies, in order to achieve image segmentation, simple binarization or frame difference method is generally used for image segmentation. However, these two image segmentation methods are heavily dependent on the lighting and color of the image background, and the accuracy and generalization ability of image segmentation are relatively weak. Summary of the invention

[0004] The present application provides an image segmentation method, device and medium to solve the problem that the existing image segmentation method is seriously dependent on the illumination and color of the image background, and the accuracy and generalization ability of image segmentation are also relatively weak.

[0005] The present application provides an image segmentation method, the method comprising:

[0006] Determine a first pixel value corresponding to each first pixel point in the obtained image to be segmented in a preset channel;

[0007] For each first pixel in the image to be segmented, determine an adjusted second pixel value corresponding to the first pixel in the preset channel according to a first difference between a first pixel value corresponding to the first pixel in the preset channel and a minimum value of the first pixel values ​​in the image to be segmented that correspond to the preset channel, and a preset ratio value greater than 1;

[0008] According to the second pixel value corresponding to each first pixel point in the image to be segmented in the preset channel, the image to be segmented is segmented to obtain a foreground image.

[0009] Further, the preset channel is channel A;

[0010] The method further comprises: determining that each first pixel point in the image to be segmented is before a first pixel value corresponding to a preset channel.

[0011] It is determined whether the image to be segmented is an image of the color model LAB type. If so, a subsequent step of determining a first pixel value corresponding to each first pixel point in the image to be segmented in a preset channel is performed.

[0012] Furthermore, the method further comprises:

[0013] If the image to be segmented is not a LAB type image, the image to be segmented is converted into a LAB type image to be segmented, and a subsequent step of determining a first pixel value corresponding to each first pixel point in the image to be segmented in a preset channel is performed.

[0014] Furthermore, the preset ratio value greater than 1 is determined according to a second difference between a maximum value of the first pixel value, a minimum value of the first pixel value and a preset parameter.

[0015] Furthermore, performing image segmentation on the image to be segmented to obtain the foreground image according to the second pixel value corresponding to the preset channel of each first pixel point in the image to be segmented includes:

[0016] According to the second pixel value corresponding to each first pixel point in the image to be segmented in the preset channel and the preset pixel value threshold, binarizing the image to be segmented to obtain a mask image corresponding to the image to be segmented;

[0017] According to the mask image, the image to be segmented is segmented to obtain a foreground image.

[0018] Furthermore, the process of determining the preset pixel value threshold includes:

[0019] Based on the second pixel value corresponding to each first pixel point in the image to be segmented in the preset channel and the maximum inter-class variance algorithm, the optimal segmentation threshold of the image to be segmented is determined, and the optimal segmentation threshold is determined as the preset pixel value threshold.

[0020] Furthermore, the method further comprises:

[0021] Determine each second pixel point in the edge area of ​​the object contained in the foreground image, and obtain a second pixel value and position information of each second pixel point;

[0022] Inputting the second pixel value and position information corresponding to each second pixel point into a pre-trained category determination model, and obtaining a probability value that the first pixel point corresponding to each second pixel point in the image to be segmented belongs to the foreground area of ​​the image to be segmented;

[0023] For each second pixel point, if the probability value of the first pixel point corresponding to the second pixel point in the image to be segmented belongs to the foreground area of ​​the image to be segmented is not greater than a preset probability value threshold, the second pixel value of the second pixel point is adjusted to a preset value; otherwise, the second pixel value corresponding to the second pixel point is kept unchanged.

[0024] The present application provides an image segmentation device, which includes:

[0025] A determination module is used to determine the first pixel value corresponding to each first pixel point in the obtained image to be segmented in the preset channel; for each first pixel point in the image to be segmented, according to the first difference between the first pixel value corresponding to the first pixel point in the preset channel and the minimum value of the first pixel value corresponding to the preset channel in the image to be segmented, and a preset ratio value greater than 1, determine the adjusted second pixel value corresponding to the first pixel point in the preset channel;

[0026] The segmentation module is used to perform image segmentation on the image to be segmented to obtain a foreground image according to a second pixel value corresponding to each first pixel point in the image to be segmented in a preset channel.

[0027] Furthermore, the device also includes:

[0028] The judging module is used to judge whether the image to be segmented is an image of the color model LAB type, and if so, trigger the determining module.

[0029] Furthermore, the judging module is further configured to convert the image to be segmented into an image to be segmented of the LAB type if the image to be segmented is not an image of the LAB type, and trigger the determining module.

[0030] Furthermore, the segmentation module is specifically used to binarize the image to be segmented according to the second pixel value corresponding to each first pixel point in the image to be segmented in a preset channel and a preset pixel value threshold, to obtain a mask image corresponding to the image to be segmented; and according to the mask image, to perform image segmentation on the image to be segmented to obtain a foreground image.

[0031] Furthermore, the determination module is specifically used to determine the optimal segmentation threshold of the image to be segmented based on the second pixel value corresponding to each first pixel point in the image to be segmented in the preset channel and the maximum inter-class variance algorithm, and determine the optimal segmentation threshold as the preset pixel value threshold.

[0032] Furthermore, the determination module is also used to determine each second pixel point in the edge area of ​​the object contained in the foreground image, and obtain the second pixel value and position information of each second pixel point; input the second pixel value and position information corresponding to each second pixel point into a pre-trained category determination model, and obtain the probability value of the first pixel point corresponding to each second pixel point in the image to be segmented belonging to the foreground area of ​​the image to be segmented; for each second pixel point, if the probability value of the first pixel point corresponding to the second pixel point in the image to be segmented belonging to the foreground area of ​​the image to be segmented is not greater than a preset probability value threshold, the second pixel value of the second pixel point is adjusted to a preset value, otherwise, the second pixel value corresponding to the second pixel point is kept unchanged.

[0033] The present application provides an electronic device, which includes a processor, and the processor is used to implement the steps of any of the above-mentioned image segmentation methods when executing a computer program stored in a memory.

[0034] The present application provides a computer-readable storage medium storing a computer program executable by a terminal. When the program is run on the terminal, the terminal executes any one of the steps of the above-mentioned image segmentation method.

[0035] In the embodiment of the present application, the first pixel value corresponding to each first pixel point in the image to be segmented in the preset channel is determined, and for each first pixel point in the image to be segmented, the second pixel value corresponding to the first pixel point in the preset channel after adjustment is determined according to the first difference between the first pixel value corresponding to the first pixel point and the minimum value of the first pixel value in the image to be segmented, and a preset ratio value greater than 1, and the image to be segmented is segmented to obtain a foreground image according to the second pixel value corresponding to each first pixel point in the image to be segmented in the preset channel. Since in the embodiment of the present application, the second pixel value corresponding to the first pixel point in the preset channel after adjustment is determined according to the first difference between the first pixel value corresponding to each first pixel point in the image to be segmented in the preset channel and the minimum value of the first pixel value in the image to be segmented, and a preset ratio value greater than 1, compared with the first pixel value, the difference between the second pixel values ​​corresponding to the pixels with different pixel values ​​is greater, which can effectively alleviate the problem that the image segmentation scheme in the prior art is seriously dependent on the illumination and color of the image background, and improve the accuracy of image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 A schematic diagram of a process of an image segmentation method provided in some embodiments of the present application;

[0038] Figure 2 A schematic diagram of a process of image segmentation and image synthesis based on a foreground image obtained by segmentation and a preset background image provided in some embodiments of the present application;

[0039] Figure 3 A schematic diagram of a process of synthesizing a synthetic image into a target video provided in some embodiments of the present application;

[0040] Figure 4A schematic diagram of the structure of an image segmentation device provided in some embodiments of the present application;

[0041] Figure 5 A schematic diagram of the structure of an electronic device provided for some embodiments of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application belong to the scope of protection of this application.

[0043] In an embodiment of the present application, a first pixel value corresponding to each first pixel point in the image to be segmented in a preset channel is determined, and for each first pixel point in the image to be segmented, a first difference between the first pixel value corresponding to the first pixel point and the minimum value of the first pixel value in the image to be segmented, and a preset ratio value greater than 1 is used to determine the adjusted second pixel value corresponding to the first pixel point in the preset channel, and according to the second pixel value corresponding to each first pixel point in the image to be segmented in the preset channel, image segmentation is performed on the image to be segmented to obtain a foreground image.

[0044] Embodiment 1:

[0045] Figure 1 A schematic diagram of a process of an image segmentation method provided in some embodiments of the present application, the process comprising the following steps:

[0046] S101: Determine a first pixel value corresponding to each first pixel point in the obtained image to be segmented in a preset channel.

[0047] The embodiment of the present application provides an image segmentation method applied to an electronic device, which may be a smart terminal, a PC, or a server, etc.

[0048] After the electronic device acquires the image to be segmented, it performs image segmentation on the image to be segmented, wherein the image to be segmented may be an image sent by other devices and received by the electronic device, may be pre-stored locally in the electronic device, or may be an image captured by the electronic device.

[0049] In order to facilitate the subsequent segmentation of the image to be segmented, the pixel value corresponding to each first pixel point in the image to be segmented can be obtained first, and the image to be segmented can be segmented based on the pixel value corresponding to each first pixel point. In order to improve the accuracy of image segmentation, in an embodiment of the present application, the first pixel value corresponding to each first pixel point in the image to be segmented in a preset channel can be determined, and the image to be segmented can be segmented based on the first pixel value corresponding to each first pixel point in the preset channel, wherein the preset channel can be any one of the R channel, the G channel, and the B channel.

[0050] S102: For each first pixel point in the image to be segmented, determine the adjusted second pixel value of the first pixel point in the preset channel according to a first difference between the first pixel value corresponding to the first pixel point in the preset channel and the minimum value of the first pixel value corresponding to the preset channel in the image to be segmented, and a preset ratio value greater than 1.

[0051] Generally, the greater the difference between the pixel values ​​of the pixels corresponding to the foreground area and the background area in the image to be segmented, the more conducive to image segmentation. Therefore, in the embodiment of the present application, in order to increase the difference between the pixel values ​​of the pixels corresponding to the foreground area and the background area, the minimum value of the first pixel value corresponding to the preset channel in the image to be segmented can be determined for the first pixel value corresponding to each first pixel in the image to be segmented in the preset channel, and the first difference between the first pixel value corresponding to the first pixel in the preset channel and the minimum value of the first pixel value corresponding to the preset channel in the image to be segmented is determined. According to the first difference and the preset ratio value greater than 1, the adjusted second pixel value corresponding to the first pixel in the preset channel is determined. For the convenience of description, the first pixel value corresponding to the first pixel in the preset channel is directly described as the first pixel value of the first pixel. Since the difference between the pixel values ​​of the pixels corresponding to the foreground area and the background area becomes larger, the problem of serious dependence on the illumination and color of the image background in the existing image segmentation technology is effectively alleviated.

[0052] In a possible implementation, after determining the first difference, the second pixel value of the first pixel after adjustment corresponding to the preset channel may be determined by multiplying the first difference and a preset ratio value greater than 1. The preset ratio value greater than 1 may be 10, 20, etc. Specifically, the preset ratio value greater than 1 is set according to demand.

[0053] S103: performing image segmentation on the image to be segmented to obtain a foreground image according to a second pixel value corresponding to each first pixel point in the image to be segmented in a preset channel.

[0054] For the convenience of description, the second pixel value corresponding to the first pixel in the preset channel is directly used to describe the second pixel value of the first pixel.

[0055] In order to perform image segmentation on the image to be segmented, in an embodiment of the present application, after determining the second pixel value of each first pixel point in the image to be segmented, the image to be segmented is segmented to obtain a foreground image according to the second pixel value of each first pixel point in the image to be segmented.

[0056] Specifically, the image to be segmented may be segmented to obtain the foreground image according to the second pixel value of each first pixel point and an edge segmentation method or a region segmentation method.

[0057] Among them, the process of performing image segmentation on the image to be segmented to obtain the foreground image, such as the edge-based segmentation method or the region-based segmentation method, is a prior art and will not be described in detail here.

[0058] In the embodiment of the present application, the adjusted second pixel value corresponding to the preset channel of each first pixel point in the image to be segmented is determined based on the first difference between the first pixel value corresponding to the preset channel and the minimum value of the first pixel value in the image to be segmented, and a preset ratio value greater than 1. Compared with the first pixel value, the difference between the second pixel values ​​corresponding to pixels with different pixel values ​​is larger, which can effectively alleviate the problem of serious dependence of the image segmentation scheme in the prior art on the lighting and color of the image background, thereby improving the accuracy of image segmentation.

[0059] Embodiment 2:

[0060] In order to determine the first pixel value, based on the above embodiment, in the embodiment of the present application, the preset channel is the color A channel;

[0061] The method further comprises: determining that each first pixel point in the image to be segmented is before a first pixel value corresponding to a preset channel.

[0062] It is determined whether the image to be segmented is an image of the color model LAB type. If so, a subsequent step of determining a first pixel value corresponding to each first pixel point in the image to be segmented in a preset channel is performed.

[0063] Since the colors in LAB type images can be well distinguished, and LAB type images are more committed to perceptual uniformity, have a wide color gamut, and are less dependent on background color changes, therefore, in an embodiment of the present application, in order to improve the accuracy of image segmentation, the preset channel can be the A channel.

[0064] Since the image to be segmented may be an RGB type image, or may be a LAB type image, etc., in an embodiment of the present application, in order to determine the first pixel value corresponding to each first pixel point in the image to be segmented in the A channel, before determining the first pixel value corresponding to each first pixel point in the obtained image to be segmented in the preset channel, it is first determined whether the image to be segmented is a LAB type image. If so, the operation of directly determining the first pixel value corresponding to each first pixel point in the obtained image to be segmented in the preset channel is performed, that is, the first pixel value corresponding to each first pixel point in the image to be segmented in the A channel is directly determined.

[0065] In order to determine the first pixel value, based on the above embodiment, in the embodiment of the present application, the method further includes:

[0066] If the image to be segmented is not a LAB type image, the image to be segmented is converted into a LAB type image, and a subsequent step of determining a first pixel value corresponding to each first pixel point in the image to be segmented in a preset channel is performed.

[0067] In an embodiment of the present application, in order to determine the first pixel value corresponding to each first pixel point in the image to be segmented in the A channel, if the image to be segmented is not a LAB type image, the other type of image corresponding to the segmented image is first converted into a LAB type image, and then the operation of determining the first pixel value corresponding to each first pixel point in the obtained image to be segmented in the preset channel is performed. That is to say, the non-LAB type image to be segmented is first converted into a LAB type image to be segmented, and then the first pixel value corresponding to each first pixel point in the converted LAB type image to be segmented in the A channel is determined.

[0068] The process of converting the image to be segmented from other types of images to LAB type images is a prior art and will not be described in detail here.

[0069] Embodiment 3:

[0070] In order to improve the accuracy of image segmentation, based on the above embodiments, the preset ratio value greater than 1 is determined according to the maximum value of the first pixel value, the minimum value of the first pixel value and the second difference of the preset parameter.

[0071] In the embodiment of the present application, since there are great differences in the first pixel values ​​corresponding to the first pixel points in different images to be segmented, when determining the adjusted second pixel values ​​corresponding to the first pixel points in different images to be segmented in the preset channels, after determining the first pixel value corresponding to the first pixel point in the image to be segmented and the first difference of the minimum value of the first pixel value in the image to be segmented, if the second pixel value corresponding to the first pixel point in the different images to be segmented in the preset channels is determined directly based on a fixed preset ratio value greater than 1 and the first difference, the image segmentation accuracy of some images to be segmented may be low. Therefore, in order to further improve the accuracy of image segmentation, the preset ratio value greater than 1 corresponding to the segmented image can be determined according to the specific situation of the first pixel value corresponding to the first pixel point contained in the image to be segmented.

[0072] In an embodiment of the present application, in order to determine the preset ratio value greater than 1 corresponding to the image to be segmented according to the specific situation of the first pixel value corresponding to the first pixel point contained in the image to be segmented, it can be determined according to the maximum value of the first pixel value in the image to be segmented, the minimum value of the first pixel value in the image to be segmented and the second difference of the preset parameter. Since the maximum value and the minimum value of the first pixel value in different images to be segmented are different, the preset ratio values ​​greater than 1 corresponding to different images to be segmented are also different.

[0073] Specifically, according to Determine the preset ratio value greater than 1;

[0074] Among them, y is a preset ratio value greater than 1, max is the maximum value of the first pixel value in the image to be segmented, min is the minimum value of the first pixel value in the image to be segmented, δ is a preset first parameter, which characterizes the scaling correction amount, ε is a preset second parameter, which is the overall mean correction amount, wherein the value range corresponding to δ is (0,30), and specifically, δ can be set according to needs, and the value range corresponding to ε is (0,50), and specifically, ε can be set according to needs.

[0075] Embodiment 4:

[0076] In order to perform image segmentation on the image to be segmented and thereby obtain a foreground image, based on the above embodiments, in an embodiment of the present application, performing image segmentation on the image to be segmented to obtain a foreground image according to a second pixel value corresponding to a preset channel of each first pixel point in the image to be segmented includes:

[0077] According to the second pixel value corresponding to each first pixel point in the image to be segmented in the preset channel and the preset pixel value threshold, binarizing the image to be segmented to obtain a mask image corresponding to the image to be segmented;

[0078] According to the mask image, the image to be segmented is segmented to obtain a foreground image.

[0079] In an embodiment of the present application, in order to perform image segmentation on an image to be segmented and thereby obtain a foreground image, the foreground area and the background area of ​​the image to be segmented may be first determined, and then the foreground image may be obtained based on the foreground area and the background area to be segmented.

[0080] In the embodiment of the present application, in order to determine the foreground area and the background area of ​​the image to be segmented, the image to be segmented can be binarized according to the second pixel value corresponding to each first pixel point in the image to be segmented in the preset channel and the preset pixel value threshold, to obtain a mask image corresponding to the image to be segmented, wherein the pixel value corresponding to the foreground area in the image to be segmented in the mask image is 1, and the pixel value corresponding to the background area in the mask image is 0. The preset pixel value threshold can be 125, etc., and specifically, the preset pixel value threshold can be set according to needs.

[0081] Specifically, for each first pixel point in the image to be segmented and the second pixel value corresponding to the preset channel, if the second pixel value corresponding to the first pixel point in the preset channel is greater than the preset pixel threshold, the pixel value of the first pixel point at the corresponding position in the mask image is set to 1; if the second pixel value corresponding to the first pixel point in the preset channel is not greater than the preset pixel threshold, the pixel value of the first pixel point at the corresponding position in the mask image is set to 0.

[0082] In an embodiment of the present application, after determining the mask image corresponding to the image to be segmented, the pixel points with pixel values ​​of 1 in the mask image corresponding to the pixel points in the image to be segmented are the pixel points in the foreground area of ​​the image to be segmented, and the pixel points with pixel values ​​of 0 in the mask image corresponding to the pixel points in the image to be segmented are the pixel points in the background area of ​​the image to be segmented.

[0083] In an embodiment of the present application, in order to perform image segmentation on the image to be segmented according to the mask image to obtain a foreground image, specifically, the pixel points corresponding to the pixel points with a pixel value of 1 in the mask image in the image to be segmented are kept unchanged, and the pixel points corresponding to the pixel points with a pixel value of 0 in the mask image in the image to be segmented are set to preset pixel values. In an embodiment of the present application, the preset pixel value can be 255.

[0084] In order to improve the accuracy of image segmentation, based on the above embodiments, in the embodiment of the present application, the process of determining the preset pixel value threshold includes:

[0085] Based on the second pixel value corresponding to each first pixel point in the image to be segmented in the preset channel and the maximum inter-class variance algorithm, the optimal segmentation threshold of the image to be segmented is determined, and the optimal segmentation threshold is determined as the preset pixel value threshold.

[0086] In an embodiment of the present application, since the second pixel values ​​corresponding to the first pixel points in different images to be segmented in the preset channels are very different, if the image to be segmented is directly binarized based on the preset pixel value threshold to obtain a mask image corresponding to the image to be segmented, the accuracy of the determined mask image will be low, thereby affecting the accuracy of subsequent image segmentation.

[0087] In the embodiment of the present application, in order to further improve the accuracy of image segmentation, the preset pixel value threshold can be determined according to the second pixel value corresponding to each first pixel point in the image to be segmented in the preset channel. Specifically, the optimal segmentation threshold of the image to be segmented can be determined based on the second pixel value corresponding to each first pixel point in the image to be segmented in the preset channel and the maximum inter-class variance algorithm, and the optimal segmentation threshold is determined as the preset pixel value threshold.

[0088] In an embodiment of the present application, in the process of determining the optimal segmentation threshold of the image to be segmented based on the second pixel value corresponding to each first pixel point in the image to be segmented in the preset channel and the maximum inter-class variance algorithm, for each second pixel value, the first number of first pixels corresponding to the second pixel value is determined, and the ratio of the first number to the second number of all first pixels contained in the image to be segmented is determined. Each second pixel value in [0,255] is traversed, and the optimal segmentation threshold is determined according to the ratio corresponding to each second pixel value.

[0089] The process of determining the optimal segmentation threshold based on the maximum inter-class variance algorithm is a prior art and will not be described in detail here.

[0090] Embodiment 5:

[0091] In order to improve the accuracy of image segmentation, based on the above embodiments, in the embodiment of the present application, the method further includes:

[0092] Determine each second pixel point in the edge area of ​​the object contained in the foreground image, and obtain a second pixel value and position information of each second pixel point;

[0093] Inputting the second pixel value and position information corresponding to each second pixel point into a pre-trained category determination model, and obtaining a probability value that the first pixel point corresponding to each second pixel point in the image to be segmented belongs to the foreground area of ​​the image to be segmented;

[0094] For each second pixel point, if the probability value of the first pixel point corresponding to the second pixel point in the image to be segmented belongs to the foreground area of ​​the image to be segmented is not greater than a preset probability value threshold, the second pixel value of the second pixel point is adjusted to a preset value; otherwise, the second pixel value corresponding to the second pixel point is kept unchanged.

[0095] Since the boundary between the foreground area and the background area in the image to be segmented is the most difficult to segment and is more likely to affect the accuracy of image segmentation, in the embodiment of the present application, after determining the foreground image, in order to further improve the accuracy of image segmentation, it is determined whether the first pixel corresponding to each second pixel in the edge area of ​​the object contained in the foreground image in the image to be segmented belongs to the foreground area in the image to be segmented. For ease of description, the first pixel corresponding to the second pixel in the image to be segmented is referred to as the first pixel corresponding to the second pixel.

[0096] Specifically, in order to determine whether the first pixel corresponding to each second pixel in the edge area of ​​an object contained in the foreground image belongs to the foreground area, in an embodiment of the present application, a category determination model is pre-trained, and after obtaining the second pixel value and position information of each second pixel, the second pixel value and position information corresponding to each second pixel are input into the pre-trained category determination model, and the category determination model outputs the probability value that the first pixel corresponding to each second pixel belongs to the foreground area of ​​the image to be segmented, wherein the higher the probability value, the greater the possibility that the first pixel corresponding to the second pixel belongs to the foreground area of ​​the image to be segmented.

[0097] In an embodiment of the present application, in order to accurately determine the probability value of each first pixel point belonging to the foreground area of ​​the image to be segmented, the category determination model is trained in advance. Specifically, a sample image set is saved in advance, and for any sample pixel point in any sample image contained in the sample image set, the sample pixel value and sample position information corresponding to the sample pixel point in the sample image are input into the original category determination model. The original category determination model outputs an estimated probability value of the sample pixel point belonging to the foreground area or the background area of ​​the sample image. According to the difference between the estimated probability value of the sample pixel point belonging to the foreground area or the background area of ​​the sample image and the actual probability value of the sample pixel point belonging to the foreground area or the background area of ​​the sample image that is marked in advance, the loss value corresponding to the sample pixel point is determined.

[0098] According to the loss value corresponding to each sample pixel of each sample image, a target loss value is determined, and based on the target loss value, the parameters of the original category determination model are adjusted. The smaller the target loss value, the higher the accuracy of the category determination model.

[0099] Specifically, according to Determine the loss value corresponding to each sample image;

[0100] Among them, y j is the loss value corresponding to the jth sample image in the sample image set, L(w|f i,j ) is the difference between the estimated probability value of the sample pixel point i belonging to the foreground area or the background area of ​​the sample image j and the actual probability value of the pre-marked sample pixel point i belonging to the foreground area or the background area of ​​the sample image j, and N is the total number of sample pixels contained in the sample image j.

[0101] according to Determine the target loss value;

[0102] Among them, Y is the target loss value, y j is the loss value corresponding to the jth sample image, and M is the number of sample images contained in the sample image set.

[0103] In order to determine whether the first pixel point corresponding to each second pixel point belongs to the foreground area of ​​the image to be segmented, in an embodiment of the present application, a probability value threshold is pre-set, wherein the pre-set probability value threshold can be 0.9 or 0.85, etc. Specifically, the pre-set probability value threshold can be set according to needs.

[0104] In an embodiment of the present application, after the foreground image is determined in the foreground area of ​​the image to be segmented, the first pixel value of each first pixel point contained in the foreground area is kept unchanged, and the first pixel value of each first pixel point contained in the background area is set to a preset pixel value. In order to facilitate the subsequent synthesis of the foreground image with other set background images, the preset pixel value can be 255.

[0105] For each second pixel point, determine whether the probability value of the first pixel point corresponding to the second pixel point belonging to the foreground area of ​​the image to be segmented is not greater than a preset probability value threshold, if so, it means that the first pixel point corresponding to the second pixel point does not belong to the foreground area of ​​the image to be segmented, and the second pixel value of the second pixel point is adjusted to a preset value, wherein the preset value may be 255. Otherwise, it means that the first pixel point corresponding to the second pixel point belongs to the foreground area of ​​the image to be segmented, and the second pixel value corresponding to the second pixel point remains unchanged.

[0106] Figure 2 A schematic diagram of a process of image segmentation and image synthesis based on a foreground image obtained by segmentation and a preset background image provided in some embodiments of the present application is now directed to Figure 2 Provide explanation.

[0107] Taking the image to be segmented obtained by the electronic device as an RGB image as an example, after receiving the RGB type image to be segmented, the electronic device converts the RGB type image to be segmented into a LAB type image to be segmented. Determine the first pixel value corresponding to each first pixel point in the obtained image to be segmented in the A channel, for each first pixel point in the image to be segmented, according to the first pixel value corresponding to the first pixel point and the minimum value of the first pixel value in the image to be segmented, determine the adjusted second pixel value corresponding to the first pixel point in A, and according to the second pixel value corresponding to each first pixel point in the image to be segmented in the A channel, perform image segmentation on the image to be segmented to obtain a foreground image.

[0108] Image synthesis is performed according to the foreground image and the preset background image. Specifically, image synthesis is performed according to W=background*(1-α)+foreground*α, wherein W is the image matrix corresponding to the synthesized image, background is the image matrix corresponding to the preset background image, foreground is the image matrix corresponding to the foreground image, and α is the preset parameter matrix, wherein for each image matrix in the image matrix corresponding to the preset background image and the image matrix corresponding to the foreground image, each element in the image matrix is ​​the pixel value of the pixel point in the corresponding image. α is the same as the matrix corresponding to the mask image corresponding to the image to be segmented, which is a matrix composed of two values ​​1 and 0.

[0109] Figure 3 A schematic diagram of a process of synthesizing a synthetic image into a target video is provided in some embodiments of the present application. Figure 3 Provide explanation.

[0110] After segmenting multiple consecutive images to be segmented in a video, multiple foreground images are obtained, and the multiple foreground images are synthesized with a preset background image to obtain multiple synthesized images. For each synthesized image, the time of the image to be segmented corresponding to the synthesized image is determined as the target time corresponding to the synthesized image. According to the target time corresponding to the multiple synthesized images, the multiple synthesized images are synthesized in time order to generate a target video. The multiple synthesized images can also be synthesized based on a cross-platform computer vision and machine learning software library (opencv) or a multimedia video processing tool (ffmpeg) to generate a target video.

[0111] Embodiment 6:

[0112] Figure 4 A schematic diagram of the structure of an image segmentation device provided in some embodiments of the present application, the device comprising:

[0113] The determination module 401 is used to determine the first pixel value corresponding to each first pixel point in the obtained image to be segmented in the preset channel; for each first pixel point in the image to be segmented, according to the first difference between the first pixel value corresponding to the first pixel point in the preset channel and the minimum value of the first pixel value corresponding to the preset channel in the image to be segmented, and a preset ratio value greater than 1, determine the adjusted second pixel value corresponding to the first pixel point in the preset channel;

[0114] The segmentation module 402 is used to perform image segmentation on the image to be segmented to obtain a foreground image according to a second pixel value corresponding to each first pixel point in the image to be segmented in a preset channel.

[0115] In a possible implementation, the device further includes:

[0116] The judging module 403 is used to judge whether the image to be segmented is an image of the color model LAB type, and if so, trigger the determining module 401.

[0117] In a possible implementation manner, the judging module 403 is further configured to convert the image to be segmented into an image to be segmented of the LAB type if the image to be segmented is not an image of the LAB type, and trigger the determining module 401 .

[0118] In a possible implementation, the segmentation module 402 is specifically used to perform binarization processing on the image to be segmented according to the second pixel value corresponding to each first pixel point in the image to be segmented in a preset channel and a preset pixel value threshold, to obtain a mask image corresponding to the image to be segmented; and perform image segmentation on the image to be segmented according to the mask image to obtain a foreground image.

[0119] In a possible implementation, the determination module 401 is specifically used to determine the optimal segmentation threshold of the image to be segmented based on the second pixel value corresponding to each first pixel point in the image to be segmented in a preset channel and the maximum inter-class variance algorithm, and determine the optimal segmentation threshold as the preset pixel value threshold.

[0120] In a possible implementation, the determination module 401 is further used to determine each second pixel point in the edge area of ​​the object contained in the foreground image, and obtain the second pixel value and position information of each second pixel point; input the second pixel value and position information corresponding to each second pixel point into a pre-trained category determination model, and obtain the probability value that the first pixel point corresponding to each second pixel point in the image to be segmented belongs to the foreground area of ​​the image to be segmented; for each second pixel point, if the probability value that the first pixel point corresponding to the second pixel point in the image to be segmented belongs to the foreground area of ​​the image to be segmented is not greater than a preset probability value threshold, the second pixel value of the second pixel point is adjusted to a preset value, otherwise, the second pixel value corresponding to the second pixel point is kept unchanged.

[0121] Embodiment 7:

[0122] Based on the above embodiments, some embodiments of the present application further provide an electronic device, such as Figure 5 As shown, it includes: a processor 501 , a communication interface 502 , a memory 503 and a communication bus 504 , wherein the processor 501 , the communication interface 502 , and the memory 503 communicate with each other via the communication bus 504 .

[0123] The memory 503 stores a computer program. When the program is executed by the processor 501, the processor 501 performs the following steps:

[0124] Determine a first pixel value corresponding to each first pixel point in the obtained image to be segmented in a preset channel;

[0125] For each first pixel in the image to be segmented, determine an adjusted second pixel value corresponding to the first pixel in the preset channel according to a first difference between a first pixel value corresponding to the first pixel in the preset channel and a minimum value of the first pixel values ​​in the image to be segmented that correspond to the preset channel, and a preset ratio value greater than 1;

[0126] According to the second pixel value corresponding to each first pixel point in the image to be segmented in the preset channel, the image to be segmented is segmented to obtain a foreground image.

[0127] Furthermore, the processor 501 is also used to determine whether the image to be segmented is an image of the color model LAB type. If so, a subsequent step of determining a first pixel value corresponding to each first pixel point in the image to be segmented in a preset channel is performed.

[0128] Furthermore, the processor 501 is also used to convert the image to be segmented into a LAB type image to be segmented if the image to be segmented is not a LAB type image, and perform a subsequent step of determining a first pixel value corresponding to each first pixel point in the image to be segmented in a preset channel.

[0129] Furthermore, the processor 501 is also used to perform binarization processing on the image to be segmented according to the second pixel value corresponding to each first pixel point in the image to be segmented in a preset channel and a preset pixel value threshold, to obtain a mask image corresponding to the image to be segmented; and perform image segmentation on the image to be segmented according to the mask image to obtain a foreground image.

[0130] Furthermore, the processor 501 is also used to determine the optimal segmentation threshold of the image to be segmented based on the second pixel value corresponding to each first pixel point in the image to be segmented in the preset channel and the maximum inter-class variance algorithm, and determine the optimal segmentation threshold as the preset pixel value threshold.

[0131] Furthermore, the processor 501 is also used to determine each second pixel point in the edge area of ​​the object contained in the foreground image, and obtain the second pixel value and position information of each second pixel point; input the second pixel value and position information corresponding to each second pixel point into a pre-trained category determination model, and obtain the probability value of the first pixel point corresponding to each second pixel point in the image to be segmented belonging to the foreground area of ​​the image to be segmented; for each second pixel point, if the probability value of the first pixel point corresponding to the second pixel point in the image to be segmented belonging to the foreground area of ​​the image to be segmented is not greater than a preset probability value threshold, the second pixel value of the second pixel point is adjusted to a preset value, otherwise, the second pixel value corresponding to the second pixel point is kept unchanged.

[0132] The communication bus mentioned in the above server can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0133] The communication interface 502 is used for communication between the electronic device and other devices.

[0134] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0135] The above-mentioned processor can be a general-purpose processor, including a central processing unit, a network processor (Network Processor, NP), etc.; it can also be a digital signal processing processor (Digital Signal Processing, DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.

[0136] Embodiment 8:

[0137] On the basis of the above embodiments, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program executable by an electronic device, and when the program is run on the electronic device, the electronic device implements the following steps when executing:

[0138] The memory stores a computer program, and when the program is executed by the processor, the processor performs the following steps:

[0139] Determine a first pixel value corresponding to each first pixel point in the obtained image to be segmented in a preset channel;

[0140] For each first pixel in the image to be segmented, determine an adjusted second pixel value corresponding to the first pixel in the preset channel according to a first difference between a first pixel value corresponding to the first pixel in the preset channel and a minimum value of the first pixel values ​​in the image to be segmented that correspond to the preset channel, and a preset ratio value greater than 1;

[0141] According to the second pixel value corresponding to each first pixel point in the image to be segmented in the preset channel, the image to be segmented is segmented to obtain a foreground image.

[0142] Further, the preset channel is channel A;

[0143] The method further comprises: determining that each first pixel point in the image to be segmented is before a first pixel value corresponding to a preset channel.

[0144] It is determined whether the image to be segmented is an image of the color model LAB type. If so, a subsequent step of determining a first pixel value corresponding to each first pixel point in the image to be segmented in a preset channel is performed.

[0145] Furthermore, the method further comprises:

[0146] If the image to be segmented is not a LAB type image, the image to be segmented is converted into a LAB type image to be segmented, and a subsequent step of determining a first pixel value corresponding to each first pixel point in the image to be segmented in a preset channel is performed.

[0147] Furthermore, the preset ratio value greater than 1 is determined according to a second difference between a maximum value of the first pixel value, a minimum value of the first pixel value and a preset parameter.

[0148] Furthermore, performing image segmentation on the image to be segmented to obtain the foreground image according to the second pixel value corresponding to the preset channel of each first pixel point in the image to be segmented includes:

[0149] According to the second pixel value corresponding to each first pixel point in the image to be segmented in the preset channel and the preset pixel value threshold, binarizing the image to be segmented to obtain a mask image corresponding to the image to be segmented;

[0150] According to the mask image, the image to be segmented is segmented to obtain a foreground image.

[0151] Furthermore, the process of determining the preset pixel value threshold includes:

[0152] Based on the second pixel value corresponding to each first pixel point in the image to be segmented in the preset channel and the maximum inter-class variance algorithm, the optimal segmentation threshold of the image to be segmented is determined, and the optimal segmentation threshold is determined as the preset pixel value threshold.

[0153] Furthermore, the method further comprises:

[0154] Determine each second pixel point in the edge area of ​​the object contained in the foreground image, and obtain a second pixel value and position information of each second pixel point;

[0155] Inputting the second pixel value and position information corresponding to each second pixel point into a pre-trained category determination model, and obtaining a probability value that the first pixel point corresponding to each second pixel point in the image to be segmented belongs to the foreground area of ​​the image to be segmented;

[0156] For each second pixel point, if the probability value of the first pixel point corresponding to the second pixel point in the image to be segmented belongs to the foreground area of ​​the image to be segmented is not greater than a preset probability value threshold, the second pixel value of the second pixel point is adjusted to a preset value; otherwise, the second pixel value corresponding to the second pixel point is kept unchanged.

[0157] In the embodiment of the present application, the adjusted second pixel value corresponding to the preset channel of each first pixel point in the image to be segmented is determined based on the first difference between the first pixel value corresponding to the preset channel and the minimum value of the first pixel value in the image to be segmented, and a preset ratio value greater than 1. Compared with the first pixel value, the difference between the second pixel values ​​corresponding to pixels with different pixel values ​​is larger, which can effectively alleviate the problem of serious dependence of the image segmentation scheme in the prior art on the lighting and color of the image background, thereby improving the accuracy of image segmentation.

[0158] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0159] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0160] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0161] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1The steps for the functions specified in one or more boxes.

[0162] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. An image segmentation method, characterized in that: The method comprises: Determine a first pixel value corresponding to each first pixel point in the obtained image to be segmented in a preset channel; For each first pixel point in the image to be segmented, determine the adjusted second pixel value corresponding to the first pixel point in the preset channel according to a first difference between a first pixel value corresponding to the first pixel point in the preset channel and a minimum value of the first pixel value corresponding to the preset channel in the image to be segmented, and a product of a preset ratio value greater than 1; wherein the preset ratio value greater than 1 is determined according to a second difference between a maximum value of the first pixel value and a minimum value of the first pixel value and a preset parameter; Determining a mask image corresponding to the image to be segmented according to a second pixel value corresponding to each first pixel point in the image to be segmented in a preset channel, and performing image segmentation on the image to be segmented based on the mask image to obtain a foreground image; Wherein, each first pixel point in the determined image to be segmented is before the first pixel value corresponding to the preset channel, and the method further includes: Determine whether the image to be segmented is an image of the color model LAB type, and if so, perform a subsequent step of determining a first pixel value corresponding to each first pixel point in the image to be segmented in a preset channel; If the image to be segmented is not a LAB type image, the image to be segmented is converted into a LAB type image to be segmented, and a subsequent step of determining a first pixel value corresponding to each first pixel point in the image to be segmented in a preset channel is performed.

2. The method according to claim 1, characterized in that The preset channel is channel A.

3. The method according to claim 1, characterized in that The step of determining a mask image corresponding to the image to be segmented according to a second pixel value corresponding to a preset channel of each first pixel point in the image to be segmented, and performing image segmentation on the image to be segmented based on the mask image to obtain a foreground image comprises: According to the second pixel value corresponding to each first pixel point in the image to be segmented in the preset channel and the preset pixel value threshold, binarizing the image to be segmented to obtain a mask image corresponding to the image to be segmented; According to the mask image, the image to be segmented is segmented to obtain a foreground image.

4. The method according to claim 3, characterized in that The process of determining the preset pixel value threshold includes: Based on the second pixel value corresponding to each first pixel point in the image to be segmented in the preset channel and the maximum inter-class variance algorithm, the optimal segmentation threshold of the image to be segmented is determined, and the optimal segmentation threshold is determined as the preset pixel value threshold.

5. The method according to claim 1, characterized in that The method further comprises: Determine each second pixel point in the edge area of ​​the object contained in the foreground image, and obtain a second pixel value and position information of each second pixel point; Inputting the second pixel value and position information corresponding to each second pixel point into a pre-trained category determination model, and obtaining a probability value that the first pixel point corresponding to each second pixel point in the image to be segmented belongs to the foreground area of ​​the image to be segmented; For each second pixel point, if the probability value of the first pixel point corresponding to the second pixel point in the image to be segmented belongs to the foreground area of ​​the image to be segmented is not greater than a preset probability value threshold, the second pixel value of the second pixel point is adjusted to a preset value; otherwise, the second pixel value corresponding to the second pixel point is kept unchanged.

6. An image segmentation device, characterized in that: The device comprises: A determination module, used to determine the first pixel value corresponding to each first pixel point in the obtained image to be segmented in a preset channel; for each first pixel point in the image to be segmented, determine the adjusted second pixel value corresponding to the first pixel point in the preset channel according to the first difference between the first pixel value corresponding to the first pixel point in the preset channel and the minimum value of the first pixel value corresponding to the preset channel in the image to be segmented, and the product of a preset ratio value greater than 1; wherein the preset ratio value greater than 1 is determined according to the second difference between the maximum value of the first pixel value and the minimum value of the first pixel value and a preset parameter; A segmentation module, configured to determine a mask image corresponding to the image to be segmented according to a second pixel value corresponding to each first pixel point in the image to be segmented in a preset channel, and perform image segmentation on the image to be segmented based on the mask image to obtain a foreground image; A judging module, used for judging whether the image to be segmented is an image of the color model LAB type, and if so, triggering the determining module; The judging module is further configured to convert the image to be segmented into an image to be segmented of the LAB type if the image to be segmented is not an image of the LAB type, and trigger the determining module.

7. An electronic device, characterized in that: The electronic device comprises a processor, and the processor is used to implement the steps of the image segmentation method according to any one of claims 1 to 5 when executing a computer program stored in a memory.

8. A computer-readable storage medium, characterized in that: It stores a computer program executable by a terminal, and when the program is run on the terminal, the terminal executes the steps of the image segmentation method according to any one of claims 1 to 5.

Images