Image matting method and device based on artificial intelligence, equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, and provides an artificial intelligence-based matting method, and the method comprises the steps: carrying out the preprocessing of a target image, and obtaining a preprocessed image; performing foreground and background segmentation on the preprocessed image based on an artificial intelligence algorithm to obtain a foreground image; performing pixel-level edge optimization on the foreground image according to the pixel values of the pixels and the adjacent pixels in the foreground image to obtain an edge-optimized foreground image; and fusing the preset background image and the edge optimization foreground image by calculating the gradient of the edge optimization foreground image and the preset background image or calculating the pixel weight of the transition region. According to the technical scheme, complex detail parts in the image can be accurately processed, the matting efficiency is improved, and the labor cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an artificial intelligence-based image cutting method, device, equipment and storage medium. Background Art

[0002] With the development of digital image processing technology, image cutout (i.e., image segmentation) has been widely used in many fields, particularly in film and television production, advertising design, virtual reality (VR), augmented reality (AR), product display on e-commerce platforms, and ID photo production. Existing image cutout methods typically rely on techniques such as manual labeling or color segmentation, but these methods often require extensive manual intervention, are inefficient when processing complex images, and are easily affected by background complexity. This can lead to loss of detail, low computational efficiency (especially for high-resolution images, which are slow and often require manual fine-tuning), and poor robustness. Summary of the Invention

[0003] The present application provides an artificial intelligence-based cutout method, apparatus, device, and storage medium, which can accurately process complex details in an image, improve cutout efficiency, and reduce labor costs.

[0004] In one aspect, the present application provides an artificial intelligence-based image cutout method, the method comprising:

[0005] Preprocessing the target image to obtain a preprocessed image;

[0006] Segmenting the preprocessed image into foreground and background based on an artificial intelligence algorithm to obtain a foreground image;

[0007] performing pixel-level edge optimization on the foreground image according to pixel values ​​of pixels in the foreground image and pixel values ​​of adjacent pixels thereof to obtain an edge-optimized foreground image;

[0008] The preset background image and the edge-optimized foreground image are fused by calculating the gradients of the edge-optimized foreground image and the preset background image or by calculating the pixel weights of a transition area.

[0009] On the other hand, the present application provides an artificial intelligence-based image cutting device, the device comprising:

[0010] A preprocessing module is used to preprocess the target image to obtain a preprocessed image;

[0011] A segmentation module, configured to segment the preprocessed image into foreground and background based on an artificial intelligence algorithm to obtain a foreground image;

[0012] an optimization module, configured to perform pixel-level edge optimization on the foreground image according to pixel values ​​of pixels in the foreground image and pixel values ​​of adjacent pixels thereof, to obtain an edge-optimized foreground image;

[0013] A fusion module is used to fuse the preset background image and the edge-optimized foreground image by calculating the gradient of the edge-optimized foreground image and the preset background image or by calculating the pixel weight of the transition area.

[0014] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the technical solution of the above-mentioned artificial intelligence-based cutout method when executing the computer program.

[0015] In a fourth aspect, the present application provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the technical solution of the above-mentioned artificial intelligence-based cutout method.

[0016] From the technical solution provided by the present application, it can be seen that by segmenting the foreground and background of the pre-processed image based on an artificial intelligence algorithm to obtain a foreground image, and performing pixel-level edge optimization on the foreground image, it is possible to accurately process complex details (such as hair, transparent objects, etc.), ensure that the transition between the foreground and the background is more natural, and reduce seams and distortion. By calculating the gradient of the edge-optimized foreground image and the preset background image or by calculating the pixel weights of the transition area, the preset background image and the edge-optimized foreground image are fused, so that the color, brightness and other information of the image in the synthesized area remain consistent. It is particularly suitable for processing images with uneven lighting and rich details, and can also ensure the naturalness and realism of the synthesized image. Since the above technical solution can be completed by relying on artificial intelligence algorithms, the need for manual intervention is greatly reduced, which not only improves the efficiency of cutouts, but also greatly reduces labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0018] Figure 1 This is a flow chart of an artificial intelligence-based image cutting method provided in an embodiment of the present application;

[0019] Figure 2Schematic diagram of the structure of the artificial intelligence-based cutout device provided in an embodiment of the present application;

[0020] Figure 3 It is a schematic diagram of the structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0022] In this specification, adjectives such as first and second may be used only to distinguish one element or action from another element or action, without necessarily requiring or implying any actual such relationship or order. Where circumstances permit, reference to an element or component or step (etc.) should not be construed as being limited to only one of the elements, components, or steps, but may be one or more of the elements, components, or steps, etc.

[0023] In this specification, for the convenience of description, the sizes of various parts shown in the drawings are not drawn according to the actual proportions.

[0024] With the development of digital image processing technology, image cutout (i.e., image segmentation) has been widely used in many fields, particularly in film and television production, advertising design, virtual reality (VR), augmented reality (AR), product display on e-commerce platforms, and ID photo production. Existing image cutout methods typically rely on techniques such as manual labeling or color segmentation, but these methods often require extensive manual intervention, are inefficient when processing complex images, and are easily affected by background complexity. This can lead to loss of detail, low computational efficiency (especially for high-resolution images, which are slow and often require manual fine-tuning), and poor robustness.

[0025] In view of the above problems of the prior art, this application proposes an artificial intelligence-based image cutting method, the flow chart of which is shown in the attached figure. Figure 1 As shown, it mainly includes steps S101 to S104, which are detailed as follows:

[0026] Step S101: pre-processing the target image to obtain a pre-processed image.

[0027] In order to improve image quality, remove noise, enhance image features, and thus improve the effect of subsequent processing (such as foreground-background segmentation, edge optimization, etc.), in an embodiment of the present application, the target image can be preprocessed first, wherein the target image is the image from which the foreground needs to be cut out. As an embodiment of the present application, preprocessing the target image to obtain the preprocessed image can be achieved through steps S1011 to S1013, which are detailed as follows:

[0028] Step S1011: performing a size normalization operation on the target image and filtering out noise in the target image through Gaussian filtering to obtain a noise-free image.

[0029] The target image may not be of standard size, for example, it may be too large or too small, which is not conducive to subsequent image processing. Therefore, the target image can be normalized. On the other hand, considering the influence of various factors during the shooting process, the target image may also have some noise. In the embodiment of the present application, the noise in the target image can be filtered out through Gaussian filtering to obtain a noise-free image.

[0030] Step S1012: performing contrast enhancement on the noise-free image using a histogram equalization method to obtain a contrast-enhanced image.

[0031] In addition to noise, the target image may also have issues such as uneven brightness and low contrast. Uneven brightness means that the target image has overexposed or underexposed areas, while low contrast can easily make it difficult to distinguish or segment the foreground and background of the target image. Since uneven brightness is also one of the causes of low contrast, whether it is uneven brightness or low contrast, in the embodiments of the present application, histogram equalization can be used to enhance the contrast of the noise-free image to obtain a contrast-enhanced image.

[0032] Step S1013: converting the color space of the contrast-enhanced image into a second color space to obtain a preprocessed image.

[0033] Generally speaking, an image is an image in the RGB color space. The RGB color space is based on an additive mixing model of the three primary colors: red (Red), green (Green), and blue (Blue). It is widely used in displays, image acquisition devices, etc. However, the main problem with the RGB color space is that its components (R, G, B) are often highly correlated, which makes it difficult to separate the foreground and background in the RGB space. In particular, when the colors of the foreground and background are very similar, the three primary color components of RGB may not intuitively reflect the semantic information in the image. To more conveniently separate the background and foreground, the color space of the contrast-enhanced image obtained in step S1012 can be converted to another color space (i.e., a second color space), such as the HSV color space. In the HSV color space, hue (Hue) represents color, saturation (Saturation) represents color purity, and value (Value) represents the brightness of the color. Since hue (H) and saturation (S) are generally better at distinguishing the colors of objects, value (V) is well suited to represent changes in lighting. Therefore, by separating brightness and chromaticity information, the HSV color space can make the distinction between foreground and background in segmentation tasks more intuitive, especially when the foreground and background have similar colors but different brightness.

[0034] As an embodiment of the present application, converting the color space of the contrast-enhanced image to a second color space (e.g., HSV color space) to obtain a preprocessed image can be achieved through the following steps S1 to S3:

[0035] Step S1: Normalize RGB values.

[0036] To facilitate subsequent calculations, the RGB values ​​are first normalized to the range [0, 1]. Specifically:

[0037]

[0038] Among them, r, g, and b are the standardized values ​​of R, G, and B respectively.

[0039] Step S2: Calculate the maximum value C of the normalized RGB value max , minimum value C min And the difference Δ between the maximum and minimum values ​​is as follows:

[0040] C max =max(r,g,b),C min =min(r,g,b) and Δ = C max -C minStep S3: Calculate the hue value, saturation value, and brightness value of the HSV color space based on the standardized RGB values ​​and their maximum, minimum, and difference values. For the hue value H, if Δ = 0, then the hue value H = 0 (i.e., grayscale image). Otherwise, calculate the hue value H based on the color channel where the maximum value is located. The calculation formula is as follows:

[0041]

[0042] The resulting range of H is [0°, 360°].

[0043] For the saturation value S, it is stipulated that if Δ=0, then the saturation value S=0 (i.e., grayscale image). Otherwise, the saturation value S is calculated according to the following formula:

[0044]

[0045] For the brightness value V, directly set it equal to the maximum RGB component, that is, V = C max .

[0046] The final HSV value is (H, S, V), where H represents hue, S represents saturation, and V represents brightness.

[0047] Step S102: Segment the foreground and background of the preprocessed image based on an artificial intelligence algorithm to obtain a foreground image.

[0048] In order to "cut out" the foreground portion of the pre-processed image, in an embodiment of the present application, the foreground and background of the pre-processed image can be segmented based on an artificial intelligence algorithm to obtain a foreground image. As an embodiment of the present application, the foreground and background of the pre-processed image can be segmented based on an artificial intelligence algorithm to obtain a foreground image through steps S1021 to S1023, as detailed below:

[0049] Step S1021: Determine the intervals to be inserted between elements of the convolution kernel according to the receptive field of the convolution kernel of the preset convolutional neural network.

[0050] In an embodiment of the present application, a convolutional neural network is a trained neural network model. The receptive field refers to the area of ​​a group of pixels in an input image, and the pixel values ​​in this area will affect the output of a particular neuron. In short, the receptive field describes the size of the area of ​​the input image that a neuron in a convolutional neural network can "see". The size of the receptive field directly determines how large a range of contextual information in the input image the convolutional neural network can perceive, thereby affecting the representation ability of the convolutional neural network. Specifically, in an embodiment of the present application, according to the receptive field of the convolution kernel of a preset convolutional neural network, determining the interval inserted between the elements of the convolution kernel can be: if the receptive field of the convolution kernel of the convolutional neural network is less than a preset threshold, the convolution kernel is configured with an interval less than the first interval threshold; if the receptive field of the convolution kernel of the convolutional neural network is greater than the preset threshold, the convolution kernel is configured with an interval greater than the first interval threshold.

[0051] Step S1022: extracting foreground image features and background image features of the preprocessed image by performing a convolution operation on the preprocessed image according to the intervals inserted between the elements of the convolution kernel.

[0052] Increasing the receptive field does not mean simply increasing the number of convolutional layers, but rather expanding the receptive field in a more efficient way. For example, by inserting gaps between the elements of the convolution kernel to expand the receptive field without increasing the amount of computation or the number of parameters, this enables the convolutional neural network to obtain more contextual information without significantly increasing the amount of computation. When the convolution kernel is inserted between its elements at a certain interval, the receptive field can be significantly expanded while keeping the size of the convolution kernel unchanged. Then, the convolution operation on the preprocessed image is performed using the convolution kernel with the inserted gaps, which can extract the foreground image features and background image features of the preprocessed image without significantly increasing the amount of computation.

[0053] Step S1023: Mapping the foreground image features and the background image features through a convolutional neural network, and determining the foreground image of the preprocessed image according to the probability output by the convolutional neural network.

[0054] Specifically, the extracted foreground image features and background image features are mapped to a pixel-level classification space through the fully connected layer of the convolutional neural network. For each pixel obtained after mapping to the pixel-level classification space, the classification layer of the convolutional neural network will output a classification probability distribution, indicating the probability that the pixel belongs to a different category. For the binary classification task of foreground and background, the classification layer outputs the probability value P(foreground|x) that each pixel belongs to the foreground, and the probability value P(background|x) = 1-P(foreground|x) that each pixel belongs to the background. For example, suppose that the classification layer of the convolutional neural network outputs a probability map, where each pixel x corresponds to a probability value P(foreground|x), ranging from 0 to 1. Each pixel value of this probability map represents the probability that the pixel belongs to the foreground. For example, a pixel with an output of 0.8 means that the pixel has an 80% probability of belonging to the foreground. Once the probability map of each pixel is obtained, the next step is to obtain the final binary segmentation map through a thresholding operation. The steps of the thresholding process are as follows:

[0055] 1) Select a threshold T, which determines whether a pixel is classified as foreground or background. If P(foreground|x)>T, the pixel is considered to belong to the foreground; if P(foreground|x)≤T, the pixel is considered to belong to the background;

[0056] 2) Based on this threshold, a binary segmentation map is generated. Each pixel is assigned a value of 1 (foreground) or 0 (background), indicating the final category of the pixel. Foreground pixels are pixels where P(foreground|x)>T, and background pixels are pixels where P(foreground|x)≤T. For example, assuming the threshold T = 0.5, if the probability value of a pixel is 0.8, it will be classified as foreground; if the probability value of the pixel is 0.4, it will be classified as background.

[0057] In the above embodiment, the extracted foreground and background image features are mapped to a pixel-level classification space through the fully connected layer of the convolutional neural network. Primarily, the fully connected layer maps the feature map in the convolutional network to a new space, where each pixel has a corresponding class probability output. Specifically, assuming a feature map of size H′×W′, after processing through several convolutional layers, each pixel in the feature map has a C-dimensional feature vector. Then, the convolution layer is used to project this C-dimensional feature vector into a class space. For example, if there is a binary classification problem (foreground and background), the fully connected layer will map each feature vector into a two-dimensional vector, representing the probability of the pixel belonging to the foreground and background. For each pixel x, its feature vector f(x) is mapped to a new space, forming a probability vector. If there are C classes (for example, binary classification, foreground and background), the output is a vector of length C, representing the probability of the pixel belonging to each class. After passing through the fully connected layer, a raw output vector is obtained. In order to convert this vector into a probability value, the Softmax activation function is used in the classification layer of the convolutional neural network to convert the probability vector of each pixel into a probability distribution, so that each output value is between 0 and 1, and the sum of these values ​​is 1.

[0058] Step S103: performing pixel-level edge optimization on the foreground image according to the pixel values ​​of the pixels in the foreground image and the pixel values ​​of the adjacent pixels thereof to obtain an edge-optimized foreground image.

[0059] After the preprocessed image is segmented into foreground and background, the edges of the segmented foreground image may be too hard or not smooth due to aliasing or "halo" effects. To make the edges of the foreground image smoother and more natural, pixel-level edge optimization can be performed on the foreground image based on the pixel values ​​of the foreground image and its adjacent pixels to obtain an edge-optimized foreground image. Specifically, this can be achieved through steps S1031 to S1033, as detailed below:

[0060] Step S1031: construct an undirected graph using pixels in the foreground image as nodes and relationships between pixels and their adjacent pixels as edges.

[0061] Specifically, each pixel in the foreground image can be regarded as a random variable, and all pixels in the foreground image are connected through a graph to form an undirected graph. Each node represents a pixel, and the edge represents the relationship between adjacent pixels. i Belongs to a category y i (foreground or background), the edge in the undirected graph connects pixels x i and adjacent pixels x j, whose weight represents the similarity between two pixels. Each node and edge in the undirected graph corresponds to a conditional random field energy function, and the ultimate goal is to obtain the optimal pixel classification (foreground or background) by minimizing the total energy of the random field energy function.

[0062] Step S1032: Define the conditional random field energy function corresponding to each pair of nodes and edges in the undirected graph.

[0063] The conditional random field energy function usually consists of two parts: data term and smooth term, where the data term E data (x i ) is used to measure any pixel x i The degree of match between its label (foreground or background), the smoothing term E smooth (x i ,x j ) is used to measure the adjacent pixels x i and x j The cost of label consistency between them.

[0064] The total energy function E of the conditional random field is:

[0065] The data item reflects the cost of a pixel belonging to the foreground or background, which is usually measured by the similarity between the pixel and the foreground or background model, that is, E data (x i )=-logP(y i |x i ), where P(y i |x i ) is the pixel x i Belongs to category (foreground or background) y i The conditional probability is usually determined by features such as pixel color and texture. The conditional probability can be estimated by a Gaussian mixture model, indicating the degree of match between the pixel color and the foreground or background. The smoothness term encourages adjacent pixels to have the same label to avoid unnatural label jumps. The smoothness term is usually determined by the color difference and spatial distance between pixels. The general form of the smoothness term is:

[0066]

[0067] Among them, I i and I j Pixels x i and x j The color (e.g. RGB value), ||I i -I j || is pixel x i and x jThe color difference between the two, λ is the weight of the smoothing term, which is used to control the strength of the smoothing. σ is the scale of the color difference, which affects the smoothing effect, Π(y i ≠y j ) is the indicator function, when the pixel x i and x j The smoothing term is only valid when the labels of are different (i.e. one is foreground and the other is background).

[0068] Step S1033: Switching pixels in the foreground image between foreground and background until the total energy value of the conditional random field energy function is minimized to obtain an edge-optimized foreground image.

[0069] In order to obtain the optimal segmentation result, it is necessary to minimize the total energy value of the conditional random field energy function. Usually, the energy value of the conditional random field energy function (that is, the value of the total energy function of the conditional random field) is minimized using the α-expansion algorithm or the α-β exchange algorithm, which are commonly used optimization methods in image segmentation. The α-expansion algorithm gradually optimizes the total energy function of the conditional random field, attempting to switch the label of each pixel from foreground to background, or vice versa, until convergence. The α-β exchange algorithm minimizes the total energy function of the conditional random field by exchanging foreground and background labels. Through these optimization methods, the problem of unnatural edge transitions can be effectively reduced and the accuracy of the foreground and background boundaries can be improved.

[0070] Step S104: merging the preset background image and the edge-optimized foreground image by calculating the gradient of the edge-optimized foreground image and the preset background image or by calculating the pixel weights of the transition area.

[0071] In the embodiments of the present application, the preset background image is a background specified according to the needs of the scene. For example, the background of an ID photo may be blue, red, or another more complex image. In another example, a real person image is seamlessly embedded as a foreground into the background of a virtual world, and the background of this virtual world may be an even more complex image. After segmenting the foreground of the target image and optimizing its edges, the preset background image and the edge-optimized foreground image can be fused by calculating the gradient of the edge-optimized foreground image and the preset background image, or by calculating the pixel weights of the transition region. The resulting fused image is the desired image.

[0072] As an embodiment of the present application, by calculating the gradients of the edge-optimized foreground image and the preset background image, the fusion of the preset background image and the edge-optimized foreground image can be achieved through steps S1041 and S1042, as detailed below:

[0073] Step S1041: Calculate the gradients of the preset background image and the edge-optimized foreground image respectively.

[0074] In the field of image processing, the gradient of an image reflects the speed and direction of the change of pixel values. When synthesizing images, if the gradients of the foreground and background are inconsistent, it will usually lead to obvious seams or edges. Therefore, this unnatural seam can be eliminated by ensuring that the gradients of the synthesized area are consistent, making the fused image more visually natural. First, the gradients of the preset background image and the edge-optimized foreground image are calculated separately. It should be noted here that the gradient of an image is usually the spatial rate of change of the image intensity. For a grayscale image, the gradient of the image describes the rate of change of the brightness value of each pixel in the horizontal and vertical directions, that is, the horizontal gradient Indicates the brightness change of the image in the horizontal direction (x-axis), vertical gradient Indicates the brightness change of the image in the vertical direction (y-axis).

[0075] Step S1042: Automatically adjust the gradient of the image to be solved so that the gradient of the image in the foreground area is consistent with the gradient of the edge-optimized foreground image and the gradient of the background area is consistent with the gradient of the preset background image, and the image to be solved is output as a fused image.

[0076] Step S1042 can be specifically performed by solving the Poisson equation To achieve, specifically, represents the divergence operator, is the Laplace operator of the image to be solved, which represents the local change of the image. F is a vector field, usually the gradient field of the image. In the application of image processing, F describes the gradient (i.e., the rate of change) of each position in the image. The solution process of Poisson's equation is to find the foreground area of ​​the target image φ with the edge optimized foreground image I f The same gradient, with the preset background image I in the background area b In other words, in the process of solving the Poisson equation, the gradient of the image to be solved is automatically adjusted so that the gradient of the foreground area of ​​the image to be solved is consistent with the gradient of the edge-optimized foreground image, and the gradient of the background area of ​​the image to be solved is consistent with the gradient of the preset background image, and the image to be solved is output as a fused image.

[0077] As an embodiment of the present application, by calculating pixel weights of the transition region, the fusion of the preset background image and the edge-optimized foreground image may be achieved through steps S'1041 to S'1043, as detailed below:

[0078] Step S'1041: determining an adjustment parameter and a threshold parameter, wherein the adjustment parameter is used to control the transition speed of the weight function, and the threshold parameter is used to control the midpoint of the weight function.

[0079] In an embodiment of the present application, the weight function is the fusion weight of each pixel in the transition area, which represents the degree of influence of the pixel in the transition area. The adjustment parameter of the weight function is used to control the transition speed of the weight function, and the threshold parameter of the weight function is used to control the midpoint of the weight function. These parameters can be determined by experiments or optimization methods, and are usually adjusted based on the visual effect of the fused image. It should be noted here that the "transition area" refers to a transition interval between the foreground and background in the image, which is generally manifested as a "fuzzy" boundary between the foreground and the background. In the transition area, the classification and feature performance of the image pixels transition from one category to another, and there is usually a gradual change process. The characteristic of the transition area is continuous and smooth change. The classification value of each pixel (such as the probability of foreground and background) changes gradually, rather than suddenly.

[0080] Step S'1042: Calculate the distance from the pixels in the preset background image and the edge-optimized foreground image to the boundary of the transition area.

[0081] In an embodiment of the present application, calculating the distance from pixels in a preset background image and an edge-optimized foreground image to the boundary of the transition area can be achieved by methods such as Euclidean distance transformation, chessboard distance transformation or fast distance transformation algorithm.

[0082] Step S'1043: The preset background image and the edge-optimized foreground image are fused according to the distance from the pixels in the preset background image and the edge-optimized foreground image to the boundary of the transition area, the adjustment parameter and the threshold parameter, and the pixel weights of the transition area are smoothed by the sigmoid function.

[0083] Specifically, if the distance from the pixel in the preset background image and the edge-optimized foreground image to the boundary of the transition area is d(x), the above adjustment parameter is α, and the threshold parameter is β, then the fusion of the preset background image and the edge-optimized foreground image based on the pixel weight of the smooth transition area of ​​the sigmoid function can be expressed as:

[0084]

[0085] Among them, I pre (x) represents the pixel value of pixel x in the original image (preset background image or edge-optimized foreground image), and represents I new (x) is the pixel value after fusion. The expression of the weight function W(x) is used to describe the relative influence of each pixel in the fusion area of ​​the foreground and background in the image, and the weight changes as the distance between the pixel and the boundary changes. e is the natural exponent.

[0086] From the above attached Figure 1From the example of the AI-based cutout method, it can be seen that the foreground and background of the pre-processed image are segmented based on the AI ​​algorithm to obtain a foreground image, and the edge optimization of the foreground image is performed at the pixel level, which can accurately process complex details (such as hair, transparent objects, etc.), ensure that the transition between the foreground and the background is more natural, and reduce seams and distortion. By calculating the gradient of the edge-optimized foreground image and the preset background image or by calculating the pixel weights of the transition area, the preset background image and the edge-optimized foreground image are fused, so that the color, brightness and other information of the image in the synthesized area can be kept consistent. It is particularly suitable for processing images with uneven lighting and rich details, and can also ensure the naturalness and realism of the synthesized image. Since the above technical solution can be completed by relying on the AI ​​algorithm, the need for manual intervention is greatly reduced, which not only improves the efficiency of cutout, but also greatly reduces labor costs.

[0087] Please see the attached Figure 2 , is an artificial intelligence-based image cutout device provided in an embodiment of the present application. The device may include a pre-processing module 201, a segmentation module 202, an optimization module 203, and a fusion module 204, as detailed below:

[0088] The preprocessing module 201 is used to preprocess the target image to obtain a preprocessed image;

[0089] A segmentation module 202 is used to segment the pre-processed image into foreground and background based on an artificial intelligence algorithm to obtain a foreground image;

[0090] An optimization module 203 is configured to perform pixel-level edge optimization on the foreground image based on pixel values ​​of pixels in the foreground image and pixel values ​​of adjacent pixels thereof, thereby obtaining an edge-optimized foreground image.

[0091] The fusion module 204 is configured to fuse the preset background image and the edge-optimized foreground image by calculating the gradient of the edge-optimized foreground image and the preset background image or by calculating the pixel weights of the transition region.

[0092] From the above attached Figure 2It can be seen from the example of the AI-based cutout device that the foreground and background of the pre-processed image are segmented based on the AI ​​algorithm to obtain a foreground image, and the edge optimization of the foreground image is performed at the pixel level, which can accurately process complex details (such as hair, transparent objects, etc.), ensure that the transition between the foreground and the background is more natural, and reduce seams and distortion. By calculating the gradient of the edge-optimized foreground image and the preset background image or by calculating the pixel weights of the transition area, the preset background image and the edge-optimized foreground image are fused, so that the color, brightness and other information of the image in the synthesized area can be kept consistent. It is particularly suitable for processing images with uneven lighting and rich details, and can also ensure the naturalness and realism of the synthesized image. Since the above technical solution can be completed by relying on the AI ​​algorithm, the need for manual intervention is greatly reduced, which not only improves the efficiency of cutout, but also greatly reduces labor costs.

[0093] Figure 3 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 3 As shown, the electronic device 3 of this embodiment mainly includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and capable of running on the processor 30, such as a program for the image cutting method based on artificial intelligence. When the processor 30 executes the computer program 32, the steps of the above-mentioned image cutting method embodiment based on artificial intelligence are implemented, such as Figure 1 Alternatively, when the processor 30 executes the computer program 32, the functions of the modules / units in the above-mentioned device embodiments are realized, for example Figure 2 The functions of the pre-processing module 201, the segmentation module 202, the optimization module 203 and the fusion module 204 are shown.

[0094] Exemplarily, the computer program 32 of the artificial intelligence-based cutout method mainly includes: preprocessing the target image to obtain a preprocessed image; segmenting the foreground and background of the preprocessed image based on an artificial intelligence algorithm to obtain a foreground image; performing pixel-level edge optimization on the foreground image according to the pixel values ​​of the pixels in the foreground image and their adjacent pixels to obtain an edge-optimized foreground image; and fusing the preset background image and the edge-optimized foreground image by calculating the gradient of the edge-optimized foreground image and the preset background image or by calculating the pixel weights of the transition area. The computer program 32 can be divided into one or more modules / units, one or more modules / units are stored in the memory 31 and executed by the processor 30 to complete the present application. The one or more modules / units can be a series of computer program instruction segments that can perform specific functions, and the instruction segments are used to describe the execution process of the computer program 32 in the electronic device 3. For example, the computer program 32 can be divided into the functions of a preprocessing module 201, a segmentation module 202, an optimization module 203 and a fusion module 204 (modules in the virtual device), and the specific functions of each module are as follows: the preprocessing module 201 is used to preprocess the target image to obtain a preprocessed image; the segmentation module 202 is used to segment the preprocessed image into foreground and background based on an artificial intelligence algorithm to obtain a foreground image; the optimization module 203 is used to perform pixel-level edge optimization on the foreground image according to the pixel values ​​of the pixels in the foreground image and their adjacent pixels to obtain an edge-optimized foreground image; the fusion module 204 is used to fuse the preset background image and the edge-optimized foreground image by calculating the gradient of the edge-optimized foreground image and the preset background image or by calculating the pixel weights of the transition area.

[0095] The electronic device 3 may include but is not limited to a processor 30 and a memory 31. Those skilled in the art will appreciate that Figure 3 It is only an example of electronic device 3 and does not constitute a limitation of electronic device 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0096] The processor 30 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0097] The memory 31 can be an internal storage unit of the electronic device 3, such as a hard drive or memory of the electronic device 3. The memory 31 can also be an external storage device of the electronic device 3, such as a plug-in hard drive, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. equipped on the electronic device 3. Furthermore, the memory 31 can include both an internal storage unit of the electronic device 3 and an external storage device. The memory 31 is used to store computer programs and other programs and data required by the electronic device. The memory 31 can also be used to temporarily store data that has been output or is about to be output.

[0098] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0099] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0100] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0101] In the embodiments provided in this application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0102] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0103] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0104] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program of the artificial intelligence-based cutout method can be stored in a storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments, that is, pre-processing the target image to obtain a pre-processed image; segmenting the pre-processed image into foreground and background based on the artificial intelligence algorithm to obtain a foreground image; performing pixel-level edge optimization on the foreground image according to the pixel values ​​of the pixels in the foreground image and their adjacent pixels to obtain an edge-optimized foreground image; and merging the preset background image and the edge-optimized foreground image by calculating the gradient of the edge-optimized foreground image and the preset background image or by calculating the pixel weights of the transition area. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form. Storage media may include any entity or device capable of carrying computer program code, recording media, USB flash drives, removable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signals, telecommunications signals, and software distribution media. It should be noted that the content of storage media may be appropriately expanded or reduced based on the requirements of legislation and patent practice within a jurisdiction. For example, in some jurisdictions, based on legislation and patent practice, storage media do not include electric carrier signals and telecommunications signals.

[0105] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application. The specific implementation methods described above further explain the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only the specific implementation method of the present application and is not used to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included in the protection scope of the present invention.

Claims

1. A method for cutting out an image based on artificial intelligence, characterized in that: The method comprises: Preprocessing the target image to obtain a preprocessed image; Segmenting the preprocessed image into foreground and background based on an artificial intelligence algorithm to obtain a foreground image; performing pixel-level edge optimization on the foreground image according to pixel values ​​of pixels in the foreground image and pixel values ​​of adjacent pixels thereof to obtain an edge-optimized foreground image; The preset background image and the edge-optimized foreground image are fused by calculating the gradients of the edge-optimized foreground image and the preset background image or by calculating the pixel weights of a transition area.

2. The artificial intelligence-based cutout method according to claim 1, wherein: The preprocessing of the target image to obtain a preprocessed image includes: Normalizing the size of the target image and filtering out noise in the target image through Gaussian filtering to obtain a noise-free image; Performing contrast enhancement on the noise-free image using a histogram equalization method to obtain a contrast-enhanced image; The color space of the contrast-enhanced image is converted to a second color space to obtain a preprocessed image.

3. The artificial intelligence-based cutout method according to claim 1, wherein: The step of segmenting the pre-processed image into foreground and background based on an artificial intelligence algorithm to obtain a foreground image includes: Determining the intervals to be inserted between elements of a convolution kernel according to the receptive field of the convolution kernel of a preset convolutional neural network; extracting foreground image features and background image features of the preprocessed image by performing a convolution operation on the preprocessed image according to the interval; By mapping the foreground image features and the background image features by the convolutional neural network, the foreground image of the preprocessed image is determined according to the probability output by the convolutional neural network.

4. The artificial intelligence-based cutout method according to claim 3, wherein: Determining the intervals to be inserted between elements of the convolution kernel according to the receptive field of the convolution kernel of the convolutional neural network includes: If the receptive field of the convolution kernel of the convolutional neural network is smaller than a preset threshold, configuring an interval smaller than a first interval threshold for the convolution kernel; If the receptive field of the convolution kernel of the convolutional neural network is greater than the preset threshold, an interval greater than the first interval threshold is configured for the convolution kernel.

5. The artificial intelligence-based cutout method according to claim 1, wherein: The step of performing pixel-level edge optimization on the foreground image according to pixel values ​​of pixels in the foreground image and pixel values ​​of adjacent pixels thereof to obtain an edge-optimized foreground image comprises: constructing an undirected graph using pixels in the foreground image as nodes and relationships between the pixels and their adjacent pixels as edges; Defining an energy function corresponding to each pair of nodes and edges in the undirected graph; The pixels in the foreground image are switched between foreground and background until the total function value of the energy function is minimized to obtain the edge-optimized foreground image.

6. The artificial intelligence-based cutout method according to claim 1, wherein: The step of calculating the gradients of the edge-optimized foreground image and the preset background image, fusing the preset background image and the edge-optimized foreground image, and outputting a fused image comprises: Calculating the gradients of the preset background image and the edge-optimized foreground image respectively; Automatically adjust the gradient of the image to be solved so that the gradient of the foreground area of ​​the image to be solved is consistent with the gradient of the edge-optimized foreground image and the gradient of the background area of ​​the image to be solved is consistent with the gradient of the preset background image, and the image to be solved is output as the fused image.

7. The artificial intelligence-based image cutting method according to claim 1, wherein: The step of fusing the preset background image and the edge-optimized foreground image by calculating pixel weights of the transition region includes: Determining an adjustment parameter and a threshold parameter, wherein the adjustment parameter is used to control a transition speed of the weight function and the threshold parameter is used to control a midpoint of the weight function; Calculating the distance from a pixel in the transition region to a boundary of the transition region; The preset background image and the edge-optimized foreground image are fused according to the distance, the adjustment parameter and the threshold parameter and based on a sigmoid function to smooth the pixel weights of the transition area.

8. An artificial intelligence-based image cutting device, characterized in that: The device comprises: A conversion module, used to securely convert the original document to obtain an initial image; a de-noising module, configured to remove noise and / or artifacts from the initial image by analyzing the initial image; an enhancement module, configured to perform detail enhancement processing on the image from which noise and / or artifacts have been removed based on an artificial intelligence algorithm to obtain a detail-enhanced image; a reconstruction module, configured to reconstruct the detail-enhanced image into a security document; The compression module is used to adaptively select a corresponding compression algorithm to compress the objects in the security document according to different objects in the security document.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images