Image matting method and device and electronic equipment

Abstract

The invention provides an image matting method, an image matting device and electronic equipment, which are used for solving the problem that the efficiency of searching and optimizing a linear index-based pixel pair solution (PPSR-LI) expressed by an image is relatively low. The method comprises the steps of obtaining a to-be-processed image, wherein the to-be-processed image comprises a to-be-solved region pixel set, a foreground pixel set and a background pixel set; for each pixel point in the pixel set of the to-be-solved region, constructing an initial pixel pair solution representation of the pixel point according to the position coordinates of the foreground pixel set and the position coordinates of the background pixel set; solving and optimizing the initial pixel pair solution representation of the pixel point by using a heuristic optimization algorithm to obtain an optimized pixel pair solution representation; calculating the opacity of each pixel point in the pixel set of the to-be-solved region according to the optimized pixel pair solution representation to obtain an opacity matrix; and generating a matting result of the to-be-processed image according to the opacity matrix.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and in particular to an image cutout method, device and electronic device. Background Art

[0002] At present, when using pixel pair optimization-based image matting methods to matte natural images, pixel pair solution representation with linear indexing (PPSR-LI) is usually used to represent candidate pixel pairs, and then a heuristic algorithm is used to search and optimize the PPSR-LI solution and matte it. However, in the actual practice, it is found that the efficiency of using a heuristic algorithm to search and optimize the pixel pair solution representation with linear indexing (PPSR-LI) for image representation is low when processing images. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide an image cutout method, device and electronic device for improving the problem of low efficiency in searching and optimizing PPSR-LI represented by an image.

[0004] An embodiment of the present application provides an image matting method, comprising: obtaining an image to be processed, the image to be processed comprising: a pixel set of a region to be solved, a foreground pixel set and a background pixel set; for each pixel point in the pixel set of the region to be solved, constructing an initial pixel pair solution representation of the pixel point according to the position coordinates of the foreground pixel set and the position coordinates of the background pixel set; using a heuristic optimization algorithm to solve and optimize the initial pixel pair solution representation of the pixel point to obtain an optimized pixel pair solution representation; calculating the opacity of each pixel point in the pixel set of the region to be solved according to the optimized pixel pair solution representation to obtain an opacity matrix; and generating a matting result of the image to be processed according to the opacity matrix. In the implementation process of the above scheme, an initial pixel pair solution representation of the pixel point is constructed according to the position coordinates of the foreground pixel set and the position coordinates of the background pixel set, and a heuristic optimization algorithm is used to solve and optimize the initial pixel pair solution representation of the pixel point. The pixel pair solution representation based on pixel position is effectively used to replace the pixel pair solution representation based on linear index, and the situation that the pixel pair solution representation based on linear index violates the fitness value continuity assumption of adjacent solutions widely adopted by the heuristic algorithm is improved, thereby effectively improving the search optimization efficiency of the heuristic optimization algorithm through the pixel pair solution representation based on pixel position.

[0005] Optionally, in an embodiment of the present application, an initial pixel pair solution representation of the pixel point is constructed based on the position coordinates of the foreground pixel set and the position coordinates of the background pixel set, including: randomly selecting a foreground pixel point from the foreground pixel set and obtaining the position coordinates of the foreground pixel point; randomly selecting a background pixel point from the background pixel set and obtaining the position coordinates of the background pixel point; constructing a coordinate binary tuple based on the position coordinates of the foreground pixel point and the position coordinates of the background pixel point, and determining the coordinate binary tuple as the initial pixel pair solution representation of the pixel point. In the implementation process of the above scheme, by constructing a coordinate binary tuple based on the position coordinates of the foreground pixel point and the position coordinates of the background pixel point, and determining the coordinate binary tuple as the initial pixel pair solution representation of the pixel point, a more complete comprehensive neighborhood structure is constructed for the heuristic optimization algorithm in the decision space, and the problem of discontinuity of the adjacent solution evaluation function value of the heuristic optimization algorithm in the decision space is improved.

[0006] Optionally, in an embodiment of the present application, a heuristic optimization algorithm is used to solve and optimize the initial pixel pair solution representation of the pixel point, including: performing cross-mutation on the initial pixel pair solution representation of the pixel point through a cross-mutation process of the heuristic optimization algorithm to obtain a mutated candidate pixel pair population, the mutated candidate pixel pair population including multiple candidate pixel pair solution representations; for each candidate pixel pair solution representation in the multiple candidate pixel pair solution representations, repairing the candidate pixel pair solution representation through the boundary of a feasible solution to obtain multiple repaired pixel pair solution representations; for each repaired pixel pair solution representation in the multiple repaired pixel pair solution representations, evaluating the repaired pixel pair solution representation using a preset evaluation function to obtain evaluation values ​​of the multiple repaired pixel pair solution representations; selecting the repaired pixel pair solution representation with the minimum evaluation value from the multiple repaired pixel pair solution representations; and adding the repaired pixel pair solution representation with the minimum evaluation value to the optimized pixel pair solution representation. In the implementation process of the above scheme, the candidate pixel pair solution representation is transformed from an infeasible solution to a feasible solution representation by performing boundary repair on the candidate pixel pair solution representation. Therefore, repairing the boundary of the feasible solution on the candidate pixel pair solution representation can ensure that the generated candidate solution is still within a reasonable range and avoid the existence of invalid solutions, thereby improving the quality of the pixel pair solution representation. Moreover, through cross-mutation and boundary repair, the algorithm can dynamically adapt to different image features and improve its adaptability to different scenes.

[0007] Optionally, in an embodiment of the present application, the candidate pixel pair solution is represented by repairing the boundary of a feasible solution, including: parsing candidate pixel points from the candidate pixel pair solution; using the foreground pixel boundary and background pixel boundary of the image to be processed as constraints, performing nearest neighbor transformation on all pixel points of the image to be processed to obtain pixel points after neighbor transformation; judging whether the candidate pixel points contained in the candidate pixel pair representation are feasible solution representations; if so, constructing a repaired pixel pair solution representation based on the candidate pixel points, otherwise, mapping the candidate pixel points to map to corresponding neighbor transformed pixel points to obtain mapped candidate pixel points, and constructing a repaired pixel pair solution representation based on the mapped candidate pixel points. In the implementation process of the above scheme, by performing nearest neighbor transformation on all pixel points of the image and using the foreground pixel boundary and background pixel boundary of the image to be processed as constraints, it is ensured that the generated solution representation does not exceed the valid range of the image, and the generation of invalid pixel pair solution representations can be avoided, thereby improving the feasibility of the pixel pair solution representation.

[0008] Optionally, in an embodiment of the present application, the candidate pixels include: candidate foreground pixels and candidate background pixels; performing a nearest neighbor transformation on all pixels of the image to be processed includes: finding a first pixel point that is closest to each pixel point in the image to be processed from all foreground pixels of the image to be processed; finding a second pixel point that is closest to each pixel point in the image to be processed from all background pixels of the image to be processed. In the implementation of the above scheme, by finding the nearest neighbor point, it can be ensured that the transformed pixel point is as similar as possible to the original candidate pixel points that should be feasible solutions in terms of color, texture, etc., thereby improving the accuracy of repairing the pixel pair solution representation.

[0009] Optionally, in an embodiment of the present application, the candidate pixels are mapped, including: if the candidate foreground pixel is not an infeasible solution, the candidate foreground pixel is optimized, and the optimized candidate foreground pixel is determined as the mapped candidate foreground pixel, otherwise, the first pixel corresponding to the candidate foreground pixel is determined as the mapped candidate pixel; if the candidate background pixel is not an infeasible solution, the candidate foreground pixel is optimized, and the optimized candidate background pixel is determined as the mapped candidate background pixel, otherwise, the second pixel corresponding to the candidate background pixel is determined as the mapped candidate pixel. In the implementation process of the above scheme, by checking whether the candidate foreground pixel is located in the foreground pixel set and whether the candidate background pixel is located in the background pixel set, it is ensured that the pixel pair solution representation finally generated always satisfies the constraint conditions, which avoids the generation of invalid pixel pair solution representations, thereby ensuring the feasibility of the pixel pair solution representation, and ultimately improving the reliability and robustness of the algorithm.

[0010] The embodiment of the present application also provides an image cutout device, including: an image processing acquisition module, used to acquire an image to be processed, the image to be processed including: a pixel set of a region to be solved, a foreground pixel set and a background pixel set; a pixel pair solution construction module, used to construct an initial pixel pair solution representation of each pixel point in the pixel set of the region to be solved according to the position coordinates of the foreground pixel set and the position coordinates of the background pixel set; a pixel pair solution optimization module, used to solve and optimize the initial pixel pair solution representation of the pixel point using a heuristic optimization algorithm to obtain an optimized pixel pair solution representation; an opacity calculation module, used to calculate the opacity of each pixel point in the pixel set of the region to be solved according to the optimized pixel pair solution representation to obtain an opacity matrix; and a cutout result generation module, used to generate a cutout result of the image to be processed according to the opacity matrix.

[0011] Optionally, in an embodiment of the present application, a pixel pair solution construction module includes: a foreground coordinate acquisition submodule, used to randomly select a foreground pixel point from a foreground pixel set and obtain the position coordinates of the foreground pixel point; a background coordinate acquisition submodule, used to randomly select a background pixel point from a background pixel set and obtain the position coordinates of the background pixel point; an initial solution representation construction module, used to construct a coordinate tuple based on the position coordinates of the foreground pixel point and the position coordinates of the background pixel point, and determine the coordinate tuple as the initial pixel pair solution representation of the pixel point.

[0012] Optionally, in an embodiment of the present application, a pixel pair solution optimization module includes: an initial cross mutation submodule, used to perform cross mutation on the initial pixel pair solution representation of the pixel point through the cross mutation process of the heuristic optimization algorithm to obtain a mutated candidate pixel pair population, wherein the mutated candidate pixel pair population includes multiple candidate pixel pair solution representations; a representation boundary repair submodule, used to perform boundary repair of a feasible solution on each candidate pixel pair solution representation in the multiple candidate pixel pair solution representations to obtain multiple repaired pixel pair solution representations; a pixel pair solution evaluation submodule, used to evaluate each repaired pixel pair solution representation in the multiple repaired pixel pair solution representations using a preset evaluation function to obtain evaluation values ​​of the multiple repaired pixel pair solution representations; a pixel pair solution selection submodule, used to select a repaired pixel pair solution representation with a minimum evaluation value from multiple repaired pixel pair solution representations; a pixel pair solution adding submodule, used to add the repaired pixel pair solution representation with the minimum evaluation value to the optimized pixel pair solution representation.

[0013] Optionally, in an embodiment of the present application, a boundary repair submodule is represented, including: a candidate pixel parsing unit, used to parse candidate pixel points from the candidate pixel pair solution representation; a pixel transformation unit, used to perform nearest neighbor transformation on all pixel points of the image to be processed with the foreground pixel boundary and background pixel boundary of the image to be processed as constraints, and obtain the pixel points after the neighbor transformation; a feasible solution judgment unit, used to judge whether the candidate pixel points included in the candidate pixel pair representation are feasible solution representations; a pixel repair mapping unit, used to construct a repair pixel pair solution representation based on the candidate pixel points if the candidate pixel points included in the candidate pixel pair are feasible solution representations, and if the candidate pixel points included in the candidate pixel pair are infeasible solution representations, map the candidate pixel points to corresponding neighbor transformed pixel points, and construct a repair pixel pair solution representation based on the mapped candidate pixel points.

[0014] Optionally, in an embodiment of the present application, the candidate pixel points include: candidate foreground pixel points and candidate background pixel points; the pixel transformation unit includes: a first pixel transformation subunit, used to find a first pixel point that is closest to each pixel point in the image to be processed from all foreground pixel points of the image to be processed; a second pixel transformation subunit, used to find a second pixel point that is closest to each pixel point in the image to be processed from all background pixel points of the image to be processed.

[0015] Optionally, in an embodiment of the present application, the pixel restoration mapping unit includes: a foreground pixel determination subunit, which is used to optimize the candidate foreground pixel if the candidate foreground pixel is not an infeasible solution, and determine the optimized candidate foreground pixel as the mapped candidate foreground pixel; otherwise, determine the first pixel corresponding to the candidate foreground pixel as the mapped candidate pixel; and a background pixel determination subunit, which is used to optimize the candidate foreground pixel if the candidate background pixel is not an infeasible solution, and determine the optimized candidate background pixel as the mapped candidate background pixel; otherwise, determine the second pixel corresponding to the candidate background pixel as the mapped candidate pixel.

[0016] An embodiment of the present application further provides an electronic device, including: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions execute the method described above when executed by the processor.

[0017] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method described above is executed.

[0018] The embodiment of the present application also provides a computer program product, including: a computer program or a computer instruction, and the computer program or the computer instruction executes the method described above when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A schematic diagram of foreground pixel representation using PPSR-LI in a trigraph in a comparative embodiment is shown;

[0021] Figure 2 A schematic diagram of using PPSR-LI to represent significant changes in fitness values ​​of adjacent pixels in space in the comparative embodiment shown;

[0022] Figure 3 A schematic diagram of the process of an image cutout method provided in an embodiment of the present application is shown;

[0023] Figure 4 A schematic diagram showing a comparison between a PPSR-LI representation and a PPSR-PP representation provided in an embodiment of the present application is shown;

[0024] Figure 5 A schematic diagram of the process flow of the image cutout device provided in an embodiment of the present application is shown;

[0025] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0026] To make the purpose, technical scheme and advantages of the embodiment of the present application clearer, the technical scheme in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application. It should be understood that the drawings in the embodiment of the present application only serve the purpose of explanation and description, and are not used to limit the protection scope of the embodiment of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in the embodiment of the present application shows the operation implemented according to some embodiments of the embodiment of the present application. It should be understood that the operation of the flowchart can be implemented out of order, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the embodiment of the present application, or remove one or more operations from the flowchart.

[0027] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all embodiments. The components of the embodiments of the present application described and shown in the drawings generally here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the embodiments of the present application claimed, but merely represents the selected embodiments of the embodiments of the present application.

[0028] It is understandable that the "first" and "second" in the embodiments of the present application are used to distinguish similar objects. Those skilled in the art will understand that the words "first" and "second" do not limit the quantity and execution order, and the words "first" and "second" do not necessarily limit the difference. In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that there may be three relationships, such as A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the related objects before and after are in an "or" relationship. The term "multiple" refers to more than two (including two), and similarly, "multiple groups" refers to more than two groups (including two groups).

[0029] Before introducing the image cutout method provided in the embodiment of the present application, some concepts involved in the embodiment of the present application are first introduced:

[0030] Natural image matting, also known as image matting, is an image processing technique specifically designed to accurately extract the foreground from natural images. This technique can estimate the opacity of the foreground object, i.e., the transparency mask, and then accurately separate the foreground and background in complex scenes.

[0031] A trimap is an image that divides the pixels in an image into three categories: an uncertain area, a determined foreground area, and a determined background area. The pixels in the determined foreground area are considered to be completely foreground, so their opacity α=1; the pixels in the determined background area are considered to be completely background, so their opacity α=0; the pixels in the uncertain area may be partially transparent, and the α value needs to be estimated by an algorithm.

[0032] It should be noted that the image cutout method provided in the embodiment of the present application can be executed by an electronic device, where the electronic device refers to a device terminal or server with the function of executing a computer program, such as a smart phone, a personal computer, a tablet computer, a personal digital assistant or a mobile Internet device. A server refers to a device that provides computing services through a network, such as an x86 server and a non-x86 server, and a non-x86 server includes a mainframe, a minicomputer and a UNIX server.

[0033] In the related art, the natural image matting problem can be modeled as the following synthesis equation:

[0034] I k =α k F k +(1-α k )B k (1)

[0035] Where k represents a pixel position in the image, I k represents the pixel value of the kth pixel in the image pixel set, F k represents the foreground pixel value at the kth pixel, B k represents the background pixel value at the kth pixel, α k is the foreground pixel value F k and background pixel value B k The weight of the convex combination at pixel position k, which represents the opacity of the foreground object at pixel position k, ranging from 0 to 1, where 0 represents completely transparent (pure background) and 1 represents completely opaque (pure foreground).

[0036] It can be understood that since the above synthesis equation has three unknowns (α k 、F k and B k ), and there is only one known number (I k ), therefore, the natural image matting problem is highly ill-posed, also known as an underdetermined problem, which means that this equation alone cannot uniquely determine the values ​​of all unknowns. This also means that additional prior knowledge is needed to constrain the problem, such as a trimap. The trimap provides a portion of the known foreground area (with an α value of 1) and a portion of the known background area (with an α value of 0). Therefore, the trimap can be used to simplify the problem to estimating the α value in the uncertain area. Specifically, it can be assumed that the foreground color value and the background color value are known in the determined area, and then this information can be used to infer the weight α value in the uncertain area.

[0037] In order to solve the weight α value, the image matting method based on pixel pair optimization (IMPPO) is one of the natural image matting methods. IMPPO selects a foreground pixel and a background pixel for each unknown pixel in the triplicate image to solve the α value of the pixel. Specifically, the α value of the unknown pixel is estimated based on the given foreground color and background color. The α value can be expressed as follows:

[0038]

[0039] in, represents the estimated weight at the kth pixel in the unknown region, I k represents the pixel value of the kth pixel in the image pixel set, F k represents the foreground pixel value in the kth given pixel pair, B k represents the background pixel value in the kth given pixel pair, and ||*|| represents the modulus of the norm *.

[0040] The above is to model the entire image matting problem as a pixel pair combination optimization problem of all unknown pixels, that is, the solution of each unknown pixel is defined as a pixel pair optimization (PPO) problem, whose goal is to select the best pixel pair consisting of foreground pixels and background pixels for each unknown pixel. The above PPO problem can be specifically expressed as follows:

[0041]

[0042] Among them, q represents unknown pixels, x q represents the pixel pair decision vector for the unknown pixel q, f q (x q ) is the pixel pair evaluation function of the unknown pixel q, represents feasible pixel pairs, specifically, represents the foreground pixel in the pixel pair, represents the background pixel in the pixel pair, Q U , Q F and Q B They represent the unknown pixel set in the area to be solved, the foreground pixel set and the background pixel set respectively.

[0043] See also Figure 1 A schematic diagram of foreground pixel representation using PPSR-LI in a tripartite graph in the comparative embodiment shown; in the process of using the IMPPO image matting method to matte a natural image, a pixel pair solution based on linear indexing (Pixel Pair Solution Representation with Linear Indexing, PPSR-LI) is usually used to represent the above-mentioned pixel pairs. PPSR-LI is a way of representing pixel pairs, encoding pixel pairs as a set of integer indices, which correspond to pixel positions in the image, and its role is to define the search space of the pixel pair optimization problem and define the mapping from the solution space to the search space. However, in the specific practice process, it is found that the fitness values ​​of adjacent solutions in the search space constructed by this PPSR-LI representation method may change significantly, which violates the continuity assumption of the fitness values ​​of adjacent solutions, resulting in low search efficiency for image pixel pairs.

[0044] Specifically, see Figure 2 The schematic diagram of using PPSR-LI to represent the significant change of fitness values ​​of adjacent pixels in space in the comparative embodiment shown in the figure; the above-mentioned PPSR-LI creates a foreground pixel array and a background pixel array arranged in column priority order, and then uses array subscripts to represent these pixels, so that all solutions in the solution space can be calculated. It can be seen from the figure that the foreground pixels F1 and F2 represent adjacent pixels in the space, and the two adjacent pixels are encoded as 3003 and 3004 in the foreground area, respectively, because the two pixel solutions of 3003 and 3004 are adjacent in the encoding space. However, the color difference between the two pixels is so large that when the pixels of 3003 and 3004 respectively form a pixel pair with pixel B1, the fitness value difference between the two on the pixel pair evaluation function of the unknown pixel U1 is very large (the pixel encoded as 3003 is 11.815, and the pixel encoded as 3004 is 24.677). This means that there will be large jumps or gaps between adjacent solutions in the decision space. Even if two pixel pairs are very close in the decision space, the fitness values ​​of these two pixel pairs may be very different, which represents a significant change in the fitness values ​​of adjacent solutions. Therefore, using PPSR-LI to represent the significant change in the fitness values ​​of adjacent pixels in the space violates the assumption of continuity of fitness values ​​of adjacent solutions. The linear index between pixel pairs does not directly reflect their spatial relationship in the image, resulting in the heuristic algorithm may spend a lot of time on low-quality pixel pairs. Therefore, the search efficiency for image pixel pairs is low.

[0045] For the above questions, please see Figure 3 The flowchart of the image matting method provided by the embodiment of the present application is shown; the main idea of ​​the image matting method provided by the embodiment of the present application is to construct a pixel pair solution representation of the pixel point according to the position coordinates of the foreground pixel set and the position coordinates of the background pixel set, also known as the pixel pair solution representation based on the pixel position (Pixel Pair Solution Representation with Pixel Position, PPSR-PP) representation, so as to adapt to the fitness value continuity assumption of adjacent solutions widely used by the heuristic algorithm, thereby improving the search optimization efficiency of the heuristic optimization algorithm. The implementation method of the above-mentioned image matting method using PPSR-PP representation may include:

[0046] Step S110: obtaining an image to be processed, wherein the image to be processed includes: a pixel set in the area to be solved, a foreground pixel set and a background pixel set.

[0047] The implementation of the above step S110 is, for example: converting the image to be processed into a trimap, and extracting a pixel set of the area to be solved, a foreground pixel set and a background pixel set from the trimap.

[0048] Step S120: for each pixel point in the pixel set of the area to be solved, construct an initial pixel pair solution representation of the pixel point according to the position coordinates of the foreground pixel set and the position coordinates of the background pixel set.

[0049] It can be understood that, since the above-mentioned initial pixel pair solution representation is a pixel pair solution based on pixel position (PPSR-PP) representation constructed by the position coordinates of the foreground pixel set and the position coordinates of the background pixel set, the above-mentioned position coordinates are the coordinates of the pixel point in the image space position. Therefore, the initial pixel pair solution representation can adapt to the fitness value continuity assumption of adjacent solutions widely used in heuristic algorithms, thereby improving the search optimization efficiency of the heuristic optimization algorithm.

[0050] Step S130: using a heuristic optimization algorithm to optimize the initial pixel pair solution representation of the pixel point to obtain an optimized pixel pair solution representation.

[0051] Step S140: Calculate the opacity of each pixel in the pixel set of the area to be solved according to the optimized pixel pair solution representation to obtain an opacity matrix.

[0052] Step S150: generating a cutout result of the image to be processed according to the opacity matrix.

[0053] In the implementation process of the above scheme, an initial pixel pair solution representation of the pixel point is constructed according to the position coordinates of the foreground pixel set and the position coordinates of the background pixel set, and a heuristic optimization algorithm is used to solve and optimize the initial pixel pair solution representation of the pixel point. The pixel pair solution representation based on pixel position is effectively used to replace the pixel pair solution representation based on linear index, and the situation that the pixel pair solution representation based on linear index violates the fitness value continuity assumption of adjacent solutions widely adopted by the heuristic algorithm is improved, thereby effectively improving the search optimization efficiency of the heuristic optimization algorithm through the pixel pair solution representation based on pixel position.

[0054] See also Figure 4The schematic diagram of the comparison between the PPSR-LI representation and the PPSR-PP representation provided by the embodiment of the present application is shown; it should be further explained that the motivation for the invention of the PPSR-PP representation of the pixel point using the position coordinates in the embodiment of the present application is that the neighborhood structure of the PPSR-LI is defined by measuring the distance in the one-dimensional genotype space, which cannot directly reflect the actual position relationship of the pixel in the two-dimensional image space. This definition ignores the actual spatial relationship between pixels and may cause adjacent pixels to be far apart in the search space. Specifically, with reference to the left sub-graph in the figure, the distance between the pixel marked as 6 and its vertically adjacent similar pixels (i.e., pixels marked as 5 and 7) in the genotype space is small, so the pixels marked as 5, 6 and 7 can construct a vertical neighborhood. However, there is a long distance between the pixel marked as 6 and its horizontally adjacent similar pixels in the genotype space, so similar pixels in the horizontal direction (such as pixels marked as 2 and 10) will still be omitted, that is, similar pixels marked as 2, 6 and 10 cannot construct a horizontal domain. Similarly, referring to the right sub-image in the figure, the color of the pixel in the second row and second column is similar to its horizontal neighboring pixels (i.e., the pixels in the second row and first column and the second row and third column). However, when using PPSR-LI, it can be observed that there is a large distance between the solution corresponding to the pixel in the second row and second column and the solutions of these similar pixels. The long distance between neighboring pixels makes it impossible for the heuristic optimization algorithm to find these neighbors, which may lead to a decrease in the local search performance of the heuristic optimization algorithm. Therefore, PPSR-LI cannot construct a comprehensive neighborhood structure with complete pixels and may miss neighboring pixels.

[0055] As an optional implementation of the above step S120, the above implementation of constructing the initial pixel pair solution representation of the pixel point according to the position coordinates of the foreground pixel set and the position coordinates of the background pixel set may include:

[0056] Step S121: randomly select a foreground pixel from the foreground pixel set, and obtain the position coordinates of the foreground pixel.

[0057] Step S122: randomly select a background pixel point from the background pixel set, and obtain the position coordinates of the background pixel point.

[0058] The implementation of the above steps S121 to S122 is as follows: Assume that the foreground pixel set is represented by ω F , and the background pixel set is represented by ω B , from the foreground pixel set ω F Randomly select a foreground pixel point from the matrix and obtain the position coordinates of the i-th foreground pixel point selected, which can be expressed as Then, from the background pixel set ω BRandomly select a background pixel point from the , and obtain the position coordinates of the i-th background pixel point selected, which can be expressed as

[0059] Step S123: construct a coordinate tuple according to the position coordinates of the foreground pixel point and the position coordinates of the background pixel point, and determine the coordinate tuple as the initial pixel pair solution representation of the pixel point.

[0060] The implementation method of the above step S123 is, for example: according to the position coordinates of the foreground pixel point and the position coordinates of the background pixels Construct a coordinate tuple, where the coordinate tuple can be expressed as Then, the coordinate tuple is determined as the initial pixel pair solution representation of the pixel point, that is, Since the above-mentioned use of position coordinates to construct the PPSR-PP of the pixel point introduces additional spatial information by using more decision variables to represent a pixel, the above-mentioned pixel pair optimization problem can be expressed by the following formula:

[0061]

[0062] Among them, Z represents the decision vector of the input image pixel pair optimization problem, H() represents the pixel pair evaluation function of the input image, and h i () represents the pixel pair evaluation function of the i-th unknown pixel to be solved, z i represents the decision vector of the i-th unknown pixel pair to be solved, The subscript representing the foreground pixel at the i-th pixel position is, The subscript representing the background pixel at the i-th pixel position, ω F represents the set of feasible foreground pixel subscripts (i.e., the foreground pixel set), ω B Represents the set of feasible background pixel subscripts (i.e., the background pixel set).

[0063] After using the position coordinates to construct the PPSR-PP of the pixel, this PPSR-PP can overcome the defects of missing adjacent solutions and discontinuous values ​​of the adjacent solution evaluation function in the decision space caused by PPSR-LI. Specifically, PPSR-PP is constructed by the position coordinates of the foreground pixel set and the background pixel set, that is, it is encoded by the row and column subscripts of the foreground pixels and the background pixels, rather than by the linear indexes of the foreground pixels and the background pixels. In measuring the similarity of feasible solutions or neighborhood relationships, the spatial correlation reflected by PPSR-PP can play a very important role. Due to the prior knowledge of the local smoothness of the image, the colors of adjacent pixels are usually similar. However, the linear index reflected by PPSR-LI can only explain the one-dimensional spatial relationship of the candidate solutions, resulting in the heuristic optimization algorithm using PPSR-LI can only provide surrounding adjacent pixels in one dimension. In contrast, the PPSR-PP constructed using the position coordinates of the foreground pixel set and the position coordinates of the background pixel set can reflect the two-dimensional spatial relationship of the candidate solutions. It is precisely because PPSR-PP can provide accurate distance measurement of candidate solutions at the representation level and build a more complete comprehensive neighborhood structure for the heuristic optimization algorithm in the decision space that PPSR-PP can overcome the defect of missing adjacent solutions caused by PPSR-LI and the defect of discontinuous values ​​of the evaluation function of adjacent solutions in the decision space of the heuristic optimization algorithm.

[0064] As an optional implementation of the above step S130, the above implementation of using a heuristic optimization algorithm to solve and optimize the initial pixel pair solution representation of the pixel point may include:

[0065] Step S131: cross-mutate the initial pixel pair solution representation of the pixel point through the cross-mutation process of the heuristic optimization algorithm to obtain a population of mutated candidate pixel pairs, wherein the population of mutated candidate pixel pairs includes multiple candidate pixel pair solution representations.

[0066] For example, the implementation method of the above step S131 is as follows: the above-mentioned use of position coordinates to construct the PPSR-PP of the pixel point introduces additional spatial information by using more decision variables to represent a pixel, resulting in the above-mentioned candidate pixel pair solution representation falling into an irregular feasible area, and part of the candidate pixel pair solution representation may fall into an infeasible area. The initial pixel pair solution representation of the pixel point can be cross-mutated through the cross-mutation process of a heuristic optimization algorithm such as a genetic algorithm, a particle swarm algorithm, a simulated degradation algorithm or an ant colony algorithm, and a mutated candidate pixel pair population can be obtained. The mutated candidate pixel pair population includes multiple candidate pixel pair solution representations that fall into feasible areas and infeasible areas. Although many kinds of heuristic optimization algorithms can be used in practice, such as a genetic algorithm, a particle swarm algorithm, a simulated degradation algorithm or an ant colony algorithm, etc., for ease of understanding and explanation, the embodiment of the present application only takes the particle swarm algorithm as an example for explanation.

[0067] Step S132: for each candidate pixel pair solution representation in the multiple candidate pixel pair solution representations, the candidate pixel pair solution representation is repaired through the boundary of a feasible solution to obtain multiple repaired pixel pair solution representations.

[0068] It can be understood that in the process of cross-mutating the initial pixel pair solution representation of the pixel point through the heuristic optimization algorithm mentioned above, the mutation process will use various evolutionary operators such as initialization operator, mutation operator and recombination operator, which may cause the candidate pixel pair solution representation to be in an infeasible area (that is, beyond the boundary of the foreground area or the boundary of the background area). Therefore, for each candidate pixel pair solution representation in multiple candidate pixel pair solution representations, the candidate pixel pair solution representation can be repaired through the boundary of the feasible solution to ensure that each candidate pixel pair solution representation is feasible, that is, it does not exceed the boundary of the foreground area or the boundary of the background area.

[0069] Since the above-mentioned candidate pixel pair solution representation may fall into an irregular feasible area, and the irregular feasible area will bring challenges of nonlinear boundary constraints to pixel pair optimization, therefore, the embodiment of the present application can overcome the above challenges through a strategy called pixel pair repair based on boundary feasible solution mapping (Pixel Pair Repair based on Boundary Feasible Solution Mapping, PPR-BFSM). The basic idea of ​​the PPR-BFSM strategy is to relax the nonlinear boundary constraints brought by PPSR-PP, and to repair the boundary of the candidate pixel pair solution representation through the feasible solution by mapping the infeasible solution to the boundary of the feasible solution area. The above-mentioned PPR-BFSM strategy can be expressed by the formula as follows:

[0070]

[0071]

[0072] Among them, Z represents the decision vector of the input image pixel pair optimization problem, H() represents the pixel pair evaluation function of the input image, and h i () represents the pixel pair evaluation function of the i-th unknown pixel to be solved, z i represents the decision vector of the i-th unknown pixel pair to be solved, The subscript representing the foreground pixel at the i-th pixel position is, The subscript represents the background pixel at the i-th pixel position, w represents the width of the input image to be processed, and h represents the height of the input image to be processed.

[0073] Assumptions represents the original foreground pixel at the i-th pixel position (i.e., the foreground pixel before restoration), and represents the foreground pixel after repair at the i-th pixel position, then the above operation of repairing the boundary of the candidate pixel pair through the feasible solution can be expressed by the formula:

[0074]

[0075] Where f() represents the mapping operation from an infeasible solution to a feasible solution (described in detail below), represents the original foreground pixel at the i-th pixel position (i.e. the foreground pixel before restoration), represents the inpainted foreground pixel at the i-th pixel position, ω F Represents the set of feasible foreground pixel indices (i.e., the foreground pixel set).

[0076] Step S133: for each of the multiple repaired pixel pair solution representations, use a preset evaluation function to evaluate the repaired pixel pair solution representation to obtain evaluation values ​​of the multiple repaired pixel pair solution representations.

[0077] An implementation example of the above-mentioned step S133 is as follows: for each repaired pixel pair solution representation in the multiple repaired pixel pair solution representations, a single function such as a color consistency evaluation function, a color difference evaluation function, a color similarity evaluation function, an edge consistency evaluation function, a texture consistency evaluation function, a spatial consistency evaluation function and / or an illumination consistency evaluation function or any combination of several functions is used as a preset evaluation function to evaluate the repaired pixel pair solution representation and obtain evaluation values ​​of the multiple repaired pixel pair solution representations.

[0078] Step S134: selecting a repaired pixel pair solution representation with a minimum evaluation value from multiple repaired pixel pair solution representations.

[0079] Step S135: adding the repaired pixel pair solution representation with the minimum evaluation value to the optimized pixel pair solution representation.

[0080] The implementation method of the above steps S134 to S135 is, for example: selecting the repaired pixel pair solution representation with the minimum evaluation value from multiple repaired pixel pair solution representations, and adding the repaired pixel pair solution representation with the minimum evaluation value to the optimized pixel pair solution representation to obtain the final optimized pixel pair solution representation. In the implementation process of the above scheme, by repairing the boundary of the feasible solution through the candidate pixel pair solution representation, it can be ensured that the generated candidate solution is still within a reasonable range, avoiding the existence of invalid solutions, thereby improving the quality of the pixel pair solution representation, and through cross-mutation and boundary repair, the algorithm can dynamically adapt to different image features and improve adaptability to different scenes.

[0081] As an optional implementation of the above step S132, the above implementation of repairing the boundary of the candidate pixel pair solution through a feasible solution may include:

[0082] Step S132a: parse out candidate pixel points from the candidate pixel pair solution.

[0083] The implementation method of the above step S132a is, for example, to use the formula The candidate pixel points are parsed from the candidate pixel pair solution; wherein, The position coordinates of the foreground pixels representing the unknown candidate pixels, Represents the position coordinates of the background pixels of the unknown candidate pixels.

[0084] Step S132b: using the foreground pixel boundary and the background pixel boundary of the image to be processed as constraints, performing nearest neighbor transformation on all pixel points of the image to be processed to obtain pixel points after neighbor transformation.

[0085] As an optional implementation of the above step S132b, the above candidate pixels may include: candidate foreground pixels and candidate background pixels; the above nearest neighbor transformation of all pixels of the image to be processed means that the determined infeasible foreground / background pixel will be transformed into a feasible foreground / background pixel, and the implementation of the nearest neighbor transformation may include: finding a first pixel point that is closest to each pixel point in the image to be processed from all foreground pixels of the image to be processed; finding a second pixel point that is closest to each pixel point in the image to be processed from all background pixels of the image to be processed. The distance between the above pixel point and each pixel point in the image can be measured using Euclidean distance or Euclidean space distance.

[0086] In the implementation process of the above scheme, by performing nearest neighbor transformation on all pixels of the image and taking the boundaries of the foreground pixels and background pixels of the image to be processed as constraints, it is ensured that the generated solution representation does not exceed the valid range of the image, thereby avoiding the generation of invalid pixel pair solution representations and improving the feasibility of pixel pair solution representations.

[0087] Step S132c: Determine whether the candidate pixel point included in the candidate pixel pair representation is a feasible solution representation.

[0088] Step S132d: If the candidate pixel points included in the candidate pixel pair representation are feasible solution representations, construct a repaired pixel pair solution representation based on the candidate pixel points.

[0089] Step S132e: If the candidate pixel points included in the candidate pixel pair representation are not feasible solution representations, the candidate pixel points are mapped to corresponding neighboring transformed pixel points to obtain mapped candidate pixel points, and a repaired pixel pair solution representation is constructed based on the mapped candidate pixel points.

[0090] As an optional implementation of the above step S132e, the implementation of the nearest neighbor transformation of the candidate pixel points contained in all candidate pixel pairs in the image to be processed may include: if the candidate foreground pixel point is not an infeasible solution, it means that the candidate foreground pixel point is a feasible solution representation, then the candidate foreground pixel point is optimized, and the optimized candidate foreground pixel point is determined as the candidate foreground pixel point after mapping; wherein, the above optimization of the candidate foreground pixel point includes: boundary optimization and color consistency optimization, etc., the boundary optimization may be a boundary adjustment optimization that can make the candidate pixel point more in line with the boundary condition by adjusting it if the candidate pixel point is close to the boundary, and the color consistency optimization may be a color matching optimization that ensures that the similarity between the color of the candidate pixel point and the surrounding pixel colors is greater than a similarity threshold. If the candidate foreground pixel point is an infeasible solution, the first pixel point corresponding to the candidate foreground pixel point is determined as the candidate foreground pixel point after mapping. If the candidate background pixel is not an infeasible solution, it means that the candidate background pixel is a feasible solution, then the candidate foreground pixel is optimized, and the optimized candidate background pixel is determined as the candidate background pixel after mapping; wherein, the above-mentioned optimization of the candidate background pixel includes: boundary optimization and color consistency optimization, etc., the boundary optimization can be a boundary adjustment optimization that can make the candidate pixel closer to the boundary to make it more consistent with the boundary conditions, and the color consistency optimization can be a color matching optimization that ensures that the similarity between the color of the candidate pixel and the surrounding pixel colors is greater than the similarity threshold. If the candidate background pixel is an infeasible solution, the second pixel corresponding to the candidate background pixel is determined as the candidate background pixel after mapping.

[0091] In the implementation process of the above scheme, by performing nearest neighbor transformation on all pixels of the image to be processed and taking the foreground pixel boundary and background pixel boundary of the image to be processed as constraints, it is ensured that the generated solution representation does not exceed the valid range of the image, thereby avoiding the generation of invalid pixel pair solution representation, thereby improving the feasibility of pixel pair solution representation.

[0092] The implementation method of the above steps S140 to S150 is, for example: after obtaining the optimized pixel pair solution representation, that is, after finding the optimized pixel pair solution representation for each unknown pixel, the above formula (2) can be used to calculate the opacity of each pixel in the pixel set of the area to be solved, and obtain the opacity value of each pixel. Finally, the opacity matrix is ​​constructed by the opacity value of each pixel in the multiple pixels, and this opacity matrix can be used as a transparency mask of the foreground object to cut out the image to be processed, thereby obtaining the final cutout result.

[0093] See also Figure 5 The flowchart of the image cutout device provided in the embodiment of the present application is shown; the embodiment of the present application provides an image cutout device 200, comprising:

[0094] The processed image acquisition module 210 is used to acquire the image to be processed, and the image to be processed includes: a pixel set of the area to be solved, a foreground pixel set and a background pixel set.

[0095] The pixel pair solution construction module 220 is used to construct an initial pixel pair solution representation of each pixel point in the pixel set of the area to be solved according to the position coordinates of the foreground pixel set and the position coordinates of the background pixel set.

[0096] The pixel pair solution optimization module 230 is used to use a heuristic optimization algorithm to solve and optimize the initial pixel pair solution representation of the pixel point to obtain an optimized pixel pair solution representation.

[0097] The opacity calculation module 240 is used to calculate the opacity of each pixel in the pixel set of the area to be solved according to the optimized pixel pair solution to obtain an opacity matrix.

[0098] The matting result generating module 250 is used to generate the matting result of the image to be processed according to the opacity matrix.

[0099] As an optional implementation of the above device, the pixel pair solution construction module includes:

[0100] The foreground coordinate acquisition submodule is used to randomly select a foreground pixel point from the foreground pixel set and obtain the position coordinates of the foreground pixel point.

[0101] The background coordinate acquisition submodule is used to randomly select a background pixel point from the background pixel set and obtain the position coordinates of the background pixel point.

[0102] The initial solution representation construction module is used to construct a coordinate tuple according to the position coordinates of the foreground pixel point and the position coordinates of the background pixel point, and determine the coordinate tuple as the initial pixel pair solution representation of the pixel point.

[0103] As an optional implementation of the above device, the pixel pair solution optimization module includes:

[0104] The initial cross-mutation submodule is used to perform cross-mutation on the initial pixel pair solution representation of the pixel point through the cross-mutation process of the heuristic optimization algorithm to obtain a population of mutated candidate pixel pairs, which includes multiple candidate pixel pair solution representations.

[0105] The representation boundary repair submodule is used to repair each candidate pixel pair solution representation in multiple candidate pixel pair solution representations through the boundary of a feasible solution to obtain multiple repaired pixel pair solution representations.

[0106] The pixel pair solution evaluation submodule is used to evaluate each of the multiple repaired pixel pair solution representations using a preset evaluation function to obtain evaluation values ​​of the multiple repaired pixel pair solution representations.

[0107] The pixel pair solution selection submodule is used to select a repaired pixel pair solution representation with a minimum evaluation value from multiple repaired pixel pair solution representations.

[0108] The pixel pair solution adding submodule is used to add the repaired pixel pair solution representation of the minimum evaluation value to the optimized pixel pair solution representation.

[0109] As an optional implementation of the above device, a boundary repair submodule is represented, including:

[0110] The candidate pixel parsing unit is used to parse out candidate pixel points from the candidate pixel pair solution representation.

[0111] The pixel transformation unit is used to perform nearest neighbor transformation on all pixel points of the image to be processed by taking the foreground pixel boundary and the background pixel boundary of the image to be processed as constraint conditions to obtain the pixel points after the neighbor transformation.

[0112] The feasible solution judgment unit is used to judge whether the candidate pixel point contained in the candidate pixel pair representation is a feasible solution representation.

[0113] The pixel restoration mapping unit is used to construct a restoration pixel pair solution representation based on the candidate pixel points if the candidate pixel points included in the candidate pixel pair are feasible solution representations; if the candidate pixel points included in the candidate pixel pair are infeasible solution representations, map the candidate pixel points to corresponding neighboring transformed pixel points, and construct a restoration pixel pair solution representation based on the mapped candidate pixel points.

[0114] As an optional implementation of the above device, the candidate pixel points include: candidate foreground pixel points and candidate background pixel points; the pixel conversion unit includes:

[0115] The first pixel point transformation subunit is used to find a first pixel point which is closest to each pixel point in the image to be processed from all foreground pixels in the image to be processed.

[0116] The second pixel point transformation subunit is used to find a second pixel point which is closest to each pixel point in the image to be processed from all background pixels of the image to be processed.

[0117] As an optional implementation of the above device, the pixel repair mapping unit includes:

[0118] The foreground pixel determination subunit is used to optimize the candidate foreground pixel if the candidate foreground pixel is not an infeasible solution, and determine the optimized candidate foreground pixel as the mapped candidate foreground pixel; if the candidate foreground pixel is an infeasible solution, determine the first pixel corresponding to the candidate foreground pixel as the mapped candidate foreground pixel.

[0119] The background pixel determination subunit is used to optimize the candidate foreground pixel if the candidate background pixel is not an infeasible solution, and determine the optimized candidate background pixel as the mapped candidate background pixel; if the candidate background pixel is an infeasible solution, determine the second pixel corresponding to the candidate background pixel as the mapped candidate background pixel.

[0120] It should be understood that the device corresponds to the above-mentioned image cutout method embodiment and can perform the various steps involved in the above-mentioned method embodiment. The specific functions of the device can be found in the description above, and the detailed description is appropriately omitted here. The device includes at least one software function module that can be stored in a memory in the form of software or firmware or solidified in the operating system (OS) of the device.

[0121] See also Figure 6The electronic device 300 provided in the embodiment of the present application includes: a processor 310 and a memory 320, wherein the memory 320 stores machine-readable instructions executable by the processor 310, and when the machine-readable instructions are executed by the processor 310, the above method is executed.

[0122] The embodiment of the present application also provides a computer-readable storage medium 330, on which a computer program is stored, and the computer program is executed by the processor 310 to execute the above method. The computer-readable storage medium 330 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, disk or optical disk.

[0123] The embodiment of the present application also provides a computer program product, including: a computer program or a computer instruction, and the computer program or the computer instruction executes the method described above when executed by a processor.

[0124] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0125] In several embodiments provided by the embodiments of the present application, it should be understood that the disclosed device and method can also be implemented in other ways. The device embodiments described above are only schematic, for example, the flowcharts and block diagrams in the accompanying drawings show the possible implementation architecture, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a part of a module, a program segment or a code, and a part of a module, a program segment or a code includes one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also be different from the order of occurrence marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which is mainly determined by the functions involved.

[0126] In addition, each functional module of each embodiment in the embodiment of the present application can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part. In addition, in the description of this specification, the description of reference terms "one embodiment", "some embodiments", "example", "specific example", "some examples", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiment of the present application. In this specification, the schematic representation of the above terms does not necessarily target the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples without contradiction.

[0127] The above description is only an optional implementation manner of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed in the embodiments of the present application, which should be covered within the protection scope of the embodiments of the present application.

Claims

1. An image cutout method, characterized in that: include: Acquire an image to be processed, wherein the image to be processed includes: a pixel set of a region to be solved, a foreground pixel set, and a background pixel set; For each pixel point in the pixel set of the area to be solved, construct an initial pixel pair solution representation of the pixel point according to the position coordinates of the foreground pixel set and the position coordinates of the background pixel set; Using a heuristic optimization algorithm to solve and optimize the initial pixel pair solution representation of the pixel point, to obtain an optimized pixel pair solution representation; Calculate the opacity of each pixel in the pixel set of the area to be solved according to the optimized pixel pair solution to obtain an opacity matrix; A matting result of the image to be processed is generated according to the opacity matrix.

2. The method according to claim 1, characterized in that The constructing the initial pixel pair solution representation of the pixel point according to the position coordinates of the foreground pixel set and the position coordinates of the background pixel set comprises: Randomly select a foreground pixel point from the foreground pixel set, and obtain the position coordinates of the foreground pixel point; Randomly select a background pixel point from the background pixel set, and obtain the position coordinates of the background pixel point; A coordinate tuple is constructed according to the position coordinates of the foreground pixel point and the position coordinates of the background pixel point, and the coordinate tuple is determined as an initial pixel pair solution representation of the pixel point.

3. The method according to claim 1, characterized in that The method of using a heuristic optimization algorithm to solve and optimize the initial pixel pair solution representation of the pixel point includes: Performing cross-mutation on the initial pixel pair solution representation of the pixel point through the cross-mutation process of the heuristic optimization algorithm to obtain a population of mutated candidate pixel pairs, wherein the population of mutated candidate pixel pairs includes a plurality of candidate pixel pair solution representations; For each candidate pixel pair solution representation in the plurality of candidate pixel pair solution representations, repairing the candidate pixel pair solution representation through a boundary of a feasible solution to obtain a plurality of repaired pixel pair solution representations; For each of the plurality of repaired pixel pair solution representations, using a preset evaluation function to evaluate the repaired pixel pair solution representation to obtain evaluation values ​​of the plurality of repaired pixel pair solution representations; Selecting a repaired pixel pair solution representation with a minimum evaluation value from the multiple repaired pixel pair solution representations; The inpainted pixel pair solution representation of the minimum evaluation value is added to the optimized pixel pair solution representation.

4. The method according to claim 3, characterized in that The step of restoring the boundary of the candidate pixel pair solution through a feasible solution comprises: Resolving candidate pixel points from the candidate pixel pair solution; Taking the foreground pixel boundary and the background pixel boundary of the image to be processed as constraint conditions, performing nearest neighbor transformation on all pixel points of the image to be processed to obtain pixel points after neighbor transformation; Determine whether the candidate pixel point included in the candidate pixel pair representation is a feasible solution representation; If so, construct the repaired pixel pair solution representation based on the candidate pixel points; otherwise, map the candidate pixel points to map them to the corresponding neighboring transformed pixel points, obtain the mapped candidate pixel points, and construct the repaired pixel pair solution representation based on the mapped candidate pixel points.

5. The method according to claim 4, characterized in that The candidate pixels include candidate foreground pixels and candidate background pixels; the nearest neighbor transformation of all pixels of the image to be processed includes: Finding a first pixel point that is closest to each pixel point in the image to be processed from all foreground pixels in the image to be processed; A second pixel point which is closest to each pixel point in the image to be processed is found from all background pixels of the image to be processed.

6. The method according to claim 5, characterized in that Mapping the candidate pixel points includes: If the candidate foreground pixel point is not an infeasible solution, the candidate foreground pixel point is optimized, and the optimized candidate foreground pixel point is determined as the mapped candidate foreground pixel point; otherwise, the first pixel point corresponding to the candidate foreground pixel point is determined as the mapped candidate pixel point; If the candidate background pixel point is not an infeasible solution, the candidate foreground pixel point is optimized and the optimized candidate background pixel point is determined as the mapped candidate background pixel point; otherwise, the second pixel point corresponding to the candidate background pixel point is determined as the mapped candidate pixel point.

7. An image cutout device, characterized in that: include: The processing image acquisition module is used to acquire the image to be processed, wherein the image to be processed includes: a pixel set of the area to be solved, a foreground pixel set and a background pixel set; A pixel pair solution construction module, used for constructing an initial pixel pair solution representation of each pixel point in the pixel set of the to-be-solved area according to the position coordinates of the foreground pixel set and the position coordinates of the background pixel set; A pixel pair solution optimization module is used to use a heuristic optimization algorithm to solve and optimize the initial pixel pair solution representation of the pixel point to obtain an optimized pixel pair solution representation; An opacity calculation module, used to calculate the opacity of each pixel in the pixel set of the area to be solved according to the optimized pixel pair solution representation, to obtain an opacity matrix; The cutout result generating module is used to generate the cutout result of the image to be processed according to the opacity matrix.

8. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions are executed by the processor to perform any method according to claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is executed.

10. A computer program product, characterized in that include: A computer program or a computer instruction, wherein when the computer program or the computer instruction is executed by a processor, the method according to any one of claims 1 to 6 is executed.

Images