Image processing method and device

Abstract

This application discloses an image processing method and apparatus, belonging to the field of image processing technology. The image processing method includes: when a first region of a target object in an image to be processed is incomplete, inputting the image to be processed into a target convolutional neural network and outputting a first image, the first image including a first region of the target object and a second region not containing the target object; traversing each column of pixels in the first image to obtain target pixels in each column, wherein the target pixels are pixels in the first region surrounding the second region; determining a first target edge of the first image based on the target pixels; and processing the first image according to the first target edge.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and specifically relates to an image processing method and apparatus. Background Technology

[0002] With the widespread use of mobile devices, more and more people are using them for everyday photography. However, not all captured images achieve the desired results. Therefore, to ensure that the images meet expectations, many people choose to use beauty-enhancing camera apps for daily shooting, such as optimizing the lips.

[0003] However, many images may contain incomplete target objects, making it difficult to accurately identify the edges of the target objects and optimize them precisely. Summary of the Invention

[0004] The purpose of this application is to provide an image processing method and apparatus that can solve the problem that when there is an incomplete target object in the image, the edge of the target object cannot be well identified and the target object in the image cannot be accurately optimized.

[0005] In a first aspect, embodiments of this application provide an image processing method, the method comprising:

[0006] If the first region of the target object in the image to be processed is incomplete, the image to be processed is input into the target convolutional neural network and the first image is output, the first image including the first region of the target object and the second region of the non-target object;

[0007] Traverse each column of pixels in the first image to obtain the target pixel in each column of pixels, wherein the target pixel is the pixel in the second region surrounded by the pixel region of the first region;

[0008] The first target edge of the first image is determined based on the target pixel, and the first image is processed according to the first target edge.

[0009] Secondly, embodiments of this application provide an image processing apparatus, including:

[0010] The output module is used to input the image to be processed into a target convolutional neural network and output a first image when the first region of the target object in the image to be processed is incomplete, the first image including the first region of the target object and the second region of the non-target object;

[0011] The first acquisition module is used to traverse each column of pixels in the first image to obtain the target pixel in each column of pixels, wherein the target pixel is the pixel in the second region surrounded by the pixel region of the first region;

[0012] The first processing module is used to determine a first target edge of the first image based on the target pixel, and to process the first image according to the first target edge.

[0013] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0014] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0015] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0016] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0017] In this embodiment, when the first region of the target object in the image to be processed is incomplete, the image to be processed is input into a target convolutional neural network to segment the image into a first image including the first region of the target object and a second region of non-target object. By traversing each column of pixels in the first image, the pixels of the second region surrounded by the pixel region of the first region are found, and the first target edge in the first image is determined. This provides a clear first target edge in the first image, solving the problem that the edge of the target object cannot be identified when the first region of the target object is incomplete, which leads to the inaccurate optimization of the first image. Based on the first target edge, the optimization area of ​​the target object can be well determined, and the target object in the first image can be accurately optimized. Attached Figure Description

[0018] Figure 1 This is one of the flowcharts of the image processing method provided in the embodiments of this application;

[0019] Figure 2 This is a schematic diagram of the target convolutional neural network image segmentation provided in the embodiments of this application;

[0020] Figure 3 This is one of the schematic diagrams of the first image provided in the embodiments of this application;

[0021] Figure 4 This is a second schematic diagram of the first image provided in the embodiments of this application;

[0022] Figure 5 This is a schematic diagram of the lip extending beyond the camera boundary, provided in an embodiment of this application.

[0023] Figure 6 This is a schematic diagram of the feature points of the lips provided in the embodiments of this application;

[0024] Figure 7 This is a schematic diagram of the first image provided in the embodiments of this application;

[0025] Figure 8 This is one of the schematic diagrams of the first image after pixel filling provided in the embodiments of this application;

[0026] Figure 9 This is the fourth schematic diagram of the first image provided in the embodiments of this application;

[0027] Figure 10 This is a second schematic diagram of the first image after pixel filling provided in the embodiments of this application;

[0028] Figure 11 This is the second flowchart of the image processing method provided in the embodiments of this application;

[0029] Figure 12 This is a schematic diagram of the circumscribed rectangle of the complete lip provided in the embodiments of this application;

[0030] Figure 13 This is a schematic diagram of the image corresponding to the smallest bounding rectangle of the complete lip provided in the embodiments of this application after being expanded.

[0031] Figure 14 This is a schematic diagram of the image to be processed provided in an embodiment of this application;

[0032] Figure 15 This is a schematic diagram of mask fusion provided in an embodiment of this application;

[0033] Figure 16 This is a schematic diagram of the Final mask provided in an embodiment of this application;

[0034] Figure 17 This is a schematic diagram of the structure of the image processing apparatus provided in the embodiments of this application;

[0035] Figure 18This is one of the structural schematic diagrams of the electronic device provided in the embodiments of this application;

[0036] Figure 19 This is a second hardware schematic diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0037] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0038] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0039] The image processing method and apparatus provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0040] Figure 1 This is one of the flowcharts of the image processing method provided in the embodiments of this application, such as... Figure 1 As shown, the image processing method includes steps 110, 120 and 130.

[0041] Step 110: If the first region of the target object in the image to be processed is incomplete, input the image to be processed into the target convolutional neural network and output a first image, wherein the first image includes the first region of the target object and the second region of the non-target object.

[0042] In this embodiment of the application, the image to be processed may be an image containing a target object in a human face, a video frame image during video shooting, an image obtained from an image shooting preview interface, a received or locally stored image, or more specifically, a pre-processed image.

[0043] In this embodiment of the application, the target object includes, but is not limited to, the lips. In this embodiment of the invention, the lips are used as an example for illustration.

[0044] The first region described in this application embodiment can specifically be the lip region of the target object in the image to be processed, and more specifically the upper lip region and lower lip region of the target object in the image to be processed.

[0045] Optionally, in the embodiments of this application, the image to be processed may more specifically be a pre-processed image.

[0046] The preprocessing can be specifically as follows: based on the feature point analysis algorithm of the lips, obtain the feature point set of the corresponding complete lips based on the incomplete lip region, obtain the minimum bounding rectangle of the complete lips based on the feature point set, and perform appropriate expansion processing on the minimum bounding rectangle. Then, perform cropping processing on the original image based on the expanded rectangle to obtain the image to be processed.

[0047] In this embodiment of the application, the incompleteness of the first region may mean that the lip region does not include the complete lip region, but only includes a part of the lip region. For example, the lip region in the first region only accounts for 1 / 2, 1 / 3, 1 / 4 of the complete lip region, etc.

[0048] The target convolutional neural network described in this application embodiment can effectively segment images and distinguish between a first region of a target object and a second region of a non-target object in the image. It can be obtained by training with a sample image set, and each image in the sample image set can be an image including a first region label and / or a second region label.

[0049] The second region described in this embodiment is all regions in the first image other than the first region.

[0050] After the image to be processed is input into the target convolutional neural network, the target convolutional neural network will segment the image to be processed, distinguish the first region of the lips and the second region of the non-lips in the image to be processed, and output a first image composed of sub-images of the first region and sub-images of the second region.

[0051] For example, Figure 2 This is a schematic diagram of the target convolutional neural network image segmentation provided in the embodiments of this application, such as... Figure 2 As shown, the target convolutional neural network extracts image features from the image to be processed through convolution and encodes them. During this process, the resolution of the feature map decreases, hence the term "downsampling." After image encoding, the convolutional neural network decodes the feature map through an upsampling process, ultimately obtaining a first image with the same resolution as the image to be processed.

[0052] In this embodiment of the application, the pixel values ​​of the first region of the target object and the second region of the non-target object in the first image are different. For example, the pixel value of the first region is 255 and the pixel value of the second region is 0; or the pixel value of the first region is 100 and the pixel value of the second region is 1.

[0053] Step 120: Traverse each column of pixels in the first image to obtain the target pixel in each column of pixels, wherein the target pixel is the pixel in the second region surrounded by the pixel region of the first region;

[0054] In this embodiment of the application, the target pixel is found by traversing each column of pixels in the first image.

[0055] In this embodiment of the application, the target pixel may specifically be a pixel in the region between the upper lip and the lower lip.

[0056] Specifically, when the lips are open, the target pixel is a pixel in the non-lip area surrounded by the upper and lower lips, and the target pixel may include one or more pixels.

[0057] For example, Figure 3 This is one of the schematic diagrams of the first image provided in the embodiments of this application, such as... Figure 3 As shown, this is the region 10 where the target pixel is located when the lips are open.

[0058] In this embodiment, when the lips are closed, since the upper and lower lips are in a closed state, there are no other areas between the upper and lower lips, meaning there are no target pixels in the first image. However, since the lips are closed, it is not necessary to identify the inner edge of the lips, and there is no problem with poor optimization fit of the inner edge of the lips.

[0059] For example, Figure 4 This is a second schematic diagram of the first image provided in the embodiments of this application, as shown below. Figure 4 As shown, there is no region corresponding to the target pixel in the first image.

[0060] Step 130: Determine the first target edge of the first image based on the target pixel, and optimize the first image according to the first target edge.

[0061] In this embodiment, the first target edge can specifically be the inner edge of the lip.

[0062] In this embodiment of the application, determining the first target edge of the first image can be either taking the boundary of the pixel region composed of the target pixels as the inner edge of the lip of the first image, or taking the pixels in the target pixels that change from pixels in the first region to pixels in the second region as the inner edge of the lip of the first image. Specifically, the inner edge of the lip can be the junction between the lip region and the region between the upper and lower lips.

[0063] In this embodiment of the application, processing the first image based on the first target edge can specifically involve optimizing the first image based on the inner edge of the lips, or more specifically, using the inner edge of the lips as the inner edge of the lips of the first image to optimize the first image.

[0064] In this embodiment, the optimization can be based on the inner edge of the lips to determine the optimization range of the first image. Only the first image within the range of the inner edge of the lips is optimized, providing a clear inner edge of the lips for the first image. This solves the problem of poor fit of the inner edge of the lips caused by the inability to properly identify the inner edge of the lips when the lips are open. By performing beautification optimization on the first image based on the inner edge of the lips, the fit between the inner edge of the lips and the lips can be effectively improved.

[0065] In other embodiments, the optimization may specifically involve applying lip gloss based on the inner edge of the lips in the first image, or performing beautification optimizations such as skin smoothing on the inner edge of the lips in the first image.

[0066] The image processing method provided in this application, when the first region of the target object in the image to be processed is incomplete, inputs the image to be processed into a target convolutional neural network, segments the image into a first image including the first region of the target object and a second region of non-target object, and finds the pixels of the second region surrounded by the pixel region of the first region by traversing each column of pixels in the first image, thereby determining the first target edge in the first image. This solves the problem that the edge of the target object cannot be identified when the first region of the target object is incomplete, which leads to the inaccurate optimization of the first image. Based on the first target edge, the optimization region of the target object can be well determined, and the target object in the first image can be accurately optimized.

[0067] Optionally, if the first region of the target object in the image to be processed is incomplete, the method may further include:

[0068] Obtain the feature point set of the complete region corresponding to the first region of the target object;

[0069] In this embodiment of the application, the complete region corresponding to the first region may specifically include the lip region of the first region and the lip region outside the terminal interface.

[0070] In this embodiment, the feature point set of the complete region can specifically be a set of feature points corresponding to the complete lip region. The feature points of the complete lip region can be obtained specifically based on the feature points of the incomplete lip region through a feature point localization algorithm. Specifically, these feature points can be the outer edge points of the lips.

[0071] For example, Figure 5 This is a schematic diagram of the lip extending beyond the camera boundary, as provided in the embodiments of this application. Figure 5 As shown, with the top left corner of the camera interface as the origin, the horizontal axis is the X-axis, and the vertical axis is the Y-axis. This complete lip region includes the first lip region 21 corresponding to the first region of the target object in the image to be processed on the phone screen, and the second lip region 22 corresponding to the lip region outside the phone screen. In this coordinate system, there are 12 feature points: A0, A1, ..., A11. The coordinates of A0, A1, ..., A11 can also be obtained from... Figure 5 get.

[0072] Curve fitting is performed on N subsets of feature points in the feature point set to obtain the edge fitting curve of each subset of feature points. The N subsets of feature points are obtained by grouping the feature point set based on the target feature points in the feature point set. The target feature points include the leftmost feature point of the complete region, the rightmost feature point of the complete region, the bottom feature point of the complete region, and the valley feature point of the complete region in the feature point set. N is an integer greater than 1.

[0073] In this embodiment of the application, the feature point subset may specifically refer to the set of feature points in any group obtained after grouping the feature point set based on the target feature point, and N can be determined according to the number of groups.

[0074] In this embodiment, the target feature point may specifically include multiple feature points of the complete lip region, specifically the leftmost feature point, the rightmost feature point, the bottom feature point, and the valley feature point of the complete lip region. These target feature points can divide the outer edge of the lip region into multiple smooth curves. The valley feature point of the complete lip region corresponds to the lowest point in the center of the upper lip; the bottom feature point corresponds to the lowest point of the lower lip; the leftmost feature point is the leftmost feature point at the junction of the upper and lower lips, more specifically, the left corner of the mouth; and the rightmost feature point is the rightmost feature point at the junction of the upper and lower lips, more specifically, the right corner of the mouth.

[0075] For example, Figure 6This is a schematic diagram of the feature points of the lips provided in the embodiments of this application, such as... Figure 6 As shown, the leftmost feature point of the complete lip region is A0, the rightmost feature point is A6, the bottom feature point is A9, and the valley feature point is A3. The feature point set is divided into four groups using A0, A3, A6, and A9: Group 1: A0, A1, A2, A3; Group 2: A3, A4, A5, A6; Group 3: A0, A11, A10, A9; Group 4: A6, A7, A8, A9. By fitting curves to each of these four groups of feature points, the outer edge of the lip region can be divided into multiple smooth curves.

[0076] In this embodiment of the application, the edge fitting curve can specifically be a curve obtained by establishing data relationships based on the discrete feature points in each feature point subset, and then finding a series of line segments to connect the discrete feature points in each feature point subset.

[0077] Based on N edge fitting curves, a second target edge of the first image is determined, and the first image is processed according to the second target edge.

[0078] In this embodiment of the application, the N edge fitting curves specifically refer to the edge fitting curves corresponding to the subsets of feature points in the N groups.

[0079] In this embodiment, determining the second target edge of the first image can be done by determining the outer edge of the lips in the first image. Specifically, the boundary of the pixel region formed by the pixels corresponding to the outermost boundary of the lips can be used as the second target edge of the first image. The outer edge of the lips can specifically be the outermost boundary of the lips.

[0080] In this embodiment of the application, processing the first image based on the second target edge can specifically involve taking the second target edge as the outer edge of the lips of the first image, determining the optimization area of ​​the first image based on the outer edge of the lips, and optimizing the first image based on the optimization area to obtain a clear and stable outer edge of the lips. Optimizing the first image using the outer edge of the lips avoids the problem of lip makeup jitter and ensures the stability of the outer edge of the lips.

[0081] In this embodiment of the application, the optimization can be based on determining the optimization range of the first image based on the outer edge of the lips, and optimizing the first image within the range of the outer edge of the lips.

[0082] The image processing method provided in this application embodiment obtains a feature point set including the complete region corresponding to the incomplete first region in the image to be processed, and performs curve fitting on the second target edge of the first image to obtain the edge fitting curve of the first image. This provides a clear second target edge of the first image and solves the problem that the edge of the target object cannot be identified when the first region of the target object is incomplete, which leads to the inaccurate optimization of the first image. Based on the obtained edge fitting curve of the first image, the optimization area of ​​the target object can be well determined, and the target object in the first image can be accurately optimized.

[0083] Optionally, after determining the second target edge of the first image based on N edge fitting curves, and processing the first image according to the second target edge, the method may further include:

[0084] The first image is processed based on the first target edge and the second target edge.

[0085] In this embodiment of the application, processing the first image based on the first target edge and the second target edge can specifically involve taking the first target edge as the inner edge of the lips of the first image and taking the second target edge as the outer edge of the lips of the first image. Optimizing the inner and outer edges of the lips of the first image based on the outer and inner edges of the lips can improve the fit between the inner and outer edges of the lips while ensuring the stability of the outer edge of the lips, thereby improving the stability of the beautification optimization and enhancing the user experience.

[0086] The image processing method provided in this application optimizes the first target edge and the second target edge of the first image based on the first target edge and the second target edge, providing clear first target edge and second target edge for the first image and improving the stability of the optimization of the first region in the first image.

[0087] Optionally, the step of traversing each column of pixels in the first image to obtain the target pixel in each column may specifically include:

[0088] Traverse each column of pixels in the first image and obtain the first changed pixel and the second changed pixel in each column of pixels. The first changed pixel is the pixel when the pixel corresponding to the first region changes to the pixel corresponding to the second region during the traversal of each column of pixels. The second changed pixel is the pixel when the pixel corresponding to the first region changes to the pixel corresponding to the second region during the traversal of each column of pixels, and then the pixel corresponding to the second region changes back to the pixel corresponding to the first region.

[0089] In this embodiment, the first changed pixel can specifically refer to the pixel in the region between the upper lip and the lower lip by traversing each column of pixels in the first image, and the pixel in that region that changes from the pixel corresponding to the first region to the pixel corresponding to the second region. Traversing each column of pixels in the first image can be done in a top-to-bottom order or in a bottom-to-top order.

[0090] In this embodiment of the application, the second changed pixel can specifically refer to the pixel in the region between the upper lip and the lower lip by traversing each column of pixels in the first image, and the pixel in the region that changed from the pixel corresponding to the first region to the pixel corresponding to the second region, and then changed back to the pixel corresponding to the first region.

[0091] The target pixel is defined as the pixel in each column that lies between the first and second changing pixel.

[0092] In this embodiment of the application, the pixels between the first changed pixel and the second changed pixel in each column of pixels are taken as the target pixel, thus obtaining the target pixel in each column of pixels in the first image.

[0093] For example, Figure 7 This is a schematic diagram of the first image provided in the embodiments of this application, as shown in Figure 3. Figure 7 As shown, since the obtained first image mask2 is the result of image segmentation, it is not stable enough at the edge of the camera preview interface (a common drawback of image segmentation). Therefore, it is necessary to remove its outline. Specifically, each column of pixels in the first image (mask2) is traversed. The starting position of each column of pixels is t0, and the ending position is th. Assume that the pixel corresponding to the first region is 255 and the pixel corresponding to the second region is 0.

[0094] When the visited pixel is 255, its position is recorded as t1; otherwise, the traversal continues. After t1, if the visited pixel is 0, its position is recorded as t2. If, after t2, the visited pixel is again 255, its position is recorded as t3, and the traversal continues. If, after t3, all pixels are 255, the traversal of this column ends. Then, the pixels between t0 and t2 are set to 255, the pixels between t2 and t3 are set to 0, and the pixels between t3 and th are set to 255.

[0095] If, after passing position t3, a pixel is found to be 0 again, and the position is recorded as t4, then the traversal of this column ends. Pixels between t0 and t2 are set to 255, pixels between t2 and t3 are set to 0, and pixels between t3 and th are set to 255.

[0096] Fill the pixels within the pixel region corresponding to the target pixel in the first image with 0, and fill the pixels in other regions with 255, thus obtaining the new lip mask3. Figure 8 As shown, Figure 8 This is one of the schematic diagrams of the first image after pixel filling provided in the embodiments of this application, wherein the pixels in the shaded area are the target pixels, which are 0, and the pixels in the non-shaded area are 255. The first region of mask3 has been eliminated.

[0097] Alternatively, in some other embodiments of this application, Figure 9 This is the fourth schematic diagram of the first image provided in the embodiments of this application, as shown below. Figure 9 As shown, with the lips closed, each column of pixels in the first image (mask2) is traversed. The starting position of each column of pixels is t0, and the ending position is th. It is assumed that the pixel corresponding to the first region is 255, and the pixel corresponding to the second region is 0.

[0098] When the visited pixel is 255, the position is recorded as t5; otherwise, the traversal continues. After t5, if a visited pixel is 0, the position is recorded as t6. After t6, if all pixels are 0, the traversal of this column ends, and all pixels in this column are filled with 255.

[0099] like Figure 10 As shown, Figure 10 This is a second schematic diagram of the first image after pixel filling provided in the embodiments of this application. There are no target pixels in the mask3, and the first region of the mask3 has been eliminated.

[0100] Optionally, each pixel in mask1 and mask3 is compared one by one, and the minimum value of the pixel at the same position is taken as the pixel value at the corresponding position in the final mask. This process is called mask fusion. The comparison method is as follows: following the row-first and column-second order, the pixels at the same position in mask1 and mask3 are traversed and compared one by one, and the minimum value of the two pixels is taken as the pixel value on the final mask. After the traversal is completed, the final mask is obtained, denoted as the final mask.

[0101] The image processing method provided in this application embodiment, by traversing each column of pixels in the first image and obtaining the first changed pixel and the second changed pixel in each column of pixels, can accurately and effectively find the target pixel in each column of pixels in the first image, and provide a clear first target edge for the first image based on the target pixel. Based on the first target edge, the optimization area of ​​the target object can be well determined, and the target object in the first image can be accurately optimized.

[0102] Optionally, the step of performing curve fitting based on N subsets of feature points in the feature point set to obtain the edge fitting curve for each subset of feature points may specifically include:

[0103] Obtain M consecutive feature point pairs from each of the feature point subsets;

[0104] In this embodiment of the application, each feature point pair may specifically include two feature points, and consecutive feature point pairs may specifically refer to two adjacent feature point pairs having a common feature point.

[0105] Based on each of the feature point pairs, construct a curve expression for each of the feature point pairs;

[0106] In this embodiment of the application, one of the feature points in each feature point pair is used as the independent variable and the other as the dependent variable. A curve expression between the feature points in each feature point pair is established by combining preset coefficients. The curve expression can be specifically a function expression.

[0107] Solve for each curve expression to obtain the fitted curve for each feature point pair;

[0108] In this embodiment, the feature points in each feature point pair are substituted into their corresponding function expressions, and a system of equations is established by combining the continuity of the derivatives of each function expression at each feature point. The system of equations is solved to obtain each function expression, and the fitting curve of each feature point pair is obtained based on the function expression.

[0109] Based on each of the fitted curves, the edge fitting curves of each subset of feature points are obtained.

[0110] In this embodiment of the application, the edge fitting curve of each feature point pair is obtained by fitting the curves corresponding to N function expressions.

[0111] Optionally, Figure 11 This is the second flowchart of the image processing method provided in the embodiments of this application, as shown below. Figure 11As shown, the image processing method may include: Step 1110, acquiring the image to be processed; Step 1120, fitting the outer edge of the lip region in the image to be processed using the feature point set corresponding to the complete lip to obtain mask1; Step 1130, inputting the image to be processed into a target convolutional neural network for segmentation to obtain mask2; Step 1140, traversing each column of pixel points in mask2; Step 1150, determining the pixel situation between t0 and th; Step 1160, only t0, t5, t6, and th exist; all pixels in this column are set to 255; Step 1170: Only t0, t1, t2, t3, and th exist; the pixels between t0 and t2 are set to 255, the pixels between t2 and t3 are set to 0, and the pixels between t3 and th are set to 0. Step 1180: Only t0, t1, t2, t3, t4, and th exist; the pixels between t0 and t2 are set to 255, the pixels between t2 and t3 are set to 0, and the pixels between t3 and th are set to 0. Step 1190: Generate mask3. Step 11901: Take the minimum value according to the pixel. Step 11902: Output the final mask.

[0112] Optionally, the preprocessing may specifically involve obtaining a set of feature points for the corresponding complete region based on the incomplete first region using a feature point analysis algorithm, obtaining the minimum bounding rectangle of the complete region based on the feature point set, appropriately expanding the minimum bounding rectangle, and cropping the original image based on the expanded rectangle to obtain the image to be processed.

[0113] For example, the feature point set is divided into four subsets: Subset 1: A0, A1, A2, A3; Subset 2: A3, A4, A5, A6; Subset 3: A0, A11, A10, A9; Subset 4: A6, A7, A8, A9. Curves are fitted to each of the four subsets, taking Subset 1 as an example.

[0114] The first subset of feature points includes four feature points: A0, A1, A2, and A3. These are divided into three consecutive pairs of feature points: [A0, A1], [A1, A2], and [A2, A3]. Each pair of feature points corresponds to a curve, the curve expression of which is:

[0115] S i (x)=a i +b i *x+c i *x 2 +d i *x 3

[0116] Where i = 1, ..., M, where i = 1, ..., 3, a i bi c i d i The preset coefficients for the curve expressions corresponding to [A0, A1], [A1, A2], and [A2, A3] are unknowns. Since there are 3 pairs of lip feature points, there are 12 unknown coefficients.

[0117] Since A1 is located in both interval 1 and interval 2, A2 is located in both interval 2 and interval 3, and A0 is located in interval 1 and A3 is located in interval 3, therefore:

[0118] S0(A0)=a0+b0*A0+c0*A0 2 +d0*A0 3

[0119] S0(A1)=a0+b0*A1+c0*A1 2 +d0*A1 3

[0120] S1(A1) = a1 + b1*A1 + c1*A1 2 +d1*A1 3

[0121] S1(A2) = a1 + b1*A2 + c1*A2 2 +d1*A2 3

[0122] S2(A2)=a2+b2*A2+c2*A2 2 +d2*A2 3

[0123] S2(A3) = a2 + b2*A3 + c2*A3 2 +d2*A3 3

[0124] Furthermore, the first derivatives are continuous at A1 and A2, therefore:

[0125] S′0(A1)=S′1(A1)

[0126] S′1(A2)=S′2(A2)

[0127] Furthermore, the second derivatives are continuous at A1 and A2, therefore:

[0128] S″0(A1)=S″1(A1)

[0129] S″1(A2)=S″2(A2)

[0130] Furthermore, the second derivative of the feature point at the endpoint of each feature point pair is 0, therefore:

[0131] Based on the above 12 equations, we can solve for a. i b i c i d i There are a total of 12 unknowns. This gives us the curve expression drawn by the first group of feature points A0, A1, A2, and A3. Similarly, we can obtain the remaining 3 groups of curves.

[0132] Fill the entire interior of the contour enclosed by the four sets of curves with a value of 255 pixels, and fill the entire exterior with a value of 0 pixels. Finally, only the lip area within the camera boundary is retained, denoted as mask1. Optionally, Figure 12 This is a schematic diagram of the circumscribed rectangle of the complete lip provided in the embodiments of this application, as shown below. Figure 12 As shown, the coordinate system is defined with the top left corner of the camera interface as the origin, the horizontal axis as the X-axis, and the vertical axis as the Y-axis. In this coordinate system, there are 12 lip feature points, A0, A1, ..., A11. By comparing the maximum and minimum X-coordinates of these 12 feature points, the leftmost and rightmost points of the lip can be obtained, denoted as Left and Right respectively. Figure 12 In the diagram, A0 and A6 are used. By comparing the maximum and minimum values ​​of the Y-coordinates of the 12 feature points, the highest and lowest points of the lip can be obtained, denoted as Top and Bottom, respectively. Top and Bottom correspond to... Figure 12 A4 and A9 in the text.

[0133] based on Figure 12 A rectangle can be drawn from the four extreme points: Left, Right, Top, and Bottom. Specifically, draw a vertical line through Left, a vertical line through Right, a horizontal line through Top, and a horizontal line through Bottom. The rectangle formed by these four lines is the minimum bounding rectangle of the complete lip. Label the X-coordinates of Left and Right as LeftX and RightX, and their Y-coordinates as LeftY and RightY. Label the X-coordinates of Top and Bottom as TOPX and BottomX, and their Y-coordinates as TOPY and BottomY. The width and height of the minimum bounding rectangle can then be calculated, denoted as Width and Height, and the center point of the bounding rectangle, denoted as CenterX, CenterY.

[0134] in,

[0135] Width = RightX - LeftX

[0136] Height = BottomY - TOPY

[0137] CenterX = α*(LeftX + RightX)

[0138] CenterY = α*(BottomY + TOPY)

[0139] α can be specifically determined based on the proportion of the lip in the first region to the whole lip. For example, when the lip in the first region is half of the whole lip, α = 0.5.

[0140] The minimum bounding rectangle of the complete lip is expanded outwards. Specifically, the four extreme points (top, bottom, left, and right) of the expanded rectangle are determined according to a fixed ratio, denoted as newLeft, newRight, newTop, and newBottom, with coordinates (newLeftX, newLeftY), (newRightX, newRightY), (newTOPX, newTOPY), and (newBottomX, newBottomY), respectively.

[0141] newLeftX = LeftX - α × Width

[0142] newLeftY = LeftY

[0143] newRightX = RightX + α × Width

[0144] newRightY = RightY

[0145] newTOPX = TOPX

[0146] newTOPY = TOPY - α × Height

[0147] newBottomX = BottomX

[0148] newBottomY=BottomY+α×Height

[0149] By obtaining the coordinates of the four extreme points (top, bottom, left, and right) of the expanded rectangle, the expanded image can be obtained, as shown below. Figure 13 As shown, Figure 13 This is a schematic diagram of the image corresponding to the smallest bounding rectangle of the complete lip provided in the embodiments of this application after being expanded.

[0150] After obtaining the expanded image, the lip region can be cropped out, which becomes the image to be processed. Since the lips are located at the camera boundary, the area outside the boundary will not be preserved; the image to be processed only contains the area within the camera boundary. See [link to relevant documentation]. Figure 14 , Figure 14 This is a schematic diagram of the image to be processed provided in the embodiments of this application.

[0151] For example, Figure 15 This is a schematic diagram of mask fusion provided in an embodiment of this application, as shown below. Figure 15 As shown, the top left image is a portion of the cropped mask1 area, and the bottom left image is the corresponding area of ​​mask3. The pixel value of the cropped mask1 in the first row and third column is 0, and the pixel value of the cropped mask3 in the first row and third column is 255. Therefore, 0 is the minimum value at this position, which means that the pixel value of the final mask at this position is 0.

[0152] After the above processing, mask3 has eliminated the outer edge of the lips. Therefore, after mask fusion, the outer edge of the lips in the final mask is formed by the curve fitted by the feature points, which can ensure the stability of the outer edge of the lips and prevent jitter. The inner edge of the lips is determined by the inner edge of the lips in the first image, which can ensure that the inner side of the lips accurately fits the lips in the open mouth position. Figure 16 This is a schematic diagram of the Final mask provided in the embodiments of this application, as shown below. Figure 16 As shown, the number of pixels in the shaded area of ​​the final mask is 0, and the number of pixels in the non-shaded area is 255.

[0153] The image processing method provided in this application embodiment performs curve fitting based on the feature points of N feature point subsets to obtain the edge fitting curve corresponding to each feature point subset. Based on the obtained edge fitting curve, a clear and stable second target edge can be obtained. The second target edge can be used to determine the optimization area of ​​the target object well, and the target object in the first image can be accurately optimized.

[0154] The image processing method provided in this application can be executed by an image processing device. This application uses an image processing device to perform the image processing method as an example to illustrate the image processing device provided in this application.

[0155] Figure 17 This is a schematic diagram of the structure of the image processing apparatus provided in the embodiments of this application, as shown below. Figure 17 As shown, the image processing apparatus may include:

[0156] The output module 1710 is used to input the image to be processed into a target convolutional neural network and output a first image when the first region of the target object in the image to be processed is incomplete, the first image including the first region of the target object and the second region of the non-target object;

[0157] The first acquisition module 1720 is used to traverse each column of pixels in the first image to obtain the target pixel in each column of pixels, wherein the target pixel is the pixel in the second region surrounded by the pixel region of the first region;

[0158] The first optimization module 1730 is used to determine the first target edge of the first image based on the target pixel, and to process the first image according to the first target edge.

[0159] Optionally, if the first region of the target object in the image to be processed is incomplete, the apparatus further includes:

[0160] The second acquisition module is used to acquire the feature point set of the complete region corresponding to the first region of the target object;

[0161] The third acquisition module is used to perform curve fitting based on N subsets of feature points in the feature point set, and to obtain the edge fitting curve of each subset of feature points. The N subsets of feature points are obtained by grouping the feature point set based on the target feature points in the feature point set. The target feature points include the leftmost feature point of the complete region, the rightmost feature point of the complete region, the bottom feature point of the complete region, and the valley feature point of the complete region in the feature point set. N is an integer greater than 1.

[0162] The second processing module is used to determine the second target edge of the first image based on N edge fitting curves, and to process the first image according to the second target edge.

[0163] Optionally, after determining the second target edge of the first image based on N edge fitting curves, and processing the first image according to the second target edge, the apparatus further includes:

[0164] The third processing module is used to process the first image based on the first target edge and the second target edge.

[0165] Optionally, the first acquisition module may include:

[0166] The first acquisition submodule is used to traverse each column of pixels in the first image and acquire the first changed pixel and the second changed pixel in each column of pixels. The first changed pixel is the pixel when the pixel corresponding to the first region changes to the pixel corresponding to the second region during the traversal of each column of pixels. The second changed pixel is the pixel when the pixel corresponding to the first region changes to the pixel corresponding to the second region during the traversal of each column of pixels, and then the pixel corresponding to the second region changes back to the pixel corresponding to the first region.

[0167] The second acquisition submodule is used to take the pixel points in each column of pixels that are between the first changed pixel point and the second changed pixel point as the target pixel points, so as to obtain the target pixel points in each column of pixels.

[0168] Optionally, the third acquisition module may include:

[0169] The third acquisition submodule is used to acquire M consecutive feature point pairs in each feature point subset;

[0170] The data processing submodule is used to construct a curve expression for each feature point pair based on each feature point pair;

[0171] The fourth acquisition submodule is used to solve each of the curve expressions and obtain the fitted curve for each of the feature point pairs;

[0172] The fifth acquisition submodule is used to acquire the edge fitting curve of each feature point subset based on each of the fitted curves.

[0173] The image processing apparatus provided in this application provides a solution when the first region of the target object in the image to be processed is incomplete. By inputting the image to be processed into a target convolutional neural network, the image is segmented into a first image that includes the first region of the target object and a second region that does not include the target object. By traversing each column of pixels in the first image, the pixels of the second region surrounded by the pixel region of the first region are found, and the first target edge in the first image is determined. This provides a clear first target edge in the first image and solves the problem that the first image cannot be accurately optimized because the edge of the target object cannot be identified when the first region of the target object is incomplete. Based on the first target edge, the optimization region of the target object can be well determined, and the target object in the first image can be accurately optimized.

[0174] The image processing device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0175] The image processing device in this application embodiment can be a device with an operating system. The operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system.

[0176] The image processing apparatus provided in this application embodiment can achieve... Figures 1 to 16 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0177] Optionally, such as Figure 18 As shown, this application embodiment also provides an electronic device 1800, including a processor 1801 and a memory 1802. The memory 1802 stores a program or instructions that can run on the processor 1801. When the program or instructions are executed by the processor 1801, they implement the various steps of the above-described image processing method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0178] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0179] Figure 19 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0180] The electronic device 1900 includes, but is not limited to, components such as: radio frequency unit 1901, network module 1902, audio output unit 1903, input unit 1904, sensor 1905, display unit 1906, user input unit 1907, interface unit 1908, memory 1909, and processor 1910.

[0181] Those skilled in the art will understand that the electronic device 1900 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1910 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 19 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0182] Among them, the network module 1902 is used to input the image to be processed into the target convolutional neural network when the first region of the target object in the image to be processed is incomplete, and output a first image, wherein the first image includes the first region of the target object and the second region of the non-target object;

[0183] The processor 1910 is used to traverse each column of pixels in the first image to obtain the target pixel in each column of pixels, wherein the target pixel is the pixel in the second region surrounded by the pixel region of the first region;

[0184] The processor 1910 is further configured to determine a first target edge of the first image based on the target pixel, and process the first image according to the first target edge.

[0185] Optionally, the network module 1902 is further configured to obtain the feature point set of the complete region corresponding to the first region of the target object;

[0186] The processor 1910 is further configured to perform curve fitting based on N subsets of feature points in the feature point set, and obtain an edge fitting curve for each subset of feature points, wherein the N subsets of feature points are obtained by grouping the feature point set based on target feature points in the feature point set, and the target feature points include the leftmost feature point of the complete region, the rightmost feature point of the complete region, the bottom feature point of the complete region, and the valley feature point of the complete region in the feature point set, and N is an integer greater than 1;

[0187] The processor 1910 is further configured to determine a second target edge of the first image based on N edge fitting curves, and process the first image according to the second target edge.

[0188] Optionally, the processor 1910 is further configured to process the first image based on the first target edge and the second target edge.

[0189] Optionally, the processor 1910 is further configured to traverse each column of pixels in the first image and obtain a first changed pixel and a second changed pixel in each column of pixels, wherein the first changed pixel is the pixel when the pixel corresponding to the first region changes to the pixel corresponding to the second region during the traversal of each column of pixels, and the second changed pixel is the pixel when the pixel corresponding to the first region changes to the pixel corresponding to the second region again during the traversal of each column of pixels.

[0190] The processor 1910 is further configured to take the pixel in each column of pixels that is between the first changed pixel and the second changed pixel as the target pixel, thereby obtaining the target pixel in each column of pixels.

[0191] Optionally, the processor 1910 is further configured to acquire M consecutive feature point pairs in each subset of said feature points;

[0192] The processor 1910 is also configured to construct a curve expression for each of the feature point pairs based on each of the feature point pairs;

[0193] The processor 1910 is also configured to solve each of the curve expressions to obtain a fitted curve for each of the feature point pairs;

[0194] The processor 1910 is also configured to obtain an edge fitting curve for each subset of the feature points based on each of the fitted curves.

[0195] The electronic device provided in this application embodiment, when the first region of the target object in the image to be processed is incomplete, inputs the image to be processed into a target convolutional neural network, segments the image into a first image including the first region of the target object and a second region of non-target object, and finds the pixels of the second region surrounded by the pixel region of the first region by traversing each column of pixels in the first image, thus determining the first target edge in the first image. This provides a clear first target edge in the first image, solving the problem that the edge of the target object cannot be identified when the first region of the target object is incomplete, which leads to the inaccurate optimization of the first image. Based on the first target edge, the optimization area of ​​the target object can be well determined, and the target object in the first image can be accurately optimized.

[0196] It should be understood that, in this embodiment, the input unit 1904 may include a graphics processing unit (GPU) 19041 and a microphone 19042. The GPU 19041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 1906 may include a display panel 19061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 1907 includes at least one of a touch panel 19071 and other input devices 19072. The touch panel 19071 is also called a touch screen. The touch panel 19071 may include a touch detection device and a touch controller. Other input devices 19072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0197] The memory 1909 can be used to store software programs and various data. The memory 1909 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1909 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1909 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0198] Processor 1910 may include one or more processing units; optionally, processor 1910 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 1910.

[0199] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image processing method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0200] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0201] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described image processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0202] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0203] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described image processing method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0204] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0205] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0206] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image processing method, characterized in that, include: If the first region of the target object in the image to be processed is incomplete, the image to be processed is input into the target convolutional neural network and the first image is output, the first image including the first region of the target object and the second region of the non-target object; Traverse each column of pixels in the first image to obtain the target pixel in each column of pixels, wherein the target pixel is the pixel in the second region surrounded by the pixel region of the first region; The first target edge of the first image is determined based on the target pixel, and the first image is processed according to the first target edge.

2. The image processing method according to claim 1, characterized in that, If the first region of the target object in the image to be processed is incomplete, the method further includes: Obtain the feature point set of the complete region corresponding to the first region of the target object; Curve fitting is performed on N subsets of feature points in the feature point set to obtain the edge fitting curve of each subset of feature points. The N subsets of feature points are obtained by grouping the feature point set based on the target feature points in the feature point set. The target feature points include the leftmost feature point of the complete region, the rightmost feature point of the complete region, the bottom feature point of the complete region, and the valley feature point of the complete region in the feature point set. N is an integer greater than 1. Based on N edge fitting curves, a second target edge of the first image is determined, and the first image is processed according to the second target edge.

3. The image processing method according to claim 2, characterized in that, The method, which determines the second target edge of the first image based on N edge fitting curves and processes the first image according to the second target edge, further includes: The first image is processed based on the first target edge and the second target edge.

4. The image processing method according to claim 1, characterized in that, The step of traversing each column of pixels in the first image to obtain the target pixel in each column includes: Traverse each column of pixels in the first image and obtain the first changed pixel and the second changed pixel in each column of pixels. The first changed pixel is the pixel when the pixel corresponding to the first region changes to the pixel corresponding to the second region during the traversal of each column of pixels. The second changed pixel is the pixel when the pixel corresponding to the first region changes to the pixel corresponding to the second region during the traversal of each column of pixels, and then the pixel corresponding to the second region changes back to the pixel corresponding to the first region. The target pixel is defined as the pixel in each column that lies between the first and second changing pixel.

5. The image processing method according to claim 2, characterized in that, The step of performing curve fitting based on N subsets of feature points in the feature point set to obtain the edge fitting curve for each subset of feature points includes: Obtain M consecutive feature point pairs from each of the feature point subsets; Based on each of the feature point pairs, construct a curve expression for each of the feature point pairs; Solve for each curve expression to obtain the fitted curve for each feature point pair; Based on each of the fitted curves, the edge fitting curves of each subset of feature points are obtained.

6. An image processing apparatus, characterized in that, include: The output module is used to input the image to be processed into a target convolutional neural network and output a first image when the first region of the target object in the image to be processed is incomplete, the first image including the first region of the target object and the second region of the non-target object; The first acquisition module is used to traverse each column of pixels in the first image to obtain the target pixel in each column of pixels, wherein the target pixel is the pixel in the second region surrounded by the pixel region of the first region; The first processing module is used to determine a first target edge of the first image based on the target pixel, and to process the first image according to the first target edge.

7. The image processing apparatus according to claim 6, characterized in that, If the first region of the target object in the image to be processed is incomplete, the device further includes: The second acquisition module is used to acquire the feature point set of the complete region corresponding to the first region of the target object; The third acquisition module is used to perform curve fitting based on N subsets of feature points in the feature point set, and to obtain the edge fitting curve of each subset of feature points. The N subsets of feature points are obtained by grouping the feature point set based on the target feature points in the feature point set. The target feature points include the leftmost feature point of the complete region, the rightmost feature point of the complete region, the bottom feature point of the complete region, and the valley feature point of the complete region in the feature point set. N is an integer greater than 1. The second processing module is used to determine the second target edge of the first image based on N edge fitting curves, and to process the first image according to the second target edge.

8. The image processing apparatus according to claim 7, characterized in that, The apparatus, after determining the second target edge of the first image based on N edge fitting curves and processing the first image according to the second target edge, further includes: The third processing module is used to process the first image based on the first target edge and the second target edge.

9. The image processing apparatus according to claim 6, characterized in that, The first acquisition module includes: The first acquisition submodule is used to traverse each column of pixels in the first image and acquire the first changed pixel and the second changed pixel in each column of pixels. The first changed pixel is the pixel when the pixel corresponding to the first region changes to the pixel corresponding to the second region during the traversal of each column of pixels. The second changed pixel is the pixel when the pixel corresponding to the first region changes to the pixel corresponding to the second region during the traversal of each column of pixels, and then the pixel corresponding to the second region changes back to the pixel corresponding to the first region. The second acquisition submodule is used to take the pixel points in each column of pixels that are between the first changed pixel point and the second changed pixel point as the target pixel points, so as to obtain the target pixel points in each column of pixels.

10. The image processing apparatus according to claim 7, characterized in that, The third acquisition module includes: The third acquisition submodule is used to acquire M consecutive feature point pairs in each feature point subset; The data processing submodule is used to construct a curve expression for each feature point pair based on each feature point pair; The fourth acquisition submodule is used to solve each of the curve expressions and obtain the fitted curve for each of the feature point pairs; The fifth acquisition submodule is used to acquire the edge fitting curve of each feature point subset based on each of the fitted curves.

Images