Image processing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an image processing method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a to-be-processed image, and determining an erasing region of the to-be-processed image; a replacement pixel value corresponding to a target pixel point in the erasure area is obtained, the replacement pixel value is determined based on a boundary pixel point corresponding to the target pixel point, and the boundary pixel point is a projection point of the target pixel value on the area boundary of the erasure area; and setting a corresponding target pixel point according to the replacement pixel value, and generating a complemented image. Pixel calculation is carried out on the target pixel points in the erasure area, the pixel values of the boundary pixel points on the edge of the erasure area are obtained to serve as the replacement pixel values to replace the pixel values of the target pixel points, rapid complementation of content in the erasure area is achieved, the content complementation speed is increased by utilizing the characteristic of high pixel calculation efficiency, and the content complementation efficiency is improved. The time consumption of the calculation process is reduced, thereby reducing the hardware performance requirement and improving the operation smoothness of the special effect function.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of Internet technology, and in particular, to an image processing method, device, electronic device, and storage medium. Background Art

[0002] Adding virtual items to images and videos uploaded by users is one of the common special effects functions in various applications with image and video editing functions. In this scenario, the existing technology usually removes the target object from the image to be processed by image erasing, and then completes the removed area based on the background image, thereby achieving the effect of erasing the target object from the image to be processed. After that, inserting special effect stickers on the erased image can further improve the visual effect of the special effect;

[0003] However, the image erasing solution in the prior art has the problems of being time-consuming to run and having high requirements on hardware performance, which affects the running smoothness of the special effect function. Summary of the invention

[0004] The embodiments of the present disclosure provide an image processing method, device, electronic device and storage medium to overcome the problems of time-consuming operation and high hardware performance requirements in the solution of erasing and completing images.

[0005] In a first aspect, an embodiment of the present disclosure provides an image processing method, including:

[0006] Acquire an image to be processed and determine an erasing area of ​​the image to be processed; acquire a replacement pixel value corresponding to a target pixel point in the erasing area, wherein the replacement pixel value is determined based on a boundary pixel point corresponding to the target pixel point, and the boundary pixel point is a projection point of the target pixel value on the area boundary of the erasing area; set a corresponding target pixel point according to the replacement pixel value to generate a completed image.

[0007] In a second aspect, an embodiment of the present disclosure provides an image processing device, including:

[0008] An acquisition module, used for acquiring an image to be processed and determining an erasing area of ​​the image to be processed;

[0009] A processing module, configured to obtain a replacement pixel value corresponding to a target pixel point in the erasing area, wherein the replacement pixel value is determined based on a boundary pixel point corresponding to the target pixel point, and the boundary pixel point is a projection point of the target pixel value on a region boundary of the erasing area;

[0010] A generation module is used to set corresponding target pixel points according to the replacement pixel value to generate a completed image.

[0011] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor and a memory;

[0012] The memory stores computer-executable instructions;

[0013] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the image processing method described in the first aspect and various possible designs of the first aspect.

[0014] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the image processing method described in the first aspect and various possible designs of the first aspect is implemented.

[0015] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the information display method described in the first aspect and various possible designs of the first aspect.

[0016] The image processing method, device, electronic device and storage medium provided in this embodiment obtain an image to be processed and determine the erasing area of ​​the image to be processed; obtain the replacement pixel value corresponding to the target pixel point in the erasing area, wherein the replacement pixel value is determined based on the boundary pixel point corresponding to the target pixel point, and the boundary pixel point is the projection point of the target pixel value on the area boundary of the erasing area; set the corresponding target pixel point according to the replacement pixel value to generate a completed image. By performing pixel calculation on the target pixel point in the erasing area, the pixel value of the boundary pixel point at the edge of the erasing area is obtained as the replacement pixel value to replace the pixel value of the target pixel point, thereby realizing the rapid completion of the content in the erasing area, and utilizing the high efficiency of pixel calculation to improve the speed of content completion and reduce the time consumption of the calculation process, thereby reducing the hardware performance requirements and improving the running smoothness of special effect functions. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0018] Figure 1 An application scenario diagram of the image processing method provided by the embodiment of the present disclosure;

[0019] Figure 2Schematic diagram of the image processing method provided in the embodiment of the present disclosure Figure 1 ;

[0020] Figure 3 A schematic diagram of a projection point provided by an embodiment of the present disclosure;

[0021] Figure 4 A schematic diagram of another projection point provided in an embodiment of the present disclosure;

[0022] Figure 5 Schematic diagram of the image processing method provided in the embodiment of the present disclosure Figure 2 ;

[0023] Figure 6 for Figure 5 A flowchart of a specific implementation method of step S203 in the illustrated embodiment;

[0024] Figure 7 A schematic diagram of the position of a row projection point provided in an embodiment of the present disclosure;

[0025] Figure 8 A schematic diagram of another position of a row projection point provided in an embodiment of the present disclosure;

[0026] Fig. 9 A schematic diagram of a column boundary pixel point provided by an embodiment of the present disclosure;

[0027] Fig.10 for Figure 5 A flowchart of a specific implementation method of step S204 in the illustrated embodiment;

[0028] Fig.11 A schematic diagram of a mirrored pixel row provided in an embodiment of the present disclosure;

[0029] Fig.12 A structural block diagram of an image processing device 3 provided in an embodiment of the present disclosure;

[0030] Fig.13 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure;

[0031] Fig.14 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0033] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0034] The application scenarios of the embodiments of the present disclosure are explained below:

[0035] Figure 1 This is an application scenario diagram of the image processing method provided by the embodiment of the present disclosure. The image processing method provided by the embodiment of the present disclosure can be applied to applications with video editing and image processing functions. More specifically, it can be applied to application scenarios in which special effects such as virtual items and stickers are added to images. The execution subject of this embodiment can be a terminal device running the above-mentioned application with video editing and image processing functions, a server running the service end corresponding to the above-mentioned application, or other electronic devices with similar functions. Reference Figure 1 As shown in, taking a terminal device as an example, in the process of inserting special effect stickers into the initial image by running an application, in order to avoid the problem that the target object in the initial image is inconsistent with the outline of the special effect sticker to be inserted and affects the visual experience, the terminal device needs to first erase the target object in the initial image, such as hair 11, to make the target object "disappear" from the initial image, and then insert the special effect sticker 13 at the position where the target object is located, that is, the erased area 13, to achieve the special effect of adding the special effect sticker 13 to the initial image.

[0036] In the prior art, the above-mentioned effect of erasing the target object from the image usually needs to be achieved using a pre-trained image processing model, that is, the target object is first removed from the image to be processed, and then the pre-trained image processing model is used to extract the image features of the background image, and the removed area is image-completed based on the image features of the retained area, thereby achieving the effect of erasing the target object from the image to be processed. However, the solutions in the prior art have the problem of large model calculation amount, which leads to a long image erasing process and high hardware performance requirements, and thus the smoothness of the operation of the special effect function. The disclosed embodiments provide an image processing method to solve the above problems.

[0037] refer to Figure 2 , Figure 2 Schematic diagram of the image processing method provided in the embodiment of the present disclosure Figure 1 The method of this embodiment can be applied in electronic devices such as terminal devices or servers. The image processing method includes:

[0038] Step S101: Acquire an image to be processed and determine an erasure area of ​​the image to be processed.

[0039] For example, reference Figure 1 As shown in the schematic diagram of the application scenario, the execution subject of the method provided in this embodiment is, for example, a terminal device. After the terminal device runs the corresponding application, it loads the image to be processed based on the user's instructions. More specifically, the image to be processed is the image to be added with special effects. The image to be processed can be stored locally on the terminal device, or it can be obtained from the outside by the terminal device through, for example, a network. The specific way and method for the terminal device to obtain the image to be processed are not limited here, and can be set as needed. Afterwards, the terminal device determines the erasure area of ​​the image to be processed. Among them, the erasure area is the area in the image to be processed that needs to be erased. More specifically, it is the image area where (in subsequent steps) the image content in the erasure area needs to be deleted and the image content outside the erasure area needs to be complemented.

[0040] In one possible implementation, the terminal device may determine the erasing area based on a user instruction or, for example, the user draws a designated area in the image to be processed through the interactive interface of the application program running on the terminal device as the erasing area of ​​the image to be processed. In another possible implementation, the terminal device determines the erasing area of ​​the image to be processed based on a pre-generated mask image, wherein the mask image may be a matrix of the same size as the image to be processed, each matrix element in the matrix corresponds to a pixel point of the image to be processed, and the element value of the matrix element may be 0 or 1, so that the erasing area of ​​the image to be processed is characterized by the element value of the matrix element. The specific implementation of the mask image is prior art and will not be described here.

[0041] Optionally, after determining the erasure area of ​​the image to be processed, the method further includes:

[0042] Step S1010: Add Gaussian blur to the image to be processed.

[0043] By adding Gaussian blur to the image to be processed, the difference between pixels in the image to be processed is reduced, and then when the erased area is subsequently completed based on the content of the unerased area, the jump between pixels is reduced and the image visual effect is improved.

[0044] Furthermore, before adding Gaussian blur, you can also include:

[0045] Obtain device performance information of the terminal device; determine Gaussian blur parameters based on the device performance information, where the Gaussian blur parameters are used to characterize the number of times the Gaussian blur is performed and / or the accuracy of the Gaussian blur.

[0046] Exemplarily, before adding Gaussian blur, the device in first determines the parameters of the algorithm for running Gaussian blur, i.e., Gaussian blur parameters, according to the device performance information. The device performance information includes, for example, the hardware identification code, memory size, hardware computing power score, etc. of the terminal device. The device performance information can be used to show the performance of the terminal device in processing data, and then based on the device performance information, a Gaussian blur parameter that matches it can be determined, so that terminal devices with poor performance use lower-quality Gaussian blur operations to reduce jams; while terminal devices with better performance use higher-quality Gaussian blur operations to improve the visual effect and fineness of Gaussian blur.

[0047] Correspondingly, after determining the Gaussian blur parameters based on the device performance information, the process of adding Gaussian blur in the above steps is specifically implemented in the following manner: adding Gaussian blur based on the Gaussian blur parameters.

[0048] Furthermore, in a possible implementation, a specific implementation of step S101 includes:

[0049] Step S1011: Identify the contour of the target object in the image to be processed.

[0050] Step S1012: Generate a mask image according to the contour of the target object, where the mask image is used to represent the erased area in the processed image.

[0051] Exemplarily, after obtaining the image to be processed, the target object in the image to be processed is identified, and the outline of the target object can be segmented. The target object is, for example, a specific target such as a person or a vehicle in the image, or a part of a specific target. More specifically, the target object is determined based on the special effect type of the triggered target special effect. For example, if the target special effect is a special effect of adding a "wig" to a portrait, then the target object identified in the corresponding image to be processed is the "hair" of the person. Through the solution of this embodiment, the target object in the image to be processed can be automatically determined, and the corresponding mask map can be generated to achieve the positioning of the target object, that is, the positioning of the erasing area.

[0052] Optionally, after step S201, the following steps may also be included:

[0053] Step S2011: Acquire special effect information, where the special effect information represents the contour shape of the target special effect added to the image to be processed.

[0054] Step S2012: Based on the special effect information, the mask image is expanded to update the erased area in the image to be processed.

[0055] Exemplarily, after obtaining the mask image, since the mask image is generated based on the contour of the target object, in the actual processing process, when the contour of the target object is relatively complex, there may be a situation where the erased area corresponding to the mask image cannot completely cover the target object (that is, the contour recognition of the target object is inaccurate), which affects the visual effect of the subsequently generated completed image. In this embodiment, by obtaining special effect information representing the contour shape of the target special effect added to the image to be processed, such as a special effect contour mask image of the target special effect, and then positioning and scaling the special effect contour mask image based on the position and size of the target object in the image to be processed, the mask image generated in the previous step is expanded according to the special effect contour mask image so that the mask image can cover the special effect contour mask image, and then based on the expanded mask image, the erased area is re-determined, that is, the erased area in the image to be processed is updated, and the visual effect of the subsequently generated completed image is improved by increasing the range of the erased area.

[0056] Step S102: Obtain a replacement pixel value corresponding to a target pixel point in the erasing area, wherein the replacement pixel value is determined based on a boundary pixel point corresponding to the target pixel point, and the boundary pixel point is a projection point of the target pixel value on the area boundary of the erasing area.

[0057] Exemplarily, after determining the erased area in the image to be processed, the image content in the erased area is complemented from the pixel dimension. Specifically, based on the mask image, the erased area in the image to be processed is determined, and the pixel points in the image to be processed that fall within the erased area are the target pixel points. Then, based on the position of the target pixel point, the boundary pixel point corresponding to it is selected, and according to the pixel value of the boundary pixel point corresponding to it, the replacement pixel value corresponding to the target pixel point is generated to achieve pixel replacement of the pixel value of the target pixel point. Among them, the boundary pixel value refers to the projection point of the target pixel point on the boundary of the area where it is located. Figure 3 A schematic diagram of a projection point provided by an embodiment of the present disclosure, such as Figure 3 As shown, exemplarily, there is a target pixel point P (shown as P in the figure) in the erased area of ​​the image to be processed, and the target pixel point P is projected onto the regional boundary L of the erased area. According to the projection direction, different projection points located on the regional boundary L can be obtained, such as P_1, P_2, P_3, etc. shown in the figure. Afterwards, one or more of the projection points P_1, P_2, and P_3 can be determined as boundary pixel points. For example, according to the coordinates of the target pixel point and each projection point, the projection point P_1 closest to the target pixel point P is determined as the boundary pixel point. Afterwards, according to the pixel value of the boundary pixel point, the replacement pixel value of the target pixel point P is obtained.

[0058] Figure 4 A schematic diagram of another projection point provided in an embodiment of the present disclosure, such as Figure 4 As shown, in another possible implementation, the target pixel point P is distributed and projected in four directions, namely, upward, downward, left, and right, so as to obtain four projection points P_1, P_2, P_3, and P_4 of the target pixel point P region boundary L. Afterwards, the boundary pixel points in the horizontal direction (pixel row) and the vertical direction (pixel column) are determined. Specifically, for example, the projection point closer to the target pixel point P among P_1 and P_3, such as P_1, is determined as the row boundary pixel point; similarly, the projection point closer to the target pixel point P among P_2 and P_4, such as P_2, is determined as the column boundary pixel point. Afterwards, a weighted average is performed based on the pixel value of the row boundary pixel point P_1 and the pixel value of the column boundary pixel point P_2 to obtain the replacement pixel value of the target pixel point P.

[0059] Step S103: setting corresponding target pixel points according to the replacement pixel values ​​to generate a completed image.

[0060] Exemplarily, after obtaining the replacement pixel value corresponding to the target pixel, the current pixel value of the corresponding target pixel is replaced based on the replacement pixel value to complete the image content in the erased area. The image generated after the completion of the above completion process is the completed image. In a possible implementation, after determining the erased area, the transparency of the target pixel in the erased area is set to 0, and then after replacing the pixel value of the target pixel (setting it to the replacement pixel value), its transparency is set to 1 to mark the target pixel. Afterwards, for the target pixel with a transparency of 0 that does not match the replacement pixel value, other methods can be used to complete it, such as by obtaining the pixel value of the pixel in the same pixel column of the target pixel, setting the pixel value of the target pixel, and completing the pixel of the target pixel that does not match the replacement pixel value.

[0061] Optionally, after step S103, the following steps may also be included:

[0062] Step S104: After the completed image is generated, Gaussian blur is added to the erased area in the completed image.

[0063] For example, after obtaining the completed image, Gaussian blur can be further added to the erased area in the completed image, thereby improving the smooth transition between the pixels of the image content in the erased area, reducing the sudden pixel points and pixel blocks, and improving the visual effect of the completed image. The specific implementation method of adding Gaussian blur to the erased area in the completed image is similar to the specific implementation method of adding Gaussian blur in step S1010, which will not be repeated here, and reference can be made to the detailed introduction in the above embodiment steps.

[0064] In this embodiment, by obtaining an image to be processed and determining the erased area of ​​the image to be processed; obtaining the replacement pixel value corresponding to the target pixel point in the erased area, wherein the replacement pixel value is determined based on the boundary pixel point corresponding to the target pixel point, and the boundary pixel point is the projection point of the target pixel value on the area boundary of the erased area; setting the corresponding target pixel point according to the replacement pixel value, and generating a completed image. By performing pixel calculation on the target pixel point in the erased area, the pixel value of the boundary pixel point at the edge of the erased area is obtained as the replacement pixel value to replace the pixel value of the target pixel point, so as to realize the rapid completion of the content in the erased area, and utilize the high efficiency of pixel calculation to improve the speed of content completion and reduce the time consumption of the calculation process, thereby reducing the hardware performance requirements and improving the running smoothness of the special effect function.

[0065] refer to Figure 5 , Figure 5 Schematic diagram of the image processing method provided in the embodiment of the present disclosure Figure 2 In this embodiment Figure 2On the basis of the illustrated embodiment, step S102 is further refined, and the image processing method includes:

[0066] Step S201: Acquire an image to be processed and determine an erasure area of ​​the image to be processed.

[0067] Step S202: obtaining the number of row projection points corresponding to target pixel points in the current pixel row of the image to be processed, where the row projection points are located at the intersection of the current pixel row and the region boundary.

[0068] Exemplarily, after determining the erased area of ​​the image to be processed, the pixel rows of the image to be processed are used as units, and the pixel points in the erased area are set pixel by pixel row to achieve image completion for the erased area. Specifically, in a possible implementation, the first pixel row of the image to be processed (i.e., the top pixel row of the image to be processed) is used as the current pixel row for processing. First, based on the erased area marked by the mask image, the target pixel points in the current pixel row are obtained. For example, the current pixel row includes pixel points P[0] to pixel point P

[1023] , a total of 1024 pixel points. Based on the mask image, it is determined that the pixel points P

[24] to pixel point P

[236] are the target pixel points located in the erased area. It can be understood that, based on the shape of the erased area, in other possible situations, the target pixel points may include multiple discontinuous pixel points, for example, in the current pixel row, pixel points P

[24] to pixel point P

[236] , P

[255] to pixel point P

[1006] are the target pixel points. Afterwards, the intersection of the current pixel row and the region boundary, i.e., the row projection point, is detected to obtain the number of row projection points. For each target pixel in the current pixel row, the corresponding row projection point and the number of projection points are the same, so the row projection point corresponding to each target pixel can be quickly determined based on one calculation, thereby improving the calculation efficiency.

[0069] Step S203: if the number of the row projection point is at least one, then obtaining the row boundary pixel point corresponding to the target pixel point in the current pixel row according to the distance between the row projection point and the corresponding target pixel point.

[0070] Exemplarily, after obtaining the number of corresponding row projection points for the current pixel row, different processing is performed based on the number of row projection points. If the number of row projection points is greater than or equal to one, that is, it includes at least one row projection point, it means that in the current pixel row, there is at least one corresponding non-erased area. Therefore, the pixel points in the non-erased area can be used to perform horizontal pixel filling to complete the image completion of the current pixel row. This process requires calculations for each target pixel point one by one based on the positional relationship between the target pixel point and the row projection point, so as to obtain the row boundary pixel point corresponding to each target pixel point. Exemplarily, as Figure 6As shown, the specific implementation of step S203 includes:

[0071] Step S2031, obtaining the position of the row projection point.

[0072] Step S2032, when the row projection points include at least one first row projection point and at least one second row projection point, and the first row projection points and the second row projection points are located on both sides of the target pixel point respectively, the first row projection point closest to the target pixel point is determined as the first row boundary pixel point;

[0073] Step S2033: determine the second row projection point closest to the target pixel point as the second row boundary pixel point.

[0074] Figure 7 A schematic diagram of the position of a row projection point provided in an embodiment of the present disclosure, such as Figure 7 As shown, exemplarily, after projection point detection is performed on the current pixel row, the position of the row projection point is obtained. When the row projection point includes at least one first row projection point and at least one second row projection point, the first row projection point and the second row projection point are respectively located on both sides of the target pixel point. For example, the P1 pixel point shown in the figure is the first row projection point; the P2 pixel point, the P3 pixel point, and the P4 pixel point are the second row projection points (shown as P1, P2, P3, and P4, respectively in the figure), and the P pixel point is the target pixel point (shown as P in the figure). Based on the distance between each row projection point and the target projection point, P1 is determined as the first row boundary pixel point and P2 is determined as the second row boundary pixel point. In subsequent steps, the replacement pixel value can be jointly determined based on the pixel values ​​of the first row boundary pixel point and the second row boundary pixel point.

[0075] Step S2034: when the row projection points include at least one first row projection point located on the same side of the target pixel point, the first row projection point closest to the target pixel point is determined as the first row boundary pixel point.

[0076] Step S2035: Determine the pixel points that are in the current pixel row and outside the erasing area as the boundary pixel points of the second row.

[0077] For example, in another possible implementation, Figure 8 Another schematic diagram of the position of the row projection point provided by the embodiment of the present disclosure is as follows: Figure 8As shown, when the row projection points include at least one first row projection point located on the same side of the target pixel point, for example, the P1 pixel point, the P2 pixel point, and the P3 pixel point shown in the figure are the first row projection points (shown as P1, P2, and P3 in the figure), and the P pixel point (shown as P in the figure) is the target pixel point, then based on the distance between each first row projection point and the target projection point, P1 is determined as the first row boundary pixel point. At the same time, since there is no boundary pixel point on the right side of the target pixel point, in this embodiment, the pixel point located outside the erasing area, such as the P4 pixel point shown in the figure (shown as P4 in the figure), is determined as the second row boundary pixel point. Among them, the P3 pixel point is the first pixel point in the non-erased area on the opposite side of the missing row boundary pixel point (the right side of the target pixel point) in the current pixel row, such as the first pixel point on the leftmost side of the current pixel row.

[0078] In a possible implementation, the specific implementation of step S2035 includes: determining the first row projection point farthest from the target pixel point or the end pixel point of the current pixel row on one side of the first row projection point as the second row boundary pixel point. Figure 8 As shown, the second row of boundary pixels can be Figure 8 In addition to point P3 in the image (the end pixel of the current pixel row on the side of the projection point of the first row), you can also Figure 8 The P4 point shown in (the first row projection point farthest from the target pixel).

[0079] In the steps of this embodiment, when there are no second-row boundary pixels in the current pixel row, the second-row boundary pixels are virtualized by using the pixels on the opposite side far away from the image boundary. This can prevent the image in the erased area from excessive lateral stretching and create a sense of symmetry, thereby improving the visual effect after image completion when a large area of ​​the image is erased.

[0080] Step S204: Obtain replacement pixel values ​​corresponding to target pixel points in the current pixel row according to the pixel values ​​of the row boundary pixel points.

[0081] Exemplarily, the row boundary pixel points include the first row boundary pixel points and the second row boundary pixel points, and the specific implementation of step S204 includes:

[0082] Step S2041: Obtain a first weighting coefficient corresponding to the first row boundary pixel point and a second weighting coefficient corresponding to the second row boundary pixel point according to the distances between the first row boundary pixel point and the second row boundary pixel point and the corresponding target pixel point respectively.

[0083] Step S2042: Calculate the weighted sum of the pixel values ​​of the first row of boundary pixels and the pixel values ​​of the second row of boundary pixels according to the first weighting coefficient and the second weighting coefficient to obtain a replacement pixel value corresponding to the target pixel.

[0084] Exemplarily, after obtaining the row boundary pixel corresponding to the target pixel through the above calculation steps, when there is only one row boundary pixel corresponding to the target pixel, in a possible implementation, the pixel value of the row boundary pixel can be directly used as the replacement pixel value corresponding to the target pixel. In another possible case, when the row boundary pixel of the target pixel includes the first row boundary pixel and the second row boundary pixel located on both sides of the target pixel, in a possible implementation, the row boundary pixel closer to the target pixel can be determined as the target row boundary pixel, and the replacement pixel value is obtained based on the target row boundary pixel. In another possible implementation, a first weighting coefficient corresponding to the first row boundary pixel point and a second weighting coefficient corresponding to the second row boundary pixel point are obtained according to the distances between the first row boundary pixel point and the second row boundary pixel point and the corresponding target pixel point, for example, the distance between the first row boundary pixel point and the target pixel point is m, and the distance between the second row boundary pixel point and the target pixel point is 4m, where m is a preset unit (i.e., the distance between the second row boundary pixel point and the target pixel point is 4 times the distance between the first row boundary pixel point and the target pixel point); accordingly, based on the distances between the first row boundary pixel point and the second row boundary pixel point and the target pixel point, after normalization, the first weighting coefficient corresponding to the first row boundary pixel point is obtained to be 0.8; the second weighting coefficient corresponding to the second row boundary pixel point is obtained to be 0.2 (i.e., the weighting coefficient is inversely proportional to the distance); thereafter, the weighted sum of the pixel value of the first row boundary pixel point and the pixel value of the second row boundary pixel point is calculated according to the first weighting coefficient and the second weighting coefficient, and the replacement pixel value is obtained to be:

[0085] sum_value=0.8*value_1+0.2*value_2

[0086] Among them, value_1 is the pixel value of the first row of boundary pixels;

[0087] value_2 is the pixel value of the second row boundary pixel;

[0088] sum_value is the replacement pixel value.

[0089] In this embodiment, when replacing the pixel value of the target pixel value for the current pixel row, a weighted summation of the pixel values ​​is performed based on the distance between the target pixel value and the row boundary pixels on both sides, so that the size of the obtained replacement pixel value is determined based on the distance between the row boundary pixels and the target pixel point, so that the replacement pixel value of the target pixel point can be closer to the pixel value of the image around its position, thereby improving the coherence between the completed image in the erased area and the image outside the erased area, and improving the visual perception effect.

[0090] In another possible implementation, the specific implementation of step S204 includes:

[0091] Step S2043: Obtain the column boundary pixel point corresponding to the target pixel point in the current pixel row.

[0092] Step S2044: Obtain a replacement pixel value corresponding to the target pixel point based on a weighted sum of the pixel values ​​of the column boundary pixel points and the pixel values ​​of the row boundary pixel points.

[0093] Further, exemplarily, based on obtaining the column boundary pixel points, the corresponding column boundary pixels can be obtained for the target pixel points in the current pixel row, wherein the column boundary pixel points are obtained by projecting the target pixel points toward the direction of the pixel column in which they are located, and obtaining the intersection of the current pixel column and the region boundary, i.e., the column projection point. The column boundary pixel points are then obtained through the column projection point. The method for obtaining the column boundary pixel points is similar to the method for obtaining the row boundary pixel points. For the specific implementation method, reference may be made to the introduction to the method for obtaining the row boundary pixel points in the previous embodiment. Fig. 9 A schematic diagram of a column boundary pixel point provided by an embodiment of the present disclosure, such as Fig. 9 As shown, within the current pixel row, for the target pixel point P, it is projected upward and downward respectively to obtain the intersection of the current pixel column where P is located and the region boundary, that is, the column projection points P1 and P2. Then, based on the distance between P1, P2 and the target pixel point, P1 is determined as the column boundary pixel point. Then, the pixel value of the row boundary pixel point obtained based on the previous step and the pixel value of the column boundary pixel point are weighted and summed. The weighting coefficient can be determined based on the distance, for example, to obtain the replacement pixel value corresponding to the target pixel point.

[0094] It should be noted that the two implementations of step S204 corresponding to the above steps S2041 to S2044 can be combined to obtain another implementation of step S204, in addition to the separate implementations described in the above embodiments. Fig.10 As shown, illustratively, the specific implementation of step S204 includes:

[0095] Step S2041: Obtain a first weighting coefficient corresponding to the first row boundary pixel point and a second weighting coefficient corresponding to the second row boundary pixel point according to the distances between the first row boundary pixel point and the second row boundary pixel point and the corresponding target pixel point respectively.

[0096] Step S2045: Obtain the first column boundary pixel point and the second column boundary pixel point corresponding to the target pixel point in the current pixel row, and obtain the third weighting coefficient corresponding to the first column boundary pixel point and the fourth weighting coefficient corresponding to the second column boundary pixel point based on the distances between the first column boundary pixel point and the second column boundary pixel point and the corresponding target pixel point respectively.

[0097] Step S2046: According to the first weighting coefficient, the second weighting coefficient, the third weighting coefficient and the fourth weighting coefficient, calculate the weighted sum of the pixel values ​​of the first row boundary pixels, the pixel values ​​of the second row boundary pixels, the pixel values ​​of the first column boundary pixels and the pixel values ​​of the second column boundary pixels to obtain the replacement pixel value corresponding to the target pixel.

[0098] Exemplarily, in the steps of this embodiment, weighted calculations are performed based on the distances between the first row boundary pixels, the second row boundary pixels, the first column boundary pixels, and the second column boundary pixels and the target pixel, respectively. This can fully utilize the image content on the boundary of the area surrounding the target pixel value, so that the replacement pixel value corresponding to the target pixel point can achieve a better transition change, thereby improving the visual effect of the image completed in the erased area.

[0099] Step S205: If the number of row projection points is less than one, determine a mirror pixel row of the current pixel row, where the position of the mirror pixel row in the image to be processed is a mirror image of the position of the current pixel row in the image to be processed.

[0100] Step S206: obtaining replacement pixel values ​​corresponding to target pixels in the current pixel row according to the pixel values ​​of the pixels in the mirrored pixel row.

[0101] Exemplarily, in another possible case, if the number of row projection points is less than one, it means that the current pixel row is entirely located within the erased area, and it is not possible to use the image information outside the erased area within the current pixel row to obtain the corresponding replacement pixel value, nor is it possible to complete the image of the target pixel point within the current pixel row. In this case, the content of the current pixel row can be completed by a pixel row that has been completed, or a pixel row that does not contain an erased area. Specifically, for example, a mirror pixel row of the current pixel row is determined, and the position of the mirror pixel row in the image to be processed is mirrored to the position of the current pixel row in the image to be processed. After that, based on the pixel value of each pixel point in the mirror pixel row, the replacement pixel value corresponding to each pixel point is determined as a whole. Fig.11 A schematic diagram of a mirrored pixel row provided in an embodiment of the present disclosure, such as Fig.11As shown, the total number of pixel rows of the image to be processed is 128 (rows 1 to 128). During the processing of the image to be processed, when the current pixel row is pixel row N (N is a positive integer less than or equal to 128), it is detected that the number of row projection points is less than one, that is, all the pixels of the current pixel row are located in the erased area. At this time, the terminal device obtains pixel row 129-N of the image to be processed. Optionally, the pixel row 129-N is then detected. If the transparency of the pixels of the pixel row 129-N is all 1 (that is, the pixel row has been image-completed or the pixel row does not include the erased area), the pixel values ​​of each pixel of the pixel row 129-N are used to determine the replacement pixel values ​​of each pixel in the pixel row N respectively; if the transparency of the pixels of the pixel row 129-N is not all 1 (that is, the image completion is not completed), the next row of the pixel row is obtained until a pixel row with all pixels having a transparency of 1 is obtained, and the above steps are completed.

[0102] In this embodiment, when the number of row projection points of the current pixel row is less than one, that is, all of them are in the erased area, the corresponding replacement pixel value is obtained as a whole by acquiring its corresponding mirror pixel row, thereby further improving the image completion efficiency in the erased area and reducing the time consumption of the image completion process.

[0103] Step S207: If the current pixel row is the last pixel row, execute step S208; otherwise, set the current pixel row as the next pixel row and return to step S202.

[0104] Step S208: setting corresponding target pixel points according to the replacement pixel values ​​to generate a completed image.

[0105] In this embodiment, the implementation method of steps S201 and S208 is the same as that of the present disclosure. Figure 2 The implementation methods of step S101 and step S103 in the illustrated embodiment are the same and will not be described in detail here.

[0106] Corresponding to the image processing method of the above embodiment, Fig.12 This is a structural block diagram of an image processing device 3 provided in an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown.

[0107] Reference Fig.12 , the image processing device 3 comprises:

[0108] An acquisition module 31 is used to acquire an image to be processed and determine an erasure area of ​​the image to be processed;

[0109] A processing module 32 is used to obtain a replacement pixel value corresponding to a target pixel point in the erasing area, wherein the replacement pixel value is determined based on a boundary pixel point corresponding to the target pixel point, and the boundary pixel point is a projection point of the target pixel value on the area boundary of the erasing area;

[0110] The generating module 33 is used to set the corresponding target pixel point according to the replacement pixel value to generate a completed image.

[0111] In one embodiment of the present disclosure, the processing module 32 is specifically used to: for each pixel row of the image to be processed, perform the following steps in sequence: obtain the number of row projection points corresponding to the target pixel point in the current pixel row, the row projection point is located at the intersection of the current pixel row and the region boundary; if the number of row projection points is at least one, then according to the distance between the row projection point and the corresponding target pixel point, obtain the row boundary pixel point corresponding to the target pixel point in the current pixel row; according to the pixel value of the row boundary pixel point, obtain the replacement pixel value corresponding to each target pixel point in the current pixel row.

[0112] In one embodiment of the present disclosure, when the row projection points include at least one first row projection point and / or at least one second row projection point, and the first row projection points and the second row projection points are respectively located on both sides of the target pixel point, the processing module 32, when obtaining the row boundary pixel point corresponding to the target pixel point in the current pixel row according to the distance between the row projection point and the corresponding target pixel point, is specifically used to: determine the first row projection point closest to the target pixel point as the first row boundary pixel point; and / or determine the second row projection point closest to the target pixel point as the second row boundary pixel point.

[0113] In one embodiment of the present disclosure, when the row projection points include at least one first row projection point located on the same side of the target pixel point, the processing module 32, when obtaining the row boundary pixel point corresponding to the target pixel point in the current pixel row based on the distance between the row projection point and the corresponding target pixel point, is specifically used to: determine the first row projection point closest to the target pixel point as the first row boundary pixel point; and determine the pixel point located in the current pixel row and outside the erased area as the second row boundary pixel point.

[0114] In one embodiment of the present disclosure, when the processing module 32 determines a pixel point located in the current pixel row and outside the erased area as a second-row boundary pixel point, it is specifically used to: determine the first-row projection point farthest from the target pixel point or the end pixel point of the current pixel row on the side of the first-row projection point as the second-row boundary pixel point.

[0115] In one embodiment of the present disclosure, row boundary pixel points include first row boundary pixel points and second row boundary pixel points; when the processing module 32 obtains the replacement pixel value corresponding to each target pixel point in the current pixel row based on the pixel value of the row boundary pixel point, it is specifically used to: obtain a first weighting coefficient corresponding to the first row boundary pixel point and a second weighting coefficient corresponding to the second row boundary pixel point based on the distances between the first row boundary pixel point and the second row boundary pixel point and the corresponding target pixel point respectively; calculate the weighted sum of the pixel value of the first row boundary pixel point and the pixel value of the second row boundary pixel point based on the first weighting coefficient and the second weighting coefficient to obtain the replacement pixel value corresponding to the target pixel point.

[0116] In one embodiment of the present disclosure, the processing module 32 is further used to: obtain the column boundary pixel points corresponding to the target pixel points in the current pixel row; when the processing module 32 obtains the replacement pixel values ​​corresponding to each target pixel point in the current pixel row according to the pixel values ​​of the row boundary pixel points, it is specifically used to: obtain the replacement pixel value corresponding to the target pixel point based on the weighted sum of the pixel values ​​of the column boundary pixel points and the pixel values ​​of the row boundary pixel points.

[0117] In one embodiment of the present disclosure, after obtaining the number of row projection points corresponding to the target pixel point in the current pixel row, the processing module 32 is also used to: if the number of row projection points is less than one, determine a mirror pixel row of the current pixel row, and the position of the mirror pixel row in the image to be processed is mirrored to the position of the current pixel row in the image to be processed; and obtain the replacement pixel value corresponding to each target pixel point in the current pixel row according to the pixel value of each pixel point in the mirror pixel row.

[0118] In one embodiment of the present disclosure, the acquisition module 31 is specifically used to: identify the contour of the target object in the image to be processed; and generate a mask image according to the contour of the target object, wherein the mask image is used to represent the erased area in the processed image.

[0119] In one embodiment of the present disclosure, before determining the erased area of ​​the image to be processed based on the mask image, the acquisition module 31 is also used to: obtain special effect information, the special effect information represents the contour shape of the target special effect added to the image to be processed; based on the special effect information, expand the mask image to update the erased area in the image to be processed.

[0120] In one embodiment of the present disclosure, the processing module is also used for at least one of the following: after determining the erased area of ​​the image to be processed, adding Gaussian blur to the image to be processed; after generating the completed image, adding Gaussian blur to the erased area in the completed image.

[0121] The acquisition module 31, the processing module 32 and the generation module 33 are connected in sequence. The image processing device 3 provided in this embodiment can execute the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, which will not be described in detail in this embodiment.

[0122] Fig.13 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure is shown in FIG. Fig.13 As shown, the electronic device 4 includes:

[0123] A processor 41, and a memory 42 communicatively connected to the processor 41;

[0124] The memory 42 stores computer executable instructions;

[0125] The processor 41 executes the computer execution instructions stored in the memory 42 to implement the following Figure 2-Figure 11 The image processing method in the illustrated embodiment.

[0126] Optionally, the processor 41 and the memory 42 are connected via a bus 43 .

[0127] For related instructions, please refer to Figure 2-Figure 11 The relevant descriptions and effects corresponding to the steps in the corresponding embodiments can be understood, and no further elaboration is made here.

[0128] The present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, which are used to implement the present invention when executed by a processor. Figure 2-Figure 11 The image processing method provided by any one of the corresponding embodiments.

[0129] In order to implement the above embodiment, the embodiment of the present disclosure also provides an electronic device.

[0130] refer to Fig.14 , which shows a schematic diagram of the structure of an electronic device 900 suitable for implementing the embodiment of the present disclosure, and the electronic device 900 may be a terminal device or a server. The terminal device may include but is not limited to mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Fig.14 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0131] like Fig.14 As shown, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 to a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0132] Typically, the following devices may be connected to the I / O interface 905: input devices 906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 908 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although Fig.14 The electronic device 900 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0133] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0134] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0135] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0136] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0137] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0138] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0139] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0140] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0141] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0142] In a first aspect, according to one or more embodiments of the present disclosure, there is provided an image processing method, comprising:

[0143] Acquire an image to be processed and determine an erasing area of ​​the image to be processed; acquire a replacement pixel value corresponding to a target pixel point in the erasing area, wherein the replacement pixel value is determined based on a boundary pixel point corresponding to the target pixel point, and the boundary pixel point is a projection point of the target pixel value on the area boundary of the erasing area; set a corresponding target pixel point according to the replacement pixel value to generate a completed image.

[0144] According to one or more embodiments of the present disclosure, the method of obtaining the replacement pixel value corresponding to the target pixel point in the erased area includes: for each pixel row of the image to be processed, sequentially performing the following steps: obtaining the number of row projection points corresponding to the target pixel point in the current pixel row, the row projection point being located at the intersection of the current pixel row and the boundary of the area; if the number of the row projection points is at least one, obtaining the row boundary pixel point corresponding to the target pixel point in the current pixel row based on the distance between the row projection point and the corresponding target pixel point; and obtaining the replacement pixel value corresponding to each target pixel point in the current pixel row based on the pixel value of the row boundary pixel point.

[0145] According to one or more embodiments of the present disclosure, when the row projection points include at least one first row projection point and / or at least one second row projection point, and the first row projection point and the second row projection point are respectively located on both sides of the target pixel point, obtaining the row boundary pixel point corresponding to the target pixel point in the current pixel row based on the distance between the row projection point and the corresponding target pixel point includes: determining the first row projection point closest to the target pixel point as the first row boundary pixel point; and / or determining the second row projection point closest to the target pixel point as the second row boundary pixel point.

[0146] According to one or more embodiments of the present disclosure, when the row projection points include at least one first row projection point located on the same side of the target pixel point, obtaining the row boundary pixel point corresponding to the target pixel point in the current pixel row based on the distance between the row projection point and the corresponding target pixel point includes: determining the first row projection point closest to the target pixel point as the first row boundary pixel point; and determining the pixel point located in the current pixel row and outside the erasing area as the second row boundary pixel point.

[0147] According to one or more embodiments of the present disclosure, the pixel points located within the current pixel row and outside the erasing area are determined as second-row boundary pixel points, including: determining the first-row projection point farthest from the target pixel point or the end pixel point of the current pixel row on the side of the first-row projection point as second-row boundary pixel points.

[0148] According to one or more embodiments of the present disclosure, the row boundary pixel points include a first row boundary pixel point and a second row boundary pixel point; obtaining the replacement pixel value corresponding to each target pixel point in the current pixel row based on the pixel value of the row boundary pixel point includes: obtaining a first weighting coefficient corresponding to the first row boundary pixel point and a second weighting coefficient corresponding to the second row boundary pixel point based on the distances between the first row boundary pixel point and the second row boundary pixel point and the corresponding target pixel point respectively; and calculating the weighted sum of the pixel value of the first row boundary pixel point and the pixel value of the second row boundary pixel point based on the first weighting coefficient and the second weighting coefficient to obtain the replacement pixel value corresponding to the target pixel point.

[0149] According to one or more embodiments of the present disclosure, the method further includes: obtaining column boundary pixel points corresponding to the target pixel points in the current pixel row; and obtaining replacement pixel values ​​corresponding to each target pixel point in the current pixel row based on the pixel values ​​of the row boundary pixel points, including: obtaining replacement pixel values ​​corresponding to the target pixel points based on a weighted sum of the pixel values ​​of the column boundary pixel points and the pixel values ​​of the row boundary pixel points.

[0150] According to one or more embodiments of the present disclosure, after obtaining the number of row projection points corresponding to the target pixel point in the current pixel row, the method further includes: if the number of the row projection points is less than one, determining a mirror pixel row of the current pixel row, the position of the mirror pixel row in the image to be processed being mirrored to the position of the current pixel row in the image to be processed; and obtaining replacement pixel values ​​corresponding to each target pixel point in the current pixel row based on the pixel value of each pixel point in the mirror pixel row.

[0151] According to one or more embodiments of the present disclosure, determining the erasure area of ​​the image to be processed includes: identifying the outline of the target object in the image to be processed; generating a mask image based on the outline of the target object, wherein the mask image is used to characterize the erasure area in the processed image.

[0152] According to one or more embodiments of the present disclosure, before determining the erased area of ​​the image to be processed according to the mask image, it also includes: obtaining special effect information, wherein the special effect information represents the contour shape of the target special effect added in the image to be processed; and based on the special effect information, dilating the mask image to update the erased area in the image to be processed.

[0153] According to one or more embodiments of the present disclosure, at least one of the following items is also included: after determining the erased area of ​​the image to be processed, adding Gaussian blur to the image to be processed; after generating the completed image, adding Gaussian blur to the erased area in the completed image.

[0154] According to one or more embodiments of the present disclosure, the method also includes: obtaining device performance information of the terminal device; determining a Gaussian blur parameter based on the device performance information, wherein the Gaussian blur parameter is used to characterize the number of times the Gaussian blur is performed and / or the accuracy of the Gaussian blur; adding Gaussian blur includes: adding Gaussian blur based on the Gaussian blur parameter.

[0155] In a second aspect, according to one or more embodiments of the present disclosure, there is provided an image processing apparatus, comprising:

[0156] An acquisition module, used for acquiring an image to be processed and determining an erasing area of ​​the image to be processed;

[0157] A processing module, configured to obtain a replacement pixel value corresponding to a target pixel point in the erasing area, wherein the replacement pixel value is determined based on a boundary pixel point corresponding to the target pixel point, and the boundary pixel point is a projection point of the target pixel value on a region boundary of the erasing area;

[0158] A generation module is used to set corresponding target pixel points according to the replacement pixel value to generate a completed image.

[0159] According to one or more embodiments of the present disclosure, the processing module is specifically used to: for each pixel row of the image to be processed, perform the following steps in sequence: obtain the number of row projection points corresponding to the target pixel point in the current pixel row, and the row projection point is located at the intersection of the current pixel row and the region boundary; if the number of the row projection points is at least one, then according to the distance between the row projection point and the corresponding target pixel point, obtain the row boundary pixel point corresponding to the target pixel point in the current pixel row; according to the pixel value of the row boundary pixel point, obtain the replacement pixel value corresponding to each target pixel point in the current pixel row.

[0160] According to one or more embodiments of the present disclosure, when the row projection points include at least one first row projection point and / or at least one second row projection point, and the first row projection point and the second row projection point are respectively located on both sides of the target pixel point, the processing module, when obtaining the row boundary pixel point corresponding to the target pixel point in the current pixel row according to the distance between the row projection point and the corresponding target pixel point, is specifically used to: determine the first row projection point closest to the target pixel point as the first row boundary pixel point; and / or determine the second row projection point closest to the target pixel point as the second row boundary pixel point.

[0161] According to one or more embodiments of the present disclosure, when the row projection points include at least one first row projection point located on the same side of the target pixel point, the processing module, when obtaining the row boundary pixel point corresponding to the target pixel point in the current pixel row based on the distance between the row projection point and the corresponding target pixel point, is specifically used to: determine the first row projection point closest to the target pixel point as the first row boundary pixel point; and determine the pixel point located in the current pixel row and outside the erasing area as the second row boundary pixel point.

[0162] According to one or more embodiments of the present disclosure, when the processing module determines a pixel point located within the current pixel row and outside the erasing area as a second-row boundary pixel point, it is specifically used to: determine the first-row projection point farthest from the target pixel point or the end pixel point of the current pixel row on the side of the first-row projection point as the second-row boundary pixel point.

[0163] According to one or more embodiments of the present disclosure, the row boundary pixel points include a first row boundary pixel point and a second row boundary pixel point; when the processing module obtains the replacement pixel value corresponding to each target pixel point in the current pixel row based on the pixel value of the row boundary pixel point, it is specifically used to: obtain a first weighting coefficient corresponding to the first row boundary pixel point and a second weighting coefficient corresponding to the second row boundary pixel point based on the distances between the first row boundary pixel point and the second row boundary pixel point and the corresponding target pixel point respectively; calculate the weighted sum of the pixel value of the first row boundary pixel point and the pixel value of the second row boundary pixel point based on the first weighting coefficient and the second weighting coefficient to obtain the replacement pixel value corresponding to the target pixel point.

[0164] According to one or more embodiments of the present disclosure, the processing module is further used to: obtain the column boundary pixel points corresponding to the target pixel points in the current pixel row; when the processing module obtains the replacement pixel values ​​corresponding to each target pixel point in the current pixel row according to the pixel values ​​of the row boundary pixel points, the processing module is specifically used to: obtain the replacement pixel value corresponding to the target pixel point based on a weighted sum of the pixel values ​​of the column boundary pixel points and the pixel values ​​of the row boundary pixel points.

[0165] According to one or more embodiments of the present disclosure, after obtaining the number of row projection points corresponding to the target pixel point in the current pixel row, the processing module is further used to: if the number of row projection points is less than one, determine a mirror pixel row of the current pixel row, and the position of the mirror pixel row in the image to be processed is mirrored to the position of the current pixel row in the image to be processed; and obtain replacement pixel values ​​corresponding to each target pixel point in the current pixel row based on the pixel value of each pixel point in the mirror pixel row.

[0166] According to one or more embodiments of the present disclosure, the acquisition module is specifically used to: identify the contour of the target object in the image to be processed; generate a mask image based on the contour of the target object, and the mask image is used to characterize the erased area in the processed image.

[0167] According to one or more embodiments of the present disclosure, before determining the erased area of ​​the image to be processed based on the mask image, the acquisition module is also used to: obtain special effect information, wherein the special effect information represents the contour shape of the target special effect added to the image to be processed; and expand the mask image based on the special effect information to update the erased area in the image to be processed.

[0168] According to one or more embodiments of the present disclosure, the processing module is also used for at least one of the following: after determining the erased area of ​​the image to be processed, adding Gaussian blur to the image to be processed; after generating the completed image, adding Gaussian blur to the erased area in the completed image.

[0169] In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: at least one processor and a memory;

[0170] The memory stores computer-executable instructions;

[0171] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the image processing method described in the first aspect and various possible designs of the first aspect.

[0172] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer execution instructions. When a processor executes the computer execution instructions, the image processing method described in the first aspect and various possible designs of the first aspect is implemented.

[0173] In a fifth aspect, according to one or more embodiments of the present disclosure, a computer program product is provided, including a computer program, which, when executed by a processor, implements the image processing method described in the first aspect and various possible designs of the first aspect.

[0174] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0175] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0176] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. An image processing method, characterized in that: include: Acquire an image to be processed, and determine an erasing area of ​​the image to be processed; Obtaining a replacement pixel value corresponding to a target pixel point in the erasing area, wherein the replacement pixel value is determined based on a boundary pixel point corresponding to the target pixel point, and the boundary pixel point is a projection point of the target pixel value on a region boundary of the erasing area; The corresponding target pixel point is set according to the replacement pixel value to generate a completed image.

2. The method according to claim 1, characterized in that The step of obtaining a replacement pixel value corresponding to a target pixel point in the erasing area includes: For each pixel row of the image to be processed, the following steps are performed in sequence: Obtaining the number of row projection points corresponding to the target pixel point in the current pixel row, wherein the row projection point is located at the intersection of the current pixel row and the region boundary; If the number of the row projection point is at least one, obtaining a row boundary pixel point corresponding to the target pixel point in the current pixel row according to the distance between the row projection point and the corresponding target pixel point; According to the pixel values ​​of the row boundary pixels, replacement pixel values ​​corresponding to the target pixel points in the current pixel row are obtained.

3. The method according to claim 2, characterized in that When the row projection points include at least one first row projection point and / or at least one second row projection point, and the first row projection point and the second row projection point are respectively located on both sides of the target pixel point, obtaining the row boundary pixel point corresponding to the target pixel point in the current pixel row according to the distance between the row projection point and the corresponding target pixel point includes: Determine the first row of projection points closest to the target pixel point as the first row of boundary pixel points; and / or, The second row projection point closest to the target pixel point is determined as the second row boundary pixel point.

4. The method according to claim 2, characterized in that: When the row projection points include at least one first row projection point located on the same side of the target pixel point, obtaining the row boundary pixel point corresponding to the target pixel point in the current pixel row according to the distance between the row projection point and the corresponding target pixel point includes: Determine the first row of projection points closest to the target pixel point as the first row of boundary pixel points; Pixel points located in the current pixel row and outside the erasing area are determined as second row boundary pixel points.

5. The method according to claim 4, characterized in that The step of determining the pixel points located in the current pixel row and outside the erasing area as the second row boundary pixel points includes: The first row projection point farthest from the target pixel point or the end pixel point of the current pixel row on one side of the first row projection point is determined as the second row boundary pixel point.

6. The method according to claim 2, characterized in that The row boundary pixel points include first row boundary pixel points and second row boundary pixel points; and obtaining the replacement pixel values ​​corresponding to the target pixel points in the current pixel row according to the pixel values ​​of the row boundary pixel points includes: Obtaining a first weighting coefficient corresponding to the first row of boundary pixels and a second weighting coefficient corresponding to the second row of boundary pixels according to the distances between the first row of boundary pixels and the corresponding target pixels respectively; A weighted sum of the pixel values ​​of the first row boundary pixel points and the pixel values ​​of the second row boundary pixel points is calculated according to the first weighting coefficient and the second weighting coefficient to obtain a replacement pixel value corresponding to the target pixel point.

7. The method according to claim 2, characterized in that The method further comprises: Obtaining a column boundary pixel point corresponding to a target pixel point in the current pixel row; The obtaining, according to the pixel values ​​of the row boundary pixels, the replacement pixel values ​​corresponding to the target pixel points in the current pixel row includes: A replacement pixel value corresponding to the target pixel point is obtained based on a weighted sum of the pixel value of the column boundary pixel point and the pixel value of the row boundary pixel point.

8. The method according to claim 2, characterized in that After obtaining the number of row projection points corresponding to the target pixel point in the current pixel row, the method further includes: If the number of the row projection points is less than one, determining a mirror pixel row of the current pixel row, wherein a position of the mirror pixel row in the image to be processed is a mirror image of a position of the current pixel row in the image to be processed; According to the pixel value of each pixel point in the mirror pixel row, the replacement pixel value corresponding to each target pixel point in the current pixel row is obtained.

9. The method according to claim 1, characterized in that: The step of determining the erasing area of ​​the image to be processed includes: Identifying the contour of the target object in the image to be processed; A mask image is generated according to the contour of the target object, and the mask image is used to represent the erased area in the processed image.

10. The method according to claim 9, characterized in that Before determining the erasing area of ​​the image to be processed according to the mask image, the method further includes: Acquire special effect information, where the special effect information represents a contour shape of a target special effect added to the image to be processed; Based on the special effect information, the mask image is expanded to update the erased area in the image to be processed.

11. The method according to claim 1, characterized in that Also includes at least one of the following: After determining the erasing area of ​​the image to be processed, adding Gaussian blur to the image to be processed; After the complement image is generated, Gaussian blur is added to the erased area in the complement image.

12. The method according to claim 11, characterized in that The method further comprises: Obtain device performance information of terminal devices; Determine a Gaussian blur parameter according to the device performance information, where the Gaussian blur parameter is used to characterize the number of times the Gaussian blur is performed and / or the accuracy of the Gaussian blur; The adding of Gaussian blur comprises: Adds a Gaussian blur based on the Gaussian blur parameters.

13. An image processing device, characterized in that: include: An acquisition module, used for acquiring an image to be processed and determining an erasing area of ​​the image to be processed; A processing module, configured to obtain a replacement pixel value corresponding to a target pixel point in the erasing area, wherein the replacement pixel value is determined based on a boundary pixel point corresponding to the target pixel point, and the boundary pixel point is a projection point of the target pixel value on a region boundary of the erasing area; A generation module is used to set corresponding target pixel points according to the replacement pixel value to generate a completed image.

14. An electronic device, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the image processing method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the image processing method according to any one of claims 1 to 12 is implemented.

16. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the image processing method according to any one of claims 1 to 12 is implemented.

Images