Image content repair method, electronic device and storage medium

Abstract

The embodiment of the present invention provides an image content repair method, an electronic device and a storage medium, the method comprising: obtaining a mask map of the area to be repaired mapped on the image to be processed, determining a filling image of the area to be repaired in the mask map of the area to be repaired; and repairing the area to be repaired on the image to be processed according to the filling image. The image content repair method, electronic device and storage medium provided by the embodiment of the present invention obtain a mask map of the area to be repaired mapped on the image to be processed, and determine a filling image of the area to be repaired in the mask map of the area to be repaired, so that the filling image repairs the area to be repaired on the image to be processed, and realizes a reasonable fusion of pixel values ​​on the filling image and pixel values ​​outside the area to be repaired on the facial image to be processed, so as to achieve a better defect repair effect on the image.

Description

Technical Field

[0001] The present invention relates to the field of information processing technology, and in particular to an image content repair method, electronic equipment and storage medium. Background Art

[0002] Sometimes, images taken by users need to be repaired to present a better picture state. Especially for users' selfies, image repair is more necessary. For this reason, in the field of digital image processing, image repair technology is needed to repair defects on the images taken by users.

[0003] The current image restoration technology mainly uses the surrounding images to fill the area to be restored, which may cause local blurring of the restored area, or uses texture matching, which may fail to match and may still have defects. Therefore, the restoration effect cannot be improved. Summary of the invention

[0004] In view of the problems existing in the prior art, embodiments of the present invention provide an image content repair method, an electronic device and a storage medium.

[0005] In a first aspect, an embodiment of the present invention provides an image content restoration method, comprising:

[0006] Acquire a mask image of the area to be repaired mapped on the image to be processed, and determine a filling image of the area to be repaired in the mask image of the area to be repaired;

[0007] The to-be-repaired area on the to-be-processed image is repaired according to the filling image.

[0008] Furthermore, the step of obtaining a mask image of the area to be repaired mapped on the image to be processed includes:

[0009] Based on the area to be repaired marked in the standard image, a standard mask image is exported;

[0010] Acquire key point data of a standard image and an image to be processed, and determine an affine transformation matrix according to the key point data;

[0011] The standard mask image is mapped using the affine transformation matrix to obtain a mask image of the area to be repaired.

[0012] Furthermore, the acquiring of key point data of the standard image and the image to be processed, and determining a mapping transformation matrix according to the key point data, includes:

[0013] Performing triangulation processing according to key point data of the standard image and the image to be processed to obtain a first grid corresponding to the standard image and a second grid corresponding to the image to be processed;

[0014] An affine transformation matrix is ​​established between the triangular faces in the first mesh and the corresponding triangular faces in the second mesh.

[0015] Furthermore, the step of determining a filling image of the area to be repaired in the mask image of the area to be repaired includes:

[0016] The mask image of the area to be repaired is moved in a preset direction on the image to be processed, and based on the position of the area to be repaired in the mask image of the area to be repaired after translation on the image to be processed, an image corresponding to the position is selected as a filling image of the area to be repaired.

[0017] Further, the repairing the to-be-repaired area on the to-be-processed image according to the filling image comprises:

[0018] Determine that a pixel point in the filling image corresponds to a matching pixel point on the image to be processed, and update the pixel value of the pixel point in the filling image according to the pixel value of the matching pixel point;

[0019] The area to be repaired on the processed image is repaired according to the filling image after the pixel value is updated.

[0020] Further, the determining that the pixel points in the filling image correspond to the matching pixel points on the image to be processed includes:

[0021] Traversing the pixel points in the filling image, selecting the target pixel point, and determining the position of the target pixel point;

[0022] Taking the position of the target pixel as the center, a texture block of a preset size is constructed;

[0023] Based on the position of each pixel point in the texture block and the position of a preset number of pixel points selected on the image to be processed, the similarity between the target pixel point and each pixel point in the preset number of pixel points is calculated;

[0024] The pixel point corresponding to the minimum similarity is determined as the matching pixel point.

[0025] Furthermore, after obtaining the mask map of the area to be repaired mapped on the image to be processed, the method further includes:

[0026] Determine the minimum bounding rectangle of the area to be repaired in the mask map of the area to be repaired, perform edge detection of the area to be repaired on the image to be processed based on the minimum bounding rectangle area, determine the updated area to be repaired, obtain the updated mask map mapped on the image to be processed according to the updated area to be repaired, and use the updated mask map as the new mask map of the area to be repaired.

[0027] Furthermore, after repairing the to-be-repaired area on the to-be-processed image according to the filling image after the pixel value is updated, the method further includes: filtering the to-be-processed image after the repair process.

[0028] In a second aspect, an embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned image content repair method when executing the program.

[0029] In a third aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, which implements the steps of the above-mentioned image content repair method when executed by a processor.

[0030] The image content restoration method, electronic device and storage medium provided by the embodiments of the present invention obtain a mask image of the area to be restored mapped on the image to be processed, and determine a filling image of the area to be restored in the mask image of the area to be restored, so that the filling image is used to restore the area to be restored on the image to be processed, and a reasonable fusion of pixel values ​​on the filling image and pixel values ​​outside the area to be restored on the facial image to be processed is achieved, so that the defect restoration effect on the image is better. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 This is a flow chart of an embodiment of the image content repair method of the present invention;

[0033] Figure 2 It is a schematic diagram of a specific process of the image content repair method of the present invention;

[0034] Figure 3 This is a structural diagram of an embodiment of an image content restoration device of the present invention;

[0035] Figure 4 It is a structural diagram of an electronic device embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] Figure 1 A schematic diagram of a process flow of an image content restoration method provided by an embodiment of the present invention is shown. Figure 1 , the method comprising:

[0038] S11, obtaining a mask image of the area to be repaired mapped on the image to be processed, and determining a filling image of the area to be repaired in the mask image of the area to be repaired;

[0039] S12, repairing the to-be-repaired area on the image to be processed according to the filling image.

[0040] Regarding step S11 and step S12, it should be noted that in an embodiment of the present invention, the image taken by the user sometimes needs to be repaired to present a better picture state. In particular, the user's selfie needs image repair. The skin condition shown in the selfie is prone to blemishes, such as wrinkles (forehead wrinkles, nasolabial folds, etc.), spots, acne (blackheads and pimples), and may also include moles, as well as stains on the skin. For this reason, it is even more necessary to repair these blemishes.

[0041] In the embodiment of the present invention, different defects are configured with different defect types. The types corresponding to the above-mentioned defects are nasolabial fold type, forehead wrinkle type, pigmentation type, acne type, etc.

[0042] Different defect types have corresponding contour areas on the image, and the contour areas are the areas to be repaired that need to be repaired.

[0043] For example, the contour area of ​​the nasolabial fold type can be an "eight-shaped" area.

[0044] The contour area of ​​the forehead wrinkle type can be a preset area of ​​the forehead.

[0045] The contour area of ​​the acne type may be a preset area of ​​the forehead, a preset area of ​​the left face, or a preset area of ​​the right face.

[0046] In the embodiment of the present invention, the method needs to map a mask map of the region to be repaired onto the image to be processed, and the mask map of the region to be repaired includes the region to be repaired.

[0047] Then, fill the image in the area to be repaired in the mask image of the area to be repaired. The filled image is used to further improve the basis for repairing the defects on the image to be processed. The filled image can collect the image in the area without defects on the image to be processed. The image has a weak texture structure, uniform color, and no obvious brightness change.

[0048] Then, the pixels in the to-be-repaired area on the processed image are repaired based on the pixels on the filled image to obtain an image with better color.

[0049] The image content restoration method provided by the embodiment of the present invention obtains a mask image of the area to be restored mapped on the image to be processed, and determines a filling image of the area to be restored in the mask image of the area to be restored, so that the filling image is used to restore the area to be restored on the image to be processed, and a reasonable fusion of pixel values ​​on the filling image and pixel values ​​outside the area to be restored on the facial image to be processed is achieved, so that the defect restoration effect on the image is better.

[0050] In a further embodiment of the above-mentioned embodiment method, the processing process of repairing the to-be-repaired area on the processed image is mainly explained based on the filling image, which is specifically as follows:

[0051] Determine that a pixel point in the filling image corresponds to a matching pixel point on the image to be processed, and update the pixel value of the pixel point in the image according to the pixel value of the matching pixel point;

[0052] The area to be repaired on the processed image is repaired according to the filling image after the pixel value is updated.

[0053] In this regard, it should be noted that the improvement of the filling image is mainly reflected in the update of the pixel value of each pixel in the image, and its purpose is to maximize the reasonable fusion of the filling image and the pixels around the area to be repaired on the processed image. To this end, it is necessary to determine the matching position of the pixel point in the filling image on the processed image, and then use the pixel value of the pixel point corresponding to the matching position to update the pixel value of the pixel point in the filling image, thereby completing the pixel improvement of the filling image.

[0054] Then, the image of the to-be-repaired area on the processed image is repaired according to the filled image with updated pixel values ​​to obtain a repaired image. The repaired image has a better picture quality than the processed image.

[0055] This embodiment matches the pixel points on the filling image of the area to be repaired on the mask image to reasonable pixel points on the image to be processed, updates the pixel values ​​on the filling image with the pixel values ​​of the matched pixel points, and then uses the filling image with updated pixel values ​​to repair the area to be repaired, thereby achieving reasonable fusion of the pixel values ​​on the filling image and the pixel values ​​outside the area to be repaired on the image to be processed, thereby achieving better image repair effect.

[0056] In a further embodiment of the above embodiment method, the process of obtaining the mask image of the area to be repaired mapped on the image to be processed is mainly explained as follows:

[0057] Based on the defect areas marked in the standard image, a standard mask map is exported.

[0058] The key point data of the standard image and the image to be processed are obtained, and the affine transformation matrix is ​​determined according to the key point data.

[0059] The standard mask image is mapped using an affine transformation matrix to obtain a mask image of the area to be repaired.

[0060] It should be noted that, in the embodiment of the present invention, for a selfie, the selected standard image may be a standard facial image, such as a frontal, expressionless model facial image.

[0061] The area to be repaired (ie, the area with defects) is automatically marked on the standard image according to the defect type, and then exported in the form of a mask based on the area to be repaired to obtain a standard mask image.

[0062] Obtain the image to be processed, perform key point detection on the standard image and the image to be processed respectively, and obtain the corresponding key point data. The key point detection can adopt a 68-point model or a 106-point model.

[0063] After the key point data is obtained, it is necessary to establish a corresponding relationship between the standard image and the image to be processed. To this end, an affine transformation relationship between the two images, that is, an affine transformation matrix, can be constructed.

[0064] After the affine transformation matrix is ​​obtained, the standard mask map can be numerically calculated based on the affine transformation matrix and mapped onto the image to be processed, which is regarded as the mask map of the area to be repaired.

[0065] In this embodiment, by setting a standard image, an affine transformation matrix between the standard image and the image to be processed is established, and the standard mask image is mapped into a mask image of the area to be repaired corresponding to the image to be processed, so that the standard image is used as a reference to obtain a mask image of the area to be repaired that is closer to the image to be processed, which facilitates subsequent image repair processing.

[0066] In a further embodiment of the above embodiment method, the process of obtaining key point data of the standard image and the image to be processed and determining the mapping transformation matrix according to the key point data is mainly explained as follows:

[0067] Performing triangulation processing according to key point data of the standard image and the image to be processed to obtain a first grid corresponding to the standard image and a second grid corresponding to the image to be processed;

[0068] An affine transformation matrix is ​​established between the triangular faces in the first mesh and the corresponding triangular faces in the second mesh.

[0069] In this regard, it should be noted that, in the embodiment of the present invention, since the key point data are position points on the image with special key feature points, a grid map can be obtained based on a reasonable division between these key points, which is regarded as a grid, namely: a first grid corresponding to the standard image, and a second grid corresponding to the facial image to be processed.

[0070] Since the mesh adopts a triangular profile, there will be multiple triangular faces in the mesh. The key point data adopts a 68-point model or a 106-point model, so each key point will have a corresponding number. At this time, the vertices of the triangular face will be uniquely determined by the number. Therefore, it can be determined that a certain triangular face in the first mesh corresponds to a certain triangular face in the second mesh. Here, it is necessary to establish an affine transformation matrix between the two corresponding triangular faces. In fact, the affine transformation matrix is ​​the affine transformation matrix between the vertices of the two triangular faces. After the affine transformation matrix is ​​determined, each image pixel located in the triangular face vertex area is transformed based on the affine matrix.

[0071] In addition, after the 68-point model is used to obtain key point data, in order to improve the accuracy of affine, the key point data can be interpolated to obtain more key point data.

[0072] In this embodiment, key point detection is performed on the two images to determine key point data, thereby obtaining a mesh, and then based on the unique correspondence between the triangular faces, an affine transformation matrix between the vertices is established to achieve accurate affine between the mask images.

[0073] Figure 2 This is a schematic diagram of the specific process of the image content restoration method of this embodiment, based on Figure 2 The process of S21-S24 in the above example is described below using the nasolabial folds on the face as an example to obtain the mask image. The details are as follows:

[0074] 1) Select a standard face image (a frontal, expressionless model face image) as the standard face image, denoted as S 1 , mark the shape area of ​​nasolabial folds on the standard facial image, and export the shape area in the form of a mask as the standard mask map, denoted as M 1 .

[0075] 2) Input the facial image to be processed, denoted as S 2, respectively, perform key point detection on the standard face image and the face image to be processed, and obtain the key point data of the face (used to mark the position or shape of facial organs, such as the general 68-point model or 106-point model). The key point data corresponding to the two images are recorded as L 1 and L 2 .

[0076] 3) Based on the above key point data L 1 and L 2 The Delaunay triangulation method is used to generate the corresponding mesh (a mesh structure consisting of lines connecting three discrete vertices), where the mesh is composed of multiple triangular faces, and the vertex of each triangular face stores the key point index number (vertex number, corresponding to 0 to 108).

[0077] 4) Traverse each triangular face in the above mesh, such as the triangular face Δ of a standard facial image 1 , the triangular face Δ of the face image to be processed 2 , calculate the affine transformation matrix between the two triangular faces (here, the affine transformation matrix between the two sets of triangular vertices is calculated, and then the image pixels in the vertex area are transformed based on the affine matrix), and use the affine transformation matrix to transform the standard mask face M 1 Δ 1 Region image mapping to Δ 2 After traversing all the triangles in the above grid, the regional mask map on the face image to be processed is obtained, which is denoted as M 2-1 .

[0078] In a further embodiment of the above embodiment method, after obtaining the mask map of the area to be repaired mapped on the image to be processed, the following subsequent steps are also included:

[0079] Determine the minimum bounding rectangle of the area to be repaired in the mask map of the area to be repaired, perform edge detection of the area to be repaired on the image to be processed based on the minimum bounding rectangle area, determine the updated area to be repaired, obtain the updated mask map mapped on the image to be processed according to the updated area to be repaired, and use the updated mask map as the new mask map of the area to be repaired.

[0080] In this regard, it should be noted that, taking the nasolabial folds as an example, the mask M of the area to be repaired is calculated. 2-1 The minimum circumscribed rectangle of the image to be processed is used to detect the nasolabial fold area in the rectangular area using Frangi filtering, and an area is determined, which is regarded as the "updated repair area". Then, the mask image mapped on the image to be processed is obtained using the updated repair area, which is regarded as the "updated mask image", denoted as M 2-2 .

[0081] In a further embodiment of the above embodiment method, the processing of determining the filling image of the area to be repaired in the mask image of the area to be repaired is mainly explained as follows:

[0082] The mask image of the area to be repaired is moved in a preset direction (horizontally, vertically or diagonally) on the image to be processed. Based on the position of the area to be repaired in the mask image of the area to be repaired after translation, the image corresponding to the position is selected as the filling image of the area to be repaired. The filling image can collect an image in the area without defects on the image to be processed. The image has a weak texture structure, uniform color, and no obvious brightness change.

[0083] In this embodiment, an image of an area close to the area to be repaired can be obtained by translating the mask image, which facilitates subsequent accurate matching of defect repair.

[0084] In a further embodiment of the above-mentioned embodiment method, the process of determining whether a pixel point in the filling image corresponds to a matching pixel point on the image to be processed is mainly explained, as follows:

[0085] Traverse the pixel points in the filling image, select the target pixel point, and determine the position of the target pixel point;

[0086] Taking the position of the target pixel as the center, a texture block of a preset size is constructed;

[0087] Based on the position of each pixel point belonging to the texture block and the position of a preset number of pixel points selected on the image to be processed, the similarity between the target pixel point and each pixel point in the preset number of pixel points is calculated;

[0088] The pixel point corresponding to the minimum similarity is determined as the matching pixel point.

[0089] In this regard, it should be noted that, in the embodiment of the present invention, it is necessary to update the pixel value of each pixel in the filling image, and for this purpose, it is necessary to traverse the pixel points in the filling image.

[0090] Select any pixel point as the target pixel point, determine the position coordinates of the target pixel point, use the position coordinates of the target pixel point as the center point coordinates, and construct a texture block of a preset size (such as 7*7).

[0091] A certain number of pixels are selected from the area to be repaired and the vicinity of the area on the image to be processed, and the position coordinates of these pixels are determined. Based on the position coordinates of each pixel belonging to the texture block and the position coordinates of a certain number of pixels on the image to be processed, the similarity between the target pixel and each pixel in the preset number of pixels is calculated.

[0092] Then the pixel point corresponding to the minimum similarity is selected as the matching pixel point.

[0093] based on Figure 2 The process of S25-S27 in the figure is described below using the nasolabial folds on the face as an example to illustrate the above matching positions in detail, as follows:

[0094] Edit the above mask M 2-2 The filling image in the area, select any pixel position (x, y) in the filling image. In the initial state, select the face image S to be processed 2 The initial matching position (u,v) on the image is (x,y), so that (u,v) = (x,y).

[0095] At this time, the similarity s between each position on the texture block centered at position (x, y) and position (u, v) is calculated and recorded as score, and score(x, y) = s. In this embodiment, the similarity calculation process is implemented using the square sum distance SSD.

[0096] Then select the face image S to be processed 2 Other positions adjacent to the initial matching position are selected (i.e., adjacent matching is performed, and the adjacent positions are selected by moving up, down, left, and right, such as (u-1, v), (u, v-1), (u+1, v) or (u, v+1)), and then the similarity s1 between each position on the texture block centered on the position (x, y) and the selected adjacent position is calculated. If s1 is less than s, score(x, y) = s1 is updated.

[0097] According to a preset number N, N adjacent positions are selected, and then the similarity between each position on the texture block centered at position (x, y) and the selected N adjacent positions is calculated. After the calculation is completed, score(x, y) = minimum similarity is selected.

[0098] Then the position of the pixel corresponding to the minimum similarity is determined as the final matching position.

[0099] This embodiment performs a proximity matching search through similarity calculation to obtain pixel information required for defect repair on the image, thereby improving the accuracy of subsequent defect repair processing.

[0100] In a further embodiment of the above embodiment method, after the image to be processed is repaired according to the filled image after the pixel value is updated, the following subsequent steps are also included:

[0101] The image to be processed is subjected to filtering processing after the defect repair processing.

[0102] The filtering process can be implemented by using Gaussian filtering, which mainly processes the image obtained by performing defect repair on the image to be processed according to the filling image updated with pixel values ​​by using Gaussian filtering to make the defect local repair smoother.

[0103] Figure 3 FIG. 1 shows a schematic diagram of the structure of an image content restoration device provided by an embodiment of the present invention. Figure 3 , the device comprises an acquisition module 31 and a processing module 32, wherein:

[0104] An acquisition module 31 is used to acquire a mask image of the area to be repaired mapped on the image to be processed, and determine a filling image of the area to be repaired in the mask image of the area to be repaired;

[0105] The processing module 32 is used to repair the to-be-repaired area on the to-be-processed image according to the filling image.

[0106] In a further embodiment of the device of the above embodiment, the processing module is specifically used for:

[0107] Determine that a pixel point in the filling image corresponds to a matching pixel point on the image to be processed, and update the pixel value of the pixel point in the filling image according to the pixel value of the matching pixel point;

[0108] The area to be repaired on the processed image is repaired according to the filling image after the pixel value is updated.

[0109] In a further embodiment of the device of the above embodiment, the acquisition module is specifically used to:

[0110] Based on the area to be repaired marked in the standard image, a standard mask image is exported;

[0111] Acquire key point data of a standard image and an image to be processed, and determine an affine transformation matrix according to the key point data;

[0112] The standard mask image is mapped using the affine transformation matrix to obtain a mask image of the area to be repaired.

[0113] In a further embodiment of the above-mentioned embodiment device, the acquisition module, in the process of acquiring key point data of the standard image and the image to be processed and determining the mapping transformation matrix according to the key point data, is specifically used to:

[0114] Performing triangulation processing according to key point data of the standard image and the image to be processed to obtain a first grid corresponding to the standard image and a second grid corresponding to the image to be processed;

[0115] An affine transformation matrix is ​​established between the triangular faces in the first mesh and the corresponding triangular faces in the second mesh.

[0116] In a further embodiment of the device of the above embodiment, the acquisition module is specifically used to:

[0117] The mask image of the area to be repaired is moved in a preset direction on the image to be processed, and based on the position of the area to be repaired in the mask image of the area to be repaired after translation on the image to be processed, an image corresponding to the position is selected as a filling image of the area to be repaired.

[0118] In a further embodiment of the apparatus of the above embodiment, the processing module, in the process of determining that a pixel point in the filling image corresponds to a matching pixel point on the image to be processed, is specifically used to:

[0119] Traversing the pixel points in the filling image, selecting the target pixel point, and determining the position of the target pixel point;

[0120] Taking the position of the target pixel as the center, a texture block of a preset size is constructed;

[0121] Based on the position of each pixel point in the texture block and the position of a preset number of pixel points selected on the image to be processed, the similarity between the target pixel point and each pixel point in the preset number of pixel points is calculated;

[0122] The pixel point corresponding to the minimum similarity is determined as the matching pixel point.

[0123] In a further embodiment of the device of the above embodiment, the device also includes an optimization module, which is used to determine the minimum enclosing rectangle of the area to be repaired in the mask map of the area to be repaired after obtaining the mask map of the area to be repaired mapped on the image to be processed, perform edge detection of the area to be repaired on the image to be processed based on the minimum enclosing rectangle area, determine the area to be repaired for update, and obtain the update mask map mapped on the image to be processed according to the update area to be repaired, so that the update mask map is used as the new mask map of the area to be repaired.

[0124] In a further embodiment of the device in the above embodiment, the device further includes an adjustment module for performing filtering processing on the image to be processed after the image to be processed is repaired according to the filling image after the pixel value is updated.

[0125] Since the principle of the device described in the embodiment of the present invention is the same as that of the method described in the above embodiment, a more detailed explanation will not be repeated here.

[0126] It should be noted that, in the embodiment of the present invention, the relevant functional modules can be implemented by a hardware processor.

[0127] An image content restoration device provided by an embodiment of the present invention obtains a mask image of a region to be restored mapped on an image to be processed, and determines a filling image of the region to be restored in the mask image of the region to be restored, so that the filling image is used to restore the region to be restored on the image to be processed, thereby achieving a reasonable fusion of pixel values ​​on the filling image and pixel values ​​outside the region to be restored on the facial image to be processed, thereby achieving a better defect restoration effect on the image.

[0128] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 41, a communication interface 42, a memory 43 and a communication bus 44, wherein the processor 41, the communication interface 42 and the memory 43 communicate with each other through the communication bus 44. The processor 41 may call the logic instructions in the memory 43 to execute the following method: obtain a mask image of the area to be repaired mapped on the image to be processed, determine a filling image of the area to be repaired in the mask image of the area to be repaired; and repair the area to be repaired on the image to be processed according to the filling image.

[0129] In addition, the logic instructions in the above-mentioned memory 43 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0130] An embodiment of the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the methods provided in the above embodiments, for example, including: obtaining a mask map of the area to be repaired mapped on the image to be processed, determining a filling image of the area to be repaired in the mask map of the area to be repaired; and repairing the area to be repaired on the image to be processed according to the filling image.

[0131] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for image content restoration, characterized in that: include: Acquire a mask image of the area to be repaired mapped on the image to be processed, move the mask image of the area to be repaired in a preset direction on the image to be processed, and based on the position of the area to be repaired in the mask image of the area to be repaired after translation on the image to be processed, select an image corresponding to the position as a filling image of the area to be repaired; The filling image captures an image in a region without defects on the image to be processed; Repairing the to-be-repaired area on the to-be-processed image according to the filling image; The repairing of the to-be-repaired area on the to-be-processed image according to the filling image comprises: Determine that a pixel point in the filling image corresponds to a matching pixel point on the image to be processed, and update the pixel value of the pixel point in the filling image according to the pixel value of the matching pixel point; Repairing the to-be-repaired area on the processed image according to the filled image after the pixel value is updated; The step of determining that a pixel point in the filled image corresponds to a matching pixel point on the image to be processed includes: Traversing the pixel points in the filling image, selecting the target pixel point, and determining the position of the target pixel point; Taking the position of the target pixel as the center, a texture block of a preset size is constructed; Based on the position of each pixel point in the texture block and the position of a preset number of pixel points selected on the image to be processed, the similarity between the target pixel point and each pixel point in the preset number of pixel points is calculated; The pixel point corresponding to the minimum similarity is determined as the matching pixel point.

2. The image content restoration method according to claim 1, characterized in that: The step of obtaining a mask image of the area to be repaired mapped on the image to be processed includes: Based on the area to be repaired marked in the standard image, a standard mask image is exported; Acquire key point data of a standard image and an image to be processed, and determine an affine transformation matrix according to the key point data; The standard mask image is mapped using the affine transformation matrix to obtain a mask image of the area to be repaired.

3. The image content restoration method according to claim 2, characterized in that: The step of acquiring key point data of the standard image and the image to be processed, and determining a mapping transformation matrix according to the key point data, comprises: Performing triangulation processing according to key point data of the standard image and the image to be processed to obtain a first grid corresponding to the standard image and a second grid corresponding to the image to be processed; An affine transformation matrix is ​​established between the triangular faces in the first mesh and the corresponding triangular faces in the second mesh.

4. The image content restoration method according to claim 1, characterized in that: After obtaining the mask map of the area to be repaired mapped on the image to be processed, the following steps are also included: Determine the minimum bounding rectangle of the area to be repaired in the mask map of the area to be repaired, perform edge detection of the area to be repaired on the image to be processed based on the minimum bounding rectangle area, determine the updated area to be repaired, obtain the updated mask map mapped on the image to be processed according to the updated area to be repaired, and use the updated mask map as the new mask map of the area to be repaired.

5. The image content restoration method according to claim 1, characterized in that: After repairing the to-be-repaired area on the to-be-processed image according to the filling image after the pixel value is updated, the method further includes: filtering the to-be-processed image after the repair process.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the image content repair method according to any one of claims 1 to 5 are implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image content repair method as claimed in any one of claims 1 to 5 are implemented.

Images