Image restoration method, device and server

Abstract

This specification provides an image restoration method, apparatus, and server. Based on this image restoration method, after determining a plurality of first image regions based on the first pixel points on the damaged boundary, first screen the second image region to be restored currently; then, instead of using fixed range parameters, based on a preset Gaussian function model and using the average gradient modulus of the pixel gray values that can reflect the change characteristics of texture information, determine the target range parameters that match the second image region; then, according to the target range parameters, determine the target image block with a relatively appropriate range size for the second image region; furthermore, the second image region can be restored by using the target image block. Thus, it is possible to balance the restoration efficiency and restoration effect, and for the second image regions with different texture information change characteristics, use the target image blocks with different target range parameters in a differentiated manner to perform targeted restoration processing efficiently and accurately.

Description

Technical Field

[0001] This specification belongs to the technical field of image processing, and particularly relates to an image repair method, apparatus, and server. Background Art

[0002] In many relatively complex image processing scenarios (such as those involving artificial intelligence and having high requirements for repair accuracy), it is often necessary to process a large number of images with partial damage to image data due to certain reasons (for example, images with partial data loss during image compression or network transmission, etc.).

[0003] When using existing image repair methods to specifically repair the above-mentioned damaged images, there are often problems such as inflexible and inaccurate repair processes, poor pertinence, and the inability to well balance repair efficiency and repair effect for damaged images in different situations, and perform targeted differential repairs.

[0004] For the above problems, no effective solution has been proposed yet. Summary of the Invention

[0005] This specification provides an image repair method, apparatus, and server, which can better balance repair efficiency and repair effect, and selectively use target image blocks with different target range parameters for different second image regions with different texture information change characteristics to perform targeted repair processing efficiently and accurately.

[0006] An embodiment of this specification provides an image repair method, including:

[0007] Determine the damaged boundary line between the normal image region and the damaged image region in the current target image; wherein, the damaged image region is the image region to be repaired;

[0008] Obtain a plurality of first pixel points on the damaged boundary line; and based on the plurality of first pixel points, determine a plurality of first image regions; wherein, each first image region contains at least one first pixel point; the first image region contains a first type of sub-image region that does not need to be repaired and a second type of sub-image region that needs to be repaired;

[0009] Screen out the current second image region to be repaired from the plurality of first image regions;

[0010] Based on a preset Gaussian function model, determine the target range parameter that matches the second image region;

[0011] According to the target range parameter, retrieve the normal image region in the current target image to determine the target image block that matches the second image region;

[0012] Use the target image block to perform restoration processing on the second image region.

[0013] In some embodiments, after performing restoration processing on the second type of sub-image region in the second image region, the method further includes:

[0014] Update the target image to obtain an updated target image;

[0015] Detect whether there is still a damaged boundary line in the updated target image;

[0016] In the case where it is determined that there is no damaged boundary line in the updated target image, determine the updated target image as the restored target image.

[0017] In some embodiments, screening out the second image region to be currently restored from multiple first image regions includes:

[0018] Obtain and calculate, according to the pixel point confidence of the first type of sub-image region in each first image region, the first type of weight parameter of each first image region based on the confidence;

[0019] Obtain and calculate, according to the texture feature of the first type of sub-image region in each first image region, the second type of weight parameter of each first image region based on the texture feature;

[0020] Calculate the priority parameter of each first image region according to the first type of weight parameter and the second type of weight parameter of each first image region; wherein, the priority parameter is used to characterize the importance of the corresponding first image region for the current target image restoration.

[0021] Screen out the first image region with the largest priority parameter as the second image region.

[0022] In some embodiments, calculating the priority parameter of each first image region according to the first type of weight parameter and the second type of weight parameter of each first image region includes:

[0023] Calculate the priority parameter of the current first image region according to the following formula:

[0024] P(p) = α(ω+(1 - ω)×C(p))+βD(p)

[0025] Wherein, P(p) is the priority parameter of the current first image region, α is the first weight coefficient, β is the second weight coefficient, ω is the normalization factor of the confidence, C(p) is the first type of weight parameter, and D(p) is the second type of weight parameter.

[0026] In some embodiments, determining the target range parameter that matches the second image region based on a preset Gaussian function model includes:

[0027] Calculating the average gradient magnitude of the gray values of the first type of sub-image regions in the second image region, and the average gradient magnitude of the gray values of the first type of sub-image regions in multiple first image regions;

[0028] Invoking the preset Gaussian function model, and calculating the target range parameter according to the average gradient magnitude of the gray values of the first type of sub-image regions in the second image region and the average gradient magnitude of the gray values of the first type of sub-image regions in multiple first image regions.

[0029] In some embodiments, invoking the preset Gaussian function model, and calculating the target range parameter according to the average gradient magnitude of the gray values of the first type of sub-image regions in the second image region and the average gradient magnitude of the gray values of the first type of sub-image regions in multiple first image regions includes:

[0030] Calculating the target range parameter according to the following formula:

[0031]

[0032] where S is the target range parameter, S min is the preset lower limit value of the target range, S max is the preset upper limit value of the target range, is the maximum value among the average gradient magnitudes of the gray values of the first type of sub-image regions in multiple first image regions, is the minimum value among the average gradient magnitudes of the gray values of the first type of sub-image regions in multiple first image regions, is the average gradient magnitude of the gray values of the first type of sub-image regions in the second image region, is the floor operation.

[0033] In some embodiments, retrieving the normal image region in the current target image according to the target range parameter to determine the target image block that matches the second image region includes:

[0034] In the normal image region of the current target image, finding multiple candidate image blocks whose range parameters are equal to the target range parameter;

[0035] Calculating the approximation parameter between the second image region and the multiple candidate image blocks;

[0036] Selecting the candidate image blocks whose approximation parameters meet the preset requirements from the multiple candidate image blocks as the target image blocks that match the second image region.

[0037] In some embodiments, calculating the approximation parameter between the second image region and the plurality of to-be-determined image blocks includes:

[0038] Calculating the approximation parameter between the second image region and the current to-be-determined image block among the plurality of to-be-determined image blocks in the following manner;

[0039] Obtaining the RGB color component values of the pixel points in the second image region and the RGB color component values of the pixel points in the current to-be-determined image block;

[0040] Calculating the sum of the squared differences of the colors of the corresponding pixel points in the second image region and the current to-be-determined image block according to the RGB color component values of the pixel points in the second image region and the RGB color component values of the pixel points in the current to-be-determined image block;

[0041] Calculating the approximation parameter between the second image region and the current to-be-determined image block according to the sum of the squared differences of the colors of the corresponding pixel points in the second image region and the current to-be-determined image block.

[0042] In some embodiments, calculating the sum of the squared differences of the colors of the corresponding pixel points in the second image region and the current to-be-determined image block according to the RGB color component values of the pixel points in the second image region and the RGB color component values of the pixel points in the current to-be-determined image block includes:

[0043] Calculating the sum of the squared differences of the colors of the corresponding pixel points in the second image region and the current to-be-determined image block according to the following formula:

[0044]

[0045] where Ψ p is the second image region, Ψ q is the current to-be-determined image block, R x and R y are the red component values of the corresponding pixel points in the second image region and the current to-be-determined image block respectively, G x and G y are the green component values of the corresponding pixel points in the second image region and the current to-be-determined image block respectively, B x and B y are the blue component values of the corresponding pixel points in the second image region and the current to-be-determined image block respectively.

[0046] In some embodiments, using the target image block to perform restoration processing on the second image region includes:

[0047] Using the pixel point data in the target image block to overwrite the corresponding pixel point data in the second type of sub-image region in the second image region to restore the second image region.

[0048] An embodiment of this specification also provides an image restoration device, including:

[0049] A first determination module, configured to determine a damaged boundary line between a normal image area and a damaged image area in a current target image; wherein, the damaged image area is an image area to be restored;

[0050] An acquisition module, configured to acquire a plurality of first pixel points on the damaged boundary line; and based on the plurality of first pixel points, determine a plurality of first image areas; wherein, each of the first image areas includes at least one first pixel point; the first image area includes a first type of sub-image area that does not need to be restored and a second type of sub-image area that needs to be restored;

[0051] A screening module, configured to screen out a second image area to be currently restored from the plurality of first image areas;

[0052] A second determination module, configured to determine a target range parameter matching the second image area based on a preset Gaussian function model;

[0053] A retrieval module, configured to retrieve the normal image area in the current target image according to the target range parameter to determine a target image block matching the second image area;

[0054] A restoration module, configured to use the target image block to perform a restoration process on the second image area.

[0055] An embodiment of this specification also provides a server, including a processor and a memory for storing processor-executable instructions, and when the processor executes the instructions, relevant steps of the image restoration method are implemented.

[0056] An embodiment of this specification also provides a computer storage medium, on which computer instructions are stored, and when the instructions are executed, relevant steps of the image restoration method are implemented.

[0057] An image restoration method, apparatus, and server provided in this specification, after determining a plurality of first image regions based on first pixel points on a damaged boundary in a current target image to be restored, first screen out one of the first image regions from the plurality of first image regions as a second image region to be currently restored; then, instead of using fixed range parameters, based on a preset Gaussian function model, and using the average gradient modulus value of the pixel grayscale values in a first type of sub-image region in the second image region that can more effectively reflect the change characteristics of texture information, determine a target range parameter that matches the second image region; then, according to this target range parameter, by retrieving a normal image region in the target image, determine a target image block with a relatively appropriate range size for this second image region; furthermore, the target image block can be used to perform corresponding restoration processing on the second image region. Thus, it is possible to better balance both the restoration efficiency and the restoration effect, and for second image regions with different texture information change characteristics, selectively and differently use target image blocks with different target range parameters to perform targeted restoration processing efficiently and accurately. Furthermore, it can effectively avoid the poor restoration accuracy of regions with large changes in some texture information in the target image and the excessive consumption of processing time and processing resources for regions with small changes in some texture information when using fixed range parameters to determine the target image blocks for restoration processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] To more clearly illustrate the embodiments of this specification, the following will briefly introduce the drawings required for use in the embodiments. The drawings described below are only some embodiments recorded in this specification. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0059] Figure 1 is a schematic diagram of an embodiment of the structural composition of a system applying the image restoration method provided in the embodiments of this specification;

[0060] Figure 2 is a schematic diagram of an embodiment of applying the image restoration method provided in the embodiments of this specification in a scenario example;

[0061] Figure 3 is a schematic diagram of an embodiment of applying the image restoration method provided in the embodiments of this specification in a scenario example;

[0062] Figure 4 is a schematic diagram of an embodiment of applying the image restoration method provided in the embodiments of this specification in a scenario example;

[0063] Figure 5It is a schematic diagram of an embodiment applying the image restoration method provided in the embodiments of this specification in a scenario example;

[0064] Figure 6 It is a schematic flowchart of the image restoration method provided in an embodiment of this specification;

[0065] Figure 7 It is a schematic diagram of an embodiment applying the image restoration method provided in the embodiments of this specification in a scenario example;

[0066] Figure 8 It is a schematic diagram of an embodiment applying the image restoration method provided in the embodiments of this specification in a scenario example;

[0067] Figure 9 It is a schematic diagram of the structural composition of a server provided in an embodiment of this specification;

[0068] Figure 10 It is a schematic diagram of the structural composition of an image restoration device provided in an embodiment of this specification. Detailed implementation manners

[0069] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of this specification, rather than all the embodiments. Based on the embodiments in this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this specification.

[0070] Considering existing image restoration methods, for example, traditional Criminisi restoration method, etc., for the image regions to be restored in the image to be restored in different situations, most do not make distinctions, use a sample window of a fixed size (i.e., a range parameter), such as a 9X9 window, to find matching image patches in the normal image regions of the image to be restored for restoration.

[0071] However, during the actual repair based on the above method, it lacks flexibility and has poor pertinence, and cannot well balance the repair effect and repair efficiency at the same time. Specifically, for example, for some regions with large changes in texture information (such as rich and complex texture structures and high frequencies of texture color changes), the sample window used is relatively large, resulting in poor repair accuracy and easy occurrence of blocking effects. For example, after repair, visible breaks in the extension of the texture structure are likely to appear in the image. On the other hand, for some regions with small changes in texture information (such as simple and single texture structures and low frequencies of texture color changes), the sample window used is relatively small, leading to excessive consumption of processing resources and processing time during the repair process, affecting the overall repair efficiency of the image.

[0072] To address the root causes of the above problems, this specification considers screening out an image region from multiple first image regions on the damaged boundary as the second image region to be repaired currently. For this second image region, instead of using fixed range parameters to set the sample window as in the existing method, the average gradient modulus value of pixel points that can effectively reflect the transformation characteristics of the surrounding neighborhood texture information is used, and based on a preset Gaussian function model, the target range parameters matching the texture change situation of this second image region are determined; then according to these target range parameters, by retrieving the normal image region, a target image block with a relatively appropriate size is determined; and then this target image block can be used to perform repair processing on the second image region.

[0073] Thus, it can better balance the repair efficiency and repair effect at the same time. For second image regions with different texture information change characteristics, different target range parameters are used discriminately to perform targeted repair processing efficiently and accurately, improving the overall repair efficiency of the image.

[0074] An embodiment of this specification provides an image repair method, which can be specifically applied to a system including a server and a terminal device. Specifically, reference can be made to Figure 1 as shown. The above server and terminal device can be connected by wired or wireless means for specific data interaction.

[0075] In this embodiment, the server can specifically include a background server applied to one side of the network platform, which can implement functions such as data transmission and data processing. Specifically, the server can be, for example, an electronic device with data operation, storage, and network interaction functions. Or, the server can also be a software program running in this electronic device, providing support for data processing, storage, and network interaction. In this embodiment, the number of servers is not specifically limited. The server can specifically be one server, or several servers, or a server cluster formed by several servers.

[0076] In this embodiment, the terminal device may specifically include a front-end device applied to the user side that can implement functions such as data collection and data transmission. Specifically, the terminal device may be, for example, a desktop computer, a tablet computer, a laptop computer, a smart phone, etc. Alternatively, the terminal device may also be a software application that can run on the above-mentioned electronic devices. For example, it may be a certain picture repair APP running on a smart phone, etc.

[0077] Specifically, the terminal device may first display the target image to be repaired to the user. Refer to Figure 2 As shown, the user can perform corresponding operations on the terminal device to mark the image area to be repaired (which can be recorded as the damaged image area) on the displayed target image with a preset marking color (for example, green), so as to obtain the calibrated target image. Among them, other image areas in the target image except the damaged image area can be understood as image areas that do not need to be repaired, which are recorded as normal image areas.

[0078] Next, the user can send the calibrated target image to the server responsible for image repair through the terminal device. Correspondingly, the server receives and obtains the target image to be repaired.

[0079] The server can first detect the preset marking color in the target image to identify the normal image area and the damaged image in the current target image; then determine the boundary line between the normal image area and the damaged image area in the current target image to determine the damaged boundary line for the current target image.

[0080] Further, the server can determine a plurality of pixel points on the damaged boundary, which are recorded as the first pixel points. At the same time, the server can calculate the confidence levels of each first pixel point on the damaged boundary line and each pixel point in the neighborhood around the damaged boundary line based on the current target image for subsequent use.

[0081] Then, for each first pixel point, the server can determine the image area for the first pixel point in combination with other pixel points in the neighborhood around the first pixel point, that is, the first image area. In this way, a plurality of first image areas can be determined, where each first image area contains at least one first pixel point. And each first image area specifically contains a first type of sub-image area that does not need to be repaired (belonging to the normal image area) and a second type of sub-image area that needs to be repaired (belonging to the damaged image area). Refer to Figure 3 As shown.

[0082] Further, the server can calculate the priority parameters of each first image region respectively by using the confidence of relevant pixel points and texture features, etc., according to the preset screening rules. Among them, the above-mentioned priority parameters are used to characterize the importance of the corresponding first image region for the current target image repair. Then, according to the priority parameters, a first image region with the largest priority parameter is selected from multiple first image regions as the second image region to be repaired currently.

[0083] Furthermore, the server can calculate the average gradient modulus of the gray values of the first type of sub-image regions in the second image region that can effectively reflect the texture change characteristics, and the average gradient modulus of the gray values of the first type of sub-image regions of multiple first image regions; then, using the above-mentioned average gradient modulus of the gray values, based on the preset Gaussian function model, determine the target range parameters that match the texture change situation of the second image region.

[0084] Next, the server can, according to the target range parameters, use a sample window of the corresponding size to retrieve in the normal image region of the current target image, and find multiple pending image blocks whose range parameters are equal to the target range parameters; then calculate the approximation parameters (such as SSD values, etc.) between the second image region and each pending image block respectively, and screen out the pending image block with the highest approximation degree to the second image region from multiple pending image blocks as the matching target image block. Thus, a target image block with good effect and appropriate range size matching for the second image region can be found. Refer to Figure 4 shown.

[0085] Then, the server can use the pixel point data in the target image block to cover the corresponding pixel point data in the second type of sub-image regions in the second image region to perform targeted repair processing on the second image region. Refer to Figure 4 shown.

[0086] Thus, the repair processing of the second image region of the current target image is completed. At this time, the server can update based on the current repaired target image to obtain an updated target image. And by detecting whether there are still damaged boundary lines in the updated target image, it is determined whether the repair of the target image is completed.

[0087] In the case where it is determined that there are no damaged boundary lines in the updated target image, it can be determined that there are no damaged image regions to be repaired in the target image, and further it can be determined that the repair of the target image is ended. At this time, the updated target image can be used as the repaired target image (i.e., the final result image). Refer to Figure 5 shown. The damaged image regions in the target image have been repaired well.

[0088] Furthermore, the server can send the repaired target image to the terminal device. The terminal device receives and displays the repaired target image to the user.

[0089] In the case where it is determined that there are still damaged boundary lines in the updated target image, it can be determined that there are still damaged image areas in the target image that need to be repaired, and further it can be determined that the repair of the target image is not completed. At this time, the server can first update the damaged boundary lines, as well as the confidence levels of the pixel points on the damaged boundary lines and the pixel points in the neighborhood around the damaged boundary lines; then, based on the updated damaged boundary lines and the updated confidence levels of the relevant pixel points, re-determine the second image area in the above manner, and perform repair processing on the second image area. Repeat the above process multiple times until it is detected and determined that there are no damaged boundary lines in the updated target image, and then determine that the repair of the target image is completed.

[0090] Through the above embodiments, it is possible to better balance the repair efficiency and the repair effect, and use different target range parameters in a differentiated manner for the second image areas with different texture information change characteristics to perform targeted repair processing efficiently and accurately.

[0091] Refer to Figure 6 As shown, the embodiments of this specification provide an image repair method. Among them, this method can be specifically applied to the server side. Specifically, this method may include the following content:

[0092] S601: Determine the damaged boundary line between the normal image area and the damaged image area in the current target image; where the damaged image area is the image area to be repaired;

[0093] S602: Obtain a plurality of first pixel points on the damaged boundary line; and based on the plurality of first pixel points, determine a plurality of first image areas; where each of the first image areas contains at least one first pixel point; the first image area contains a first type of sub-image area that does not need to be repaired and a second type of sub-image area that needs to be repaired;

[0094] S603: Screen out the second image area to be repaired currently from the plurality of first image areas;

[0095] S604: Based on a preset Gaussian function model, determine the target range parameter that matches the second image area;

[0096] S605: According to the target range parameter, retrieve the normal image area in the current target image to determine the target image block that matches the second image area;

[0097] S606: Use the target image block to perform restoration processing on the second image region.

[0098] Through the above embodiments, when specifically restoring the current target image, the second image region to be currently restored can be first screened out from multiple first image regions regarding the damaged boundary line, and the above second image region is determined as the restoration object for the current time; when specifically restoring the second image region, by using the average gradient modulus value of the pixel gray values that can effectively reflect the change characteristics of the texture information, and based on the preset Gaussian function model, the target range parameter matching the second image region is flexibly determined; then according to the above target range parameter, a target image block with a relatively appropriate range size for the second image region is determined; furthermore, the second image region can be restored by using the target image block. Thus, it is possible to better balance the restoration efficiency and restoration effect for the target image, and for the second image regions with different texture information change characteristics, selectively choose and use target image blocks based on different target range parameters to perform targeted restoration processing efficiently and accurately.

[0099] In some embodiments, the above target image can specifically be the image to be restored. Specifically, the above target image can be an image with partial data loss during image compression or network transmission, or an old photo with damage or an image obtained by digitizing ancient cultural relic calligraphy and paintings, or an image that needs to remove or hide some target objects due to some special requirements (for example, the wire in a movie film), etc. Of course, the above-listed target images are only illustrative. In specific implementation, according to the specific application scenario and processing requirements, the above target image can also include other types of images. Regarding this, this specification does not make a limitation.

[0100] In some embodiments, the above target image is specifically still an image in which the damaged image region is pre-marked through calibration. Among them, the above damaged image region (which can be denoted as Ω) can be specifically understood as the image region in the target image that needs to be restored.

[0101] In the target image, the image region other than the damaged image region can be denoted as the normal image region. Correspondingly, the above normal image region (which can be denoted as Φ) can be specifically understood as the image region in the target image that does not need to be restored.

[0102] In some embodiments, the above target image can specifically be obtained by the user manually marking the damaged image region in the target image with a preset marking color. It can also be obtained by the server automatically identifying and marking the damaged image region by processing the target image by calling a pre-trained damage recognition model.

[0103] In some embodiments, the above-mentioned current target image may specifically be an image that has not been repaired using the image repair method provided in this specification. It may also be an image that has been repaired one or more times using the image repair method provided in this specification and still has damaged image areas to be repaired.

[0104] In some embodiments, before specific implementation, it may be possible to first detect whether there is still a damaged boundary line (which may be denoted as δΩ) in the current target image. Among them, the above-mentioned damaged boundary line can be specifically understood as a boundary line located between the normal image area and the damaged image in the current target image. Further, there may be multiple pixel points on the above-mentioned damaged boundary line, which may be denoted as the first pixel points. The above-mentioned first pixel points (for example, pixel point P) may be pixel points in the normal image area. Reference may be made to Figure 3 as shown.

[0105] In some embodiments, in the case where it is detected that there is no damaged boundary line in the current target image, it can be determined that there are no longer damaged image areas to be repaired in the current target image. Furthermore, it can be determined that the image repair is completed, and the current target image is determined as the result image of the completed repair.

[0106] In the case where it is detected that there is still a damaged boundary line in the current target image, it can be determined that there are still damaged image areas to be repaired in the current target image. Furthermore, it can trigger the application of the image repair method provided in this specification to perform the current image repair on the current target image.

[0107] In some embodiments, when it is detected that there is still a damaged boundary line in the current target image, the damaged boundary line can be identified and determined in the current target image. Further, multiple first pixel points on the damaged boundary line can be determined and obtained; at the same time, the confidence levels of multiple first pixel points on the damaged boundary line and multiple pixel points in the surrounding neighborhood outside the damaged boundary line can also be calculated and obtained.

[0108] In some embodiments, when specifically determining multiple first image areas based on the multiple first pixel points, taking the determination of the current first image area corresponding to the current first pixel point among the multiple first pixel points as an example, in specific implementation, it may include: in the current target image, with the current first pixel point as the center, based on a preset range parameter, determine an image area within a corresponding range as the current first image area. Specifically, for example, reference may be made to Figure 3 as shown, a 9X9 square image area can be determined with the current first pixel point as the center based on a preset range parameter as the current first image area. In the above manner, the respective first image areas corresponding to the respective first pixel points can be determined. And each first image area contains at least one first pixel point.

[0109] In some embodiments, each first image region (which can be denoted as Ψ) may further include a first type of sub-image region (which can be denoted as Ψ Φ ) that belongs to the normal image region and does not need to be repaired, and a second type of sub-image region (which can be denoted as Ψ Ω ) that belongs to the damaged image region and needs to be repaired.

[0110] In some embodiments, during specific implementation, the priority parameter of each first image region can be calculated and based on it, one first image region with the highest importance for the current target image repair can be selected from multiple first image regions as the second image region to be repaired currently (which can be denoted as Ψ p ) to trigger the targeted repair process for the second image region in the current iteration.

[0111] In some embodiments, the above-mentioned selection of the second image region to be repaired currently from multiple first image regions may specifically include the following contents during specific implementation:

[0112] S1: Obtain and calculate the first type of weight parameter based on confidence of each first image region according to the pixel point confidence of the first type of sub-image region in each first image region;

[0113] S2: Obtain and calculate the second type of weight parameter based on texture feature of each first image region according to the texture feature of the first type of sub-image region in each first image region;

[0114] S3: Calculate the priority parameter of each first image region according to the first type of weight parameter and the second type of weight parameter of each first image region; wherein, the priority parameter is used to represent the importance of the corresponding first image region for the current target image repair;

[0115] S4: Select the first image region with the largest priority parameter as the second image region.

[0116] Through the above embodiments, for the current target image, considering two factors of confidence and texture feature in the image, the second image region that relatively more needs to be repaired currently can be accurately selected from multiple first image regions.

[0117] In some embodiments, the above-mentioned calculation of the first type of weight parameter based on confidence of each first image region according to the pixel point confidence of the first type of sub-image region in each first image region may specifically include:

[0118] Calculate the first type of weight parameter based on confidence of the first image region according to the following formula:

[0119]

[0120] Among them, C(p) is the first type of weight parameter of the first image region labeled p, q is any pixel point in the first type of sub-image region (Ψ Φ ) in this first image region, C(q) is the confidence of pixel point q, and Num(Ψ) is the total number of pixel points included in this first image region. Among them, p is the label of the first image region, and specifically, it can be represented by the number of the first pixel point in this first image region.

[0121] In some embodiments, before specific implementation, the confidence of pixel points in the target image (or pixel points on the damaged boundary line and in the neighborhood around the damaged boundary line) can be initialized first.

[0122] Specifically, the confidence can be initialized according to the following formula:

[0123]

[0124] Among them, i is the number of the pixel point, and C(i) is the initialized confidence of the pixel point numbered i.

[0125] In some embodiments, the texture feature of the above-mentioned first type of sub-image region can specifically include the product of the unit normal vector n p and the isophote vector at the first pixel point p in the first type of sub-image region.

[0126] Specifically, as shown in Figure 7 , the above-mentioned unit normal vector can specifically refer to a unit vector starting from the first pixel point p in the first image region, perpendicular to the tangent of the damaged boundary line, and with the direction from the normal image region to the damaged image region.

[0127] The above-mentioned isophote vector can specifically refer to a unit vector starting from the first pixel point p in the first image region, parallel to the extension direction of the texture structure in the first type of sub-image region, and with the direction from the normal image region to the damaged image region.

[0128] In some embodiments, the second type of weight parameter of the first image region based on the texture feature can be calculated according to the following formula:

[0129]

[0130] Among them, D(p) is the second type of weight parameter of the first image region labeled p, is the isophote vector in the first image region, n pis the unit normal vector in the first image region, α is the normalization parameter, and ε is a very small positive constant. Here, ε is used to prevent D(p) from having a zero value.

[0131] Based on the above second type of weight parameter, when the unit normal vector n in the first image region p and the isophote vector The smaller the angle between them, correspondingly, the larger D(p) will be. That is, when the extension direction of the texture structure in the first image region at the damaged boundary is perpendicular to the tangent of the damaged boundary (or parallel to the normal vector), this value is the largest; relatively, more repair is needed, and the effect on the current target image after repair is also relatively greater.

[0132] In some embodiments, calculating the priority parameter of each first image region according to the first type of weight parameter and the second type of weight parameter of each first image region, specifically in implementation, may include:

[0133] Calculate the priority parameter of the current first image region according to the following formula:

[0134] P(p) = aC(p) + bD(p)

[0135] Where, P(p) is the priority parameter of the current first image region, a is the first type of weight coefficient, b is the second type of weight coefficient, C(p) is the first type of weight parameter, and D(p) is the second type of weight parameter.

[0136] Among them, the priority parameter is used to represent the importance of the corresponding first image region for repairing the current target image. Usually, the larger the priority parameter of the first image region, it indicates that the repair of this first image region is more important for the current target image.

[0137] In some embodiments, further considering that in the specific image repair process, due to the first type of weight parameter C(p) based on confidence and the second type of weight parameter D(p) based on texture features showing different change trends in numerical changes. Specifically, in the later stage of image repair, the values of the first type of weight parameter based on confidence between the first image regions gradually approach zero, resulting in the numerical difference of the first type of weight parameter between the first image regions becoming smaller. At this time, when calculating the priority parameter, the influence of the first type of weight parameter will be weakened, and the value of the priority parameter basically depends on the second type of weight parameter. This will lead to a poor effect of the calculated priority parameter and low reference value. Based on the above considerations, a regularization factor can be introduced to optimize and improve the formula for calculating the priority parameter of the first image region, so as to more accurately calculate the priority parameter of each first image region.

[0138] In some embodiments, according to the first type of weight parameters and the second type of weight parameters of each first image region, calculate the priority parameters of each first image region. Specifically, the implementation may include the following:

[0139] Calculate the priority parameter of the current first image region according to the following formula:

[0140] P(p) = aC′(p) + bD(p) = α(ω + (1 - ω)×C(p)) + βD(p)

[0141] where P(p) is the priority parameter of the current first image region, α is the first weight coefficient, β is the second weight coefficient, ω is the normalization factor of the confidence level, C(p) is the first type of weight parameter, and D(p) is the second type of weight parameter.

[0142] Based on the above optimized formula, the normalization factor can be used to normalize C(p) first, so that the value of the normalized C′(p) is within the range of [ω, 1], and at the same time, its curve shape can be well maintained, avoiding the minimum value of the confidence level C(p). Thus, a more accurate and better-performing priority parameter can be calculated.

[0143] Refer to Figure 8 As shown, it can be seen that the normalized first type of weight parameter curve based on the confidence level can indeed avoid the minimum value and maintain a good curve shape compared with the first type of weight parameter curve based on the confidence level before normalization.

[0144] In some embodiments, specifically, the priority parameters of each first image region can be calculated in the above manner; then, select the first image region with the relatively largest priority parameter from multiple first image regions as the current second image region to be repaired.

[0145] In some embodiments, based on the preset Gaussian function model, determine the target range parameter that matches the second image region. Specifically, the implementation may include the following:

[0146] S1: Calculate the average gradient modulus of the gray values of the first type of sub-image regions in the second image region, and the average gradient modulus of the gray values of the first type of sub-image regions of multiple first image regions;

[0147] S2: Call the preset Gaussian function model, and calculate the target range parameter according to the average gradient modulus of the gray values of the first type of sub-image regions in the second image region and the average gradient modulus of the gray values of the first type of sub-image regions of multiple first image regions.

[0148] Through the above embodiments, the average gradient modulus of the gray values of the pixel points that can reflect the variation characteristics of the texture information (for example, the variation frequency of the surrounding neighborhood texture colors of the pixel points, etc.) can be calculated and utilized, and combined with the preset Gaussian function model, to determine more targeted and better-effect target range parameters for the specific texture variation situation of the second image region.

[0149] In some embodiments, the above-mentioned preset Gaussian function model is called, and according to the average gradient modulus of the gray values of the first type of sub-image regions in the second image region, and the average gradient modulus of the gray values of the first type of sub-image regions of multiple first image regions, the target range parameters are calculated. Specifically, the following contents may be included during implementation:

[0150] Calculate the target range parameters according to the following formula:

[0151]

[0152] where S is the target range parameter, S min is the preset lower limit value of the target range, S max is the preset upper limit value of the target range, is the maximum value among the average gradient moduli of the gray values of the first type of sub-image regions of multiple first image regions, is the minimum value among the average gradient moduli of the gray values of the first type of sub-image regions of multiple first image regions, is the average gradient modulus of the gray values of the first type of sub-image regions in the second image region, is the floor operation.

[0153] Through the above embodiments, the texture variation characteristics reflected by the average gradient modulus of the gray values of the pixel points can be fully utilized to specifically determine more suitable target range parameters that match the second image region, avoiding using the same numerical target range parameters as in the existing methods without distinguishing the specific texture variation situation.

[0154] In some embodiments, the value of the above S min may specifically be a range containing 3 pixel points, and the value of the above S max may specifically be a range containing 15 pixel points, etc. Of course, the above-listed preset lower limit value and upper limit value of the target range are only illustrative. During specific implementation, according to the specific situation and processing requirements, other appropriate ranges of other sizes may also be used as the preset lower limit value and upper limit value of the target range.

[0155] In some embodiments, the above-mentioned target range parameter is used to define the window size of the sample window used when subsequently searching for a target image block for repairing the second type of sub-image region in the second image region.

[0156] In some embodiments, retrieving a normal image region in the current target image according to the target range parameter to determine a target image block that matches the second image region may specifically include the following content when implemented:

[0157] S1: In the normal image region of the current target image, find multiple pending image blocks whose range parameters are equal to the target range parameter;

[0158] S2: Calculate the approximation parameter between the second image region and the multiple pending image blocks;

[0159] S3: Screen out the pending image blocks whose approximation parameters meet the preset requirements from the multiple pending image blocks as the target image blocks that match the second image region.

[0160] Through the above embodiments, the target range parameter that matches the second image region can be used to find pending image blocks with relatively appropriate range sizes first, avoiding the range sizes of the found pending image blocks being too large or too small; then, by calculating and based on the approximation parameter between the second image region and the above-mentioned pending image blocks, the target image blocks suitable for repairing the second type of sub-image region in the second image region can be accurately and efficiently determined.

[0161] In some embodiments, the above-mentioned approximation parameter may specifically refer to the sum of squared differences (SSD) between gray values.

[0162] Specifically, the above-mentioned SSD can be calculated according to the following formula:

[0163]

[0164] where, Ψ p is the second image region, Ψ q is the current pending image block, and d(Ψ p , Ψ q ) is the sum of the squared differences of the colors of the corresponding pixel points in the second image region and the current pending image block. Among them, the pending image block Ψ q is an image region in the normal image region whose range parameter is equal to the target range parameter.

[0165] In some embodiments, when specifically implementing the calculation of the approximation parameter between the second image region and the multiple to-be-determined image blocks, the following may be included: calculating the approximation parameter between the second image region and the current to-be-determined image block among the multiple to-be-determined image blocks in the following manner;

[0166] S1: Obtain the RGB color component values of the pixel points in the second image region and the RGB color component values of the pixel points in the current to-be-determined image block;

[0167] S2: Calculate the sum of the squared differences in color of the corresponding pixel points between the second image region and the current to-be-determined image block based on the RGB color component values of the pixel points in the second image region and the RGB color component values of the pixel points in the current to-be-determined image block;

[0168] S3: Calculate the approximation parameter between the second image region and the current to-be-determined image block based on the sum of the squared differences in color of the corresponding pixel points between the second image region and the current to-be-determined image block.

[0169] Through the above embodiments, the sum of the squared differences in color of the corresponding pixel points between the second image region and each to-be-determined image block among the multiple to-be-determined image blocks can be accurately calculated and utilized to obtain a better approximation parameter.

[0170] In some embodiments, when specifically implementing the calculation of the sum of the squared differences in color of the corresponding pixel points between the second image region and the current to-be-determined image block based on the RGB color component values of the pixel points in the second image region and the RGB color component values of the pixel points in the current to-be-determined image block, the following may be included:

[0171] Calculate the sum of the squared differences in color of the corresponding pixel points between the second image region and the current to-be-determined image block according to the following formula:

[0172]

[0173] where Ψ p is the second image region, Ψ q is the current to-be-determined image block, R x and R y are the red component values of the corresponding pixel points in the second image region and the current to-be-determined image block respectively, G x and G y are the green component values of the corresponding pixel points in the second image region and the current to-be-determined image block respectively, B x and B y are the blue component values of the corresponding pixel points in the second image region and the current to-be-determined image block respectively, and d(Ψ p ,Ψ q) is the sum of the squares of the differences in the colors of the corresponding pixel points in the second image region and the currently undetermined image block.

[0174] Through the above embodiments, the sum of the squares of the differences in the colors of the corresponding pixel points in the second image region and each undetermined image block can be calculated efficiently and accurately using the above formula.

[0175] In some embodiments, when performing the restoration process on the second image region using the target image block, specifically, it may include the following: using the pixel point data in the target image block to overwrite the corresponding pixel point data in the second type of sub-image region in the second image region to restore the second image region.

[0176] Through the above embodiments, the pixel point data of the target image block can be effectively used to perform targeted restoration processing on the second type of sub-image region in the second image region, thereby completing the current image restoration.

[0177] In some embodiments, specifically, the pixel point data in the target image block can be copied out; then the copied pixel point data in the target image block is copied to the second type of sub-image region in the second image region, thereby achieving the restoration of the second image region. For details, refer to Figure 4 as shown.

[0178] In some embodiments, after performing the restoration process on the second type of sub-image region in the second image region, the method may further include the following when specifically implemented:

[0179] S1: Update the target image to obtain the updated target image;

[0180] S2: Detect whether there are still damaged boundary lines in the updated target image;

[0181] S3: When it is determined that there are no such damaged boundary lines in the updated target image, determine the updated target image as the restored target image.

[0182] In this embodiment, when it is determined that there are no longer damaged boundary lines in the updated target image, it can be determined that there are no longer damaged image regions to be restored in the target image, and it is determined that the restoration of the target image is completed, and the image restoration process is stopped. And the above updated target image is determined as the restored target image that has completed the image restoration.

[0183] In the case where there are still damaged boundary lines in the determined updated target image, it can be determined that there are still damaged image areas to be repaired in the target image, and it can be determined that the repair of the target image has not ended. At this time, the damaged boundary lines, as well as the confidence levels of the pixel points on the damaged boundary lines and the pixel points in the neighborhood around the damaged boundary lines, can be updated; then, based on the updated damaged boundary lines and the updated confidence levels of the relevant pixel points, the above-mentioned method is repeated to perform image repair processing one or more times. Until there are no damaged boundary lines in the updated target image.

[0184] Through the above embodiments, the repair of the target image can be completed by performing multiple image repair processes and update detections on the target image, and a repaired target image with better effects can be obtained. The continuity of the texture structure in the obtained repaired target image is better, and at the same time, the repair of the texture information is relatively ideal, the connection between the texture structures is relatively more natural, and the block effect phenomenon in the image is also significantly weakened.

[0185] As can be seen from the above, based on the image repair method provided in the embodiments of this specification, after determining multiple first image areas based on the first pixel points on the damaged boundary of the current target image to be repaired, first, an image area is selected from the multiple first image areas as the current second image area to be repaired; then, instead of using fixed range parameters, based on a preset Gaussian function model and using the average gradient modulus value of the pixel point gray values that can effectively reflect the change characteristics of the texture information, the target range parameters matching the second image area are determined; then, according to the target range parameters, by retrieving the normal image area in the target image, the target image block with a relatively appropriate range size for the second image area is determined; furthermore, the target image block can be used to perform corresponding repair processing on the second image area. Thus, it is possible to better balance the repair efficiency and the repair effect, and use different target range parameters in a differentiated manner for the second image areas with different texture information change characteristics to perform targeted repair processing efficiently and accurately.

[0186] An embodiment of this specification further provides a server, including a processor and a memory for storing executable instructions of the processor. When specifically implemented, the processor may execute the following steps according to the instructions: determining a damaged boundary line between a normal image area and a damaged image area in a current target image; wherein the damaged image area is an image area to be repaired; obtaining a plurality of first pixel points on the damaged boundary line; and determining a plurality of first image areas based on the plurality of first pixel points; wherein each of the first image areas includes at least one first pixel point; the first image area includes a first type of sub-image area that does not need to be repaired and a second type of sub-image area that needs to be repaired; screening out a second image area to be repaired currently from the plurality of first image areas; determining a target range parameter matching the second image area based on a preset Gaussian function model; retrieving a normal image area in the current target image according to the target range parameter to determine a target image block matching the second image area; and using the target image block to perform a repair process on the second image area.

[0187] To be able to more accurately complete the above instructions, refer to Figure 9 As shown, an embodiment of this specification further provides another specific server. Among them, the server includes a network communication port 901, a processor 902, and a memory 903. The above structures are connected by internal cables so that each structure can perform specific data interactions.

[0188] Among them, the network communication port 901 can specifically be used to receive a target image to be repaired.

[0189] The processor 902 can specifically be used to determine a damaged boundary line between a normal image area and a damaged image area in a current target image; wherein the damaged image area is an image area to be repaired; obtaining a plurality of first pixel points on the damaged boundary line; and determining a plurality of first image areas based on the plurality of first pixel points; wherein each of the first image areas includes at least one first pixel point; the first image area includes a first type of sub-image area that does not need to be repaired and a second type of sub-image area that needs to be repaired; screening out a second image area to be repaired currently from the plurality of first image areas; determining a target range parameter matching the second image area based on a preset Gaussian function model; retrieving a normal image area in the current target image according to the target range parameter to determine a target image block matching the second image area; and using the target image block to perform a repair process on the second image area.

[0190] The memory 903 can specifically be used to store corresponding instruction programs.

[0191] In this embodiment, the network communication port 901 can be bound to different communication protocols, so as to send or receive different data through virtual ports. For example, the network communication port can be a port responsible for web data communication, or a port responsible for FTP data communication, or a port responsible for mail data communication. In addition, the network communication port can also be a physical communication interface or communication chip. For example, it can be a wireless mobile network communication chip, such as GSM, CDMA, etc.; it can also be a Wifi chip; it can also be a Bluetooth chip.

[0192] In this embodiment, the processor 902 can be implemented in any suitable manner. For example, the processor can take the form of, for example, a microprocessor or a processor, and a computer-readable medium storing computer-readable program code (such as software or firmware) executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, etc. This specification does not make any limitations.

[0193] In this embodiment, the memory 903 can include multiple levels. In a digital system, anything that can store binary data can be a memory; in an integrated circuit, a circuit without a physical form but with a storage function is also called a memory, such as RAM, FIFO, etc.; in a system, a storage device with a physical form is also called a memory, such as a memory module, a TF card, etc.

[0194] This embodiment of the specification also provides a computer storage medium based on the above image repair method. The computer storage medium stores computer program instructions, which when executed, implement: determining a damaged boundary line between a normal image area and a damaged image area in the current target image; wherein, the damaged image area is the image area to be repaired; obtaining a plurality of first pixel points on the damaged boundary line; and based on the plurality of first pixel points, determining a plurality of first image areas; wherein, each of the first image areas contains at least one first pixel point; the first image area contains a first type of sub-image area that does not need to be repaired and a second type of sub-image area that needs to be repaired; screening out a second image area to be repaired currently from the plurality of first image areas; determining a target range parameter matching the second image area based on a preset Gaussian function model; according to the target range parameter, retrieving the normal image area in the current target image to determine a target image block matching the second image area; and using the target image block to perform a repair process on the second image area.

[0195] In this embodiment, the above storage medium includes, but is not limited to, a Random Access Memory (RAM), a Read-Only Memory (ROM), a Cache, a Hard Disk Drive (HDD), or a Memory Card. The memory can be used to store computer program instructions. The network communication unit can be set according to the standards specified by the communication protocol and is used as an interface for network connection communication.

[0196] In this embodiment, the functions and effects specifically implemented by the program instructions stored in the computer storage medium can be explained in comparison with other embodiments and will not be elaborated here.

[0197] Refer to Figure 10 As shown, at the software level, an embodiment of this specification also provides an image repair device, which specifically may include the following structural modules:

[0198] The first determination module 1001 can specifically be used to determine the damaged boundary line between the normal image area and the damaged image area in the current target image; wherein, the damaged image area is the image area to be repaired;

[0199] The acquisition module 1002 can specifically be used to acquire a plurality of first pixel points on the damaged boundary line; and based on the plurality of first pixel points, determine a plurality of first image areas; wherein, each of the first image areas contains at least one first pixel point; the first image area contains a first type of sub-image area that does not need to be repaired and a second type of sub-image area that needs to be repaired;

[0200] The screening module 1003 can specifically be used to screen out the second image area to be repaired currently from the plurality of first image areas;

[0201] The second determination module 1004 can specifically be used to determine the target range parameter matching the second image area based on a preset Gaussian function model;

[0202] The retrieval module 1005 can specifically be used to retrieve the normal image area in the current target image according to the target range parameter to determine the target image block matching the second image area;

[0203] The repair module 1006 can specifically be used to use the target image block to perform repair processing on the second image area.

[0204] It should be noted that the units, devices, modules, etc. illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. For the convenience of description, when describing the above devices, they are divided into various modules according to functions and described separately. Of course, when implementing this specification, the functions of each module can be implemented in the same or multiple software and / or hardware, or the modules implementing the same function can be realized by a combination of multiple sub-modules or sub-units, etc. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be electrical, mechanical or other forms.

[0205] As can be seen from the above, based on the image restoration device provided in the embodiments of this specification, the restoration efficiency and restoration effect can be better balanced at the same time. For the second image regions with different texture information change characteristics, different target range parameters are used in a differentiated manner to perform targeted restoration processing efficiently and accurately.

[0206] Although this specification provides method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-creative means. The step order listed in the embodiments is only one of the many step execution orders and does not represent the only execution order. When the actual device or client product executes, it can be executed in the order of the method shown in the embodiments or the drawings or executed in parallel (for example, in an environment of parallel processors or multi-threaded processing, or even in a distributed data processing environment). The term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, product or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, product or device. Without further limitation, it does not exclude the existence of additional identical or equivalent elements in the process, method, product or device including the said elements. The words "first", "second", etc. are used to denote names and do not denote any particular order.

[0207] Those skilled in the art also know that, in addition to implementing the controller in the form of pure computer-readable program code, the method steps can be logically programmed to enable the controller to implement the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Therefore, such a controller can be regarded as a hardware component, and the devices included therein for implementing various functions can also be regarded as the structures within the hardware component. Or even, the devices for implementing various functions can be regarded as either software modules for implementing the method or structures within the hardware component.

[0208] This specification can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform specific tasks or implement specific abstract data types. This specification can also be practiced in a distributed computing environment, where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0209] From the description of the above embodiments, those skilled in the art can clearly understand that this specification can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solutions of this specification can essentially be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, magnetic disk, optical disc, etc., and includes several instructions to enable a computer device (which can be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this specification.

[0210] The various embodiments in this specification are described in a progressive manner. For the same or similar parts between the various embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. This specification can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and so on.

[0211] Although this specification is depicted through embodiments, those of ordinary skill in the art know that this specification has many variations and changes without departing from the spirit of this specification. It is hoped that the appended claims will cover these variations and changes without departing from the spirit of this specification.

Claims

1. An image inpainting method, characterized in that, it includes: Determine the damaged boundary line between the normal image area and the damaged image area in the current target image; wherein, the damaged image area is the image area to be inpainted; Obtain a plurality of first pixel points on the damaged boundary line; and based on the plurality of first pixel points, determine a plurality of first image areas; wherein, each of the first image areas contains at least one first pixel point; the first image area contains a first type of sub-image area that does not need to be inpainted and a second type of sub-image area that needs to be inpainted; Screen out the second image area to be currently inpainted from the plurality of first image areas; Based on a preset Gaussian function model, determine the target range parameter that matches the second image area; According to the target range parameter, retrieve the normal image area in the current target image to determine the target image block that matches the second image area; Use the target image block to perform inpainting processing on the second image area; Wherein, based on a preset Gaussian function model, determining the target range parameter that matches the second image area includes: calculating the average gradient modulus value of the gray value of the first type of sub-image area in the second image area, and the average gradient modulus value of the gray value of the first type of sub-image area of the plurality of first image areas; call the preset Gaussian function model, and according to the average gradient modulus value of the gray value of the first type of sub-image area in the second image area, and the average gradient modulus value of the gray value of the first type of sub-image area of the plurality of first image areas, calculate the target range parameter; Wherein, calling the preset Gaussian function model, and according to the average gradient modulus value of the gray value of the first type of sub-image area in the second image area, and the average gradient modulus value of the gray value of the first type of sub-image area of the plurality of first image areas, calculating the target range parameter includes: Calculate the target range parameter according to the following formula: Among them, S is the target range parameter, S min is the preset lower limit value of the target range, S max is the preset upper limit value of the target range, is the maximum value among the average gradient modulus values of the gray values of the first type of sub-image regions of multiple first image regions, is the minimum value among the average gradient modulus values of the gray values of the first type of sub-image regions of multiple first image regions, is the average gradient modulus value of the gray value of the first type of sub-image region in the second image region, is the floor operation.

2. The method according to claim 1, characterized in that, After performing inpainting processing on the second type of sub-image area in the second image area, the method further includes: Update the target image to obtain an updated target image; Detect whether there is still a damaged boundary line in the updated target image; In the case of determining that there is no such damaged boundary line in the updated target image, determine the updated target image as the inpainted target image.

3. The method according to claim 1, characterized in that, Screening out the second image area to be currently inpainted from the plurality of first image areas includes: Obtain and calculate the first type of weight parameter based on confidence of each first image area according to the pixel point confidence of the first type of sub-image area in each first image area; Obtain and calculate the second type of weight parameter based on texture features of each first image area according to the texture features of the first type of sub-image area in each first image area; Calculate the priority parameter of each first image region according to the first type of weight parameter and the second type of weight parameter of each first image region; wherein, the priority parameter is used to characterize the importance of the corresponding first image region for the current target image restoration. Select the first image region with the largest priority parameter as the second image region.

4. The method according to claim 3, wherein, Calculating the priority parameter of each first image region according to the first type of weight parameter and the second type of weight parameter of each first image region includes: Calculate the priority parameter of the current first image region according to the following formula: P(p) = α(ω+(1 - ω)×C(p))+βD(p) where P(p) is the priority parameter of the current first image region, α is the first weight coefficient, β is the second weight coefficient, ω is the normalization factor of the confidence level, C(p) is the first type of weight parameter, and D(p) is the second type of weight parameter.

5. The method according to claim 1, wherein, Retrieve the normal image region in the current target image according to the target range parameter to determine the target image block that matches the second image region, including: In the normal image region of the current target image, find multiple pending image blocks whose range parameters are equal to the target range parameter; Calculate the approximation parameter between the second image region and the multiple pending image blocks; Select the pending image blocks whose approximation parameters meet the preset requirements from the multiple pending image blocks as the target image blocks that match the second image region.

6. The method according to claim 5, wherein, Calculating the approximation parameter between the second image region and the multiple pending image blocks includes: Calculate the approximation parameter between the second image region and the current pending image block among the multiple pending image blocks in the following manner; Obtain the RGB color component values of the pixel points in the second image region and the RGB color component values of the pixel points in the current pending image block; Calculate the sum of the squared differences of the colors of the corresponding pixel points in the second image region and the current pending image block according to the RGB color component values of the pixel points in the second image region and the RGB color component values of the pixel points in the current pending image block; Calculate the approximation parameter between the second image region and the current pending image block according to the sum of the squared differences of the colors of the corresponding pixel points in the second image region and the current pending image block.

7. The method according to claim 6, wherein, Calculating the sum of the squared differences of the colors of the corresponding pixel points in the second image region and the current pending image block according to the RGB color component values of the pixel points in the second image region and the RGB color component values of the pixel points in the current pending image block includes: Calculate the sum of the squared differences of the colors of the corresponding pixel points in the second image region and the current pending image block according to the following formula: Among them, Ψ p is the second image region, Ψ q is the currently undetermined image block, R x , R y are respectively the red component values of the corresponding pixel points in the second image region and the currently undetermined image block, G x , G y are respectively the green component values of the corresponding pixel points in the second image region and the currently undetermined image block, B x , B y are respectively the blue component values of the corresponding pixel points in the second image region and the currently undetermined image block, d(Ψ p , Ψ q ) is the sum of the squared differences of the colors of the corresponding pixel points in the second image region and the currently undetermined image block.

8. The method according to claim 1, wherein, Use the target image block to perform restoration processing on the second image region, including: Using the pixel data in the target image block to overwrite the corresponding pixel data in the second type of sub-image area in the second image area to repair the second image area.

9. An image repair device, characterized in that, it includes: A first determination module, configured to determine the damaged boundary line between the normal image area and the damaged image area in the current target image; wherein, the damaged image area is the image area to be repaired; An acquisition module, configured to acquire a plurality of first pixel points on the damaged boundary line; and based on the plurality of first pixel points, determine a plurality of first image areas; wherein, each of the first image areas includes at least one first pixel point; the first image area includes a first type of sub-image area that does not need to be repaired and a second type of sub-image area that needs to be repaired; A screening module, configured to screen out the second image area to be repaired currently from the plurality of first image areas; A second determination module, configured to determine the target range parameter matching the second image area based on a preset Gaussian function model; A retrieval module, configured to retrieve the normal image area in the current target image according to the target range parameter to determine the target image block matching the second image area; A repair module, configured to use the target image block to perform a repair process on the second image area; wherein, the second determination module is specifically configured to calculate the average gradient modulus of the gray value of the first type of sub-image area in the second image area, and the average gradient modulus of the gray value of the first type of sub-image area of the plurality of first image areas; call the preset Gaussian function model, and calculate the target range parameter according to the average gradient modulus of the gray value of the first type of sub-image area in the second image area, and the average gradient modulus of the gray value of the first type of sub-image area of the plurality of first image areas; The second determination module is specifically further configured to calculate the target range parameter according to the following formula: Among them, S is the target range parameter, S min is the preset lower limit value of the target range, S max is the preset upper limit value of the target range, is the maximum value among the average gradient modulus values of the gray values of the first-type sub-image regions of multiple first image regions, is the minimum value among the average gradient modulus values of the gray values of the first-type sub-image regions of multiple first image regions, is the average gradient modulus value of the gray value of the first-type sub-image region in the second image region, is the floor operation.

10. A server, characterized in that, it includes a processor and a memory for storing processor-executable instructions, and when the processor executes the instructions, it implements the steps of the method according to any one of claims 1 to 8.

11. A computer storage medium, characterized in that, computer instructions are stored thereon, and when the instructions are executed, the steps of the method according to any one of claims 1 to 8 are implemented.

Images