An image inpainting method and apparatus

Abstract

Embodiments of the present application disclose an image repairing method and device; an embodiment of the present application can acquire a histogram of a to-be-repaired ID photo, then determine a feature value of a left waveform (a waveform where a dark part peak value is located) and a feature value of a right waveform (a waveform where a bright part peak value is located) in the histogram, generate an image recovery transform function and a dynamic pull-up factor according to the feature value of the left waveform and the feature value of the right waveform, and finally repair the to-be-repaired ID photo through the image recovery transform function and recover or enhance the brightness and details of the to-be-repaired ID photo by using the dynamic pull-up factor; the scheme is simple to implement, has strong real-time performance, and has better repairing effect, greatly reduces the probability of waste during certificate making, and is beneficial to improving the success rate of certificate making.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more particularly to an image restoration method and apparatus. Background Technology

[0002] With the development of society and the economy, identity documents such as passports, ID cards, driver's licenses, and medical insurance cards have gradually become an indispensable part of our daily lives. Because of these special purposes, the images used for ID card production have stringent requirements for image quality. For example, ID photos must meet the requirements of the Digital Photographic Technology Standard (GA461-2004), meaning the portrait must be clear, have rich detail, a natural expression, no obvious distortion, and consistent angle, pose, and color. However, in actual shooting, issues such as dust on the lens or lighting often result in blurry or hazy portraits. If these problems are not detected and corrected during the document production process, the produced documents can easily become unusable. Therefore, image restoration techniques for substandard ID photos are essential.

[0003] Existing image restoration technologies mainly include texture pixel-based restoration methods and machine learning-based image restoration algorithms. During research and practice on existing technologies, the inventors of this application discovered that existing texture pixel-based restoration methods are only effective for images with low contrast. However, they easily lead to color distortion in images with relatively normal contrast. Machine learning-based image restoration methods are not only time-consuming but also highly dependent on the quantity and diversity of training samples. However, collecting a sufficient number of diverse real training samples is extremely difficult in reality, often resulting in insufficient training and reducing the effectiveness and robustness of the model for image restoration. Therefore, existing solutions are not very suitable for ID photo restoration. They are not only difficult to implement and have poor real-time performance, but also easily cause color distortion or loss of some detailed features during restoration, resulting in poor restoration effects and significantly impacting the success rate of ID photo production. Summary of the Invention

[0004] The main purpose of this application is to provide an image restoration method and apparatus that can be applied to the image restoration of ID photos. It is not only simple to implement and highly real-time, but also can restore or enhance brightness and details during restoration, greatly improving the restoration effect. This helps to reduce the probability of producing defective ID cards and increase the success rate of ID card production.

[0005] To achieve the above objectives, this application provides an image restoration method, comprising:

[0006] Obtain the histogram of the ID photo to be repaired. The histogram includes a left waveform and a right waveform. The left waveform is the waveform where the peak value of the dark part is located, and the right waveform is the waveform where the peak value of the bright part is located.

[0007] Determine the eigenvalues ​​of the left and right waveforms of the histogram;

[0008] An image restoration transform function and a dynamic stretching factor are generated based on the feature values ​​of the left and right waveforms.

[0009] The image restoration transformation function is used to process the ID photo to be restored to obtain the processed ID photo;

[0010] The brightness and details of the processed ID photo are restored using the dynamic upscaling factor to obtain the restored ID photo.

[0011] Accordingly, embodiments of this application also provide an image restoration apparatus, including:

[0012] The acquisition unit is used to acquire the histogram of the ID photo to be repaired. The histogram includes a left waveform and a right waveform. The left waveform is the waveform where the peak value of the dark part is located, and the right waveform is the waveform where the peak value of the bright part is located.

[0013] A determining unit is used to determine the characteristic values ​​of the left waveform and the right waveform of the histogram;

[0014] The generation unit is used to generate an image restoration transform function and a dynamic stretching factor based on the feature values ​​of the left waveform and the right waveform.

[0015] The processing unit is used to process the ID photo to be repaired using the image restoration transformation function to obtain the processed ID photo;

[0016] The repair unit is used to repair the brightness and details of the processed ID photo using the dynamic upscaling factor to obtain the repaired ID photo.

[0017] After obtaining the ID photo to be repaired, this embodiment can determine the feature values ​​of the left waveform (the waveform containing the peaks in the dark areas) and the right waveform (the waveform containing the peaks in the bright areas) in the histogram of the ID photo. Based on these features, an image restoration transformation function and a dynamic upscaling factor are generated. Then, the ID photo to be repaired is processed through the image restoration transformation function, and the brightness and details are restored or enhanced using the dynamic upscaling factor. Since this scheme mainly relies on the feature values ​​of the left and right waveforms in the histogram rather than the texture pixels of the image, there are no special requirements for the image contrast. Even if an image with relatively normal contrast or even a normal image is repaired, it will not cause color distortion. Moreover, since the dynamic upscaling factor can restore brightness and details, it can also avoid the loss of detail features that may occur in the early repair process, greatly improving the final repair effect. In addition, since this scheme does not require a lot of time to collect a large number of diverse training samples, it not only has good real-time performance but is also simple to implement. In summary, this solution is not only simple to implement and highly real-time, but also has a better repair effect, greatly reducing the probability of producing invalid certificates and improving the success rate of certificate production. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of the image restoration method provided in the embodiments of this application;

[0020] Figure 2 This is an example diagram of the histogram of the ID photo sample in the embodiments of this application;

[0021] Figure 3 This is an example diagram of the histogram of the ID photo to be repaired in the embodiments of this application;

[0022] Figure 4 This is another example diagram of the histogram of the ID photo to be repaired in the embodiments of this application;

[0023] Figure 5 Another flowchart of the image restoration method provided in the embodiments of this application;

[0024] Figure 6 A framework diagram of the image restoration method provided in the embodiments of this application;

[0025] Figure 7This is a schematic diagram of the image restoration device provided in an embodiment of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] This application provides an image restoration method and apparatus, which will be described in detail below.

[0028] This embodiment will be described from the perspective of an image restoration device, which can be integrated into network devices, such as servers or terminals.

[0029] An image restoration method includes: acquiring a histogram of a photo of an ID card to be restored, wherein the histogram includes a left waveform and a right waveform; determining the feature values ​​of the left waveform and the right waveform of the histogram; generating an image restoration transform function and a dynamic upscaling factor based on the feature values ​​of the left waveform and the right waveform; then processing the photo of the ID card to be restored using the image restoration transform function; and finally restoring the brightness and details of the processed photo of the ID card using the dynamic upscaling factor to obtain a restored photo of the ID card.

[0030] like Figure 1 As shown, the specific process of this image restoration method can be as follows;

[0031] 101. Obtain the histogram of the ID photo to be repaired.

[0032] For example, a photo of the ID document to be repaired can be obtained, and then the brightness value of each pixel in the photo can be obtained. A brightness histogram of the photo can be generated based on the obtained brightness values ​​of the pixels. For ease of description, in this embodiment, the brightness histogram is simply referred to as a histogram.

[0033] There are several ways to obtain the brightness value of each pixel in the ID photo to be repaired. For example, it can be done as follows:

[0034] Obtain the red channel value (R, Red), green channel value (G, Green), and blue channel value (B, Blue) of each pixel in the ID photo to be repaired, and select the maximum value among the three as the brightness value of the pixel.

[0035] In other words, the formula for extracting the brightness of a pixel can be as follows:

[0036] bI(x,y)=max(R,G,B);

[0037] Wherein, R, G, and B are the red, green, and blue channels of the ID photo (color image) to be evaluated, (x,y) is the coordinate value of the image pixel in the image coordinate system, max(R,G,B) is the maximum value of R, G, and B of the pixel point (x,y) in the image coordinate system as the brightness value of the pixel point, and bI(x,y) is the brightness image.

[0038] In this histogram, a two-dimensional coordinate system is used. The horizontal axis x represents different levels of pixel brightness, which usually corresponds to values ​​from 0 to 255 from left to right, also known as color levels. The vertical axis y represents the number of pixels v corresponding to different color levels.

[0039] Due to the histogram of a normal ID photo Figure 1 Generally, they have the characteristic of being far apart and having two distinct waveforms on the left and right. Therefore, for the sake of convenience, in the embodiments of this application, the waveform where the dark part peak is located (the hair area of ​​the portrait is darker) is called the left waveform, and the waveform where the bright part peak is located (the skin area of ​​the portrait is brighter) is called the right waveform.

[0040] For example, see Figure 2 and Figure 3 ,from Figure 2 and 3 As can be seen, the histogram has two distinct waveforms, one on the left and one on the right. The waveform on the left is called the left waveform, and the waveform on the right is called the right waveform.

[0041] In other words, the histogram of the obtained ID photo to be repaired Figure 1 It can generally include a left waveform and a right waveform. The left waveform is the waveform where the peak of the dark part is located, and the right waveform is the waveform where the peak of the bright part is located.

[0042] It should be noted that if there is only one obvious waveform in the histogram, such as in the case of a bald head, there may only be one right waveform (the waveform where the bright part peaks). See [link to relevant documentation]. Figure 4 In this case, the left waveform can be considered non-existent or can be ignored. Therefore, it is sufficient to set the parameter value related to the waveform to 0. For example, if there is only one left waveform (the waveform where the peak of the dark part is located), such as a person with dark skin, then the right waveform can be considered non-existent or can be ignored. Therefore, it is sufficient to set the parameter value related to the waveform to 0. And so on.

[0043] 102. Determine the characteristic values ​​of the left and right waveforms of the histogram of the ID photo to be repaired.

[0044] Specifically, the feature values ​​of the left waveform can include the left and right boundary points of the left waveform, and the feature values ​​of the right waveform can include the left and right boundary points of the right waveform. Optionally, the step "determining the feature values ​​of the left and right waveforms of the histogram of the ID photo to be repaired" can include:

[0045] The left and right boundary points of the left waveform of the histogram of the ID photo to be repaired are determined to obtain the characteristic value of the left waveform; and the left and right boundary points of the right waveform of the histogram of the ID photo to be repaired are determined to obtain the characteristic value of the right waveform.

[0046] For example, a first straight line parallel to the horizontal axis can be set in the histogram of the ID photo to be repaired. This first straight line gradually moves upward along the vertical axis until it intersects with the rising and falling edges of the left waveform and the rising and falling edges of the right waveform, thus obtaining an intersection line. The horizontal axis coordinate value corresponding to the intersection point of the intersection line with the falling edge of the left waveform is determined to obtain the left boundary point of the left waveform. The horizontal axis coordinate value corresponding to the intersection point of the intersection line with the falling edge of the left waveform is determined to obtain the right boundary point of the left waveform. The horizontal axis coordinate value corresponding to the intersection point of the intersection line with the rising edge of the right waveform is determined to obtain the left boundary point of the right waveform. The horizontal axis coordinate value corresponding to the intersection point of the intersection line with the falling edge of the right waveform is determined to obtain the right boundary point of the right waveform.

[0047] For example, such as Figure 3 As shown, taking the histogram of the ID photo to be repaired as an example where there are two distinct waveforms, the left waveform is called the left waveform, and the right waveform is called the right waveform. A straight line f = v (the first straight line) is set in the histogram, where v is a threshold variable. Increasing the value of v causes the line f = v to slide upwards along the vertical axis until it intersects with the rising edge, falling edge, rising edge, and falling edge of the left waveform (i.e., until both left and right waveforms are obtained). The value of v at this point is recorded as waveThr, meaning the first straight line is y = waveThr at this point. Figure 3 It can be seen that y = waveThr has an intersection point L1 with the rising edge of the left waveform, an intersection point L2 with the falling edge of the left waveform, an intersection point R1 with the rising edge of the right waveform, and an intersection point R2 with the falling edge of the right waveform. The horizontal axis coordinate value corresponding to L1 is the left boundary point of the left waveform, the horizontal axis coordinate value corresponding to L2 is the right boundary point of the left waveform, the horizontal axis coordinate value corresponding to R1 is the left boundary point of the right waveform, and the horizontal axis coordinate value corresponding to R2 is the right boundary point of the right waveform.

[0048] For example, such as Figure 4As shown, taking the histogram of the ID photo for evaluation as an example, there is a clear waveform. Since this waveform is where the bright peak is located, it can be considered the right waveform. The left waveform does not exist or can be ignored (i.e., considered as 0). Therefore, the left and right endpoints of the left waveform can be considered as 0. Similarly, a straight line f = v (i.e., the first straight line) can be set in the histogram, where v is a threshold variable. Increasing the value of v makes the straight line f = v slide upward along the vertical axis until it intersects both the rising and falling edges of the right waveform (i.e., until a clear waveform is obtained). Record the value of v at this time as waveThr. That is, at this time, the first straight line is y = waveThr. Figure 3 It can be seen that y = waveThr has an intersection point R1 with the rising edge of the right waveform and an intersection point R2 with the falling edge of the right waveform. The horizontal axis coordinate value corresponding to R1 is the left boundary point of the right waveform, and the horizontal axis coordinate value corresponding to R2 is the right boundary point of the right waveform.

[0049] 103. Generate an image restoration transform function and a dynamic upscaling factor based on the eigenvalues ​​of the left and right waveforms; for example, it can be done as follows:

[0050] The variance of the left waveform is calculated based on the left and right boundary points of the left waveform to obtain the left endpoint of the left waveform. The variance of the right waveform is calculated based on the left and right boundary points of the right waveform to obtain the right endpoint of the right waveform. Then, an image restoration transform function is generated based on the left and right endpoints of the left and right waveforms, and a dynamic stretching factor is constructed based on the left and right endpoints of the left and right waveforms.

[0051] Optionally, since the distribution of each waveform in the histogram is close to the normal distribution function, the left endpoint δl of the left waveform δ interval and the right endpoint δr of the right waveform δ interval can be determined according to the 3δ rule (three sigma criterion) in the normal distribution function, where δ is the variance of the normal function.

[0052] Optionally, the step "calculate the variance of the left waveform based on the left boundary point and the right boundary point of the left waveform to obtain the left endpoint of the left waveform" may include: determining the left endpoint δl of the δ interval of the left waveform using the 3δ rule in the normal distribution function based on the left boundary point and the right boundary point of the left waveform. In this embodiment of the invention, the left endpoint δl of the δ interval of the left waveform is simply referred to as the left endpoint of the left waveform.

[0053] Similarly, the step "calculate the variance of the right waveform based on the left boundary point and the right boundary point of the right waveform, and obtain the right endpoint of the right waveform" may include: determining the right endpoint δr of the right waveform δ interval using the 3δ rule in the normal distribution function based on the left boundary point and the right boundary point of the right waveform. In this embodiment of the invention, the right endpoint δr of the right waveform δ interval is simply referred to as the right endpoint of the right waveform.

[0054] The method for determining the left endpoint of the left waveform and the right endpoint of the right waveform using the 3δ rule in the normal distribution function can be as follows:

[0055] According to the 3δ rule in the normal distribution function, although the normal variable in the normal function is (-∞, ∞), its value has a 99.7% probability of falling within the interval (u-3δ, u+3δ), where u is the mean of the normal variable. Therefore, the probability of a value not falling within the interval (u-3δ, u+3δ) can be determined to be (1-0.997). Since there are two waveforms, left and right, the probability wTH of a point on each waveform not falling within the interval (u-3δ, u+3δ) is:

[0056] wTH = (1 - 0.997) / 2 = 0.0015;

[0057] Traverse from the left boundary point of a waveform to its right boundary point. Stop when the cumulative number of pixels equals or for the first time exceeds wTH (i.e., 0.0015) of the area enclosed by the waveform and the y = waveThr line (y is the vertical axis of the histogram) on the histogram (because it is not necessarily exactly equal to 0.0015 of the area enclosed by the waveform and the y = waveThr line). The position value of this point is the left endpoint of the δ interval, which can be expressed by the formula:

[0058]

[0059] Where subHist(i) is the waveform in the histogram, S peak _l represents the sum of the number of pixels on waveform subHist(i) in the histogram when traversing from the left boundary point to the right boundary point of the waveform; lPos represents the left boundary point of waveform subHist(i) in the histogram; rPos represents the right boundary point of waveform subHist(i) in the histogram; tn represents the cumulative sum of pixel counts when traversing the histogram; when the cumulative tn does not meet the condition tn peak_l When *wTH is used, Loc(·) takes the value of i at this time as the left endpoint δl of the δ interval.

[0060] ​Similarly, traversing from the right boundary point of a waveform to its left boundary point, stopping when the cumulative number of pixels equals or exceeds for the first time the area wTH (i.e., 0.0015) enclosed by the waveform and the y = waveThr line on the histogram, the position value of this point is the right endpoint of the δ interval, expressed by the formula:

[0061]

[0062] Where subHist(i) is the waveform in the histogram, S peak _r represents the sum of the number of pixels on waveform subHist(i) in the histogram when traversing from the left boundary point to the right boundary point of the waveform; lPos represents the left boundary point of waveform subHist(i) in the histogram; rPos represents the right boundary point of waveform subHist(i) in the histogram; tn represents the cumulative sum of pixel counts when traversing the histogram; if the cumulative tn does not meet the condition tn peak_r When *wTH is used, Loc(·) takes the value of i at this time as the right endpoint δr of the δ interval.

[0063] In other words, the step "determine the left endpoint of the left waveform using the 3δ rule in the normal distribution function based on the left and right boundary points of the left waveform" can include: in the histogram, traverse from the left boundary point of the left waveform to the right boundary point of the left waveform, stop when the accumulated pixels are equal to 0.0015 of the area enclosed by the left waveform and the intersection line, and take the position value of the currently traversed point as the left endpoint of the left waveform.

[0064] The step "Determine the right endpoint of the right waveform using the 3δ rule in the normal distribution function based on the left and right boundary points of the right waveform" can include: In the histogram, traverse from the right boundary point of the right waveform to the left boundary point of the right waveform, and stop when the accumulated pixels are equal to 0.0015 of the area enclosed by the right waveform and the intersection line, and take the position value of the currently traversed point as the right endpoint of the right waveform.

[0065] After obtaining the left endpoint of the left waveform and the right endpoint of the right waveform, an image restoration transform function can be generated based on the left endpoint of the left waveform and the right endpoint of the right waveform, and a dynamic stretching factor can be constructed based on the left endpoint of the left waveform and the right endpoint of the right waveform.

[0066] For example, the step "generating an image restoration transform function based on the left endpoint of the left waveform and the right endpoint of the right waveform" may specifically include:

[0067] ​A first dynamic factor is constructed based on the left endpoint of the left waveform and the right endpoint of the right waveform. Then, a second dynamic factor is constructed based on the left endpoint of the left waveform, the right endpoint of the right waveform, and the first dynamic factor. Finally, an image restoration transform function is generated based on the first dynamic factor and the second dynamic factor.

[0068] For example, the formula for calculating the image restoration transform function can be as follows:

[0069]

[0070] Wherein, δl is the left endpoint of the left waveform, δr is the right endpoint of the right waveform, I(x,y) is the ID photo to be evaluated with a quality of "grayish image", d(x,y) is the image restored by the image restoration transformation function (i.e., the processed ID photo), e(·) is the natural index, and parameters α and β are dynamic factors of image restoration. The term "dynamic" means that the values ​​of parameters α and β vary depending on the image to be restored (i.e., the ID photo to be evaluated with a quality of "grayish image"). (For different ID photos to be evaluated, the values ​​of δl and δr will also change accordingly, so the values ​​of α and β will also change accordingly.) For ease of description, in this embodiment, α is referred to as the first dynamic factor, and β is referred to as the second dynamic factor.

[0071] For example, the step "constructing a dynamic pull-up factor based on the left endpoint of the left waveform and the right endpoint of the right waveform" may specifically include:

[0072] Calculate the product of the right endpoint and 2 to obtain the first product. Calculate the difference between the right endpoint and the left endpoint to obtain the first difference. Add the ratio between the first product and the first difference to 1 to obtain the dynamic escalation factor.

[0073] For example, the formula for calculating this dynamic boost factor can be as follows:

[0074]

[0075] Where δl is the left endpoint of the left waveform, δr is the right endpoint of the right waveform, and r is the dynamic stretching factor.

[0076] It should be noted that if there is only one obvious waveform in the histogram, such as a bald head, there may only be one right waveform (the waveform where the bright part peaks are located). In this case, the left waveform can be considered non-existent or can be ignored, that is, the values ​​of the left and right endpoints of the left waveform are set to 0. Similarly, if there is only one left waveform (the waveform where the dark part peaks are located), such as a person with dark skin, the right waveform can be considered non-existent or can be ignored, that is, the values ​​of the left and right endpoints of the right waveform are set to 0, and so on.

[0077] 104. The image restoration transformation function is used to process the ID photo to be restored, and the processed ID photo is obtained.

[0078] For example, the ID photo I(x,y) to be repaired can be input into the image restoration transformation function "d(x,y)=I(x,y)·α+β" to obtain the processed ID photo d(x,y).

[0079] Optionally, after obtaining the processed ID photo, if the processed ID photo is an 8-bit portrait bitmap, to prevent pixel values ​​from overflowing, the range of pixel values ​​within the image matrix can be limited to between 0 and 255. For example, pixel values ​​exceeding 255 can be adjusted to 255, and pixel values ​​less than 0 can be set to 0, etc. That is, after the step "processing the ID photo to be repaired using a preset image restoration transformation function to obtain the processed ID photo", the image restoration method may further include:

[0080] The pixel values ​​within the image matrix of the processed ID photo are limited to the range of 0 to 255 to obtain the adjusted ID photo.

[0081] 105. The brightness and details of the processed ID photo are restored using the dynamic upscaling factor to obtain the restored ID photo.

[0082] For example, the processed ID photo d(x,y) can be input into the following formula:

[0083]

[0084] The repaired ID photo outI(x,y) can then be obtained, where r is the dynamic scaling factor.

[0085] Optionally, in order to reduce the amount of computation and image processing time, a mapping table can be generated based on the dynamic stretching factor. In this way, by using the mapping table, the repaired ID photo can be quickly obtained simply by inputting the processed ID photo.

[0086] Optionally, the step "using the dynamic upscaling factor to repair the brightness and details of the processed ID photo to obtain the repaired ID photo" may include: based on the dynamic upscaling factor, establishing a mapping table according to a preset function relationship, and using the mapping table to repair the brightness and details of the repaired ID photo to obtain the repaired ID photo.

[0087] The relationship between this mapping table and the dynamic pull-up factor can be specifically shown in the following formula:

[0088]

[0089] Where δl is the left endpoint of the left waveform, δr is the right endpoint of the right waveform, r is the dynamic stretching factor, and list(i) is the mapping table.

[0090] The repaired ID photo will look like this:

[0091] outI(x,y) = list[d(x,y)]

[0092] Where outI(x,y) is the repaired ID photo, d(x,y) is the processed ID photo, and list(i) is the mapping table.

[0093] By adjusting the dynamic upscaling factor, not only can the brightness and details of the processed ID photo be restored to a large extent, but it can also correct slight color casts in the ID photo, and improve images of normal quality as well.

[0094] It should be noted that if, in step 104, the pixel value range within the image matrix of the processed ID photo has been limited to between 0 and 255 to obtain the adjusted ID photo, then the step "using a preset dynamic upscaling factor to repair the brightness and details of the processed ID photo to obtain the repaired ID photo" specifically means: using a preset dynamic upscaling factor to repair the brightness and details of the adjusted ID photo to obtain the repaired ID photo.

[0095] The method of using a preset dynamic upscaling factor to restore the brightness and details of the adjusted ID photo is the same as the method of restoring the processed ID photo, and will not be described in detail here.

[0096] Optionally, in step 101, the ID photo to be repaired can be provided by the user, or the image repair device can evaluate the ID photo and automatically trigger the repair process after selecting the ID photo whose evaluation result is "grayish image". Alternatively, before the step "obtain the histogram of the ID photo to be repaired", the image repair method may further include:

[0097] Obtain the ID photo to be evaluated and its brightness image. Based on the brightness image, perform an image quality assessment on the ID photo. If the assessment result is a gray image, then the ID photo to be evaluated is designated as the ID photo to be repaired.

[0098] There are several ways to obtain the brightness image of the ID photo to be evaluated. For example, the brightness value of each pixel in the ID photo to be evaluated can be obtained separately, and then the brightness image of the ID photo to be evaluated can be generated based on the obtained pixel brightness values.

[0099] The process of obtaining the brightness value of each pixel in the ID photo to be evaluated is similar to that of the ID photo to be repaired. That is, the red channel value, green channel value and blue channel value of each pixel in the ID photo to be evaluated can be obtained. The maximum value among the three values ​​is selected as the brightness value of the pixel. See the previous description for details, which will not be repeated here.

[0100] Optionally, there are various ways to evaluate the image quality of the ID photo based on its brightness image. For example, the following methods can be used, as detailed in steps S1 to S7:

[0101] S1. Segment the brightness image of the ID photo to be evaluated into a brightness map of the hair area and a brightness map of the skin area, and then perform step S2.

[0102] For example, preset hair segmentation thresholds, high skin segmentation thresholds, and low skin segmentation thresholds can be obtained. Then, on one hand, regions in the brightness image of the ID photo to be evaluated whose brightness is greater than the hair segmentation threshold are identified as non-hair regions. The brightness values ​​of these non-hair regions in the brightness image are adjusted to 0, resulting in a brightness map of the hair region. On the other hand, regions in the brightness image of the ID photo to be evaluated whose brightness is less than or equal to the high skin segmentation threshold and greater than the low skin segmentation threshold are identified as skin regions. The brightness values ​​of all regions in the brightness image except for these skin regions are adjusted to 0, resulting in a brightness map of the skin region. This can be expressed by the following formula:

[0103]

[0104]

[0105] Where bI(x,y) is the brightness image of the ID photo to be evaluated, hBI(x,y) is the brightness map of the hair region, sBI(x,y) is the brightness map of the skin region, hSegTH is the hair segmentation threshold, sSegTHl is the low threshold for skin segmentation, and sSegTHh is the high threshold for skin segmentation.

[0106] Since the brightness map of the hair region is the same size as the brightness image bI(x,y) in this embodiment, the brightness map of the hair region can be obtained simply by setting the brightness value of the non-hair region to 0. Similarly, since the brightness map of the skin region is the same size as the brightness image bI(x,y), the brightness map of the skin region can be obtained simply by setting the brightness value of the non-skin region to 0.

[0107] The hair segmentation threshold, high skin segmentation threshold, and low skin segmentation threshold can be preset according to the needs of the actual application. This is because, during the imaging process, light reflects and scatters differently on different object surfaces. In other words, the emission and scattering of light on human hair and skin are different, resulting in significant differences in their brightness components in the image. Therefore, the hair region and the skin region can be distinguished based on the average brightness value. Specifically, multiple high-quality ID photo samples can be collected, and then the average brightness values ​​of these ID photo samples in these two regions can be statistically analyzed. Based on these average brightness values, various segmentation thresholds can be set to segment the portrait in the ID photo into skin and hair regions. That is, before the step "obtaining the preset hair segmentation threshold, high skin segmentation threshold, and low skin segmentation threshold", this image restoration method may also include:

[0108] Multiple ID photo samples are collected, including positive samples, which are normal images. The average brightness of the hair region and the average brightness of the skin region of the multiple ID photo samples are obtained. The average brightness of the hair region of the multiple ID photo samples is set as the hair segmentation threshold and the low threshold for skin segmentation, and the average brightness of the skin region of the multiple ID photo samples is set as the high threshold for skin segmentation.

[0109] As mentioned above, due to the histogram of a normal ID photo... Figure 1 Generally, these patterns exhibit a relatively large distance between them and show two distinct waveforms on the left and right sides. Therefore, this characteristic can be used to obtain the average brightness of the hair and skin areas. For example, it can be done as follows:

[0110] Obtain the histogram of the ID photo sample, which includes a left waveform and a right waveform. The left waveform represents the peak value in the dark area, and the right waveform represents the peak value in the bright area. Determine the right boundary point of the left waveform, the left boundary point of the right waveform, and the right boundary point of the right waveform. Calculate the mean of the sum of the right boundary points of the left waveform and the left boundary points of the right waveform to obtain the mean brightness of the multiple ID photo samples in the hair area. Calculate the mean of the right boundary point of the right waveform to obtain the mean brightness of the multiple ID photo samples in the skin area.

[0111] In other words, the hair segmentation threshold, the high skin segmentation threshold, and the low skin segmentation threshold can be expressed by the following formula:

[0112]

[0113] Where hSegTH is the hair segmentation threshold, sSegTHl is the low threshold for skin color segmentation, sSegTHh is the high threshold for skin color segmentation, lpeak_r is the right boundary point of the left waveform on the histogram of the ID photo sample, rpeak_l is the left boundary point of the right waveform on the histogram of the ID photo sample, and rpeak_r is the right boundary point of the right waveform on the histogram of the ID photo sample.

[0114] Optionally, the left and right boundary points of the left and right waveforms can be determined in the following way:

[0115] In this histogram, a first straight line parallel to the horizontal axis is set. This first straight line gradually moves upward along the vertical axis until it intersects with the falling edge of the left waveform, the rising edge of the right waveform, and the falling edge of the right waveform, thus obtaining the intersection line. The horizontal coordinate value corresponding to the intersection point of the intersection line with the rising edge of the left waveform is determined to obtain the left boundary point of the left waveform. The horizontal coordinate value corresponding to the intersection point of the intersection line with the falling edge of the left waveform is determined to obtain the right boundary point of the left waveform. The horizontal coordinate value corresponding to the intersection point of the intersection line with the rising edge of the right waveform is determined to obtain the left boundary point of the right waveform. The horizontal coordinate value corresponding to the intersection point of the intersection line with the falling edge of the right waveform is determined to obtain the right boundary point of the right waveform.

[0116] For example, with Figure 2 Taking the histogram shown as an example, specifically, a straight line f = v (i.e., the first straight line) can be set in the histogram, where v is a threshold variable. Increasing the value of v causes the line f = v to slide upwards along the vertical axis until it intersects with the falling edge of the left waveform, the rising edge of the right waveform, and the falling edge of the right waveform (i.e., until both left and right waveforms are obtained). Record the value of v at this time as histTh. In other words, at this time, the line is f = histTh (i.e., the intersection line mentioned above). Figure 2 It can be seen that f = histTh has an intersection point L1 with the rising edge of the left waveform, an intersection point L2 with the falling edge of the left waveform, an intersection point R1 with the rising edge of the right waveform, and an intersection point R2 with the falling edge of the right waveform. The x-coordinate value corresponding to L2 is the right boundary point of the left waveform, the x-coordinate value corresponding to R1 is the left boundary point of the right waveform, and the x-coordinate value corresponding to R2 is the right boundary point of the right waveform.

[0117] It should be noted that if the brightness image of the ID photo to be evaluated cannot be segmented into a brightness map of the hair area and a brightness map of the skin area, then step S6 is executed.

[0118] S2. Count the number of pixels with non-zero brightness values ​​in the hair region according to the brightness map of the hair region, and calculate the proportion of the number of pixels with non-zero brightness values ​​in the brightness image. If the proportion is greater than or equal to the preset empirical value, then execute S3; otherwise, if the proportion is less than the preset empirical value, then execute S6.

[0119] This percentage is expressed by the formula:

[0120]

[0121]

[0122] Where bI(x,y) is the brightness image of the ID photo to be evaluated, (x,y) is the coordinate value of the image pixel in the image coordinate system, noZeroNum is the number of pixels with non-zero brightness values ​​in the hair region, H and W are the height and width of the brightness image, respectively, and numRatio is the proportion of the number of pixels with non-zero brightness values ​​in the hair region in the brightness image.

[0123] Optionally, this empirical value can be set based on requirements, experiments, or practical experience. For example, by collecting multiple ID photo samples, such as positive samples, and then by statistically analyzing the proportion of pixels with non-zero brightness values ​​in the hair area of ​​these positive samples in their brightness images, and calculating the average of these proportions, the value can be set based on this average, and so on. This will not be elaborated on here.

[0124] S3. Calculate the average brightness of the hair area based on the brightness map of the hair area in the ID photo to be evaluated, and calculate the average brightness of the skin area based on the brightness map of the skin area in the ID photo to be evaluated, and then execute S4.

[0125] For example, you can obtain the dimensions of the brightness map of the hair region, such as its length and width, and then calculate the average brightness of the hair region based on the brightness map and dimensions. Similarly, you can obtain the dimensions of the brightness map of the skin region, such as its length and width, and then calculate the average brightness of the skin region based on the brightness map and dimensions, and so on.

[0126] For example, the average brightness can be calculated using the following formula:

[0127]

[0128] Where vI(x,y) is the input brightness image, which can be the brightness image bI(x,y) of the ID photo to be evaluated, or the brightness image hBI(x,y) of the hair region or the brightness image sBI(x,y) of the skin region; vH and vW are the height and width of the brightness image, respectively, and bmean is the brightness mean. When the input image is the brightness image hBI(x,y) of the hair region, the brightness mean hBmean of the hair region is obtained; when the input image is the brightness image sBI(x,y) of the skin region, the brightness mean sBmean of the skin region is obtained. The formulas for the brightness mean hBmean of the hair region and the brightness mean sBmean of the skin region are as follows:

[0129]

[0130]

[0131] S4. Calculate the ratio of the average brightness of the hair region to the average brightness of the skin region to obtain the first ratio, and then execute S5.

[0132] The first ratio is expressed by the formula:

[0133]

[0134] Where hsRatio is the first ratio, hBmean is the mean brightness of the hair region, and sBmean is the mean brightness of the skin region.

[0135] S5. When the average brightness of the hair area is greater than the preset hair brightness threshold, the ID photo to be evaluated is determined to be a grayscale image.

[0136] Otherwise, if the average brightness of the hair area is less than or equal to the preset hair brightness threshold, and the first ratio is greater than the preset ratio threshold, then the ID photo to be evaluated is determined to be a normal image.

[0137] When the average brightness of the hair area is less than or equal to the preset hair brightness threshold, and the first ratio is less than or equal to the preset ratio threshold, step S6 can be executed.

[0138] The hair brightness threshold and ratio threshold can be preset according to the needs of actual applications. For example, multiple normal-quality ID photo samples and grayed-out ID photo samples can be collected separately. Then, the average brightness of these normal and grayed-out ID photo samples can be calculated separately, and the hair brightness threshold and ratio threshold can be set based on these average brightness values. Optionally, when collecting ID photo samples, in addition to positive samples, negative samples can also be included, where negative samples are grayed-out images.

[0139] Then at this time:

[0140] The step "obtain the average brightness of the multiple ID photo samples in the hair area and the average brightness of the multiple ID photo samples in the skin area" can specifically include: obtaining the average brightness of multiple positive samples in the hair area, the average brightness of multiple positive samples in the skin area, and the average brightness of multiple negative samples in the hair area; optionally, the average brightness of multiple negative samples in the skin area can also be obtained.

[0141] After the step "obtain the average brightness value of the multiple ID photo samples in the hair area and the average brightness value in the skin area", it may further include: setting a hair brightness threshold and a ratio threshold based on the average brightness values ​​of the multiple positive samples in the hair area, the multiple positive samples in the skin area, and the multiple negative samples in the hair area. For example, it can be as follows:

[0142] A preset clustering algorithm is used to calculate the average brightness of multiple positive samples in the hair region, obtaining the cluster center value of the positive sample hair region. The same clustering algorithm is used to calculate the average brightness of multiple positive samples in the skin region, obtaining the cluster center value of the positive sample skin region. The same clustering algorithm is used to calculate the average brightness of multiple negative samples in the hair region, obtaining the cluster center value of the negative sample hair region. Then, the average value of the cluster center values ​​of the positive sample hair region and the negative sample hair region is set as the hair brightness threshold, and the ratio of the cluster center value of the positive sample hair region to the cluster center value of the positive sample skin region is set as the ratio threshold.

[0143] Optionally, when setting the hair brightness threshold and ratio threshold, the average brightness of multiple negative samples in the skin color region can also be considered as one of the factors. Specifically, the clustering algorithm can be used to calculate the average brightness of multiple negative samples in the skin color region to obtain the cluster center of the skin color region of multiple negative samples. Then, when setting the hair brightness threshold and ratio threshold, the cluster center of the skin color region of multiple negative samples can be used as a reference factor, and so on. These details will not be elaborated here.

[0144] The clustering algorithm can be configured according to the needs of the actual application. For example, the following formula can be used:

[0145]

[0146] Among them, C n Let C1 be the sample space, and n takes values ​​in the range [1,2]. When n=1, it represents a positive sample, that is, C1 is the sample space of normal ID photos, called the positive sample space. When n=2, it represents a negative sample, that is, C2 is the sample space of gray ID photos, called the negative sample space.

[0147] P nkC n The mean brightness of the sample in the hair region and skin region in space, i.e., when n=1, k takes values ​​of 1 and 2 respectively, P 11 and P 12 Let P be the average brightness of the samples in the hair region and skin color region of the positive sample space C1, respectively. When n = 2, and k takes values ​​of 1 and 2, respectively, P... 21 and P 22 P represents the average brightness of the samples in the hair and skin regions of the negative sample space C2, respectively. nk (m) is C n The mean brightness of a sample m in the hair or skin region of space, for example, P. 11 (m) represents the mean brightness of a sample m in the hair region within the C1 space, P 12 (m) represents the mean brightness of a sample m in the skin-colored region in the C1 space, P 21 (m) represents the mean brightness of a sample m in the hair region within the C2 space, P 22 (m) represents the average brightness of a sample m in the skin color region in the C2 space, and so on.

[0148] E n Let E1 be the clustering error, where E1 is the clustering error in the positive sample space C1 when n = 1, and E2 is the clustering error in the negative sample space C2 when n = 2. n C n The number of samples in the space, m is M n Specific samples in the text.

[0149] O nk The center point of the cluster is where, when n = 1 and k = 1, O 11 The cluster centers of the hair regions of samples in the positive sample space C1 are determined when n = 1 and k = 2. 12 For the cluster centers of skin color regions in the positive sample space C1, when n=2 and k takes 1, O 21 For the cluster centers of the hair regions of samples in the C2 sample space of the grayscale image, when n=2 and k takes 2, O 22 The cluster centers are the skin color regions of the samples in the C2 sample space of the gray image.

[0150] In the clustering loop iteration, when the clustering error E n When it no longer changes, take O. nk The value of O is used as the center value for the two classes, hair region and skin color region. That is, when E1 no longer changes, the value of O at this point is taken. 11 The value is used as the center value of the cluster "hair region of positive samples", and the value of O at this time is taken. 12The value is used as the center value of the cluster "skin color region of positive sample". In other words, the cluster center value O of the hair region of positive sample can be obtained at this time. 11 And obtain the cluster center value O of the skin color region of the positive sample. 12 Similarly, when E2 no longer changes, we take the value of O at this point. 21 The value is used as the center value of the cluster "hair region of negative samples", and the value of O at this time is taken. 22 The value is used as the center value of the cluster "skin color region of negative sample". In other words, the cluster center value O of the hair region of negative sample can be obtained at this time. 21 And obtain the cluster center value O of the skin color region of the negative sample. 22 .

[0151] The cluster center value O of the positive sample hair region is obtained. 11 Cluster center value O for positive skin color regions 12 The average O of the cluster centers of the negative sample hair regions 21 Then, the cluster center values ​​O of the positive sample hair regions can be calculated. 11 The average O of the cluster centers of the negative sample hair regions 21 Set the hair brightness threshold hsMeanTh, and set the cluster center value O of the positive sample hair region. 11 Cluster center value O of the skin color region of the positive sample 12 The ratio is set as the ratio threshold hsRatioTh, which can be expressed by the following formula:

[0152]

[0153] S6. Obtain the average edge intensity of the ID photo to be evaluated. If the average edge intensity is greater than the preset edge intensity threshold, the ID photo to be evaluated is determined to be a normal image; otherwise, if the average edge intensity is less than or equal to the preset edge intensity threshold, proceed to step S7.

[0154] For example, the photo of the ID to be evaluated can be converted into a grayscale image, and the edge image of the grayscale image can be calculated. False edges in the edge image can be removed to obtain an adjusted edge image. Then, the variance of the adjusted edge image can be calculated to obtain the average edge intensity of the photo of the ID to be evaluated.

[0155] For example, the edge image of the grayscale image can be calculated using the following formula:

[0156]

[0157] Where gI(x,y) is the grayscale image, eI(x,y) is the edge image, and (x,y) is the coordinate value of the image pixel in the image coordinate system.

[0158] After obtaining the edge image, false edges caused by noise points can be removed. For example, a small threshold can be used to remove edges with intensity values ​​less than that threshold, resulting in an adjusted edge image. Then, the variance of this adjusted edge image can be calculated. The standard deviation of the calculated edge image is the average edge intensity of the ID photo being evaluated. For example, the average edge intensity of the ID photo being evaluated can be calculated using the following formula:

[0159]

[0160] Where eIvar is the standard deviation of the edge image eI(x,y), emean is the mean of the edge image eI(x,y), (x,y) is the coordinate value of the image pixel in the image coordinate system, and H and W are the height and width of the brightness image, respectively.

[0161] The edge strength threshold can be set based on requirements, experiments, or practical experience. For example, by collecting multiple ID photo samples, such as positive samples, calculating the average edge strength of these positive samples, and statistically calculating the mean of these average edge strengths, the edge strength threshold can be set. This will not be elaborated on here.

[0162] S7. Obtain the feature value of the center position of the histogram peak of the ID photo to be evaluated. If the feature value of the center position of the histogram peak is less than the preset position threshold, the ID photo to be evaluated is determined to be a gray image; otherwise, if the feature value of the center position of the histogram peak is greater than or equal to the preset position threshold, the ID photo to be evaluated is determined to be a normal image.

[0163] For example, the horizontal position of the maximum value of the histogram of the brightness image in the coordinate system can be taken as the position of the peak center point (i.e., the feature value of the histogram peak center position, which can be expressed by the following formula):

[0164] w_c = Loc(max(Hist(i)));

[0165] Where w_c is the feature value of the center position of the peak of the histogram, Hist(i) is the histogram, max(Hist(i)) is the maximum value of the histogram, and Loc(·) takes the horizontal position of the histogram where the maximum value is located.

[0166] The location threshold can be set based on requirements, experiments, or practical experience. For example, it can be set by collecting multiple ID photo samples, such as positive samples, calculating the feature values ​​of the center position of the histogram peaks of the brightness images of these positive samples, and statistically calculating the average value of the center position of the histogram peaks, etc., which will not be elaborated here.

[0167] As can be seen from the above, after obtaining the ID photo to be repaired, this embodiment can determine the feature values ​​of the left and right waveforms in the histogram of the ID photo, and generate an image restoration transformation function and a dynamic scaling factor accordingly. Then, the ID photo to be repaired is processed by the image restoration transformation function, and its brightness and details are restored or enhanced by the dynamic scaling factor. Since this scheme mainly relies on the feature values ​​of the left and right waveforms in the histogram rather than the texture pixels of the image when repairing the image, there are no special requirements for the image contrast. Even if the image with relatively normal contrast or even a normal image is repaired, it will not cause color distortion. Moreover, since the dynamic scaling factor can repair its brightness and details, it can also avoid the loss of detail features that may be caused in the early repair process, which greatly improves the final repair effect. In addition, since this scheme does not require a lot of time to collect a large number of diverse training samples, it not only has good real-time performance, but is also simple to implement. In summary, this solution is not only simple to implement and highly real-time, but also has a better repair effect, greatly reducing the probability of producing invalid certificates and improving the success rate of certificate production.

[0168] Based on the methods described in the above embodiments, the following examples will provide further detailed explanations.

[0169] In this embodiment, the image restoration device will be specifically integrated into a network device as an example for explanation.

[0170] like Figure 5 and 6 As shown, an image restoration method can be described in the following steps:

[0171] 201. Network devices obtain photos of the documents to be repaired.

[0172] For example, a network device can receive a photo of an ID card that needs to be repaired, such as a grayed-out photo; or, the network device can evaluate the photo of the ID card to be assessed and use the photo of the ID card that is assessed as a "grayed-out image" as the photo of the ID card to be repaired, and so on.

[0173] It should be noted that normal quality ID photos can also be input. For normal quality ID photos, the repair method provided in this embodiment can also improve their brightness and some details, thus optimizing their quality.

[0174] 202. The network device acquires the brightness value of each pixel in the ID photo to be repaired, and generates a histogram of the ID photo based on the acquired pixel brightness values. For example, it can be done as follows:

[0175] Obtain the red, green, and blue channel values ​​of each pixel in the ID photo to be repaired. Select the maximum value from the three values ​​as the brightness value of the pixel. Then, generate a histogram of the ID photo to be repaired based on the obtained brightness values ​​of the pixels. For details, please refer to the previous embodiments, which will not be repeated here.

[0176] 203. The network device determines the left and right boundary points of the left waveform of the histogram of the ID photo to be repaired, and the left and right boundary points of the right waveform of the histogram of the ID photo to be repaired.

[0177] The specific methods for determining the left boundary point, right boundary point, left boundary point, and right boundary point of the left waveform can be found in the previous embodiments, and will not be repeated here.

[0178] It should be noted that if there is only one obvious waveform in the histogram, such as a bald head, there may only be one right waveform. In this case, the left waveform can be considered non-existent or negligible, that is, the left boundary point and the right boundary point of the left waveform are considered non-existent or negligible (equivalent to 0). Similarly, if there is only one left waveform, such as a dark-skinned person, the right waveform can be considered non-existent or negligible, that is, the left boundary point and the right boundary point of the right waveform are considered non-existent or negligible (equivalent to 0), and so on.

[0179] 204. The network device calculates the variance of the left waveform based on the left boundary point and the right boundary point of the left waveform to obtain the left endpoint of the left waveform. It also calculates the variance of the right waveform based on the left boundary point and the right boundary point of the right waveform to obtain the right endpoint of the right waveform.

[0180] For example, a network device can determine the left endpoint of the left waveform using the 3δ rule in the normal distribution function, based on the left and right boundary points of the left waveform. For instance, in a histogram, it can traverse from the left boundary point of the left waveform to the right boundary point, stopping when the accumulated pixels equal 0.0015 of the area enclosed by the intersection line between the left waveform and the right boundary point. The position value of the currently traversed point is then taken as the left endpoint of the left waveform.

[0181] Similarly, network devices can determine the right endpoint of the right waveform using the 3δ rule in the normal distribution function, based on the left and right boundary points of the right waveform. For example, in the histogram, the device can traverse from the right boundary point of the right waveform to the left boundary point, stopping when the accumulated pixels equal 0.0015 of the area enclosed by the intersection line of the right waveform and the left boundary point. The position value of the currently traversed point is then taken as the right endpoint of the right waveform, and so on.

[0182] For details on how to determine the left and right endpoints, please refer to the previous embodiments, which will not be repeated here.

[0183] It should be noted that if there is only one obvious waveform in the histogram, such as a bald head, there may only be one right waveform. In this case, the left waveform can be considered non-existent or can be ignored, that is, the values ​​of the left and right endpoints of the left waveform are set to 0. Similarly, if there is only one left waveform, such as a dark-skinned person, the right waveform can be considered non-existent or can be ignored, that is, the values ​​of the left and right endpoints of the right waveform are set to 0, and so on.

[0184] 205. The network device generates an image restoration transform function based on the left endpoint of the left waveform and the right endpoint of the right waveform.

[0185] For example, taking the left endpoint of the left waveform as δl and the right endpoint of the right waveform as δr, the network device can construct the first dynamic factor α based on the left endpoint δl of the left waveform and the right endpoint δr of the right waveform, as follows:

[0186]

[0187] Then, based on the left endpoint δl of the left waveform, the right endpoint δr of the right waveform, and the first dynamic factor α, the second dynamic factor β is constructed as follows:

[0188]

[0189] Finally, an image restoration transformation function is generated based on the first dynamic factor α and the second dynamic factor β, for example, as follows:

[0190] d(x,y)=I(x,y)·α+β

[0191] Where I(x,y) is the ID photo to be evaluated with the quality of "grayish image", d(x,y) is the image restored by the image restoration transformation function (i.e. the processed ID photo), and e(·) is the natural index.

[0192] 206. The network device constructs a dynamic scaling factor based on the left endpoint of the left waveform and the right endpoint of the right waveform, and establishes a mapping table based on the dynamic scaling factor and a preset functional relationship.

[0193] For example, taking the left endpoint of the left waveform as δl and the right endpoint of the right waveform as δr, the network device can calculate the product of the right endpoint and 2 to obtain the first product "2δl". r And calculate the difference between the right endpoint and the left endpoint to obtain the first difference "δ". r -δ lThen, add the ratio between the first product and the first difference to 1 to obtain the dynamic boosting factor r. For example, its formula can be as follows:

[0194]

[0195] To reduce computational load and image processing time, the network device can generate a mapping table `list(i)` based on the dynamic scaling factor. This allows the network to quickly obtain the repaired ID photo by inputting the processed ID photo through `list(i)`. The relationship between the mapping table `list(i)` and the dynamic scaling factor `r` is as follows:

[0196]

[0197] It should be noted that, in practice, the execution order of steps 205 and 206 is not important and can be determined according to the actual application requirements.

[0198] 207. The network device uses the image restoration transformation function to process the ID photo to be restored, obtains the processed ID photo, and then executes step 208.

[0199] For example, as long as the ID photo to be repaired is input as I(x,y), the processed ID photo d(x,y) can be obtained through the image restoration transformation function “d(x,y)=I(x,y)·α+β”.

[0200] 208. The network device limits the pixel value range within the image matrix of the processed ID photo to between 0 and 255 to obtain the adjusted ID photo, and then executes step 209.

[0201] For example, network devices can adjust pixel values ​​exceeding 255 in the processed ID photo d(x,y) to 255, set pixel values ​​less than 0 to 0, and so on. In this way, the range of pixel values ​​within the image matrix can be limited to between 0 and 255, thereby preventing pixel values ​​from overflowing the range.

[0202] 209. The network device uses the mapping table to repair the brightness and details of the adjusted ID photo to obtain the repaired ID photo.

[0203] For example, taking the adjusted ID photo as d′(x,y), after inputting the adjusted ID photo as d′(x,y), the repaired ID photo outI(x,y) can be obtained through the following mapping table, as follows:

[0204] outI(x,y)=list[d′(x,y)];

[0205] Where outI(x,y) is the repaired ID photo, d′(x,y) is the adjusted ID photo, and list(i) is the mapping table.

[0206] It should be noted that if the input at this time is the processed ID photo d(x,y), then the repaired ID photo outI(x,y) will be:

[0207] outI(x,y) = list[d(x,y)].

[0208] As can be seen from the above, after obtaining the ID photo to be repaired, this embodiment can determine the feature values ​​of the left and right waveforms in the histogram of the ID photo, and generate an image restoration transformation function and a dynamic enhancement factor accordingly. Then, the ID photo to be repaired is processed by the image restoration transformation function, and its brightness and details are restored or enhanced by the dynamic enhancement factor. Since this scheme mainly relies on the feature values ​​of the left and right waveforms in the histogram rather than the texture pixels of the image when repairing the image, there are no special requirements for the contrast of the image. Even if the image with relatively normal contrast or even a normal image is repaired, it will not cause color distortion. Moreover, since the dynamic enhancement factor can repair its brightness and details, it can also avoid the loss of detail features that may be caused in the early repair process, greatly improving the final repair effect, thereby reducing the probability of producing unusable ID cards and improving the success rate of ID card production.

[0209] Furthermore, since this scheme does not require spending too much time collecting a large number of diverse training samples, it is not only simple to implement, but also has good real-time performance.

[0210] Furthermore, this embodiment generates a mapping table based on the dynamic upscaling factor, so that when restoring the brightness and details of the image, there is no need to perform too many tedious calculations. Instead, the result can be obtained directly using the mapping table. Therefore, the amount of computation can be greatly reduced, the image processing time can be reduced, and the real-time performance can be further improved.

[0211] To better implement the above methods, embodiments of this application also provide an image restoration apparatus, such as... Figure 7 As shown, the image restoration device includes: an acquisition unit 301, a determination unit 302, a generation unit 303, a processing unit 304, and a restoration unit 305, as detailed below:

[0212] (1) Obtain unit 301;

[0213] The acquisition unit 301 is used to acquire the histogram of the ID photo to be repaired.

[0214] For example, the acquisition unit 301 can specifically acquire the ID photo to be repaired, then acquire the brightness value of each pixel in the ID photo to be repaired, and generate a histogram of the ID photo to be repaired based on the acquired pixel brightness values. The histogram includes a left waveform and a right waveform, where the left waveform represents the peak value in the dark area, and the right waveform represents the peak value in the bright area.

[0215] (2) Determine unit 302;

[0216] The determining unit 302 is used to determine the characteristic values ​​of the left waveform and the right waveform of the histogram.

[0217] The feature values ​​of the left waveform may include the left boundary point and the right boundary point of the left waveform, and the feature values ​​of the right waveform may include the left boundary point and the right boundary point of the right waveform. The method for determining the left boundary point and the right boundary point can be referred to the previous embodiments, and will not be repeated here.

[0218] (3) Generation unit 303;

[0219] The generation unit 303 is used to generate an image restoration transform function and a dynamic stretching factor based on the feature values ​​of the left waveform and the right waveform.

[0220] For example, the generation unit 303 can be specifically used to calculate the variance of the left waveform based on the left boundary point and the right boundary point of the left waveform to obtain the left endpoint of the left waveform, calculate the variance of the right waveform based on the left boundary point and the right boundary point of the right waveform to obtain the right endpoint of the right waveform, and then generate an image restoration transformation function based on the left endpoint of the left waveform and the right endpoint of the right waveform, and construct a dynamic stretching factor based on the left endpoint of the left waveform and the right endpoint of the right waveform.

[0221] (4) Processing unit 304;

[0222] The processing unit 304 is used to process the ID photo to be repaired using the image restoration transformation function to obtain the processed ID photo.

[0223] (5) Repair unit 305;

[0224] The repair unit 305 is used to repair the brightness and details of the processed ID photo using the dynamic upscaling factor to obtain the repaired ID photo.

[0225] Optionally, in order to reduce the amount of computation and image processing time, a mapping table can be generated based on the dynamic stretching factor. In this way, by using the mapping table, the repaired ID photo can be quickly obtained simply by inputting the processed ID photo.

[0226] Specifically, the repair unit 305 can be used to establish a mapping table based on the dynamic upscaling factor and a preset functional relationship, and then use the mapping table to repair the brightness and details of the repaired ID photo to obtain the repaired ID photo.

[0227] Optionally, after obtaining the processed ID photo, if the processed ID photo is an 8-bit portrait bitmap, to prevent pixel values ​​from overflowing, the range of pixel values ​​within the image matrix can be limited to between 0 and 255. For example, pixel values ​​exceeding 255 can be adjusted to 255, and pixel values ​​less than 0 can be set to 0, etc. That is, the processing unit 304 can also be used to limit the range of pixel values ​​within the image matrix of the processed ID photo to between 0 and 255 to obtain the adjusted ID photo.

[0228] At this point, the repair unit 305 can be used to repair the brightness and details of the adjusted ID photo using a preset dynamic upscaling factor, thereby obtaining the repaired ID photo.

[0229] Optionally, the ID photo to be repaired can be provided by the user, or the image restoration device can evaluate the ID photo and use the ID photo whose evaluation result is "grayish image" as the ID photo to be repaired. That is, the image restoration device can also include an evaluation unit, as follows:

[0230] The evaluation unit is used to acquire the ID photo to be evaluated and acquire the brightness image of the ID photo to be evaluated. Based on the brightness image of the ID photo to be evaluated, the image quality of the ID photo to be evaluated is evaluated. If the evaluation result is a gray image, the ID photo to be evaluated is regarded as an ID photo to be repaired.

[0231] For details on the implementation of each of the above units, please refer to the preceding method embodiments; they will not be repeated here.

[0232] It should be noted that, in practical implementation, the above units can be arbitrarily combined and integrated into one or more modules, or implemented as independent entities. Furthermore, the above units can be implemented in hardware or as software functional modules. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The aforementioned storage medium can be a read-only memory, a hard disk, or an optical disk, etc.

[0233] The image restoration device provided in this embodiment is not only simple to implement and highly real-time, but also restores or enhances brightness and detail when restoring ID photos, greatly improving the restoration effect. This helps reduce the probability of producing defective ID cards and increases the success rate of ID card production.

[0234] The above provides a detailed description of an image restoration method and apparatus provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and its core ideas. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An image restoration method, characterized in that, include: Obtain the histogram of the ID photo to be repaired. The histogram includes a left waveform and a right waveform. The left waveform is the waveform where the peak value of the dark part is located, and the right waveform is the waveform where the peak value of the bright part is located. Determine the eigenvalues ​​of the left and right waveforms of the histogram; An image restoration transform function and a dynamic stretching factor are generated based on the feature values ​​of the left and right waveforms. The image restoration transformation function is used to process the ID photo to be restored to obtain the processed ID photo; The brightness and details of the processed ID photo are restored using the dynamic upscaling factor to obtain the restored ID photo; The step of generating the image restoration transform function and the dynamic upscaling factor based on the feature values ​​of the left and right waveforms includes: The variance of the left waveform is calculated based on the left and right boundary points of the left waveform, and the left endpoint of the left waveform is obtained. The variance of the right waveform is calculated based on the left boundary point and the right boundary point of the right waveform, and the right endpoint of the right waveform is obtained. The first dynamic factor is constructed based on the left endpoint of the left waveform and the right endpoint of the right waveform; The second dynamic factor is constructed based on the left endpoint of the left waveform, the right endpoint of the right waveform, and the first dynamic factor; An image restoration transformation function is generated based on the first dynamic factor and the second dynamic factor. Calculate the product of the right endpoint and 2 to obtain the first product; Calculate the difference between the right endpoint and the left endpoint to obtain the first difference; Add the ratio between the first product and the first difference to 1 to obtain the dynamic boost factor.

2. The method according to claim 1, characterized in that, Determining the eigenvalues ​​of the left and right waveforms of the histogram includes: Determine the left and right boundary points of the left waveform of the histogram to obtain the characteristic values ​​of the left waveform; The left and right boundary points of the right waveform of the histogram are determined to obtain the characteristic values ​​of the right waveform.

3. The method according to claim 2, characterized in that, Determining the left and right boundary points of the left waveform of the histogram, and determining the left and right boundary points of the right waveform of the histogram, includes: In the histogram, a first straight line parallel to the horizontal axis is set. The first straight line gradually moves upward along the vertical axis until it intersects with the rising and falling edges of the left waveform and the rising and falling edges of the right waveform, thus obtaining an intersection line. The horizontal axis coordinate value corresponding to the intersection point of the intersection line and the falling edge of the left waveform is determined to obtain the left boundary point of the left waveform; The horizontal axis coordinate value corresponding to the intersection point of the intersection line and the falling edge of the left waveform is determined to obtain the right boundary point of the left waveform; The horizontal axis coordinate value corresponding to the intersection point of the intersecting line and the rising edge of the right waveform is determined to obtain the left boundary point of the right waveform; The right boundary point of the right waveform is obtained by determining the horizontal axis coordinate value corresponding to the intersection point of the intersecting line and the falling edge of the right waveform.

4. The method according to claim 1, characterized in that, The step of calculating the variance of the left waveform based on the left boundary point and the right boundary point of the left waveform to obtain the left endpoint of the left waveform includes: in the histogram, traversing from the left boundary point of the left waveform to the right boundary point of the left waveform, stopping when the accumulated pixels are equal to 0.0015 of the area enclosed by the intersection line of the left waveform, and taking the position value of the currently traversed point as the left endpoint of the left waveform. The step of calculating the variance of the right waveform based on the left boundary point and the right boundary point of the right waveform to obtain the right endpoint of the right waveform includes: in the histogram, traversing from the right boundary point of the right waveform to the left boundary point of the right waveform, stopping when the accumulated pixels are equal to 0.0015 of the area enclosed by the right waveform and the intersection line, and taking the position value of the currently traversed point as the right endpoint of the right waveform.

5. The method according to any one of claims 1 to 4, characterized in that, The process of using the dynamic upscaling factor to restore the brightness and details of the processed ID photo to obtain a restored ID photo includes: Based on the dynamic pull-up factor, a mapping table is established according to a preset functional relationship; The brightness and details of the repaired ID photo are restored using the mapping table to obtain the repaired ID photo.

6. The method according to any one of claims 1 to 4, characterized in that, Before obtaining the histogram of the ID photo to be repaired, the process also includes: Obtain the ID photo to be evaluated, and obtain the brightness image of the ID photo to be evaluated; The image quality of the ID photo to be evaluated is assessed based on the brightness image. If the evaluation result is a gray image, then the ID photo to be evaluated will be used as the ID photo to be repaired.

7. An image restoration device, characterized in that, include: The acquisition unit is used to acquire the histogram of the ID photo to be repaired. The histogram includes a left waveform and a right waveform. The left waveform is the waveform where the peak value of the dark part is located, and the right waveform is the waveform where the peak value of the bright part is located. A determining unit is used to determine the characteristic values ​​of the left waveform and the right waveform of the histogram; The generation unit is used to generate an image restoration transform function and a dynamic stretching factor based on the feature values ​​of the left waveform and the right waveform. The processing unit is used to process the ID photo to be repaired using the image restoration transformation function to obtain the processed ID photo; The repair unit is used to repair the brightness and details of the processed ID photo using the dynamic upscaling factor to obtain the repaired ID photo. The generation unit is specifically used to calculate the variance of the left waveform based on the left boundary point and the right boundary point of the left waveform, and obtain the left endpoint of the left waveform; and to calculate the variance of the right waveform based on the left boundary point and the right boundary point of the right waveform, and obtain the right endpoint of the right waveform. The first dynamic factor is constructed based on the left endpoint of the left waveform and the right endpoint of the right waveform; The second dynamic factor is constructed based on the left endpoint of the left waveform, the right endpoint of the right waveform, and the first dynamic factor; An image restoration transformation function is generated based on the first dynamic factor and the second dynamic factor. Calculate the product of the right endpoint and 2 to obtain the first product; calculate the difference between the right endpoint and the left endpoint to obtain the first difference; Add the ratio between the first product and the first difference to 1 to obtain the dynamic boost factor.

Images