A portrait sharpening method, device, equipment and medium based on Hessian filter

Abstract

The invention discloses a portrait sharpening method, device, equipment and medium based on Hessian filter, belonging to the technical field of image processing; the method specifically comprises: obtaining an original image; performing low-pass filtering on the original image to obtain first original image information, and obtaining a first label image; performing image subtraction between the original image and the first label image to obtain second original image information, and obtaining a second label image; performing Hessian filtering on the original image to obtain third original image information, and obtaining a third label image; performing linear fusion of the original image and the second label image based on the third label image to obtain a result image; the invention processes structural texture and low-frequency texture of the skin separately, detects the structural texture by using a Hessian filter as a guide mask, and then strengthens the structural texture of the facial features to achieve a sharpening effect without strengthening skin defects; the invention has the advantage that it can strengthen the structural texture of the facial features of the portrait without enhancing the image noise of the low-frequency skin texture, and is very effective in improving the sharpness of the portrait.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a portrait sharpening method, device, equipment and medium based on a Hessian filter. Background Art

[0002] Image sharpening is to compensate for the contour of the image, enhance the edge of the image and the grayscale jump part, so that the image becomes clear. It is divided into two categories: spatial domain processing and frequency domain processing. Image sharpening is to highlight the edges, contours, or features of certain linear target elements of the objects on the image. This filtering method increases the contrast between the edge of the object and the surrounding pixels, so it is also called edge enhancement. Some image enhancement and sharpening software currently on the market make post-processing image sharpening very fast or convenient. With the development of mobile Internet, a variety of convenient mobile hard application software have also appeared on mobile devices, but it will be found that the algorithms of these hard application software often sharpen the portrait indiscriminately, so that the skin defects are magnified, that is, global sharpening; although the details of the face are improved, the corresponding facial defect details are also highlighted. Therefore, the applicant proposes a portrait sharpening method based on the Hessian filter. Technically, the structural texture of the portrait and the low-frequency texture of the skin are processed separately, and the structural texture of the facial features is detected by using the Hessian filter as a guide mask, and then the structural texture of the facial features is enhanced without enhancing the skin defects. The sharpening effect is achieved.

[0003] When this application was initiated, a search for prior art was conducted and two comparative documents on image sharpening were found, the applicant of which was Nubia Technology Co., Ltd.

[0004] Comparative Document 1: CN201510716021.8 discloses an image sharpening method, including: a mobile terminal obtains an original RGB image, and converts the original RGB image into a YCbCr space image; obtains the intensity information of each pixel of the brightness component in the YCbCr space image, and determines the black edges and white edges and their corresponding sharpening intensities according to the intensity information of each pixel; sharpens the black edges and white edges according to their corresponding sharpening intensities; converts the sharpened YCbCr space image into a new RGB image to obtain a sharpened image. The present invention also discloses a mobile terminal for image sharpening. The present invention realizes sharpening of the black edges and white edges separately to a corresponding degree, thereby improving the sharpening effect of the image. The method of this application is the closest prior art to this application, and it also uses YUV format images for sharpening processing; it also uses the brightness channel as the entry point to sharpen the image; but the difference is that this application uses contrast to determine the texture information of the image, and then uses contrast to control the intensity of sharpening; while the distinguishing technical feature of this application is that it first obtains the texture information of the image to obtain the structural texture, and then uses the structural texture as a guide to strengthen the structural texture of the facial features; compared with Comparative Document 2, the substantial progress and significant feature of this application is that Comparative Document 2 uses the brightness channel to extract image information and then merges the structural texture and skin low-frequency texture. Technically, this application processes the portrait structural texture and skin low-frequency texture separately, detects the structural texture as a guide mask by using the Hessian filter, and then strengthens the structural texture of the facial features to achieve the sharpening effect without strengthening the skin blemishes.

[0005] Comparative Document 2: CN201611248625.5 discloses an image sharpening method, the method comprising: identifying facial information contained in a captured image according to a face recognition method, and extracting biometric information from the facial information; determining that the image corresponding to the biometric information in the captured image is a first sharpening area, and the image other than the image corresponding to the biometric information in the captured image is a second sharpening area; receiving a first sharpening instruction for the first sharpening area, and obtaining a first sharpened image; receiving a second sharpening instruction for the second sharpening area, and obtaining a second sharpened image; fusing the first sharpened image and the second sharpened image to obtain a fused sharpened image. The embodiment of the present invention also discloses an image sharpening terminal. The problem of over-sharpening when sharpening a portrait is solved, and the rationality of the degree of portrait sharpening is ensured. By comparison, it is found that the method of the application is actually to obtain facial information, and then use the facial information as a boundary to divide the image into a face area and a non-face area, and then sharpen the two parts separately, and finally fuse them to obtain a sharpened result. The problem of over-sharpening when sharpening portraits is solved, and the rationality of the portrait sharpening degree is ensured, but the problem is that the details of facial blemishes such as acne marks, acne pits, freckles, etc. are also enhanced accordingly. Although the details of the face are improved, the corresponding facial blemish details are also highlighted. The combination of comparative file 1 and comparative file 2 is also difficult to solve the problem of magnified skin blemishes. Summary of the invention

[0006] (I) Technical solution

[0007] The present invention is implemented by the following technical solution: a portrait sharpening method based on a Hessian filter, the method specifically comprising:

[0008] Get the original image;

[0009] Performing low-pass filtering on the brightness channel of the original image to obtain first original image information, thereby obtaining a first label image;

[0010] Subtracting the original image from the first labeled image to obtain second original image information, thereby obtaining a second labeled image;

[0011] Perform Hessian filtering on the original image to obtain information of a third original image, thereby obtaining a third labeled image;

[0012] The original image and the second labeled image are linearly fused according to the third labeled image to obtain a result image.

[0013] As a further illustration of the above solution, the original image is obtained by a smart terminal device.

[0014] As a further illustration of the above solution, the original image format includes a YUV format.

[0015] As a further illustration of the above solution, the first original image information includes original image texture structure information.

[0016] As a further illustration of the above solution, the low-pass filtering adopts Gaussian filtering.

[0017] As a further illustration of the above solution, the second original image information includes original image texture information.

[0018] As a further explanation of the above scheme, the original image Hessian filtering calculation method is as follows:

[0019]

[0020] Where G(X,s) is the Gaussian distribution weight, L(X,s) is the image at point x o Taylor expansion of .

[0021] As a further illustration of the above solution, the third original image information includes structural texture information of the original image.

[0022] As a further illustration of the above scheme, the calculation formula of the linear fusion is as follows:

[0023]

[0024] Where D st Represents the output result graph, S rc Represents the original input image, I Detail Represents the second original image information, I Mask Indicates the third original image information.

[0025] The present invention also proposes a portrait sharpening device based on a Hessian filter, the device comprising:

[0026] An acquisition unit, used for acquiring the original image;

[0027] Processing unit: used for performing low-pass filtering on the brightness channel of the original image to obtain the first original image information, and obtain the first label image; performing image subtraction between the original image and the first label image to obtain the second original image information, and obtain the second label image; performing Hessian filtering on the original image to obtain the third original image information, and obtain the third label image; performing linear fusion of the original image and the second label image according to the third label image to obtain the result image;

[0028] Preview unit: used to preview the result graph output by the processing unit.

[0029] The present invention also proposes a portrait sharpening device based on a Hessian filter, comprising a processor, a memory, and a computer program stored in the memory, wherein the computer program can be executed by the processor to implement the portrait sharpening method based on a Hessian filter.

[0030] The present invention also proposes a computer-readable storage medium, characterized in that the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the portrait sharpening method based on the Hessian filter.

[0031] (III) Beneficial effects

[0032] Compared with the prior art, the present invention has the following beneficial effects: the present invention processes structural texture and low-frequency skin texture separately, detects structural texture by using Hessian filter as a guide mask, and then strengthens the structural texture of facial features to achieve sharpening effect without strengthening skin defects; the advantage of the present invention is that it can strengthen the structural texture of facial features without enhancing the image noise of low-frequency skin texture, which is very effective in improving the sharpness of portraits. Since different information of the picture is processed separately, it can be processed more delicately than the commonly used unified sharpening solution, and facial defects and noise will not be strengthened. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0034] Figure 1 It is a schematic diagram of the process of an embodiment of the present invention;

[0035] Figure 2 The original image obtained by the embodiment of the present invention;

[0036] Figure 3 A structural texture image extracted by an embodiment of the present invention;

[0037] Figure 4 The output diagram is an embodiment of the present invention. DETAILED DESCRIPTION

[0038] Example 1

[0039] See also Figure 1 , a portrait sharpening method based on Hessian filter, the method specifically comprising:

[0040] To get the original image, see Figure 2; It should be further explained that the original image does not limit the format of the picture, the hardware and software of the picture source. In this embodiment, the original image format is preferably an image in YUV format. YUV is a color encoding method. It is often used in various video processing components. When encoding photos or videos, YUV allows the bandwidth of chromaticity to be reduced in consideration of human perception. YUV is a type of true-color color space (color space) compilation. Y'UV, YUV, YCbCr, YPbPr and other special terms can all be called YUV, and there are overlaps with each other. "Y" represents brightness (Luminance or Luma), that is, the grayscale value, and "U" and "V" represent chromaticity (Chrominance or Chroma), which is used to describe the color and saturation of the image and is used to specify the color of the pixel. The advantage of the YUV encoding format over the RGB encoding format is that the memory used per pixel in the YUV encoding format is lower, but the effects of the two are the same. On a device with a width and height of w*h, the number of bytes occupied by the RGB encoding is w*h*3, and the temporary memory used by the YUV encoding is w*h*+w*h / 4+w*h / 4=w*h*3 / 2.

[0041] It should be further explained that the original image is obtained by a smart terminal device; the smart terminal device may include a smart phone, a tablet computer, a notebook, a wearable device, a vehicle-mounted smart terminal, a videophone, a conference terminal, etc. This embodiment assumes that the terminal is a smart phone, but those skilled in the art will understand that, in addition to originals specifically for mobile purposes, the construction according to the embodiment of the present invention can also be applied to fixed-type terminals.

[0042] The original image is low-pass filtered to obtain first original image information, and a first label image is obtained; the first original image information includes original image texture structure information; it should be further explained that this step specifically performs low-pass filtering on the brightness channel of the original image to obtain the texture structure information of the image, which is named I Low , the low-pass filter is preferably Gaussian filter, the formula is as follows:

[0043]

[0044] In this embodiment, Gaussian filtering is preferably used for filtering, and the brightness channel of the original image is used to scan each pixel in the image, and the weighted average grayscale value of the pixels in the neighborhood determined by the brightness channel is used to replace the value of the central pixel of the template. Gaussian filtering is very effective in suppressing noise that obeys normal distribution. If a Gaussian filter is used, the system function is smooth and ringing phenomenon is avoided. However, the original source of the image in this embodiment is an intelligent terminal. Due to the volume of the intelligent terminal, the area of ​​the sensor is also small, so that the noise generated by the sensor under low illumination or high temperature conditions is mostly Gaussian noise. Therefore, it is more appropriate to use Gaussian filtering for processing in this embodiment.

[0045] The image texture structure information is a visual feature that reflects the homogeneous phenomenon in the image. It reflects the surface structure organization and arrangement properties of the object surface with slow or periodic changes. Texture has three major characteristics: 1) a certain local sequence is repeated continuously; 2) non-random arrangement; 3) the texture area is roughly a uniform unity. Unlike image features such as grayscale and color, texture is expressed by the grayscale distribution of pixels and their surrounding spatial neighborhoods, that is, local texture information. In addition, the repeatability of local texture information to varying degrees is global texture information. While texture features reflect the properties of global features, they also describe the surface properties of the scene corresponding to the image or image area. The purpose of obtaining the texture structure information of the image through the above steps is to outline the texture information in the subsequent steps, which will be explained in detail in the subsequent steps.

[0046] The original image is subtracted from the first label image to obtain the second original image information, and the second label image is obtained; the second original image information includes the texture information of the original image. The original image in step 1 is subtracted from the result image in step 2 to obtain the texture information of the image, which is named I Detail ; It needs to be further explained that texture information is the texture information of the image, including structural texture and detail texture. The texture information mentioned here is different from the texture structure information described in the previous step. The surface properties corresponding to the image area obtained in the previous step include both the texture information described in this step and the corresponding grayscale information of the original image; this step is equivalent to extracting the data of the original image and removing the redundant data.

[0047] The original image is subjected to Hessian filtering to obtain third original image information, and a third label image is obtained; the third original image information includes structural texture information of the original image; please refer to 3, the original image is subjected to Hessian filtering to obtain structural texture, which is named I Mask The purpose of this step is to extract features from texture structure information, so that the edge features of people, scenes, and objects in the image are more prominent and reduce the interference of other information.

[0048] The original image Hessian filtering calculation method is as follows:

[0049]

[0050] Where G(X,s) is the Gaussian distribution weight, L(X,s) is the image at point x o Taylor expansion of . D is equal to 2 for a two-dimensional image, x is the distance, and γ and s are generally equal to 1.

[0051] It should be further explained that the purpose of the above steps is to record color and light and shadow at low frequency, and save texture details at high frequency; the high and low frequencies are adjusted separately and do not interfere with each other. The low-frequency layer is used to control the color and light and shadow of the image, and the adjustment will not affect the details of the picture. The high-frequency layer is used to control the details without changing the color. The purpose of this embodiment is to sharpen the high-frequency layer, while the information in the low-frequency layer is not processed, so that it can enhance the structural texture of the facial features of the portrait without enhancing the image noise of the low-frequency skin texture; but there is a lot of information in the high-frequency layer, such as structural texture and detail texture, and the structural texture needs to be filtered out through Hessian filtering. The structural texture is commonly understood as edge feature extraction, please refer to Figure 3 After completing the above steps, it can be found that the facial contour and facial features are extracted, while the facial details such as skin texture are excluded. If the detailed texture is retained, the algorithm of the app will often sharpen the portrait indiscriminately, making the skin flaws magnified, and it will not achieve the effect of both beautifying and contour details. The role of structural texture is to guide, which will be further explained in the subsequent steps.

[0052] See also Figure 4 , the original image and the second label image are linearly fused according to the third label image to obtain the result image, and the calculation formula of the linear fusion is as follows:

[0053]

[0054] Where D st Represents the output result graph, S rc Represents the original input image, I Detail Represents the original texture information, I Mask Represents structural texture.

[0055] It needs to be further explained that the purpose of linear fusion is to complete the sharpening step. Through the above steps, it can be known that the actual purpose is to extract the texture information and structural texture in the high-frequency signal respectively, and use the position information of the structural texture information as a guide to guide and enhance the texture information of the image, while retaining the detail texture in the original image. In layman's terms, the texture information is first extracted through filtering and image subtraction. This part of the texture information will be stored in the high-frequency layer, while the corresponding light and shadow and color are retained in the low-frequency layer; the high frequency and low frequency are separated to avoid enhancing the image noise of the low-frequency skin texture during the sharpening process; and the texture information includes structural texture and detail texture, but the part that needs to be sharpened is only the structural texture information. Therefore, the present invention uses Hessian filtering to extract the structural texture information in the original image, and uses the structural texture information as a guide to determine the range and position that needs to be sharpened, so that the original skin state can be retained while enhancing the edge area of ​​the facial features of the portrait, which will not sharpen the skin defects and can improve the three-dimensional sense of the facial features.

[0056] A portrait sharpening device based on a Hessian filter, the device comprising:

[0057] An acquisition unit, used for acquiring the original image;

[0058] Processing unit: used for performing low-pass filtering on the brightness channel of the original image to obtain the first original image information, and obtain the first label image; performing image subtraction between the original image and the first label image to obtain the second original image information, and obtain the second label image; performing Hessian filtering on the original image to obtain the third original image information, and obtain the third label image; performing linear fusion of the original image and the second label image according to the third label image to obtain the result image;

[0059] Preview unit: used to preview the result graph output by the processing unit.

[0060] The present invention also proposes a portrait sharpening device based on a Hessian filter, comprising a processor, a memory, and a computer program stored in the memory, wherein the computer program can be executed by the processor to implement the portrait sharpening method based on a Hessian filter.

[0061] The present invention also proposes a computer-readable storage medium, characterized in that the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the portrait sharpening method based on the Hessian filter.

[0062] Exemplarily, the computer program may be divided into one or more units, which are stored in the memory and executed by the processor to complete the present invention. The one or more units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the portrait sharpening device based on the Hessian filter.

[0063] The portrait sharpening device based on the Hessian filter may include but is not limited to a processor and a memory. Those skilled in the art will appreciate that the schematic diagram is merely an example of a portrait sharpening device based on the Hessian filter and does not constitute a limitation on the portrait sharpening device based on the Hessian filter. The device may include more or fewer components than shown in the figure, or may combine certain components, or different components. For example, the portrait sharpening device based on the Hessian filter may also include input and output devices, network access devices, buses, etc.

[0064] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The control center of the portrait sharpening device based on the Hessian filter uses various interfaces and lines to connect various parts of the entire portrait sharpening device based on the Hessian filter.

[0065] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the portrait sharpening device based on the Hessian filter by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0066] Wherein, if the integrated unit of the portrait sharpening device based on the Hessian filter is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, the steps of the above-mentioned method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc.

[0067] The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0068] It should be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art may understand and implement it without paying any creative effort.

[0069] The implementation schemes in the above embodiments can be further combined or replaced, and the embodiments are only descriptions of the preferred embodiments of the present invention, and do not limit the concept and scope of the present invention. Without departing from the design concept of the present invention, various changes and improvements made to the technical solutions of the present invention by professional and technical personnel in this field all belong to the protection scope of the present invention.

Claims

1. A portrait sharpening method based on Hessian filter, characterized in that: The method specifically comprises: Get the original image; Gaussian filtering is performed on the brightness channel of the original image to obtain first original image information, thereby obtaining a first labeled image; the first original image information includes original image texture structure information; Subtracting the original image from the first label image to obtain second original image information, thereby obtaining a second label image; the second original image information includes original image texture information; Performing Hessian filtering on the original image to obtain third original image information, thereby obtaining a third labeled image; the third original image information includes structural texture information of the original image; The original image and the second label image are linearly fused according to the third label image to obtain a result image; The calculation formula of the linear fusion is as follows: Where D st Represents the output result graph, S rc Represents the original input image, I Detail Represents the second original image information, I Mask Indicates the third original image information.

2. The portrait sharpening method based on Hessian filter according to claim 1, characterized in that: The original image is obtained by using a smart terminal device.

3. The portrait sharpening method based on Hessian filter according to claim 1, characterized in that: The original image Hessian filtering calculation method is as follows: Where G(X,s) is the Gaussian distribution weight, ; L(X,s) is the image at point x o Taylor expansion of ; D is the two-dimensional image equal to 2, x is the distance, γ and s are equal to 1.

4. A portrait sharpening device based on a Hessian filter, characterized in that: The device comprises: An acquisition unit, used for acquiring the original image; Processing unit: used for performing Gaussian filtering on the brightness channel of the original image to obtain first original image information, and obtain a first label image; the first original image information includes original image texture structure information; Subtracting the original image from the first label image to obtain second original image information, thereby obtaining a second label image; the second original image information includes original image texture information; Performing Hessian filtering on the original image to obtain third original image information, thereby obtaining a third labeled image; the third original image information includes structural texture information of the original image; The original image and the second label image are linearly fused according to the third label image to obtain a result image; The calculation formula of the linear fusion is as follows: Where D st Represents the output result graph, S rc Represents the original input image, I Detail Represents the second original image information, I Mask represents third original image information; Preview unit: used to preview the result graph output by the processing unit.

5. A portrait sharpening device based on a Hessian filter, characterized in that: The invention comprises a processor, a memory and a computer program stored in the memory, wherein the computer program can be executed by the processor to implement a portrait sharpening method based on a Hessian filter as claimed in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute a portrait sharpening method based on a Hessian filter as described in any one of claims 1 to 3.

Images