Image processing method and device, equipment and storage medium

Abstract

The embodiment of the invention provides an image processing method and device, electronic equipment and a storage medium. The method comprises the steps of performing face recognition on a to-be-enhanced image to determine a target face; performing face image analysis on the target face to determine a face effective area; performing blurred face analysis on the face effective area to determine whether the target face is a blurred face; and performing face enhancement processing on the target face of the non-blurred face. According to the method, the effective face area is obtained by performing face image analysis on the target face and removing the related interference area, so that blurred face judgment can be more effectively performed, and the accuracy of blurred face judgment is improved. By accurately recognizing the blurred face, the blurred face which is not subjected to enhancement processing can be protected, so that the picture quality of related video contents is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image processing method, device, electronic device, storage medium and computer program product. Background Art

[0002] With the increasing popularity of short video social consumption, there are more and more videos containing faces on short video platforms. Since faces are usually the area of ​​interest for users when watching videos, image enhancement of the face area can effectively improve the user's visual experience. Based on this, image enhancement technology specifically for faces has emerged.

[0003] However, not every face in the video needs to be enhanced. For example, when a user uses a blur effect or a background blur effect, the relevant area is expected to be displayed with a blurred effect. If the face in this area is enhanced, it violates the content consistency with the original video and will bring a bad viewing experience to the user. Summary of the invention

[0004] Embodiments of the present disclosure provide an image processing method, apparatus, electronic device, storage medium, and computer program product.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an image processing method, comprising: performing face recognition on an image to be enhanced to determine at least one target face; performing face image analysis on the target face to determine a valid face area; performing blurred face analysis on the valid face area to determine whether the corresponding target face is a blurred face; in response to the target face being a non-blurred face, determining that the target face is a face to be enhanced; and performing face enhancement processing on the face to be enhanced.

[0006] In some exemplary embodiments of the present disclosure, in response to the target face being a blurred face, a corresponding area in the image to be enhanced is retained.

[0007] In some exemplary embodiments of the present disclosure, performing facial image analysis on the target face to determine the effective face area includes: performing text content recognition on the facial area corresponding to the target face to determine the text content area; generating a text mask map based on the text content area; and determining the effective face area based on the facial area and the text mask map.

[0008] In some exemplary embodiments of the present disclosure, performing facial image analysis on the target face to determine the effective face area includes: performing occluder recognition on the facial area corresponding to the target face to determine the occluder area; generating an occluder mask map based on the occluder area; and determining the effective face area based on the facial area and the occluder mask map.

[0009] In some exemplary embodiments of the present disclosure, performing facial image analysis on the target face to determine the effective face area includes: performing facial analysis on the face area corresponding to the target face, and marking the face area for each pixel in the face area; the face area marks respectively correspond to a preset face area category; generating a face mask map based on the face area marks of each pixel in the face area; and determining the effective face area based on the face area and the face mask map.

[0010] In some exemplary embodiments of the present disclosure, the method further includes: performing corrosion processing on the effective face area to obtain the corroded effective face area.

[0011] In some exemplary embodiments of the present disclosure, the blurred face analysis is performed on the effective face area to determine whether the corresponding target face is a blurred face, including: performing Laplace filtering on the effective face area to extract first face edge information; performing edge variance calculation based on the first face edge information to obtain a first blurred eigenvalue; in response to the first blurred eigenvalue being greater than or equal to a first blurred threshold, determining that the corresponding target face is a non-blurred face; in response to the first blurred eigenvalue being less than the first blurred threshold, determining that the corresponding target face is a blurred face.

[0012] In some exemplary embodiments of the present disclosure, the performing of blurred face analysis on the effective face area to determine whether the corresponding target face is a blurred face also includes: blurring the effective face area to obtain a blurred face effective area; performing Laplace filtering on the blurred face effective area to extract second face edge information; calculating edge variance based on the second face edge information to obtain a second blurred eigenvalue; and comparing the first blurred eigenvalue and the second blurred eigenvalue to determine whether the corresponding target face is a blurred face.

[0013] In some exemplary embodiments of the present disclosure, the comparison of the first blur feature value and the second blur feature value to determine whether the corresponding target face is a blurred face includes: calculating the difference between the first blur feature value and the second blur feature value to obtain a feature difference; in response to the feature difference being greater than or equal to a second blur threshold, determining that the corresponding target face is a non-blurred face; in response to the feature difference being less than the second blur threshold, determining that the corresponding target face is a blurred face.

[0014] In some exemplary embodiments of the present disclosure, performing face recognition on the image to be enhanced to determine at least one target face includes: performing face recognition on the image to be enhanced to determine at least one pending face and a corresponding face confidence; in response to the face confidence being greater than a preset confidence threshold, determining the pending face to be the target face.

[0015] According to a second aspect of an embodiment of the present disclosure, there is provided an image processing apparatus, comprising: a face recognition module, configured to perform face recognition on an image to be enhanced, and determine at least one target face; a face image parsing module, configured to perform face image parsing on the target face, and determine a valid face area; a blurred face analysis module, configured to perform blurred face analysis on the valid face area, and determine whether the corresponding target face is a blurred face; a to-be-enhanced face determination module, configured to determine that the target face is a to-be-enhanced face in response to the target face being a non-blurred face; and a face enhancement module, configured to perform face enhancement processing on the to-be-enhanced face.

[0016] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, characterized in that it includes: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the executable instructions to implement any one of the image processing methods described.

[0017] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute any one of the image processing methods described.

[0018] According to a fifth aspect of an embodiment of the present disclosure, there is provided a computer program product, comprising a computer program / instruction, wherein the computer program / instruction implements any one of the image processing methods when executed by a processor.

[0019] The image processing method provided by the embodiment of the present disclosure performs face recognition on the image to be enhanced to determine the target face; performs face image analysis on the target face to determine the effective face area; performs blurred face analysis on the effective face area to determine whether the target face is a blurred face; and performs face enhancement processing on the target face that is not a blurred face. The method performs face image analysis on the target face, removes the effective face area obtained by the relevant interference area, and can more effectively perform blurred face judgment, thereby improving the accuracy of blurred face judgment. By accurately identifying the blurred face, the blurred face that should not be enhanced can be protected, thereby improving the picture quality of the relevant video content.

[0020] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0022] Figure 1 A schematic diagram showing an exemplary system architecture to which the method according to an embodiment of the present disclosure can be applied is shown.

[0023] Figure 2 The figure is a flowchart of an image processing method according to an exemplary embodiment.

[0024] Figure 3 The following is a flow chart of a facial image analysis method according to an exemplary embodiment. Figure 1 .

[0025] Figure 4 The following is a flow chart of a facial image analysis method according to an exemplary embodiment. Figure 2 .

[0026] Figure 5 The following is a flow chart of a facial image analysis method according to an exemplary embodiment. Figure 3 .

[0027] Figure 6 The following is a flow chart of a method for analyzing a blurred face according to an exemplary embodiment. Figure 1 .

[0028] Figure 7 The following is a flow chart of a method for analyzing a blurred face according to an exemplary embodiment. Figure 2 .

[0029] Figure 8 is a block diagram of an image processing apparatus according to an exemplary embodiment.

[0030] Fig. 9 It is a schematic diagram showing the structure of an electronic device suitable for implementing the exemplary embodiments of the present disclosure according to an exemplary embodiment. DETAILED DESCRIPTION

[0031] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0032] The features, structures or characteristics described in the present disclosure may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.

[0033] The accompanying drawings are only schematic diagrams of the present disclosure, and the same reference numerals in the drawings represent the same or similar parts, so their repeated description will be omitted. Some block diagrams shown in the accompanying drawings do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in at least one hardware module or integrated circuit, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0034] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and steps, nor must they be executed in the order described. For example, some steps can be decomposed, and some steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0035] In this specification, the terms "a", "an", "the", "said" and "at least one" are used to indicate the presence of at least one element / component / etc.; the terms "comprising", "including" and "having" are used to express an open-ended inclusion and mean that additional elements / components / etc. may exist in addition to the listed elements / components / etc.; the terms "first", "second" and "third" etc. are used only as labels and are not intended to limit the quantity of their objects.

[0036] Figure 1 A schematic diagram showing an exemplary system architecture to which the method according to an embodiment of the present disclosure can be applied is shown.

[0037] like Figure 1 As shown, the system architecture may include a server 101, a network 102, a terminal device 103, a terminal device 104, and a terminal device 105. The network 102 is used to provide a medium for a communication link between the terminal device 103, the terminal device 104, or the terminal device 105 and the server 101. The network 102 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0038] The server 101 may be a server that provides various services, such as a background management server that provides support for devices operated by users using the terminal device 103, the terminal device 104, or the terminal device 105. The background management server may analyze and process the received request and other data, and feed back the processing results to the terminal device 103, the terminal device 104, or the terminal device 105.

[0039] Terminal device 103, terminal device 104 and terminal device 105 can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a wearable smart device, a virtual reality device, an augmented reality device, etc., but are not limited thereto.

[0040] It should be understood that Figure 1 The number of terminal devices 103, terminal devices 104, terminal devices 105, networks 102 and servers 101 are merely illustrative. Server 101 may be a physical server, a server cluster consisting of multiple servers, or a cloud server. Depending on actual needs, any number of terminal devices, networks and servers may be provided.

[0041] Below, each step of the method in the exemplary embodiment of the present disclosure will be described in more detail with reference to the accompanying drawings and embodiments.

[0042] Figure 2 The figure is a flowchart of an image processing method according to an exemplary embodiment. Figure 2The method provided in the embodiment is applied to the client and can be executed by any electronic device, such as the above Figure 1 The terminal device in Figure 1 Servers in, or Figure 1 The terminal device and the server in the embodiment are jointly executed, but the present disclosure is not limited to this.

[0043] In step S210, face recognition is performed on the image to be enhanced to determine at least one target face.

[0044] As mentioned above, in the related art, image enhancement technology cannot distinguish which faces in the image need to be enhanced and which do not. As a result, some facial images that need to be blurred are enhanced, which violates the content consistency with the original video and brings a bad viewing experience to users. The image processing method provided by the present disclosure solves the problem of identifying blurred faces in image and video scenes, protects faces that should not be enhanced, and maintains the content consistency of the video while improving the image quality of facial multimedia content consumption.

[0045] In the disclosed embodiment, face recognition is performed on the image to be enhanced to determine at least one target face. The image to be enhanced may be a picture or any image frame in a video. The image to be enhanced contains a face image and needs to be enhanced. By using face recognition technology, at least one target face contained in the image to be enhanced is identified for subsequent determination of whether to perform face enhancement processing on the target face. Face recognition technology is a well-known technology in the art and there are many ways to implement it, which are not specifically limited here.

[0046] In an exemplary embodiment, face recognition is performed on the image to be enhanced by face recognition technology. In the image to be enhanced, at least one pending face and the face confidence corresponding to each pending face are determined. The face confidence is used to represent the probability value of the pending face being recognized as a face. According to a preset confidence threshold, each pending face is preliminarily screened, and the pending face with a face confidence greater than the confidence threshold is determined as the target face. In this way, each pending face can be preliminarily screened based on the face confidence, and faces with low confidence can be directly ignored, and subsequent blurred face judgment is not performed, and face enhancement processing is not performed on the relevant faces. Through this preliminary screening process, the amount of data processing for subsequent blurred face judgment can be reduced.

[0047] In step S220, facial image analysis is performed on the target face to determine a valid facial area.

[0048] In the disclosed embodiment, the face area corresponding to the target face is extracted to perform blurred face judgment. Usually, the extracted face area is a rectangular area including the face image. The face area contains non-face image content. At the same time, short videos often have subtitles or stickers and other content to block the face image. These blocking information will also affect the subsequent blurred face judgment. For example, some subtitles are superimposed on the blurred face, and the subtitles are edited later, and their edges are clear. In this way, the pixel edge variance increases when the blurred face is judged, which may cause misjudgment. For another example, the blurred face is superimposed with some special effects stickers, and the stickers themselves are edited, and their edges are clear, which will also cause similar effects.

[0049] In the disclosed embodiment, the face area corresponding to the target face is not directly subjected to the blurred face judgment, but the face image analysis is first performed on the target face. Through the face image analysis, the face area is subjected to image analysis, and the non-face area, the occluded area, and the area interfering with the blurred face judgment are removed, and finally the effective face area is determined. By removing the effective face area obtained by the relevant interference area, the blurred face judgment can be performed more effectively, thereby improving the accuracy of the blurred face judgment.

[0050] In step S230, a blurred face analysis is performed on the effective face area to determine whether the corresponding target face is a blurred face.

[0051] In the disclosed embodiment, the above-obtained effective face area is subjected to a blurred face analysis. The effective face area is analyzed and judged by a preset blurred face analysis algorithm to determine whether the corresponding target face is a blurred face. That is, it is determined whether the target face has been blurred using a blur effect or blurred using a background blur effect.

[0052] In step S240, in response to the target face being a non-blurred face, the target face is determined to be a face to be enhanced.

[0053] In the disclosed embodiment, the blurred face analysis algorithm is used to determine whether the target face is a blurred face. If the target face is a non-blurred face, it means that the target face has not been blurred and blurred in advance, and the target face is determined to be a face to be enhanced. That is, the target face needs to be enhanced to optimize the image quality of the relevant target face area. If the target face is a blurred face, it means that the target face has been blurred and blurred in advance, and the target face is determined to be a blurred face.

[0054] In the disclosed embodiment, the blurred face is protected and the target face is not enhanced to keep the blurring effect of the area consistent with the surrounding area, thereby avoiding a bad viewing experience for the user.

[0055] In an exemplary embodiment, in response to the target face being a blurred face, it means that the target face has been blurred using a blur effect, or blurred using a background blur effect. Therefore, the area corresponding to the target face is not enhanced, and the area corresponding to the target face in the image to be enhanced is directly retained. In this way, the blurred face that should not be enhanced can be protected, thereby improving the picture quality of the relevant video content.

[0056] In step S250, face enhancement processing is performed on the face to be enhanced.

[0057] In the disclosed embodiment, the face to be enhanced is subjected to face enhancement processing to optimize the image quality of the relevant target face region. The face enhancement processing algorithm is a technology well known in the art, and there are many implementation methods, which are not specifically limited here. At the same time, the enhanced face region is re-fused with other regions in the image to be enhanced to form an enhanced image.

[0058] The image processing method provided by the embodiment of the present disclosure performs face recognition on the image to be enhanced to determine the target face; performs face image analysis on the target face to determine the effective face area; performs blurred face analysis on the effective face area to determine whether the target face is a blurred face; and performs face enhancement processing on the target face that is not a blurred face. The method performs face image analysis on the target face, removes the effective face area obtained by the relevant interference area, and can more effectively perform blurred face judgment, thereby improving the accuracy of blurred face judgment. By accurately identifying the blurred face, the blurred face that should not be enhanced can be protected, thereby improving the picture quality of the relevant video content.

[0059] Figure 3 The following is a flow chart of a facial image analysis method according to an exemplary embodiment. Figure 1 In the embodiment of the present disclosure, Figure 2 Based on the image processing method shown, the aforementioned step S220 may include the following steps.

[0060] In step S310, text content recognition is performed on the face area corresponding to the target face to determine the text content area.

[0061] As mentioned above, short videos often contain text information such as subtitles. When there is relevant text information in the face area corresponding to the target face, it will interfere with the judgment of the blurred face and easily lead to misjudgment.

[0062] In the disclosed embodiment, the face area corresponding to the target face is subjected to text content recognition to determine the area range involving text content in the face area. The text content recognition technology can be implemented in a variety of ways, and can be implemented based on OCR technology or based on a pre-trained text recognition model, which is not specifically limited here. The text content area can be an area range that extends outward by a certain range based on the recognized text content.

[0063] In step S320, a text mask map is generated according to the text content area.

[0064] In the disclosed embodiment, a text mask image is generated based on the text content area. The so-called mask image is an image that hides or blocks a part of the area in the image.

[0065] In step S330, the effective face area is determined according to the face area and the text mask image.

[0066] In the disclosed embodiment, the text content area in the face area is shielded by the face area corresponding to the target face and the text mask image to remove the text content related area, thereby determining the effective face area. By removing the interference of the text content area, the blurred face judgment is performed based on the effective face area, which can reduce the probability of misjudgment and improve the accuracy of judgment.

[0067] Figure 4 The following is a flow chart of a facial image analysis method according to an exemplary embodiment. Figure 2 In the embodiment of the present disclosure, Figure 2 Based on the image processing method shown, the aforementioned step S220 may include the following steps.

[0068] In step S410, occluder recognition is performed on the face area corresponding to the target face to determine the occluder area.

[0069] As mentioned above, since stickers and other occluding contents are often added to the face area in short videos, when there are relevant occluding objects in the face area corresponding to the target face, it will interfere with the judgment of the blurred face and easily lead to misjudgment.

[0070] In the disclosed embodiment, the face area corresponding to the target face is subjected to occlusion recognition to determine the area range of the face area involving the occlusion. The occlusion recognition technology can be implemented in a variety of ways, and can be implemented based on face analysis technology or based on a pre-trained occlusion recognition model, which is not specifically limited here. The occlusion area can be an area range that extends outward by a certain range based on the identified occlusion.

[0071] In step S420, an occluder mask map is generated according to the occluder region.

[0072] In the disclosed embodiment, an occluder mask image is generated based on the occluder region. The so-called mask image is an image that hides or blocks a part of the image.

[0073] In step S430, the effective face area is determined according to the face area and the occluder mask image.

[0074] In the disclosed embodiment, the occluder area in the face area is blocked by the face area corresponding to the target face and the occluder mask image, so as to remove the occluder-related area, thereby determining the effective face area. By removing the interference of the occluder area, the blurred face judgment is performed based on the effective face area, which can reduce the probability of misjudgment and improve the accuracy of judgment.

[0075] Figure 5 The following is a flow chart of a facial image analysis method according to an exemplary embodiment. Figure 3 In the embodiment of the present disclosure, Figure 2 Based on the image processing method shown, the aforementioned step S220 may include the following steps.

[0076] In step S510, face analysis is performed on the face region corresponding to the target face, and face region marking is performed on each pixel in the face region; the face region markings respectively correspond to a preset face region category.

[0077] Since the human face includes many different parts. Different parts of the face have different effects on the judgment of blurred faces. Therefore, different parts of the face can be screened, and the parts of the face that have a more significant effect on the judgment of blurred faces can be pre-selected for blurred face judgment. For example, the parts of the face can include: hair, forehead, face, eyes, nose, ears, mouth, decorations, etc. Based on the previous analysis, it is believed that hair does not have much effect on the judgment of blurred faces, and decorations may interfere with the judgment of blurred faces. Therefore, when generating a face mask map, only the areas corresponding to the forehead, face, eyes, nose, ears, mouth and other parts are selected to generate the corresponding mask map.

[0078] In the disclosed embodiment, a face analysis is performed on the face region corresponding to the target face by a pre-trained face analysis model to identify different face parts in the face region. Based on the face analysis, the face analysis model performs face region marking on each pixel in the face region. The face region markings correspond to a pre-set face region category. By marking the face region of each pixel, the face region category corresponding to the pixel, i.e., the face part, can be determined.

[0079] In the disclosed embodiment, before the face parsing model is trained, multiple face region categories (i.e., face parts) are determined. After model training, the face parsing model parses the face region image, determines the face region category to which each pixel in the face region image belongs, and marks each pixel as a face region label corresponding to the face region category. The face region label corresponds to the face region category one by one.

[0080] In step S520, a face mask image is generated according to the face region label of each pixel in the face region.

[0081] In the disclosed embodiment, based on the face region labels of each pixel in the face region, the region ranges of different parts of the face are determined. Based on the face parts pre-selected for blurred face recognition, a face mask image is generated. The so-called mask image is an image that hides or blocks part of the image.

[0082] In step S530, the effective face area is determined according to the face area and the face mask image.

[0083] In the disclosed embodiment, the area corresponding to the non-selected face parts in the face area is shielded by the face area corresponding to the target face and the face mask image, so as to remove the relevant area, thereby determining the effective face area. By removing the area corresponding to the non-selected face parts, the effective face area is more concentrated in the area corresponding to the pre-selected face parts, which can reduce the probability of misjudgment in the blurred face judgment and improve the judgment accuracy. It should be noted that by generating a mask image through this method, non-face area parts such as the background in the face area can be filtered out at the same time, further improving the effectiveness of the effective face area.

[0084] It should be pointed out that the aforementioned Figure 3 , 4 The embodiments shown in , 5 respectively provide different mask image generation schemes for text content, occluders and face parts. Through these different embodiments, the interference area in the face area can be removed from different angles, so as to determine the effective face area. By filtering the face area, the probability of misjudgment in the blurred face judgment is reduced, and the judgment accuracy is improved. In practical applications, according to the needs of the actual use scenario, a combination of one or more mask strategies can be selected to filter the face area, which should be regarded as within the protection scope of the present disclosure.

[0085] In some embodiments of the present disclosure, in order to further eliminate the influence of the above-mentioned related interference factors, the effective face area obtained in the above-mentioned implementation method can also be corroded to obtain the corroded effective face area. The so-called corrosion process refers to removing a certain number of pixel contents along the edge of the image. By performing the above corrosion process, the effective face area can be further optimized to avoid the influence of mis-segmentation at the edge on the blurred face judgment.

[0086] The image processing method provided by the embodiment of the present disclosure uses a variety of different facial image analysis methods to identify text content, occlusion content and different facial parts in the facial area. By removing the text content area, non-face area, occlusion area and other areas that interfere with the blur judgment in the target face, the effective face area is finally determined. By removing the effective face area obtained by removing the relevant interference area, the blurred face judgment can be performed more effectively, thereby improving the accuracy of the blurred face judgment.

[0087] Figure 6 The following is a flow chart of a method for analyzing a blurred face according to an exemplary embodiment. Figure 1 In the embodiment of the present disclosure, Figure 2 Based on the image processing method shown, the aforementioned step S230 may include the following steps.

[0088] In step S610, Laplace filtering is performed on the effective face area to extract first face edge information.

[0089] In the disclosed embodiment, the above-obtained effective face area is subjected to Laplacian filtering. Through Laplacian filtering, the details and edge features in the image can be highlighted, making the edge in the image more obvious, thereby extracting the first face edge information from the effective face area. The face edge information is used to represent the face edge area.

[0090] In step S620, edge variance calculation is performed based on the first face edge information to obtain a first blurring feature value.

[0091] In the disclosed embodiment, edge variance calculation is performed on pixels in the edge region of the face according to the first face edge information to obtain a first blur feature value. Through the edge variance calculation, the first blur feature value is used to characterize the gradient change between pixels in the edge region of the face. The larger the gradient change, the clearer the image. Conversely, the smaller the gradient change, the more blurred the image.

[0092] In step S630, in response to the first blurring feature value being greater than or equal to a first blurring threshold, it is determined that the corresponding target face is a non-blurred face.

[0093] In the embodiment of the present disclosure, in response to the first blur feature value being greater than or equal to the first blur threshold, it indicates that the image of the edge area of ​​the face is relatively clear, thereby determining that the target face has not been pre-blurred, that is, determining that the target face is a non-blurred face.

[0094] In step S640, in response to the first blurring feature value being less than the first blurring threshold, it is determined that the corresponding target face is a blurred face.

[0095] In the disclosed embodiment, in response to the first blur feature value being less than the first blur threshold, it indicates that the image of the edge area of ​​the face is relatively blurred, thereby determining that the target face may have been blurred in advance, that is, determining that the target face is a blurred face.

[0096] The image processing method provided by the embodiment of the present disclosure performs blurred face judgment based on the effective face area. By calculating the edge variance of the face edge information in the effective face area, the clarity of the face edge area can be determined, and then it can be determined whether the target person is a blurred face or a non-blurred face.

[0097] Figure 7 The following is a flow chart of a method for analyzing a blurred face according to an exemplary embodiment. Figure 2 .

[0098] In the disclosed embodiment, Figure 7 In the blurred face analysis method shown in FIG. Figure 6 Steps S610 and S620 in the blurred face analysis method are corresponding to each other and will not be repeated here.

[0099] In the embodiment of the present disclosure, Figure 6 Based on the blurred face analysis method shown, Figure 7 The blurred face analysis method shown may further include the following steps.

[0100] In step S730, the effective face area is blurred to obtain a blurred effective face area.

[0101] In the embodiment of the present disclosure, as Figure 6The supplement of the blurred face analysis method shown in the embodiment further improves the accuracy of blurred face judgment. The core idea of ​​the blurred face analysis method provided in this embodiment is to artificially superimpose a blurred layer on the image to be analyzed, and compare the image difference before and after superimposing the blurred layer. If the image to be analyzed is originally blurred (that is, it has been blurred), the image will still be blurred after superimposing the blurred layer, and the image difference before and after superimposing will not change much. On the contrary, if the image to be analyzed is originally clear (that is, it has not been blurred), the image will become blurred after superimposing the blurred layer, and the image difference before and after superimposing will change greatly. Based on this, this embodiment further performs blurred face analysis and judgment on the effective face area.

[0102] In the disclosed embodiment, the effective face area obtained above is firstly subjected to blurring processing to obtain a blurred effective face area. The blurring processing is to perform consistent blurring processing on the entire effective face area. In an exemplary embodiment, the blurring processing can be implemented by using Gaussian blurring processing technology.

[0103] In step S740, Laplace filtering is performed on the blurred face effective area to extract the second face edge information.

[0104] In the embodiment of the present disclosure, Laplace filtering is performed on the blurred face effective area after blurring to extract the edge information of the second face. This processing is similar to the aforementioned step S610 and will not be repeated here.

[0105] In step S750, edge variance is calculated based on the second face edge information to obtain a second blurring feature value.

[0106] In the embodiment of the present disclosure, edge variance calculation is performed on pixels in the face edge region according to the second face edge information to obtain a second blur feature value. This processing is similar to the aforementioned step S620 and will not be repeated here.

[0107] In step S760, the first blur feature value and the second blur feature value are compared to determine whether the corresponding target face is a blurred face.

[0108] As mentioned above, if the effective face area is originally blurry, the difference between the images before and after superposition is not much. On the contrary, if the effective face area is originally clear, the difference between the images before and after superposition is large. In the disclosed embodiment, by comparing the first blur feature value and the second blur feature value, it can be determined whether the target face corresponding to the effective face area is a blurred face.

[0109] like Figure 7 As shown, step S760 may include the following steps.

[0110] In step S761, a difference between the first virtualized feature value and the second virtualized feature value is calculated to obtain a feature difference value.

[0111] In the embodiment of the present disclosure, based on the first blur feature value obtained in the aforementioned step S620 and the second blur feature value obtained in step S750, the difference between the two blur feature values ​​is calculated to obtain a feature difference value. The feature difference value represents the difference in the image before and after the blur layer is superimposed.

[0112] In step S762, in response to the feature difference being greater than or equal to the second blur threshold, the corresponding target face is determined to be a non-blurred face.

[0113] In the embodiment of the present disclosure, in response to the feature difference being greater than or equal to the second blur threshold, it indicates that the image difference before and after superposition of the effective area of ​​the face is significantly different, thereby determining that the target face has not been pre-blurred, that is, determining that the target face is a non-blurred face.

[0114] In step S763, in response to the feature difference being less than the second blur threshold, it is determined that the corresponding target face is a blurred face.

[0115] In the disclosed embodiment, in response to the feature difference being less than the second blur threshold, it indicates that the image difference before and after superposition of the effective face area does not change much, thereby determining that the target face may have been blurred in advance, that is, determining that the target face is a blurred face.

[0116] The image processing method provided by the embodiment of the present disclosure performs blurring processing on the effective face area and performs blurred face judgment based on the blurred effective face area. The method supplements the blurred face judgment, thereby further improving the accuracy of the blurred face judgment.

[0117] The blurred face is judged based on the effective face area. By calculating the edge variance of the face edge information in the effective face area, the clarity of the face edge area can be determined, and then it can be determined whether the target person is a blurred face or a non-blurred face.

[0118] The following are embodiments of the device disclosed herein, which can be used to execute the method embodiments disclosed herein. For details not disclosed in the device embodiments disclosed herein, please refer to the method embodiments disclosed herein.

[0119] Figure 8 is a block diagram of an image processing device according to an exemplary embodiment. Figure 8 The image processing device 800 may include: a face recognition module 810, a face image parsing module 820, a blurred face analysis module 830, a to-be-enhanced face determination module 840 and a face enhancement module 850.

[0120] The face recognition module 810 is configured to perform face recognition on the image to be enhanced and determine at least one target face.

[0121] The face image analysis module 820 is configured to perform face image analysis on the target face and determine a valid face area.

[0122] The blurred face analysis module 830 is configured to perform blurred face analysis on the effective face area to determine whether the corresponding target face is a blurred face.

[0123] The to-be-enhanced face determination module 840 is configured to determine, in response to the target face being a non-blurred face, that the target face is a to-be-enhanced face.

[0124] The face enhancement module 850 is configured to perform face enhancement processing on the face to be enhanced.

[0125] In some exemplary embodiments of the present disclosure, the to-be-enhanced face determination module 840 is further configured to retain a corresponding area in the to-be-enhanced image in response to the target face being a blurred face.

[0126] In some exemplary embodiments of the present disclosure, the facial image parsing module 820 is further configured to perform text content recognition on the facial area corresponding to the target face to determine the text content area; generate a text mask map based on the text content area; and determine the effective facial area based on the facial area and the text mask map.

[0127] In some exemplary embodiments of the present disclosure, the facial image parsing module 820 is further configured to perform occluder recognition on the facial area corresponding to the target face to determine the occluder area; generate an occluder mask map based on the occluder area; and determine the effective facial area based on the facial area and the occluder mask map.

[0128] In some exemplary embodiments of the present disclosure, the face image parsing module 820 is further configured to perform face analysis on the face area corresponding to the target face, and mark the face area for each pixel in the face area; the face area marks respectively correspond to a preset face area category; generate a face mask map based on the face area marks of each pixel in the face area; and determine the effective face area based on the face area and the face mask map.

[0129] In some exemplary embodiments of the present disclosure, the face image parsing module 820 is further configured to perform corrosion processing on the effective face area to obtain the corroded effective face area.

[0130] In some exemplary embodiments of the present disclosure, the blurred face analysis module 830 is further configured to perform Laplace filtering on the effective face area to extract first face edge information; perform edge variance calculation based on the first face edge information to obtain a first blurred feature value; in response to the first blurred feature value being greater than or equal to a first blurred threshold, determine that the corresponding target face is a non-blurred face; in response to the first blurred feature value being less than the first blurred threshold, determine that the corresponding target face is a blurred face.

[0131] In some exemplary embodiments of the present disclosure, the blurred face analysis module 830 is further configured to blur the effective face area to obtain a blurred face effective area; perform Laplace filtering on the blurred face effective area to extract second face edge information; perform edge variance calculation based on the second face edge information to obtain a second blurred eigenvalue; compare the first blurred eigenvalue and the second blurred eigenvalue to determine whether the corresponding target face is a blurred face.

[0132] In some exemplary embodiments of the present disclosure, the blurred face analysis module 830 is further configured to calculate the difference between the first blurred feature value and the second blurred feature value to obtain a feature difference; in response to the feature difference being greater than or equal to a second blurred threshold, determine that the corresponding target face is a non-blurred face; in response to the feature difference being less than the second blurred threshold, determine that the corresponding target face is a blurred face.

[0133] In some exemplary embodiments of the present disclosure, the face recognition module 810 is further configured to perform face recognition on the image to be enhanced, determine at least one pending face and a corresponding face confidence; in response to the face confidence being greater than a preset confidence threshold, determine that the pending face is the target face.

[0134] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0135] Refer to the following Fig. 9 hereinafter describes an electronic device 900 according to such an embodiment of the present disclosure. Fig. 9 The electronic device 900 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0136] like Fig. 9As shown, the electronic device 900 is in the form of a general computing device. The components of the electronic device 900 may include, but are not limited to: the at least one processing unit 910, the at least one storage unit 920, a bus 930 connecting different system components (including the storage unit 920 and the processing unit 910), and a display unit 940.

[0137] The storage unit stores program codes, which can be executed by the processing unit 910, so that the processing unit 910 performs the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 910 can perform the following steps: Figure 2 The steps shown.

[0138] For example, electronic devices can achieve Figure 2 The steps shown.

[0139] The storage unit 920 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 921 and / or a cache memory unit 922 , and may further include a read-only memory unit (ROM) 923 .

[0140] The storage unit 920 may also include a program / utility 924 having a set (at least one) of program modules 925, such program modules 925 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0141] Bus 930 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0142] The electronic device 900 may also communicate with one or more external devices 970 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 900, and / or communicate with any device that enables the electronic device 900 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 950. Furthermore, the electronic device 900 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 960. As shown, the network adapter 960 communicates with other modules of the electronic device 900 via a bus 930. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0143] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiment of the present disclosure.

[0144] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the above instructions can be executed by a processor of the device to complete the above method. Optionally, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0145] In an exemplary embodiment, a computer program product is also provided, including a computer program / instruction, and when the computer program / instruction is executed by a processor, the method in the above embodiment is implemented.

[0146] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0147] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that: include: Performing face recognition on the image to be enhanced to determine at least one target face; Performing facial image analysis on the target face to determine a valid facial area; Performing a blurred face analysis on the effective face area to determine whether the corresponding target face is a blurred face; In response to the target face being a non-blurred face, determining the target face to be a face to be enhanced; Performing face enhancement processing on the face to be enhanced.

2. The method according to claim 1, characterized in that The method further comprises: In response to the target face being a blurred face, a corresponding area in the image to be enhanced is retained.

3. The method according to claim 1, characterized in that The performing facial image analysis on the target face to determine the effective face area includes: Performing text content recognition on the face area corresponding to the target face to determine the text content area; Generate a text mask image according to the text content area; The effective face area is determined according to the face area and the text mask image.

4. The method according to claim 1, characterized in that: The performing facial image analysis on the target face to determine the effective face area includes: Performing occlusion recognition on the face area corresponding to the target face to determine the occlusion area; Generate an occluder mask map according to the occluder area; The effective face area is determined according to the face area and the occluder mask image.

5. The method according to claim 1, characterized in that The performing facial image analysis on the target face to determine the effective face area includes: By performing face analysis on the face area corresponding to the target face, each pixel in the face area is marked as a face area; the face area marks respectively correspond to a preset face area category; Generate a face mask image according to the face area mark of each pixel in the face area; The effective face area is determined according to the face area and the face mask image.

6. The method according to claim 2, 3 or 4, characterized in that: The method further comprises: The effective face area is corroded to obtain an corroded effective face area.

7. The method according to claim 1, characterized in that The performing a blurred face analysis on the effective face area to determine whether the corresponding target face is a blurred face includes: Performing Laplace filtering on the effective face area to extract first face edge information; Calculate the edge variance based on the first face edge information to obtain a first blurring feature value; In response to the first blur feature value being greater than or equal to a first blur threshold, determining that the corresponding target face is a non-blurred face; In response to the first blurring feature value being less than the first blurring threshold, it is determined that the corresponding target face is a blurred face.

8. The method according to claim 7, characterized in that The performing of a blurred face analysis on the effective face area to determine whether the corresponding target face is a blurred face also includes: Performing fuzzy processing on the effective face area to obtain a fuzzy effective face area; Performing Laplace filtering on the blurred face effective area to extract edge information of the second face; Calculate the edge variance based on the second face edge information to obtain a second blurring feature value; The first blur feature value and the second blur feature value are compared to determine whether the corresponding target face is a blurred face.

9. The method according to claim 8, characterized in that The comparing the first blur feature value and the second blur feature value to determine whether the corresponding target face is a blur face includes: Calculating a difference between the first virtualized feature value and the second virtualized feature value to obtain a feature difference; In response to the feature difference being greater than or equal to a second blur threshold, determining that the corresponding target face is a non-blurred face; In response to the feature difference being less than the second blur threshold, determining that the corresponding target face is a blurred face.

10. The method according to claim 1, characterized in that The step of performing face recognition on the image to be enhanced to determine at least one target face comprises: Performing face recognition on the image to be enhanced to determine at least one pending face and a corresponding face confidence level; In response to the face confidence being greater than a preset confidence threshold, the pending face is determined to be the target face.

11. An image processing device, characterized in that: include: A face recognition module is configured to perform face recognition on the image to be enhanced and determine at least one target face; A face image analysis module is configured to perform face image analysis on the target face to determine a valid face area; A blurred face analysis module is configured to perform blurred face analysis on the effective face area to determine whether the corresponding target face is a blurred face; a to-be-enhanced face determination module, configured to, in response to the target face being a non-blurred face, determine that the target face is a to-be-enhanced face; The face enhancement module is configured to perform face enhancement processing on the face to be enhanced.

12. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the executable instructions to implement the image processing method according to any one of claims 1 to 10. 13 . A computer-readable storage medium, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image processing method according to any one of claims 1 to 10.

14. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the image processing method according to any one of claims 1 to 10 is implemented.

Images