Image processing method, device, intelligent terminal and storage medium

Abstract

The present invention discloses an image processing method, device, intelligent terminal and storage medium, wherein the image processing method comprises: obtaining an image to be processed containing a human face; obtaining an image of a region of interest of the human face to be processed based on the image to be processed; determining the relationship between the pixel value of each pixel in the image of the region of interest of the human face to be processed and a preset pixel threshold, and taking pixels less than the preset pixel threshold as target pixels; performing a linear transformation on the pixel value of each of the target pixels based on a preset objective function, wherein the pixel value of the target pixel after the linear transformation is greater than the pixel value before the linear transformation. Compared with the prior art, the pixel value of the target pixel with a smaller pixel value is linearly transformed to increase it, thereby increasing the brightness of the corresponding pixel, so that the image meets the brightness requirements of the subsequent algorithm or model, thereby improving the recognition effect. The transformed image is adapted to the original model, and there is no need to retrain the model, which can save computing resources.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method, device, intelligent terminal and storage medium. Background Art

[0002] With the advancement of science and technology, image processing technology has also developed rapidly. For example, tasks such as facial recognition can be performed based on acquired images. Currently, the processing and application of acquired images containing faces are becoming increasingly widespread. However, images containing faces may suffer from insufficient lighting, affecting recognition performance and even failing to meet the requirements of the algorithms or models used to perform recognition.

[0003] In existing technologies, images that meet the requirements are typically obtained by adjusting the lighting and re-acquiring the image. However, this method is not time-efficient and the images obtained after adjusting the lighting are not compatible with the original model. Therefore, the corresponding model needs to be retrained after adjusting the lighting, which is not conducive to saving computing resources and affects the user experience. Summary of the Invention

[0004] The main purpose of the present invention is to provide an image processing method, device, intelligent terminal and storage medium, aiming to solve the problem that the method of adjusting the lighting to re-acquire the image in the prior art is not conducive to saving time, and the image obtained after adjusting the lighting is not compatible with the original model. Therefore, the corresponding model needs to be retrained after adjusting the lighting, which is not conducive to saving computing resources and affects the user experience.

[0005] In order to achieve the above object, the present invention provides an image processing method in a first aspect, wherein the method comprises:

[0006] Get the image to be processed containing the face;

[0007] Based on the above image to be processed, an image of a region of interest of the face to be processed is obtained;

[0008] Determine the relationship between the pixel value of each pixel in the image of the face region of interest to be processed and a preset pixel threshold, and use pixels smaller than the preset pixel threshold as target pixels;

[0009] The pixel values ​​of the target pixels are linearly transformed based on a preset target function, and the pixel values ​​of the target pixels after the linear transformation are greater than the pixel values ​​before the linear transformation.

[0010] Optionally, the step of obtaining the image of the facial region of interest to be processed based on the image to be processed includes:

[0011] Detect the above-mentioned image to be processed based on the face detection model to obtain an image of the face region of interest;

[0012] Determine whether the face in the face region of interest image is blocked;

[0013] When the face in the above-mentioned facial region of interest image is not blocked, the average illumination value corresponding to the above-mentioned facial region of interest image is obtained. When the above-mentioned average illumination value is less than a preset illumination threshold, the above-mentioned facial region of interest image is used as the facial region of interest image to be processed.

[0014] Optionally, the above-mentioned average illumination value is the average illumination value of the lower half of the face in the above-mentioned facial region of interest image.

[0015] Optionally, the determining whether the face in the face region of interest image is blocked includes:

[0016] Detecting the above-mentioned facial region of interest image based on the facial attribute model to determine whether the facial object in the above-mentioned facial region of interest image is wearing a mask;

[0017] When the face object wears a mask, it is determined that the face in the face region of interest image is blocked;

[0018] When the above-mentioned face object is not wearing a mask, it is determined that the face in the above-mentioned face region of interest image is not blocked.

[0019] Optionally, the preset objective function is:

[0020] A=B*n+m, where A is the pixel value of the target pixel after linear transformation, B is the pixel value of the target pixel before linear transformation, A is greater than B, n is the preset first transformation coefficient, and m is the preset second transformation coefficient.

[0021] Optionally, after performing linear transformation on the pixel values ​​of the target pixels based on the preset objective function so that the pixel values ​​of the target pixels after the linear transformation are greater than the pixel values ​​before the linear transformation, the method further includes:

[0022] Obtain a processed face region-of-interest image after all the target pixels have undergone linear transformation, and perform a face processing task based on the processed face region-of-interest image, where the face processing task includes face recognition.

[0023] A second aspect of the present invention provides an image processing device, wherein the device comprises:

[0024] The module for acquiring an image to be processed is used to acquire an image to be processed containing a human face;

[0025] An interest region image acquisition module is used to acquire an interest region image of a face to be processed based on the image to be processed;

[0026] A target pixel acquisition module is used to determine the relationship between the pixel value of each pixel in the above-mentioned face region of interest image to be processed and a preset pixel threshold, and to use pixels smaller than the preset pixel threshold as target pixels;

[0027] The linear transformation module is used to perform linear transformation on the pixel values ​​of the target pixels based on a preset target function, and the pixel values ​​of the target pixels after the linear transformation are greater than the pixel values ​​before the linear transformation.

[0028] A third aspect of the present invention provides an intelligent terminal, comprising a memory, a processor, and an image processing program stored in the memory and executable on the processor, wherein the image processing program, when executed by the processor, implements any one of the steps of the image processing method.

[0029] Optionally, the intelligent terminal further includes an image acquisition module, and the processor utilizes the image acquisition module to acquire an image to be processed including a face to execute any one of the above-mentioned image processing methods.

[0030] A fourth aspect of the present invention provides a computer-readable storage medium, on which an image processing program is stored. When the image processing program is executed by a processor, the image processing program implements the steps of any one of the above-mentioned image processing methods.

[0031] As can be seen from the above, in the solution of the present invention, an image containing a human face is obtained; based on the image to be processed, an image of a region of interest of the human face to be processed is obtained; the relationship between the pixel value of each pixel in the image to be processed and a preset pixel threshold is determined, and pixels less than the preset pixel threshold are used as target pixels; and the pixel values ​​of each target pixel are linearly transformed based on a preset objective function, so that the pixel value of the target pixel after the linear transformation is greater than the pixel value before the linear transformation. Compared with the prior art method of adjusting the illumination to reacquire the image, the pixel value of the target pixel with a smaller pixel value is linearly transformed to increase it. Generally, the larger the pixel value, the brighter the image. Therefore, increasing the smaller pixel value can increase the brightness of the corresponding pixel, thereby improving the overall brightness of the image, so that the image meets the brightness requirements (illumination requirements) of the subsequent algorithm or model, and improving the recognition effect. At the same time, there is no need to adjust the illumination and reacquire the image to be processed, which helps save time. Since the pixel values ​​are linearly transformed, the transformed image can adapt to the original model, without the need to retrain the corresponding model, which helps save computing resources and improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0033] Figure 1 is a flowchart of an image processing method provided by an embodiment of the present invention;

[0034] Figure 2 The present invention is implemented Figure 1 Specific flow diagram of step S200;

[0035] Figure 3 1 is a structural diagram of a face detection model provided by an embodiment of the present invention;

[0036] Figure 4 The present invention is implemented Figure 2 Specific flow diagram of step S202;

[0037] Figure 5 This is a schematic structural diagram of a face attribute model provided by an embodiment of the present invention;

[0038] Figure 6 This is a schematic diagram of a specific flow of an image processing method provided by an embodiment of the present invention;

[0039] Figure 7 is a structural diagram of an image processing device provided by an embodiment of the present invention;

[0040] Figure 8 This is a block diagram of the internal structure principle of a smart terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In the following description, specific details such as particular system structures and techniques are provided for purposes of illustration, not limitation, to facilitate a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

[0042] It will be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0043] It should also be understood that the terms used in the present specification are only for the purpose of describing particular embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0044] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0045] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0046] The following is a clear and complete description of the technical solutions in the embodiments of the present invention, in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0047] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0048] At present, face recognition and other detection tasks can be performed based on the obtained images containing faces, thereby meeting user needs. However, the obtained images containing faces may affect the recognition effect due to insufficient lighting, and may even not meet the requirements of the algorithm or model used to recognize the image. For example, near-infrared images are images formed by a remote sensor receiving the near-infrared spectrum reflected or radiated by the target object. They have the characteristic of being unaffected by visible light and are increasingly used in the field of face recognition. However, near-infrared images are limited by distance. At long distances, the attenuation of near-infrared light will cause the image to be too dark, so that the lighting of the distant face image does not meet the requirements of the algorithm or model, affecting the user experience. Specifically, when the lighting of the distant face image is dark, it may affect the subsequent model recognition effect (affecting the accuracy and efficiency of recognition), and may also cause the corresponding algorithm (or model) to directly malfunction due to the image being too dark.

[0049] To address the problems existing in the prior art, this embodiment proposes an image processing method, device, intelligent terminal, and storage medium for enhancing the brightness of acquired images to meet the brightness requirements of subsequent algorithms or models, thereby improving recognition performance. This method also eliminates the need for adjusting lighting and reacquiring the image to be processed, saving time. Furthermore, the transformed image is guaranteed to be compatible with the original model, eliminating the need for retraining the corresponding model, conserving computing resources and improving the user experience.

[0050] Exemplary Methods

[0051] like Figure 1 As shown, an embodiment of the present invention provides an image processing method. Specifically, the method includes the following steps:

[0052] Step S100: Acquire an image to be processed containing a face.

[0053] The image to be processed is an image that needs to be processed to meet the recognition requirements of a face recognition model or other model. For example, the image to be processed may be an image containing a face directly obtained by a camera, which needs to be processed to extract the area corresponding to the face and adjust its brightness to meet the brightness requirements of the face recognition model in order to perform face recognition (or perform liveness detection, etc.).

[0054] In one embodiment, the image to be processed is a near-infrared image. While near-infrared images are unaffected by visible light, they can become too dark at greater distances due to attenuation of near-infrared light. Therefore, in this embodiment, brightness adjustment is performed on the near-infrared image based on the aforementioned image processing method to enhance its brightness, preventing it from being too dark and affecting recognition performance or failing to meet algorithm and model requirements. Furthermore, the image can be transformed without retraining the near-infrared model, ensuring that the transformed image is compatible with the original model.

[0055] Among them, the above-mentioned near-infrared model may include models such as face attribute model, face quality model, face liveness recognition model, face feature extraction model, etc. for performing different tasks, which are not specifically limited here.

[0056] Step S200: Based on the image to be processed, an image of a region of interest of the face to be processed is obtained.

[0057] In one embodiment, the above step S200 more specifically includes the following steps: Figure 2 Steps shown:

[0058] Step S201: Detect the image to be processed based on a face detection model to obtain an image of a facial region of interest.

[0059] Step S202: determine whether the face in the face region of interest image is blocked.

[0060] Step S203, when the face in the above-mentioned facial region of interest image is not blocked, obtain the average illumination value corresponding to the above-mentioned facial region of interest image; when the above-mentioned average illumination value is less than a preset illumination threshold, use the above-mentioned facial region of interest image as the facial region of interest image to be processed.

[0061] It should be noted that the aforementioned facial ROI image to be processed is the facial ROI image that requires image processing. Specifically, the aforementioned image processing involves brightness adjustment. When the brightness of the corresponding face in a facial ROI image is low, brightness adjustment is required to meet the requirements of the algorithm and model. The corresponding facial ROI image with low brightness is used as the facial ROI image to be processed. Conversely, if the brightness of the face in a facial ROI image is high enough to meet the requirements, no processing is required.

[0062] In a specific embodiment, before the above step S201, the above method further includes: normalizing the image to be processed into an 8-bit image, that is, adjusting the pixel bit depth of the image to be processed to 8 bits. Specifically, the near-infrared image is a 16-bit image with an effective number of 10 bits, while the input image corresponding to the current deep learning model is generally 8 bits, which is convenient for training. Therefore, in this specific embodiment, the corresponding bit depth is divided by 4 (i.e., 2 2 ), convert the near-infrared image to be processed into an 8-bit image.

[0063] Specifically, when performing subsequent tasks such as face recognition and facial feature detection, the main focus is on the facial part of the image to be processed. Therefore, in this embodiment, the processed 8-bit image to be processed is detected based on the face detection model to obtain a face region of interest image. The above-mentioned face region of interest image is an image corresponding to the ROI (region of interest) of the face. In this embodiment, the face region of interest image is an image including the entire face contour obtained after detection and extraction from the image to be processed. In one application scenario, the above-mentioned face detection model can be a model based on a multi-task convolutional neural network (MTCNN) or a RetinaFace model, which is not specifically limited here.

[0064] Figure 3 This is a structural diagram of a face detection model provided by an embodiment of the present invention. In this embodiment, the face detection model is composed of a series of (the specific number can be determined according to actual needs) convolutional layers, batch normalization layers, and activation layers. The last two output layers are classification layers and regression layers. The classification layer is used to classify the face detection output results to distinguish whether they are faces, and the regression layer is used to regress the coordinates of the face detection frame to locate the position of the face. Among them, the classification loss and regression loss are used to train the face detection model in the training stage. The classification loss is the cross entropy loss, and the regression loss is the Smooth L1 loss. The Smooth L1 loss function is shown in the following formula (1):

[0065]

[0066] Furthermore, after obtaining the facial ROI image, it is necessary to determine whether the face is occluded. If the face is occluded (for example, by a mask), the illumination statistics will be inaccurate, and the occluded face image cannot be used for subsequent face recognition, facial feature extraction, and other tasks. Therefore, in this embodiment, the facial ROI image with an occluded face is not processed to save computing resources and processing time.

[0067] If the face in the facial ROI image is not obscured, the image can be used for subsequent face recognition, facial feature extraction, and other tasks. At this point, it is necessary to determine whether the image's brightness meets the requirements of the subsequent model. If so, no brightness adjustment is required.

[0068] Specifically, in this embodiment, for facial region of interest images where the face is not obstructed, the corresponding average illumination value is statistically calculated. If the average illumination value is less than a preset illumination threshold, it indicates that the brightness does not meet the requirements and the image is processed as the facial region of interest image for brightness adjustment. Otherwise, brightness adjustment is not required. The average illumination value may be the average pixel value, and the preset illumination threshold is a pre-set threshold used to reflect the brightness level of the image. It can be set and adjusted according to actual needs and is not specifically limited here.

[0069] It should be noted that, in this embodiment, the above-mentioned average illumination value is the average illumination value of the lower half of the face in the above-mentioned face region of interest image. Specifically, if there are glasses in the eye area, a large light spot may be caused by reflection, resulting in inaccurate statistical illumination. Therefore, in this embodiment, only the lower half of the face is counted. Specifically, the average illumination value of the lower half of the face is recorded as light_value, and the preset illumination threshold is light_threshold. If light_value is greater than or equal to light_threshold, there is no need to perform brightness adjustment processing, and subsequent face processing tasks (face recognition, face feature extraction, etc.) can be directly performed. Otherwise, brightness adjustment processing is required. In one application scenario, brightness adjustment processing is performed by the illumination processing module.

[0070] In one embodiment, the above step S202 more specifically includes the following steps: Figure 4 Steps shown:

[0071] Step S2021: Detect the above-mentioned facial region of interest image based on the facial attribute model to determine whether the facial object in the above-mentioned facial region of interest image is wearing a mask.

[0072] Step S2022: When the above-mentioned face object wears a mask, the face in the above-mentioned face region of interest image is blocked.

[0073] Step S2023: When the above-mentioned face object is not wearing a mask, the face in the above-mentioned face region of interest image is not blocked.

[0074] Because the object to be detected (i.e., the human face object) is usually covered by a mask, this embodiment mainly detects the mask. However, in the actual detection process, other possible obstructions can also be detected, which is not specifically limited here.

[0075] The above-mentioned face object is a person in the image of the face region of interest, and the above-mentioned face attribute model is a pre-trained model for determining whether the face is wearing a mask. Figure 5 FIG. 1 is a schematic diagram of the structure of a face attribute model provided by an embodiment of the present invention. Figure 5 As shown, in this embodiment, the facial attribute model consists of a series of convolutional layers (the specific number can be determined according to actual needs), batch normalization layers, and activation layers, with the final output layer being a classification layer. The classification loss is the binary cross entropy loss function (BCE loss), which is used to train the facial attribute model during the training phase.

[0076] Step S300 , determining the relationship between the pixel value of each pixel in the image of the facial region of interest to be processed and a preset pixel threshold, and taking pixels smaller than the preset pixel threshold as target pixels.

[0077] Among them, the above-mentioned target pixels are pixels that need to be brightness-adjusted. Specifically, although the overall brightness of the above-mentioned image of the region of interest of the face to be processed is relatively low, for one image, the brightness corresponding to different pixels is not exactly the same, some pixels have higher brightness, and some pixels have lower brightness. In this embodiment, only the pixels with lower brightness (i.e., the target pixels whose pixel values ​​are less than the preset pixel threshold) are brightness-adjusted to avoid amplifying the light spots on the face (such as the light spots caused by the reflection of glasses) and affecting the judgment of the subsequent face algorithm. The above-mentioned preset pixel threshold is a pre-set threshold for judging whether the brightness of each pixel can meet the requirements (the requirements of the subsequent face algorithm or model). The specific value can be set and adjusted according to actual needs and is not specifically limited here.

[0078] In step S400 , linear transformation is performed on the pixel values ​​of the target pixels based on a preset target function, so that the pixel values ​​of the target pixels after the linear transformation are greater than the pixel values ​​before the linear transformation.

[0079] The objective function is a pre-set function used to increase the pixel value of the target pixel. It is a linear function that linearly transforms the pixel values ​​to ensure that the transformed image still fits the original model (i.e., the model previously trained under corresponding lighting conditions for subsequent face recognition tasks). The specific form of the objective function and the corresponding parameter values ​​can be set and adjusted according to actual needs and are not specifically limited here.

[0080] In this embodiment, the above-mentioned preset objective function is: A=B*n+m, where A is the pixel value of the above-mentioned target pixel after linear transformation, B is the pixel value of the above-mentioned target pixel before linear transformation, A is greater than B, n is the preset first transformation coefficient, and m is the preset second transformation coefficient.

[0081] Specifically, the above-mentioned objective function is a linear function, and can be set and adjusted according to actual needs during actual use, and is not specifically limited here. The above-mentioned first transformation coefficient and second transformation coefficient are pre-set parameter values ​​for controlling the degree of change of pixel values, and can be set and adjusted according to actual needs. Among them, the first transformation coefficient n is used to adjust the brightness, and the second transformation coefficient m is used to adjust the contrast. When m is equal to 0, increasing the pixel value only affects the brightness. When m is not equal to 0, increasing the pixel value will affect the brightness and contrast of the image. The increase in brightness can be used to judge the illumination through the model and algorithm, and because it is a linear transformation, it can be adapted to the original model. The larger n is, the brighter the image obtained after the transformation, but excessive brightness may also affect recognition (for example, causing light spots), so the specific value of n needs to be determined according to actual needs.

[0082] Furthermore, when m = 0, the linear transformation corresponding to the above objective function is the optimal transformation. At this time, the image of the region of interest of the face to be processed only changes the brightness, not the contrast, and does not uniformly transform all pixels, avoiding converting more pixels into invalid pixels (i.e., pixel values ​​greater than or equal to 255) and changing the information of the eyes. Due to the problem of wearing glasses or pupil reflection, there are generally light spots on the eyes of the face in near-infrared images. If the pixel threshold is not set and the face is directly uniformly transformed in brightness, the light spots on the eyes will be magnified, which will affect the judgment of the subsequent face algorithm. Therefore, in this embodiment, a preset pixel threshold is set, and only pixels with pixel values ​​less than the preset pixel threshold are processed to avoid amplifying the light spots. Among them, the above pixel threshold is set according to actual needs and the test set of the corresponding model. In one application scenario, n is set to be greater than 1 and m is set to be greater than 0 to appropriately adjust the brightness and contrast at the same time to improve the effect of the processed image.

[0083] In a specific embodiment, each pixel in the image of the facial region of interest to be processed can be traversed through a program loop and each target pixel can be processed. Specifically, assuming that the image of the facial region of interest to be processed is img_roi, the near-infrared image is a single channel, and the preset pixel threshold is pixel_threshold, the specific processing flow is as follows:

[0084] for y in img_roi.height:

[0085] for x in img_roi.width:

[0086] ifimg_roi[y,x] <pixel_threshold:

[0087] img_roi[y,x]=img_roi[y,x]*n+m;

[0088] Here, img_roi[y,x] represents the pixel value corresponding to the yth row and xth column of the image img_roi. y indicates the row of the pixel, and x indicates the column of the pixel. By adjusting the values ​​of x and y, each pixel in img_roi can be traversed and processed. In this way, pixels that do not meet the lighting requirements are transformed. Generally, in near-infrared images of human faces, except for the eyeglasses area, which will reflect light and cause highlights, the rest of the face generally has a continuous brightness, without sudden changes. This can achieve the adjustment of the overall brightness of the image in the area of ​​interest of the face.

[0089] In a specific embodiment, after the pixel values ​​of the above-mentioned target pixels are linearly transformed based on the preset objective function, and the pixel values ​​of the above-mentioned target pixels after the linear transformation are greater than the pixel values ​​before the linear transformation, the above-mentioned method further includes: obtaining a processed face region of interest image after all the above-mentioned target pixels have completed the linear transformation, and performing a face processing task based on the above-mentioned processed face region of interest image, and the above-mentioned face processing task includes face recognition.

[0090] Among them, the above-mentioned processed face area of ​​interest image is an image that has been brightness adjusted and meets the subsequent model requirements. The above-mentioned face processing task is a related processing task based on the image corresponding to the face, such as face recognition, face liveness recognition, face feature extraction and other tasks.

[0091] In this embodiment, the above image processing method is further described in detail based on a specific application scenario. Figure 6 FIG. 1 is a schematic diagram of a specific flow of an image processing method provided by an embodiment of the present invention. Figure 6 As shown, in this embodiment, a near-infrared image containing a face (i.e., the image to be processed) is obtained and normalized. The image of the region of interest of the face to be processed is extracted through the face detection model to determine whether the face object is wearing a mask. If a mask is worn, the next image to be processed is directly obtained without processing the current image. If a mask is not worn, the average illumination value of the lower half of the face is calculated, and when the average illumination value is less than a preset illumination threshold, the target pixel is adjusted (the specific processing process is described above), and then the next face processing task is performed on the adjusted image. On the contrary, when the average illumination value is not less than the preset illumination threshold, no processing is required, and the next face processing task is performed directly. In this way, it is ensured that the illumination (brightness) of the image for executing the next face processing task meets the requirements of the corresponding model, thereby improving the execution effect of the face processing task.

[0092] Exemplary devices

[0093] like Figure 7 As shown in , corresponding to the above-mentioned image processing method, an embodiment of the present invention further provides an image processing device, and the above-mentioned image processing device includes:

[0094] The to-be-processed image acquisition module 510 is configured to acquire an to-be-processed image containing a human face.

[0095] The image to be processed is an image that needs to be processed to meet the recognition requirements of a face recognition model or other model. For example, the image to be processed may be an image containing a face directly obtained by a camera, which needs to be processed to extract the area corresponding to the face and adjust its brightness to meet the brightness requirements of the face recognition model in order to perform face recognition (or perform liveness detection, etc.).

[0096] The region of interest image acquisition module 520 is used to acquire the region of interest image of the face to be processed based on the image to be processed.

[0097] The aforementioned facial ROI image to be processed is a facial ROI image that requires image processing. Specifically, the aforementioned image processing is brightness adjustment. When the brightness of the corresponding face in a facial ROI image is low, brightness adjustment processing is required to meet the requirements of the algorithm and model. The facial ROI image with low brightness is used as the facial ROI image to be processed. Conversely, if the brightness of the face in a facial ROI image is high and meets the requirements, no processing is required.

[0098] The target pixel acquisition module 530 is used to determine the relationship between the pixel value of each pixel in the above-mentioned face ROI image to be processed and the preset pixel threshold, and to use the pixel smaller than the preset pixel threshold as the target pixel.

[0099] Among them, the above-mentioned target pixels are pixels that need to be brightness-adjusted. Specifically, although the overall brightness of the above-mentioned image of the facial region of interest to be processed is relatively low, for one image, the brightness corresponding to different pixels is not exactly the same. Some pixels have higher brightness, and some pixels have lower brightness. In this embodiment, only the pixels with lower brightness (i.e., target pixels whose pixel values ​​are less than the preset pixel threshold) are brightness-adjusted. This can avoid amplifying light spots on the face (such as light spots caused by reflections from glasses) and affecting the judgment of subsequent face algorithms. The above-mentioned preset pixel threshold is a pre-set threshold used to judge whether the brightness of each pixel meets the requirements (the requirements of subsequent face algorithms or models). Its specific value can be set and adjusted according to actual needs and is not specifically limited here.

[0100] The linear transformation module 540 is configured to perform linear transformation on the pixel values ​​of the target pixels based on a preset target function, so that the pixel values ​​of the target pixels after the linear transformation are greater than the pixel values ​​before the linear transformation.

[0101] The objective function is a pre-set function used to increase the pixel value of the target pixel. It is a linear function that linearly transforms the pixel values ​​to ensure that the transformed image still fits the original model (i.e., the model previously trained under corresponding lighting conditions for subsequent face recognition tasks). The specific form of the objective function and the corresponding parameter values ​​can be set and adjusted according to actual needs and are not specifically limited here.

[0102] In this embodiment, the pixel values ​​of target pixels with smaller pixel values ​​are linearly transformed to increase them. Generally, the larger the pixel value, the brighter the image. Therefore, increasing the smaller pixel value can increase the brightness of the corresponding pixel, thereby increasing the overall brightness of the image, so that the image meets the brightness requirements (lighting requirements) of subsequent algorithms or models, improving the recognition effect. At the same time, there is no need to adjust the lighting and re-acquire the image to be processed, which helps save time. Since the pixel values ​​are linearly transformed, the transformed image can adapt to the original model without retraining the corresponding model, which helps save computing resources and improve the user experience.

[0103] Specifically, in this embodiment, the specific functions of the above-mentioned image processing device and its various modules can also refer to the corresponding descriptions in the above-mentioned image processing method, and will not be repeated here.

[0104] Based on the above embodiment, the present invention also provides an intelligent terminal, whose principle block diagram can be shown as follows: Figure 8 As shown. The above-mentioned smart terminal includes a processor and a memory. The memory of the smart terminal includes an image processing program, and the memory provides an environment for the operation of the image processing program. When the image processing program is executed by the processor, the steps of any one of the above-mentioned image processing methods are implemented for the above-mentioned image to be processed. It should be noted that the smart terminal may also include an image acquisition module, which is used to acquire images to be processed containing faces. The image acquisition module can be integrated into the smart terminal, or can be connected to the smart terminal wirelessly and / or wired, or can be associated with the cloud, which is not limited here.

[0105] Those skilled in the art will understand that Figure 8 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the solution of the present invention and does not constitute a limitation on the smart terminal to which the solution of the present invention is applied. Specifically, the smart terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0106] In one embodiment, the image processing program performs the following operation instructions when executed by the processor:

[0107] Get the image to be processed containing the face;

[0108] Based on the above image to be processed, an image of a region of interest of the face to be processed is obtained;

[0109] Determine the relationship between the pixel value of each pixel in the image of the face region of interest to be processed and a preset pixel threshold, and use pixels smaller than the preset pixel threshold as target pixels;

[0110] Based on a preset objective function, the pixel values ​​of the target pixels are linearly transformed so that the pixel values ​​of the target pixels after the linear transformation are greater than the pixel values ​​before the linear transformation.

[0111] An embodiment of the present invention further provides a computer-readable storage medium, on which an image processing program is stored. When the image processing program is executed by a processor, the steps of any one of the image processing methods provided in the embodiment of the present invention are implemented.

[0112] It should be understood that the sequence numbers of the steps in the above embodiments do not imply a specific order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0113] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0114] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0115] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0116] In the embodiments provided by the present invention, it should be understood that the disclosed apparatus / terminal device and method can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For example, the division of the modules or units described above is merely a logical functional division. In actual implementation, other division methods may be used. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not implemented.

[0117] If the above-mentioned integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code can be in source code form, object code form, executable file or some intermediate form. The above-mentioned computer-readable medium may include: any entity or device capable of carrying the above-mentioned computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the above-mentioned computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0118] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. An image processing method, characterized in that: The method comprises: Get the image to be processed containing the face; Based on the image to be processed, obtaining an image of a region of interest of a face to be processed; Determine the relationship between the pixel value of each pixel in the image of the face region of interest to be processed and a preset pixel threshold, and use pixels smaller than the preset pixel threshold as target pixels; performing a linear transformation on the pixel value of the target pixel based on a preset objective function, so that the pixel value of the target pixel after the linear transformation is greater than the pixel value before the linear transformation; wherein the objective function is a preset linear function for increasing the pixel value of the target pixel; The step of obtaining a face region of interest image to be processed based on the image to be processed includes: Adjusting the pixel bit depth of the image to be processed to 8 bits; Detecting the image to be processed based on a face detection model to obtain an image of a facial region of interest; Determining whether a face in the face region of interest image is blocked; When the face in the facial region of interest image is not obscured, the average illumination value corresponding to the facial region of interest image is obtained; when the average illumination value is less than a preset illumination threshold, the facial region of interest image is used as the facial region of interest image to be processed; the average illumination value is the average illumination value of the lower half of the face in the facial region of interest image.

2. The image processing method according to claim 1, wherein: The determining whether the face in the face region of interest image is blocked includes: Detecting the facial region of interest image based on a facial attribute model to determine whether the facial subject in the facial region of interest image is wearing a mask; When the face subject wears a mask, it is determined that the face in the face region of interest image is blocked; When the face object is not wearing a mask, it is determined that the face in the face region of interest image is not blocked.

3. The image processing method according to claim 1, wherein: The preset objective function is: A=B*n+m, where A is the pixel value of the target pixel after linear transformation, B is the pixel value of the target pixel before linear transformation, A is greater than B, n is the preset first transformation coefficient, and m is the preset second transformation coefficient.

4. The image processing method according to claim 1, wherein: After performing a linear transformation on the pixel value of the target pixel based on a preset objective function so that the pixel value of the target pixel after the linear transformation is greater than the pixel value before the linear transformation, the method further includes: A processed face region-of-interest image after linear transformation of all target pixels is obtained, and a face processing task is performed based on the processed face region-of-interest image, where the face processing task includes face recognition.

5. An image processing device, characterized in that: The device comprises: The module for acquiring an image to be processed is used to acquire an image to be processed containing a human face; An interest region image acquisition module is used to acquire an interest region image of a face to be processed based on the image to be processed; A target pixel acquisition module is used to determine the relationship between the pixel value of each pixel in the image of the facial region of interest to be processed and a preset pixel threshold, and to use pixels smaller than the preset pixel threshold as target pixels; a linear transformation module, configured to perform linear transformation on the pixel values ​​of the target pixels based on a preset objective function, so that the pixel values ​​of the target pixels after the linear transformation are greater than the pixel values ​​before the linear transformation; wherein the objective function is a preset linear function for increasing the pixel values ​​of the target pixels; The step of obtaining a face region of interest image to be processed based on the image to be processed includes: Adjusting the pixel bit depth of the image to be processed to 8 bits; Detecting the image to be processed based on a face detection model to obtain an image of a facial region of interest; Determining whether a face in the face region of interest image is blocked; When the face in the facial region of interest image is not obscured, the average illumination value corresponding to the facial region of interest image is obtained; when the average illumination value is less than a preset illumination threshold, the facial region of interest image is used as the facial region of interest image to be processed; the average illumination value is the average illumination value of the lower half of the face in the facial region of interest image.

6. An intelligent terminal, characterized in that: The intelligent terminal includes a memory, a processor, and an image processing program stored in the memory and executable on the processor. When the image processing program is executed by the processor, the steps of the image processing method according to any one of claims 1 to 4 are implemented.

7. The intelligent terminal according to claim 6, characterized in that: The intelligent terminal further includes an image acquisition module, and the processor uses the image to be processed acquired by the image acquisition module to execute the image processing method according to any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores an image processing program, which, when executed by a processor, implements the steps of the image processing method according to any one of claims 1 to 4.

Images