A method and system for face recognition of people in a vehicle based on deep learning

Abstract

The present invention discloses a method and system for in-vehicle face recognition based on deep learning. The present invention first performs illumination balance processing on a face image to obtain an illumination balance image; then, performs illumination denoising processing on the illumination balance image to obtain a denoised image; in this way, the illumination balance processing can reduce the influence of illumination on the face image to a certain extent; at the same time, the illumination denoising of the image can further reduce the pollution of illumination on the image, thereby making the image clearer; and after the illumination denoising is completed, the present invention performs face posture correction to reduce the adverse influence of posture on face recognition; then, performs de-occlusion processing on the corrected face image to obtain a de-occluded face image; finally, the de-occluded face image is input into a face recognition model to obtain a face recognition result; in this way, the present invention can reduce the influence of factors such as illumination, occlusion and posture on face recognition, thereby improving the accuracy of face recognition.

Description

Technical Field

[0001] The present invention belongs to the technical field of face recognition, and in particular relates to a method and system for face recognition of people in a vehicle based on deep learning. Background Art

[0002] In automatic intelligent driving technology, image acquisition devices are installed in the car to identify the user's identity, monitor the user's status, etc., and using face recognition technology to determine the identity and number of users in the car is also a key technology in the current smart cockpit field. At present, the illumination generated by the car is different during continuous movement, and the differences in the appearance of the face under different illumination conditions are also different. Studies have shown that the difference between the images of the same person under different illumination conditions is often greater than the difference between the images of different people under the same imaging conditions. Therefore, illumination changes will seriously affect the accuracy of face recognition. At the same time, since the face angle of the person in the car is not fixed, it will produce corresponding posture differences with the activities of the person, and the person will also wear items such as sunglasses when riding or driving. In this way, under the influence of factors such as posture change and occlusion, the accuracy of face recognition of the person in the car is further reduced. Based on this, how to provide a face recognition method that can reduce the influence of illumination factors, posture factors and occlusion factors on face recognition, thereby improving the accuracy of face recognition, has become a problem to be solved urgently. Summary of the invention

[0003] The purpose of the present invention is to provide a method and system for in-vehicle face recognition based on deep learning, so as to solve the problem of low face recognition accuracy caused by the influence of lighting factors, posture factors and face occlusion factors in the prior art.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] In a first aspect, a method for face recognition of a person in a vehicle based on deep learning is provided, comprising:

[0006] Acquire a facial image of a person in the vehicle, and perform illumination balance processing on the facial image to obtain an illumination balance image;

[0007] Determine a calibration image of the illumination balance image, and perform illumination denoising processing on the illumination balance image based on the calibration image, so as to obtain a denoised image after the illumination denoising processing;

[0008] Performing facial posture correction processing on the denoised image to obtain a corrected facial image;

[0009] Performing de-occlusion processing on the corrected face image to obtain a de-occluded face image;

[0010] The de-occluded face image is input into a face recognition model for face recognition processing to obtain a face recognition result corresponding to the face image.

[0011] Based on the above disclosed content, the face recognition method provided by the present invention first performs illumination balance processing on the face image of the person in the car to obtain an illumination balance image; then, obtains a calibration image of the illumination balance image, and based on the calibration image, performs illumination denoising processing on the illumination balance image to obtain a denoised image; in this way, the illumination balance processing can reduce the influence of illumination on the face image to a certain extent; at the same time, the illumination denoising of the image can further reduce the pollution of the image by illumination, thereby enhancing the image details and making the image clearer; and after completing the illumination denoising, the present invention performs face posture correction, thereby converting faces at different angles into faces at standard angles, so as to reduce the adverse effects of posture on face recognition; then, the corrected face image is de-occluded to obtain a de-occluded face image; finally, the de-occluded face image is input into the face recognition model to obtain the face recognition result of the person in the car.

[0012] Through the above design, the present invention performs illumination balance, illumination denoising, posture correction and face de-occlusion processing before performing face recognition. In this way, the influence of factors such as illumination, occlusion and posture on face recognition can be reduced, thereby improving the accuracy of face recognition; therefore, it is very suitable for large-scale application and promotion in the field of face recognition.

[0013] In a possible design, performing illumination balance processing on the face image to obtain an illumination balanced image includes:

[0014] Performing grayscale processing on the face image to obtain a face grayscale image;

[0015] Calculating a grayscale mean of the face grayscale image, and determining a lighting balance function of the face grayscale image based on the grayscale mean;

[0016] Based on the illumination balance function, performing initial illumination balance processing on the face grayscale image to obtain an initial illumination balance image after the initial illumination balance processing;

[0017] The initial illumination balanced image is subjected to a histogram equalization process to obtain the illumination balanced image after the histogram equalization process.

[0018] In a possible design, determining the illumination balance function of the face grayscale image according to the grayscale mean value includes:

[0019] If the grayscale mean is in [0,100), the illumination balance function of the face grayscale image is as shown in the following formula (1):

[0020]

[0021] In the above formula (1), f′(x, y) represents the illumination balance function, f(x, y) represents the face grayscale image, and α, β, and c all represent illumination balance factors;

[0022] If the grayscale mean is in [100, 180], the illumination balance function of the face grayscale image is as shown in the following formula (2):

[0023] f′(x,y)=f(x,y) (2)

[0024] If the grayscale mean is in (180, 255], the illumination balance function of the face grayscale image is as shown in the following formula (3):

[0025] f′(x,y)=β c[f(x,y)-α] -1 (3).

[0026] In a possible design, the image size and image content of the calibration image are the same as those of the illumination balance image, wherein based on the calibration image, illumination denoising is performed on the illumination balance image to obtain a denoised image after the illumination denoising, including:

[0027] Determine the illumination denoising weight of each pixel in the illumination balance image according to the calibration image and the illumination balance image, and construct a denoising weight matrix using the illumination denoising weight of each pixel;

[0028] Performing image multiplication processing on the calibration image and the illumination balance image to obtain a first enhanced image;

[0029] Based on the calibration image, generating a second enhanced image;

[0030] Using the denoising weight matrix, performing weighted averaging processing on the calibration image, the illumination balance image, the first enhanced image, and the second enhanced image, respectively, so as to obtain a first weighted average value, a second weighted average value, a third weighted average value, and a fourth weighted average value after the weighted averaging processing;

[0031] Calculating a first initial denoising filter parameter according to the first weighted average, the second weighted average, the third weighted average, and the fourth weighted average, and calculating a second initial denoising filter parameter using the first initial denoising filter parameter;

[0032] Determining first denoising filter parameters according to the denoising weight matrix and the first initial denoising filter parameters, and determining second denoising filter parameters according to the denoising weight matrix and the second initial denoising filter parameters;

[0033] Generate an initial denoised image of the illumination balance image using the first denoising filter parameter, the second denoising filter parameter and the calibration image;

[0034] The initial denoised image is subjected to high-pass filtering to obtain the denoised image after high-pass filtering.

[0035] In a possible design, determining the illumination denoising weight of each pixel in the illumination balance image according to the calibration image and the illumination balance image includes:

[0036] Obtaining a first denoising weight factor and a second denoising weight factor;

[0037] For the nth pixel in the light balance image, a first weight parameter is calculated based on the first denoising weight factor, the pixel abscissa and the pixel ordinate of the nth pixel, and according to the following formula (4);

[0038]

[0039] In the above formula (4), R1 represents the first weight parameter, i and j represent the pixel horizontal coordinate and pixel vertical coordinate of the nth pixel point respectively, and τ1 represents the first denoising weight factor;

[0040] Obtain all adjacent pixels of the target pixel in the calibration image, and select any one pixel from all the adjacent pixels as a neighboring pixel, wherein the pixel coordinates of the target pixel are (i, j);

[0041] Calculate a second weight parameter according to the grayscale values ​​of the n pixels, the grayscale values ​​of the neighboring pixels and the second denoising weight factor and in accordance with the following formula (5);

[0042]

[0043] In the above formula (5), R2 represents the second weight parameter, I n Represents the gray value of the nth pixel, I l represents the grayscale value of the adjacent pixel point, and τ2 represents the second denoising weight factor;

[0044] According to the first weight parameter and the second weight parameter, and using the following formula (6), the illumination denoising weight of the n-th pixel is calculated;

[0045]

[0046] In the above formula (6), W n represents the illumination denoising weight of the n-th pixel, and K represents a constant;

[0047] Add 1 to n, and recalculate the first weight parameter based on the first denoising weight factor and the pixel horizontal coordinate and the pixel vertical coordinate of the nth pixel until n is equal to N, so as to obtain the illumination denoising weight of each pixel in the illumination balance image, wherein the initial value of n is 1, and N is the total number of pixels in the illumination balance image.

[0048] In one possible design, calculating a first initial denoising filter parameter according to the first weighted average, the second weighted average, the third weighted average, and the fourth weighted average includes:

[0049] According to the following formula (7), the first initial denoising filter parameter is calculated;

[0050]

[0051] In the above formula (7), L1 represents the first initial denoising filter parameter, wavg1, wavg2, wavg3, wavg4 represent the first weighted average value, the second weighted average value, the third weighted average value and the fourth weighted average value respectively, and δ represents the regularization coefficient;

[0052] The method of calculating the second initial denoising filter parameters by using the first initial denoising filter parameters includes:

[0053] According to the following formula (8), the second initial denoising filter parameter is calculated;

[0054] L2=wavg2-L1×wavg1 (8)

[0055] In the above formula (8), L2 represents the second initial denoising filter parameter;

[0056] Correspondingly, using the first denoising filter parameter, the second denoising filter parameter and the calibration image to generate an initial denoised image of the illumination balance image includes:

[0057] According to the following formula (9), an initial denoised image of the illumination balance image is generated;

[0058] Q′=L1′B+L2′ (9)

[0059] In the above formula (9), Q′ represents the initial denoising image, B represents the calibration image, L1′ and L2′ represent the first denoising filter parameters and the second denoising filter parameters respectively.

[0060] In a possible design, performing face posture correction processing on the denoised image to obtain a corrected face image includes:

[0061] Performing pupil positioning processing on the face in the denoised image to obtain coordinates of two pupils on the face in the denoised image;

[0062] According to the coordinates of the two pupils, the posture correction angle is calculated according to the following formula (10);

[0063]

[0064] In the above formula (10), θ represents the posture correction angle, X L ,Y L Indicates the horizontal and vertical coordinates of the left pupil of the two pupils, X R ,Y R Indicates the horizontal and vertical coordinates of the right pupil of the two pupils;

[0065] Based on the posture correction angle, and using the following formula (11), position correction is performed on each pixel point in the denoised image, so as to obtain a corrected face image after position correction;

[0066]

[0067] In the above formula (11), x m ,y m Respectively represent the horizontal coordinate and vertical coordinate of the mth pixel in the denoised image, x′ m ,y′ m They respectively represent the horizontal coordinate and vertical coordinate of the m-th pixel after position correction, where m=1, 2, 3, ..., M, and M is the total number of pixels in the denoised image.

[0068] In the second aspect, a deep learning-based face recognition system for in-vehicle personnel is provided, comprising:

[0069] A light balancing unit, used to obtain a facial image of a person in the vehicle, and perform light balancing processing on the facial image to obtain a light balanced image;

[0070] an illumination denoising unit, configured to determine a calibration image of the illumination balance image, and perform illumination denoising processing on the illumination balance image based on the calibration image, so as to obtain a denoised image after the illumination denoising processing;

[0071] A posture correction unit, used for performing face posture correction processing on the denoised image to obtain a corrected face image;

[0072] A de-occlusion unit, used for performing a de-occlusion process on the corrected face image to obtain a de-occluded face image;

[0073] The face recognition unit is used to input the de-occluded face image into a face recognition model for face recognition processing to obtain a face recognition result corresponding to the face image.

[0074] In the third aspect, a deep learning-based face recognition device for people in a vehicle is provided. Taking the device as an electronic device as an example, the device includes a memory, a processor and a transceiver that are communicatively connected in sequence, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the deep learning-based face recognition method for people in a vehicle as in the first aspect or any possible design of the first aspect.

[0075] In a fourth aspect, a storage medium is provided, on which instructions are stored. When the instructions are run on a computer, the deep learning-based face recognition method for people in a vehicle as described in the first aspect or any possible design of the first aspect is executed.

[0076] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, causes the computer to execute the deep learning-based face recognition method for people inside a vehicle as described in the first aspect or any possible design of the first aspect.

[0077] Beneficial effects:

[0078] (1) Before performing face recognition, the present invention performs illumination balancing, illumination denoising, posture correction and face de-occlusion processing, thereby reducing the influence of factors such as illumination, occlusion and posture on face recognition, thereby improving the accuracy of face recognition; therefore, the present invention is very suitable for large-scale application and promotion in the field of face recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 A schematic diagram of the steps of a method for face recognition of a person in a vehicle based on deep learning provided by an embodiment of the present invention;

[0080] Figure 2 A schematic diagram of the structure of a deep learning-based in-vehicle face recognition system provided by an embodiment of the present invention;

[0081] Figure 3 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0082] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the present invention will be briefly introduced below in combination with the drawings and the description of the embodiments or the prior art. Obviously, the following description of the structure of the drawings is only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention.

[0083] It should be understood that although the terms first, second, etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are only used to distinguish one unit from another unit. For example, a first unit can be referred to as a second unit, and similarly, a second unit can be referred to as a first unit without departing from the scope of the exemplary embodiments of the present invention.

[0084] It should be understood that the term "and / or" that may appear in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, B exists alone, and A and B exist at the same time. The term " / and" that may appear in this article describes another type of association object relationship, indicating that two relationships may exist. For example, A / and B can represent two situations: A exists alone, and A and B exist alone. In addition, the character " / " that may appear in this article generally indicates that the previous and next associated objects are in an "or" relationship.

[0085] Example:

[0086] See also Figure 1 As shown, the deep learning-based face recognition method for people in a vehicle provided in this embodiment, before performing face recognition, first performs illumination balance, illumination denoising, posture correction and face de-occlusion processing on the face image, so that after the above-mentioned processing, a de-occluded face image can be obtained; then, the de-occluded face image is input into the face recognition model, and the face recognition result of the people in the vehicle can be obtained; thus, the method can reduce the influence of factors such as illumination, occlusion and posture on face recognition, thereby improving the accuracy of face recognition; therefore, it is very suitable for large-scale application and promotion in the field of face recognition; among them, for example, the method can be but not limited to running on the face recognition end side, optionally, the face recognition end can be but not limited to a vehicle-mounted computer, it can be understood that the above-mentioned execution subject does not constitute a limitation on the embodiments of the present application, and accordingly, the running steps of the method can be but not limited to the following steps S1 to S5.

[0087] S1. Obtain a facial image of a person in the car, and perform illumination balance processing on the facial image to obtain an illumination balance image; in specific applications, the camera in the car can be used, but is not limited to, to photograph the person in the car to obtain a corresponding facial image; then, the camera transmits the photographed facial image to the on-board computer to realize facial recognition of the person in the car.

[0088] At the same time, since the external lighting will change during the driving of the car, it will cause light pollution to the facial images of the people in the car, thereby reducing the image quality (such as causing blurred images, unclear details, etc.); based on this, this embodiment needs to perform light removal processing before performing face recognition; among them, the light removal processing is mainly divided into two steps, one step is to perform light balance to reduce the impact of light on the image; the second step is to perform light denoising on the basis of the first step to restore the image and achieve the purpose of image enhancement.

[0089] Optionally, for example but not limited to, the following steps S11 to S14 may be used to implement the aforementioned illumination balance processing of the facial image.

[0090] S11. grayscale the face image to obtain a face grayscale image; in this embodiment, the grayscale processing can provide a data basis for the subsequent illumination balance on the one hand, and on the other hand, can also reduce the amount of calculation, thereby improving the processing speed; wherein, after the grayscale of the face image is completed, the illumination balance function used in this illumination balance processing can be determined based on the grayscale mean of the face grayscale image; optionally, the illumination balance function determination process can be but is not limited to as shown in the following step S12.

[0091] S12. Calculate the grayscale mean of the grayscale image of the face, and determine the illumination balance function of the grayscale image of the face according to the grayscale mean. In this embodiment, images correspond to different grayscale mean values, and their corresponding illumination standards are different. When the grayscale mean is less than 100, the image is considered to be too dark. When the grayscale mean is greater than 180, the illumination is considered to be too strong. When the grayscale mean is between [100, 180], the image is considered to be in standard illumination, and at this time, the illumination will not have an adverse effect on the image. Therefore, based on the above description, this embodiment determines different illumination balance functions according to the interval in which the grayscale mean is located, so as to achieve balanced processing when the image is too bright or too dark.

[0092] Specifically, if the grayscale mean is in [0, 100), the illumination balance function of the face grayscale image may be, but is not limited to, as shown in the following formula (1).

[0093]

[0094] In the above formula (1), f′(x, y) represents the illumination balance function, f(x, y) represents the face grayscale image, and α, β, c all represent illumination balance factors. In this embodiment, α, β, c can be, but are not limited to, 0.01, 255×ln1.2, 255 respectively.

[0095] Similarly, when the grayscale mean is in [100, 180], the illumination balance function of the face grayscale image may be, but is not limited to, as shown in the following formula (2).

[0096] f′(x,y)=f(x,y) (2)

[0097] In addition, when the grayscale mean is in (180, 255], the illumination balance function of the face grayscale image can be, but is not limited to, as shown in the following formula (3).

[0098] f′(x,y)=β c[f(x,y)-α] -1 (3).

[0099] In this embodiment, when formula (3) is used to perform illumination balance processing on a grayscale face image, α, β, and c can be, but are not limited to, 0, 255, and 1 / 255.

[0100] Thus, through the above three illumination balance functions, it can be known that when the mean value of the grayscale image of the face is less than 100, it means that the image is too dark. At this time, a logarithmic function is needed to expand the image in the low grayscale range, thereby enhancing the contrast of the image; similarly, when the mean value of the grayscale image of the face is greater than 180, it means that the image is too bright. At this time, the image will be bleached. Therefore, an exponential function is used to achieve bleaching compensation for the overbright image; and when the grayscale image of the face is in [100,180], it means that the image is under normal lighting conditions and no processing is required.

[0101] After the illumination balance function of the face grayscale image is obtained, illumination balance processing of the face grayscale image can be performed based on the function, and the process can be but not limited to the following step S13.

[0102] S13. Based on the illumination balance function, the facial grayscale image is subjected to initial illumination balance processing to obtain an initial illumination balance image after the initial illumination balance processing. In the present embodiment, the grayscale value of each pixel in the facial grayscale image is substituted into the aforementioned illumination balance function to obtain the grayscale value of each pixel after the initial illumination balance processing. After all pixels are processed, the initial illumination balance function can be obtained.

[0103] At the same time, since the aforementioned illumination balance function is a global transformation of the entire image, but since the illumination changes are diverse, only performing global illumination balance on the grayscale image of the face cannot achieve better illumination processing results. Therefore, this embodiment also provides a secondary processing step, and the process can be but is not limited to as shown in the following step S14.

[0104] S14. Performing histogram equalization processing on the initial light-balanced image, so as to obtain the light-balanced image after the histogram equalization processing; in specific applications, the histogram equalization processing is essentially an image enhancement technology, and the principle is to improve the contrast of the image by adjusting the histogram distribution of the image; in this way, after the histogram equalization processing, the contrast of the image can be enhanced, thereby reducing the light pollution in the initial light-balanced image.

[0105] Therefore, through the aforementioned steps S11 to S14, by using the illumination balance function and histogram equalization processing, the illumination balance processing of the face image can be achieved; and after obtaining the illumination balance image corresponding to the face image, the image illumination denoising can be performed, and the process can be but not limited to as shown in the following step S2.

[0106] S2. Determine a calibration image of the illumination balance image, and based on the calibration image, perform illumination denoising on the illumination balance image to obtain a denoised image after the illumination denoising. In specific applications, it is equivalent to using the calibration image to guide the illumination balance image to perform filtering, which is essentially equivalent to a local linear image filter, which can achieve smooth filtering while having good edge retention performance. Optionally, the image size and image content of the calibration image are the same as those of the illumination balance image, and the calibration image of the illumination balance image can be but is not limited to itself, that is, it is used by itself to achieve guided filtering of the image.

[0107] In a specific application, for example, but not limited to, the following steps S21 to S28 may be used to implement guided filtering denoising of the illumination balance image.

[0108] S21. Determine the illumination denoising weight of each pixel in the illumination balance image according to the calibration image and the illumination balance image, and construct a denoising weight matrix using the illumination denoising weight of each pixel. In specific applications, the illumination denoising weight of each pixel in the illumination balance image may be calculated by, for example but not limited to, the following steps S21a to S21f.

[0109] S21a. Obtain a first denoising weight factor and a second denoising weight factor; in specific applications, the first denoising weight factor and the second denoising weight factor are mainly used to calculate subsequent weight parameters, where, for example, the first denoising weight factor can be but not limited to being set to 2, 4 or 8, and the second denoising weight factor can be but not limited to being set to 0.1, 0.2 or 0.4; in this embodiment, the two are preferably set to 8 and 0.1.

[0110] After the first denoising weight factor and the second denoising weight factor are obtained, the first weight parameter and the second weight parameter may be calculated, and the calculation process may be, but is not limited to, as shown in the following steps S21b to S21e.

[0111] S21b. For the nth pixel in the light balance image, based on the first denoising weight factor, and the pixel horizontal coordinate and pixel vertical coordinate of the nth pixel, and according to the following formula (4), a first weight parameter is calculated.

[0112]

[0113] In the above formula (4), R1 represents the first weight parameter, i, j represent the pixel horizontal coordinate and pixel vertical coordinate of the nth pixel point respectively, and τ1 represents the first denoising weight factor.

[0114] After calculating the first weight parameter of the nth pixel based on the above formula (4), the corresponding second weight parameter can be calculated using the calibration image and the second denoising weight factor; wherein, this embodiment needs to obtain the grayscale value of the pixel used to calculate the second weight parameter from the calibration image based on the nth pixel; optionally, the acquisition process can be but is not limited to as shown in the following step S21c.

[0115] S21c. Obtain all adjacent pixel points of the target pixel point in the calibration image, and select any one pixel point from all the adjacent pixel points as a neighboring pixel point, wherein the pixel coordinates of the target pixel point are (i, j); in specific applications, it is equivalent to first determining the pixel point with the same position as the nth pixel point from the calibration image as the target pixel point; then, select any one pixel point from all the adjacent pixel points of the target pixel point as a neighboring pixel point (that is, select any one of the four pixel points above, below, left and right); and after determining the neighboring pixel point, the second denoising weight factor can be combined to calculate the second weight parameter of the nth pixel point; optionally, the calculation process of the second weight parameter can be but is not limited to the following step S21d as shown.

[0116] S21d. Calculate a second weight parameter based on the grayscale values ​​of the n pixels, the grayscale values ​​of the adjacent pixels and the second denoising weight factor and in accordance with the following formula (5).

[0117]

[0118] In the above formula (5), R2 represents the second weight parameter, I n Represents the gray value of the nth pixel, I l represents the grayscale value of the adjacent pixel point, and τ2 represents the second denoising weight factor.

[0119] In this way, after calculating the second weight parameter of the nth pixel point based on the aforementioned formula (5), the illumination denoising weight of the nth pixel point can be calculated in combination with the first weight parameter; wherein, the calculation process of the illumination denoising weight can be but is not limited to as shown in the following step S21e.

[0120] S21e. Calculate the illumination denoising weight of the n-th pixel according to the first weight parameter and the second weight parameter and using the following formula (6).

[0121]

[0122] In the above formula (6), W n represents the illumination denoising weight of the n-th pixel, and K represents a constant.

[0123] Therefore, based on the aforementioned steps S21b to S21e, the illumination denoising weight of the nth pixel can be determined; then, using the same principle, the illumination denoising weight of the next pixel can be calculated; in this way, the cycle is continued until all pixels in the illumination balance image are polled, and the illumination denoising weight of each pixel can be obtained; wherein, the cyclic calculation process can be but is not limited to as shown in the following step S21f.

[0124] S21f. Add 1 to n, and recalculate the first weight parameter based on the first denoising weight factor and the pixel horizontal coordinate and pixel vertical coordinate of the nth pixel, until n is equal to N, and the illumination denoising weight of each pixel in the illumination balance image is obtained, wherein the initial value of n is 1, and N is the total number of pixels in the illumination balance image.

[0125] Based on the aforementioned steps S21a to S21f, the illumination denoising weight of each pixel in the illumination balance image can be calculated, and then, the illumination balance image can be guided filtered in combination with the calibration image, and the process can be but is not limited to as shown in the following steps S22 to S28.

[0126] S22. Perform image multiplication processing on the calibration image and the illumination balance image to obtain a first enhanced image. In this embodiment, image multiplication is essentially the multiplication of pixels at corresponding positions, which suppresses areas of no interest in the image (such as areas outside the face) and highlights the details of the area of ​​interest in the image to make it clearer.

[0127] After the first enhanced image is obtained, a second enhanced image may be generated based on the calibration image, and the process may be, but is not limited to, as shown in the following step S23.

[0128] S23. Based on the calibration image, generate a second enhanced image; in this embodiment, the calibration image is multiplied by the calibration image to generate the second enhanced image; after obtaining the second enhanced image, the first enhanced image, the calibration image, the illumination balance image and the illumination denoising weight can be combined to perform weighted averaging processing, and the process can be but is not limited to as shown in the following step S24.

[0129] S24. Using the denoising weight matrix, weighted averaging is performed on the calibration image, the illumination balance image, the first enhanced image and the second enhanced image, respectively, so as to obtain a first weighted average value, a second weighted average value, a third weighted average value and a fourth weighted average value after the weighted averaging. In this embodiment, since the calibration image, the first enhanced image and the second enhanced image have the same size as the illumination balance image, the pixels in the illumination balance image can correspond one-to-one in the aforementioned three images (that is, the pixels at the corresponding coordinates can be determined in the remaining three images based on the pixel coordinates of the pixels in the illumination balance image). Therefore, the calibration image is taken as an example to explain the weighted averaging process in detail: that is, the grayscale value of each pixel in the calibration image is directly multiplied by the illumination denoising weight corresponding to each pixel in the denoising weight matrix; then, the sum is calculated and divided by the total number of pixels to obtain the first weighted average value; of course, the weighted averaging process of the remaining images is the same as the weighted averaging process of the aforementioned calibration image, and will not be repeated here.

[0130] After obtaining the weighted average of the above four images, the first initial denoising filter parameter and the second initial denoising filter parameter can be calculated based on the weighted average of the above four images. The process can be but not limited to the one shown in the following step S25.

[0131] S25. Calculate the first initial denoising filter parameter based on the first weighted average, the second weighted average, the third weighted average and the fourth weighted average, and calculate the second initial denoising filter parameter using the first initial denoising filter parameter; in specific applications, the first initial denoising filter parameter can be calculated, for example but not limited to, according to the following formula (7).

[0132]

[0133] In the above formula (7), L1 represents the first initial denoising filter parameter, wavg1, wavg2, wavg3, wavg4 represent the first weighted average, the second weighted average, the third weighted average and the fourth weighted average respectively, and δ represents the regularization coefficient; optionally, the regularization coefficient can be, but is not limited to, set to 0.16.

[0134] After obtaining the first initial denoising filter parameter, the second initial denoising filter parameter can be calculated based on the first initial denoising filter parameter. For example, but not limited to, the second initial denoising filter parameter can be calculated according to the following formula (8).

[0135] L2=wavg2-L1×wavg1 (8)

[0136] In the above formula (8), L2 represents the second initial denoising filter parameter.

[0137] Thus, according to the aforementioned formula (7) and formula (8), two initial denoising filter parameters can be calculated; then, based on this, the denoising filter parameters used in actual filtering can be calculated; wherein, the specific calculation process of the two denoising filter parameters can be but is not limited to as shown in the following step S26.

[0138] S26. Determine the first denoising filter parameters according to the denoising weight matrix and the first initial denoising filter parameters, and determine the second denoising filter parameters according to the denoising weight matrix and the second initial denoising filter parameters; in this embodiment, the first initial denoising filter parameters are multiplied by each element in the denoising weight matrix; then the sum is calculated and the mean is taken to obtain the first denoising filter parameters; similarly, the calculation process of the second denoising filter parameters is the same, which will not be repeated here.

[0139] After the first denoising filter parameter and the second denoising filter parameter are obtained, an initial denoising process of the illumination balance image may be performed, and the process may be, but is not limited to, as shown in the following step S27.

[0140] S27. Generate an initial denoised image of the illumination balance image using the first denoising filter parameter, the second denoising filter parameter and the calibration image. In this embodiment, the initial denoised image of the illumination balance image may be generated by, but is not limited to, the following formula (9).

[0141] Q′=L1′B+L2′ (9)

[0142] In the above formula (9), Q′ represents the initial denoising image, B represents the calibration image, L1′ and L2′ represent the first denoising filter parameters and the second denoising filter parameters respectively.

[0143] Furthermore, in this embodiment, after obtaining the initial denoised image, the following formula (12) may be used, but is not limited to, to further perform image enhancement.

[0144] Q″=Z(BQ′)+Q′ (12)

[0145] In the above formula (12), Q″ represents the enhanced initial denoised image, and Z represents the enhancement parameter. In this embodiment, it can be set to 5 but is not limited to it. Of course, it can be set specifically according to actual use and is not limited to the above example.

[0146] In this way, the filtering denoising effect can be increased through the aforementioned formula (12); however, in actual application, after illumination denoising, although the image is enhanced, some areas have the problem of over-sharpening; therefore, this embodiment also performs high-pass filtering on the initial denoised image to suppress local over-sharpening, and the process can be but is not limited to as shown in the following step S28.

[0147] S28. Perform high-pass filtering on the initial denoised image to obtain the denoised image after high-pass filtering; in this embodiment, if the initial denoised image is enhanced, then high-pass filtering is performed on the enhanced initial denoised image to obtain the denoised image after high-pass filtering.

[0148] Therefore, through the aforementioned steps S21 to S28 and their sub-steps, the illumination denoising processing of the illumination balanced image can be completed, thereby obtaining a face image with light pollution removed; in this way, the pollution of light to the image can be reduced, thereby enhancing the image details and making the image clearer.

[0149] After completing the illumination balance and illumination denoising of the face image, the face posture correction can be performed to avoid the adverse effect of the face angle change of the occupant on face recognition; wherein, the face posture correction process can be but is not limited to as shown in the following step S3.

[0150] S3. Perform facial posture correction on the denoised image to obtain a corrected facial image; in a specific application, the pupil distance is the most stable factor of the facial image; therefore, on this basis, this embodiment selects the pupil as a position reference for facial posture correction; among them, for example, but not limited to, the following steps S31 to S33 can be used to complete the facial posture correction.

[0151] S31. Perform pupil positioning processing on the face in the denoised image to obtain the coordinates of the two pupils on the face in the denoised image. In a specific implementation, the pupil coordinates may be obtained by, but not limited to: (1) performing vertical grayscale projection on the denoised image to obtain a vertical grayscale projection curve; (2) determining the left and right boundaries of the face in the denoised face image based on the vertical grayscale projection curve (in this embodiment, the average grayscale value of the face is obviously different from that of other objects in the background, and the distribution of grayscale values ​​inside the face has no change except for the eyebrows). The average grayscale value has little effect, and the rest is relatively uniform; therefore, the vertical grayscale projection curve will appear relatively flat in the area where the face is located, and there will be a sudden change at the junction of the face and the background; based on this, there will be a large gradient value at the boundary, which will be reflected in the vertical grayscale projection curve, and there will be a large bulge. Therefore, based on the bulge, the optimal boundary of the face can be determined); (3) Based on the left and right boundaries of the face, the face image area is cut out; (4) The face image area is horizontally projected to obtain the horizontal grayscale projection curve; (5) According to the horizontal grayscale projection curve, the eye region is determined from the face image region (in this embodiment, the maximum point and the second maximum point of the horizontal grayscale projection curve correspond to the forehead and the middle of the nose, respectively. Therefore, the horizontal coordinate of the eye is located between the maximum point and the second maximum point. Based on this, the position of the eye can be obtained, thereby cutting out the eye region from the face image region); (6) Obtain a horizontal difference operator and a vertical difference operator, and perform a difference operation on the eye region based on the horizontal difference operator and the vertical difference operator. A horizontal difference image and a vertical difference image are obtained; (7) the horizontal difference image is multiplied by the horizontal difference image, and the vertical difference image is multiplied by the vertical difference image, to obtain a horizontal difference enhanced image and a vertical difference enhanced image; (8) the horizontal difference enhanced image and the vertical difference enhanced image are Gaussian smoothing filtered, respectively, to obtain a horizontal difference filtered image and a vertical difference filtered image; (8) according to the horizontal difference image, the vertical difference image, the horizontal difference filtered image and the vertical difference filtered image, the coordinates of the two pupils are determined from the human eye area.

[0152] In specific applications, the horizontal difference operator is exemplified as: The vertical difference operator is:

[0153] At the same time, examples can include but are not limited to calculating the corner point matrix of the face image based on the aforementioned horizontal difference image, vertical difference image, horizontal difference filter image and vertical difference filter image according to the following formula (13).

[0154] E=[T′1×T′2-(T1×T2) 2 ]-κ[T′1+T′2] 2 (13)

[0155] In the above formula (13), E represents the corner point matrix, T1 and T2 represent the horizontal difference image and the vertical difference image respectively, and T1′ and T2′ represent the horizontal difference filter image and the vertical difference filter image respectively.

[0156] Thus, through the above formula (13), it can be known that the corner point matrix is ​​actually obtained by image operation of the above four images, and each element inside it corresponds to the corner point feature value of each pixel in the face area; therefore, from the corner point matrix, the element values ​​greater than the threshold are screened out (there are only two); then, the pixel points corresponding to the two screened element values ​​are used as the two pupils, thereby obtaining the corresponding pupil coordinates.

[0157] In this way, after obtaining the coordinates of the two pupils, the posture correction angle can be calculated based on this, so that the face posture correction can be performed based on this angle later; wherein, the calculation process of the posture correction angle can be but is not limited to as shown in the following step S32.

[0158] S32. Calculate the posture correction angle based on the coordinates of the two pupils and according to the following formula (10).

[0159]

[0160] In the above formula (10), θ represents the posture correction angle, X L ,Y L Indicates the horizontal and vertical coordinates of the left pupil of the two pupils, X R ,Y R Represents the horizontal and vertical coordinates of the right eye pupil among the two pupils.

[0161] After the posture correction angle is calculated based on the above formula (10), the position correction of each pixel in the denoised image can be performed based on the angle. The process can be but is not limited to the following step S33.

[0162] S33. Based on the posture correction angle, and using the following formula (11), position correction is performed on each pixel in the denoised image to obtain a corrected face image after position correction.

[0163]

[0164] In the above formula (11), x m ,y m Respectively represent the horizontal coordinate and vertical coordinate of the mth pixel in the denoised image, x′m, y′ m They respectively represent the horizontal coordinate and vertical coordinate of the m-th pixel after position correction, where m=1, 2, 3, ..., M, and M is the total number of pixels in the denoised image.

[0165] Thus, through the aforementioned steps S31 to S33, the posture correction of the face in the denoised image can be completed, so that faces at different angles can be corrected to standard faces, thereby avoiding the problem of low face recognition accuracy caused by posture changes.

[0166] After completing the face posture correction, face de-occlusion processing is also required to avoid the influence of items such as sunglasses worn by people in the car on face recognition; wherein, the face de-occlusion processing process can be but is not limited to the following step S4.

[0167] S4. De-occlusion processing is performed on the corrected face image to obtain a de-occluded face image. In this embodiment, face occlusion recognition processing can be performed first, such as using a trained occlusion recognition model to perform recognition processing of occlusions in the corrected face image, thereby obtaining a recognition result. The occlusion recognition model can be, but is not limited to, an R-CNN series model, a YOLO series model, an SSD model, etc., and its training process is: (1) obtaining a training data set, which includes a number of face images wearing sunglasses and a number of face images not wearing sunglasses; (2) using each training data in the training data set as input and the occlusion recognition results of each training data as output to train an initial occlusion recognition model, so as to obtain an occlusion recognition model after the training is completed; in this embodiment, the initial occlusion recognition model is an untrained occlusion recognition model.

[0168] Among them, in this embodiment, if the occlusion recognition result of the corrected facial image is that occlusion exists, then de-occlusion processing is required, otherwise it is not required; at the same time, for example, but not limited to, a method based on sparse expression classification can be used to remove occlusion of objects such as sunglasses in the corrected facial image.

[0169] Specifically, the occlusion removal process based on sparse expression classification can be but is not limited to: (1) obtaining an occlusion training set, wherein the occlusion training set includes multiple occluded face images and an unoccluded image corresponding to each occluded face image; (2) calculating the sparse coefficient of each occluded face image on the corresponding unoccluded face image; (3) determining the reconstructed face sparse coefficient based on the sparse coefficient of each occluded face image; (4) performing face reconstruction of the corrected face image based on the reconstructed face sparse coefficient to obtain a de-occluded face image.

[0170] Of course, the aforementioned de-occlusion method based on sparse representation is a commonly used technology for face de-occlusion, and its principle will not be repeated here.

[0171] After the de-occlusion processing of the corrected face image is completed, face recognition can be performed, and the process can be but not limited to the following step S5.

[0172] S5. Input the de-occluded facial image into a facial recognition model for facial recognition processing to obtain a facial recognition result corresponding to the facial image; in this embodiment, examples may include but are not limited to using commonly used facial recognition models for facial recognition processing, such as a deep convolutional neural network model (CNN model), a VDDFACE model, a YOLOv4 model, etc.; of course, the specific model type can be specifically set according to actual use, and is not limited to the aforementioned examples.

[0173] Therefore, through the deep learning-based face recognition method for people in the car described in detail in the aforementioned steps S1 to S5, the present invention first performs illumination balance, illumination denoising, posture correction and face de-occlusion processing on the face image before performing face recognition. In this way, after the aforementioned processing, a de-occluded face image can be obtained; then, the de-occluded face image is input into the face recognition model, and the face recognition result of the people in the car can be obtained; thus, the present invention can reduce the influence of factors such as illumination, occlusion and posture on face recognition, thereby improving the accuracy of face recognition; therefore, it is very suitable for large-scale application and promotion in the field of face recognition.

[0174] like Figure 2 As shown, the second aspect of this embodiment provides a hardware system for implementing the in-vehicle person face recognition method based on deep learning described in the first aspect of the embodiment, including:

[0175] The light balancing unit is used to obtain a facial image of a person in the vehicle and perform light balancing processing on the facial image to obtain a light balanced image.

[0176] The illumination denoising unit is used to determine a calibration image of the illumination balance image, and perform illumination denoising processing on the illumination balance image based on the calibration image, so as to obtain a denoised image after the illumination denoising processing.

[0177] The posture correction unit is used to perform face posture correction processing on the denoised image to obtain a corrected face image.

[0178] The de-occlusion unit is used to perform de-occlusion processing on the corrected face image to obtain a de-occluded face image.

[0179] The face recognition unit is used to input the de-occluded face image into a face recognition model for face recognition processing to obtain a face recognition result corresponding to the face image.

[0180] The working process, working details and technical effects of the system provided in this embodiment can be found in the first aspect of the embodiment and will not be described in detail here.

[0181] like Figure 3As shown, the third aspect of this embodiment provides a deep learning-based face recognition device for people in a vehicle. Taking the device as an electronic device as an example, it includes: a memory, a processor and a transceiver that are communicatively connected in sequence, wherein the memory is used to store computer programs, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the deep learning-based face recognition method for people in a vehicle as described in the first aspect of the embodiment.

[0182] For example, the memory may include, but is not limited to, random access memory (RAM), read only memory (ROM), flash memory, first input first output (FIFO) and / or first in last out (FILO), etc. Specifically, the processor may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). At the same time, the processor may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state.

[0183] In some embodiments, the processor may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. For example, the processor may not be limited to a microprocessor of the STM32F105 series, a reduced instruction set computer (RISC) microprocessor, an X86 architecture processor, or a processor with an integrated embedded neural network processing unit (NPU); the transceiver may be, but is not limited to, a wireless fidelity (WIFI) wireless transceiver, a Bluetooth wireless transceiver, a general packet radio service technology (General Packet Radio Service, GPRS) wireless transceiver, a ZigBee protocol (a low-power local area network protocol based on the IEEE802.15.4 standard, ZigBee) wireless transceiver, a 3G transceiver, a 4G transceiver and / or a 5G transceiver, etc. In addition, the device may also include, but is not limited to, a power module, a display screen, and other necessary components.

[0184] The working process, working details and technical effects of the electronic device provided in this embodiment can be found in the first aspect of the embodiment and will not be described in detail here.

[0185] The fourth aspect of this embodiment provides a storage medium storing instructions for the method for face recognition of people in a vehicle based on deep learning as described in the first aspect of the embodiment, that is, the storage medium stores instructions, and when the instructions are run on a computer, the method for face recognition of people in a vehicle based on deep learning as described in the first aspect of the embodiment is executed.

[0186] The storage medium refers to a carrier for storing data, which may include but is not limited to a floppy disk, a CD, a hard disk, a flash memory, a USB flash drive and / or a memory stick, and the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0187] The working process, working details and technical effects of the storage medium provided in this embodiment can be found in the first aspect of the embodiment and will not be described in detail here.

[0188] The fifth aspect of this embodiment provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the deep learning-based face recognition method for people inside a vehicle as described in the first aspect of the embodiment, wherein the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0189] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for face recognition of people in a vehicle based on deep learning, characterized in that: include: Acquire a facial image of a person in the vehicle, and perform illumination balance processing on the facial image to obtain an illumination balance image; Determine a calibration image of the illumination balance image, and perform illumination denoising processing on the illumination balance image based on the calibration image, so as to obtain a denoised image after the illumination denoising processing; Performing facial posture correction processing on the denoised image to obtain a corrected facial image; Performing de-occlusion processing on the corrected face image to obtain a de-occluded face image; Inputting the de-occluded face image into a face recognition model for face recognition processing to obtain a face recognition result corresponding to the face image; Performing illumination balance processing on the face image to obtain an illumination balanced image includes: Performing grayscale processing on the face image to obtain a face grayscale image; Calculating a grayscale mean of the face grayscale image, and determining a lighting balance function of the face grayscale image based on the grayscale mean; Based on the illumination balance function, performing initial illumination balance processing on the face grayscale image to obtain an initial illumination balance image after the initial illumination balance processing; Performing a histogram equalization process on the initial illumination balanced image to obtain the illumination balanced image after the histogram equalization process; The image size and image content of the calibration image are the same as those of the illumination balance image, wherein based on the calibration image, illumination denoising is performed on the illumination balance image to obtain a denoised image after the illumination denoising, including: Determine the illumination denoising weight of each pixel in the illumination balance image according to the calibration image and the illumination balance image, and construct a denoising weight matrix using the illumination denoising weight of each pixel; Performing image multiplication processing on the calibration image and the illumination balance image to obtain a first enhanced image; Based on the calibration image, generating a second enhanced image; Using the denoising weight matrix, performing weighted averaging processing on the calibration image, the illumination balance image, the first enhanced image, and the second enhanced image, respectively, so as to obtain a first weighted average value, a second weighted average value, a third weighted average value, and a fourth weighted average value after the weighted averaging processing; Calculating a first initial denoising filter parameter according to the first weighted average, the second weighted average, the third weighted average, and the fourth weighted average, and calculating a second initial denoising filter parameter using the first initial denoising filter parameter; Determining first denoising filter parameters according to the denoising weight matrix and the first initial denoising filter parameters, and determining second denoising filter parameters according to the denoising weight matrix and the second initial denoising filter parameters; Generate an initial denoised image of the illumination balance image using the first denoising filter parameter, the second denoising filter parameter and the calibration image; The initial denoised image is subjected to high-pass filtering to obtain the denoised image after high-pass filtering.

2. The method according to claim 1, characterized in that Determining the illumination balance function of the face grayscale image according to the grayscale mean value includes: If the grayscale mean is in [0,100), the illumination balance function of the face grayscale image is as shown in the following formula (1): In the above formula (1), f′(x, y) represents the illumination balance function, f(x, y) represents the face grayscale image, and α, β, and c all represent illumination balance factors; If the grayscale mean is in [100, 180], the illumination balance function of the face grayscale image is as shown in the following formula (2): f′(x,y)=f(x,y) (2) If the grayscale mean is in (180, 255], the illumination balance function of the face grayscale image is as shown in the following formula (3): f′(x,y)=β c[f(x,y)-α] -1 (3).

3. The method according to claim 1, characterized in that Determining the illumination denoising weight of each pixel in the illumination balance image according to the calibration image and the illumination balance image, including: Obtaining a first denoising weight factor and a second denoising weight factor; For the nth pixel in the light balance image, a first weight parameter is calculated based on the first denoising weight factor, the pixel abscissa and the pixel ordinate of the nth pixel, and according to the following formula (4); In the above formula (4), R1 represents the first weight parameter, i and j represent the pixel horizontal coordinate and pixel vertical coordinate of the nth pixel point respectively, and τ1 represents the first denoising weight factor; Obtain all adjacent pixels of the target pixel in the calibration image, and select any one pixel from all the adjacent pixels as a neighboring pixel, wherein the pixel coordinates of the target pixel are (i, j); Calculate a second weight parameter according to the grayscale values ​​of the n pixels, the grayscale values ​​of the neighboring pixels and the second denoising weight factor and in accordance with the following formula (5); In the above formula (5), R2 represents the second weight parameter, I n Represents the gray value of the nth pixel, I l represents the grayscale value of the adjacent pixel point, and τ2 represents the second denoising weight factor; According to the first weight parameter and the second weight parameter, and using the following formula (6), the illumination denoising weight of the n-th pixel is calculated; In the above formula (6), W n represents the illumination denoising weight of the n-th pixel, and K represents a constant; Add 1 to n, and recalculate the first weight parameter based on the first denoising weight factor and the pixel horizontal coordinate and the pixel vertical coordinate of the nth pixel until n is equal to N, so as to obtain the illumination denoising weight of each pixel in the illumination balance image, wherein the initial value of n is 1, and N is the total number of pixels in the illumination balance image.

4. The method according to claim 1, characterized in that Calculating a first initial denoising filter parameter according to the first weighted average value, the second weighted average value, the third weighted average value, and the fourth weighted average value includes: According to the following formula (7), the first initial denoising filter parameter is calculated; In the above formula (7), L1 represents the first initial denoising filter parameter, wavg1, wavg2, wavg3, wavg4 represent the first weighted average value, the second weighted average value, the third weighted average value and the fourth weighted average value respectively, and δ represents the regularization coefficient; The method of calculating the second initial denoising filter parameters by using the first initial denoising filter parameters includes: According to the following formula (8), the second initial denoising filter parameter is calculated; L2=wavg2-L1×wavg1 (8) In the above formula (8), L2 represents the second initial denoising filter parameter; Correspondingly, using the first denoising filter parameter, the second denoising filter parameter and the calibration image to generate an initial denoised image of the illumination balance image includes: According to the following formula (9), an initial denoised image of the illumination balance image is generated; Q′=L1′B+L2′ (9) In the above formula (9), Q′ represents the initial denoising image, B represents the calibration image, L1′ and L2′ represent the first denoising filter parameters and the second denoising filter parameters respectively.

5. The method according to claim 1, characterized in that Performing facial posture correction processing on the denoised image to obtain a corrected facial image includes: Performing pupil positioning processing on the face in the denoised image to obtain coordinates of two pupils on the face in the denoised image; According to the coordinates of the two pupils, the posture correction angle is calculated according to the following formula (10); In the above formula (10), θ represents the posture correction angle, X L ,Y L Indicates the horizontal and vertical coordinates of the left pupil of the two pupils, X R ,Y R Indicates the horizontal and vertical coordinates of the right pupil of the two pupils; Based on the posture correction angle, and using the following formula (11), position correction is performed on each pixel point in the denoised image, so as to obtain a corrected face image after position correction; In the above formula (11), x m ,y m Respectively represent the horizontal coordinate and vertical coordinate of the mth pixel in the denoised image, x′ m ,y′ m They respectively represent the horizontal coordinate and vertical coordinate of the m-th pixel after position correction, where m=1, 2, 3, ..., M, and M is the total number of pixels in the denoised image.

6. A deep learning-based face recognition system for in-vehicle personnel, characterized in that: include: A light balancing unit, used to obtain a facial image of a person in the vehicle, and perform light balancing processing on the facial image to obtain a light balanced image; an illumination denoising unit, configured to determine a calibration image of the illumination balance image, and perform illumination denoising processing on the illumination balance image based on the calibration image, so as to obtain a denoised image after the illumination denoising processing; A posture correction unit, used for performing face posture correction processing on the denoised image to obtain a corrected face image; A de-occlusion unit, used for performing a de-occlusion process on the corrected face image to obtain a de-occluded face image; A face recognition unit, used for inputting the de-occluded face image into a face recognition model for face recognition processing to obtain a face recognition result corresponding to the face image; Performing illumination balance processing on the face image to obtain an illumination balanced image includes: Performing grayscale processing on the face image to obtain a face grayscale image; Calculating a grayscale mean of the face grayscale image, and determining a lighting balance function of the face grayscale image based on the grayscale mean; Based on the illumination balance function, performing initial illumination balance processing on the face grayscale image to obtain an initial illumination balance image after the initial illumination balance processing; Performing a histogram equalization process on the initial illumination balanced image to obtain the illumination balanced image after the histogram equalization process; The image size and image content of the calibration image are the same as those of the illumination balance image, wherein based on the calibration image, illumination denoising is performed on the illumination balance image to obtain a denoised image after the illumination denoising, including: Determine the illumination denoising weight of each pixel in the illumination balance image according to the calibration image and the illumination balance image, and construct a denoising weight matrix using the illumination denoising weight of each pixel; Performing image multiplication processing on the calibration image and the illumination balance image to obtain a first enhanced image; Based on the calibration image, generating a second enhanced image; Using the denoising weight matrix, performing weighted averaging processing on the calibration image, the illumination balance image, the first enhanced image, and the second enhanced image, respectively, so as to obtain a first weighted average value, a second weighted average value, a third weighted average value, and a fourth weighted average value after the weighted averaging processing; Calculating a first initial denoising filter parameter according to the first weighted average, the second weighted average, the third weighted average, and the fourth weighted average, and calculating a second initial denoising filter parameter using the first initial denoising filter parameter; Determining first denoising filter parameters according to the denoising weight matrix and the first initial denoising filter parameters, and determining second denoising filter parameters according to the denoising weight matrix and the second initial denoising filter parameters; Generate an initial denoised image of the illumination balance image using the first denoising filter parameter, the second denoising filter parameter and the calibration image; The initial denoised image is subjected to high-pass filtering to obtain the denoised image after high-pass filtering.

7. An electronic device, characterized in that: include: A memory, a processor and a transceiver that are sequentially communicatively connected, wherein the memory is used to store a computer program, the transceiver is used to send and receive messages, and the processor is used to read the computer program to execute the deep learning-based face recognition method for occupants of a vehicle as described in any one of claims 1 to 5.

8. A computer program product comprising instructions, characterized in that When the instructions are executed on a computer, the computer is enabled to execute the deep learning-based face recognition method for occupants of a vehicle as described in any one of claims 1 to 5.

Images