A dual-camera face recognition method, device and medium without white light fill light

Abstract

The present invention provides a dual-camera face recognition method without a white light fill light, comprising the following steps: acquiring a real color image and an infrared image; processing an experimental color image; inputting the experimental color image and the infrared image for training, and outputting a fused image; calculating a loss function and adjusting model parameters; starting a detection program, waiting for the light illuminance value to stabilize, acquiring the current light illuminance value and white balance value, and switching between a bright field mode and a dark field mode when the light illuminance value and the white balance value meet a switching condition; if the bright field mode is currently in the bright field mode, performing face recognition based on the color camera and the infrared camera; if the dark field mode is currently in the dark field mode, collecting a target color image and a target infrared image in the current environment, inputting them into an image fusion model, obtaining a target fused image, and performing face recognition based on the target fused image and the infrared camera. The present invention also provides an electronic device and a medium for solving the problem of people not being dazzled by white light fill lights at night.

Description

Technical Field

[0001] The present invention relates to the field of face recognition technology, and in particular to a dual-camera face recognition method, device, and medium without a white light fill lamp. Background Art

[0002] There is an increasing demand for smaller building intercom systems. Key challenges for these devices include space utilization and heat dissipation. People are demanding a higher level of comfort and quality of life. Conventional fill light systems require a prominent display of the fill light's position. Furthermore, to support facial recognition in dark environments, the fill light must be dimmed, resulting in glare. This can be particularly problematic when the light is turned on in a dark environment, creating a negative user experience.

[0003] The existing technology of using white light to perform face recognition in a dark environment has the following problems:

[0004] 1. Use white light fill light for fill light. White light fill light requires high power consumption. In order to ensure the life of the light, the fill light needs to be heat-dissipated. Aluminum substrate and other plates with fast heat dissipation will be used, which will increase the cost.

[0005] 2. The fill light effect and experience of white light fill light vary. Some users need to start fill light at a long distance, so the white light fill light needs to be brighter. If the brightness of the white light fill light is too bright, it will cause the white light to be very dazzling, so the brightness of the white light fill light needs to be reduced.

[0006] 3. The appearance of the product will be slightly abrupt due to the white fill light uniformity film, affecting the aesthetics. Summary of the Invention

[0007] To solve the above problems, the present invention provides a dual-camera face recognition method without a white light fill light, which eliminates the need for a white light fill light to provide fill light, thereby solving the problem that people do not feel glared by the white light fill light at night.

[0008] The present invention is achieved through the following technical solutions:

[0009] The present invention provides a dual-camera face recognition method without a white light fill light, the method comprising the following steps:

[0010] Step 1: Acquire multiple real color images and infrared images under sufficient light conditions; and darken all the real color images to obtain experimental color images;

[0011] Step 2: Each time, a set of the experimental color images and infrared images under the same environment are input to the image fusion model for training. The image fusion model fuses the experimental color images and infrared images according to the image fusion method and model parameters, and outputs a fused image.

[0012] Step 3: Calculate the loss function based on the fused image, the true color image, and the loss function formula. Adjust the model parameters according to the result of the loss function and continue training until the value of the loss function stabilizes. The image fusion model training is considered successful.

[0013] Step 4: Start the detection program. The environmental scenes include bright field mode and dark field mode. The default mode is switched to bright field mode. Wait for the light brightness value to stabilize. The image signal processor of the color camera obtains the light brightness value and white balance value of the current environment. When the light brightness value and white balance value meet the switching conditions, the bright field mode and dark field mode are switched.

[0014] Step 5: If the current mode is bright field, face recognition is performed based on the color camera and infrared camera. If the current mode is dark field, the color camera and infrared camera are used to collect the target color image and target infrared image in the current environment, input the target color image and target infrared image into the trained image fusion model, obtain the target fusion image for echo, and perform face recognition based on the target fusion image and the infrared camera.

[0015] Furthermore, the step 1 specifically includes:

[0016] Step 11: Using a color camera to capture multiple real color images under sufficient lighting conditions, and simultaneously using an infrared camera to capture multiple infrared images under the same conditions;

[0017] Step 12: reducing the resolution and contrast of all the real color images to obtain a low-resolution experimental color image;

[0018] Step 13: All experimental color images and infrared images are used as a data set, where the data set includes multiple sets of image data, each set of image data includes an infrared image and an experimental color image under the same environment, and the real color image is used as the real label of the data set.

[0019] Furthermore, the step 2 specifically includes:

[0020] Step 21: Input a set of experimental color images and infrared images under the same environment to the image fusion model for training each time;

[0021] Step 22: The image fusion model is fused using the FusionCNN image fusion method, and the model parameters include the convolution kernel parameters corresponding to each convolution kernel;

[0022] Step 23: The experimental color image MS is increased to 32 channels by using a 3*3 convolution kernel and its convolution kernel parameters, thereby obtaining an experimental color image MS1; the experimental color image MS1 is increased to 64 channels by using a 3*3 convolution kernel and its convolution kernel parameters, thereby obtaining an experimental color image MS2; the experimental color image MS2 is increased to 128 channels by using a 3*3 convolution kernel and its convolution kernel parameters, thereby obtaining an experimental color image MS3;

[0023] Step 24: The number of channels of the infrared image PAN is increased to 32 by using a 3*3 convolution kernel and its convolution kernel parameters, thereby obtaining an infrared image PAN1. The number of channels of the infrared image PAN1 is increased to 64 by using a 3*3 convolution kernel and its convolution kernel parameters, thereby obtaining an infrared image PAN2. The number of channels of the infrared image PAN2 is increased to 128 by using a 3*3 convolution kernel and its convolution kernel parameters, thereby obtaining an infrared image PAN3.

[0024] Step 25: The experimental color image MS and the infrared image PAN are stitched together, and the number of channels is increased to 64 by using a 1*1 convolution kernel and its convolution kernel parameters to obtain a stitched image PANMS1; the stitched image PANMS1 is stitched together with the experimental color image MS1 and the infrared image PAN1, and the number of channels is adjusted to 128 by using the 1*1 convolution kernel and its convolution kernel parameters to obtain a stitched image PANMS2; the stitched image PANMS2 is stitched together with the experimental color image MS2 and the infrared image PAN2, and the number of channels is adjusted to 256 by using the 1*1 convolution kernel and its convolution kernel parameters to obtain a stitched image PANMS3;

[0025] Step 26: Concatenate the experimental color image MS3, the stitched image PANMS3, and the infrared image PAN3, and use a 1*1 convolution kernel and its convolution kernel parameters to adjust the number of channels to 3 to obtain a fused image F and output it.

[0026] Furthermore, the step 3 specifically includes:

[0027] Step 31: During the training of the image fusion model, all image data sets are randomly distributed and divided into training set, validation set and test set according to the set ratio;

[0028] Step 32: Use the model previously trained in the CIFAR open source dataset as a pre-trained model for the image fusion model;

[0029] Step 33: input a set of image data in the training set into the image fusion model for training each time, and output a fused image;

[0030] Step 34: Substitute the fused image and the true color image into the loss function formula to calculate and obtain the loss function. The loss function formula is: Where I is the real color image, F is the fused image, n is the training set capacity, PAN is the infrared image, MS is the experimental color image, θ is the model parameter of the image fusion model, and L(θ) is the loss function;

[0031] Step 35: After each training cycle, the validation set is used for validation. The changes in the loss function values ​​calculated by the training set and the validation set are observed to determine whether the loss function value tends to be stable. If so, it is considered that the image fusion model is successfully fitted and the image fusion model converges; if not, proceed to step 36.

[0032] Step 36: Fine-tune and optimize the model parameters of the image fusion model. The optimization algorithm uses the Adam descent algorithm, sets the initial step size, and optimizes according to the step size. Repeat steps 33-35 to continue training until the value of the loss function stabilizes.

[0033] Step 37: After the image fusion model is trained, multiple tests are performed using the test set.

[0034] Furthermore, the step 4 specifically includes:

[0035] Step 41: Preset the brightness thresholds: DN_MAX and DN_MIN, and the white balance thresholds: AWB_MAX and AWB_MIN;

[0036] Step 42: Start the detection program, switch to bright field mode by default, and wait for the light intensity value to stabilize;

[0037] Step 43: The image signal processor of the color camera obtains the light intensity gain and white balance value TWB in the current environment, and determines whether the light intensity gain is greater than DN_MAX. If not, the bright field mode is maintained, and the light intensity value is again stabilized. If so, the white balance value TWB is less than AWB_MIN. If not, the bright field mode is maintained, and the light intensity value is again stabilized. If so, step 43 is repeated m times, and if the results are the same, the mode is switched to dark field mode, and the light intensity value is again stabilized.

[0038] Step 44: Obtain the light luminance value gain and white balance value TWB in the current environment through the image signal processor of the color camera, and determine whether the light luminance value gain is less than DN_MAX. If not, maintain dark field mode and wait again for the light luminance value to stabilize. If so, then determine whether the white balance value TWB is greater than AWB_MAX. If not, maintain dark field mode and wait again for the light luminance value to stabilize. If so, repeat step 44 for m times and if the results are the same, switch to bright field mode.

[0039] Furthermore, the step 4 of waiting for the light brightness value to stabilize may include:

[0040] Step 45: Capture a current color image in the current environment through a color camera;

[0041] Step 46: The image signal processor of the color camera obtains the light brightness value of the current color image according to the current color image;

[0042] Step 47: continuously collect n times of light illuminance values, and detect whether the n consecutive light illuminance values ​​are all daytime light illuminance values ​​or nighttime light illuminance values. If so, it is considered that the light is stable and detection can be performed; otherwise, continue to obtain light illuminance values.

[0043] Furthermore, the step 5 specifically includes:

[0044] Step 51: If the current mode is bright field, indicating that the current ambient light is sufficient, the color camera captures the target color image and displays it. The color camera then performs face detection and draws a face frame. The color camera and infrared camera detect whether the face is alive based on the face frame. If so, facial features are extracted based on the face frame and the target color image, and compared with the facial features in the database for face recognition. If not, the process ends.

[0045] Step 52: If the current mode is dark field, the color camera and the red camera are used to capture the target color image and the target infrared image in the current environment;

[0046] Step 53: Input the target color image and the target infrared image into the trained image fusion model to obtain the target fusion image for display;

[0047] Step 54: Face detection is then performed using an infrared camera, and a face frame is drawn. The infrared camera then detects whether the subject is alive based on the face frame. If so, facial features are extracted based on the face frame and the target fusion image, and compared with the facial features in the database for face recognition. If not, the process ends.

[0048] Furthermore, the face detection process using the infrared camera in step 54 also includes:

[0049] After entering dark field mode, the infrared fill light under the infrared camera is adjusted to maximum power by default. When a face is detected, the size of the detected face is mapped to the distance between the face and the infrared camera. The infrared camera automatically adjusts the power of the infrared fill light based on the distance and uses the PWM interface to adjust the brightness;

[0050] A warm-toned solid-color picture is used as the background in the upper half of the screen of the color camera and the infrared camera, and the lower half of the screen is used as an image preview interface.

[0051] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method for dual-camera face recognition without a white light fill lamp as described above is implemented.

[0052] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned dual-camera face recognition method without a white light fill light.

[0053] The beneficial effects of the present invention are:

[0054] The present invention enables face recognition in a dark environment without the use of a white fill light, and uses a color camera and a red camera to complete image echo and face recognition. The color camera ensures that the image can display normal colors, and the red camera ensures image clarity. By repeatedly inputting color images and infrared images in a dark environment into an image fusion model for fusion, a real color image under sufficient light is simulated. The trained image fusion model can obtain an image close to reality, resulting in better image echo and face recognition effects.

[0055] 1. Reduced power consumption and cost of the equipment, and removed white lights and heat dissipation components;

[0056] 2. Improve user experience by eliminating the need for white light fill light, so that users will not feel glare at night due to the white light fill light;

[0057] 3. There are more options for appearance, which can enhance the beauty of the product and improve the overall competitiveness of the product. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0059] Figure 1 This is a flow chart of a dual-camera face recognition method without white light fill light provided by an embodiment of the present invention.

[0060] Figure 2 Schematic diagram of the structure of the image fusion model in an embodiment of the present invention.

[0061] Figure 34 is a flow chart of switching between the bright field mode and the dark field mode under different switching conditions in an embodiment of the present invention.

[0062] Figure 4 This is a flow chart of light brightness stability judgment in an embodiment of the present invention.

[0063] Figure 5 This is a flowchart of face recognition in different environmental modes in an embodiment of the present invention.

[0064] Figure 6 is a schematic diagram of an electronic device provided by an embodiment of the present invention.

[0065] Figure 7 It is a schematic diagram of a computer-readable storage medium provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the invention for which protection is sought, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0067] like Figure 1-5 As shown, an embodiment of the present invention provides a dual-camera face recognition method without white light fill light, the method comprising the following steps:

[0068] Step 1: Acquire multiple real color images and infrared images under sufficient light conditions; and darken all the real color images to obtain experimental color images;

[0069] In this embodiment, the step specifically includes:

[0070] Step 11: Using a color camera to capture multiple real color images under sufficient lighting conditions, and simultaneously using an infrared camera to capture multiple infrared images under the same conditions;

[0071] Step 12: reducing the resolution and contrast of all the real color images to obtain a low-resolution experimental color image;

[0072] Step 13: All experimental color images and infrared images are used as a data set, wherein the data set includes multiple sets of image data, each set of image data includes an infrared image and an experimental color image under the same environment, and the real color image is used as the real label of the data set;

[0073] The purpose of this step is to collect data to facilitate the training of the image fusion model. Each training requires infrared images and experimental color images under the same environment to fuse a fused image that simulates the real color image. This is the premise of the image fusion model.

[0074] Step 2: Each time, a set of the experimental color images and infrared images under the same environment are input to the image fusion model for training. The image fusion model fuses the experimental color images and infrared images according to the image fusion method and model parameters, and outputs a fused image.

[0075] In this embodiment, if Figure 2 As shown, this step specifically includes:

[0076] Step 21: Input a set of experimental color images and infrared images under the same environment to the image fusion model for training each time;

[0077] Step 22: The image fusion model is fused using the FusionCNN image fusion method, and the model parameters include the convolution kernel parameters corresponding to each convolution kernel;

[0078] Step 23: The experimental color image MS is increased to 32 channels by using a 3*3 convolution kernel and its convolution kernel parameters, thereby obtaining an experimental color image MS1; the experimental color image MS1 is increased to 64 channels by using a 3*3 convolution kernel and its convolution kernel parameters, thereby obtaining an experimental color image MS2; the experimental color image MS2 is increased to 128 channels by using a 3*3 convolution kernel and its convolution kernel parameters, thereby obtaining an experimental color image MS3;

[0079] Step 24: The number of channels of the infrared image PAN is increased to 32 by using a 3*3 convolution kernel and its convolution kernel parameters, thereby obtaining an infrared image PAN1. The number of channels of the infrared image PAN1 is increased to 64 by using a 3*3 convolution kernel and its convolution kernel parameters, thereby obtaining an infrared image PAN2. The number of channels of the infrared image PAN2 is increased to 128 by using a 3*3 convolution kernel and its convolution kernel parameters, thereby obtaining an infrared image PAN3.

[0080] Step 25: The experimental color image MS and the infrared image PAN are stitched together, and the number of channels is increased to 64 by using a 1*1 convolution kernel and its convolution kernel parameters to obtain a stitched image PANMS1; the stitched image PANMS1 is stitched together with the experimental color image MS1 and the infrared image PAN1, and the number of channels is adjusted to 128 by using the 1*1 convolution kernel and its convolution kernel parameters to obtain a stitched image PANMS2; the stitched image PANMS2 is stitched together with the experimental color image MS2 and the infrared image PAN2, and the number of channels is adjusted to 256 by using the 1*1 convolution kernel and its convolution kernel parameters to obtain a stitched image PANMS3;

[0081] Step 26: Concatenate the experimental color image MS3, the stitched image PANMS3, and the infrared image PAN3, and use a 1*1 convolution kernel and its convolution kernel parameters to adjust the number of channels to 3 to obtain a fused image F and output it.

[0082] This step uses the method of fusing the grayscale image of the infrared camera with the color image of the color camera to obtain a better preview effect. The principle is to fuse the texture information obtained from the grayscale image with the color of the color image. The FusionCNN image fusion method proposed by Ye fanjie et al. is used. This step is the basis for image fusion.

[0083] Step 3: Calculate the loss function based on the fused image, the true color image, and the loss function formula. Adjust the model parameters according to the result of the loss function and continue training until the value of the loss function stabilizes. The image fusion model training is considered successful.

[0084] In this embodiment, if Figure 3 As shown, this step specifically includes:

[0085] Step 31: During the training of the image fusion model, all image data sets are randomly distributed and divided into training, validation, and test sets according to a set ratio. The ratio can be set to 8:1:1. The 8:1:1 ratio of training, validation, and test sets means that for every eight Bs restored to A, the mean square error of the function will be added to B1-B8. An image is used to verify whether the function can be restored to A1-A8 to a large extent. If the restoration is poor, the model parameters are adjusted. An image of B9, which was not included in the training set, is added to verify whether A9 can be restored to a large extent.

[0086] Step 32: Use the model previously trained in the CIFAR open source dataset as a pre-trained model for the image fusion model;

[0087] Step 33: input a set of image data in the training set into the image fusion model for training each time, and output a fused image;

[0088] Step 34: Substitute the fused image and the true color image into the loss function formula to calculate and obtain the loss function. The loss function formula is: Where I is the true color image, F is the fused image, n is the training set size, PAN is the infrared image, MS is the experimental color image, θ is the model parameter of the image fusion model, and L(θ) is the loss function. A true color image refers to an image taken at a specific location under normal conditions using normal equipment, denoted as Image A (good results). The true color image is processed by reducing resolution, contrast, and color brightness to obtain the processed experimental true color image, denoted as Image B. The process of restoring Image B to Image A yields a function that defines the restoration process. Each process of restoring Image B to Image A yields many sets of functions. By combining these function definitions into a formula for the mean square error, we obtain a loss function formula. This loss function formula is our algorithm.

[0089] Step 35: After each training cycle, the validation set is used for validation. The changes in the loss function values ​​calculated by the training set and the validation set are observed to determine whether the loss function value tends to be stable. If so, it is considered that the image fusion model is successfully fitted and the image fusion model converges; if not, proceed to step 36. Verification by the validation set is to ensure that overfitting does not occur.

[0090] Step 36: Fine-tune and optimize the model parameters of the image fusion model. The optimization algorithm uses the Adam descent algorithm, sets the initial step size to 0.001, optimizes according to the step size, and saves the model parameters every 10 cycles; repeat steps 33-35 to continue training until the value of the loss function stabilizes;

[0091] Step 37: After the image fusion model is trained, it is tested multiple times using the test set. After the image fusion model converges, it is considered that the normal dark field color image and the infrared image can be input into the image fusion model to obtain the desired fused image.

[0092] This step is the process of training the image fusion model, which provides the prerequisite for later face recognition in dark field mode.

[0093] Step 4: Start the detection program. The environmental scenes include bright field mode and dark field mode. The default mode is switched to bright field mode. Wait for the light brightness value to stabilize. The image signal processor of the color camera obtains the light brightness value and white balance value of the current environment. When the light brightness value and white balance value meet the switching conditions, the bright field mode and dark field mode are switched.

[0094] In this embodiment, the step specifically includes:

[0095] Step 41: Preset the brightness thresholds: DN_MAX and DN_MIN, and the white balance thresholds: AWB_MAX and AWB_MIN; the setting of the thresholds is the basis for the switching conditions;

[0096] Step 42: Start the detection program, switch to bright field mode by default, and wait for the light intensity value to stabilize;

[0097] Step 43: Obtain the light intensity gain and white balance value TWB in the current environment through the image signal processor (ISP) of the color camera, and determine whether the light intensity gain is greater than DN_MAX. If not, maintain bright field mode and wait again for the light intensity value to stabilize. If so, then determine whether the white balance value TWB is less than AWB_MIN. If not, maintain bright field mode and wait again for the light intensity value to stabilize. If so, repeat step 43 m times for the same test results, then switch to dark field mode and wait again for the light intensity value to stabilize.

[0098] Step 44: The image signal processor of the color camera obtains the light intensity gain and white balance value TWB in the current environment, and determines whether the light intensity gain is less than DN_MAX. If not, the dark field mode is maintained, and the light intensity value is again stabilized. If so, the white balance value TWB is again determined to be greater than AWB_MAX. If not, the dark field mode is maintained, and the light intensity value is again stabilized. If so, step 44 is repeated m times, and if the results are the same, the mode is switched to bright field mode.

[0099] This step describes the specific process of switching between different ambient modes (bright field mode and dark field mode). Different ambient modes employ different facial recognition methods. After obtaining a stable gain value, the system performs three status checks based on the brightness gain and white balance values. During this process, a critical judgment is made on the switching condition. If the three results are the same, the switch is made; if the three results are different, the switch is not made. This mechanism effectively avoids the adverse effects of oscillation and improves the user experience.

[0100] In this embodiment, if Figure 4 As shown, the step 4 waits for the light brightness value to be stable; specifically includes:

[0101] Step 45: Capture a current color image in the current environment through a color camera;

[0102] Step 46: The image signal processor of the color camera obtains the light brightness value of the current color image according to the current color image;

[0103] Step 47: continuously collect n times of light illuminance values ​​and detect whether the n consecutive light illuminance values ​​are all daytime light illuminance values ​​or all nighttime light illuminance values. If so, it is considered that the light is stable and the detection can be performed; otherwise, continue to obtain light illuminance values; the value n is set by the user; here n = 3;

[0104] Because ambient brightness can fluctuate frequently within a certain range, the brightness gain values ​​obtained by the ISP are unstable. To avoid data fluctuations that could cause switching between bright and dark field modes, the camera first checks whether the gain value is stable at the beginning of detection. To prevent constant switching when an object is constantly moving in front of the lens, the design concept is to not switch if the light in front of the lens is considered to be fluctuating too much, and wait for exposure to stabilize. The light intensity must be tested for three consecutive times to show either daytime brightness or nighttime brightness before switching.

[0105] Step 5: If the current mode is bright field, face recognition is performed based on the color camera and infrared camera. If the current mode is dark field, the color camera and infrared camera are used to capture the target color image and target infrared image in the current environment. The target color image and target infrared image are input into the trained image fusion model, the target fusion image is obtained and displayed, and face recognition is performed based on the target fusion image and the infrared camera.

[0106] In this embodiment, if Figure 5 As shown, this step specifically includes:

[0107] Step 51: If the current mode is bright field, indicating that the current ambient light is sufficient, the color camera captures the target color image and displays it. The color camera then performs face detection and draws a face frame. The color camera and infrared camera detect whether the face is alive based on the face frame. If so, facial features are extracted based on the face frame and the target color image, and compared with the facial features in the database for face recognition. If not, the process ends.

[0108] Step 52: If the current mode is dark field, the color camera and the red camera are used to capture the target color image and the target infrared image in the current environment;

[0109] Step 53: Input the target color image and the target infrared image into the trained image fusion model to obtain the target fusion image for display;

[0110] Step 54: Face detection is then performed using an infrared camera, and a face frame is drawn. The infrared camera then detects whether the subject is alive based on the face frame. If so, facial features are extracted based on the face frame and the target fusion image, and compared with facial features in the database for face recognition. If not, the process ends.

[0111] This step describes the face recognition method under different environmental modes. In bright field mode, the image input by the color camera is used to detect the face, extract facial features for face recognition and interface preview, and an infrared camera is used for auxiliary liveness detection. In dark field mode, an infrared camera is used as the main camera, and an infrared fill light is used for face detection, recognition, and liveness detection. The FusionCNN image fusion method is combined for interface preview.

[0112] In this embodiment, the face detection process using the infrared camera in step 54 further includes:

[0113] After entering dark field mode, the infrared fill light under the infrared camera is adjusted to maximum power by default. When a face is detected, the size of the detected face is mapped to the distance between the face and the infrared camera. The infrared camera automatically adjusts the power of the infrared fill light based on this distance and uses a PWM interface to adjust the brightness. When performing face recognition in dark field, to enable the infrared camera to capture clearer images from farther away, improve the recognition distance, and ensure that the image is not overexposed when a face approaches, the infrared fill light uses a PWM interface combined with the infrared camera's automatic power adjustment algorithm to automatically adjust the power. The higher the power, the greater the brightness. After entering dark field mode, the infrared fill light is adjusted to maximum power by default, allowing for face detection at greater distances. The size of the detected face is mapped to the distance between the face and the camera. When the face enters a set close distance threshold, the infrared fill light power is reduced to resolve the problem of face overexposure and improve image quality and recognition results.

[0114] A warm-toned solid-color picture is used as the background in the upper half of the screen of the color camera and the infrared camera, and the lower half of the screen is used as the image preview interface; this is to reduce the impact of brightness fluctuations caused by large changes in image content in the camera preview interface under the critical state of bright field and dark field on the switching algorithm. The face recognition interface has been adjusted, and a large area of ​​a gentle warm-toned solid-color picture is used as the background in the upper part of the screen, and a smaller area of ​​the camera image preview interface is used in the lower half of the screen.

[0115] like Figure 6 As shown, an embodiment of the present invention further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the above-mentioned dual-camera face recognition method without white light fill light is implemented.

[0116] like Figure 7 As shown, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned dual-camera face recognition method without white light fill light.

[0117] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A dual-camera face recognition method without white light fill light, characterized in that: The method comprises the following steps: Step 1: Acquire multiple real color images and infrared images under sufficient light conditions; and darken all the real color images to obtain experimental color images; Step 2: Each time, a set of the experimental color images and infrared images under the same environment are input to the image fusion model for training. The image fusion model fuses the experimental color images and infrared images according to the image fusion method and model parameters, and outputs a fused image. Step 3: Calculate the loss function based on the fused image, the true color image, and the loss function formula. Adjust the model parameters according to the result of the loss function and continue training until the value of the loss function stabilizes. The image fusion model training is considered successful. Step 4: Start the detection program. The environmental scenes include bright field mode and dark field mode. The default mode is switched to bright field mode. Wait for the light brightness value to stabilize. The image signal processor of the color camera obtains the light brightness value and white balance value of the current environment. When the light brightness value and white balance value meet the switching conditions, the bright field mode and dark field mode are switched. Step 5: If the current mode is bright field, face recognition is performed based on the color camera and infrared camera. If the current mode is dark field, the color camera and infrared camera are used to collect the target color image and target infrared image in the current environment, input the target color image and target infrared image into the trained image fusion model, obtain the target fusion image for display, and perform face recognition based on the target fusion image and the infrared camera.

2. The dual-camera face recognition method without white light fill light according to claim 1, characterized in that: The step 1 specifically includes: Step 11: Using a color camera to capture multiple real color images under sufficient lighting conditions, and simultaneously using an infrared camera to capture multiple infrared images under the same conditions; Step 12: reducing the resolution and contrast of all the real color images to obtain a low-resolution experimental color image; Step 13: All experimental color images and infrared images are used as a data set, where the data set includes multiple sets of image data, each set of image data includes an infrared image and an experimental color image under the same environment, and the real color image is used as the real label of the data set.

3. The dual-camera face recognition method without white light fill light according to claim 1, characterized in that: The step 2 specifically includes: Step 21: Input a set of experimental color images and infrared images under the same environment to the image fusion model for training each time; Step 22: The image fusion model is fused using the FusionCNN image fusion method, and the model parameters include the convolution kernel parameters corresponding to each convolution kernel; Step 23: The experimental color image MS is increased to 32 channels by using a 3*3 convolution kernel and its convolution kernel parameters, thereby obtaining an experimental color image MS1; the experimental color image MS1 is increased to 64 channels by using a 3*3 convolution kernel and its convolution kernel parameters, thereby obtaining an experimental color image MS2; the experimental color image MS2 is increased to 128 channels by using a 3*3 convolution kernel and its convolution kernel parameters, thereby obtaining an experimental color image MS3; Step 24: The number of channels of the infrared image PAN is increased to 32 by using a 3*3 convolution kernel and its convolution kernel parameters, thereby obtaining an infrared image PAN1. The number of channels of the infrared image PAN1 is increased to 64 by using a 3*3 convolution kernel and its convolution kernel parameters, thereby obtaining an infrared image PAN2. The number of channels of the infrared image PAN2 is increased to 128 by using a 3*3 convolution kernel and its convolution kernel parameters, thereby obtaining an infrared image PAN3. Step 25: The experimental color image MS and the infrared image PAN are stitched together, and the number of channels is increased to 64 by using a 1*1 convolution kernel and its convolution kernel parameters to obtain a stitched image PANMS1; the stitched image PANMS1 is stitched together with the experimental color image MS1 and the infrared image PAN1, and the number of channels is adjusted to 128 by using the 1*1 convolution kernel and its convolution kernel parameters to obtain a stitched image PANMS2; the stitched image PANMS2 is stitched together with the experimental color image MS2 and the infrared image PAN2, and the number of channels is adjusted to 256 by using the 1*1 convolution kernel and its convolution kernel parameters to obtain a stitched image PANMS3; Step 26: Concatenate the experimental color image MS3, the stitched image PANMS3, and the infrared image PAN3, and use a 1*1 convolution kernel and its convolution kernel parameters to adjust the number of channels to 3 to obtain a fused image F and output it.

4. The dual-camera face recognition method without white light fill light according to claim 2, characterized in that: The step 3 specifically includes: Step 31: During the training of the image fusion model, all image data sets are randomly distributed and divided into training set, validation set and test set according to the set ratio; Step 32: Use the model previously trained in the CIFAR open source dataset as a pre-trained model for the image fusion model; Step 33: input a set of image data in the training set into the image fusion model for training each time, and output a fused image; Step 34: Substitute the fused image and the true color image into the loss function formula to calculate and obtain the loss function. The loss function formula is: Where I is the real color image, F is the fused image, n is the training set capacity, PAN is the infrared image, MS is the experimental color image, θ is the model parameter of the image fusion model, and L(θ) is the loss function; Step 35: After each training cycle, the validation set is used for validation. The changes in the loss function values ​​calculated by the training set and the validation set are observed to determine whether the loss function value tends to be stable. If so, it is considered that the image fusion model is successfully fitted and the image fusion model converges; if not, proceed to step 36. Step 36: Fine-tune and optimize the model parameters of the image fusion model. The optimization algorithm uses the Adam descent algorithm, sets the initial step size, and optimizes according to the step size. Repeat steps 33-35 to continue training until the value of the loss function stabilizes. Step 37: After the image fusion model is trained, multiple tests are performed using the test set.

5. The dual-camera face recognition method without white light fill light according to claim 1, characterized in that: The step 4 specifically includes: Step 41: Preset the brightness thresholds: DN_MAX and DN_MIN, and the white balance thresholds: AWB_MAX and AWB_MIN; Step 42: Start the detection program, switch to bright field mode by default, and wait for the light intensity value to stabilize; Step 43: The image signal processor of the color camera obtains the light intensity gain and white balance value TWB in the current environment, and determines whether the light intensity gain is greater than DN_MAX. If not, the bright field mode is maintained, and the light intensity value is again stabilized. If so, the white balance value TWB is less than AWB_MIN. If not, the bright field mode is maintained, and the light intensity value is again stabilized. If so, step 43 is repeated m times, and if the results are the same, the mode is switched to dark field mode, and the light intensity value is again stabilized. Step 44: Obtain the light luminance value gain and white balance value TWB in the current environment through the image signal processor of the color camera, and determine whether the light luminance value gain is less than DN_MAX. If not, maintain dark field mode and wait again for the light luminance value to stabilize. If so, then determine whether the white balance value TWB is greater than AWB_MAX. If not, maintain dark field mode and wait again for the light luminance value to stabilize. If so, repeat step 44 for m times and if the results are the same, switch to bright field mode.

6. The dual-camera face recognition method without white light fill light according to claim 1, characterized in that: The step 4 of waiting for the light brightness value to stabilize includes: Step 45: Capture a current color image in the current environment through a color camera; Step 46: The image signal processor of the color camera obtains the light brightness value of the current color image according to the current color image; Step 47: continuously collect n times of light illuminance values, and detect whether the n consecutive light illuminance values ​​are all daytime light illuminance values ​​or nighttime light illuminance values. If so, it is considered that the light is stable and detection can be performed; otherwise, continue to obtain light illuminance values.

7. The dual-camera face recognition method without white light fill light according to claim 1, characterized in that: The step 5 specifically includes: Step 51: If the current mode is bright field, indicating that the current ambient light is sufficient, the color camera captures the target color image and displays it. The color camera then performs face detection and draws a face frame. The color camera and infrared camera detect whether the face is alive based on the face frame. If so, facial features are extracted based on the face frame and the target color image, and compared with the facial features in the database for face recognition. If not, the process ends. Step 52: If the current mode is dark field, the color camera and the red camera are used to capture the target color image and the target infrared image in the current environment; Step 53: Input the target color image and the target infrared image into the trained image fusion model to obtain the target fusion image for display; Step 54: Face detection is then performed using an infrared camera, and a face frame is drawn. The infrared camera then detects whether the subject is alive based on the face frame. If so, facial features are extracted based on the face frame and the target fusion image, and compared with the facial features in the database for face recognition. If not, the process ends.

8. The dual-camera face recognition method without white light fill light according to claim 7, characterized in that: The face detection process using the infrared camera in step 54 also includes: After entering dark field mode, the infrared fill light under the infrared camera is adjusted to maximum power by default. When a face is detected, the size of the detected face is mapped to the distance between the face and the infrared camera. The infrared camera automatically adjusts the power of the infrared fill light based on the distance and uses the PWM interface to adjust the brightness; A warm-toned solid-color picture is used as the background in the upper half of the screen of the color camera and the infrared camera, and the lower half of the screen is used as an image preview interface.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the dual-camera face recognition method without a white light fill light as described in any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the dual-camera face recognition method without a white light fill light as described in any one of claims 1 to 8 is implemented.

Images