A face recognition method, system, computer device and storage medium

Abstract

The present invention belongs to the field of computer technology, and specifically relates to a face recognition method, system, computer equipment and storage medium. The method comprises: acquiring captured image data; initializing the acquired image data, judging whether there is a rotation flag; if not, cutting out the face and reading the characteristic value of the captured face; according to the processed characteristic value and the face direction recognition judgment, obtaining the rotation value, assigning the image and saving the rotation flag; if yes, performing the corresponding angle rotation according to the value of the rotation flag to obtain normal image data. The present invention can quickly and efficiently complete the adaptive normal display of the face direction, and all big data operations such as image processing are processed in an embedded neural network processor (NPU), which greatly improves efficiency, reduces power consumption, can identify living bodies, avoids the situation that the recognition can be passed by using photos, and greatly improves the recognition security.

Description

Technical Field

[0001] The present invention belongs to the field of computer technology, and in particular relates to a face recognition method, system, computer equipment and storage medium. Background Art

[0002] Face recognition is a biometric technology that identifies people based on their facial features. It uses a camera or camcorder to collect images or video streams containing faces, automatically detects and tracks faces in images, and then performs facial recognition on the detected faces. It is also commonly called portrait recognition or facial recognition.

[0003] Face recognition technology is mainly based on face collection by camera sensors. The direction of the camera sensor has a great influence on the collection of face recognition. Since the image sensor is directional, if the patch is soldered upside down, the image presented will be reversed. If the image is upside down, the face cannot be recognized normally. If the SMT patch soldering direction of the sensor components is different, the sensor has different patch directions, and the sensor has different patch directions, the patch may appear in 90°, 180° or 270° directions that are prone to recognition errors, the collected and displayed image may be upside down, affecting the rationality of the image display and the acquisition of algorithm features, and then recognition errors.

[0004] The usual practice is to match the sensor driver and perform some adaptation operation control on the sensor in software, which may generate multiple firmwares, where firmware refers to programs written in EPROM (erasable programmable read-only memory) or EEPROM (electrically erasable programmable read-only memory). This is not conducive to version management. Summary of the invention

[0005] In order to solve the problem that the image displayed in the prior art is inverted, the sensor needs to be driven and matched, which is not conducive to version management, the present invention provides a face recognition method, system, computer equipment and storage medium, and proposes face recognition based on RGA (2D hardware accelerator) for image rotation processing and ISP (image signal processor) for rotation exposure processing, adaptive sensor patch angle and dark environment. The present invention can adaptively rotate image data in different sensor patch directions to achieve the normal display of the image, and transmit the corrected image data to the algorithm to obtain the correct feature value, so as to achieve the effect of adaptive correct face recognition.

[0006] The present invention is implemented by the following technical solutions:

[0007] A face recognition method, comprising:

[0008] Get the captured image data;

[0009] Initialize the acquired image data and determine whether there is a rotation flag; if not, crop the face and read the feature value of the captured face; based on the processed feature value and face direction recognition, obtain the rotation value, assign a value to the image and save the rotation flag; if yes, perform a corresponding angle rotation according to the value of the rotation flag to obtain normal image data.

[0010] As a further solution of the present invention, before the captured image data, the method further includes:

[0011] The camera module is connected to an image processor, which collects images containing human faces. The image processor processes the collected images. The camera module is equipped with an IR camera / infrared lens, which can identify living bodies, avoiding the situation where identification can be achieved using photos, and greatly improving identification security.

[0012] Furthermore, when the camera module uses a video stream containing a face, video frame processing is performed to obtain an image containing a face.

[0013] Furthermore, the processing of the collected images or video streams includes linear correction, white balance, automatic exposure and noise removal on the collected images containing faces to obtain high-quality images.

[0014] Furthermore, the image processor internally includes a CPU, SUP IP (a general term for various functional modules) and an IF device, and the CPU chip of the image processor is integrated with an RGA accelerator for performing 2D image operations.

[0015] Furthermore, the 2D image operation includes but is not limited to image processing operations such as image scaling, rotation, bitBlt, and alpha blending.

[0016] Furthermore, before obtaining the captured image data, the method also includes: initializing the camera module through the V4l2 framework; after the initialization process, the upper layer of the application issues an instruction to capture the image.

[0017] As a further solution of the present invention, when there is no rotation flag, before cutting out the face and reading the feature value of the captured face, the method further includes:

[0018] The image data without the rotation flag is transmitted to the sensor direction discrimination function for processing to discriminate the current direction of the image data;

[0019] The processed image data is input into the NPU algorithm model, and face recognition processing is performed on the input image to crop out the face, read the feature values ​​of the captured face, assign values ​​to the image, and save the rotation flag.

[0020] Furthermore, when reading the feature values ​​of the captured face, the feature values ​​of the face in the current direction of the image data are compared with a preset correct facial feature model, the image is assigned a value and a rotation flag is generated for storage, wherein the facial feature model is a basic point model of the eyes, nose, and mouth that outlines the basic shape of a person.

[0021] As a further solution of the present invention, the operation of performing corresponding angle rotation according to the value of the rotation flag is performed in the RGA accelerator. According to the Boolean value or logical quantity that needs to be rotated indicated by the rotation flag, the determined rotation value is input into the RGA accelerator to obtain a correct image.

[0022] Furthermore, the normal image data obtained is also used to be sent to a display for display and to an algorithm for processing to obtain correct feature values.

[0023] The present invention also includes a face recognition system, which adopts the above-mentioned face recognition method to realize image rotation processing and rotation exposure processing, adaptive sensor patch angle and face recognition in dark environment, and displays the image positively; the face recognition system includes a camera module, an image processor, a capture module, a rotation flag position judgment module, a rotation flag position setting module and a rotation module.

[0024] The camera module is used to capture images containing human faces; the image processor is connected to the camera module and is used to process the captured images containing human faces to obtain high-quality images; the capture module is used to capture images according to instructions issued by the upper application layer and obtain captured image data; the rotation flag judgment module is used to judge whether the captured image data has a rotation flag; the rotation flag setting module is used to judge the direction of the video face image when the image data does not have a rotation flag, compare whether rotation is required, and if rotation is required, cut out the face and read the feature value of the captured face, set and save the rotation flag on the image data, and clear the rotation flag when rotation is not required; and the rotation module is used to input the value of the rotation flag into the RGA accelerator when it is judged that the captured image data has a rotation flag, and perform a corresponding angle rotation to obtain normal image data.

[0025] The present invention also includes a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the face recognition method when executing the computer program.

[0026] The present invention also includes a storage medium storing a computer program, which implements the steps of the face recognition method when executed by a processor.

[0027] The technical solution provided by the present invention has the following beneficial effects:

[0028] The present invention is based on the platform's RGA (2D hardware accelerator) and neural network unit (NPU). The camera acquires video data, processes the facial features and determines the direction through the software algorithm of the internally integrated neural network processor, and performs the corresponding direction rotation through the RGA, which can quickly and efficiently complete the adaptive normal display of the face direction. In the present invention, all big data operations such as image processing are processed in the embedded neural network processor (NPU), and the efficiency is greatly improved. The recognition time is about 200-300ms, which is less than most algorithms on the market; the power consumption is also significantly reduced compared to the traditional CPU processing method; and because it is equipped with an IR camera / infrared lens, it can identify living bodies, avoiding the situation where photos can be used for recognition, and the recognition security is greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0030] Figure 1 FIG. 4 is a system block diagram of a face recognition system in one embodiment of the present invention.

[0031] Figure 2 The present invention is a flowchart of a face recognition system for implementing face recognition image rotation in one embodiment of the present invention.

[0032] Figure 3 The figure is a general design block diagram of a face recognition system in one embodiment of the present invention.

[0033] Figure 4 The present invention is a flowchart of video acquisition of a face recognition system in one embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0035] The present invention provides a face recognition method, system, computer device and storage medium. When the sensor has different patch directions, the collected and displayed image may be inverted, which affects the rationality of the image display and the acquisition of algorithm feature values, and then the problem of recognition error. This is different from the conventional practice of driving and matching the sensor, which may produce multiple firmware. The present invention proposes face recognition based on RGA (2D hardware accelerator) for image rotation processing and ISP (image signal processor) for rotation exposure processing, adaptive sensor patch angle and dark environment. The image data can be adaptively rotated in different sensor patch directions to achieve the display of the image, and the corrected image data is transmitted to the algorithm to obtain the correct feature value, so as to achieve the effect of adaptive correct face recognition, and can quickly and efficiently complete the adaptive normal display of the face direction.

[0036] In one embodiment of the present invention, a face recognition system is provided. Figure 1 As shown, the system includes a camera module 11, an image processor 12, a capture module 13, a rotation mark position determination module 14, a rotation mark position setting module 15 and a rotation module 16.

[0037] The system is used to realize image rotation processing and rotation exposure processing, adaptive sensor patch angle and face recognition in dark environment, and display images in a positive direction. The camera module 11 is used to collect images containing faces. The camera module 11 has an IR camera / infrared lens, which can identify living bodies, avoiding the situation where photos can be used for recognition, and greatly improving recognition security.

[0038] The image processor 12 is connected to the camera module 11 and is used to process the collected image containing the face to obtain a high-quality image. Figure 2 As shown, the image processor 12 includes ISP, NPU, RGA, the platform ISP chip is connected to the camera module 11, and the CPU is internally integrated with processors such as NPU and RGA. RGA (Raster Graphic Acceleration Unit) is an independent 2D hardware accelerator that can be used to accelerate point / line drawing, perform common 2D image operations such as image scaling, rotation, bitBlt, alpha blending, etc. Among them, the image processor 12 is connected to a memory for storing collected image data and image processing data.

[0039] See also Figure 3As shown, the capture module 13 is used to capture images according to instructions issued by the upper application layer and obtain captured image data. The rotation flag judgment module 14 is used to judge whether the captured image data has a rotation flag. If there is no rotation flag in the image data, the rotation flag setting module 15 is used to judge the direction of the video face image and compare whether rotation is required. If rotation is required, the face is cut out and the feature value of the captured face is read, and the rotation flag is set on the image data to save it. If rotation is not required, the rotation flag is cleared.

[0040] The rotation module 16 is used for inputting the value of the rotation flag into the RGA accelerator to perform a corresponding angle rotation to obtain normal image data when it is determined that the captured image data has a rotation flag.

[0041] See also Figure 1 , Figure 2 and Figure 3 As shown, when performing image acquisition operations, the camera module 11 is connected to the ISP (Image Signal Processor 12), which can process the images obtained from the camera module 11, such as linear correction, white balance, automatic exposure, noise removal, etc., which can greatly improve the image quality. The CPU chip integrates RGA. RGA (Raster Graphic Acceleration Unit) is an independent 2D hardware accelerator that can be used to accelerate point / line drawing, perform common 2D image operations such as image scaling, rotation, bitBlt, alpha blending, etc.; NPU (embedded neural network processor) is very good at processing multimedia resources such as video images. After all initializations are completed, the upper layer of the application can issue instructions to capture images. In this solution, the camera is initialized and captured through the v4l2 framework.

[0042] The upper layer captures the image data and initializes to determine whether there is a rotation flag. If not, it is sent to the sensor direction discrimination function for processing. Next, the data is passed to the NPU (embedded neural network processor). The NPU will process the incoming image according to the algorithm model. The face recognition process is generally to cut out the face and read the feature values ​​of the captured face. The processed data is subjected to face direction recognition and judgment to obtain the rotation value, such as 0, 90, 270, etc., and the rotation flag is assigned and saved. After initialization, the corresponding angle rotation can be performed in the RGA according to the value of the rotation flag to obtain normal image data. This system is also used to transmit normal image data display and algorithm processing.

[0043] In this embodiment, the application upper layer refers to the upper layer application, and the operating system is divided into the bottom kernel layer, also known as the driver layer, the middle layer and the upper application layer. In the v4l2 framework, V4L2 is the abbreviation of Video for Linux2, which is the kernel driver for video devices in Linux. In Linux, video devices are device files, which can be read and written like accessing ordinary files. The camera is under / dev / video*. If there is only one video device, it is usually / dev / video0.

[0044] In this embodiment, the discriminant function is:

[0045] RK_S32

[0046] RK_MPI_MB_GetImageInfo(MEDIA_BUFFER mb,MB_IMAGE_INFO_S*pstImageInfo).

[0047] See also Figure 4 As shown, the camera module 11 can also collect video. When the video device is turned on, the data and collection method are set, and the video device is closed after the collected data is processed. When the camera module 11 uses a video stream containing a face, video frame processing is performed to obtain an image containing a face. The collection method is that after the video device is turned on, the properties of the video device can be set, such as cropping, scaling, etc. This step is optional. In Linux programming, the ioctl function is generally used to manage the I / O channels of the device. When performing video collection, it is necessary to open the device file, obtain the capability of the device, and see what functions the device has, such as whether it has video input, or audio input and output, etc. Select the video input. A video device can have multiple video inputs; set the video format and frame format. The formats include PAL, NTSC, and the frame formats include width and height, etc.

[0048] In the present invention, a frame buffer is applied to the driver, generally not more than 5, and the applied frame buffer is mapped to the user space, so that the collected frames can be directly operated without copying. All the applied frame buffers are queued to store the collected data. As time goes by, the video collection starts, and the queue is dequeued to obtain the frame buffer of the collected data and obtain the original collected data. VIDIOC_DQBUF, the buffer is re-entered into the end of the queue, so that the collection can be cyclic. Finally, the video collection is stopped and the video device is turned off.

[0049] join Figure 2 and Figure 3As shown, in one embodiment of the present invention, a face recognition method is also provided to realize image rotation processing and rotation exposure processing, adaptive sensor patch angle and face recognition in dark environment, and display the image in a positive direction, including:

[0050] Step 1: Get the captured image data.

[0051] In this embodiment, before capturing image data, the camera module 11 is connected to the image processor 12. The camera module 11 captures images containing faces, and the image processor 12 processes the captured images. The camera module 11 is equipped with an IR camera / infrared lens, which can identify living bodies, avoiding the situation where identification can be achieved using photos, and greatly improving identification security.

[0052] When the camera module 11 uses a video stream containing a face, video frame processing is performed to obtain an image containing a face, and the captured image or video stream is processed, including linear correction, white balance, automatic exposure and noise removal of the captured image containing a face, to obtain a high-quality image.

[0053] The image processor 12 includes a CPU, SUP IP (a general term for various functional modules) and an IF device. The CPU chip of the image processor 12 is integrated with an RGA accelerator for performing 2D image operations, including but not limited to image scaling, rotation, bitBlt, alpha blending and other image processing operations.

[0054] In this embodiment, before acquiring the captured image data, the process also includes: initializing the camera module 11 through the V4l2 framework; after the initialization process, the upper layer of the application issues an instruction to capture the image.

[0055] Step 2: Initialize the acquired image data and determine whether there is a rotation flag.

[0056] In this embodiment, when there is no rotation flag, the face is cut out and the feature value of the captured face is read; the rotation value is obtained based on the processed feature value and the face direction recognition judgment, and the image is assigned and the rotation flag is saved. Specifically, the image data without the rotation flag is transmitted to the sensor direction discrimination function for processing to determine the current direction of the image data; the processed image data is input into the NPU algorithm model, and the input image is subjected to face recognition processing to cut out the face, read the feature value of the captured face, and assign the image and save the rotation flag.

[0057] In this embodiment, when reading the feature values ​​of the captured face, the feature values ​​of the face in the current direction of the image data are compared with the preset correct facial feature model, the image is assigned a value and a rotation flag is generated and saved, wherein the facial feature model is the basic points of the eyes, nose, and mouth that outline the basic shape of a person.

[0058] When there is a rotation flag, a corresponding angle rotation is performed according to the value of the rotation flag to obtain normal image data. The operation of performing a corresponding angle rotation according to the value of the rotation flag is performed in the RGA accelerator. According to the Boolean value and logical quantity that need to be rotated indicated by the rotation flag, the determined rotation value is input into the RGA accelerator to obtain the correct image.

[0059] In an embodiment of the present invention, the normal image data is also sent to a display for display and to an algorithm for processing to obtain correct feature values.

[0060] When using the face recognition method of the present invention to perform image recognition, see Figure 2 and Figure 3 As shown. At the beginning, the image video stream is intercepted to determine whether there is a rotation flag. If not, the video face direction is determined to compare whether it needs to be rotated. If it needs to be rotated, the rotation flag is set and saved, otherwise the rotation flag is cleared; when the intercepted image video stream has a rotation flag, RGA rotation is used, and face detection is performed to determine whether the face is correct. If it is, the face is partially exposed and input into the algorithm for processing. If the face detection is negative, the return value starts.

[0061] When RGA rotates, it obtains a frame of data from ISP, and obtains width, height, and video format parameters from the intercepted data, such as the format parameters of 640*720 width and height. According to the intercepted data, two data spaces, source and destination, are allocated to allocate memory, and then rga_set_rect sets the source rect, destination rect, and conversion angle value, where the source is the required image data and the destination is the video stream data; finally, RkRgaBlit synthesizes the source and the converted destination and sends them to the display and the ISP displays the rotated image.

[0062] This embodiment is based on the platform's RGA (2D hardware accelerator) and neural network unit (NPU). The camera acquires video data, processes facial features and determines the direction through software algorithms for the internally integrated neural network processor, and performs corresponding direction rotation through RGA, which can quickly and efficiently complete the adaptive normal display of the face direction. In the present invention, all big data operations such as image processing are processed in the embedded neural network processor (NPU), which greatly improves efficiency. The recognition time is about 200-300ms, which is less than most algorithms on the market; the power consumption is also significantly reduced compared to the traditional CPU processing method; and because it is equipped with an IR camera / infrared lens, it can identify living bodies, avoiding the situation where photos can be used for recognition, and greatly improving recognition security.

[0063] In one embodiment of the present invention, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiment are implemented:

[0064] Get the captured image data;

[0065] Initialize the acquired image data and determine whether there is a rotation flag; if not, crop the face and read the feature value of the captured face; based on the processed feature value and face direction recognition, obtain the rotation value, assign a value to the image and save the rotation flag; if yes, perform a corresponding angle rotation according to the value of the rotation flag to obtain normal image data.

[0066] In an embodiment of the present invention, a storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented:

[0067] Get the captured image data;

[0068] Initialize the acquired image data and determine whether there is a rotation flag; if not, crop the face and read the feature value of the captured face; based on the processed feature value and face direction recognition, obtain the rotation value, assign a value to the image and save the rotation flag; if yes, perform a corresponding angle rotation according to the value of the rotation flag to obtain normal image data.

[0069] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory.

[0070] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A face recognition method, characterized in that: The face recognition method comprises: Get the captured image data; Initialize the acquired image data and determine whether there is a rotation flag; if not, crop the face and read the feature value of the captured face; perform face direction recognition and judgment based on the processed feature value, obtain the rotation value, assign a value to the image and save the rotation flag; if yes, perform a corresponding angle rotation based on the value of the rotation flag to obtain normal image data.

2. The face recognition method according to claim 1, characterized in that: Before the captured image data, it also includes: a camera module is connected to an image processor, the camera module captures images containing faces, and the image processor processes the captured images.

3. The face recognition method according to claim 2, characterized in that: The processing of the collected image includes linear correction, white balance, automatic exposure and noise removal on the collected image containing the face, so as to obtain a high-quality image.

4. The face recognition method according to claim 3, characterized in that: The image processor includes a CPU and an IF device. The CPU chip of the image processor is integrated with an RGA accelerator for performing 2D image operations.

5. The face recognition method according to claim 4, characterized in that: Before obtaining the captured image data, the method also includes: initializing the camera module through the V4l2 framework; after the initialization process, the upper layer of the application issues an instruction to capture the image.

6. The face recognition method according to claim 1, wherein: When there is no rotation flag, before cutting out the face and reading the feature value of the captured face, it also includes: The image data without the rotation flag is transmitted to the sensor direction discrimination function for processing to discriminate the current direction of the image data; The processed image data is input into the NPU algorithm model, and face recognition processing is performed on the input image to crop out the face, read the feature values ​​of the captured face, assign values ​​to the image, and save the rotation flag.

7. The face recognition method according to claim 1, characterized in that: The operation of performing corresponding angle rotation according to the value of the rotation flag is performed in the RGA accelerator. According to the logical amount that needs to be rotated indicated by the rotation flag, the determined rotation value is input into the RGA accelerator to obtain a correct image. The normal image data obtained is also used to be sent to the display for display and to the algorithm for processing to obtain the correct feature value.

8. A face recognition system, characterized in that: The face recognition system adopts the face recognition method described in any one of claims 1 to 7 to realize image rotation processing, adapt to the sensor patch angle, and display the image in a positive direction; The face recognition system comprises: A camera module, used to capture images containing human faces; An image processor, connected to the camera module, for processing the collected image containing the face to obtain a high-quality image; A capture module is used to capture images according to instructions issued by the upper application layer and obtain captured image data; A rotation flag determination module is used to determine whether the captured image data has a rotation flag; A rotation flag setting module, used to determine the direction of the video face image when the image data does not have a rotation flag, compare whether rotation is required, and if rotation is required, cut out the face and read the feature value of the captured face, set the save rotation flag on the image data, and clear the rotation flag when rotation is not required; and The rotation module is used to input the value of the rotation flag into the RGA accelerator when it is determined that the captured image data has a rotation flag, and perform a corresponding angle rotation to obtain normal image data.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images