A face recognition optimization method and device

Abstract

This invention provides a face recognition optimization method, comprising: acquiring video data collected by the camera; determining the face pose in the video data by utilizing the correspondence between the coordinates of facial key points in the video data in the image coordinate system and the world coordinate system; and adjusting the yaw angle of the camera accordingly using the face pose to facilitate the acquisition of a frontal face image during the face recognition process.

Description

Technical Field

[0001] The present invention relates to the field of face recognition, and in particular to a face recognition optimization method and a device for face recognition in a driver's cab. Background Art

[0002] Trams are a common type of public transportation passenger vehicle, including rail trams, light rail trams, and streetcars. Existing rail trams, light rail trams, and streetcars require special power systems and tracks to operate, and the cost of infrastructure construction and vehicle purchase is high. To solve this problem, CRRC Corporation has proposed a tram concept that can follow virtual tracks on the ground. This new type of tram eliminates steel rails and follows the ground virtual track by means of rubber wheel bearing and steering.

[0003] The virtual ground track can be arranged flexibly, without the need for special infrastructure construction on the ground. It is only necessary to draw the virtual track for the new tram on the ground, like lane lines and zebra crossings. This new tram no longer needs to run along fixed tracks, and greatly reduces infrastructure costs, which has huge operational advantages compared to trams. At the same time, the operating characteristics of this new tram, "sharing road rights and mixed traffic", give the transportation system the advantage of flexible organization in terms of ground lane layout.

[0004] The tram developed by CRRC Zhuzhou in 2016 has realized the vision-based automatic tracking function, further improving the intelligence level of this new tram. From the release of the intelligent express system in June 2017 to the opening of the T1 line of the Sichuan Yibin Intelligent Express System in June 2019, this new tram has been put into operation in three cities, including Zhuzhou, Hunan, Yongxiu, Jiangxi, and Yibin, Sichuan.

[0005] Identification of people in the driver's cab of a tram is an important part of ensuring the safety of tram driving. Among them, face recognition is a relatively safe identification method that is being developed.

[0006] Despite years of development, face recognition products are still affected by factors such as lighting, posture, expression, occlusion, age, etc. Traditional face recognition technology can be divided into five categories: template matching-based methods, geometric feature-based methods, algebraic feature-based methods, artificial neural network-based methods, and sparse representation-based methods.

[0007] Although traditional methods have achieved a lot of results in the research of face recognition technology, due to the complexity of face recognition itself, such as the diversity of backgrounds and the change of postures, these nonlinear factors make traditional methods have great defects, which greatly limits the application of these methods in practice. In addition, due to the limitations of algorithms and computer performance, there is no ability to train large-scale face data sets, which further deepens the defects of traditional methods.

[0008] In order to solve the problems existing in the existing face recognition methods, the present invention aims to propose a face recognition optimization method, which can adjust the angle of the camera shooting face data to collect the frontal face video of the driver, thereby improving the accuracy of face recognition. Summary of the invention

[0009] A brief summary of one or more aspects is given below to provide a basic understanding of these aspects. This summary is not an exhaustive overview of all conceived aspects, and is neither intended to identify the key or critical elements of all aspects nor to define the scope of any or all aspects. Its only purpose is to give some concepts of one or more aspects in a simplified form as a prelude to a more detailed description that will be given later.

[0010] According to one aspect of the present invention, a face recognition optimization method is provided, comprising: acquiring video data captured by the camera; determining the face posture in the video data using the correspondence between the coordinates of face key points in the video data in the image coordinate system and the world coordinate system; and adjusting the yaw angle of the camera accordingly using the face posture to facilitate the capture of a frontal face image during the face recognition process.

[0011] In one embodiment, the method of determining the facial posture in the video data by using the correspondence between the coordinates of the facial key points in the video data in the image coordinate system and the world coordinate system includes: determining the 2D coordinates of at least three facial key points in the image coordinate system by using a regression tree set facial key point positioning algorithm; determining the 3D coordinates of the at least three facial key points in the world coordinate system by using a universal face 3D model; determining the rotation matrix from the world coordinate system to the image coordinate system by using the 2D coordinates of the at least three facial key points and the corresponding 3D coordinates; and calculating the Euler angles of the face based on the rotation matrix, wherein the Euler angles represent the facial posture.

[0012] In one embodiment, the use of the 2D coordinates of the at least three facial key points and the corresponding 3D coordinates to determine the rotation matrix from the world coordinate system to the image coordinate system includes: substituting the 2D coordinates (x, y) and the corresponding 3D coordinates (U, V, W) of the at least three facial key points into the equation group

[0013]

[0014] To calculate the rotation matrix Among them, f x and f y is the focal length of the camera in both directions of the image coordinate system, (c x ,c y ) is the optical center, is the translation matrix from the camera coordinate system to the image coordinate system.

[0015] In one embodiment, calculating the Euler angles of the face based on the rotation matrix includes: using the Euler angle calculation formula Calculate the Euler angle of the face, where θ x is the pitch angle, θ y is the yaw angle, θ z is the roll angle, the rotation matrix

[0016] In one embodiment, the face recognition optimization method further includes: using the adjusted camera to collect video data of the driver's cab; and using a deep learning algorithm to identify facial information in the video data to identify driver information.

[0017] In one embodiment, the facial information includes a facial feature vector, and a local facial feature database stores a number of drivers and their facial feature vectors. The use of a deep learning algorithm to identify the facial information in the video data to identify the driver information includes: using a YOLOv3 Tiny algorithm to detect the facial image in the video data; using a regression tree set facial key point positioning algorithm to align the facial image to obtain a standard facial image; using an ArcFace deep face recognition method to extract the facial feature vector in the standard facial image; and comparing the facial feature vector with the facial feature vector of each driver in the local facial feature database to identify the driver corresponding to the facial image.

[0018] In one embodiment, detecting the facial image in the video data using the YOLOv3 Tiny algorithm includes: training the facial image data using the YOLOv3 Tiny algorithm to obtain a face detection model; and inputting the video data into the face detection model to obtain the facial image.

[0019] In one embodiment, the method of extracting the facial feature vector from the standard facial image using the ArcFace deep face recognition method includes: training a deep neural network model using a standard face data set to obtain a face recognition model; and obtaining a 512-dimensional facial feature vector using a deep representation of the deep learning image of the face recognition model.

[0020] In one embodiment, comparing the facial feature vector with the facial feature vector of each driver in a local facial feature database to identify the driver corresponding to the facial image includes: comparing and identifying the facial feature vector with each driver in the local facial feature database to calculate the corresponding cosine similarity; in response to the cosine similarity between the facial feature vector and the facial feature vector of any driver being greater than a preset threshold, determining the driver as the driver corresponding to the facial image; and in response to the cosine similarities between the facial feature vector and the facial feature vectors of all drivers being less than or equal to a preset threshold, judging that facial recognition has failed.

[0021] According to another aspect of the present invention, a face recognition optimization device is also provided, including: a memory; and a processor connected to the memory, the processor being configured to: obtain video data captured by the camera; determine the face posture in the video data using the correspondence between the coordinates of face key points in the video data in the image coordinate system and the world coordinate system; and adjust the yaw angle of the camera accordingly using the face posture to facilitate the capture of a frontal face image during the face recognition process.

[0022] In one embodiment, the processor is further configured to: determine the 2D coordinates of at least three facial key points in the image coordinate system using a regression tree set facial key point location algorithm; determine the 3D coordinates of the at least three facial key points in the world coordinate system using a universal face 3D model; determine the rotation matrix from the world coordinate system to the image coordinate system using the 2D coordinates of the at least three facial key points and the corresponding 3D coordinates; and calculate the Euler angles of the face based on the rotation matrix, wherein the Euler angles represent the facial posture.

[0023] In one embodiment, the processor is further configured to: substitute the 2D coordinates (x, y) and the corresponding 3D coordinates (U, V, W) of the at least three facial key points into the equation system

[0024]

[0025] To calculate the rotation matrix Among them, f x and f y is the focal length of the camera in both directions of the image coordinate system, (c x ,c y ) is the optical center, is the translation matrix from the camera coordinate system to the image coordinate system.

[0026] In one embodiment, the processor is further configured to: use the Euler angle calculation formula Calculate the Euler angle of the face, where θ x is the pitch angle, θ y is the yaw angle, θ z is the roll angle, the rotation matrix

[0027] In one embodiment, the processor is further configured to: collect video data of the driver's cab using the adjusted camera; and identify facial information in the video data using a deep learning algorithm to identify driver information.

[0028] In one embodiment, the facial information includes a facial feature vector, and a local facial feature database stores a number of drivers and their facial feature vectors. The processor is further configured to: detect facial images in the video data using the YOLOv3 Tiny algorithm; align the facial images using a regression tree set facial key point positioning algorithm to obtain a standard facial image; extract the facial feature vector from the standard facial image using the ArcFace deep face recognition method; and compare the facial feature vector with the facial feature vector of each driver in the local facial feature database to identify the driver corresponding to the facial image.

[0029] In one embodiment, the processor is further configured to: train facial image data using the YOLOv3 Tiny algorithm to obtain a face detection model; and input the video data into the face detection model to obtain the facial image.

[0030] In one embodiment, the processor is further configured to: train a deep neural network model using a standard face data set to obtain a face recognition model; and obtain a 512-dimensional face feature vector using a deep representation of the deep learning image of the face recognition model.

[0031] In one embodiment, the processor is further configured to: compare and identify the facial feature vector with each driver in a local facial feature database to calculate the corresponding cosine similarity; in response to the cosine similarity between the facial feature vector and the facial feature vector of any driver being greater than a preset threshold, determine the driver as the driver corresponding to the facial image; and in response to the cosine similarity between the facial feature vector and the facial feature vectors of all drivers being less than or equal to a preset threshold, determine that facial recognition has failed.

[0032] According to another aspect of the present invention, there is further provided a computer storage medium on which a computer program is stored. When the computer program is executed, the steps of the face recognition optimization method described in any of the above embodiments are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The above features and advantages of the present invention will be better understood after reading the detailed description of the embodiments of the present disclosure in conjunction with the following drawings.

[0034] Figure 1 is a flow chart of a face recognition optimization method in an embodiment according to one aspect of the present invention;

[0035] Figure 2 is a schematic diagram of facial posture angles according to one aspect of the present invention;

[0036] Figure 3 is a schematic diagram of the correspondence relationship between three coordinate systems drawn according to one aspect of the present invention;

[0037] Figure 4 is a partial flow chart of a face recognition optimization method in an embodiment according to one aspect of the present invention;

[0038] Figure 5 is a schematic diagram of 2D-3D correspondence relationship of facial feature points drawn according to one aspect of the present invention;

[0039] Figure 6 is a schematic diagram showing the relative position relationship between a camera and a driver in a cab according to one aspect of the present invention;

[0040] Figure 7 It is a simple schematic diagram of the relative position relationship between the camera and the driver in the driver's cab from a top view according to one aspect of the present invention;

[0041] Figure 8 is a partial flow chart of a face recognition optimization method in an embodiment according to one aspect of the present invention;

[0042] Fig. 9 is a partial flow chart of a face recognition optimization method in an embodiment according to one aspect of the present invention;

[0043] Fig.10 1 is a schematic diagram of the target detection principle of the YOLO algorithm according to one aspect of the present invention;

[0044] Fig.11 is a schematic diagram of the actual effect of face alignment achieved by the ERT algorithm according to one aspect of the present invention;

[0045] Fig.12is a schematic diagram of additional angle margins according to one aspect of the present invention;

[0046] Fig.13 is a module block diagram of face recognition optimization in an embodiment according to another aspect of the present invention. DETAILED DESCRIPTION

[0047] The following description is given to enable those skilled in the art to implement and use the invention and incorporate it into a specific application context. Various modifications, as well as various uses in different applications will be readily apparent to those skilled in the art, and the general principles defined herein are applicable to a wide range of embodiments. Thus, the present invention is not limited to the embodiments given herein, but should be granted the broadest scope consistent with the principles and novel features disclosed herein.

[0048] In the following detailed description, many specific details are set forth to provide a more thorough understanding of the present invention. However, it will be apparent to those skilled in the art that the practice of the present invention need not be limited to these specific details. In other words, known structures and devices are shown in block diagram form without detailed display to avoid obscuring the present invention.

[0049] The reader is reminded of all documents and literature filed simultaneously with this specification and open to public inspection, and the contents of all such documents and literature are incorporated herein by reference. Unless otherwise directly stated, all features disclosed in this specification (including any attached claims, abstracts and drawings) can be replaced by alternative features for achieving the same, equivalent or similar purposes. Therefore, unless otherwise explicitly stated, each feature disclosed is only an example of a group of equivalent or similar features.

[0050] Note that, where used, the symbols left, right, front, back, top, bottom, forward, reverse, clockwise and counterclockwise are used only for convenience and do not imply any specific fixed direction. In fact, they are used to reflect the relative position and / or direction between the various parts of the object. In addition, the terms "first" and "second" are used only for descriptive purposes and should not be understood as indicating or implying relative importance.

[0051] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0052] Note that, in the case of use, further, preferably, further and more preferably are simple beginnings for explaining another embodiment based on the previous embodiment, and the content of further, preferably, further or more preferably followed by the above embodiment is combined with the previous embodiment as a complete composition of another embodiment. Several further, preferably, further or more preferably settings followed by the same embodiment can be arbitrarily combined to form another embodiment.

[0053] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. Note that the aspects described below in conjunction with the accompanying drawings and specific embodiments are only exemplary and should not be construed as limiting the scope of protection of the present invention in any way.

[0054] According to one aspect of the present invention, there is provided a face recognition optimization method applicable to a camera in a driver's cab for capturing face video data.

[0055] In one embodiment, if Figure 1 As shown, the face recognition optimization method 100 may include steps S110 to S130.

[0056] Wherein, step S110 is: obtaining video data collected by the camera.

[0057] It can be understood that in order to capture the frontal face video data of the driver in the cab, it is necessary to first collect the current video data of the driver to determine the face angle of the driver.

[0058] Step S120 is: determining the facial posture in the video data by using the correspondence between the coordinates of the facial key points in the video data in the image coordinate system and the world coordinate system.

[0059] In computer vision, the face posture is its direction and position relative to the camera, which is usually expressed in Euler angles. Euler angles include three angles: pitch, yaw, and roll. These three angles are also called posture angles. Figure 2 As shown in Figure 1, determining the face pose requires calculating its Euler angle relative to the camera.

[0060] Face pose estimation requires three coordinate systems, namely the world coordinate system, the camera coordinate system, and the image coordinate system. The 3D coordinates of facial feature points are expressed in the world coordinate system. If the rotation and translation of the world coordinate system relative to the camera coordinate system are known, the 3D points in the world coordinate system can be converted to 3D points in the camera coordinate system. Then, the 3D points in the camera coordinate system can be projected onto the image plane using the intrinsic parameters of the camera (focal length, optical center, etc.), which become 2D points in the image coordinate system.

[0061] like Figure 3 As shown, o is the camera center, i.e. the origin of the camera coordinate system, and c is the image center, i.e. the origin of the image coordinate system. The position (U, V, W) of the 3D point P in the world coordinate system is transformed by rotation R and translation t, and then the coordinates (X, Y, Z) of the point P (U, V, W) in the world coordinate system when it is converted to the point P in the camera coordinate system can be calculated by equation (1).

[0062]

[0063] Expanding formula (1) yields:

[0064]

[0065]

[0066] Then, formula (1) can be converted into:

[0067]

[0068] Given the known camera intrinsics and scaling factor s, the coordinates (x, y) of point p in the image coordinate system are calculated as follows:

[0069]

[0070] Among them, f x and f y is the focal length in the x-axis and y-axis directions of the image coordinate system, (c x ,c y ) is the optical center. Combining equation (4) and equation (5), we can get equation (6).

[0071]

[0072] The matrix multiplication expansion in equation (6) can yield equations (7) and (8):

[0073]

[0074]

[0075] Substituting equation (8) into equation (7), we can get equation group (9):

[0076]

[0077] In formula (9), at f x 、f y 、c x and c yGiven that (x, y) and (U, V, W) are a set of corresponding 2D and 3D coordinates, there exists r 00 、r 01 、r 02 、r 10 、r 11 、r 12 、r 20 、r 21 、r 22 ,t x ,t y and t z There are 12 unknowns in total. Since the rotation matrix R is an orthogonal matrix, each row and column is a unit vector and intersects each other, the degree of freedom of the rotation matrix R is 3 and the rank is also 3. 00 、r 01 and r 10 In this case, the remaining unknowns of the rotation matrix R can be calculated, plus the 3 unknowns of the translation vector, a total of 6 unknowns, and a set of corresponding 2D coordinates and 3D coordinates can establish a system of equations. Through 3 sets of corresponding 2D coordinates and 3D coordinates, 3 systems of equations can be established to solve the 6 unknowns, so at least 3 sets of corresponding 2D coordinates and 3D coordinates are required to obtain the rotation matrix R.

[0078] It can be understood that after the rotation matrix R of the world coordinate system relative to the image coordinate system is determined using the currently collected face image, the rotation matrix R can be used to calculate the Euler angle θ of the face.

[0079] Preferably, in one embodiment, if Figure 4 As shown, step S120 may include steps S121 to S124.

[0080] Wherein, step S121 is: using the regression tree set facial key point location algorithm to determine the 2D coordinates of at least three facial key points in the image coordinate system.

[0081] like Figure 5 As shown, the facial key points for calculating the facial posture include the left corner of the left eye, the right corner of the right eye, the nose tip, the left corner of the mouth, the right corner of the mouth and the lower jaw. At least any three facial key points among the left corner of the left eye, the right corner of the right eye, the nose tip, the left corner of the mouth, the right corner of the mouth and the lower jaw can be selected and the 2D coordinates of the selected facial key points in the image coordinate system can be determined using the regression tree set positioning algorithm.

[0082] Step S122 is: using the universal face 3D model to determine the 3D coordinates of the at least three face key points in the world coordinate system.

[0083] Correspondingly, the 3D coordinates of the facial key points selected in step S121 in the world coordinate system are determined. The 2D coordinates of each facial key point in the image coordinate system correspond to the 3D coordinates of the facial key point in the world coordinate system.

[0084] Step S123 is: using the 2D coordinates of the at least three facial key points and the corresponding 3D coordinates to determine a rotation matrix from the world coordinate system to the image coordinate system.

[0085] Specifically, the 2D coordinates (x, y) and the corresponding 3D coordinates (U, V, W) of the facial key points selected in step S121 are substituted into equation group (9), and the rotation matrix R can be calculated using the PnP algorithm in OpenCV.

[0086] Step S124 is: calculating the Euler angles of the face based on the rotation matrix, where the Euler angles represent the face posture.

[0087] Specifically, the Euler angle calculation formula can be used Calculate the Euler angle of the face, where θ x is the pitch angle, θ y is the yaw angle, θ z is the roll angle.

[0088] Furthermore, step S130 is: adjusting the yaw angle of the camera accordingly according to the facial posture.

[0089] Figure 6 The relative position relationship between the camera and the driver in the cab of a tram is shown. Since the camera is fixed on the driving platform, the camera cannot roll or rotate. At the same time, the height of the driver on the seat and the pitch angle of the camera are fixed, so the camera cannot pitch or rotate either.

[0090] Figure 7 A simple top view schematic diagram of the relative position relationship between the camera and the driver in the cab of a tram is shown, wherein the base of the camera is an arc, the camera is centered on the face of the person, and can move 30° left and right on the base. Then when the yaw angle θ of the face is determined in step S120 y When the camera moves to the corresponding angle θ y .

[0091] Further, after adjusting the angle of the camera, the camera can capture the front face image of the driver in the cab, thereby increasing the accuracy of face recognition. Preferably, after adjusting the angle of the camera, the face recognition optimization method 100 can also include a face recognition step.

[0092] In one embodiment, if Figure 8As shown, the face recognition optimization method 100 also includes steps S140 to S150.

[0093] Among them, step S140 is: using the adjusted camera to collect video data of the driver's cab.

[0094] It can be understood that the video data captured by the adjusted camera includes the frontal facial image of the driver.

[0095] Step S150 is: using a deep learning algorithm to identify facial information in the video data to identify the driver information.

[0096] Specifically, Fig. 9 As shown, step S150 can be embodied as steps S151 to S154.

[0097] Wherein, step S151 is: using the YOLOv3 Tiny algorithm to detect the face image in the video data.

[0098] The YOLO algorithm uses a separate CNN model to achieve end-to-end object detection. The network input image size is 416*416. The core of YOLO is to use the entire image as the input of the network model and directly regress the position of the bounding box and its category at the output. Fig.10 As shown in the figure, an image is divided into S×S grids. If the center of an object falls in this grid, then this grid is responsible for predicting this object. Each bounding box needs to predict 5 parameter values ​​(x, y, w, h) and confidence. At the same time, the grid also needs to predict a category information, which is recorded as C. Then, in the S×S grids, each grid needs to predict B bounding boxes, and each bounding box has probability values ​​corresponding to C categories. The output is a vector of S×S×B×(5+C).

[0099] YOLOv3 Tiny has a total of 23 layers, including: 13 convolutional layers, 6 pooling layers, 2 routing layers, 1 upsampling layer, and 2 yolo output layers. In YOLOv3 Tiny, except for the convolutional layer before the yolo layer, each convolutional layer is followed by a BN layer, and each convolutional layer is followed by a Leaky RELU activation layer. The output sizes of the two yolo output layers are 13×13 and 26×26 respectively. Each grid can predict 3 bounding boxes, for a total of 80 categories.

[0100] Furthermore, the YOLOv3 Tiny algorithm may be used to train face image data to obtain a face detection model, and then the video data collected by the adjusted camera may be input into the face detection model to obtain a face image.

[0101] The experiment verified that the face detection model trained by the YOLOv3 Tiny algorithm can quickly and accurately detect faces and obtain face images under different lighting, background, posture and occlusion conditions. At the same time, because the network model of the YOLOv3 Tiny algorithm is small, it can also achieve real-time face detection speed on edge computing devices.

[0102] Step S152 is: aligning the face images using the regression tree set (ERT) facial key point location algorithm to obtain a standard face image.

[0103] The ERT facial key point location algorithm regresses the face shape to its true shape step by step by building a cascaded residual regression tree. Each leaf node of each residual regression tree stores a residual regressor. When the input falls on a node, the residual is added to the input to achieve the purpose of regression. Finally, all the residuals are superimposed together to complete the purpose of face alignment.

[0104] Preferably, the ERT algorithm implemented in the open source Dlib framework can be used to detect facial key points in face images. Fig.11 A schematic diagram of the actual effect of face alignment in an embodiment is shown, wherein the left image is an original face image, the middle image is a 68-point facial key point positioning image, and the right image is a standard face image using facial key points for face alignment.

[0105] Step S153 is: using the ArcFace deep face recognition method to extract the facial feature vector in the standard face image.

[0106] ArcFace deep face recognition method proposes a new loss function and additional angle margin (such as Fig.12 As shown). The loss function L is shown in formula (10), θ is the angle between classes, m is the additional angle margin, and s is the size of the feature vector. When θ∈[0,π-m], cos(θ+m) is less than cos(θ), so the constraints in the loss function L are stricter for face classification.

[0107]

[0108] The backbone network of ArcFace uses the ResNet residual network. Preferably, ResNet50 can be selected as the backbone network of ArcFace.

[0109] Specifically, a standard face data set may be used to train a deep neural network model to obtain a face recognition model, and then a 512-dimensional face feature vector may be obtained by using the deep representation of the deep learning image of the face recognition model.

[0110] Step S154 is: comparing the facial feature vector with the facial feature vector of each driver in the local facial feature database to identify the driver corresponding to the facial image.

[0111] Assuming that the local facial feature database stores information of N drivers, and the information of the drivers at least includes facial feature vectors, the facial feature vectors obtained in step S153 can be compared and identified with the N drivers in the local facial feature database, and N groups of cosine similarities can be calculated.

[0112] When any cosine similarity in the N groups of cosine similarities is greater than a preset threshold, that is, the cosine similarity between the facial feature vector obtained in step S153 and the facial feature vector of any driver is greater than the preset threshold, the driver associated with the cosine similarity greater than the preset threshold is determined as the driver corresponding to the collected facial image, and face recognition is completed.

[0113] Correspondingly, when the N groups of cosine similarities are all less than or equal to the preset threshold, that is, the cosine similarities between the facial feature vector obtained in step S153 and the facial feature vectors of all drivers are all less than or equal to the preset threshold, it is judged that face recognition has failed, and the face recognition failure result can be output and displayed.

[0114] Although the above methods are illustrated and described as a series of actions for simplicity of explanation, it should be understood and appreciated that these methods are not limited by the order of the actions, because according to one or more embodiments, some actions may occur in a different order and / or concurrently with other actions from those illustrated and described herein or not illustrated and described herein but understandable to those skilled in the art.

[0115] According to another aspect of the present invention, a face recognition optimization device is also provided. Fig.13 As shown, it includes a memory 1310 and a processor 1320 .

[0116] The memory 1310 is used to store computer programs.

[0117] The processor 1320 is connected to the memory 1310 and is used to execute the computer program in the memory 1310. When the computer program is executed, the steps of the face recognition optimization method in any of the above embodiments are implemented.

[0118] Those skilled in the art will appreciate that information, signals, and data may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips cited throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or optical particles, or any combination thereof.

[0119] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of the two. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps are generally described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. The technician may implement the described functionality in different ways for each specific application, but such implementation decisions should not be interpreted as resulting in a departure from the scope of the present invention.

[0120] The various illustrative logic modules and circuits described in conjunction with the embodiments disclosed herein may be implemented or executed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in cooperation with a DSP core, or any other such configuration.

[0121] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software module may reside in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read and write information from / to the storage medium. In an alternative, a storage medium may be integrated into a processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and the storage medium may reside in a user terminal as discrete components.

[0122] In one or more exemplary embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented as a computer program product in software, each function may be stored on or transmitted by a computer-readable medium as one or more instructions or codes. Computer-readable media include both computer storage media and communication media, including any medium that facilitates the transfer of a computer program from one place to another. Storage media may be any available medium that can be accessed by a computer. As an example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer. Any connection is also properly referred to as a computer-readable medium. For example, if the software is transmitted from a website, a server, or other remote source using a coaxial cable, a fiber optic cable, a twisted pair, a digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of the medium. Disk and disc as used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, wherein disk often reproduces data magnetically, while disc reproduces data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0123] The foregoing description is provided to enable any person skilled in the art to practice the various aspects described herein. However, it should be understood that the scope of protection of the present invention shall be subject to the appended claims and shall not be limited to the specific structures and components of the above-described embodiments. Those skilled in the art may make various changes and modifications to the embodiments within the spirit and scope of the present invention, and these changes and modifications also fall within the scope of protection of the present invention.

Claims

1. A face recognition optimization method, comprising: Get the video data collected by the camera; Determine the facial posture in the video data by using the correspondence between the coordinates of the facial key points in the video data in the image coordinate system and the world coordinate system, wherein the 2D coordinates of at least three facial key points in the image coordinate system are determined by using a regression tree set facial key point location algorithm, and the 3D coordinates of the at least three facial key points in the world coordinate system are determined by using a universal face 3D model; Substitute the 2D coordinates (x, y) and the corresponding 3D coordinates (U, V, W) of the at least three facial key points into the equation system To calculate the rotation matrix Among them, f x and f y is the focal length of the camera in both directions of the image coordinate system, (c x ,c y ) is the optical center, is a translation matrix from the camera coordinate system to the image coordinate system, so as to determine a rotation matrix from the world coordinate system to the image coordinate system; Based on the rotation matrix, the Euler angle of the face is calculated using the Euler angle calculation formula, and the Euler angle represents the face posture. The Euler angle calculation formula is: Among them, θ x is the pitch angle, θ y is the yaw angle, θ z is the roll angle; and The yaw angle of the camera is correspondingly adjusted using the facial posture so as to collect a frontal facial image during the facial recognition process.

2. The face recognition optimization method according to claim 1, characterized in that: Determining the facial posture in the video data by using the correspondence between the coordinates of the facial key points in the video data in the image coordinate system and the world coordinate system includes: Determine the 2D coordinates of at least three facial key points in the image coordinate system using the regression tree set facial key point location algorithm; The 3D coordinates of the at least three facial key points in the world coordinate system are determined using a universal 3D face model.

3. The face recognition optimization method according to claim 1, characterized in that: Also includes: Collect video data from the driver's cab using the adjusted camera; A deep learning algorithm is used to identify facial information in the video data to identify the driver information.

4. The face recognition optimization method according to claim 3, characterized in that: The facial information includes a facial feature vector, the local facial feature database stores a number of drivers and their facial feature vectors, and the method of using a deep learning algorithm to identify the facial information in the video data to identify the driver information includes: Detecting facial images in the video data using the YOLOv3 Tiny algorithm; Using a regression tree set facial key point location algorithm to align the facial images to obtain a standard facial image; Extracting facial feature vectors from the standard facial image using ArcFace deep face recognition method; and The facial feature vector is compared with the facial feature vector of each driver in a local facial feature database to identify the driver corresponding to the facial image.

5. The face recognition optimization method according to claim 4, characterized in that: The detecting of the face image in the video data by using the YOLOv3 Tiny algorithm comprises: Use the YOLOv3 Tiny algorithm to train face image data to obtain a face detection model; and The video data is input into the face detection model to obtain the face image.

6. The face recognition optimization method according to claim 4, characterized in that: The method of extracting the facial feature vector from the standard facial image using the ArcFace deep face recognition method includes: Using a standard face dataset to train a deep neural network model to obtain a face recognition model; and The deep representation of the deep learning image of the face recognition model is used to obtain a 512-dimensional face feature vector.

7. The face recognition optimization method according to claim 4, characterized in that: The comparing the facial feature vector with the facial feature vector of each driver in a local facial feature database to identify the driver corresponding to the facial image comprises: Compare and identify the facial feature vector with each driver in a local facial feature database to calculate the corresponding cosine similarity; In response to a cosine similarity between the facial feature vector and a facial feature vector of any driver being greater than a preset threshold, determining the driver as the driver corresponding to the facial image; and In response to the cosine similarities between the facial feature vector and the facial feature vectors of all drivers being less than or equal to a preset threshold, it is determined that face recognition has failed.

8. A face recognition optimization device, comprising: Memory; as well as a processor connected to the memory, the processor being configured to: Get the video data collected by the camera; Determine the facial posture in the video data by using the correspondence between the coordinates of the facial key points in the video data in the image coordinate system and the world coordinate system, wherein the 2D coordinates of at least three facial key points in the image coordinate system are determined by using a regression tree set facial key point location algorithm, and the 3D coordinates of the at least three facial key points in the world coordinate system are determined by using a universal face 3D model; Substitute the 2D coordinates (x, y) and the corresponding 3D coordinates (U, V, W) of the at least three facial key points into the equation system To calculate the rotation matrix Among them, f x and f y is the focal length of the camera in both directions of the image coordinate system, (c x ,c y ) is the optical center, is a translation matrix from the camera coordinate system to the image coordinate system, so as to determine a rotation matrix from the world coordinate system to the image coordinate system; Based on the rotation matrix, the Euler angle of the face is calculated using the Euler angle calculation formula, and the Euler angle represents the face posture. The Euler angle calculation formula is: Among them, θ x is the pitch angle, θ y is the yaw angle, θ z is the roll angle; and The yaw angle of the camera is correspondingly adjusted using the facial posture so as to collect a frontal facial image during the facial recognition process.

9. The face recognition optimization device according to claim 8, characterized in that: The processor is further configured to: Determine the 2D coordinates of at least three facial key points in the image coordinate system using the regression tree set facial key point location algorithm; The 3D coordinates of the at least three facial key points in the world coordinate system are determined using a universal 3D face model.

10. The face recognition optimization device according to claim 8, characterized in that: The processor is further configured to: Collect video data from the driver's cab using the adjusted camera; A deep learning algorithm is used to identify facial information in the video data to identify the driver information.

11. The face recognition optimization device according to claim 10, characterized in that: The facial information includes a facial feature vector, and the local facial feature database stores a number of drivers and their facial feature vectors. The processor is further configured to: Detecting facial images in the video data using the YOLOv3 Tiny algorithm; Using a regression tree set facial key point location algorithm to align the facial images to obtain a standard facial image; Extracting facial feature vectors from the standard facial image using ArcFace deep face recognition method; as well as The facial feature vector is compared with the facial feature vector of each driver in a local facial feature database to identify the driver corresponding to the facial image.

12. The face recognition optimization device according to claim 11, characterized in that: The processor is further configured to: Use the YOLOv3 Tiny algorithm to train face image data to obtain a face detection model; and The video data is input into the face detection model to obtain the face image.

13. The face recognition optimization device according to claim 11, characterized in that: The processor is further configured to: Using a standard face dataset to train a deep neural network model to obtain a face recognition model; and The deep representation of the deep learning image of the face recognition model is used to obtain a 512-dimensional face feature vector.

14. The face recognition optimization device according to claim 11, characterized in that: The processor is further configured to: Compare and identify the facial feature vector with each driver in a local facial feature database to calculate the corresponding cosine similarity; In response to a cosine similarity between the facial feature vector and a facial feature vector of any driver being greater than a preset threshold, determining the driver as the driver corresponding to the facial image; and In response to the cosine similarities between the facial feature vector and the facial feature vectors of all drivers being less than or equal to a preset threshold, it is determined that face recognition has failed.

15. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed, the steps of the face recognition optimization method according to any one of claims 1 to 7 are implemented.

Images