Face recognition method, device, storage medium and electronic device

Abstract

The embodiment of the present invention discloses a face recognition method, device, storage medium and electronic device. The method includes: a face recognition method, characterized in that it includes: obtaining a face image to be recognized; inputting the face image into a face recognition model to obtain a recognition result output by the face recognition model. The input face image is alternately extracted by a multi-head attention module and a multi-order mixed attention module to obtain high-order connection features in the face image, thereby realizing the representation of complex features in the face image. Furthermore, the global features and local features of the face image are obtained by a pooling model, which improves the comprehensiveness of the features, and the recognition probability is determined by the global features and the local features respectively, thereby improving the accuracy of face recognition.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of face recognition technology, and in particular to a face recognition method, device, storage medium and electronic device. Background Art

[0002] Facial recognition is widely used in smart security and smart cities.

[0003] Currently, the representation of faces is usually completed by manually constructing features or extracting features through CNN or transformer. After that, in the training phase, the features are used by the classifier to classify the faces in the training set, and in the testing phase, the features are matched with the facial features in the identity database and assigned a matching identity.

[0004] However, in the process of implementing the present invention, the inventors found that there are at least the following technical problems in the prior art: the current recognition of facial images is generally limited to learning local discriminative features in the face, which leads to the misjudgment of matching people of different identities with similar faces when performing identity matching. Summary of the invention

[0005] The embodiments of the present invention provide a face recognition method, device, storage medium and electronic device to improve the recognition accuracy of face recognition.

[0006] In a first aspect, an embodiment of the present invention provides a face recognition method, comprising:

[0007] Obtain a face image to be recognized;

[0008] Inputting the face image into a face recognition model to obtain a recognition result output by the face recognition model;

[0009] Among them, the face recognition model includes a feature extraction module, a pooling module and a classification module. The feature extraction module includes a first number of multi-head attention modules and a second number of multi-order mixed attention modules, which are used to extract visual sequence features with global connections. The pooling module is used to obtain global features and local features based on the visual sequence features. The classification module is used to obtain recognition results of face images based on the global features and the local features.

[0010] Optionally, the multi-head attention module and the multi-order mixed attention module are connected based on a preset connection order, and the multi-order mixed attention module includes multiple attention units, and a convolution unit is arranged between any two adjacent attention units.

[0011] Optionally, the convolution unit includes a deformable convolution and a depthwise separable convolution, which respectively perform convolution processing on the feature information output by the previous attention unit, and the processing results of the deformable convolution and the depthwise separable convolution are merged and input into the next attention unit.

[0012] Optionally, the attention parameters of any of the attention units are determined based on the first-order attention parameters and prior information of the current-order attention unit.

[0013] Optionally, the feature extraction module also includes an embedding layer, which is used to extract initial visual sequence features from the input face image and output the initial visual sequence features to a multi-head attention module connected to the embedding layer.

[0014] Optionally, the pooling module includes a global pooling unit and a local pooling unit, the global pooling unit is used to perform global pooling processing on the visual sequence features output by the feature extraction module to obtain global features; the local pooling unit is used to perform local pooling processing on the visual sequence features output by the feature extraction module to obtain multiple local features.

[0015] Optionally, the classification module includes multiple fully connected layers and a result processing unit, each fully connected layer is used to determine the recognition probability based on the global feature or any local feature, and the result processing unit determines the recognition result based on the recognition probability output by each of the fully connected layers.

[0016] In a second aspect, an embodiment of the present invention further provides a face recognition device, comprising:

[0017] A face image acquisition module, used to acquire a face image to be recognized;

[0018] A face recognition module, used to input the face image into a face recognition model to obtain a recognition result output by the face recognition model;

[0019] Among them, the face recognition model includes a feature extraction module, a pooling module and a classification module. The feature extraction module includes a first number of multi-head attention modules and a second number of multi-order mixed attention modules, which are used to extract visual sequence features with global connections. The pooling module is used to obtain global features and local features based on the visual sequence features. The classification module is used to obtain recognition results of face images based on the global features and the local features.

[0020] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a face recognition method as provided in any embodiment of the present invention is implemented.

[0021] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, wherein the computer executable instructions, when executed by a computer processor, are used to execute any face recognition method described in the embodiments of the present invention.

[0022] The technical solution provided in this embodiment performs high-precision face recognition on face images by setting a face recognition model, wherein the feature extraction module in the face model includes a multi-head attention module and a multi-order mixed attention module, and alternately extracts features from the input face image through the multi-head attention module and the multi-order mixed attention module to obtain high-order connection features in the face image, thereby realizing the representation of complex features in the face image. Furthermore, the global features and local features of the face image are obtained through the pooling model, which improves the comprehensiveness of the features, and the recognition probability is determined by the global features and the local features respectively, thereby improving the accuracy of face recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flow chart of a face recognition method provided by an embodiment of the present invention;

[0024] Figure 2 is a structural schematic diagram of a face recognition model provided by an embodiment of the present invention;

[0025] Figure 3 is a structural schematic diagram of a multi-order hybrid attention module provided by an embodiment of the present invention;

[0026] Figure 4 is a structural schematic diagram of a multi-order hybrid attention module provided by an embodiment of the present invention;

[0027] Figure 5 is a structural schematic diagram of a face recognition model provided by an embodiment of the present invention;

[0028] Figure 6 is a structural schematic diagram of a face recognition device provided by an embodiment of the present invention;

[0029] Figure 7 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0031] Figure 1This is a flow chart of a face recognition method provided by an embodiment of the present invention. This embodiment is applicable to the case of high-precision recognition of face images. The method can be executed by a face recognition device provided by an embodiment of the present invention. The face recognition device can be implemented by software and / or hardware. The face recognition device can be configured on an electronic device such as a computer, a mobile phone, etc., and specifically includes the following steps:

[0032] S110: Obtain a face image to be recognized.

[0033] S120: Input the facial image into a facial recognition model to obtain a recognition result output by the facial recognition model.

[0034] In this embodiment, obtaining a facial image may be receiving an imported image or obtaining a collected facial image, and the method for obtaining the facial image is not limited. Optionally, obtaining a facial image to be identified may be obtaining an image to be processed, identifying a facial region in the image, performing facial segmentation on the image to be processed, and obtaining a facial image to be identified based on the facial region obtained by segmentation. Specifically, a monochrome background may be added to the facial region obtained by segmentation to obtain a facial image to be identified. For example, the monochrome background may be a black background or a white background, etc., to reduce the interference of background information in the original image on facial recognition.

[0035] In some embodiments, after acquiring the facial image, the acquired facial image is further enhanced to improve the image quality, so as to facilitate the accurate recognition of the facial image by the facial recognition model. The implementation method of image enhancement is not limited in this embodiment. For example, it can be implemented based on a preset image enhancement model; it can also be based on a gamma correction method to perform adaptive white balance correction on the facial image; it can also be based on a preset power function to adjust the image brightness of the facial image, wherein the preset power function can be a power function of 0.9 or a power function of 1.3.

[0036] In some embodiments, the face image can also be scaled based on the input requirements of the face recognition model, for example, the face image can be scaled to a 256×256 image, where the target size of the face image is determined according to the input requirements of the face recognition model and is not limited to this.

[0037] The face recognition model includes a feature extraction module, a pooling module and a classification module. The feature extraction module includes a first number of multi-head attention modules and a second number of multi-order mixed attention modules, which are used to extract visual sequence features with global connections. The pooling module is used to obtain global features and local features based on the visual sequence features. The classification module is used to obtain the recognition result of the face image based on the global features and the local features. For example, see Figure 2 , Figure 2 It is a structural schematic diagram of a face recognition model provided by an embodiment of the present invention.

[0038] The preprocessed face image is input into the face recognition model, which is a recognition model that extracts and integrates local, full, high-order, and connected features of the face. It has high robustness and improves the recognition accuracy of the face.

[0039] In this embodiment, the face recognition model includes a feature extraction module, a pooling module and a classification module, the feature extraction module is used to extract visual sequence features with global connections, the pooling module is used to obtain global features and local features based on the visual sequence features, and the classification module is used to obtain recognition results of face images based on the global features and the local features. Optionally, the face image is input into the face recognition model to obtain the recognition result output by the face recognition model, including: inputting the face image into the feature extraction module to obtain visual sequence features with global connections output by the feature extraction module, inputting the visual sequence features into the pooling module to obtain global features and local features output by the pooling module, and inputting the global features and the local features into the classification module to obtain the recognition result of the face image output by the classification module.

[0040] The feature extraction module includes a first number of multi-head attention modules and a second number of multi-order hybrid attention modules, wherein the multi-head attention module can perform multiple parallel attention operations on the input information, and splice the attention information obtained by each multi-head to obtain the final multi-head attention information. The information of each multi-head input is the same, and an independent attention operation is performed on the input information. The multi-head attention information is obtained by splicing, which can achieve the effect of preventing overfitting.

[0041] The multiple multi-head attention modules in this embodiment may include the same number of heads or different numbers of heads, and there is no limitation on this.

[0042] The multi-order mixed attention module includes multi-order attention units. The feature information output by the previous-order attention unit is used as the input information of the next-order attention unit. The feature information of each order output by the multi-order attention unit is fused to obtain the target feature information output by the multi-order mixed attention module. By setting up a multi-order mixed attention module, the high-order feature information in the face image can be obtained in advance, and the feature information of each order can be fused at the same time to achieve the fusion of feature information of different orders, which can achieve the effect of preventing overfitting.

[0043] The multiple multi-order mixed attention modules in this embodiment may include the same number of orders or different numbers of orders, which is not limited to this. In some embodiments, the order of each multi-order mixed attention module may be 2 or 3.

[0044] For example, see Figure 3 , Figure 3 It is a structural schematic diagram of a multi-order hybrid attention module provided by an embodiment of the present invention. Figure 3 The input information X is the feature information output by the previous module, where the previous model can be a multi-head attention module or a multi-order mixed attention module. The input information is converted into a feature vector that meets the requirements of the current attention unit through the norm function. The feature vector is the input information of the first-order attention unit. The first-order attention unit generates the first-order attention parameters based on the input information, namely Q1, K1 and V1. Specifically, it can be obtained by dividing the input information X with the weight W. Q1 , W K1 and W V1 Calculated, Q1 = XW Q1 、K1=XW K1 and V1 = XW V1 The first-order attention information A1 is obtained through the first-order attention parameters Q1, K1 and V1. 1 =solfmax(S 1 )·V 1 , where the similarity coefficient is the normalization parameter, It can be 8 or other values ​​to ensure the stability of the gradient during training.

[0045] The output information of the first-order attention unit is used as the input information of the second-order attention unit. The first-order attention information A2 is obtained based on the output information of the first-order attention unit. The output information of the second-order attention unit is used as the input information of the third-order attention unit, and so on, to obtain high-order attention information and high-order features of the face image.

[0046] In the multi-order hybrid attention module, as the depth increases, the high-order attention unit has the problem of large amount of calculation, especially in the stage of attention acquisition. To address the above problem, the multi-order hybrid attention module in this embodiment sets a parameter sharing mechanism, reuses the first-order attention parameters Q1, K1 and V1 in the high-order attention unit, and determines the high-order attention parameters based on the reused first-order attention parameters, without the need for a large number of calculations based on the output features of the previous attention unit, thereby improving the reuse rate of the first-order attention parameters and reducing the amount of calculation.

[0047] Correspondingly, the attention parameter of any of the attention units is determined based on the first-order attention parameter and the prior information of the current-order attention unit. The attention parameter here is the similarity coefficient S i , Figure 3The similarity coefficient S of each order attention point unit in i Based on Qi and Ki, in this embodiment, the similarity coefficient S of each order attention point unit is i Based on the first-order attention parameter S 1 The prior information W of the current attention unit is calculated. Specifically, based on the first-order attention parameter S 1 Multiply it with the prior information W. For example, see Figure 4 , Figure 4 is a schematic diagram of the structure of a multi-order hybrid attention module provided by an embodiment of the present invention. It should be noted that the attention parameter V of each order attention unit i Determined based on the input information of the attention unit, and based on the obtained similarity coefficient S i and the attention parameter V i Determine the current level of attention information A i By simplifying the similarity coefficient S i The calculation method reduces the amount of calculation in the multi-order mixed attention module and reduces the consumption of computing resources.

[0048] The prior information W of each order of attention units is obtained during the training process of the face recognition model. The prior information of different order attention units may be different. Correspondingly, the prior information of each order of attention units in different multi-order mixed attention modules may be different.

[0049] Based on the above embodiment, a convolution unit is arranged between any two adjacent attention units in the multi-order mixed attention module. The convolution unit is used to perform convolution processing on the output information of the previous-order attention unit to reduce the sequence length of the output information, thereby achieving the effect of reducing the computational complexity of the next-order attention unit.

[0050] In some embodiments, the convolution unit includes but is not limited to at least one of a deformable convolution and a depth-separable convolution, and performs convolution processing on the feature information output by the previous attention unit. Among them, the deformable convolution (DFConv) includes an offset parameter, which is trained in the end-to-end training process of the face recognition model. The size and position of the convolution kernel of the deformable convolution can be dynamically adjusted according to the image content that needs to be recognized at present. The intuitive effect is that the position of the convolution kernel sampling points at different positions will change adaptively according to the image content, so as to adapt to the geometric deformations such as the shape and size of different objects. In this embodiment, the sampling point position can be adjusted according to the size and position of the face area in the face image to collect the image content in the face area, avoid the interference of background information outside the face area on face recognition, improve the extraction accuracy of feature information in the face area, and further improve the face recognition accuracy.

[0051] The depthwise convolution (DWConv) includes at least two layers of convolution operations. For each channel data in the input information, convolution processing is performed based on the first convolution kernel to obtain the feature information of each channel, and then the feature information is traversed through n (for example, the number of channels) 1×1 convolution kernels to perform feature fusion. Optionally, the feature information obtained by the above fusion can be further processed based on n 1×1 convolution kernels to obtain deep information. By setting up a deep separable convolution, the spatial information in the input information, that is, the spatial information in the face image, can be extracted, which improves the spatial reference basis for face recognition and reduces the model parameters, thereby improving the efficiency of face recognition.

[0052] See also Figure 3 Any convolution unit includes a deformable convolution and a depthwise separable convolution, which respectively receive the output information of the previous-order attention unit, and merge the processing results of the deformable convolution and the depthwise separable convolution and input them to the next attention unit. By performing convolution processing of different dimensions on the output information of the previous-order attention unit, the accuracy and diversity of the features are improved, so that the next-order attention unit can perform high-precision feature extraction based on the output information of the convolution unit.

[0053] On the basis of the above embodiments, the multi-head attention module and the multi-order mixed attention module in the feature extraction module are connected based on a preset connection order. The number of multi-head attention modules and multi-order mixed attention modules can be set according to user needs. In some embodiments, the data range of the first number of multi-head attention modules can be 6-10, and the data range of the second number of multi-order mixed attention modules can be 2-6, which is not limited. In some embodiments, the first number of multi-head attention modules can be 8, and the second number of multi-order mixed attention modules can be 4.

[0054] In some embodiments, the multi-order hybrid attention module is located at the center of the feature extraction module, and the multi-head attention modules are distributed at both ends of the feature extraction module.

[0055] In some embodiments, the multi-head attention module and the multi-order mixed attention module in the feature extraction module are alternately connected, and at least one multi-head attention module is respectively provided at both ends of the feature extraction module.

[0056] On the basis of the above embodiment, the feature extraction module further includes an embedding layer for extracting initial visual sequence features from the input face image, and outputting the initial visual sequence features to the multi-head attention module connected to the embedding layer. The embedding layer can be implemented by an embedder, and the embedding layer embeds the layer, so that the multi-head attention module can quickly extract features from the initial visual sequence features and accelerate the feature extraction efficiency.

[0057] Based on the above embodiment, the pooling module includes a global pooling unit and a local pooling unit. The global pooling unit and the local pooling unit are respectively connected to the output end of the feature extraction module, and process the visual sequence features output by the feature extraction module respectively.

[0058] The global pooling unit is used to perform global pooling processing on the visual sequence features output by the feature extraction module to obtain global features. In some embodiments, the global pooling unit can be implemented by a global pooling layer, and the global pooling layer can perform average pooling or maximum pooling on the visual sequence features.

[0059] The local pooling unit is used to perform local pooling processing on the visual sequence features output by the feature extraction module to obtain multiple local features. In some embodiments, the local pooling unit can be implemented by a local pooling layer, and the local pooling layer can be average pooling or maximum pooling. For example, the visual sequence features output by the feature extraction module are divided into multiple sub-features, and average pooling or maximum pooling is performed on each of them to obtain multiple local features, wherein the number of local features can be 6.

[0060] On the basis of the above embodiment, the classification module includes a plurality of fully connected layers and a result processing unit, each fully connected layer is used to determine the recognition probability based on the global feature or any local feature, and the result processing unit determines the recognition result based on the recognition probability output by each fully connected layer. Specifically, the result processing unit may be a face recognition result obtained by performing mean processing on the recognition probabilities output by the plurality of fully connected layers; the result processing unit may also be a face recognition result obtained by performing weighted processing on the recognition probabilities output by the plurality of fully connected layers. The weight of each recognition probability may be preset, and optionally, the weight of the recognition probability corresponding to the global feature may be higher than the weight of the recognition probability corresponding to the local feature, so as to increase the importance of the global feature and avoid recognition errors caused by the similarity of local features in different facial features.

[0061] In this embodiment, by extracting global features and local features and determining the recognition probability respectively by the global features and the local features, the comprehensiveness of the features is improved, the recognition error rate caused by recognizing faces based only on local features is reduced, and the accuracy of face recognition is improved.

[0062] The technical solution of this embodiment performs high-precision face recognition on face images by setting a face recognition model, wherein the feature extraction module in the face model includes a multi-head attention module and a multi-order mixed attention module, and alternately extracts features from the input face image through the multi-head attention module and the multi-order mixed attention module to obtain high-order connection features in the face image, thereby realizing the representation of complex features in the face image. Furthermore, the global features and local features of the face image are obtained through the pooling model, which improves the comprehensiveness of the features, and the recognition probability is determined by the global features and the local features respectively, thereby improving the accuracy of face recognition.

[0063] Based on the above embodiments, the present invention also provides a preferred embodiment. Figure 5 , Figure 5 It is a structural schematic diagram of a face recognition model provided by an embodiment of the present invention. The face recognition model includes an embedder, a feature extraction module in which a multi-head attention module and a multi-order mixed attention module are alternately connected, a pooling model and a classification module, wherein the feature extraction module in which the multi-head attention module and the multi-order mixed attention module are alternately connected includes 8 multi-head attention modules and 4 multi-order mixed attention modules, wherein the embedder is connected to a multi-head attention module for extracting initial visual sequence features from an input face image. The feature extraction module includes a multi-head attention module, a multi-head attention module, a multi-order mixed attention module, a multi-head attention module, a multi-order mixed attention module, a multi-head attention module, a multi-order mixed attention module, a multi-head attention module, a multi-order mixed attention module, a multi-head attention module, a multi-head attention module, and a multi-head attention module connected in sequence. The attention parameters in each multi-order mixed attention module are shared, and each high-order attention unit reuses the first-order attention parameters output by the first-order attention unit to simplify the operation process and reduce the amount of calculation. The last multi-head attention module outputs visual sequence features with global connections. The pooling model includes global pooling units and local pooling units, which output global features and local features respectively. The classification module outputs face recognition results based on the initial visual sequence features extracted from the input face image.

[0064] Figure 6 : is a structural schematic diagram of a face recognition device provided by an embodiment of the present invention, the device comprising:

[0065] A face image acquisition module 210 is used to acquire a face image to be recognized;

[0066] A face recognition module 220, used to input the face image into a face recognition model to obtain a recognition result output by the face recognition model;

[0067] Among them, the face recognition model includes a feature extraction module, a pooling module and a classification module. The feature extraction module includes a first number of multi-head attention modules and a second number of multi-order mixed attention modules, which are used to extract visual sequence features with global connections. The pooling module is used to obtain global features and local features based on the visual sequence features. The classification module is used to obtain recognition results of face images based on the global features and the local features.

[0068] Optionally, the multi-head attention module and the multi-order mixed attention module are connected based on a preset connection order, and the multi-order mixed attention module includes multiple attention units, and a convolution unit is arranged between any two adjacent attention units.

[0069] Optionally, the convolution unit includes a deformable convolution and a depthwise separable convolution, which respectively perform convolution processing on the feature information output by the previous attention unit, and the processing results of the deformable convolution and the depthwise separable convolution are merged and input into the next attention unit.

[0070] Optionally, the attention parameters of any of the attention units are determined based on the first-order attention parameters and prior information of the current-order attention unit.

[0071] Optionally, the feature extraction module also includes an embedding layer, which is used to extract initial visual sequence features from the input face image and output the initial visual sequence features to a multi-head attention module connected to the embedding layer.

[0072] Optionally, the pooling module includes a global pooling unit and a local pooling unit, the global pooling unit is used to perform global pooling processing on the visual sequence features output by the feature extraction module to obtain global features; the local pooling unit is used to perform local pooling processing on the visual sequence features output by the feature extraction module to obtain multiple local features.

[0073] Optionally, the classification module includes multiple fully connected layers and a result processing unit, each fully connected layer is used to determine the recognition probability based on the global feature or any local feature, and the result processing unit determines the recognition result based on the recognition probability output by each of the fully connected layers.

[0074] The face recognition device provided in the embodiment of the present invention can execute the face recognition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0075] Figure 7 A schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. Figure 7 A block diagram of an electronic device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 7The electronic device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention. The device 12 is typically an electronic device that undertakes the image classification function.

[0076] like Figure 7 As shown, the electronic device 12 is in the form of a general purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors 16, a storage device 28, and a bus 18 connecting various system components (including the storage device 28 and the processor 16).

[0077] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. For example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0078] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0079] The storage device 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 7 not shown, usually called a "hard drive"). Although Figure 7Not shown in the figure, a disk drive for reading and writing a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing a removable non-volatile optical disk (e.g., a read-only optical disk (Compact Disc-Read Only Memory, CD-ROM), a digital video disk (Digital Video Disc-Read Only Memory, DVD-ROM) or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The storage device 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of various embodiments of the present invention.

[0080] A program 36 having a set (at least one) of program modules 26 may be stored, for example, in a storage device 28, such program modules 26 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination thereof may include an implementation of a gateway environment. The program modules 26 generally perform the functions and / or methods of the embodiments described herein.

[0081] The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, cameras, displays 24, etc.), may communicate with one or more devices that enable a user to interact with the electronic device 12, and / or may communicate with any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. In addition, the electronic device 12 may also communicate with one or more gateways (e.g., a local area network (LAN), a wide area network (WAN), and / or a public gateway, such as the Internet) via a gateway adapter 20. As shown, the gateway adapter 20 communicates with other modules of the electronic device 12 via a bus 18. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, disk arrays (Redundant Arrays of Independent Disks, RAID) systems, tape drives, and data backup storage systems.

[0082] The processor 16 executes various functional applications and data processing by running the programs stored in the storage device 28, such as implementing the face recognition method provided in the above embodiment of the present invention.

[0083] Embodiment 5 of the present invention provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the face recognition method provided in the embodiment of the present invention is implemented.

[0084] Of course, the computer-readable storage medium provided by the embodiment of the present invention stores a computer program which is not limited to the method operation described above, but can also execute the face recognition method provided by any embodiment of the present invention.

[0085] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0086] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable source code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0087] Source code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wire, electrical wire, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0088] Computer source code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The source code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of gateway, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0089] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A face recognition method, characterized in that: include: Obtain a face image to be recognized; Inputting the face image into a face recognition model to obtain a recognition result output by the face recognition model; The face recognition model includes a feature extraction module, a pooling module and a classification module. The feature extraction module includes a first number of multi-head attention modules and a second number of multi-order mixed attention modules, which are used to extract visual sequence features with global connections. The pooling module is used to obtain global features and local features based on the visual sequence features. The classification module is used to obtain a recognition result of a face image based on the global features and the local features. Among them, the multi-order hybrid attention module is located at the center of the feature extraction module, and the multi-head attention modules are distributed at both ends of the feature extraction module; or, the multi-head attention module and the multi-order hybrid attention module in the feature extraction module are alternately connected, and at least one multi-head attention module is respectively arranged at both ends of the feature extraction module.

2. The method according to claim 1, characterized in that The multi-head attention module and the multi-order mixed attention module are connected based on a preset connection order. The multi-order mixed attention module includes multiple attention units, and a convolution unit is arranged between any two adjacent attention units.

3. The method according to claim 2, characterized in that The convolution unit includes a deformable convolution and a depthwise separable convolution, which respectively perform convolution processing on the feature information output by the previous attention unit. The processing results of the deformable convolution and the depthwise separable convolution are merged and input into the next attention unit.

4. The method according to claim 2, characterized in that: The attention parameters of any of the attention units are determined based on the first-order attention parameters and the prior information of the current-order attention unit.

5. The method according to claim 1, characterized in that The feature extraction module also includes an embedding layer for extracting initial visual sequence features from the input face image and outputting the initial visual sequence features to a multi-head attention module connected to the embedding layer.

6. The method according to claim 1, characterized in that The pooling module includes a global pooling unit and a local pooling unit. The global pooling unit is used to perform global pooling processing on the visual sequence features output by the feature extraction module to obtain global features; the local pooling unit is used to perform local pooling processing on the visual sequence features output by the feature extraction module to obtain multiple local features.

7. The method according to claim 1, characterized in that The classification module includes multiple fully connected layers and a result processing unit, each fully connected layer is used to determine the recognition probability based on the global feature or any local feature, and the result processing unit determines the recognition result based on the recognition probability output by each of the fully connected layers.

8. A face recognition device, characterized in that: include: A face image acquisition module, used to acquire a face image to be recognized; A face recognition module, used to input the face image into a face recognition model to obtain a recognition result output by the face recognition model; The face recognition model includes a feature extraction module, a pooling module and a classification module. The feature extraction module includes a first number of multi-head attention modules and a second number of multi-order mixed attention modules, which are used to extract visual sequence features with global connections. The pooling module is used to obtain global features and local features based on the visual sequence features. The classification module is used to obtain a recognition result of a face image based on the global features and the local features. Among them, the multi-order hybrid attention module is located at the center of the feature extraction module, and the multi-head attention modules are distributed at both ends of the feature extraction module; or, the multi-head attention module and the multi-order hybrid attention module in the feature extraction module are alternately connected, and at least one multi-head attention module is respectively arranged at both ends of the feature extraction module.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the face recognition method as described in any one of claims 1-7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the face recognition method as described in any one of claims 1 to 7 is implemented.

Images