Face recognition model training method, face recognition method and related equipment

Abstract

The invention discloses a face recognition model training method, a face recognition method and related equipment, and the method comprises the steps: collecting an initial data set, and carrying out the data cleaning and alignment of the initial data set, and obtaining an aligned data set; inputting the aligned data set into a multi-modal model CLIP to extract image features and text features; fusing the image features and the text features through a cross-modal interaction module to generate multi-modal fusion features; and inputting the multi-modal fusion features into an AdaFace classification head, and performing training based on a combined loss function to obtain a face recognition model. According to the invention, the feature learning ability and classification precision of the face recognition model are improved, and the face recognition precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of biometric identification technology, and in particular to a face recognition model training method, a face recognition method and related equipment. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, facial recognition is increasingly being used in security surveillance, financial payments, healthcare, smart devices, and other fields. However, real-world scenarios often present issues such as uneven lighting, varying postures, and partial occlusion (e.g., caused by wearing masks or glasses), which pose significant challenges to the accuracy and robustness of traditional facial recognition systems. This is especially true in critical scenarios such as public safety and identity verification, where recognition errors can pose significant risks. Technological breakthroughs are urgently needed to improve recognition reliability in complex environments.

[0003] Currently, mainstream face recognition methods are primarily based on single-modal image data, employing convolutional neural networks (CNNs) or visual transformer (ViT) architectures for feature extraction, combined with improved loss functions (such as ArcFace and AdaFace) to optimize classification boundaries. Existing technologies present at least the following technical issues: (1) Reliance on a single image modality: Only facial image information is used, and there is a lack of integration of multimodal data such as text and voice. It is difficult to assist in inferring identity through semantic descriptions (such as age, gender, and facial attributes), resulting in a significant performance degradation in scenes with blurred, occluded, or low-resolution images. (2) Modal information isolation: Existing technologies do not effectively use auxiliary modal information such as text to supplement image features, which makes the model sensitive to image quality and prone to misjudgment in occluded or low-light scenes; (3) Inefficient cross-modal interaction: Traditional multimodal fusion methods (such as feature splicing) lack a dynamic weight adjustment mechanism, making it difficult to capture the semantic association between images and text, and unable to achieve deep feature complementarity; (4) Insufficient training efficiency and generalization: Training the model from scratch requires a large amount of labeled data, and a single task loss function is difficult to take into account both multimodal alignment and classification optimization, which limits the adaptability of the model in complex environments; (5) Limited scalability: The existing framework does not reserve a multimodal expansion interface, making it difficult to be compatible with new modalities (such as voice and behavioral data) in the future, and the cost of technology iteration is high.

[0004] Therefore, there is an urgent need for a method that can interact across modalities and perform face recognition with high precision. Summary of the Invention

[0005] The present invention provides a face recognition model training method, a face recognition method and related equipment to improve the accuracy of face recognition.

[0006] In order to solve the above technical problems, the present invention provides a face recognition model training method, including: Collect an initial data set, and perform data cleaning and alignment on the initial data set to obtain an aligned data set, wherein the initial data set includes a facial image data set and a corresponding attribute description text data set, and the facial image of each person ID in the aligned data set corresponds one-to-one with the attribute description text; Inputting the aligned dataset into a multimodal model CLIP to extract image features and text features, wherein the multimodal model CLIP includes an image encoder and a text encoder, the image encoder is used to extract image features from the facial image data in the aligned dataset, and the text encoder is used to extract text features from the attribute description text in the aligned dataset; Fusing the image features with the text features through a cross-modal interaction module to generate multimodal fusion features; The multimodal fusion features are input into an AdaFace classification head and trained based on a combined loss function to obtain a face recognition model, wherein the combined loss function includes a multimodal alignment loss and an AdaFace classification loss.

[0007] Optionally, performing data cleaning and alignment on the initial data set to obtain an aligned data set includes: The face area of the image is cropped using a face detection model, and the image size is uniformly adjusted to a preset resolution; the attribute description text includes at least one of age, gender, facial attributes and face shape.

[0008] Optionally, the image encoder is a visual model pre-trained by the multimodal model CLIP, which outputs image features of fixed dimensions; the text encoder is a text model pre-trained by the multimodal model CLIP, which outputs text features with the same dimension as the image features.

[0009] Optionally, the cross-modal interaction module interacts with image features and text features through a self-attention mechanism, and extracts the fused global features as multimodal fusion features.

[0010] Optionally, the multimodal alignment loss is optimized by calculating a comparative relationship between similarities of matching image-text pairs and similarities of non-matching pairs, wherein the similarity of matching pairs is higher than that of non-matching pairs.

[0011] In order to solve the above technical problems, the embodiment of the present application further provides a face recognition method, including: Obtain the image to be recognized; The image to be recognized is input into a face recognition model to obtain a face recognition result.

[0012] In order to solve the above technical problems, the embodiment of the present application further provides a face recognition model training device, comprising: An acquisition module is used to acquire an initial data set, and perform data cleaning and alignment on the initial data set to obtain an aligned data set, wherein the initial data set includes a facial image data set and a corresponding attribute description text data set, and the facial image of each person ID in the aligned data set corresponds one-to-one with the attribute description text; An extraction module, configured to input the aligned dataset into a multimodal model CLIP to extract image features and text features, wherein the multimodal model CLIP includes an image encoder and a text encoder, wherein the image encoder is configured to extract image features from the facial image data in the aligned dataset, and the text encoder is configured to extract text features from the attribute description text in the aligned dataset; A fusion module, configured to fuse the image features with the text features through a cross-modal interaction module to generate multimodal fusion features; A training module is used to input the multimodal fusion features into an AdaFace classification head and perform training based on a combined loss function to obtain a face recognition model, wherein the combined loss function includes a multimodal alignment loss and an AdaFace classification loss.

[0013] In order to solve the above technical problems, the present application also provides a face recognition device, including: An acquisition module, used to acquire an image to be identified; The recognition module is used to input the image to be recognized into the face recognition model to obtain a face recognition result.

[0014] In order to solve the above technical problems, an embodiment of the present application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the computer program, the steps of the above-mentioned face recognition model training method are implemented, or when the processor executes the computer program, the steps of the above-mentioned face recognition method are implemented.

[0015] In order to solve the above technical problems, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the steps of the above-mentioned face recognition model training method, or when the computer program is executed by a processor, it implements the steps of the above-mentioned face recognition method.

[0016] The face recognition model training method, face recognition method, apparatus, computer device, and storage medium provided in embodiments of the present invention collect an initial data set, cleanse, and align it to obtain an aligned data set; input the aligned data set into a multimodal model CLIP to extract image and text features; fuse the image and text features through a cross-modal interaction module to generate multimodal fusion features; and input the multimodal fusion features into an AdaFace classification head and train based on a combined loss function to obtain a face recognition model. The implementation of multimodal data fusion, a cross-modal interaction mechanism, and a combined loss function design significantly improves the feature learning capability, classification accuracy, and scenario adaptability of the face recognition model, providing a technical foundation for high-precision identity authentication in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 is an exemplary system architecture diagram to which the present application may be applied; Figure 2 This is a flowchart of an embodiment of the face recognition model training method of the present application; Figure 3 is a flowchart of an embodiment of the face recognition method of the present application; Figure 4 This is a structural diagram of an embodiment of a face recognition model training device according to the present application; Figure 5 is a structural diagram of an embodiment of a face recognition device according to the present application; Figure 6 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meanings as commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of the application are for the purpose of describing specific embodiments only and are not intended to limit this application. The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of this application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0020] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0021] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0022] See also Figure 1 ,like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is a medium for providing communication links between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0023] Users can use terminal devices 101 , 102 , 103 to interact with server 105 via network 104 to receive or send messages, etc.

[0024] The terminal devices 101, 102, and 103 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptop computers, desktop computers, etc.

[0025] The server 105 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal devices 101 , 102 , and 103 .

[0026] It should be noted that the face recognition model training method / face recognition method provided in the embodiment of the present application is executed by a server, and accordingly, the face recognition model training device / face recognition device is set in the server.

[0027] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only illustrative. According to the implementation requirements, there can be any number of terminal devices, networks and servers. The terminal devices 101, 102, 103 in the embodiment of the present application can specifically correspond to application systems in actual production.

[0028] See also Figure 2 , Figure 2 A face recognition model training method provided by an embodiment of the present invention is shown. Figure 1 The server in the example is used as an example, and the details are as follows: S201: Collect an initial data set, clean and align the initial data set to obtain an aligned data set, wherein the initial data set includes a facial image data set and a corresponding attribute description text data set, and the facial image of each person ID in the aligned data set corresponds one-to-one to the attribute description text.

[0029] Specifically, in a specific example, a face dataset and a face attribute description text dataset totaling 10 million people are collected, and then data cleaning is performed, such as using retinaface to crop the face image and crop the face dataset to the adaface input dimension size dataset. Format. Use image_tools in mxnet to convert the training dataset to mxrecord, which is then converted to binary image data format for easy reading and other operations to obtain face recognition image data. Through ID matching, each image folder is used as a person ID. A text file (.txt) describing the person ID attributes is added for each person ID. For example, ID: 1, Age: 23, Gender: Male, Wearing Glasses: No, Wearing Mask: No, Face Shape: Square. Each person has a corresponding attribute description file, enabling pairing of person attributes and person images, increasing the credibility of multimodal information and providing basic data support for subsequent face recognition algorithm training using a multimodal training architecture.

[0030] Optionally, the initial data set is cleaned and aligned to obtain an aligned data set including: The face area of the image is cropped using a face detection model, and the image size is uniformly adjusted to a preset resolution; the attribute description text includes at least one of age, gender, facial attributes and face shape.

[0031] Furthermore, the face detection model is used to crop the face area of the image and the image size is uniformly adjusted to a preset resolution, including: Use the RetinaFace model to detect and crop faces in the image, ensuring that only the valid face area is retained; resize the cropped face images to a preset resolution (e.g., 112×112 pixels). Convert images to binary format (such as MXNet's MXRecord format) to improve data reading efficiency; A corresponding attribute description text file is created for each person ID. The attributes include at least one of age, gender, whether wearing glasses, whether wearing a mask, and face shape, and ID matching is used to ensure strict alignment between the image and the text.

[0032] In this embodiment, by combining multimodal data from facial images and attribute description text, the model can use textual information to supplement semantic features (such as age, gender, and facial attributes) that are difficult to capture in images, thereby more comprehensively learning facial features. For example, in cases where the image is blurred or partially occluded, the auxiliary information from the text description can help the model more accurately infer identity and reduce the risk of misjudgment.

[0033] S202: Input the aligned dataset into the multimodal model CLIP to extract image features and text features, wherein the multimodal model CLIP includes an image encoder and a text encoder. The image encoder is used to extract image features from the facial image data in the aligned dataset, and the text encoder is used to extract text features from the attribute description text in the aligned dataset.

[0034] Optionally, the image encoder is a visual model pre-trained by the multimodal model CLIP, which outputs image features of fixed dimensions; the text encoder is a text model pre-trained by the multimodal model CLIP, which outputs text features with the same dimension as the image features.

[0035] Among them, the image encoder uses the Vision Transformer (ViT-B / 32) pre-trained by CLIP to output 512-dimensional image features; the text encoder uses the Transformer model pre-trained by CLIP to output 512-dimensional text features with the same dimension as the image features.

[0036] In this example, a CLIP-pretrained encoder is used to map image and text features into the same semantic space, ensuring consistency in the joint feature space. A cross-modal interaction module (such as the self-attention mechanism) dynamically adjusts the weights of image and text features, achieving efficient information complementation. For example, the attribute "wearing glasses" in a text description can be deeply integrated with eye features in an image, enhancing the model's perception of detail.

[0037] At the same time, using the CLIP pre-trained model to initialize the encoder can significantly reduce training time and data requirements, avoiding the resource consumption of training from scratch; the multimodal data framework supports flexible expansion of other modal information (such as voice and infrared images), providing a compatibility basis for future technology iterations.

[0038] S203: Fusing the image features and text features through a cross-modal interaction module to generate multimodal fusion features.

[0039] Specifically, the specific implementation of the cross-modal interaction module includes: Concatenate image features and text features into an input of shape batch_size,2,512; Cross-modal interaction is carried out through the self-attention mechanism of Transformer Encoder, dynamically adjusting the weights of the two modal features; The fused global features are extracted as multimodal fusion features, specifically by taking CLS tags or performing average pooling on the feature sequence.

[0040] Optionally, the cross-modal interaction module interacts with image features and text features through a self-attention mechanism, and extracts the fused global features as multimodal fusion features.

[0041] In this embodiment, the self-attention mechanism can capture the semantic association between images and text. For example, the attribute "wearing glasses" described in the text can be deeply integrated with the eye features in the image, enhancing the model's perception of details. At the same time, the global feature extraction strategy (such as CLS tagging) retains key information of cross-modal interaction and improves the robustness of feature expression.

[0042] S204: Input the multimodal fusion features into the AdaFace classification head and train based on the combined loss function to obtain a face recognition model, wherein the combined loss function includes the multimodal alignment loss and the AdaFace classification loss.

[0043] Optionally, the multimodal alignment loss is optimized by calculating the contrast between the similarity of matched image-text pairs and the similarity of non-matching pairs, where the similarity of matched pairs is higher than that of non-matching pairs.

[0044] The combined loss function includes multimodal alignment loss and AdaFace classification loss, where: The multimodal alignment loss is used to optimize the matching between image and text features through contrastive learning; The AdaFace classification loss is used to adjust the distance between the feature and the category center through adaptive boundaries to optimize the classification effect; The combined loss function is the weighted sum of the two.

[0045] Specifically, the design of the combined loss function balances the requirements of multimodal alignment and classification tasks: Multimodal alignment loss: Based on contrastive learning, it optimizes the similarity of matching image-text pairs, forcing positive pairs (matching image-text pairs) to be close in feature space and negative pairs (non-matching pairs) to be far apart. Furthermore, a temperature parameter is used to control the smoothness of the similarity calculation, narrowing the semantic gap between modalities and enhancing cross-modal retrieval capabilities. AdaFace classification loss: Adaptively adjusts the boundary between features and class centers, dynamically optimizes the classification decision surface for samples of different quality, and enhances the discrimination of difficult samples. In this embodiment, an adaptive boundary adjustment factor g(z) is introduced based on ArcFace to dynamically adjust the classification decision boundary according to feature quality; Automatically enlarge the boundaries of low-quality samples (such as blurred images) to enhance the discrimination of difficult samples.

[0046] The combination of the two enables the model to have stronger generalization capabilities while ensuring the accuracy of identity classification, and is suitable for complex scenes (such as lighting changes and posture differences).

[0047] Furthermore, the multimodal alignment loss is calculated using the following formula: ; Where N is the batch size, i.e., the number of image-text pairs in a batch; img_i is the feature vector of the i-th image; text_i is the feature vector of the text that matches the i-th image (positive examples); text_j is the feature vector of all texts in the batch (both positive and negative examples); sim(...) is the similarity function, using cosine similarity (sim(a, b) = a·b / (||a||·||b||)). T is the temperature parameter, a scalar (e.g., 0.07) that controls the smoothness of the softmax function; and ∑ is the sum of all examples in the batch.

[0048] In this formula, the numerator exp(sim(img_i, text_i) / T) represents the similarity between the i-th image and its matching text (positive sample), after exponential and temperature scaling. The denominator: / T) represents the sum of similarities between the i-th image and all texts in the batch (including positive and negative samples). log: Calculates the logarithm of the numerator divided by the denominator, measuring the relative probability of positive sample pairs. It means taking the average of negative logarithms as the loss, indicating that we want to maximize the similarity of positive sample pairs.

[0049] In this embodiment, the Adaface classification loss is an improved version of ArcFace, which adds an adaptive boundary and adopts the following formula: ; Among them, N is the batch size, that is, the number of samples in a batch, : The angle between the i-th sample feature and the center of the target category (correct identity). : The angle between the jth sample feature and the center of the non-target class (other identity). s is the scale factor. m is the margin. g(z) is the adaptive adjustment factor related to feature quality. ∑ is the sum of all samples in the batch.

[0050] In this formula, the numerator: represents the score of the ith sample and its correct identity category, after scaling and boundary adjustment. represents the sum of all scores for all categories, including correct and incorrect categories. log is the logarithm of the relative probability of calculating the correct category score. Taking the average of the negative logarithms as the loss means that we want to maximize the score of the correct category.

[0051] The loss function used is a weighted combination of adafaceloss and cliploss ; Where, λ1 and λ2 are hyperparameters that balance multimodal alignment and classification.

[0052] Furthermore, this embodiment utilizes a multi-task design combining identity classification and attribute prediction, allowing the model to share multimodal features during training, promoting the decoupling and reuse of feature representations. For example, attribute prediction tasks (such as gender classification) can constrain the model to learn general features unrelated to identity, indirectly improving the discriminability of identity recognition.

[0053] In a specific example, the multi-task header output includes: 1. Identity classification head: Based on AdaFace, outputs identity logits.

[0054] 2. Attribute prediction head: outputs attribute categories (such as gender, age group).

[0055] After the identity classification head: fully connected layer + AdaFace Loss, the face recognition facial feature output is obtained; Attribute prediction head: fully connected layer + cross entropy loss, predicts attribute categories and obtains multiple facial attribute features.

[0056] Combining the results of the two features, we can get a multi-feature face recognition result, which improves the accuracy of each face.

[0057] In this example, an initial dataset is collected, cleaned, and aligned to produce an aligned dataset. This dataset is then fed into the multimodal CLIP model to extract image and text features. The image and text features are then fused using a cross-modal interaction module to generate multimodal fusion features. This fusion feature is then fed into the AdaFace classification head and trained using a combined loss function to produce a face recognition model. This multimodal data fusion, cross-modal interaction mechanism, and combined loss function design significantly improve the face recognition model's feature learning capabilities, classification accuracy, and scenario adaptability, providing a technical foundation for high-precision identity authentication in complex environments.

[0058] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0059] See also Figure 3 , Figure 3 A face recognition method provided by an embodiment of the present invention is shown. Figure 1 The server in the example is used as an example, and the details are as follows: Obtain the image to be recognized; Input the image to be recognized into the face recognition model to obtain the face recognition result.

[0060] In this embodiment, the trained face recognition model is used to recognize the image to be recognized, thereby improving the sales of face recognition.

[0061] Figure 4 FIG. 1 is a block diagram showing a face recognition model training device corresponding to the face recognition model training method in the above embodiment. Figure 4 As shown, the face recognition model training device includes an acquisition module 31, an extraction module 32, a fusion module 33 and a training module 34. The functional modules are described in detail as follows: An acquisition module 31 is configured to acquire an initial data set, clean and align the initial data set, and obtain an aligned data set, wherein the initial data set includes a facial image data set and a corresponding attribute description text data set, and the facial image of each person ID in the aligned data set corresponds one-to-one with the attribute description text; an extraction module 32, configured to input the aligned dataset into a multimodal model CLIP to extract image features and text features, wherein the multimodal model CLIP includes an image encoder and a text encoder, wherein the image encoder is configured to extract image features from the facial image data in the aligned dataset, and the text encoder is configured to extract text features from the attribute description text in the aligned dataset; A fusion module 33 is configured to fuse the image features with the text features through a cross-modal interaction module to generate multimodal fusion features; The training module 34 is used to input the multimodal fusion features into the AdaFace classification head and perform training based on a combined loss function to obtain a face recognition model, wherein the combined loss function includes a multimodal alignment loss and an AdaFace classification loss.

[0062] Figure 5 FIG. 1 shows a principle block diagram of a face recognition device corresponding to the face recognition method in the above embodiment. Figure 5 As shown, the face recognition model training device includes an acquisition module 35 and a recognition module 36. The functional modules are described in detail as follows: An acquisition module 35 is used to acquire an image to be identified; The recognition module 36 is used to input the image to be recognized into the face recognition model to obtain a face recognition result.

[0063] For the specific definition of the face recognition model training device / face recognition device, please refer to the definition of the face recognition model training method / face recognition method above, which will not be repeated here. The various modules in the above-mentioned face recognition model training device / face recognition device can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0064] To solve the above technical problems, the present application also provides a computer device. Figure 6 , Figure 6 This is a basic structural block diagram of the computer device in this embodiment.

[0065] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components connected to the memory 41, the processor 42, and the network interface 43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0066] The computer device may be a desktop computer, notebook computer, PDA, cloud server, etc. The computer device may interact with the user via a keyboard, mouse, remote control, touchpad, or voice control device.

[0067] The memory 41 includes at least one type of readable storage medium, including flash memory, a hard disk, a multimedia card, card-type memory (e.g., SD or D-interface display memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic storage, a magnetic disk, an optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or internal memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash memory card, etc. Of course, the memory 41 may also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as the program code of the face recognition model training method. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0068] In some embodiments, the processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute program code stored in the memory 41 or process data, such as executing program code for a face recognition model training method.

[0069] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0070] The present application also provides another embodiment, namely, providing a computer-readable storage medium, which stores an interface display program, and the interface display program can be executed by at least one processor to enable the at least one processor to perform the steps of the above-mentioned face recognition model training method.

[0071] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of this application, or the part that contributes to the existing technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of this application.

[0072] Obviously, the embodiments described above are only some of the embodiments of the present application, rather than all of the embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions described in the aforementioned specific embodiments, or to make equivalent replacements for some of the technical features therein. Any equivalent structure made using the contents of the present application specification and the accompanying drawings, directly or indirectly used in other related technical fields, is also within the scope of patent protection of the present application.

Claims

1. A face recognition model training method, characterized in that: include: Collect an initial data set, and perform data cleaning and alignment on the initial data set to obtain an aligned data set, wherein the initial data set includes a facial image data set and a corresponding attribute description text data set, and the facial image of each person ID in the aligned data set corresponds one-to-one with the attribute description text; Inputting the aligned dataset into a multimodal model CLIP to extract image features and text features, wherein the multimodal model CLIP includes an image encoder and a text encoder, the image encoder is used to extract image features from the facial image data in the aligned dataset, and the text encoder is used to extract text features from the attribute description text in the aligned dataset; Fusing the image features with the text features through a cross-modal interaction module to generate multimodal fusion features; The multimodal fusion features are input into an AdaFace classification head and trained based on a combined loss function to obtain a face recognition model, wherein the combined loss function includes a multimodal alignment loss and an AdaFace classification loss.

2. The face recognition model training method according to claim 1, wherein: Cleaning and aligning the initial data set to obtain an aligned data set includes: The face area of the image is cropped using a face detection model, and the image size is uniformly adjusted to a preset resolution; the attribute description text includes at least one of age, gender, facial attributes and face shape.

3. The face recognition model training method according to claim 1, wherein: The image encoder is a visual model pre-trained by the multimodal model CLIP, which outputs image features with fixed dimensions; the text encoder is a text model pre-trained by the multimodal model CLIP, which outputs text features with the same dimensions as the image features.

4. The face recognition model training method according to claim 1, wherein: The cross-modal interaction module interacts with image features and text features through a self-attention mechanism, and extracts the fused global features as multimodal fusion features.

5. The face recognition model training method according to claim 1, wherein: The multimodal alignment loss is optimized by calculating the contrast between the similarity of matched image-text pairs and the similarity of non-matching pairs, where the similarity of matched pairs is higher than that of non-matching pairs.

6. A face recognition method, characterized in that: include: Obtain the image to be recognized; The image to be recognized is input into a face recognition model to obtain a face recognition result, wherein the face recognition model is trained according to the face recognition model training method according to any one of claims 1 to 5.

7. A face recognition model training device, characterized in that: include: An acquisition module is used to acquire an initial data set, and perform data cleaning and alignment on the initial data set to obtain an aligned data set, wherein the initial data set includes a facial image data set and a corresponding attribute description text data set, and the facial image of each person ID in the aligned data set corresponds one-to-one with the attribute description text; An extraction module, configured to input the aligned dataset into a multimodal model CLIP to extract image features and text features, wherein the multimodal model CLIP includes an image encoder and a text encoder, wherein the image encoder is configured to extract image features from the facial image data in the aligned dataset, and the text encoder is configured to extract text features from the attribute description text in the aligned dataset; A fusion module, configured to fuse the image features with the text features through a cross-modal interaction module to generate multimodal fusion features; A training module is used to input the multimodal fusion features into an AdaFace classification head and perform training based on a combined loss function to obtain a face recognition model, wherein the combined loss function includes a multimodal alignment loss and an AdaFace classification loss.

8. A face recognition device, characterized in that: include: An acquisition module, used to acquire an image to be identified; A recognition module is used to input the image to be recognized into a face recognition model to obtain a face recognition result, wherein the face recognition model is trained according to the face recognition model training method according to any one of claims 1 to 5.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, it implements the face recognition model training method according to any one of claims 1 to 5, or when the processor executes the computer program, it implements the face recognition method according to claim 6.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, it implements the face recognition model training method according to any one of claims 1 to 5, or when the computer program is executed by the processor, it implements the face recognition method according to claim 6.

Images