Large model face recognition method and electronic device based on synthetic data

Abstract

The present application discloses a large-model face recognition method and electronic device based on synthetic data, belonging to the field of face recognition technology. The method includes: determining the labels of the synthetic data and the categories corresponding to each label; generating corresponding text descriptions based on the labels; generating a face training data set based on the text descriptions; and fine-tuning the first model using the face training data set to obtain a target model. The present application improves the accuracy of face recognition and provides stronger support and broader application prospects for facial expression recognition technology.

Description

Technical Field

[0001] The present application belongs to the field of face recognition technology, and in particular to a large-model face recognition method and electronic equipment based on synthetic data. Background Art

[0002] There are still some problems with the currently widely used facial expression recognition technology. First, it is difficult to obtain facial expression data and there is a lack of data, especially the data of certain expression categories are extremely scarce, resulting in poor performance of the model on these categories. Secondly, facial expression data is easily restricted by various factors such as the environment and equipment during the collection process, making it difficult to obtain high-quality and diverse expression data. In addition, although the existing large models perform well on general visual tasks, due to the lack of specialized optimization for facial expression recognition, their effects on expression recognition tasks are often unsatisfactory. These shortcomings limit the effectiveness and popularity of expression recognition technology in practical applications.

[0003] In order to solve the above problems, a large-model face recognition method and electronic device based on synthetic data are proposed in this application. Summary of the invention

[0004] In order to address the deficiencies of the prior art, the present application provides a large-model face recognition method and electronic device based on synthetic data, which solves the problems in the prior art such as the difficulty in obtaining facial expression data, insufficient data, and inaccurate facial expression recognition.

[0005] The technical effects to be achieved by this application are achieved through the following solutions: In a first aspect, the present application provides a large model face recognition method based on synthetic data, the method comprising: Determine the labels of the synthetic data and the category corresponding to each label; Generate a corresponding text description based on the tag; Generate a face training data set based on the text description; The first model is fine-tuned using the face training data set to obtain a target model.

[0006] In some embodiments, the tags include: age, gender, race, and expression; and Age-corresponding categories include: children, adolescents, young adults, middle-aged, and elderly; Categories corresponding to gender include: male and female; The categories corresponding to race include: Asian, White, Black, Latino, Middle Eastern, and Other; The categories of expressions include: joy, sadness, anger, surprise, fear and disgust.

[0007] In some embodiments, generating a corresponding text description based on the tag includes: Get the target category corresponding to each label; The target category is combined with the environment to generate a corresponding text description.

[0008] In some embodiments, generating a face training data set based on the text description includes: Based on the text description, using multiple generation models, N expression pictures are generated, each expression picture has corresponding label information, wherein the label information includes a target category corresponding to the expression picture, and N is a positive integer; Associating and storing each expression picture with a text description corresponding to the expression picture and corresponding tag information as an image data set; Generate a corresponding expression picture dialogue for each expression picture, associate each expression picture with the expression picture dialogue corresponding to the expression picture and store it as an expression picture dialogue data set, wherein the expression picture dialogue includes an expression picture question and an expression picture answer; The image data set and the expression picture dialogue data set constitute the face training data set.

[0009] In some embodiments, fine-tuning the first model using the face training data set to obtain a target model includes: Inputting the image dataset and the expression picture dialogue dataset in the face training dataset into the first model; A visual input channel and a language input channel are respectively used as feedback inputs of the first model, wherein the image data set is used as a training part of the visual input channel, and the expression picture dialogue data set is used as a training part of the language input channel; The model output of the first model is obtained.

[0010] In some embodiments, after obtaining the model output of the first model, the method further includes: Using a supervised learning method, the model output is compared with the label information through a cross entropy loss function to obtain a comparison result; Optimizing parameters of the first model based on the comparison result; The first model is trained and verified based on the optimized parameters to obtain a target model, which is used in the facial expression recognition task.

[0011] In some embodiments, the multiple generation models include: FLUX model, Stable Diffusion 3 model, MidJourney model and Keling model; for each of the generation models, a corresponding expression picture is generated.

[0012] In some embodiments, before fine-tuning the first model using the face training dataset, the method further includes: Loading the first model; Configure the training hyperparameters of the first model, wherein the training hyperparameters include: optimizer, initial learning rate, batch size, number of epochs, gradient clipping value, and drop probability.

[0013] In a second aspect, the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the aforementioned large-model face recognition methods based on synthetic data when executing the computer program.

[0014] In a third aspect, the present application provides a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement the large model face recognition method based on synthetic data as described in any of the above.

[0015] Through the large-model face recognition method based on synthetic data and electronic equipment provided by the present application, the method generates rich and diverse facial expressions based on text descriptions, and fine-tunes the multimodal large model based on diverse facial expressions, effectively solving the problems of data collection difficulties, insufficient expression data, and poor performance of large models in expression recognition tasks, increasing the diversity of facial expression data, thereby improving the accuracy of face recognition, and providing stronger support and broader application prospects for facial expression recognition technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the existing technical solutions, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 This is a flow chart of a large model face recognition method based on synthetic data in one embodiment of the present application; Figure 2 It is a schematic block diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0019] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in one or more embodiments of the present application should be understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in one or more embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0020] Related technologies mainly focus on facial expression recognition by training deep learning models through large-scale data sets. However, there are problems such as difficulty in data set collection and insufficient data, which lead to inaccurate facial expression recognition.

[0021] This application mainly focuses on using a model to generate synthetic facial expression data, and then conducting the entire process of model training based on the synthetic data.

[0022] Based on the generation of rich facial expression data, the facial expression recognition ability is significantly improved by fine-tuning the multimodal large model.

[0023] Specifically, this application uses mainstream generation models, such as the open source FLUX model, Stable Diffusion 3 model, and the closed source MidJourney model and KeLing model, to generate diverse, high-quality synthetic facial expression data to make up for the lack of real data.

[0024] At the same time, these high-quality synthetic data are used to fine-tune the large multimodal model, so that it can use these data more efficiently when processing facial expression recognition tasks, thereby improving recognition accuracy and generalization ability.

[0025] Through this, it can effectively solve the problems of data collection difficulties, insufficient expression data, and poor performance of large models in expression recognition tasks, providing stronger support and broader application prospects for facial expression recognition technology.

[0026] Various non-limiting implementations of the present application are described in detail below in conjunction with the accompanying drawings.

[0027] First, refer to Figure 1 , the large model face recognition method based on synthetic data of this application is described in detail.

[0028] The present application provides a large model face recognition method based on synthetic data, the method comprising: S1: Determine the labels of the synthetic data and the category corresponding to each label; S2: Generate a corresponding text description based on the label; S3: Generate a face training data set based on the text description; S4: Fine-tune the first model using the face training data set to obtain a target model.

[0029] The large-model face recognition method based on synthetic data provided by the present application generates rich and diverse facial expressions based on text descriptions, and fine-tunes the multimodal large model based on diverse facial expressions, thereby effectively solving the problems of data collection difficulties, insufficient expression data, and poor performance of large models in expression recognition tasks, increasing the diversity of facial expression data, thereby improving the accuracy of face recognition, and providing stronger support and broader application prospects for facial expression recognition technology.

[0030] Through the following steps, the synthetic expression data generation method and fine-tuning process proposed in this application can significantly improve the effect of the facial expression recognition model and provide more powerful support for practical applications.

[0031] In some embodiments, the tags include: age, gender, race, and expression; and Age-corresponding categories include: children, adolescents, young adults, middle-aged, and elderly; Categories corresponding to gender include: male and female; The categories corresponding to race include: Asian, White, Black, Latino, Middle Eastern, and Other; The categories of expressions include: joy, sadness, anger, surprise, fear, and disgust.

[0032] Specifically, determine the labels of synthetic data (i.e., facial expression data) and the corresponding category distribution. To ensure the diversity and balance of the synthetic data set, first determine the following labels and the corresponding category distribution: 1) Age: The data is divided into five age groups, namely children (0-12 years old), teenagers (13-18 years old), young people (19-35 years old), middle-aged (36-55 years old), and the elderly (over 56 years old). The sample proportion of each age group can be 20%.

[0033] 2) Gender: Ensure that the sample is balanced between males and females, accounting for 50% each.

[0034] 3) Race: including Asian, white, black, Latino, and Middle Eastern, with balanced distribution of racial samples, for example, 20% each; other races may also be included. If other races are involved, the proportion of each race may be distributed according to other proportions known to those skilled in the art, and there is no limitation here.

[0035] 4) Expressions: Select seven main expressions including joy, sadness, anger, surprise, fear, disgust, and neutrality. The samples of each expression should be balanced, for example, each accounting for 1 / 7.

[0036] The above-mentioned categories and their corresponding ratios are exemplary only, and other categories and their corresponding ratios known to those skilled in the art may be applied here without limitation.

[0037] In this application, we choose age, gender, race and expression as labels for synthetic data to ensure the diversity and balance of the dataset in these key dimensions. This can effectively improve the model's ability to recognize and generate different groups of people and expressions, reduce bias, and improve its generalization ability in real application scenarios.

[0038] In some embodiments, generating a corresponding text description based on the tag includes: Get the target category corresponding to each label; The target category is combined with the environment to generate a corresponding text description.

[0039] Specifically, the large language model GPT-4 can be used to generate text descriptions for the text graph model based on the above labels. Each description will combine age, gender, race, expression and environment. For example: Example 1: An Asian woman in her 30s was laughing on the grass; Example 2: A white male in his 50s displays an angry expression in the office.

[0040] Generate a specific text description for each group of labels. According to the predetermined category ratio, a total of 10,000 text descriptions can be generated (here is an example). Take a one-by-one approach, combine the corresponding label information for each description, and generate a description. The prompt words given to GPT-4 each time are as follows: For example, based on the following tag information, generate a text description containing the following tag information: {age} {gender} {race} {expression}.

[0041] example: - Tag information: 30, female, Asian, joy; - Output: An Asian female, approximately 30 years old, laughing on the grass.

[0042] In some embodiments, generating a face training data set based on the text description includes: Based on the text description, multiple generation models are used to generate N expression pictures, each of which has corresponding label information, wherein the label information includes a target category corresponding to the expression picture, wherein N is a positive integer; the target category refers to the categories corresponding to the labels used when generating the text description, such as 30, female, Asian, and joy, that is, the actual categories corresponding to age, gender, race, and expression.

[0043] Associating and storing each expression picture with a text description corresponding to the expression picture and corresponding tag information as an image data set; Generate a corresponding expression picture dialogue for each expression picture, associate each expression picture with the expression picture dialogue corresponding to the expression picture and store it as an expression picture dialogue data set, wherein the expression picture dialogue includes an expression picture question and an expression picture answer; The image data set and the expression picture dialogue data set constitute the face training data set.

[0044] In some embodiments, the multiple generation models include: FLUX model, Stable Diffusion 3 model, MidJourney model, and Keling model; for each of the generation models, a corresponding expression picture is generated.

[0045] In some embodiments, four generative models, namely, the FLUX model, the Stable Diffusion 3 model, the MidJourney model, and the Keling model, are used to generate 40,000 facial expression images with expression labels. The label information and the generated text description corresponding to each image should also be retained and correspond to the image one by one to form an image data set; the example here is that each model generates 10,000 images.

[0046] Generate emoji image dialogues: Each emoji image needs to be equipped with at least one set of dialogues, where the dialogues contain descriptive questions and answers. Conversational data is used for model training. Questions can be randomly selected from the template library, and the answers are the emojis in the label. The template library contains a large number of sentences like "What is the expression of the person in the picture?" For example: Image Description: The image shows an Asian woman in her 30s laughing on the grass.

[0047] Dialogue generation, generates the following dialogue according to the description: Expression picture question: "What kind of emotion does the person in the picture express?" The emoji replied: "Joy." In some embodiments, fine-tuning the first model using the face training data set to obtain a target model includes: Inputting the image dataset and the expression picture dialogue dataset in the face training dataset into the first model; A visual input channel and a language input channel are respectively used as feedback inputs of the first model, wherein the image data set is used as a training part of the visual input channel, and the expression picture dialogue data set is used as a training part of the language input channel; The model output of the first model is obtained.

[0048] In some embodiments, the fine-tuning process includes: using a face training dataset (i.e., a synthetic dataset) to fine-tune the model, using a standard supervised learning method, and optimizing the model parameters through a cross-entropy loss function. Use a synthetic image dataset and the above expression picture dialogue dataset as input, and feed them to the model through visual and language input channels respectively. The image dataset is used to train the visual part of the model, and the expression picture dialogue dataset is used to train the language understanding and generation part, that is, the expression picture dialogue dataset is used as the training part of the language input channel, and then the model predicts the output expression; using a standard supervised learning method, the model output is compared with the real label through a cross-entropy loss function to optimize the model parameters. Specifically, the model predicts certain labels (such as generating a correct image description or answering questions related to the image) based on the input image and text description, and the loss function is used to measure the difference between the predicted result and the actual label.

[0049] In some embodiments, after obtaining the model output of the first model, the method further includes: Using a supervised learning method, the model output is compared with the label information through a cross entropy loss function to obtain a comparison result; Optimizing parameters of the first model based on the comparison result; The first model is trained and verified based on the optimized parameters to obtain a target model, which is used in the facial expression recognition task.

[0050] In some embodiments, before fine-tuning the first model using the face training dataset, the method further includes: Loading the first model; Configure the training hyperparameters of the first model, wherein the training hyperparameters include: optimizer, initial learning rate, batch size, number of epochs, gradient clipping value, and drop probability. Specifically, one epoch refers to the process of training all training samples once.

[0051] In some embodiments, model initialization includes: loading a pre-trained LLaVA-OneVision model, and configuring training hyperparameters such as the optimizer AdamW and the learning rate scheduler. The hyperparameter configuration includes using the AdamW optimizer, the initial learning rate is 1e-4 and a linear decay strategy is adopted, the batch size is 32, training is 10 epochs, the gradient clipping value is set to 1.0, and the dropout probability is 0.1. The specific values ​​of each parameter here are exemplary, and those skilled in the art can determine the specific parameter values ​​according to the actual situation, and there is no limitation on this.

[0052] In some embodiments, the evaluation and verification include: during the fine-tuning process, regularly using the validation set to evaluate the model performance, monitoring its performance on different expression categories, and ensuring the improvement of the model's generalization ability.

[0053] In some embodiments, final model saving includes: after sufficient fine-tuning and verification, saving the optimized model for further application in actual facial expression recognition tasks.

[0054] The beneficial effect of this application is that the performance of facial expression recognition is significantly improved by utilizing high-quality synthetic data and fine-tuning a large multimodal model.

[0055] Compared with the prior art, this application has at least the following advantages: 1. Improve data diversity and quality: The use of Wensheng graph models (such as FLUX model, Stable Diffusion 3 model, MidJourney model, Keling model, etc.) to generate diverse and high-quality facial expression data makes up for the insufficiency and scarcity of real data in the acquisition process. With rich data input, the model is more robust in processing different types of expressions, and the recognition accuracy has been significantly improved.

[0056] 2. Optimize the model’s expression recognition capabilities: By fine-tuning the multimodal large model using synthetic data, this application makes the model more adaptable and specialized in the task of expression recognition. Compared with the unfine-tuned model, the fine-tuned model can more effectively extract features related to expression and improve recognition accuracy.

[0057] 3. Enhance the generalization ability of the model: Since a large amount of expression data of different scenes and different characters is generated and the model is fine-tuned on this basis, the generalization ability of the model is improved. When the model is applied to actual scenes or different groups of people, its recognition effect is more stable and consistent, and it can cope with a wider range of usage scenarios.

[0058] 4. Reduce data acquisition costs and time: The traditional facial expression data collection process is usually time-consuming and laborious, and is easily restricted by various external conditions. This application reduces the reliance on large-scale real data through synthetic data generation, reduces the cost and time of data acquisition, and provides a more economical and efficient solution for the development and deployment of expression recognition systems.

[0059] It should be noted that the method of one or more embodiments of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of one or more embodiments of the present application, and the multiple devices will interact with each other to complete the described method.

[0060] It should be noted that the above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0061] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also discloses an electronic device.

[0062] Specifically, Figure 2 The hardware structure diagram of an electronic device for a large-model face recognition method based on synthetic data provided in this embodiment is shown, and the device may include: a processor 410, a memory 420, an input / output interface 430, a communication interface 440, and a bus 450. The processor 410, the memory 420, the input / output interface 430, and the communication interface 440 are connected to each other through the bus 450 in the device.

[0063] The processor 410 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0064] The memory 420 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 420 may store an operating system and other application programs. When the technical solution provided in the embodiment of the present application is implemented by software or firmware, the relevant program code is stored in the memory 420 and is called and executed by the processor 410.

[0065] The input / output interface 430 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0066] The communication interface 440 is used to connect a communication module (not shown in the figure) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired manner (for example, USB, network cable, etc.) or a wireless manner (for example, mobile network, Wi-Fi, Bluetooth, etc.).

[0067] The bus 450 includes a path that transmits information between the various components of the device (eg, the processor 410 , the memory 420 , the input / output interface 430 , and the communication interface 440 ).

[0068] It should be noted that, although the above device only shows the processor 410, the memory 420, the input / output interface 430, the communication interface 440 and the bus 450, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiment of the present application, and does not necessarily include all the components shown in the figure.

[0069] The electronic device of the above-mentioned embodiment is used to implement the corresponding large-model face recognition method based on synthetic data in any of the above-mentioned embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0070] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, one or more embodiments of the present application also provide a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the large model face recognition method based on synthetic data as described in any of the above embodiments.

[0071] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0072] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the large model face recognition method based on synthetic data as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0073] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application is limited to these examples. Based on the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0074] In addition, to simplify the description and discussion, and in order not to make one or more embodiments of the present application difficult to understand, the well-known power / ground connections to the integrated circuit (IC) chip and other components may or may not be shown in the provided figures. In addition, the device may be shown in the form of a block diagram to avoid making one or more embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which one or more embodiments of the present application will be implemented (that is, these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that one or more embodiments of the present application can be implemented without these specific details or with changes in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0075] Although the present application has been described in conjunction with specific embodiments of the present application, many alternatives, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the discussed embodiments.

[0076] One or more embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A large model face recognition method based on synthetic data, characterized in that: include: Determine the labels of the synthetic data and the category corresponding to each label; Generate a corresponding text description based on the tag; Generate a face training data set based on the text description; The first model is fine-tuned using the face training data set to obtain a target model.

2. The large model face recognition method based on synthetic data according to claim 1 is characterized in that: The labels include: age, gender, race and expression; and Age-corresponding categories include: children, adolescents, young adults, middle-aged, and elderly; Categories corresponding to gender include: male and female; The categories corresponding to race include: Asian, White, Black, Latino, Middle Eastern, and Other; The categories of expressions include: joy, sadness, anger, surprise, fear and disgust.

3. The large model face recognition method based on synthetic data according to claim 1 or 2, characterized in that: The generating a corresponding text description based on the tag includes: Get the target category corresponding to each label; The target category is combined with the environment to generate a corresponding text description.

4. The large model face recognition method based on synthetic data according to claim 3 is characterized in that: Generating a face training data set based on the text description includes: Based on the text description, using multiple generation models, N expression pictures are generated, each expression picture has corresponding label information, wherein the label information includes a target category corresponding to the expression picture, and N is a positive integer; Associating and storing each expression picture with a text description corresponding to the expression picture and corresponding tag information as an image data set; Generate a corresponding expression picture dialogue for each expression picture, associate each expression picture with the expression picture dialogue corresponding to the expression picture and store it as an expression picture dialogue data set, wherein the expression picture dialogue includes an expression picture question and an expression picture answer; The image data set and the expression picture dialogue data set constitute the face training data set.

5. The large model face recognition method based on synthetic data according to claim 4 is characterized in that: The fine-tuning training of the first model using the face training data set to obtain the target model includes: Inputting the image dataset and the expression picture dialogue dataset in the face training dataset into the first model; A visual input channel and a language input channel are respectively used as feedback inputs of the first model, wherein the image data set is used as a training part of the visual input channel, and the expression picture dialogue data set is used as a training part of the language input channel; The model output of the first model is obtained.

6. The large model face recognition method based on synthetic data according to claim 5 is characterized in that: After obtaining the model output of the first model, the method further includes: Using a supervised learning method, the model output is compared with the label information through a cross entropy loss function to obtain a comparison result; Optimizing parameters of the first model based on the comparison result; The first model is trained and verified based on the optimized parameters to obtain a target model, which is used in the facial expression recognition task.

7. The large model face recognition method based on synthetic data according to claim 4 is characterized in that: The multiple generation models include: FLUX model, Stable Diffusion 3 model, MidJourney model and Keling model; for each generation model, a corresponding expression picture is generated.

8. The large model face recognition method based on synthetic data according to claim 1, characterized in that: Before fine-tuning the first model using the face training data set, the method further includes: Loading the first model; Configure the training hyperparameters of the first model, wherein the training hyperparameters include: optimizer, initial learning rate, batch size, number of epochs, gradient clipping value, and drop probability.

9. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the large model face recognition method based on synthetic data as described in any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the large model face recognition method based on synthetic data as described in any one of claims 1 to 8.

Images