Face recognition interaction methods, devices, storage media and equipment

Abstract

This specification discloses a face recognition interaction method, apparatus, storage medium, and device. The method includes: acquiring a face image containing a target face; performing facial landmark detection processing on the target face in the face image to obtain a facial landmark detection result corresponding to the target face; performing emotion recognition processing on the target face to obtain an emotion category corresponding to the target face; determining the face position of the target face in the face image based on the facial landmark detection result; acquiring a virtual face internal component corresponding to the emotion category; and displaying the virtual face internal component, the virtual face external component, and interactive text corresponding to the face position on a display interface based on the face position.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a face recognition interaction method, apparatus, storage medium and device. Background Technology

[0002] As facial recognition technology matures, its commercial applications are becoming increasingly widespread, such as in financial transactions, access control systems, and mobile terminals. However, when users perform facial recognition on their mobile phones or offline IoT devices, if the user's face position does not meet the facial recognition requirements, the facial recognition device will interact with the user through the display screen, prompting the user to move to a suitable position. Summary of the Invention

[0003] This specification provides a facial recognition interaction method, apparatus, storage medium, and device. By recognizing facial images captured by a camera, the facial recognition device enables intelligent, emotion-based interaction between the user and the user. The technical solution is as follows:

[0004] In a first aspect, embodiments of this specification provide a face recognition interaction method, the method comprising:

[0005] Obtain a face image containing the target person's face;

[0006] Facial landmark detection is performed on the target face in the face image to obtain the facial landmark detection result corresponding to the target face, and emotion recognition is performed on the target face to obtain the emotion category corresponding to the target face.

[0007] Based on the facial key point detection results, the facial position of the target face in the facial image is determined, and the internal components of the virtual face corresponding to the emotion category are obtained;

[0008] Based on the face location, the internal components of the virtual face, the external components of the virtual face, and the interactive text corresponding to the face location are displayed on the display interface.

[0009] Secondly, embodiments of this specification provide a method for training a face detection model, the method comprising:

[0010] Establish a training image dataset, which includes each sample training image and the key point coordinate labels and sentiment category labels corresponding to each sample training image;

[0011] The sample training image is input into the face detection model to obtain the key point coordinate prediction result and emotion category prediction result output by the face detection model for the sample training image;

[0012] Based on the key point coordinate prediction results and the key point coordinate labels, the key point prediction loss value is calculated using a preset key point loss function.

[0013] Based on the emotion category prediction results and the emotion category labels, the emotion prediction loss value is calculated using a preset emotion loss function;

[0014] The model parameters of the face detection model are updated based on the key point prediction loss value and the emotion prediction loss value.

[0015] Determine whether the face detection model with updated parameters meets the preset convergence condition. If it does, stop training and obtain the trained face detection model. If it does not meet the condition, execute the step of inputting the sample training image into the face detection model.

[0016] Thirdly, embodiments of this specification provide a face recognition interaction device, the device comprising:

[0017] The image acquisition module is used to acquire a face image containing the target face;

[0018] The image processing module is used to perform facial landmark detection processing on the target face in the face image to obtain the facial landmark detection result corresponding to the target face, and to perform emotion recognition processing on the target face to obtain the emotion category corresponding to the target face.

[0019] The component determination module is used to determine the face position of the target face in the face image based on the face key point detection results, and to obtain the internal components of the virtual face corresponding to the emotion category;

[0020] The interactive display module is used to display the internal components of the virtual face, the external components of the virtual face, and the interactive text corresponding to the face position on the display interface based on the face position.

[0021] Fourthly, embodiments of this specification provide a face detection model training device, the device comprising:

[0022] The training data acquisition module is used to establish a training image dataset, which includes each sample training image and the key point coordinate labels and sentiment category labels corresponding to each sample training image.

[0023] The face detection module is used to input the sample training image into the face detection model to obtain the key point coordinate prediction result and emotion category prediction result output by the face detection model for the sample training image;

[0024] The first loss calculation module is used to calculate the key point prediction loss value based on the key point coordinate prediction result and the key point coordinate label, using a preset key point loss function.

[0025] The second loss calculation module is used to calculate the emotion prediction loss value based on the emotion category prediction result and the emotion category label using a preset emotion loss function.

[0026] The model parameter update module is used to update the model parameters of the face detection model based on the key point prediction loss value and the emotion prediction loss value.

[0027] The model convergence judgment module is used to determine whether the face detection model with updated parameters meets the preset convergence condition. If it does, training is stopped and a trained face detection model is obtained. If it does not meet the condition, the step of inputting the sample training image into the face detection model is executed.

[0028] Fifthly, embodiments of this specification provide a computer program product that stores at least one instruction adapted to be loaded by a processor and executed in accordance with the above-described method steps.

[0029] Sixthly, embodiments of this specification provide a storage medium storing a computer program adapted to be loaded by a processor and to execute the above-described method steps.

[0030] In a seventh aspect, embodiments of this specification provide an electronic device that may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the above-described method steps.

[0031] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following:

[0032] The face recognition interaction method provided in the embodiments of this specification involves performing facial landmark detection and emotion recognition on a face image to obtain the facial landmark detection results and emotion category corresponding to the face image. Then, the face position is determined based on the facial landmark detection results, and the internal components of the virtual face are determined based on the emotion category. Finally, the internal components of the virtual face, the external components of the virtual face, and the interactive text corresponding to the face position are displayed on the display interface based on the face position. This allows for intelligent interaction with the user based on a virtual face on a face recognition device, thereby improving the user experience. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 A flowchart illustrating a face recognition interaction method provided in an embodiment of this specification;

[0035] Figure 2 This is an example diagram illustrating the acquisition of a human face image, provided as an embodiment of this specification.

[0036] Figure 3 A flowchart illustrating a face recognition interaction method provided in an embodiment of this specification;

[0037] Figure 4 This is a schematic diagram illustrating an example of image cropping provided in an embodiment of this specification;

[0038] Figure 5 This is a schematic diagram illustrating an example of internal and external components of a virtual face provided in an embodiment of this specification.

[0039] Figure 6 This is an example diagram illustrating a virtual face provided in an embodiment of this specification;

[0040] Figure 7 This is an example diagram illustrating the display of interactive text and a virtual face, provided as an embodiment of this specification.

[0041] Figure 8 This is a flowchart illustrating a face detection model training method provided in an embodiment of this specification.

[0042] Figure 9 This is a schematic diagram of the structure of a face recognition interactive device provided in the embodiments of this specification;

[0043] Figure 10 This is a schematic diagram of the structure of a face recognition interactive device provided in the embodiments of this specification;

[0044] Figure 11 This is a schematic diagram of the structure of a face detection model training device provided in the embodiments of this specification;

[0045] Figure 12 This is a structural block diagram of an electronic device provided as an embodiment of this specification. Detailed Implementation

[0046] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0047] In the description of this specification, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. In the description of this specification, it should be noted that, unless otherwise expressly specified and limited, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. Those skilled in the art can understand the specific meaning of the above terms in this specification based on the specific circumstances. Furthermore, in the description of this specification, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0048] The following detailed description is provided in conjunction with embodiments of the examples in this specification. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims. The flowcharts shown in the accompanying drawings are merely illustrative and are not necessarily to be performed in accordance with the steps shown. For example, some steps are parallel and do not have a strict logical order; therefore, the actual execution order is variable.

[0049] In related technologies, when a user performs facial recognition on a facial recognition device, the device captures an image of the face within the camera's field of view and displays the captured image directly on the screen. The captured image includes not only the user's target face but also other scenes. In some places where privacy is paramount, directly displaying the captured image on the screen can lead to a poor privacy experience for the target user.

[0050] Based on this, embodiments of this specification provide a face recognition interaction method. This method involves performing facial landmark detection and emotion recognition on a face image to obtain the facial landmark detection results and emotion category corresponding to the face image. Then, the face position is determined based on the facial landmark detection results, and the internal components of the virtual face are determined based on the emotion category. Finally, based on the face position, the internal components of the virtual face, the external components of the virtual face, and the interactive text corresponding to the face position are displayed on the display interface. This allows for interaction between the virtual face and the user on the face recognition device's display interface, ensuring user privacy and security. Furthermore, determining the internal components of the virtual face based on the emotion category enables intelligent, emotionally-inspired interaction between the virtual face and the user on the face recognition device, further enhancing the user experience.

[0051] Please see Figure 1 This is a flowchart illustrating a face recognition interaction method provided in an embodiment of this specification. In the embodiments of this specification, the face recognition interaction method is applied to a face recognition interaction device or an electronic device equipped with a face recognition interaction device. The following will focus on... Figure 1 The process shown will be described in detail. The face recognition interaction method may specifically include the following steps:

[0052] S102, Obtain a face image containing the target face;

[0053] In the embodiments described in this specification, the face recognition interaction device acquires a face image containing the target face based on the image acquisition device.

[0054] It should be noted that, in the embodiments of this specification, the face recognition interaction device can be a face recognition interaction device, including but not limited to terminal devices such as mobile phones, computers, tablets, smart wearable devices, in-vehicle devices, and IoT devices. It can also be a module within a terminal device used to implement the face recognition interaction method.

[0055] In the embodiments described in this specification, the face recognition interaction device acquires a face image containing the target face through a camera.

[0056] Please see Figure 2 This is an example diagram illustrating the acquisition of a facial image provided in an embodiment of this specification. Figure 2 As shown, an image acquisition device is installed on the face recognition interactive device, which can be used to acquire the face image of the target user.

[0057] S104, Perform facial landmark detection processing on the target face in the face image to obtain the facial landmark detection result corresponding to the target face, and perform emotion recognition processing on the target face to obtain the emotion category corresponding to the target face.

[0058] In the embodiments of this specification, after obtaining a face image containing the target face, facial landmark detection and emotion recognition processing are performed on the target face in the face image. The facial landmark detection processing can obtain the coordinates of the key points of the target face in the face image, which is used to determine the position of the target face in the face image. The emotion recognition processing can identify the emotion category of the target face in the face image.

[0059] Optionally, facial landmark detection and emotion recognition can be performed on the target face in the face image based on a pre-trained face detection model. The acquired face image is input into the pre-trained face detection model, and the face detection model outputs the facial landmark detection results and emotion category corresponding to the target face in the face image.

[0060] The face detection model is a neural network model based on deep learning.

[0061] Optionally, the face detection model includes a feature extraction network and a feature prediction network. The feature extraction network is used to extract the facial features of the target face in the face image, and the feature prediction network is used to predict facial landmarks and emotions based on the extracted facial features, so as to obtain the facial landmark detection results and emotion category corresponding to the target face.

[0062] Optionally, the face detection model includes a first feature extraction network, a second feature extraction network, a first feature prediction network, and a second feature prediction network. The first feature extraction network extracts facial keypoint features of the target face in the face image; the second feature extraction network extracts facial emotion features of the target face in the face image; the first feature prediction network predicts keypoints based on the facial keypoint features to obtain the facial keypoint detection result corresponding to the target face in the face image; and the second feature prediction network predicts emotion based on the emotion features to obtain the predicted emotion category corresponding to the target face in the face image.

[0063] In the embodiments of this specification, before performing facial landmark detection processing on the target face in the face image to obtain the facial landmark detection result corresponding to the target face, and performing emotion recognition processing on the target face to obtain the emotion category corresponding to the target face, the method further includes: performing face detection processing on the face image to obtain a face bounding box containing the target face, and cropping the face image based on the face bounding box to obtain a cropped face image. Then, facial landmark detection processing and emotion recognition processing are performed on the cropped face image.

[0064] It should be noted that when performing facial landmark detection and emotion recognition on a target face in a face image, the face image is first cropped to obtain a face image with a larger proportion of the target face. In this way, performing facial landmark detection and emotion recognition on the cropped face image can reduce the proportion of non-face features in the face image, reduce interference, and improve the detection accuracy of facial landmark detection and emotion recognition.

[0065] S106, Based on the facial key point detection results, determine the face position of the target face in the face image, and obtain the internal components of the virtual face corresponding to the emotion category;

[0066] In the embodiments of this specification, the position of the target face in the face image is determined based on the facial key point detection results of the target face in the face image. The facial key point detection results include the facial key point coordinates corresponding to each facial key point. Based on the facial key point coordinates corresponding to each facial key point, the position of the target face in the face image can be determined. Furthermore, the component corresponding to the emotion category is determined in the predefined virtual face internal components according to the emotion category.

[0067] It should be noted that the virtual face is generated from internal and external components. The internal components represent the emotional state of the virtual face. A predefined set of internal components is used, including components corresponding to various emotional categories. After determining the emotional category of the target face in the face image, the corresponding internal component is selected from the predefined set based on the target face's emotional category. This ensures that the emotional state of the virtual face generated based on the determined internal and external components matches the target face in the face image, enabling intelligent interaction with the user and conveying emotions.

[0068] In the embodiments of this specification, determining the face position of the target face in the face image based on the face key point detection results includes: calculating the first average value of each horizontal coordinate in the coordinates of each face key point, and calculating the second average value of each vertical coordinate in the coordinates of each face key point, using the first average value as the horizontal coordinate and the second average value as the vertical coordinate to generate face position coordinates representing the face position.

[0069] S108, based on the face position, displays the internal components of the virtual face, the external components of the virtual face, and the interactive text corresponding to the face position on the display interface.

[0070] In the embodiments of this specification, after determining the face position of the target face and the internal components of the virtual face, a virtual face is generated based on the internal components and the external components of the virtual face, and the corresponding interactive text is determined based on the face position. The virtual face and the interactive text are then displayed on the display interface of the face recognition device.

[0071] Understandably, after obtaining the target face's location in the face image, if the corresponding face location does not meet the face recognition requirements, the face recognition device will prompt the user to move their face to a designated position for easier recognition. The interactive text is a prompt generated based on the face's location, used to guide the user to move their face so that the face recognition device can accurately identify the target face. For example, if the target face's location in the face image is off-center and near the top left corner, the face recognition device will generate an interactive text prompting the user to move to the bottom right, displaying the interactive text and the virtual face on the display interface.

[0072] Furthermore, the virtual face internal components are determined based on the emotional category of the target face in the face image. The virtual face internal components express the same emotional state as the target face, which can enable the virtual face to interact with the user with the same emotion, thereby improving the user interaction experience.

[0073] Optionally, before generating a virtual face based on the virtual face internal components and the virtual face external components, the relative positions of the virtual face internal components and the virtual face external components are determined based on the face position, and the virtual face internal components and the virtual face external components are combined based on the determined relative positions to obtain a virtual face.

[0074] Understandably, adjusting the relative positions of the internal and external components of the virtual face based on the target face position allows the generated virtual face to interact in a state facing the target face, thus improving the user's interactive experience.

[0075] In the embodiments of this specification, facial landmark detection and emotion recognition are performed on a face image to obtain the facial landmark detection results and emotion category corresponding to the face image. Then, the face position is determined based on the facial landmark detection results, and the internal components of the virtual face are determined based on the emotion category. Finally, based on the face position, the internal components of the virtual face, the external components of the virtual face, and the interactive text corresponding to the face position are displayed on the display interface. The virtual face interacts with the user on the display interface of the face recognition device, ensuring the user's privacy and security. Furthermore, by determining the internal components of the virtual face based on the emotion category, intelligent interaction with the user based on the virtual face can be realized on the face recognition device, further enhancing the user experience.

[0076] Please see Figure 3 This is a flowchart illustrating a face recognition interaction method provided in an embodiment of this specification. Figure 3 As shown, the face recognition interaction method may include the following steps:

[0077] S202, Obtain a face image containing the target face;

[0078] In the embodiments of this specification, step S202 is described in detail in another embodiment of this specification for step S102, and will not be repeated here.

[0079] S204, Perform face detection processing on the face image to obtain a face bounding box containing the target face;

[0080] In the embodiments of this specification, after obtaining a face image including the target face, a face detection algorithm is used to identify the target face in the face image and generate a face bounding box including the target face.

[0081] S206, Crop the face image based on the face bounding box to obtain the cropped face image;

[0082] In the embodiments of this specification, the face image is further cropped based on the generated face bounding box to obtain a cropped face image including the target face.

[0083] Understandably, when acquiring facial images that include the target face, most of the area outside the target face may be background. A large background area can negatively impact the accuracy of facial landmark detection and emotion recognition. Therefore, cropping the facial image reduces the amount of background data, thus ensuring the accuracy of facial landmark detection and emotion recognition.

[0084] Please see Figure 4 This is an example diagram illustrating an image cropping method provided in an embodiment of this specification. Figure 4 As shown, the initial face image is a captured face image that has not been cropped. By performing face detection processing on the initial face image, a face bounding box including the target face is generated when the target face is detected in the initial face image. Then, the initial face image is cropped based on the face bounding box to obtain the cropped face image.

[0085] S208, Input the face image into the pre-trained face detection model to obtain the face key point detection results and emotion category corresponding to the target face in the face image;

[0086] In the embodiments of this specification, the cropped face image is input into a pre-trained face detection model, which performs facial landmark detection and emotion recognition processing on the target face in the face image to obtain the facial landmark detection result and emotion category corresponding to the target face in the face image.

[0087] Optionally, the face detection model includes a feature extraction network and a feature prediction network. The feature extraction network is used to extract the facial features of the target face in the face image, and the feature prediction network is used to predict facial landmarks and emotions based on the extracted facial features, so as to obtain the facial landmark detection results and emotion category corresponding to the target face.

[0088] Optionally, the face detection model includes a first feature extraction network, a second feature extraction network, a first feature prediction network, and a second feature prediction network. The first feature extraction network extracts facial keypoint features of the target face in the face image; the second feature extraction network extracts facial emotion features of the target face in the face image; the first feature prediction network predicts keypoints based on the facial keypoint features to obtain the facial keypoint detection result corresponding to the target face in the face image; and the second feature prediction network predicts emotion based on the emotion features to obtain the emotion category corresponding to the target face in the face image.

[0089] Optionally, the face detection model includes a first feature extraction network, a second feature extraction network, a feature fusion network, a first feature prediction network, and a second feature prediction network. Specifically, the first feature extraction network extracts facial key point features of the target face in the face image; the second feature extraction network extracts facial emotion features of the target face in the face image; the feature fusion network fuses the facial key point features and facial emotion features to obtain a fused emotion feature; the first feature prediction network predicts key points based on the facial key point features to obtain the facial key point detection result corresponding to the target face in the face image; and the second feature prediction network predicts the emotion based on the fused emotion feature to obtain the emotion category corresponding to the target face in the face image.

[0090] S210, calculate the first average value of each horizontal coordinate in the coordinates of each facial key point, and calculate the second average value of each vertical coordinate in the coordinates of each facial key point.

[0091] In the embodiments of this specification, the facial landmark detection results include a preset number of facial landmark coordinates, a first average value of each horizontal coordinate in each facial landmark coordinate, and a second average value of each vertical coordinate in each facial landmark coordinate.

[0092] S212, using the first average value as the horizontal axis and the second average value as the vertical axis, generate face position coordinates representing the face position;

[0093] In the embodiments of this specification, the first average value is used as the horizontal axis and the second average value is used as the vertical axis to generate face position coordinates representing the face position.

[0094] It is understood that in the embodiments of this specification, the coordinates of each obtained facial key point are averaged, and the coordinates corresponding to the average value are used as the facial position of the target face in the facial image.

[0095] S214, Obtain the internal components of the virtual face corresponding to the emotion category;

[0096] It should be noted that the virtual face is generated from internal and external components. The internal components represent the emotional state of the virtual face. A predefined set of internal components is used, including components corresponding to various emotional categories. After determining the emotional category of the target face in the face image, the corresponding internal component is selected from the predefined set based on the target face's emotional category. This ensures that the emotional state of the virtual face generated based on the determined internal and external components matches the target face in the face image, enabling intelligent interaction with the user and conveying emotions.

[0097] Please see Figure 5 This is a schematic diagram illustrating an example of internal and external components of a virtual face provided in an embodiment of this specification. Figure 5 As shown, a virtual face consists of an external virtual face component and one of multiple internal virtual face components.

[0098] S216, Determine the relative positions of the inner and outer components of the virtual face based on the face position;

[0099] In the embodiments of this specification, an inscribed rectangle is constructed for the outer component of the virtual face, and the relative position of the inner component of the virtual face within the inscribed rectangle is determined based on the position of the target face in the face image.

[0100] For example, if the target face is located in the lower left corner of the face image, then the relative position of the inner component of the virtual face is determined to be the lower left corner of the inscribed rectangle of the outer component of the virtual face. The relative position can be determined based on a proportional method.

[0101] S218, The virtual face internal components and virtual face external components are combined based on their relative positions to obtain a virtual face;

[0102] In the embodiments of this specification, after determining the relative positions of the inner components and outer components of the virtual face, the inner components and outer components of the virtual face are combined based on their relative positions to obtain a virtual face.

[0103] Understandably, adjusting the relative positions of the internal and external components of the virtual face based on the target face position allows the generated virtual face to interact in a state facing the target face, thus improving the user's interactive experience.

[0104] Please see Figure 6 This is an example diagram illustrating a virtual face provided in an embodiment of this specification. Figure 6 As shown, if the user's target face is located near the left side of the face image frame, then the relative position of the inner component of the virtual face should be located to the left of the inscribed rectangle of the outer component of the virtual face.

[0105] Optionally, the virtual face external component may include a male virtual face external component and a female virtual face external component. By performing face recognition on the target face in the face image, when the face is recognized as female, a virtual face is generated based on the female virtual face external component and the virtual face internal component; when the face is recognized as male, a virtual face is generated based on the male virtual face external component and the virtual face internal component.

[0106] S220 displays the virtual face and the corresponding interactive text on the display interface.

[0107] In the embodiments described in this specification, the corresponding interactive text is determined based on the face location, and the virtual face and the interactive text are displayed on the display interface of the face recognition device.

[0108] Understandably, after obtaining the target face's location in the face image, if the corresponding face location does not meet the face recognition requirements, the face recognition device will prompt the user to move their face to a designated position for easier recognition. The interactive text is a prompt generated based on the face's location, used to guide the user to move their face so that the face recognition device can accurately identify the target face. For example, if the target face's location in the face image is off-center and near the top left corner, the face recognition device will generate an interactive text prompting the user to move to the bottom right, displaying the interactive text and the virtual face on the display interface.

[0109] Please see Figure 7 This is an example diagram illustrating the display of interactive text and a virtual face, provided in an embodiment of this specification. Figure 7As shown, the virtual face is generated based on the relative positions of the internal and external components of the virtual face, determined by the face position. As can be seen from the virtual face, the target face is located near the right side of the face image. Therefore, interactive text corresponding to the face position is displayed below the virtual face: Please move a little to the left.

[0110] In the embodiments of this specification, a face image containing the target face is first acquired. Then, face detection processing is performed on the face image to obtain a face bounding box containing the target face. Based on the face bounding box, the face image is cropped to obtain a cropped face image. By cropping the face image, the background area of ​​the non-target image is reduced, which can ensure the accuracy of facial key point detection and emotion recognition. Next, the cropped face image is input into a pre-trained face detection model to obtain the facial key point detection results and emotion category corresponding to the target face in the face image. Then, the face position is determined based on the facial key point detection results, and the internal components of the virtual face are determined based on the emotion category. Finally, based on the face position, the internal components of the virtual face, the external components of the virtual face, and the interactive text corresponding to the face position are displayed on the display interface. The virtual face interacts with the user on the display interface of the face recognition device, ensuring the user's privacy and security. Furthermore, by determining the internal components of the virtual face based on the emotion category, intelligent interaction with the user based on the virtual face can be realized on the face recognition device, further improving the user experience.

[0111] Please see Figure 8 This is a flowchart illustrating a face detection model training method provided in an embodiment of this specification. Figure 8 As shown, the face detection model training method may include the following steps:

[0112] S302, Establish a training image dataset. The training image dataset includes each sample training image and the key point coordinate labels and sentiment category labels corresponding to each sample training image.

[0113] In the embodiments of this specification, a training image dataset is first created, sample training images of each sample face are collected for face recognition, and corresponding key point coordinate labels and emotion category labels are labeled for each sample training image.

[0114] S304, Input the sample training image into the face detection model to obtain the key point coordinate prediction result and emotion category prediction result output by the face detection model for the sample training image;

[0115] In the embodiments of this specification, sample training images from the training image dataset are input into the constructed face detection model to obtain the key point coordinate prediction results and emotion category prediction results output by the face detection model for the sample training images.

[0116] Optionally, the face detection model may include a feature extraction network and a feature prediction network. The feature extraction network is used to extract facial features of sample faces in the sample training images, and the feature prediction network is used to predict facial key points and emotions based on the extracted facial features, so as to obtain the key point coordinate prediction results and emotion category prediction results corresponding to the sample face.

[0117] In the embodiments of this specification, the step of inputting the sample training image into the face detection model to obtain the key point coordinate prediction result and emotion category prediction result output by the face detection model for the sample training image includes: extracting facial key point features from the sample training image based on the first feature extraction network in the face detection model; extracting facial expression features from the sample training image based on the second feature extraction network in the face detection model; performing key point prediction on the facial key point features based on the first feature prediction network in the face detection model to obtain the key point coordinate prediction result corresponding to the sample training image; and performing emotion prediction on the facial expression features based on the second feature prediction network in the face detection model to obtain the emotion category prediction result corresponding to the sample training image.

[0118] Optionally, the face detection model may include a first feature extraction network, a second feature extraction network, a first feature prediction network, and a second feature prediction network. The first feature extraction network extracts facial keypoint features from the sample training images; the second feature extraction network extracts facial emotion features from the sample training images; the first feature prediction network predicts keypoint coordinates based on the facial keypoint features to obtain the predicted coordinates of the keypoints corresponding to the sample faces in the sample training images; and the second feature prediction network predicts the emotion category based on the emotion features to obtain the predicted emotion category of the sample faces in the sample training images.

[0119] Optionally, the face detection model may include a first feature extraction network, a second feature extraction network, a feature fusion network, a first feature prediction network, and a second feature prediction network. Specifically, the first feature extraction network extracts facial keypoint features from the sample training images; the second feature extraction network extracts facial emotion features from the sample training images; the feature fusion network fuses the facial keypoint features and facial emotion features to obtain fused emotion features; the first feature prediction network predicts keypoints based on the facial keypoint features to obtain the predicted coordinates of the keypoints corresponding to the sample faces in the sample training images; and the second feature prediction network predicts the emotion based on the fused emotion features to obtain the predicted emotion category of the sample faces in the sample training images.

[0120] S306, based on the key point coordinate prediction results and key point coordinate labels, calculates the key point prediction loss value through a preset key point loss function;

[0121] S308, based on the sentiment category prediction results and sentiment category labels, calculates the sentiment prediction loss value through a preset sentiment loss function;

[0122] Optionally, the key point prediction loss value and the sentiment prediction loss value are weighted and summed to obtain a weighted loss value, and the model parameters of the face detection model are updated based on the weighted loss value.

[0123] S310, update the model parameters of the face detection model based on the key point prediction loss value and the sentiment prediction loss value;

[0124] Optionally, the key point prediction loss value and the sentiment prediction loss value are weighted and summed to obtain a weighted loss value, and the model parameters of the face detection model are updated based on the weighted loss value.

[0125] S312, determine whether the face detection model with updated parameters meets the preset convergence condition. If it does, stop training and obtain the trained face detection model. If it does not meet the condition, proceed to step S304.

[0126] In the embodiments of this specification, a face detection model is trained by constructing a training image dataset and training the face detection model based on each sample training image in the training image dataset. Finally, a face detection model that meets the preset convergence condition is obtained. Based on the trained face detection model, the face key point detection and emotion recognition of the target face in the face image can be performed accurately, and the face key point detection results and emotion categories can be generated.

[0127] Please see Figure 9 This is a schematic diagram of the structure of a face recognition interaction device provided in an embodiment of this specification. Figure 9As shown, the face recognition interactive device 1 can be implemented as all or part of an electronic device through software, hardware, or a combination of both. According to some embodiments, the face recognition interactive device 1 includes an image acquisition module 11, an image processing module 12, a component determination module 13, and an interactive display module 14, specifically including:

[0128] Image acquisition module 11 is used to acquire a face image containing the target face;

[0129] Image processing module 12 is used to perform facial key point detection processing on the target face in the face image to obtain the facial key point detection result corresponding to the target face, and to perform emotion recognition processing on the target face to obtain the emotion category corresponding to the target face.

[0130] The component determination module 13 is used to determine the face position of the target face in the face image based on the face key point detection results, and to obtain the internal components of the virtual face corresponding to the emotion category;

[0131] The interactive display module 14 is used to display the internal components of the virtual face, the external components of the virtual face, and the interactive text corresponding to the face position on the display interface based on the face position.

[0132] Optional, please see Figure 10 This is a schematic diagram of the structure of a face recognition interaction device provided in an embodiment of this specification. Figure 10 As shown, the face recognition interaction device further includes an image cropping module 15, used for:

[0133] The face image is subjected to face detection processing to obtain a face bounding box containing the target face;

[0134] The face image is cropped based on the face bounding box to obtain the cropped face image.

[0135] Optionally, the facial landmark detection results include a preset number of facial landmark coordinates, and the component determination module 13 is specifically used for:

[0136] Calculate the first average value of each horizontal coordinate in the coordinates of each of the aforementioned facial key points, and calculate the second average value of each vertical coordinate in the coordinates of each of the aforementioned facial key points;

[0137] Using the first average value as the horizontal axis and the second average value as the vertical axis, a face position coordinate representing the face position is generated.

[0138] Optionally, the image processing module 12 is specifically used for:

[0139] The face image is input into a pre-trained face detection model to obtain the facial landmark detection results and emotion category corresponding to the target face in the face image.

[0140] Optionally, the interactive display module 14 is specifically used for:

[0141] The relative positions of the internal components and the external components of the virtual face are determined based on the face position.

[0142] The virtual face is obtained by combining the internal components and the external components of the virtual face based on the relative positions.

[0143] The virtual face and the corresponding interactive text at the face's location are displayed on the display interface.

[0144] In the embodiments of this specification, facial landmark detection and emotion recognition are performed on a face image to obtain the facial landmark detection results and emotion category corresponding to the face image. Then, the face position is determined based on the facial landmark detection results, and the internal components of the virtual face are determined based on the emotion category. Finally, based on the face position, the internal components of the virtual face, the external components of the virtual face, and the interactive text corresponding to the face position are displayed on the display interface. The virtual face interacts with the user on the display interface of the face recognition device, ensuring the user's privacy and security. Furthermore, by determining the internal components of the virtual face based on the emotion category, intelligent interaction with the user based on the virtual face can be realized on the face recognition device, further enhancing the user experience.

[0145] It should be noted that the face recognition interaction device provided in the above embodiments is only illustrated by the division of the above functional modules when executing the face recognition interaction method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the face recognition interaction device and the face recognition interaction method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0146] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0147] Please see Figure 11 This is a schematic diagram of the structure of a face detection model training device provided in an embodiment of this specification. Figure 11As shown, the face detection model training device 2 can be implemented as all or part of an electronic device through software, hardware, or a combination of both. According to some embodiments, the face detection model training device 2 includes a training data acquisition module 21, a face detection module 22, a first loss calculation module 23, a second loss calculation module 24, a model parameter update module 25, and a model convergence judgment module 26, specifically including:

[0148] Training data acquisition module 21 is used to establish a training image dataset, which includes each sample training image and the key point coordinate labels and sentiment category labels corresponding to each sample training image.

[0149] The face detection module 22 is used to input the sample training image into the face detection model to obtain the key point coordinate prediction result and emotion category prediction result output by the face detection model for the sample training image;

[0150] The first loss calculation module 23 is used to calculate the key point prediction loss value based on the key point coordinate prediction result and the key point coordinate label by a preset key point loss function.

[0151] The second loss calculation module 24 is used to calculate the emotion prediction loss value based on the emotion category prediction result and the emotion category label using a preset emotion loss function.

[0152] The model parameter update module 25 is used to update the model parameters of the face detection model based on the key point prediction loss value and the emotion prediction loss value;

[0153] The model convergence judgment module 26 is used to determine whether the face detection model with updated parameters meets the preset convergence condition. If it meets the condition, training is stopped and a trained face detection model is obtained. If it does not meet the condition, the step of inputting the sample training image into the face detection model is executed.

[0154] Optionally, the face detection module 22 is specifically used for:

[0155] The face key point features in the sample training image are extracted based on the first feature extraction network in the face detection model;

[0156] The facial expression features in the sample training images are extracted based on the second feature extraction network in the face detection model.

[0157] Based on the first feature prediction network in the face detection model, key point prediction is performed on the facial key point features to obtain the key point coordinate prediction result corresponding to the sample training image;

[0158] Based on the second feature prediction network in the face detection model, emotion prediction is performed on the facial expression features to obtain the emotion category prediction result corresponding to the sample training image.

[0159] Optionally, the face detection module 22 is further configured to:

[0160] Based on the feature fusion network in the face detection model, the facial key point features and the facial expression features are fused to obtain emotion fusion features;

[0161] Based on the second feature prediction network in the face detection model, emotion prediction is performed on the emotion fusion features to obtain the emotion category prediction result corresponding to the sample training image.

[0162] Optionally, the model parameter update module 25 is specifically used for:

[0163] The weighted loss value is obtained by weighting and summing the key point prediction loss value and the sentiment prediction loss value.

[0164] The model parameters of the face detection model are updated based on the weighted loss value.

[0165] In the embodiments of this specification, a face detection model is trained by constructing a training image dataset and training the face detection model based on each sample training image in the training image dataset. Finally, a face detection model that meets the preset convergence condition is obtained. Based on the trained face detection model, the face key point detection and emotion recognition of the target face in the face image can be performed accurately, and the face key point detection results and emotion categories can be generated.

[0166] It should be noted that the face detection model training device provided in the above embodiments is only illustrated by the division of the above functional modules when executing the face detection model training method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the face detection model training device and the face detection model training method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0167] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0168] This specification also provides an embodiment of a computer storage medium that can store multiple instructions adapted to be loaded and executed by a processor as described above. Figures 1 to 8 The face recognition interaction method described in the illustrated embodiment can be found in the following document for a detailed execution process. Figures 1 to 8The specific details of the illustrated embodiments will not be elaborated here.

[0169] This specification also provides a computer program product that stores at least one instruction, said at least one instruction being loaded and executed by the processor as described above. Figures 1 to 8 The face recognition interaction method described in the illustrated embodiment can be found in the following document for a detailed execution process. Figures 1 to 8 The specific details of the illustrated embodiments will not be elaborated here.

[0170] Please refer to Figure 12 This is a structural block diagram of an electronic device provided in an embodiment of this specification. The electronic device in this specification may include one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, memory 120, input device 130, and output device 140 can be connected via the bus 150.

[0171] Processor 110 may include one or more processing cores. Processor 110 connects to various parts of the terminal using various interfaces and lines, and performs various functions and processes data of terminal 100 by running or executing instructions, programs, code sets, or instruction sets stored in memory 120, and by calling data stored in memory 120. Optionally, processor 110 may be implemented using at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). Processor 110 may integrate one or more of the following: central processing unit (CPU), graphics processing unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 110 and may be implemented separately using a communication chip.

[0172] The memory 120 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 120 may include non-transitory computer-readable storage medium. The memory 120 may be used to store instructions, programs, code, code sets, or instruction sets.

[0173] The input device 130 is used to receive input instructions or data, and includes, but is not limited to, a keyboard, mouse, camera, microphone, or touch device. The output device 140 is used to output instructions or data, and includes, but is not limited to, a display device and a speaker. In this embodiment, the input device 130 can be a temperature sensor to obtain the operating temperature of the terminal. The output device 140 can be a speaker to output audio signals.

[0174] In addition, those skilled in the art will understand that the structure of the terminal shown in the above figures does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the terminal may also include radio frequency circuits, input units, sensors, audio circuits, wireless fidelity (WIFI) modules, power supplies, Bluetooth modules, etc., which will not be described in detail here.

[0175] In the embodiments of this specification, the executing entity for each step can be the terminal described above. Optionally, the executing entity for each step is the terminal's operating system. The operating system can be Android, iOS, or other operating systems; this specification does not limit this.

[0176] exist Figure 12 In the electronic device, the processor 110 can be used to call the face recognition interaction program stored in the memory 120 and execute it to implement the face recognition interaction method as described in the various method embodiments of this specification.

[0177] In the embodiments of this specification, facial landmark detection and emotion recognition are performed on a face image to obtain the facial landmark detection results and emotion category corresponding to the face image. Then, the face position is determined based on the facial landmark detection results, and the internal components of the virtual face are determined based on the emotion category. Finally, based on the face position, the internal components of the virtual face, the external components of the virtual face, and the interactive text corresponding to the face position are displayed on the display interface. The virtual face interacts with the user on the display interface of the face recognition device, ensuring the user's privacy and security. Furthermore, by determining the internal components of the virtual face based on the emotion category, intelligent interaction with the user based on the virtual face can be realized on the face recognition device, further enhancing the user experience.

[0178] Those skilled in the art will clearly understand that the technical solutions in this specification can be implemented using software and / or hardware. In this specification, "unit" and "module" refer to software and / or hardware capable of independently performing or cooperating with other components to perform a specific function. Hardware may include, for example, a Field-Programmable Gate Array (FPGA), an Integrated Circuit (IC), etc.

[0179] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this specification is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this specification. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this specification.

[0180] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0181] In the several embodiments provided in this specification, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0182] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0183] Furthermore, the functional units in the various embodiments of this specification can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0184] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0185] The foregoing descriptions are merely exemplary embodiments of this specification and should not be construed as limiting the scope of this specification. Any equivalent changes and modifications made in accordance with the teachings of this specification shall still fall within the scope of this specification. Other embodiments of this specification will be readily apparent to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This specification is intended to cover any variations, uses, or adaptations that follow the general principles of this specification and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this specification are defined by the claims.

Claims

1. A face recognition interaction method, the method comprising: Obtain a face image containing the target person's face; Facial landmark detection is performed on the target face in the face image to obtain the facial landmark detection result corresponding to the target face, and emotion recognition is performed on the target face to obtain the emotion category corresponding to the target face. The facial landmark detection result includes a preset number of facial landmark coordinates. Based on the facial landmark detection results, the face position of the target face in the face image is determined, and the internal components of the virtual face corresponding to the emotion category are obtained. The determination of the face position of the target face in the face image includes: calculating a first average value of the horizontal coordinates of each facial landmark in the facial landmark detection results, and calculating a second average value of each vertical coordinate; using the first average value as the horizontal coordinate and the second average value as the vertical coordinate to generate face position coordinates representing the face position. Based on the face location, the internal components of the virtual face, the external components of the virtual face, and the interactive text corresponding to the face location are displayed on the display interface; The step of displaying the internal components of the virtual face, the external components of the virtual face, and the interactive text corresponding to the face position on the display interface based on the face position includes: Construct the inscribed rectangle of the outer component of the virtual face; Based on the face position, the relative position of the inner component of the virtual face within the inscribed rectangle of the outer component of the virtual face is determined proportionally. The virtual face internal components and the virtual face external components are combined based on the relative position to obtain a virtual face, and the virtual face interacts with the target face in a state of facing the target face. The virtual face and the corresponding interactive text at the face's location are displayed on the display interface.

2. The method according to claim 1, before performing facial landmark detection processing on the target face in the face image to obtain the facial landmark detection result corresponding to the target face, and performing emotion recognition processing on the target face to obtain the emotion category corresponding to the target face, further comprising: Perform face detection processing on the face image to obtain a face bounding box containing the target face; The face image is cropped based on the face bounding box to obtain the cropped face image.

3. The method according to claim 1, wherein the virtual face external component includes a male virtual face external component and a female virtual face external component; the method further includes: Gender identification is performed on the target face in the face image; When identified as female, a virtual face is generated based on the female virtual face external component and the virtual face internal component; when identified as male, a virtual face is generated based on the male virtual face external component and the virtual face internal component.

4. The method according to claim 1, wherein performing facial landmark detection processing on the target face in the face image to obtain the facial landmark detection result corresponding to the target face, and performing emotion recognition processing on the target face to obtain the emotion category corresponding to the target face, includes: The face image is input into a pre-trained face detection model to obtain the facial landmark detection results and emotion category corresponding to the target face in the face image.

5. A method for training a face detection model, comprising: Establish a training image dataset, which includes each sample training image and the key point coordinate labels and sentiment category labels corresponding to each sample training image; The sample training image is input into the face detection model to obtain the key point coordinate prediction result and emotion category prediction result output by the face detection model for the sample training image; Based on the key point coordinate prediction results and the key point coordinate labels, the key point prediction loss value is calculated using a preset key point loss function. Based on the emotion category prediction result and the emotion category label, the emotion prediction loss value is calculated using a preset emotion loss function; The model parameters of the face detection model are updated based on the key point prediction loss value and the emotion prediction loss value. Determine whether the face detection model with updated parameters meets the preset convergence condition. If it does, stop training and obtain the trained face detection model. If it does not meet the condition, execute the step of inputting the sample training image into the face detection model. The trained face detection model is used to perform facial landmark detection and emotion recognition on the target face in the face image, and obtain the facial landmark detection result and emotion category corresponding to the target face. The facial landmark detection result includes a preset number of facial landmark coordinates. The facial landmark detection results are used to calculate the first average value of each horizontal coordinate and the second average value of each vertical coordinate in the facial landmark detection results. The first average value is used as the horizontal coordinate and the second average value is used as the vertical coordinate to generate facial position coordinates representing the facial position. The emotion category is used to determine the internal components of the virtual face. The facial position is used to construct the inscribed rectangle of the external components of the virtual face and determine the relative position of the internal components of the virtual face within the inscribed rectangle of the external components of the virtual face in a proportional manner. Based on the relative position, the internal components of the virtual face and the external components of the virtual face are combined to obtain a virtual face, and the virtual face interacts with the target face. The virtual face and the interactive text corresponding to the facial position are displayed together in the display interface.

6. The method according to claim 5, wherein inputting the sample training image into a face detection model to obtain the key point coordinate prediction result and emotion category prediction result output by the face detection model for the sample training image includes: The face key point features in the sample training image are extracted based on the first feature extraction network in the face detection model; The facial expression features in the sample training images are extracted based on the second feature extraction network in the face detection model. Based on the first feature prediction network in the face detection model, key point prediction is performed on the facial key point features to obtain the key point coordinate prediction result corresponding to the sample training image; Based on the second feature prediction network in the face detection model, emotion prediction is performed on the facial expression features to obtain the emotion category prediction result corresponding to the sample training image.

7. The method according to claim 6, further comprising: Based on the feature fusion network in the face detection model, the facial key point features and the facial expression features are fused to obtain emotion fusion features; The second feature prediction network in the face detection model performs emotion prediction on the facial expression features to obtain the emotion category prediction result corresponding to the sample training image, including: Based on the second feature prediction network in the face detection model, emotion prediction is performed on the emotion fusion features to obtain the emotion category prediction result corresponding to the sample training image.

8. The method according to claim 5, wherein updating the model parameters of the face detection model based on the key point prediction loss value and the emotion prediction loss value comprises: The weighted sum of the key point prediction loss value and the sentiment prediction loss value is obtained by weighting the sum of the two values. The model parameters of the face detection model are updated based on the weighted loss value.

9. A facial recognition interactive device, comprising: The image acquisition module is used to acquire a face image containing the target face; The image processing module is used to perform facial key point detection processing on the target face in the face image to obtain the facial key point detection result corresponding to the target face, and to perform emotion recognition processing on the target face to obtain the emotion category corresponding to the target face. The facial key point detection result includes a preset number of facial key point coordinates. A component determination module is used to determine the face position of the target face in the face image based on the face key point detection results, and to obtain the virtual face internal components corresponding to the emotion category. The determination of the face position of the target face in the face image includes: calculating a first average value of the horizontal coordinates of each face key point in the face key point detection results, and calculating a second average value of each vertical coordinate; using the first average value as the horizontal coordinate and the second average value as the vertical coordinate to generate face position coordinates representing the face position. An interactive display module is used to display the internal components of the virtual face, the external components of the virtual face, and the interactive text corresponding to the face position on the display interface based on the face position; The interactive display module is specifically used to form the inscribed rectangle of the virtual face outer component. Based on the face position, it determines the relative position of the virtual face inner component and the virtual face outer component in a proportional manner. The relative position is the relative position of the virtual face inner component in the inscribed rectangle of the virtual face outer component. The virtual face internal components and the virtual face external components are combined based on the relative position to obtain a virtual face, and the virtual face interacts with the target face in a state of facing the target face. The virtual face and the corresponding interactive text at the face's location are displayed on the display interface.

10. A face detection model training device, comprising: The training data acquisition module is used to establish a training image dataset, which includes each sample training image and the key point coordinate labels and sentiment category labels corresponding to each sample training image. The face detection module is used to input the sample training image into the face detection model to obtain the key point coordinate prediction result and emotion category prediction result output by the face detection model for the sample training image; The first loss calculation module is used to calculate the key point prediction loss value based on the key point coordinate prediction result and the key point coordinate label, using a preset key point loss function. The second loss calculation module is used to calculate the emotion prediction loss value based on the emotion category prediction result and the emotion category label using a preset emotion loss function. The model parameter update module is used to update the model parameters of the face detection model based on the key point prediction loss value and the emotion prediction loss value. The model convergence judgment module is used to determine whether the face detection model with updated parameters meets the preset convergence condition. If it meets the condition, training is stopped and a trained face detection model is obtained. If it does not meet the condition, the step of inputting the sample training image into the face detection model is executed. The trained face detection model is used to perform facial landmark detection and emotion recognition on the target face in the face image, and obtain the facial landmark detection result and emotion category corresponding to the target face. The facial landmark detection result includes a preset number of facial landmark coordinates. The facial landmark detection results are used to calculate the first average value of each horizontal coordinate and the second average value of each vertical coordinate in the facial landmark detection results. The first average value is used as the horizontal coordinate and the second average value is used as the vertical coordinate to generate facial position coordinates representing the facial position. The emotion category is used to determine the internal components of the virtual face. The facial position is used to construct the inscribed rectangle of the external components of the virtual face and determine the relative position of the internal components of the virtual face within the inscribed rectangle of the external components of the virtual face in a proportional manner. Based on the relative position, the internal components of the virtual face and the external components of the virtual face are combined to obtain a virtual face, and the virtual face interacts with the target face. The virtual face and the interactive text corresponding to the facial position are displayed together in the display interface.

11. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 4 or 5 to 8.

12. An electronic device, characterized in that, include: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the steps of the method as claimed in any one of claims 1 to 4 or 5 to 8.

13. A computer program product having at least one instruction stored thereon, characterized in that, When the at least one instruction is executed by the processor, it implements the steps of the method according to any one of claims 1 to 4 or 5 to 8.

Images