Methods, devices, equipment and storage media for acquiring facial feature points

Abstract

The application provides a face feature point acquisition method and device, equipment and storage medium, and relates to the technical field of face recognition. The method comprises the following steps: acquiring a current frame image from a first video; when it is detected that the current frame image comprises a face image, intercepting the face image from the current frame image; acquiring face images in a preset number of historical frame images from the first video, and a face feature point set corresponding to each of the face images in the preset number of historical frame images; inputting the face image in the current frame image, the face images in the preset number of historical frame images, and the face feature point set corresponding to each of the face images in the preset number of historical frame images into a face feature point prediction model, and determining a face feature point set corresponding to the face image in the current frame image. The method is suitable for the detection process of face feature points, and is used for solving the problem of face feature point jitter.

Description

Technical Field

[0001] This application relates to the field of facial recognition technology, and in particular to a method, apparatus, device and storage medium for acquiring facial feature points. Background Technology

[0002] Facial landmarks (also known as facial key points) refer to points, corners, and dividing points in a facial image that possess clear semantic features. Examples include features such as eyes, eyebrows, nose, mouth, and facial contours. Typically, when analyzing facial images, facial landmarks can be predicted (also known as localization or detection).

[0003] However, the prediction results of facial feature points are often affected by noise and exhibit irregular jitter, resulting in low stability in facial feature point prediction. Therefore, when applying jittery facial feature points to facial image analysis in fields such as video surveillance and in-vehicle driver analysis, misjudgments may occur. Furthermore, applying jittery facial feature points to entertainment video applications can negatively impact the user experience. Summary of the Invention

[0004] Based on the above-mentioned technical problems, this application provides a method, apparatus, device and storage medium for obtaining facial feature points. When continuously predicting multiple consecutive frames of facial images to obtain a set of multiple facial feature points of a person, the set of multiple facial feature points appears more stable to the human eye and is less prone to jitter.

[0005] In a first aspect, this application provides a method for obtaining facial feature points, the method comprising: obtaining a current frame image from a first video; when detecting that the current frame image includes a facial image, cropping a facial image from the current frame image; obtaining facial images from a preset number of historical frame images from the first video, and a set of facial feature points corresponding to the facial images in the preset number of historical frame images respectively; inputting the set of facial feature points corresponding to the facial images in the current frame image, the set of facial images in the preset number of historical frame images, and the set of facial feature points corresponding to the facial images in the preset number of historical frame images respectively into a facial feature point prediction model to determine the set of facial feature points corresponding to the facial image in the current frame image.

[0006] Optionally, when it is detected that the current frame image includes a face image, the face image is extracted from the current frame image, including: determining the face sequence frame number of the current frame image; the face sequence refers to a sequence of multiple consecutive frames in the first video that include face images; and extracting the face image from the current frame image according to the face sequence frame number.

[0007] In one possible implementation, extracting a face image from the current frame image based on the face sequence frame number includes: if the face sequence frame number of the current frame image is 1, then detecting the current frame image according to a preset face detector to obtain the face bounding box output by the face detector; and extracting a face image from the current frame image using the face bounding box output by the face detector.

[0008] In another possible implementation, the face image is extracted from the current frame image based on the face sequence frame number, including: if the face sequence frame number of the current frame image is greater than 1, then a face bounding box is generated using the set of face feature points corresponding to the face images in the previous images, and the face image is extracted from the current frame image using the generated face bounding box.

[0009] Optionally, when the current frame image is detected to include a face image, the face image is extracted from the current frame image, including: detecting the current frame image according to a preset face detector to obtain a face bounding box output by the face detector; and extracting the face image from the current frame image using the face bounding box output by the face detector.

[0010] Optionally, before determining the set of facial feature points corresponding to the face image in the current frame image based on the face image in the current frame image, the face images in a preset number of historical frames, the set of facial feature points corresponding to the face images in the preset number of historical frames, and the facial feature point prediction model, the method further includes: obtaining a training sample set; the training sample set includes multiple training samples; each training sample includes the set of facial feature points corresponding to N frames of face images, the set of facial feature points corresponding to the N-1 frames of face images preceding the Nth frame of the N frames of face images, and a label; the label is the set of facial feature points corresponding to the Nth frame of the N frames of face images; and training a preset neural network based on the training sample set to obtain a facial feature point prediction model.

[0011] Optionally, obtaining a training sample set includes: obtaining a second video, which includes multiple frames of face images; for the first frame of a face image in the second video, copying the first frame of the face image N-1 times to obtain N-1 reference face images; detecting the N-1 reference face images using a face detection model to obtain N-1 sets of reference face feature points; performing a weighted average on the N-1 sets of reference face feature points to obtain an average set of face feature points as the set of face feature points corresponding to each frame of the N-1 reference face images; using the average set of face feature points as a first label, and obtaining training samples based on the first frame of the face image, the N-1 reference face images, the set of face feature points corresponding to the N-1 reference face images, and the first label; for the Xth frame of the face image in the second video, obtaining the set of face feature points corresponding to the Xth frame of the face image. Combine; use the set of facial feature points corresponding to the Xth frame face image as the second label, and obtain training samples based on the Xth frame face image, the X-1 frames face images preceding the Xth frame face image in the second video, the NX frame reference face image, the X-1 sets of facial feature points corresponding to the X-1 frames face image, the sets of facial feature points corresponding to the NX frame reference face image, and the second label; X is an integer greater than 1 and less than N; for the Yth frame face image in the second video, obtain the set of facial feature points corresponding to the Yth frame face image; use the set of facial feature points corresponding to the Yth frame face image as the third label, and obtain training samples based on the Yth frame face image, the N-1 frames face images preceding the Yth frame face image, the N-1 sets of facial feature points corresponding to the N-1 frames face images preceding the Yth frame face image, and the third label; Y is an integer greater than N.

[0012] Using the facial feature point acquisition method provided in this application, facial images from the current frame and a preset number of historical frames can be obtained from a first video, and sets of facial feature points corresponding to the facial images in the preset number of historical frames can be obtained respectively. These sets of facial feature points corresponding to the facial images in the historical frames may contain detection errors, and multiple detection errors may appear jittery when viewed continuously by the human eye. By inputting the facial images from the current frame, the preset number of historical frames, and the sets of facial feature points corresponding to the facial images in the preset number of historical frames into a facial feature point prediction model, the model can fully learn the characteristics of the detection errors of the sets of facial feature points corresponding to the facial images in the preset number of historical frames. The sets of facial feature points corresponding to multiple frames of facial images in the first video continuously predicted by this model all have the same or similar characteristics of the detection errors. Multiple sets of facial feature points with the same or similar characteristics are less prone to jitter.

[0013] Secondly, this application provides a facial feature point acquisition device, which includes various modules for the method described in the first aspect or any possible implementation of the first aspect.

[0014] Thirdly, this application provides a computer program product that, when run on a computer, causes the computer to perform the steps of the related method described in the first aspect, so as to implement the method described in the first aspect.

[0015] Fourthly, this application provides an electronic device comprising: a processor and a memory; the memory storing processor-executable instructions; when the processor is configured to execute the instructions, causing the electronic device to perform the method described in the first aspect above.

[0016] Fifthly, this application provides a computer-readable storage medium comprising: computer software instructions; when the computer software instructions are executed in an electronic device, they cause the electronic device to perform the method described in the first aspect above.

[0017] The beneficial effects of the second to fifth aspects mentioned above can be referred to the first aspect, and will not be repeated here. Attached Figure Description

[0018] Figure 1 A schematic diagram of facial feature points;

[0019] Figure 2 A diagram illustrating facial feature point jitter;

[0020] Figure 3 This is a schematic diagram illustrating the composition of the facial feature point acquisition system provided in the embodiments of this application;

[0021] Figure 4 A schematic diagram illustrating the composition of the first electronic device provided in an embodiment of this application;

[0022] Figure 5 A flowchart illustrating the method for obtaining facial feature points provided in this application embodiment;

[0023] Figure 6 A schematic diagram of a face frame provided for an embodiment of this application;

[0024] Figure 7 A schematic diagram of a face sequence provided in an embodiment of this application;

[0025] Figure 8 A schematic diagram illustrating a face sequence frame number of a preset number of historical frame images corresponding to the current frame image, provided in an embodiment of this application;

[0026] Figure 9This is another flowchart illustrating the method for obtaining facial feature points provided in the embodiments of this application;

[0027] Figure 10 This is another flowchart illustrating the method for obtaining facial feature points provided in the embodiments of this application;

[0028] Figure 11 A flowchart illustrating a training method for a facial feature point prediction model provided in an embodiment of this application;

[0029] Figure 12 This is a schematic diagram of face frame position perturbation provided in an embodiment of this application;

[0030] Figure 13 This is a schematic diagram of the composition of the facial feature point acquisition device provided in the embodiments of this application. Detailed Implementation

[0031] Hereinafter, the terms "first," "second," and "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first," "second," or "third," etc., may explicitly or implicitly include one or more of that feature.

[0032] Facial landmarks (also known as facial key points) refer to points, corners, and dividing points in a face image that have clear semantic features. Figure 1 This is a diagram illustrating facial feature points. (Example:) Figure 1 As shown, facial feature points can include facial feature points such as eyebrows, eyes, the root of the nose, the tip of the nose, ears, mouth, jaw, and chin. Figure 1 The example shown uses 32 facial feature points at the intersection of the black lines. In actual scenarios, the number of facial feature points can be greater than, equal to or less than 32, and this application embodiment does not impose any restrictions on this.

[0033] Alternatively, when analyzing facial images, a facial landmark detection model (also known as a facial landmark localization model) can be used to predict facial landmarks. However, since the predicted results of facial landmarks are often affected by noise and exhibit irregular fluctuations, the stability of obtaining facial landmarks through the facial landmark detection model is relatively low.

[0034] Facial landmark prediction technology can be applied to driver analytics in vehicles. For example, in-vehicle cameras can capture driver monitoring video, and the onboard computer can predict facial landmarks from this video to estimate the driver's gaze. If the driver's gaze deviates from the road path, a warning can be issued. However, if facial landmarks are jittery, the gaze estimation may misjudge the situation, leading to incorrect warnings.

[0035] The technology for predicting facial feature points can also be applied to the analysis of entertainment video applications. For example, virtual plugins on facial images in live videos can be added to different positions in the facial image, using facial feature points as reference points. When there is jitter in the facial feature points, the virtual plugin will also jitter even when the face is still, affecting the user experience.

[0036] For example, Figure 2 This is a diagram illustrating the shaking of facial feature points. (Example:) Figure 2 As shown, taking feature point 8 at the corner of the left eye as an example, assuming that in the previous frame of the face image detected using the face feature point detection model, this feature point 8 is similar to feature point 8 in the ideal state (i.e., ... Figure 1 In the first frame, feature point 8 is shifted to the left compared to the ideal feature point 8; in the second frame, after detecting a face using the face feature point detection model, feature point 8 is shifted to the right compared to the ideal feature point 8. Since the shift directions of feature point 8 are different in these two frames, the shift error between these two frames will appear as temporal jitter to the human eye.

[0037] Based on this, embodiments of this application provide a method for obtaining facial feature points, which can be used to obtain the aforementioned facial feature points. Figure 2 The offset error shown is also fed into the face feature point prediction model that has learned the offset error, so as to detect stable face feature points.

[0038] Figure 3 This is a schematic diagram illustrating the composition of a facial feature point acquisition system provided in an embodiment of this application. Figure 3 As shown, the facial feature point acquisition system 100 may include an image acquisition device 10 and a computing processing device 20. The image acquisition device 10 and the computing processing device 20 may be connected via a wired network or a wireless network.

[0039] Image acquisition device 10 can be a mobile phone camera, laptop camera, tablet camera, vehicle camera, or... Figure 3 The image acquisition device 10 shown in this application does not limit the specific form of the image acquisition device 10.

[0040] The computing processing device 20 can be a device with computing processing capabilities, such as a mobile phone, laptop, tablet computer, in-vehicle computer, general-purpose computer, or server (e.g., the device can be referred to as the first electronic device), or a functional module within the first electronic device; there is no limitation on this. The server can be a single server or a server cluster composed of multiple servers. In some embodiments, the server cluster can also be a distributed cluster. Optionally, the server can also be implemented on a cloud platform. For example, the cloud platform can include private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, inter-cloud, and multi-cloud, or any combination thereof. This application does not limit the specific form of the computing processing device 20.

[0041] The image acquisition device 10 can be integrated with the computing processing device 20 in a single electronic device (e.g., a first electronic device equipped with a camera). Alternatively, the image acquisition device 10 can be separated from the computing processing device 20 and housed in different electronic devices. This application embodiment does not impose any limitations on this.

[0042] The image acquisition device 10 can acquire images or videos of the area (also known as the target area) within the field of view of the device and send them to the computing processing device 20 via a wired or wireless network.

[0043] The computing processing device 20 can receive images or videos sent by the image acquisition device 10 and predict the set of facial feature points in the image frames of the image or video. The specific prediction process can be referred to in S101 to S103 below, and will not be repeated here.

[0044] It should be noted that the facial feature point acquisition method provided in this embodiment is executed by a facial feature point acquisition device. Optionally, the function implemented by this device can be achieved through the above-described... Figure 3 The facial feature point acquisition system 100 shown is implemented; or, the device can be the one described above. Figure 3 The facial feature point acquisition system 100 shown includes a computing processing device 20 (e.g., a first electronic device); alternatively, the device may be an application (APP) with facial feature point acquisition function in the first electronic device; alternatively, the device may be a central processing unit (CPU) in the first electronic device; or alternatively, the device may be a functional module in the first electronic device for executing the facial feature point acquisition method. This application embodiment does not impose any limitations on these aspects.

[0045] The following description will use a facial feature point acquisition device as the first electronic device.

[0046] Figure 4 This is a schematic diagram illustrating the composition of the first electronic device provided in an embodiment of this application. (As shown...) Figure 4 As shown, the first electronic device may include a processor 21, a memory 22, a communication line 23, and a communication interface 24.

[0047] The processor 21, memory 22 and communication interface 24 can be connected via communication line 23.

[0048] Processor 21 is used to execute instructions stored in memory 22 to implement the facial feature point acquisition method provided in the following embodiments of this application. Processor 21 may be a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), a microprocessor, a microcontroller, a programmable logic device (PLD), or any combination thereof. Processor 21 may also be any other device with processing capabilities, such as a circuit, device, or software module; this application embodiment does not limit this. In one example, processor 21 may include one or more CPUs, for example... Figure 4 CPU0 and CPU1 are mentioned. As an optional implementation, the first electronic device may include multiple processors, for example, in addition to processor 21, it may also include processor 25.

[0049] The memory 22 is used to store instructions. These instructions can be computer programs. The memory 22 can be a read-only memory (ROM) or other type of static storage device capable of storing static information and / or instructions; it can also be a random access memory (RAM) or other type of dynamic storage device capable of storing information and / or instructions; it can also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, etc. This application embodiment does not limit these possibilities.

[0050] It should be noted that the memory 22 can exist independently of the processor 21, or it can be integrated with the processor 21. The memory 22 can be located inside or outside the first electronic device, and this embodiment does not impose any restrictions on this.

[0051] Communication line 23 is used to transmit information between the components included in the first electronic device.

[0052] The communication interface 24 is used to communicate with other devices (such as the image acquisition device 10 described above) or other communication networks. These other communication networks can be Ethernet, radio access network (RAN), wireless local area network (WLAN), etc. The communication interface 24 can be a module, circuit, transceiver, or any device capable of enabling communication.

[0053] It should be noted that, Figure 4 The structure shown does not constitute a limitation on the first electronic device, except... Figure 4 In addition to the components shown, the first electronic device may include more or fewer components than those shown, or combine certain components, or have different component arrangements.

[0054] Figure 5 This is a flowchart illustrating a method for obtaining facial feature points provided in an embodiment of this application. Optionally, this method can be implemented by someone with... Figure 4 The first electronic device in the hardware structure shown executes the commands. For example... Figure 5 As shown, the method may include S101 to S104.

[0055] S101, The first electronic device obtains the current frame image from the first video.

[0056] The first video may be the aforementioned driver monitoring video or live video, etc., and this application embodiment does not limit this. The specific method for acquiring the first video can be referred to the relevant description in the image acquisition device 10 above, and will not be repeated here.

[0057] S102. When it is detected that the current frame image includes a face image, the first electronic device extracts the face image from the current frame image.

[0058] One possible implementation is that the first electronic device can detect the current frame image according to a preset face detector, obtain the face bounding box output by the face detector, and use the face bounding box output by the face detector to extract the face image from the current frame image.

[0059] For example, the preset face detector can be a BlazeFace face detector, a Viola-Jones face detector, or a practical facial landmark detector (PFLD), etc. This application does not limit the specific type of preset face detector.

[0060] For example, Figure 6 A schematic diagram of a face frame provided for an embodiment of this application. For example... Figure 6 As shown, with Figure 6 The image of the little person shown is an example of the current frame image mentioned above. A rectangular face bounding box can then be used. Figure 6 (Using a dashed rectangle as an example) Capture the face image in the current frame.

[0061] Another possible implementation, S102 above may include: the first electronic device determining the face sequence frame number of the current frame image, and extracting the face image from the current frame image according to the face sequence frame number.

[0062] The face sequence frame number is the sequence number of the current frame image within the face sequence. A face sequence refers to a sequence of multiple consecutive frames in the first video that include face images.

[0063] Specifically, when a face image is detected for the first time in a frame of the first video, that frame is designated as image number 1 in a face sequence (i.e., frame number 1 of the face sequence). Optionally, when the first electronic device begins to mark an image containing a face image as frame number j (j is an integer), that is, when the first electronic device begins to capture a face sequence, if it detects that subsequent images containing frame number j do not include face images, the first electronic device can determine that the face sequence has ended. Furthermore, when the first electronic device detects an image containing a face image again, it can capture another face sequence.

[0064] For example, Figure 7 This is a schematic diagram of a face sequence provided in an embodiment of this application. Figure 7 As shown, the first video includes multiple frames of images. The first video includes face images, and three consecutive frames of images constitute a face sequence. The frame numbers of the face sequence images in this face sequence can be 1, 2, and 3, respectively.

[0065] Optionally, if the face sequence frame number of the current frame image is 1, the first electronic device can detect the current frame image according to the preset face detector, obtain the face bounding box output by the face detector, and use the face bounding box output by the face detector to extract the face image from the current frame image.

[0066] The preset face detector can be implemented as described in one possible way in S102 above, and will not be repeated here. Optionally, if the face sequence frame number of the current frame image is greater than 1, the first electronic device can generate a face bounding box using the set of face feature points corresponding to the face images in the images preceding the current frame image, and use the generated face bounding box to extract the face image from the current frame image. The method for obtaining the set of face feature points corresponding to the face images in the images preceding the current frame image can be described in S103 below, and will not be repeated here.

[0067] It should be noted that each frame in a face sequence may include a face image of one person or multiple people. When multiple people's face images are included, the first electronic device can predict the face images of each person according to the face feature point prediction model provided in the embodiments of this application, and obtain multiple sets of face feature points corresponding to each of the multiple people's face images. The following description will uniformly use a single frame containing a face image of one person as an example.

[0068] For example, if the face images in the current frame include the face images of person 1 and person 2, the first electronic device can respectively capture the face images of person 1 and person 2 and use the face feature point prediction model to predict the set of face feature points corresponding to the face image of person 1 and the set of face feature points corresponding to the face image of person 2 in the current frame.

[0069] S103. The first electronic device obtains face images from a preset number of historical frame images in the first video, and sets of face feature points corresponding to the face images in the preset number of historical frame images respectively.

[0070] For ease of description, this application embodiment will be described below using a preset quantity of M-1 as an example. In this case, the first electronic device acquires face images from a preset quantity of historical frame images, and the set of facial feature points corresponding to each face image in these preset quantity of historical frame images is: the first electronic device acquires face images from M-1 frames preceding the current frame image, and the set of M-1 facial feature points corresponding to each face image in these M-1 frames. Here, M is an integer greater than or equal to 2. For example, M can be 2, 5, 8, or 10, etc. This application embodiment does not limit the specific value of M. The method for acquiring historical frame face images can refer to the method for acquiring face images in S102 above, and will not be repeated here. The set of facial feature points corresponding to historical frame face images can refer to the process for acquiring the set of facial feature points corresponding to face images in S104 below, and will not be repeated here.

[0071] Optionally, the first electronic device may include a buffer area, which may cache face images from images preceding the current frame and sets of facial feature points corresponding to the face images from images preceding the current frame. The first electronic device acquiring face images from M-1 frames preceding the current frame, and M-1 sets of facial feature points corresponding to the face images in those M-1 frames, may include: the first electronic device retrieving the face images from the M-1 frames preceding the current frame cached in the buffer, and the M-1 sets of facial feature points corresponding to each face image in those M-1 frames.

[0072] For example, taking the above-mentioned face sequence as an example, the first electronic device retrieves the face images from the M-1 frames preceding the current frame image cached in the buffer, as well as the M-1 sets of face feature points corresponding to each face image in the M-1 frames. This can be divided into three cases:

[0073] Scenario 1: The face sequence frame number of the current frame image is 1, meaning there are no face images preceding the face image in the current frame image. In this case, the face images in the M-1 frames preceding the current frame image can be M-1 virtual reference face images obtained by the first electronic device copying the face image in the current frame image M-1 times. The M-1 sets of facial feature points corresponding to the face images in the M-1 frames preceding the current frame image can be the same average set of facial feature points. This average set of facial feature points is obtained by the first electronic device detecting the M-1 reference face images, obtaining M-1 sets of reference facial feature points, and then weighting and averaging the facial feature points in the M-1 sets of reference facial feature points. Specific detection methods can be found in the face detection model, and will not be elaborated here.

[0074] Scenario 2: The face sequence frame number of the current frame image is greater than 1 and less than M, meaning there are face images in the images preceding the current frame image, but the number of face images in the images preceding the current frame image is less than M-1. For example, if the face sequence frame number of the current frame image is K, and 1 < K < M, then the first electronic device can use the MK frame reference face image and the face images in the K-1 frames preceding the face image of the current frame image cached in the buffer as the face images in the M-1 frames preceding the current frame image. Based on the average set of facial feature points corresponding to the MK frame reference face image and the set of facial feature points corresponding to the face images in the K-1 frames preceding the current frame image cached in the buffer, the set of facial feature points corresponding to the M-1 historical frame face images is obtained.

[0075] Case 3: The face sequence frame number of the current frame image is greater than or equal to M, meaning the number of face images in the images preceding the current frame image is greater than M-1. In this case, the first electronic device can directly retrieve the face images from the M-1 frames preceding the current frame image cached in the buffer, and the M-1 sets of face feature points corresponding to the face images from the M-1 frames preceding the current frame image.

[0076] For example, Figure 8 This is a schematic diagram illustrating a preset number of historical frame images corresponding to the current frame image, as provided in an embodiment of this application. For example... Figure 8 As shown, a rectangle represents a face image, and the number inside the rectangle represents the frame number of the face sequence corresponding to the face image. For example, assuming M is 5, the first electronic device can display the first frame face image (e.g., Figure 8 The face image in the image with frame number 1 is copied 5-1=4 times to obtain 4 reference face images. The face sequence frame numbers of the images corresponding to these 4 reference face images are 0, -1, -2 and -3 respectively (because they are virtual reference face images, they are represented by frame numbers less than 1 here). In this scenario, when the current frame is the first frame of a face sequence, the face sequence frame numbers of the M-1 historical frame images and the current frame image are -3, -2, -1, 0, and 1, respectively; when the current frame is the second frame of a face sequence, the face sequence frame numbers of the M-1 historical frame images and the current frame image are -2, -1, 0, 1, and 2, respectively; when the current frame is the third frame of a face sequence, the face sequence frame numbers of the M-1 historical frame images and the current frame image are -1, 0, 1, 2, and 3, respectively; when the current frame is the fourth frame of a face sequence, the face sequence frame numbers of the M-1 historical frame images and the current frame image are 0, 1, 2, 3, and 4, respectively; when the current frame is the fifth frame of a face sequence, the face sequence frame numbers of the M-1 historical frame images and the current frame image are 1, 2, 3, 4, and 5, respectively; and when the current frame is the sixth frame of a face sequence, the face sequence frame numbers of the M-1 historical frame images and the current frame image are 2, 3, 4, 5, and 6, respectively.

[0077] S104. The first electronic device inputs the set of facial feature points corresponding to the face image in the current frame image, the face images in a preset number of historical frame images, and the set of facial feature points corresponding to the face images in a preset number of historical frame images into the face feature point prediction model to determine the set of facial feature points corresponding to the face image in the current frame image.

[0078] The facial feature point prediction model is pre-installed in the first electronic device. This model can predict the set of facial feature points corresponding to the face image in the current frame image based on the face image in the current frame image, the face images in a preset number of historical frames image, and the set of facial feature points corresponding to the face images in the preset number of historical frames image. The training process of this model can be referred to as follows. Figure 11 As mentioned above, it will not be repeated here.

[0079] For example, the first electronic device can input the face image in the current frame and the face images in the M-1 historical frames (the Mth frame face image is the face image in the current frame) and the M-1 sets of face feature points corresponding to the face images in the M-1 historical frames into the face feature point prediction model to obtain the set of face feature points corresponding to the face image in the current frame (the Mth frame).

[0080] Optionally, the specific form of the set of facial feature points corresponding to the face image in the current frame image may include any one of the following or a combination thereof: a facial feature point map (e.g., the above-mentioned...). Figure 1 (As shown), a set of facial feature point coordinate data, an incremental set with any point in the aforementioned face frame as a reference, or an incremental set with the facial feature point coordinate data corresponding to the face image in the previous frame as a reference, etc. This application does not impose any limitations on these embodiments.

[0081] For example, Figure 9 This is another flowchart illustrating the facial feature point acquisition method provided in this application embodiment. For example... Figure 9 As shown, each rectangle represents a face image, and the number inside the rectangle represents the face sequence frame number of the corresponding image. Assuming the preset quantity (M-1 above) is 4, the face image from the current frame and the face images from the four historical frames (a total of 5 frames) are input into the face feature point prediction model, along with the four sets of face feature points corresponding to the face images from the four historical frames. This yields the set of face feature points corresponding to the face image in the current frame (the 5th frame). Therefore, when the face image in the current frame is frame 1 (i.e., the face sequence frame number of the current frame is 1), the input to the face feature point prediction model can include: the face image of frame 1, the face images from the historical frames with face sequence frame numbers -3, -2, -1, and 0, and the set of face feature points T corresponding to the face images from these four historical frames. -3 T -2 T -1And T0, the output of the face feature point prediction model can be the set of face feature points T1 corresponding to the face image in the first frame. When the face image in the current frame is the second frame, the input of the face feature point prediction model can include: the face image in the second frame, the face images in the historical frame images with face sequence frame numbers -2, -1, 0, and 1, and the set of face feature points T corresponding to the face images in these four historical frame images respectively. -2 T -1 The output of the face feature point prediction model can be the set of face feature points T2 corresponding to the face image in the second frame, T0, and T1. When the face image in the current frame is the third frame, the input of the face feature point prediction model can include: the face image in the third frame, the face images in the historical frame images with face sequence frame numbers -1, 0, 1, and 2, and the set of face feature points T1 corresponding to the face images in these four historical frame images. -1 The input to the face feature point prediction model can be T0, T1, and T2, and the output can be the face feature point set T3 corresponding to the face image in the third frame. When the face image in the current frame is the fourth frame, the input to the face feature point prediction model can include: the face image in the fourth frame, the face images in the historical frame images with frame numbers 0, 1, 2, and 3, and the face feature point sets T0, T1, T2, and T3 corresponding to the face images in these four historical frame images, respectively. The output of the face feature point prediction model can be the face feature point set T4 corresponding to the face image in the fourth frame. When the face image in the current frame is the fifth frame, the input to the face feature point prediction model can include: the face image in the fifth frame, the face images in the historical frame images with frame numbers 1, 2, 3, and 4, and the face feature point sets T1, T2, T3, and T4 corresponding to the face images in these four historical frame images, respectively. The output of the face feature point prediction model can be the face feature point set T5 corresponding to the face image in the fifth frame. When the face image in the current frame is frame 6, the input to the face feature point prediction model can include: the face image of frame 6, the face images in the historical frame images with face sequence frame numbers 2, 3, 4 and 5, and the face feature point sets T2, T3, T4 and T5 corresponding to the face images in the four historical frame images respectively. The output of the face feature point prediction model can be the face feature point set T6 corresponding to the face image of frame 6.

[0082] For example, Figure 10 This is another flowchart illustrating the facial feature point acquisition method provided in this application embodiment. For example... Figure 10As shown, the input to the face feature point prediction model can include the set of face feature points corresponding to the face images in M ​​frames of face images and M-1 historical frames, respectively. The M frames of face images can include the face images in the previous M-1 historical frames and the face image in the current frame (the Mth frame face image). The output of the face feature point prediction model can be the set of face feature points corresponding to the face image in the current frame. The set of face feature points corresponding to the face image in the current frame can also be used as the set of face feature points corresponding to the historical frame face images in the next frame (M+1th frame), and input into the face feature point prediction model when predicting the set of face feature points corresponding to the face image in the next frame.

[0083] Optionally, as described above, the buffer can cache face images from historical frames and sets of face feature points corresponding to those face images. After obtaining (predicting) the set of face feature points corresponding to the face image in the current frame using the face feature point prediction model, the first electronic device can also cache the set of face feature points corresponding to the face image in the current frame output by the face feature point prediction model into the buffer to update the buffer.

[0084] The updating of the cache by the first electronic device can be divided into the following three cases:

[0085] Case 1: If the face sequence frame number of the current frame image is less than or equal to M (i.e., the above-mentioned preset number + 1), then the first electronic device caches the face image in the current frame image and the set of face feature points corresponding to the face image in the current frame image output (predicted) by the face feature point prediction model in the buffer area.

[0086] Case 2: If the face sequence frame number of the image corresponding to the face image is greater than M, then the first electronic device caches the face image in the current frame image and the set of face feature points corresponding to the face image in the current frame image output (predicted) by the face feature point prediction model in the buffer area, while deleting the face image in the historical frame image with the smallest face sequence frame number and the set of face feature points corresponding to the face image in the historical frame image with the smallest face sequence frame number.

[0087] Case 3: If no face image is detected in the current frame, the first electronic device can initialize the buffer.

[0088] Using the facial feature point acquisition method provided in this application, facial images from the current frame and a preset number of historical frames can be obtained from a first video, and sets of facial feature points corresponding to the facial images in the preset number of historical frames can be obtained respectively. These sets of facial feature points corresponding to the facial images in the historical frames may contain detection errors, and multiple detection errors may appear jittery when viewed continuously by the human eye. By inputting the facial images from the current frame, the preset number of historical frames, and the sets of facial feature points corresponding to the facial images in the preset number of historical frames into a facial feature point prediction model, the model can fully learn the characteristics of the detection errors of the sets of facial feature points corresponding to the facial images in the preset number of historical frames. The sets of facial feature points corresponding to multiple frames of facial images in the first video continuously predicted by this model all have the same or similar characteristics of the detection errors. Multiple sets of facial feature points with the same or similar characteristics are less prone to jitter.

[0089] In some possible embodiments, prior to S101 described above, the first electronic device may also acquire a facial feature point prediction model.

[0090] The facial feature point prediction model can be obtained by the first electronic device training the neural network in advance based on training samples. Of course, the model can also be obtained by any computing device with computing power training the neural network in advance based on training samples and then sending it to the first electronic device, or it can be transferred to the first electronic device through an intermediate storage medium. This application embodiment does not limit this.

[0091] Optionally, taking the example of a facial feature point prediction model obtained by the first electronic device after training the neural network based on training samples, refer to... Figure 11 , Figure 11 This is a flowchart illustrating a training method for a facial feature point prediction model provided in an embodiment of this application. Figure 11 As shown, before S101, the method may also include S201 to S202.

[0092] S201, The first electronic device acquires the training sample set.

[0093] The training sample set may include multiple training samples. Each training sample may include N (continuous) face images, the set of facial feature points corresponding to the N-1 face images preceding the Nth face image in the N frames, and a label, where the label is the set of facial feature points corresponding to the Nth face image in the N frames. N is an integer greater than or equal to 2, and N may be equal to M or not equal to M. This application embodiment does not impose any limitations on this.

[0094] Optionally, the sets of facial feature points corresponding to the N-1 frames of facial images preceding the Nth frame and the set of facial feature points corresponding to the Nth frame can be directly obtained from the face detection model or obtained through manual annotation. This application does not impose any limitations on this.

[0095] In one possible implementation, S201 may specifically include: a first electronic device acquiring a second video, the second video including multiple frames of face images; the first electronic device acquiring training samples based on the multiple frames of face images in the second video according to the following three cases:

[0096] Scenario 1: For the first frame of the face image in the second video, the first electronic device copies the first frame of the face image N-1 times to obtain N-1 reference face images; it uses a face detection model to detect the N-1 reference face images to obtain N-1 sets of reference face feature points; the first electronic device performs a weighted average of the N-1 sets of reference face feature points to obtain an average set of face feature points, which serves as the set of face feature points corresponding to each frame of the N-1 reference face images; the first electronic device uses the average set of face feature points as the first label, and obtains training samples based on the first frame of the face image, the N-1 reference face images, the set of face feature points corresponding to the N-1 reference face images, and the first label.

[0097] Case 2: For the Xth frame face image in the second video, obtain the set of facial feature points corresponding to the Xth frame face image; use the set of facial feature points corresponding to the Xth frame face image as the second label; based on the Xth frame face image, the X-1 frames face images preceding the Xth frame face image in the second video, the NX frame reference face image, the X-1 sets of facial feature points corresponding to the X-1 frame face image, the sets of facial feature points corresponding to the NX frame reference face image, and the second label, obtain the training samples; X is an integer greater than 1 and less than N;

[0098] Case 3: For the Y-th frame face image in the second video, obtain the set of facial feature points corresponding to the Y-th frame face image; use the set of facial feature points corresponding to the Y-th frame face image as the third label, and obtain the training samples based on the N-1 sets of facial feature points corresponding to the Y-th frame face image, the N-1 frames face images before the Y-th frame face image, and the N-1 frames face images before the Y-th frame face image, as well as the third label; Y is an integer greater than N.

[0099] It should be noted that the specific descriptions of the three cases of obtaining training samples mentioned above can be found in the section on the three cases of the first electronic device retrieving the face image in the M-1 frames before the current frame image cached in the buffer and the M-1 sets of face feature points corresponding to the face images in the M-1 frames, which will not be repeated here.

[0100] Optionally, the first electronic device acquiring the second video may include: the first electronic device acquiring historical video (i.e., the historical video of the aforementioned driver monitoring video and live video); for each frame in the historical video, when a face image is detected for the first time, the first electronic device acquires a face sequence starting from that image, the face sequence may include multiple frames; the first electronic device extracts multiple face images from the multiple frames of the face sequence; the first electronic device acquires the second video based on the multiple face images. The face sequence here can refer to the above. Figure 8 The face sequence described above will not be repeated here.

[0101] Optionally, the first electronic device may extract multiple frames of face images from a multi-frame image sequence, which may include: the first electronic device extracting multiple frames of face images from the multi-frame image sequence using a face bounding box. The face bounding box can be referenced... Figure 6 As mentioned above, it will not be repeated here.

[0102] Optionally, when the number of frames in the multi-frame image of the face sequence is small, the first electronic device can also extract multiple frames of face images by perturbing the position of the face bounding box. In this case, the first electronic device extracts multiple frames of face images from the multi-frame image of the face sequence using the face bounding box, which may include: for each frame in the multi-frame image of the face sequence, the first electronic device perturbs the position of the face bounding box and extracts face images multiple times to obtain multiple frames of face images.

[0103] Among them, positional perturbation refers to randomly changing the position of the face frame within a preset range.

[0104] For example, Figure 12 This is a schematic diagram illustrating the perturbation of the face bounding box position provided in an embodiment of this application. Figure 12 As shown, the first electronic device utilizes the above-mentioned Figure 7 After the face bounding box shown captures the face image, the position of the face bounding box can be perturbed. Each perturbed face bounding box can define a face image. After multiple perturbing, multiple face images can be captured from the same frame.

[0105] S202. The first electronic device trains a preset neural network based on a training sample set to obtain a facial feature point prediction model.

[0106] For example, as described above, the training sample set can include multiple training samples. The first electronic device inputs one training sample into the neural network each time, obtaining a predicted value (the set of facial feature points corresponding to the Nth frame face image predicted by the neural network). Based on the predicted value and the labels in the training samples (the set of facial feature points corresponding to the Nth frame face image), a loss function is calculated, and the parameters of the neural network are adjusted. This process is repeated iteratively until the neural network converges.

[0107] Optionally, the conditions for the neural network to converge (end training) may include: the first electronic device inputting training samples into the neural network a preset number of times, or the error between the predicted value and the label is less than a preset error threshold.

[0108] The foregoing primarily describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the aforementioned functions, it includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0109] In an exemplary embodiment, this application also provides a facial feature point acquisition device, which can be applied to the aforementioned first electronic device. Figure 13 This is a schematic diagram illustrating the composition of the facial feature point acquisition device provided in an embodiment of this application. Figure 13 As shown, the device may include an acquisition module 1301 and a processing module 1302. The acquisition module 1301 and the processing module 1302 are connected.

[0110] The acquisition module 1301 is used to acquire the current frame image from the first video; when the current frame image is detected to include a face image, to extract the face image from the current frame image; and to acquire face images from a preset number of historical frame images from the first video, as well as the set of face feature points corresponding to the face images from the preset number of historical frame images respectively.

[0111] The processing module 1302 is used to input the set of facial feature points corresponding to the face image in the current frame image, the face images in a preset number of historical frame images, and the set of facial feature points corresponding to the face images in a preset number of historical frame images into the face feature point prediction model to determine the set of facial feature points corresponding to the face image in the current frame image.

[0112] In other possible embodiments, the acquisition module 1301 is specifically used to determine the face sequence frame number of the current frame image, where the face sequence refers to a sequence of multiple consecutive frames in the first video that includes face images; and to extract the face image from the current frame image according to the face sequence frame number.

[0113] In some other possible embodiments, the acquisition module 1301 is specifically used to detect the current frame image according to a preset face detector if the face sequence frame number of the current frame image is 1, and obtain the face bounding box output by the face detector; and use the face bounding box output by the face detector to extract the face image from the current frame image.

[0114] In some other possible embodiments, the acquisition module 1301 is specifically used to generate a face bounding box by using the set of face feature points corresponding to the face images in the previous images if the face sequence frame number of the current frame image is greater than 1, and to extract the face image from the current frame image using the generated face bounding box.

[0115] In some other possible embodiments, the acquisition module 1301 is specifically used to detect the current frame image according to a preset face detector, obtain the face bounding box output by the face detector, and use the face bounding box output by the face detector to extract the face image from the current frame image.

[0116] In some other possible embodiments, the acquisition module 1301 is further configured to acquire a training sample set, which includes multiple training samples. Each training sample includes N frames of face images, a set of facial feature points corresponding to the N-1 frames of face images preceding the Nth frame of the N frames, and a label, where the label is the set of facial feature points corresponding to the Nth frame of the N frames. The processing module 1302 is further configured to train a preset neural network based on the training sample set to obtain a facial feature point prediction model.

[0117] In some other possible embodiments, the acquisition module 1301 is specifically used to acquire a second video, which includes multiple frames of face images.

[0118] Processing module 1302 is specifically used to: copy the first frame face image in the second video N-1 times to obtain N-1 reference face images; detect the N-1 reference face images using a face detection model to obtain N-1 sets of reference face feature points; perform a weighted average on the N-1 sets of reference face feature points to obtain an average set of face feature points, which serves as the set of face feature points corresponding to each frame of the N-1 reference face images; use the average set of face feature points as the first label, and obtain training samples based on the first frame face image, the N-1 reference face images, the set of face feature points corresponding to the N-1 reference face images, and the first label; for the Xth frame face image in the second video, obtain the set of face feature points corresponding to the Xth frame face image; and compare the Xth frame face image with the... The corresponding set of facial feature points is used as the second label. Training samples are obtained based on the face image of frame X, the face images of frames X-1 before frame X in the second video, the reference face image of frame N, the X-1 sets of facial feature points corresponding to the face image of frame X-1, the sets of facial feature points corresponding to the reference face image of frame N, and the second label; X is an integer greater than 1 and less than N. For the face image of frame Y in the second video, the set of facial feature points corresponding to the face image of frame Y is obtained. The set of facial feature points corresponding to the face image of frame Y is used as the third label. Training samples are obtained based on the face image of frame Y, the face images of frames N-1 before frame Y, the sets of facial feature points corresponding to the face images of frames N-1 before frame Y, and the third label; Y is an integer greater than N.

[0119] Those skilled in the art will readily recognize that, based on the units and algorithm steps described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is implemented in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0120] It should be noted that, Figure 13 The module division shown is illustrative and represents only one logical functional division; in actual implementation, other division methods are possible. For example, two or more functions can be integrated into a single processing module. These integrated modules can be implemented either in hardware or as software functional modules.

[0121] In an exemplary embodiment, this application also provides a computer-readable storage medium including computer-executable instructions that, when run on an electronic device, cause the electronic device to perform any of the methods provided in the above embodiments.

[0122] In an exemplary embodiment, this application also provides a computer program product containing computer execution instructions, which, when run on an electronic device, causes the electronic device to perform any of the methods provided in the above embodiments.

[0123] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer-executable instructions. When these computer-executable instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer-executable instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer-executable instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks, SSDs).

[0124] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple components. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0125] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

[0126] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A face feature point acquisition method, characterized by, The method comprises: obtaining a current frame image from a first video; when it is detected that the current frame image comprises a face image, cutting a face image from the current frame image; obtaining face images in a preset number of historical frame images from the first video, and a face feature point set corresponding to each of the face images in the preset number of historical frame images; inputting the face image in the current frame image, the face images in the preset number of historical frame images, and the face feature point set corresponding to each of the face images in the preset number of historical frame images into a face feature point prediction model to determine a face feature point set corresponding to the face image in the current frame image; the face feature point prediction model is used to learn a detection error feature of the face feature point set corresponding to each of the face images in the preset number of historical frame images, and generate the face feature point set corresponding to the current frame image; the detection error feature of the face feature point set corresponding to the current frame image satisfies a similarity condition with a detection error feature corresponding to the historical frame image.

2. The method of claim 1, wherein, When it is detected that the current frame image comprises a face image, the face image is cut from the current frame image, which comprises: determining a face sequence frame number of the current frame image; the face sequence refers to a sequence formed by multiple frame images comprising face images and being continuous in the first video; cutting a face image from the current frame image according to the face sequence frame number.

3. The method of claim 2, wherein, Cutting a face image from the current frame image according to the face sequence frame number comprises: if the face sequence frame number of the current frame image is 1, detecting the current frame image according to a preset face detector to obtain a face frame output by the face detector; cutting a face image from the current frame image by using the face frame output by the face detector.

4. The method of claim 2, wherein, Cutting a face image from the current frame image according to the face sequence frame number comprises: if the face sequence frame number of the current frame image is greater than 1, generating a face frame by using a face feature point set corresponding to a face image in an image before the current frame image, and cutting a face image from the current frame image by using the generated face frame.

5. The method of claim 1, wherein, When it is detected that the current frame image comprises a face image, the face image is cut from the current frame image, which comprises: detecting the current frame image according to a preset face detector to obtain a face frame output by the face detector; cutting a face image from the current frame image by using the face frame output by the face detector.

6. The method of claim 1, wherein, Before determining the face feature point set corresponding to the face image in the current frame image according to the face image in the current frame image, the face images in the preset number of historical frame images, the face feature point set corresponding to each of the face images in the preset number of historical frame images, and the face feature point prediction model, the method further comprises: obtain a training sample set; the training sample set includes a plurality of training samples; each of the training samples includes N frames of face images, a face feature point set corresponding to each of the N-1 frames of face images before an Nth frame of face image in the N frames of face images, and a label; the label is a face feature point set corresponding to the Nth frame of face image in the N frames of face images; N is an integer greater than or equal to 2; train a preset neural network based on the training sample set to obtain a face feature point prediction model.

7. The method of claim 6, wherein, The obtaining of the training sample set includes: obtaining a second video, the second video including a plurality of frames of face images; for a first frame of face image in the second video, copying the first frame of face image N-1 times to obtain N-1 frames of reference face images; detecting the N-1 frames of reference face images by using a face detection model to obtain N-1 reference face feature point sets; performing weighted average on the N-1 reference face feature point sets to obtain an average face feature point set as a face feature point set corresponding to each of the N-1 frames of reference face images; taking the average face feature point set as a first label, and obtaining a training sample according to the first frame of face image, the N-1 frames of reference face images, the face feature point set corresponding to each of the N-1 frames of reference face images, and the first label; for an Xth frame of face image in the second video, obtaining a face feature point set corresponding to the Xth frame of face image; taking the face feature point set corresponding to the Xth frame of face image as a second label, and obtaining a training sample according to the Xth frame of face image, X-1 frames of face images before the Xth frame of face image in the second video, N-X frames of reference face images, X-1 face feature point sets corresponding to the X-1 frames of face images respectively, face feature point sets corresponding to the N-X frames of reference face images respectively, and the second label; X is an integer greater than 1 and less than N; for a Yth frame of face image in the second video, obtaining a face feature point set corresponding to the Yth frame of face image; taking the face feature point set corresponding to the Yth frame of face image as a third label, and obtaining a training sample according to the Yth frame of face image, N-1 frames of face images before the Yth frame of face image, N-1 face feature point sets corresponding to the N-1 frames of face images before the Yth frame of face image respectively, and the third label; Y is an integer greater than N.

8. A face feature point acquisition apparatus characterized by comprising: The device includes an obtaining module and a processing module; the obtaining module is configured to obtain a current frame image from a first video; when detecting that the current frame image includes a face image, cut the face image from the current frame image; obtain face images in a preset number of historical frame images from the first video, and face feature point sets corresponding to the face images in the preset number of historical frame images respectively; The processing module is configured to input the face feature point set corresponding to the face image in the current frame image, the face feature point set corresponding to the face image in each of the preset number of historical frame images, and the face feature point set corresponding to the face image in each of the preset number of historical frame images into a face feature point prediction model, and determine the face feature point set corresponding to the face image in the current frame image; the face feature point prediction model is configured to learn detection error features of the face feature point set corresponding to the face image in each of the preset number of historical frame images, and generate the face feature point set corresponding to the current frame image, wherein the detection error features of the face feature point set corresponding to the current frame image and the detection error features corresponding to the historical frame images satisfy a similarity condition.

9. An electronic device, comprising: The electronic device comprises a processor and a memory; The memory stores instructions executable by the processor; The processor is configured to execute the instructions, so that the electronic device implements the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium comprises computer software instructions; When the computer software instructions run in the electronic device, the electronic device implements the method according to any one of claims 1-7.

Images