Method and related apparatus for face tracking

Abstract

Embodiments of the present application disclose a face tracking method and related device, applied to an electronic device, the method comprising: detecting that a target user is moving, acquiring motion parameters of the target user; determining a motion trajectory according to the motion parameters; acquiring a plurality of images of the target user in the motion trajectory, each of the plurality of images corresponding to a position information; and determining a target face image according to the plurality of images and the position information of each of the plurality of images. The present application is beneficial to improving the accuracy of face tracking.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to methods and related apparatus for face tracking. Background Technology

[0002] With the maturity of facial recognition technology, it has been widely used in various scenarios, such as identity authentication and photography. Facial recognition can accurately locate and track the position of facial areas, quickly pinpointing the face as its position changes, and is applicable to different expressions, genders, ages, postures, lighting conditions, etc. However, in photography scenarios, facial recognition cannot accurately capture images as the user moves, resulting in blurry images. Summary of the Invention

[0003] This application provides a face tracking method and related apparatus, which helps to improve the accuracy of face tracking control.

[0004] In a first aspect, embodiments of this application provide a face tracking method applied to an electronic device, the method comprising:

[0005] When the target user is detected to be moving, the motion parameters of the target user are acquired;

[0006] The motion trajectory is determined based on the motion parameters.

[0007] Multiple images of the target user are acquired along the motion trajectory, and each of the multiple images corresponds to a location information.

[0008] The target face image is determined based on the multiple images and the location information of each image.

[0009] Secondly, embodiments of this application provide a face tracking device applied to an electronic device. The face tracking device includes a detection unit, a determination unit, and an acquisition unit, wherein...

[0010] The detection unit is used to detect when the target user is moving and to acquire the motion parameters of the target user.

[0011] The determining unit is used to determine the motion trajectory based on the motion parameters;

[0012] The acquisition unit is used to acquire multiple images of the target user in the motion trajectory, wherein each of the multiple images corresponds to a location information.

[0013] The determining unit is further configured to determine the target face image based on the plurality of images and the position information of each image.

[0014] Thirdly, embodiments of this application provide an electronic device, including a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for performing steps in any method of the first aspect of this application.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program causes a computer to perform some or all of the steps described in any method of the first aspect of this application, the computer including an electronic device.

[0016] Fifthly, embodiments of this application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, the computer program being operable to cause a computer to perform some or all of the steps described in any method of the first aspect of this application. The computer program product may be a software installation package, and the computer includes an electronic device.

[0017] As can be seen, in this embodiment, the electronic device first detects the target user's movement and acquires the target user's motion parameters. Then, based on the motion parameters, it determines the motion trajectory. Next, it acquires multiple images of the target user along the motion trajectory, each image corresponding to a location information. Finally, it determines the target face image based on the multiple images and the location information of each image. It is evident that the electronic device can further determine the motion trajectory based on the motion parameters and acquire multiple images based on the motion trajectory, ensuring that the multiple images are on the trajectory rather than deviating from it. This avoids increasing image errors during subsequent face image processing, thus obtaining accurate images for further face image processing. This effectively solves the technical problem of poor face recognition accuracy in existing face localization and tracking methods when dealing with complex face movements, and is beneficial for improving the accuracy of face tracking. Attached Figure Description

[0018] The accompanying drawings involved in the embodiments of this application will be briefly described below.

[0019] Figure 1 This is a schematic diagram of the structure of a face tracking control device;

[0020] Figure 2A This is a flowchart illustrating a face tracking method provided in an embodiment of this application;

[0021] Figure 2BThis is an image illustration of a face tracking method provided in an embodiment of this application;

[0022] Figure 3 This is a flowchart illustrating a face tracking method disclosed in an embodiment of this application;

[0023] Figure 4 This is a flowchart illustrating a face tracking method disclosed in an embodiment of this application;

[0024] Figure 5 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application;

[0025] Figure 6 This is a block diagram of the functional units of an electronic device disclosed in an embodiment of this application. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0027] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0028] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0029] The electronic devices involved in the embodiments of this application may include various handheld devices, vehicle-mounted devices, wearable devices, computing devices, or other processing devices connected to a wireless modem with wireless communication capabilities, as well as various forms of user equipment (UE), mobile station (MS), terminal device, etc. For ease of description, the devices mentioned above are collectively referred to as electronic devices. The operating system involved in the embodiments of this invention is a software system that uniformly manages hardware resources and provides service interfaces to users.

[0030] like Figure 1 As shown, Figure 1 This is a schematic diagram of a face tracking control device provided in an embodiment of this application. The face tracking control device 100 includes: a control chip 101, a depth camera 102, a distance sensor 103, and a 3D face recognition device 104. The control chip 101 is connected to and controls the depth camera 102, the distance sensor 103, and the 3D face recognition device 104.

[0031] Among them, the control chip 101 is the control center of the face tracking control device 100, which is used to receive information and issue operation commands to the depth camera 102, distance sensor 103 and 3D face recognition device 104 through the information.

[0032] Among them, the depth camera 102 is a new type of stereo vision sensor and three-dimensional depth perception module, which can acquire high-resolution, high-precision, and low-latency depth and RGB video streams in real time and generate 3D images in real time for real-time target recognition, motion capture, or scene perception of three-dimensional images.

[0033] Among them, the 3D face recognition device 104 is a movable device that can move up and down and adjust the shooting angle, and has a camera installed inside it.

[0034] The embodiments of this application will be described in detail below.

[0035] Please see Figure 2A , Figure 2A This application provides a flowchart illustrating a face tracking method, applicable to electronic devices, such as... Figure 2A As shown, the methods for face tracking include:

[0036] S201, when the electronic device detects that the target user is moving, it acquires the motion parameters of the target user.

[0037] The motion parameters may include, but are not limited to, initial velocity, acceleration, velocity direction, and time, and are not uniquely limited here.

[0038] Optionally, the electronic device may activate a camera or an accelerometer to acquire the motion parameters of the target user.

[0039] S202, the electronic device determines the motion trajectory based on the motion parameters.

[0040] The motion trajectory can be a motion trajectory simulated by using a preset formula based on initial velocity, acceleration, and time, or it can be obtained by analyzing multiple images captured from the user in real time.

[0041] The preset formula can be obtained from a cubic polynomial of the trajectory with intermediate points or a linear function with a parabolic mixed segment; it is not limited to a single formula.

[0042] S203, the electronic device acquires multiple images of the target user in the motion trajectory, and each of the multiple images corresponds to a location information.

[0043] Each image corresponds to different location information, and some images may be partially repeated, but not completely repeated. Multiple images are acquired within a fixed preset time interval by setting a preset time interval.

[0044] S204, the electronic device determines the target face image based on the plurality of images and the position information of each image.

[0045] The target face image is a clear face.

[0046] As can be seen, in this embodiment, the electronic device first detects the target user's movement and acquires the target user's motion parameters. Then, based on the motion parameters, it determines the motion trajectory. Next, it acquires multiple images of the target user along the motion trajectory, each image corresponding to a location information. Finally, it determines the target face image based on the multiple images and the location information of each image. It is evident that the electronic device can further determine the motion trajectory based on the motion parameters and acquire multiple images based on the motion trajectory, ensuring that the multiple images are on the trajectory rather than deviating from it. This avoids increasing image errors during subsequent face image processing, thus obtaining accurate images for further face image processing. This effectively solves the technical problem of poor face recognition accuracy in existing face localization and tracking methods when dealing with complex face movements, and is beneficial for improving the accuracy of face tracking.

[0047] In one possible example, determining the target face image based on the plurality of images and the positional information of each image includes: determining the face region of each image in the plurality of images to obtain at least one face image; inputting the at least one face image into a preset deep neural network model to obtain the face feature information of the at least one face image; obtaining the clarity of the face feature information; filtering out the region images of each face part that exceed a preset clarity; and combining the region images of each face part to obtain the target face image.

[0048] The face region can be obtained by selecting a bounding box around a face, thus recognizing the facial contour and obtaining the face region. For example... Figure 2B As shown in the figure, user A is being filmed jumping. Based on the movement trajectory, multiple images of user A are captured. In this case, the face of user A at each location is selected by bounding box, as shown in the figure. If there are overlapping parts of the face images, the two overlapping face images can be selected together.

[0049] Among them, the extraction of facial feature points can be based on prior visual knowledge of facial structure. When locating key facial feature points, a trained prior scale relationship is added, namely the Y-coordinate ratio relationship from the forehead to the eyes, eyes to the nostrils, nostrils to the mouth, and mouth to the chin. This can eliminate obvious erroneous localization of individual key points and obtain accurate facial feature points.

[0050] The preset resolution can be set by the manufacturer at the factory or obtained from the image quality analysis of the aforementioned face images; there is no single limitation here.

[0051] For example, if the clarity of facial feature information in three images A, B, and C is obtained, and the eye area in image A is the clearest; the nose area in image B is the clearest; and the mouth area in image C is the clearest; then, according to a preset face ratio, the eye area in image A, the nose area in image B, and the mouth area in image C are combined to obtain the target face image. The preset face ratio can be 1 / 3:1 / 3:1 / 3.

[0052] As can be seen in this example, the electronic device extracts facial feature information from the face image frame, further obtains the clarity of each part of the face in each image, and then selects the parts with higher clarity to combine, thus obtaining a clearer face image and avoiding partially clear or blurry face images, thereby improving the accuracy of face tracking.

[0053] In one possible example, combining the region images of each face part to obtain a target face image includes: determining the face edge region in each image based on the positional information of each image; calculating the mean of the face edge regions in each image to obtain the target face edge region; determining the face area based on the target face edge region and obtaining the area of ​​the region image of each face part; calculating the face proportion in the at least one face image, where the face proportion is the percentage of the face part to the face area; and combining the region images of each face part according to the face proportion to obtain the target face image.

[0054] Optionally, the center point of each image is obtained, and a rectangular coordinate system is established with the center point as the origin. That is, based on the position information of each image, at least one edge coordinate of the face edge region in each image and the coordinates of each face part are obtained.

[0055] The area of ​​a face and the area of ​​each facial region can be determined by calculation based on at least one edge coordinate.

[0056] The calculation of the face proportions in the at least one face image can be: the proportion of the eye area to the total face area; the proportion of the nose area to the total face area; and the proportion of the mouth area to the total face area.

[0057] As can be seen in this example, the electronic device determines the edge region of the target face by using the edge regions of multiple faces. That is, it first limits the image area of ​​the target face image, and then proportionally reduces or enlarges the image areas of each face part that were previously selected as exceeding the preset resolution, according to the image area of ​​the limited target face image. This avoids the situation where the image is distorted due to the incoordination of face proportions when combining faces, and improves the accuracy of face tracking.

[0058] Optionally, multiple face proportions are obtained, the highest and lowest proportion values ​​are removed, the average of the remaining proportion values ​​is calculated, the average value is the target face proportion, and the region images of each face part are combined according to the target face proportion to obtain the target face image.

[0059] In one possible example, determining the target face image based on the plurality of images and the position information of each image includes: detecting the clarity of the plurality of images; filtering out at least one image with a clarity exceeding a preset threshold; acquiring the position information of each of the at least one image; filtering out the target image within the preset position information; and extracting facial feature points from the target image to obtain the target face image.

[0060] The preset resolution can be set by the manufacturer at the factory or obtained from the image quality analysis of the aforementioned face images; there is no single limitation here.

[0061] The preset position information refers to the position on the aforementioned movement trajectory.

[0062] The preset facial feature points are obtained by pre-recording the facial images of at least one user, extracting and storing the facial feature points in the facial images.

[0063] As can be seen, in this example, the application first selects at least one image with good clarity, and then performs a second selection on the at least one image to remove images that are not within the motion trajectory, thereby further improving the accuracy of face tracking.

[0064] In one possible example, acquiring multiple images of the target user in the motion trajectory includes: determining at least one movement position of the target user based on the motion trajectory; acquiring multiple images of each of the at least one movement position based on the at least one movement position; filtering the multiple images of each position to obtain a target image of each position; and assembling multiple images from the target images of each position.

[0065] Optionally, the step of filtering multiple images at each location to obtain a target image at each location includes: performing face localization on multiple images at each location; further preprocessing the images based on the face localization, comparing the grayscale values ​​and contrast of multiple images, and filtering out images that exceed a preset threshold, which are the target images at each location.

[0066] As can be seen in this example, the electronic device further obtains the target image with the highest image quality at each location by filtering multiple images based on each location, and then performs subsequent image combination steps on multiple target images, thereby improving the intelligence and accuracy of face tracking.

[0067] In one possible example, before acquiring multiple images of the target user in the motion trajectory, the method further includes: acquiring an initial image in the motion trajectory; and inputting the initial image into a preset motion trajectory algorithm to obtain a reference image for each position in the reference motion trajectory.

[0068] The reference image is simulated based on the initial image; that is, before acquiring subsequent images, the reference motion trajectory contains a reference image for each position.

[0069] Optionally, after acquiring multiple images of the target user in the motion trajectory, the method further includes: matching the multiple images with a reference image at each location to obtain an image matching rate at each location; filtering out target images at each location with a matching rate greater than a preset value; and combining the target images at each location into multiple images, which are then used for face image processing.

[0070] The preset matching rate can be set by the manufacturer at the factory or set by the user; there is no single limitation here.

[0071] As can be seen in this example, the electronic device matches the captured image with the reference image to obtain a more accurate image, which helps to improve the security of face tracking.

[0072] With the above Figure 2A The embodiments shown are consistent; please refer to [link / reference]. Figure 3 , Figure 3 This is a flowchart illustrating a face tracking method provided in an embodiment of this application, applied to an electronic device. As shown in the figure, the face tracking method includes:

[0073] S301, when the electronic device detects that the target user is moving, it acquires the motion parameters of the target user.

[0074] S302, the electronic device determines the motion trajectory based on the motion parameters.

[0075] S303, the electronic device acquires multiple images of the target user in the motion trajectory.

[0076] S304, the electronic device determines the face region of each of the multiple images to obtain at least one face image.

[0077] S305, the electronic device inputs the at least one face image into a preset deep neural network model to obtain the face feature information of the at least one face image.

[0078] S306, the electronic device acquires the clarity of the facial feature information.

[0079] S307, the electronic device filters out regional images of each facial feature that exceed a preset resolution.

[0080] S308, the electronic device combines the regional images of each facial feature to obtain a target facial image.

[0081] As can be seen, in this embodiment, the electronic device first detects the target user's movement and acquires the target user's motion parameters. Then, based on the motion parameters, it determines the motion trajectory. Next, it acquires multiple images of the target user along the motion trajectory, each image corresponding to a location information. Finally, it determines the target face image based on the multiple images and the location information of each image. It is evident that the electronic device can further determine the motion trajectory based on the motion parameters and acquire multiple images based on the motion trajectory, ensuring that the multiple images are on the trajectory rather than deviating from it. This avoids increasing image errors during subsequent face image processing, thus obtaining accurate images for further face image processing. This effectively solves the technical problem of poor face recognition accuracy in existing face localization and tracking methods when dealing with complex face movements, and is beneficial for improving the accuracy of face tracking.

[0082] In addition, electronic devices extract facial feature information from the face image frame to obtain the clarity of each part of the face in each image. Then, they select the parts with higher clarity and combine them to obtain a clearer face image, avoiding partially clear or blurry face images, which helps to improve the accuracy of face tracking.

[0083] With the above Figure 2A The embodiments shown are consistent; please refer to [link / reference]. Figure 4 , Figure 4 This is a flowchart illustrating a face tracking method provided in an embodiment of this application, applied to an electronic device. As shown in the figure, the face tracking method includes:

[0084] S401, when the electronic device detects that the target user is moving, it acquires the motion parameters of the target user.

[0085] S402, the electronic device determines the motion trajectory based on the motion parameters.

[0086] S403, the electronic device acquires multiple images of the target user in the motion trajectory.

[0087] S404, the electronic device determines the face region of each of the plurality of images to obtain at least one face image.

[0088] S405, the electronic device inputs the at least one face image into a preset deep neural network model to obtain the face feature information of the at least one face image.

[0089] S406, the electronic device acquires the clarity of the facial feature information.

[0090] S407, the electronic device filters out regional images of each facial feature that exceed a preset resolution.

[0091] S408, the electronic device determines the face edge region in each image based on the position information of each image.

[0092] S409, the electronic device calculates the average value of the face edge region in each image to obtain the target face edge region.

[0093] S410, the electronic device determines the face area based on the target face edge region and acquires the area of ​​the region image of each face part.

[0094] S411, the electronic device calculates the face proportion in the at least one face image, the face proportion being the percentage of the face portion to the face area.

[0095] S412, the electronic device combines the regional images of each facial part according to the facial proportion to obtain the target facial image.

[0096] As can be seen, in this embodiment, the electronic device first detects the target user's movement and acquires the target user's motion parameters. Then, based on the motion parameters, it determines the motion trajectory. Next, it acquires multiple images of the target user along the motion trajectory, each image corresponding to a location information. Finally, it determines the target face image based on the multiple images and the location information of each image. It is evident that the electronic device can further determine the motion trajectory based on the motion parameters and acquire multiple images based on the motion trajectory, ensuring that the multiple images are on the trajectory rather than deviating from it. This avoids increasing image errors during subsequent face image processing, thus obtaining accurate images for further face image processing. This effectively solves the technical problem of poor face recognition accuracy in existing face localization and tracking methods when dealing with complex face movements, and is beneficial for improving the accuracy of face tracking.

[0097] In addition, electronic devices extract facial feature information from the face image frame to obtain the clarity of each part of the face in each image. Then, they select the parts with higher clarity and combine them to obtain a clearer face image, avoiding partially clear or blurry face images, which helps to improve the accuracy of face tracking.

[0098] In addition, the electronic device determines the edge region of the target face by using the edge regions of multiple faces. That is, it first limits the image area of ​​the target face image, and then proportionally reduces or enlarges the image areas of each face part that were previously selected as exceeding the preset resolution, according to the image area of ​​the limited target face image. This avoids the situation where the image is distorted due to the incoordination of face proportions when combining faces, thus improving the accuracy of face tracking.

[0099] With the above Figure 2A , Figure 3 , Figure 4 The embodiments shown are consistent with those shown. Figure 5 This is a schematic diagram of the structure of an electronic device 500 provided in an embodiment of this application. As shown in the figure, the electronic device 500 includes an application processor 510, a memory 520, a communication interface 530, and one or more programs 521. The one or more programs 521 are stored in the memory 520 and configured to be executed by the application processor 510. The one or more programs 521 include instructions for performing the following steps.

[0100] When the target user is detected to be moving, the motion parameters of the target user are acquired;

[0101] The motion trajectory is determined based on the motion parameters.

[0102] Multiple images of the target user are acquired along the motion trajectory, and each of the multiple images corresponds to a location information.

[0103] The target face image is determined based on the multiple images and the location information of each image.

[0104] As can be seen, in this embodiment, the electronic device first detects the target user's movement and acquires the target user's motion parameters. Then, based on the motion parameters, it determines the motion trajectory. Next, it acquires multiple images of the target user along the motion trajectory, each image corresponding to a location information. Finally, it determines the target face image based on the multiple images and the location information of each image. It is evident that the electronic device can further determine the motion trajectory based on the motion parameters and acquire multiple images based on the motion trajectory, ensuring that the multiple images are on the trajectory rather than deviating from it. This avoids increasing image errors during subsequent face image processing, thus obtaining accurate images for further face image processing. This effectively solves the technical problem of poor face recognition accuracy in existing face localization and tracking methods when dealing with complex face movements, and is beneficial for improving the accuracy of face tracking.

[0105] In one possible example, regarding the determination of the target face image based on the plurality of images and the location information of each image, the instructions in the program are specifically used to perform the following operations: determine the face region of each image in the plurality of images to obtain at least one face image; input the at least one face image into a preset deep neural network model to obtain the face feature information of the at least one face image; obtain the clarity of the face feature information; filter out the region images of each face part that exceed the preset clarity; and combine the region images of each face part to obtain the target face image.

[0106] In one possible example, regarding the combination of the region images of each face part to obtain a target face image, the instructions in the program are specifically used to perform the following operations: determine the face edge region in each image based on the positional information of each image; calculate the mean of the face edge regions in each image to obtain the target face edge region; determine the face area based on the target face edge region and obtain the area of ​​the region image of each face part; calculate the face proportion in the at least one face image, where the face proportion is the percentage of the face part to the face area; and combine the region images of each face part according to the face proportion to obtain the target face image.

[0107] In one possible example, regarding the determination of the target face image based on the plurality of images and the positional information of each image, the instructions in the program are specifically used to perform the following operations: detect the sharpness of the plurality of images; filter out at least one image with a sharpness exceeding a preset threshold; obtain the positional information of each of the at least one image; filter out the target image within the preset positional information; and extract the facial feature points of the target image to obtain the target face image.

[0108] In one possible example, regarding the acquisition of multiple images of the target user in the motion trajectory, the instructions in the program are specifically used to perform the following operations: determining at least one motion position of the target user based on the motion trajectory; acquiring multiple images of each of the at least one motion position based on the at least one motion position; filtering the multiple images of each position to obtain a target image of each position; and assembling multiple images from the target images of each position.

[0109] In one possible example, before acquiring multiple images of the target user in the motion trajectory, the instructions in the program are specifically used to perform the following operations: acquire an initial image in the motion trajectory; and input the initial image into a preset motion trajectory algorithm to obtain a reference image for each position in the reference motion trajectory.

[0110] In one possible example, after acquiring multiple images of the target user in the motion trajectory, the instructions in the program are further specifically used to perform the following operations: matching the multiple images with a reference image at each location to obtain an image matching rate at each location; filtering out target images at each location with a matching rate greater than a preset value; and combining the target images at each location into multiple images, which are used for face image processing.

[0111] The above primarily describes the solutions of the embodiments of this application from the perspective of the method execution process. It is understood that, in order to achieve the above functions, the electronic device includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0112] This application embodiment can divide the electronic device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0113] The following are embodiments of the apparatus of the present invention, which are used to execute the methods implemented in the embodiments of the present invention. Figure 6 The face tracking device 600 shown is applied to an electronic device. The face tracking device 600 includes a detection unit 601, a determination unit 602, and an acquisition unit 603.

[0114] The detection unit 601 is used to detect when the target user is moving and to acquire the motion parameters of the target user.

[0115] The determining unit 602 is used to determine the motion trajectory based on the motion parameters;

[0116] The acquisition unit 603 is used to acquire multiple images of the target user in the motion trajectory, wherein each of the multiple images corresponds to a location information.

[0117] The determining unit 602 is further configured to determine the target face image based on the plurality of images and the position information of each image.

[0118] The face tracking device may further include a storage unit 604 for storing program code and data of the electronic device. The storage unit 604 may be a memory.

[0119] As can be seen, in this embodiment, the electronic device first detects the target user's movement and acquires the target user's motion parameters. Then, based on the motion parameters, it determines the motion trajectory. Next, it acquires multiple images of the target user along the motion trajectory, each image corresponding to a location information. Finally, it determines the target face image based on the multiple images and the location information of each image. It is evident that the electronic device can further determine the motion trajectory based on the motion parameters and acquire multiple images based on the motion trajectory, ensuring that the multiple images are on the trajectory rather than deviating from it. This avoids increasing image errors during subsequent face image processing, thus obtaining accurate images for further face image processing. This effectively solves the technical problem of poor face recognition accuracy in existing face localization and tracking methods when dealing with complex face movements, and is beneficial for improving the accuracy of face tracking.

[0120] In one possible example, regarding the determination of the target face image based on the plurality of images and the positional information of each image, the determining unit 602 is specifically configured to: determine the face region of each image in the plurality of images to obtain at least one face image; input the at least one face image into a preset deep neural network model to obtain the face feature information of the at least one face image; obtain the clarity of the face feature information; filter out the region images of each face part that exceed the preset clarity; and combine the region images of each face part to obtain the target face image.

[0121] In one possible example, regarding the combination of the region images of each face part to obtain a target face image, the determining unit 602 is specifically configured to: determine the face edge region in each image based on the position information of each image; calculate the mean of the face edge regions in each image to obtain the target face edge region; determine the face area based on the target face edge region and obtain the area of ​​the region image of each face part; calculate the face proportion in the at least one face image, where the face proportion is the percentage of the face part to the face area; and combine the region images of each face part according to the face proportion to obtain the target face image.

[0122] In one possible example, in determining the target face image based on the plurality of images and the position information of each image, the determining unit 602 is specifically configured to: detect the sharpness of the plurality of images; filter out at least one image with a sharpness exceeding a preset threshold; acquire the position information of the at least one image respectively; filter out the target image within the preset position information; and extract the facial feature points of the target image to obtain the target face image.

[0123] In one possible example, regarding the acquisition of multiple images of the target user in the motion trajectory, the acquisition unit 603 is specifically configured to: determine at least one motion position of the target user based on the motion trajectory; acquire multiple images of each of the at least one motion position based on the at least one motion position; filter the multiple images of each position to obtain a target image of each position; and compose multiple images from the target images of each position.

[0124] In one possible example, before acquiring multiple images of the target user in the motion trajectory, the acquisition unit 603 is specifically used to: acquire an initial image in the motion trajectory; and input the initial image into a preset motion trajectory algorithm to obtain a reference image for each position in the reference motion trajectory.

[0125] In one possible example, after acquiring multiple images of the target user in the motion trajectory, the acquisition unit 603 is further specifically used to: match the multiple images with the reference image at each position to obtain the image matching rate at each position; filter out the target images at each position with a matching rate greater than a preset value; and combine the target images at each position into multiple images, which are used for face image processing.

[0126] This application also provides a computer storage medium storing a computer program for electronic data interchange, which causes a computer to perform some or all of the steps of any of the methods described in the above method embodiments, wherein the computer includes an electronic device.

[0127] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments. The computer program product may be a software installation package, and the computer may include an electronic device.

[0128] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0129] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0130] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0131] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0132] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0133] If the integrated units described above are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0134] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0135] The embodiments of this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for face tracking, characterized in that, Applied to electronic devices, the method includes: When the target user is detected to be moving, the motion parameters of the target user are acquired; The motion trajectory is determined based on the motion parameters. Based on the motion trajectory, at least one motion position of the target user is determined. Multiple images of each of the at least one motion position are acquired. Face localization is performed on the multiple images of each position. The images are preprocessed based on the face localization. The grayscale values ​​and contrast of the multiple images of each position are compared. Images exceeding a preset threshold are selected as target images for each position. Multiple images are formed by combining the target images of each position. Each image in the multiple images corresponds to a location information. The face region of each of the multiple images is determined to obtain at least one face image; the at least one face image is input into a preset deep neural network model to obtain the face feature information of the at least one face image; the clarity of the face feature information is obtained; and the region images of each face part that exceed the preset clarity are filtered out. The facial edge region in each image is determined based on the positional information of each image; the mean value of the facial edge region in each image is calculated to obtain the target facial edge region; the facial area is determined based on the target facial edge region, and the area of ​​the region image of each facial part is obtained; the facial proportion in at least one facial image is calculated, where the facial proportion is the percentage of the facial part to the facial area; the region images of each facial part are combined according to the facial proportion to obtain the target facial image.

2. The method according to claim 1, characterized in that, The step of determining the target face image based on the plurality of images and the positional information of each image includes: Detect the sharpness of the multiple images; Select at least one image that exceeds the preset resolution; Obtain the position information of each of the at least one image; Filter out target images within the preset location information; The facial feature points of the target image are extracted to obtain the target face image.

3. The method according to claim 1, characterized in that, Before acquiring multiple images of the target user from the motion trajectory, the method further includes: Obtain the initial image from the motion trajectory; The initial image is fed into a preset motion trajectory algorithm to obtain a reference image for each position in the reference motion trajectory.

4. The method according to claim 3, characterized in that, After acquiring multiple images of the target user within the motion trajectory, the method further includes: The multiple images are matched with the reference image at each location to obtain the image matching rate at each location; Filter out target images at each location with a matching rate greater than the preset target rate; The target image at each location is combined into multiple images, which are then used for face image processing.

5. A face tracking device, characterized in that, Applied to electronic devices, the face tracking device includes a detection unit, a determination unit, and an acquisition unit, wherein... The detection unit is used to detect when the target user is moving and to acquire the motion parameters of the target user. The determining unit is used to determine the motion trajectory based on the motion parameters; The acquisition unit is configured to determine at least one movement position of the target user based on the movement trajectory, acquire multiple images of each of the at least one movement position, perform face localization on the multiple images of each position, preprocess the images based on the face localization, compare the grayscale value and contrast of the multiple images of each position, filter out images that exceed a preset threshold as target images of each position, and form multiple images from the target images of each position, with each image in the multiple images corresponding to a location information. The determining unit is further configured to determine the face region of each of the plurality of images to obtain at least one face image; input the at least one face image into a preset deep neural network model to obtain face feature information of the at least one face image; obtain the clarity of the face feature information; and filter out the region images of each face part that exceed the preset clarity. The facial edge region in each image is determined based on the positional information of each image; the mean value of the facial edge region in each image is calculated to obtain the target facial edge region; the facial area is determined based on the target facial edge region, and the area of ​​the region image of each facial part is obtained; the facial proportion in at least one facial image is calculated, where the facial proportion is the percentage of the facial part to the facial area; the region images of each facial part are combined according to the facial proportion to obtain the target facial image.

6. An electronic device, characterized in that, The device includes a processor, a memory, a communication interface, and one or more programs, said programs being stored in the memory and configured to be executed by the processor, said programs including instructions for performing the steps of the method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, A computer program for storing electronic data interchange, wherein the computer program causes a computer to perform the method as described in any one of claims 1-4.

Images