A face image tracking method and device, and a computer readable medium

Abstract

The application provides a face image tracking method and device, and a computer readable medium, the method comprising the steps of: obtaining face size information and face position information from an image (S100); calculating target display frame size information according to the face size information (S200), and obtaining current frame display frame size information according to at least the target display frame size information; calculating target display frame position information according to the face size information, the face position information and the current frame display frame size information (S300), and obtaining current frame display frame position information according to at least the target display frame position information; obtaining window image information according to the current frame display frame size information and the current frame display frame position information (S400); and displaying a target image corresponding to the window image information (S500). The application can make face image display more stable and improve user experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, in particular to a face image tracking method and device, and computer readable medium. BACKGROUND

[0002] Face recognition is realized by an AI (Artificial Intelligence) algorithm, and the AI face recognition algorithm is a machine learning algorithm in the field of artificial intelligence. The algorithm is used after being trained to a certain precision based on a large amount of face data, and when used, an image is input, and then the face information contained in the image is inferred. Since the AI face recognition algorithm based on machine learning is an algorithm based on various data in the learning process, it does not have as powerful logical thinking as a human being, so slight changes in color, brightness and other parameters of the input image of the two frames will cause slight changes in the detected face, that is, the shaking of the face position and size. At the same time, the person in the video image cannot always remain still, and the person himself will slightly sway or move back and forth, which will also cause the position and size of the face to change.

[0003] The goal of face tracking is to place the face image in a display window, so according to the shaking or slight change data obtained by face recognition, the window position and window size calculated will always have slight shaking, and the feeling of a person watching it will be like watching TV, where the camera is not fixed but is constantly shaking, which is not a good experience. SUMMARY

[0004] Based on the above status, in order to overcome the above problems, the present application provides a face image tracking method and device, and computer readable medium.

[0005] The present application provides a face image tracking method, which comprises:

[0006] Step S100: obtaining face size information and face position information from an image;

[0007] Step S200: calculating target display frame size information according to the face size information, and obtaining this frame display frame size information according to at least the target display frame size information;

[0008] Step S300: calculating target display frame position information according to the face size information, the face position information and the this frame display frame size information, and obtaining this frame display frame position information according to at least the target display frame position information;

[0009] Step S400: obtaining window image information according to the this frame display frame size information and the this frame display frame position information;

[0010] Step S500: display a target image corresponding to the window image information;

[0011] When the image is not the first frame image, the size corresponding to the current frame display box size information is between the size corresponding to the target display box size information and the size corresponding to the current display box size information; the position corresponding to the current frame display box position information is between the position corresponding to the target display box position information and the position corresponding to the current display box position information.

[0012] Preferably, the face size information includes a face width face_width and a face height face_height; and the target display box size information is calculated according to the face size information, and includes:

[0013] Step S201a: calculate a width scale multiple scale_width and a height scale multiple scale_height; the width scale multiple scale_width is a multiple of the face width face_width relative to a preset width minimum scale width, and the height scale multiple scale_height is a multiple of the face height face_height relative to a preset height minimum scale height;

[0014] Step S201b: take the maximum of the width scale multiple scale_width and the height scale multiple scale_height as a maximum scale multiple scale;

[0015] Step S201c: calculate a display box width w and a display box height h included in the target display box size information:

[0016] w = scale * width * N;

[0017] h = scale * height * N;

[0018] N is a constant greater than or equal to 1.

[0019] Preferably, when the image is the first frame image, the target display box size information is the current frame display box size information.

[0020] When the image is not the first frame image, the current frame display box size information is obtained according to at least the target display box size information, and includes:

[0021] Step S202a: judging whether the size difference between the size corresponding to the target display frame size information and the size corresponding to the current display frame size information exceeds a preset first size; if yes, executing step S202b and step S202c; otherwise, taking the current display frame size information as the display frame size information of the current frame;

[0022] The step S202b is: generating the display frame size information of the current frame based on the current display frame size information;

[0023] The step S202c is: judging whether the size difference between the size corresponding to the current display frame size information and the size corresponding to the target display frame size information is less than a preset second size, the second size being less than the first size; if yes, taking the current display frame size information as the display frame size information of the current frame; if no, continuing to execute the step S202b to generate the display frame size information of the current frame.

[0024] Preferably, in the step S202b, the generating the display frame size information of the current frame based on the current display frame size information comprises:

[0025] adjusting the size corresponding to the current display frame size information by a minimum scale or adjusting k*A to obtain the display frame size information of the current frame, wherein A is the size difference between the size corresponding to the target display frame size information and the size corresponding to the current display frame size information, k is greater than 0 and less than 1.

[0026] Preferably, the face size information comprises a face width face_width and a face height face_height; the face position information comprises a face starting x coordinate face_xst and a face starting y coordinate face_yst.

[0027] In the step S300, the target display frame position information comprises a window starting x coordinate xst and a window starting y coordinate yst:

[0028] xst = face_xst - |win_width - face_width| / 2;

[0029] yst = face_yst - |win_height - face_height| / 2;

[0030] win_width and win_height are respectively the width and height determined by the display frame size information of the current frame.

[0031] Preferably, when the image is a first frame image, the target display frame position information is the display frame position information of the current frame.

[0032] When the image is not the first frame image, the obtaining of the current frame display frame position information according to the target display frame position information comprises:

[0033] Step S301: judging whether the distance between the position corresponding to the target display frame position information and the position corresponding to the current display frame position information exceeds a preset first distance; if yes, executing step S302 and step S303; otherwise, taking the current display frame position information as the current frame display frame position information;

[0034] The step S302 comprises: generating the current frame display frame position information based on the current display frame position information.

[0035] The step S303 comprises: judging whether the distance between the position corresponding to the current display frame position information and the position corresponding to the target display frame position information is less than a preset second distance, the second distance being less than the first distance; if yes, taking the current display frame position information as the current frame display frame position information; if no, continuing to execute the step S302 to generate the current frame display frame position information.

[0036] Preferably, the generating of the current frame display frame position information based on the current display frame position information comprises:

[0037] Step S302a: judging whether the distance between the position corresponding to the target display frame position information and the position corresponding to the current display frame position information exceeds a preset third distance; if yes, executing step S302b; otherwise, executing step S302c; wherein the third distance is greater than the first distance.

[0038] The step S302b comprises: moving the position corresponding to the current display frame position information by k*B to obtain the current frame display frame position information, wherein B is the difference between the position corresponding to the target display frame position information and the position corresponding to the current display frame position information; k is greater than 0 and less than 1.

[0039] The step S302c comprises: moving the position corresponding to the current display frame position information by a minimum scale to obtain the current frame display frame position information.

[0040] Preferably, in step S300, after the target display frame position information is obtained by calculation, it is detected whether the target display frame position information is located in a preset coordinate range; if no, taking the minimum coordinate value or the maximum coordinate value of the preset coordinate range as the current frame display frame position information.

[0041] The present application further provides a face image tracking device, which comprises:

[0042] The face information recognition module is configured to acquire face size information and face position information from the image.

[0043] The current frame display box size information generation module is configured to determine current frame display box size information according to at least the face size information.

[0044] The current frame display box position information generation module is configured to calculate target display box position information according to the face size information, the face position information and the current frame display box size information, and to obtain current frame display box position information according to at least the target display box position information.

[0045] The window image information generation module is configured to acquire window image information according to the current frame display box size information and the current frame display box position information.

[0046] The display module is configured to display a target image corresponding to the window image information.

[0047] The display module is configured to display a target image corresponding to the window image information.

[0048] When the image is not a first frame image, the size corresponding to the current frame display box size information is between the size corresponding to the target display box size information and the size corresponding to the current display box size information; and the position corresponding to the current frame display box position information is between the position corresponding to the target display box position information and the position corresponding to the current display box position information.

[0049] The present application also provides a computer readable medium storing a computer program, which, when executed by a processor, can implement the face image tracking method.

[0050] The face image tracking method and device and the computer readable medium provided by the present application can acquire window information by calculating current frame display box size information and current frame display box position information, and can make face image display more stable, effectively alleviate the discomfort caused by position and scaling jitter, and improve user experience.

[0051] Other beneficial effects of the present application will be illustrated in the specific embodiments by the introduction of specific technical features and technical solutions, and those skilled in the art should be able to understand the beneficial technical effects brought by the technical features and technical solutions through the introduction of the technical features and technical solutions. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 A flow chart of a face image tracking method according to an embodiment of the present application.

[0053] Figure 2 A detailed flow chart of step S200 in a face image tracking method according to an embodiment of the present application.

[0054] Figure 3 A detailed flow chart of step S201 in a face image tracking method according to an embodiment of the present application.

[0055] Figure 4 A detailed flow chart of step S202 in a face image tracking method according to an embodiment of the present application.

[0056] Figure 5 A detailed flow chart of step S300 in a face image tracking method according to an embodiment of the present application.

[0057] Figure 6 A detailed flow chart of step S302 in a face image tracking method according to an embodiment of the present application. DETAILED DESCRIPTION

[0058] The present application is described below based on embodiments, but the present application is not limited to only these embodiments. In the following detailed description of the present application, some specific details are described in detail in order to avoid obscuring the essence of the present application, and well-known methods, processes, flows, elements are not described in detail.

[0059] In addition, those of ordinary skill in the art should understand that the drawings provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0060] Unless the context clearly requires otherwise, throughout the description and the claims, the words "comprise", "comprising", and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense; that is to say, in the sense of "including, but not limited to".

[0061] In the description of the present application, it should be understood that the terms "first", "second", etc. are only for the purpose of description and should not be understood as indicating or implying relative importance. In addition, in the description of the present application, unless otherwise specified, the meaning of "multiple" is two or more.

[0062] The present application relates to face image tracking technology, and in particular to: an image device acquires an image from the outside world, which can be obtained by the image device or obtained from a video stream recorded by the image device. The image is input into a face recognition model for inference, and the face recognition model identifies whether there is a face and face information, including face position representing the position of the face in the image and face size representing the size of the face. After the face recognition model outputs the face information, the tracking system processes the window image information using the output face size and face position. The window image information is the information corresponding to the content displayed to the user for viewing. In the face tracking scenario, the window image information is the face information. It can be understood that the specific method of obtaining the window image information is not limited, such as in the application scenario of picture cropping, the tracking system calculates the display frame position information and the display frame size information, and the window image information is obtained by cutting the image of the size corresponding to the display frame size information at the position of the image corresponding to the display frame position information. In the application scenario of the zoom camera, the tracking system calculates the display frame position information and the display frame size information, the display frame position information determines the rotation angle of the zoom camera, and the display frame size information determines the focal length of the camera, and then the window image information is output by adjusting the rotation and focal length of the camera. The window image information is displayed in a window.

[0063] It can be understood that the area surrounded by the display frame refers to the area on the image corresponding to the window image information, and the width and height of the area correspond to the display frame size information, that is, the focal length of the zoom camera or the width and height of the picture cropping. The adjustment of the size corresponds to the adjustment of the focal length of the zoom camera and the adjustment of the length and height of the picture cropping, and the change of the size can magnify or reduce the image seen in the display. The coordinates (origin) defining the relative position of the area correspond to the display frame position information. In some embodiments, the center coordinates of the image captured by the zoom camera or the center coordinates of the picture cropping are taken as the display frame position information. The coordinates correspond to the physical movement of the zoom camera or the movement of the center coordinates of the cropping window during picture cropping, which can move the image seen in the display up, down, left and right. In some other embodiments, a certain fixed point in the image captured by the zoom camera can also be taken as the display frame position information, such as a certain vertex in the image. The present application is not limited to the selection of the display position information.

[0064] For example, in the picture cropping scenario, assuming that the original image is 1920*1080 and the window requires an image of 640*360, a 160*90 image (window image information) is cut out from the original image according to the display frame position information and the display frame size information calculated by the tracking system, and is enlarged to 640*360 and displayed in the window. If it is in the zoom camera scenario, the angle and focal length of the camera can be adjusted to obtain the window image information.

[0065] It can be understood that the face tracking algorithm is to display the same face information recognized by the face model in the same fixed window after filtering and calculation by the algorithm. For example, if two faces are detected in the original image: face 1 and face 2, and tracking frame 1 and tracking frame 2 display face 1 and face 2 and the surrounding image within a certain range, if face 1 and face 2 in the video stream move constantly, the image in the tracking frame moves or zooms. Once the tracking frame and a certain face id are matched, each frame of the tracking frame only displays the face corresponding to the face id and the surrounding image until the face disappears completely. The tracking frame and the face are one-to-one, and the display effect is better.

[0066] Please refer to Figure 1 A face image tracking method includes steps S100 to S500.

[0067] In step S100, face size information and face position information are obtained from an image.

[0068] In step S200, target display frame size information is calculated according to the face size information, and at least according to the target display frame size information, the current frame display frame size information is obtained.

[0069] In step S300, target display frame position information is calculated according to the face size information, the face position information and the current frame display frame size information, and at least according to the target display frame position information, the current frame display frame position information is obtained.

[0070] In step S400, window image information is obtained according to the current frame display frame size information and the current frame display frame position information.

[0071] In step S500, the target image corresponding to the window image information is displayed.

[0072] When the image is not a first frame image, the size corresponding to the current frame display frame size information is between the size corresponding to the target display frame size information and the size corresponding to the current display frame size information; the position corresponding to the current frame display frame position information is between the position corresponding to the target display frame position information and the position corresponding to the current display frame position information.

[0073] It can be understood that the area surrounded by the target display frame corresponding to the target display frame size information and the target display frame position information refers to the area where the current face is displayed under ideal conditions.

[0074] It can be understood that, in order to improve the user experience, in some cases, after the face size information and the face position information are acquired, the window image information determined by the target display frame size information and the target display frame position information is not directly displayed, but the window image information determined by the current frame display frame size information and the current frame display frame position information is displayed, and the size and the position corresponding to the current frame display frame size information and the current frame display frame position information are close to the size and the position corresponding to the target display frame size information and the target display frame position information.

[0075] It can be understood that, in the face image tracking method provided by the present application, the window information is acquired by calculating the current frame display frame size information and the current frame display frame position information, and since when the image is not the first frame image, the size corresponding to the current frame display frame size information is between the size corresponding to the target display frame size information and the size corresponding to the current display frame size information, and the position corresponding to the current frame display frame position information is between the position corresponding to the target display frame position information and the position corresponding to the current display frame position information, the face image display is more stable, the discomfort caused by the position and the zooming jitter to the user is effectively alleviated, there is no abrupt feeling of immediately cutting the image, and the user experience is improved.

[0076] In step S100 included in the face image tracking method provided by the present application, face size information and face position information are acquired from an image; wherein the face size information includes a face width face_width and a face height face_height; and the face position information includes a face starting x-coordinate face_xst and a face starting y-coordinate face_yst.

[0077] It can be understood that the face size information and the face position information acquired from the image can be realized by any existing technical means, such as using a face recognition model to recognize whether there is a face in the image and face information, and since this part of content belongs to the prior art, it will not be described in detail in the present application.

[0078] In step S200 included in the face image tracking method provided by the present application, the current frame display frame size information is determined at least according to the face size information.

[0079] It can be understood that the size corresponding to the current frame display frame size information is the size corresponding to the display frame of the target image to be displayed.

[0080] It can be understood that, in order to more completely display the face, the size corresponding to the current frame display frame size information is greater than the size corresponding to the face size information.

[0081] Please refer to Figure 2 In some embodiments, step S200 specifically includes:

[0082] Step S201: calculating target display frame size information according to the face size information;

[0083] Step S202: determining the current frame display frame size information according to the target display frame size information and the current display frame size information or based on the target display frame size information.

[0084] It can be understood that in some embodiments, when the target display frame size information calculated according to the face size information is greatly different from the size corresponding to the current display frame size information, if the current display frame size is directly adjusted to the size corresponding to the target display frame size information, the display frame will be too large or too small, which gives people the feeling that the distance between the camera and the target scene is not fixed but constantly zooming in and out, which is not a good experience. However, the intermediate value of the current frame display frame size information can effectively alleviate the bad experience.

[0085] It can be understood that the current frame display frame size information can be updated round by round, so that the size corresponding to the current frame display frame size information gradually approaches the size corresponding to the target display frame size information.

[0086] In some embodiments, the current frame display frame size information is directly determined according to the face size information, such as the size corresponding to the face size information plus a set value to determine the target display frame size information or the current frame display frame size information.

[0087] Please refer to Figure 3 In some embodiments, calculating the target display frame size information according to the face size information includes:

[0088] Step S201a: calculating a width scale multiple scale_width and a height scale multiple scale_height; the width scale multiple scale_width is a multiple of the face width face_width relative to a preset minimum width scale width, and the height scale multiple scale_height is a multiple of the face height face_height relative to a preset minimum height scale height;

[0089] Step S201b: taking the maximum of the width scale multiple scale_width and the height scale multiple scale_height as a maximum scale multiple scale;

[0090] Step S201c: calculating the display frame width w and the display frame height h included in the target display frame size information:

[0091] w = scale * width * N;

[0092] h = scale * height * N;

[0093] N is a constant greater than or equal to 1.

[0094] It can be understood that the width and height mentioned in the present application have a fixed ratio.

[0095] As a specific embodiment, the minimum scale width and the minimum scale height can be 16:9, which is not specifically limited and can be set according to the image size, window size, etc. Generally, the minimum scale width and the minimum scale height take a value less than the window size and much less than the image size.

[0096] In order to better display the face and the surrounding image, the target display frame size is set to be N times the face size, where N is a positive integer and N > 1. As a specific embodiment, N = 2. In some embodiments, the target display frame size and the face size do not have an integer multiple relationship. In order to ensure that the face can be completely displayed, N is obtained by rounding up. For example, if the ratio of the target display frame size to the face size is 3.2, then after rounding up, the value of N is 4.

[0097] As an embodiment, in step S201a, the calculation formula of the width scale multiple scale width and the height scale multiple scale height is as follows:

[0098] scale width = face width / width + 1;

[0099] scale height = face height / height + 1.

[0100] In the above two calculation formulas, "+1" is because in the integer division operation rule, the part less than the divisor at the end will be discarded. Therefore, in order to ensure that the face can be completely displayed, "+1" will be performed after face width / width or face height is calculated. For example, the theoretical calculation result of face width / width is 2.1, but the calculation system displays the result as 2, and scale width is equal to 3 (2 + 1 = 3). It can be understood that by using the above calculation method, the complete display of the face can be ensured, and the target display frame size information suitable for the face can be calculated.

[0101] In some embodiments, the target display frame size information is the display frame size information of the current frame.

[0102] In step S202, the current display frame size information is determined based on the target display frame size information or the target display frame size information.

[0103] As an embodiment, when the image is a first frame image, the target display frame size information is the current display frame size information. Since there is no current target display frame size at this time, the target display frame size information calculated is directly used as the current display frame size information. When the image is not a first frame image, please refer to Figure 4 Then, the current display frame size information is determined based on the target display frame size information.

[0104] Step S202a: determining whether the difference between the size corresponding to the target display frame size information and the size corresponding to the current display frame size information exceeds a preset first size; if yes, executing step S202b and step S202c; otherwise, using the current display frame size information as the current display frame size information.

[0105] It can be understood that the display frame size information is updated only when the difference between the size corresponding to the target display frame size information and the size corresponding to the current display frame size information exceeds the first size, and is not updated if not. Therefore, when the image color brightness or the face movement causes the size corresponding to the target display frame size information calculated to change slightly, the display frame size is not adjusted, which guarantees the user experience and realizes zooming anti-shake, and can effectively filter the zooming back and forth problem caused by the size change due to the face movement.

[0106] As an embodiment, the first size can be m*the size corresponding to the current display frame size information or the target display frame size information. For example, m=1 / 4, the first size is the size corresponding to the current display frame size information / 4 or the size corresponding to the target display frame size information / 4.

[0107] The step S202b is: generating the current display frame size information based on the current display frame size information.

[0108] As an embodiment, the size obtained by adjusting the size corresponding to the current display frame size information by a minimum scale is the size corresponding to the current display frame size information. It can be understood that the minimum scale can be the aforementioned width minimum scale and height minimum scale, or another minimum scale. It can be understood that the height and the width are always adjusted according to a fixed ratio.

[0109] As an embodiment, the current display frame size information is adjusted by k*A to obtain the display frame size information of the current frame, wherein A is the difference between the size corresponding to the target display frame size information and the size corresponding to the current display frame size information; k is greater than 0 and less than 1. For example, k is 1 / 2, the difference between the size corresponding to the target display frame size information and the size corresponding to the current display frame size information is L, and the single adjustment range of the current display frame size is L / 2. In this way, the current display frame size can be greatly adjusted, and the adjustment efficiency is improved.

[0110] The step S202c is to determine whether the difference between the size corresponding to the current display frame size information and the size corresponding to the target display frame size information is less than a preset second size, wherein the second size is less than the first size; if yes, the current display frame size information is taken as the display frame size information of the current frame; if no, the step S202b is continuously executed to generate the display frame size information of the current frame.

[0111] It can be understood that when the difference between the size corresponding to the current display frame size information and the size corresponding to the target display frame size information is less than the second size, the sizes of the two are close, and the current display frame size does not need to be adjusted any more, thereby further guaranteeing the user experience. When the difference is greater than the second size, in the image tracking of the current frame, the size corresponding to the display frame size information of the current frame can be continuously adjusted, so that the size corresponding to the current display frame size is closer and closer to the size corresponding to the target display frame size information.

[0112] In some embodiments, when the difference between the size corresponding to the target display frame size information and the size corresponding to the current display frame size information exceeds a preset first size, in the image tracking of the current frame, the gradual adjustment of the current display frame size is realized by generating new display frame size information of the current frame, so that the size corresponding to the current display frame size is closer and closer to the size corresponding to the target display frame size information.

[0113] In the step S300 included in the face image tracking method provided in the present application, the target display frame position information is calculated according to the face size information, the face position information and the display frame size information of the current frame, and the display frame position information of the current frame is obtained according to at least the target display frame position information.

[0114] As an embodiment, in the step S300, the target display frame position information includes a window start x coordinate xst and a window start y coordinate yst of a determined coordinate:

[0115] xst = face_xst - |win_width - face_width| / 2;

[0116] yst = face_yst - |win_height - face_height| / 2;

[0117] win_width and win_height are width and height of the display frame of the current frame determined by the display frame size information.

[0118] The method can ensure that the face is always near the center of the window and there is no abrupt feeling of immediate cutting, thus improving user experience.

[0119] As an embodiment, in the step S300, the obtaining of the display frame position information of the current frame according to the target display frame position information comprises: obtaining the display frame position information of the current frame based on the target display frame position information and the current display frame position information or based on the target display frame position information.

[0120] As an embodiment, when the image is a first frame image, the target display frame position information is the display frame position information of the current frame; when the image is not a first frame image, please refer to Figure 5 , the obtaining of the display frame position information of the current frame based on the target display frame position information and the current display frame position information comprises:

[0121] Step S301: judging whether the distance between the position corresponding to the target display frame position information and the position corresponding to the current display frame position information exceeds a preset first distance; if yes, executing step S302 and step S303, otherwise, taking the current display frame position information as the display frame position information of the current frame.

[0122] It can be understood that the distance comparison or adjustment between positions comprises position adjustment in a single direction or multiple directions, and the position adjustment in multiple directions can be understood as adjustment in multiple single directions. Since the aspect ratio is fixed, the distance comparison or position adjustment in one dimension can correspondingly obtain the distance comparison result or position adjustment amplitude in another dimension.

[0123] As an embodiment, the first distance can be n times the distance corresponding to the current display frame position information or the target display frame position information. For example, n = 1 / 4, the first distance is 1 / 4 of the position corresponding to the current display frame position information or the position corresponding to the target display frame position information.

[0124] It can be understood that the display frame position information is only updated when the distance between the position corresponding to the target display frame position information and the position corresponding to the current display frame position information exceeds the first distance, and is not updated when the distance does not exceed the first distance. Therefore, when the target display frame position calculated by the image color brightness or the face movement slightly changes, the display frame position is not adjusted, the user experience is guaranteed, the position anti-shake algorithm is realized, and the image shaking caused by the face shaking can be effectively filtered out.

[0125] The step S302 is to generate the display frame position information of the current frame based on the current display frame position information.

[0126] Please refer to Figure 6 As an embodiment, the step of generating the display frame position information of the current frame based on the current display frame position information includes:

[0127] Step S302a: determining whether the distance between the position corresponding to the target display frame position information and the position corresponding to the current display frame position information exceeds a preset third distance, if yes, performing step S302b, otherwise, performing step S302c; wherein the third distance is greater than the first distance.

[0128] The step S302b is to move the position corresponding to the current display frame position information by k*B to obtain the display frame position information of the current frame, wherein B is the difference between the position corresponding to the target display frame position information and the position corresponding to the current display frame position information; k is greater than 0 and less than 1. In this way, the current display frame position can be greatly adjusted, and the adjustment efficiency is improved.

[0129] The step S302c is to move the position corresponding to the current display frame position information by a position corresponding to a minimum scale to obtain the display frame position information of the current frame. In this way, the adjustment range of the position can be reduced.

[0130] It can be understood that when the distance between the position corresponding to the target display frame position information and the position corresponding to the current display frame position information exceeds the third distance, it means that the current display frame position and the target display frame position are quite different, and the adjustment efficiency can be improved by using step S302b. When the distance does not exceed the third distance, i.e. the positions are close, step S302c is used for adjustment, which realizes the adjustment purpose while guaranteeing the user experience.

[0131] The step S303 is to determine whether the distance between the current display frame position information and the position corresponding to the target display frame position information is less than a preset second distance, the second distance being less than the first distance; if yes, the current display frame position information is taken as the current frame display frame position information; if no, the step S302 is continuously executed (in the tracking of the current frame image) to generate the current frame display frame position information.

[0132] It can be understood that when the size corresponding to the current display frame position information and the size corresponding to the target display frame position information differ by less than the second distance, the distance between them is close, and there is no need to adjust the current display frame position, further guaranteeing the user experience. When the position difference between them is greater than the second distance, the size corresponding to the current frame display frame position information can be continuously adjusted in the tracking of the current frame image, so that the current display frame position and the position corresponding to the target display frame position information are increasingly close.

[0133] In some embodiments, when the distance difference between the position corresponding to the target display frame position information and the position corresponding to the current display frame position information exceeds the preset first distance, the gradual adjustment of the current display frame position is realized by generating new current frame display frame position information in the tracking of the current frame image, so that the current display frame position and the position corresponding to the target display frame position information are increasingly close. That is, in the tracking of the current frame image, the size and position of the display frame are gradually close to the target display frame size and position. The current frame display frame size information obtained in the last round of calculation is taken as the current display frame size information in the next round of calculation, and the current frame display frame position information obtained in the last round of calculation is taken as the current display frame position information in the next round of calculation.

[0134] As an embodiment, the current frame display frame size information is determined according to the target display frame size information, for example, the target display frame size information is increased or decreased by a preset value to obtain the current frame display frame size information.

[0135] As an embodiment, in the step S300, after the target display frame position information is calculated, it is detected whether the target display frame position information is located in a preset coordinate range, if not, the minimum coordinate value or the maximum coordinate value of the preset coordinate range is taken as the current frame display frame position information.

[0136] It can be understood that the window corresponds to a coordinate range limit, such as a preset coordinate range set according to the hardware conditions of running the method, display requirements, etc., which has a minimum coordinate value and a maximum coordinate value. If the coordinate calculated in step S300 is greater than the minimum coordinate value and less than the maximum coordinate value, it is considered that the target display frame position information is located in the preset coordinate range. If the coordinate calculated in step S300 is less than the minimum coordinate value corresponding to the window, the minimum coordinate value is taken as the target display frame position information. If the coordinate calculated in step S300 is greater than the maximum coordinate value corresponding to the window, the maximum coordinate value is taken as the target display frame position information.

[0137] In some embodiments, the display frame disappears when the face matched by the current display frame disappears for N frames, otherwise the display frame is continuously displayed.

[0138] It can be understood that in the process of face recognition, especially when the face is far away and the corresponding face size is small, the face may appear and disappear, that is, the face is at the edge of being able to be recognized and not being able to be recognized, at this time the display of the display frame will be interrupted and the display effect will be poor. To solve this problem, in the present application, when tracking the face id of the recognition model, M frame buffering processing is added, which allows the face to be interrupted for M (for example, N=10) frames in detection and refreshing, that is, if a certain existing face id is detected in a new frame, the face information is not discarded immediately, but when the face corresponding to the id is still not detected for M consecutive frames, the face information is discarded. The purpose of this is to have a M-frame anti-shake function for the face output by the face recognition model.

[0139] When the display frame matches the face id, N frame buffering processing is added, which allows the display frame to be interrupted for N (currently N=10) frames in display refreshing, that is, if a certain display frame does not match the face information corresponding to the id in a new frame, the display is not immediately closed, but when the face corresponding to the id is still not detected for N consecutive frames, the window corresponding to the display frame is closed. The purpose of this is to have a N-frame anti-shake function for the face output by the display frame, which does not affect the display for a few frames in the middle.

[0140] In some embodiments, the image is cropped according to the current frame display frame size information and the current frame display frame position information to obtain the window image information; or the focal length and rotation angle of the camera are adjusted according to the current frame display frame size information and the current frame display frame position information to obtain the window image information.

[0141] In some embodiments, the image corresponding to the window image information is processed by magnification or the like before being displayed through the window. In other embodiments, the image size corresponding to the window image information is consistent with the image size displayed through the window.

[0142] In some embodiments, the window can be a separate display device, or a running window of a certain software.

[0143] It can be understood that the present application can be used in various image tracking and image processing systems, and is not limited to tracking of faces obtained by face recognition. It can also be used in other image tracking systems that require adjustment of input parameters.

[0144] The embodiment of the present application also provides a face image tracking device, which comprises a face information recognition module, a current frame display frame size information generation module, a current frame display frame position information generation module, a window image information generation module and a display module. The face information recognition module is used to obtain face size information and face position information from an image. The current frame display frame size information generation module is used to calculate target display frame size information according to the face size information, and obtain current frame display frame size information according to at least the target display frame size information. The current frame display frame position information generation module is used to calculate target display frame position information according to the face size information, the face position information and the current frame display frame size information, and obtain current frame display frame position information according to at least the target display frame position information. The window image information generation module is used to obtain window image information according to the current frame display frame size information and the current frame display frame position information. The display module is used to display a target image corresponding to the window image information.

[0145] When the image is not a first frame image, the size corresponding to the current frame display frame size information is between the size corresponding to the target display frame size information and the size corresponding to the current display frame size information; and the position corresponding to the current frame display frame position information is between the position corresponding to the target display frame position information and the position corresponding to the current display frame position information.

[0146] As an embodiment, the current frame display frame position information is obtained based on the target display frame position information and the current display frame position information, or based on the target display frame position information.

[0147] As an embodiment, the current frame display frame size information generation module is used to calculate target display frame size information according to the face size information, and determine the current frame display frame size information according to the target display frame size information and the current display frame size information, or based on the target display frame size information.

[0148] As an embodiment, the face size information comprises a face width face_width and a face height face_height. The current frame display box size information generation module comprises a scale multiple calculation module, a maximum scale calculation module and a target display box size information generation module. The scale multiple calculation module is configured to calculate a width scale multiple scale_width and a height scale multiple scale_height. The width scale multiple scale_width is a multiple of the face width face_width relative to a preset minimum width scale width, and the height scale multiple scale_height is a multiple of the face height face_height relative to a preset minimum height scale height. The maximum scale calculation module is configured to take a maximum value of the width scale multiple scale_width and the height scale multiple scale_height as a maximum scale multiple scale. The target display box size information generation module is configured to generate the target display box size information comprising a display box width w and a display box height h:

[0149] w = scale * width * N;

[0150] h = scale * height * N;

[0151] N is a constant greater than or equal to 1.

[0152] As an embodiment, the current frame display box size information generation module comprises a size judgment module configured to judge whether a size corresponding to the target display box size information and a size corresponding to the current display box size information differ by more than a preset first size or less than a preset second size. When the image is a first frame image, the current frame display box size information generation module sets the target display box size information as the current frame display box size information. When the image is not a first frame image, if the size corresponding to the target display box size information and the current display box size information differ by less than the first size, the current frame display box size information generation module sets the current display box size information as the current frame display box size information. Otherwise, the current frame display box size information generation module generates the current frame display box size information based on the current display box size information, and judges whether the size corresponding to the current display box size information and the size corresponding to the target display box size information differ by less than the preset second size through the size judgment module. If yes, the current frame display box size information generation module sets the current display box size information as the current frame display box size information. If no, the current frame display box size information generation module continues to generate the current frame display box size information (in the tracking of the current frame image).

[0153] As an embodiment, the current frame display box size information generating module generates the current frame display box size information based on the target display box size information and the current display box size information, and generates the target frame display box size information based on the target display box size information and the current frame display box size information.

[0154] As an embodiment, the face size information includes a face width face_width and a face height face_height; the face position information includes a face starting x coordinate face_xst and a face starting y coordinate face_yst; the target display box position information generated by the current frame display box position information generating module includes a window starting x coordinate xst and a window starting y coordinate yst:

[0155] xst = face_xst - |win_width - face_width| / 2;

[0156] yst = face_yst - |win_height - face_height| / 2;

[0157] win_width and win_height are width and height determined by the current frame display box size information respectively.

[0158] As an embodiment, the current frame display box position information generating module includes a distance judging module, which is used to judge whether the distance between the position corresponding to the target display box position information and the position corresponding to the current display box position information exceeds a preset first distance or is less than a preset second distance; when the image is a first frame image, the current frame display box position information generating module sets the target display box position information as the current frame display box position information; when the image is not a first frame image, if the distance between the position corresponding to the target display box position information and the position corresponding to the current display box position information does not exceed the first distance, the current frame display box position information generating module sets the current display box position information as the current frame display box position information; otherwise, the current frame display box position information generating module generates the current frame display box position information based on the current display box position information, and judges whether the distance between the position corresponding to the current display box position information and the position corresponding to the target display box position information is less than the preset second distance through the distance judging module; if yes, the current frame display box position information generating module sets the current display box position information as the current frame display box position information; if no, the current frame display box position information generating module continues to generate the current frame display box position information (in the tracking of the current frame image).

[0159] As an embodiment, the current frame display frame position information generating module generates the current frame display frame position information based on the target display frame position information.

[0160] The distance judging module judges whether the distance between the position corresponding to the target display frame position information and the position corresponding to the current display frame position information exceeds a preset third distance (the third distance is greater than the first distance), and if yes, the current frame display frame position information generating module moves the position corresponding to the current display frame position information by k*B to obtain the current frame display frame position information, wherein B is the difference between the position corresponding to the target display frame position information and the position corresponding to the current display frame position information, and k is greater than 0 and less than 1. Otherwise, the current frame display frame position information generating module moves the position corresponding to the current display frame position information by a minimum scale to obtain the current frame display frame position information.

[0161] As an embodiment, the device further comprises a coordinate range judging module, which is used to detect whether the target display frame position information is located in a preset coordinate range after the target display frame position information is calculated, and if not, the minimum coordinate value or the maximum coordinate value of the preset coordinate range is taken as the current frame display frame position information.

[0162] It can be understood that the device embodiment in the present application corresponds to the method embodiment, and the content in the method embodiment is applicable to the device embodiment.

[0163] The present application also provides a computer readable medium, which stores a computer program, and the computer program is executed by a processor to realize the face image tracking method.

[0164] It should be noted that the computer readable storage medium in the embodiments of the present disclosure is not limited to the above-mentioned embodiments, for example, it can also be an electric, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or instrument, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electric connection with one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the embodiments of the present disclosure, the computer readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, device or instrument.

[0165] Those skilled in the art can understand that the above-mentioned preferred embodiments can be freely combined and superimposed without conflict. Among them, the flowcharts and block diagrams in the drawings illustrate the possible implementation architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code containing one or more executable instructions for implementing the specified logic function. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different orders than those noted in the drawings, for example, two blocks indicated in succession can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the function involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of special-purpose hardware and computer instructions. The numbering of the steps herein is only for the convenience of description and reference, and does not limit the front and rear sequences, and the specific execution sequence is determined by the technology itself, and those skilled in the art can determine various allowed and reasonable sequences according to the technology itself.

[0166] It should be noted that the step numbering (letter or number) is used in the present application to refer to certain specific method steps, only for the purpose of convenience and brevity, and absolutely not to limit the order of the method steps by letters or numbers. Those skilled in the art can understand that the order of the related method steps should be determined by the technology itself, and should not be improperly limited by the existence of step numbering, and those skilled in the art can determine various allowed and reasonable step sequences according to the technology itself.

[0167] Those skilled in the art can understand that the above-mentioned preferred embodiments can be freely combined and superimposed without conflict.

[0168] It should be understood that the above-described embodiments are only exemplary and not limiting, and those skilled in the art can make various obvious or equivalent modifications or replacements to the above-described details without departing from the essential principles of the present application, which will be included in the scope of the claims of the present application.

Claims

1. A face image tracking method, characterized in that, The method includes: Step S100: Obtain face size information and face location information from the image; Step S200: Calculate the target display frame size information based on the face size information, and at least obtain the display frame size information of the current frame based on the target display frame size information; Step S300: Calculate the target display frame position information based on the face size information, the face position information, and the display frame size information of the current frame, and obtain the display frame position information of the current frame based at least on the target display frame position information; Step S400: Obtain window image information based on the current frame display frame size information and the current frame display frame position information; Step S500: Display the target image corresponding to the window image information; When the image is not the first frame image, the size corresponding to the display frame size information of this frame is between the size corresponding to the target display frame size information and the size corresponding to the current display frame size information; the position corresponding to the display frame position information of this frame is between the position corresponding to the target display frame position information and the position corresponding to the current display frame position information. The face size information includes face width (face_width) and face height (face_height); the face position information includes the starting x-coordinate (face_xst) and the starting y-coordinate (face_yst). In step S300, the target display box position information includes the window's starting x-coordinate xst and the window's starting y-coordinate yst: xst=face_xst-|win_width-face_width| / 2; yst=face_yst-|win_height-face_height| / 2; win_width and win_height are the width and height determined by the display frame size information of this frame, respectively.

2. The face image tracking method as described in claim 1, characterized in that, The step of calculating the target display frame size information based on the face size information includes: Step S201a: Calculate the width scale multiple scale_width and the height scale multiple scale_height; the width scale multiple scale_width is the multiple of the face width face_width relative to the preset minimum width scale width, and the height scale multiple scale_height is the multiple of the face height face_height relative to the preset minimum height scale height; Step S201b: Take the maximum value of the width scale multiple scale_width and the height scale multiple scale_height as the maximum scale multiple scale; Step S201c: Calculate the width w and height h of the display frame: w = scale * width * N; h = scale * height * N; N is a constant greater than or equal to 1.

3. The face image tracking method as described in claim 1, characterized in that, When the image is the first frame image, the target display frame size information is the display frame size information of this frame; When the image is not a first frame image, obtaining the display frame size information of the current frame based at least on the target display frame size information includes: Step S202a: Determine whether the difference between the size corresponding to the target display frame size information and the size corresponding to the current display frame size information exceeds a preset first size; If the size exceeds the limit, steps S202b and S202c are executed; otherwise, the current display frame size information is used as the display frame size information for this frame. Step S202b is: generating the display frame size information for the current frame based on the current display frame size information; Step S202c is as follows: determine whether the difference between the size corresponding to the current display frame size information and the size corresponding to the target display frame size information is less than a preset second size, wherein the second size is less than the first size; if yes, then use the current display frame size information as the display frame size information of this frame; if no, then continue to execute step S202b to generate the display frame size information of this frame.

4. The face image tracking method as described in claim 3, characterized in that, In step S202b, generating the display frame size information for the current frame based on the current display frame size information includes: Adjust the size corresponding to the current display frame size information by a minimum scale, or adjust k*A to obtain the display frame size information of this frame, where A is the size difference between the size corresponding to the target display frame size information and the size corresponding to the current display frame size information; k is greater than 0 and less than 1.

5. The face image tracking method as described in claim 1, characterized in that, When the image is the first frame image, the target display box position information is the display box position information of this frame; When the image is not a first frame image, obtaining the display frame position information of the current frame based at least on the target display frame position information includes: Step S301: Determine whether the distance between the position corresponding to the target display frame position information and the position corresponding to the current display frame position information exceeds a preset first distance; If the position exceeds the limit, steps S302 and S303 are executed; otherwise, the current display frame position information is used as the display frame position information for this frame. Step S302 is: generating the current frame display frame position information based on the current display frame position information; Step S303 is as follows: determine whether the distance between the current display frame position information and the position corresponding to the target display frame position information is less than a preset second distance, wherein the second distance is less than the first distance; if yes, then use the current display frame position information as the display frame position information of this frame; if no, then continue to execute step S302 to generate the display frame position information of this frame.

6. The face image tracking method as described in claim 5, characterized in that, The step of generating the display frame position information for the current frame based on the current display frame position information includes: Step S302a: If the distance between the position corresponding to the target display frame position information and the position corresponding to the current display frame position information exceeds a preset third distance, then if it exceeds the third distance, proceed to step S302b; otherwise, proceed to step S302c; wherein, the third distance is greater than the first distance. Step S302b is as follows: the position corresponding to the current display frame position information is moved by k*B to obtain the display frame position information of the current frame, where B is the position difference between the position corresponding to the target display frame position information and the position corresponding to the current display frame position information; k is greater than 0 and less than 1; Step S302c is: move the position corresponding to the current display frame position information by a minimum scale to obtain the display frame position information of this frame.

7. The face image tracking method as described in claim 1, characterized in that, In step S300, after calculating the target display frame position information, it is detected whether the target display frame position information is within a preset coordinate range. If not, the minimum or maximum coordinate value of the preset coordinate range is used as the display frame position information for this frame.

8. A face image tracking device, characterized in that, The device includes: The face recognition module is used to obtain face size and face location information from images; The current frame display frame size information generation module is used to calculate the target display frame size information, and at least obtain the current frame display frame size information based on the target display frame size information; The current frame display frame position information generation module is used to calculate the target display frame position information based on the face size information, the face position information and the current frame display frame size information, and to obtain the current frame display frame position information based at least on the target display frame position information; The window image information generation module is used to obtain window image information based on the current frame display frame size information and the current frame display frame position information; The display module is used to display the target image corresponding to the window image information; When the image is not the first frame image, the size corresponding to the display frame size information of this frame is between the size corresponding to the target display frame size information and the size corresponding to the current display frame size information; the position corresponding to the display frame position information of this frame is between the position corresponding to the target display frame position information and the position corresponding to the current display frame position information. The face size information includes face width (face_width) and face height (face_height); the face position information includes the face's starting x-coordinate (face_xst) and face's starting y-coordinate (face_yst); the target display box position information includes the window's starting x-coordinate (xst) and window's starting y-coordinate (yst). The method by which the current frame display frame position information generation module obtains the current frame display frame position information is as follows: xst=face_xst-|win_width-face_width| / 2; yst=face_yst-|win_height-face_height| / 2; win_width and win_height are the width and height determined by the display frame size information of this frame, respectively.

9. A computer-readable medium, characterized in that, The computer-readable medium stores a computer program that, when executed by a processor, enables the face image tracking method as described in any one of claims 1-7.

Images