Video tracking method and device and electronic equipment
By obtaining the face and head combination in the video tracking technology and performing full image detection, matching the tracking results and detection results, the problem of high calculation cost of face and head tracking in the prior art is solved, and tracking efficiency and accuracy are improved.
Patent Information
- Application Number
- CN202310631313.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-30
- Publication Date
- 2024-12-03
AI Technical Summary
When existing video tracking technology tracks face and head at the same time, the calculation cost is high and the efficiency is low.
A video tracking method is proposed, by obtaining the face and head combination in the current image frame, performing face or head tracking, and performing full image detection when the full image detects the frame, matching the tracking results and detection results to determine the tracking object. Reduce unnecessary matching calculations.
It improves the efficiency and accuracy of video tracking, reduces calculation costs, avoids redundancy of frame-by-frame detection, and ensures tracking effect.
Smart Images

Figure CN119068518A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of target tracking technology, and in particular to a video tracking method, device and electronic equipment. Background Art
[0002] Video tracking generally refers to locating the position and size of a face or head in a video, and tracking the face or head. Face tracking is performed when the face is clear and unobstructed, and head tracking is performed when the face is invisible. Face and head tracking is common in the camera's movie mode or portrait mode, and requires continuous tracking of the face or head position for focusing.
[0003] Currently, when tracking a head or face, the face tracking algorithm and the head tracking algorithm need to run simultaneously, which results in high computational cost. Summary of the invention
[0004] Embodiments of the present application provide a video tracking method, device, and electronic device.
[0005] The first embodiment of the present application provides a video tracking method, including:
[0006] Acquire a first combination of a face and a head in the current image frame, and perform face tracking or head tracking on each of the first combinations to obtain a tracking result of the first combination;
[0007] When the current image frame is a full-image detection frame, performing full-image detection of a face and a head on the current image frame to obtain a detection result of a second combination of a face and a head in the current image frame;
[0008] matching the tracking result of the first assembly with the detection result of the second assembly;
[0009] The tracking object of the second combination is determined according to the matching result, and the tracking object is continued to be tracked in the next image frame.
[0010] In one embodiment of the present application, performing face tracking or head tracking on each of the first assemblies to obtain a tracking result of the first assemblies includes:
[0011] Determine a tracking result of the same assembly in a previous image frame corresponding to the first assembly;
[0012] According to the tracking result of the same combination in the previous image frame, a tracking object of the first combination is determined, and the tracking object is tracked to obtain the tracking result of the first combination.
[0013] In one embodiment of the present application, determining the tracking object of the first assembly according to the tracking result of the same assembly in the previous image frame, tracking the tracking object, and obtaining the tracking result of the first assembly includes:
[0014] If the tracking result of the same assembly is valid for face tracking, determining that the tracking object of the first assembly is a face, and performing face tracking on the first assembly;
[0015] If the face tracking is successful, the face frame information and the head frame information are updated and it is determined that the face identifier and the head identifier are both valid, thereby obtaining the tracking result of the first combination;
[0016] If the face tracking fails, the process switches to head tracking of the first assembly to obtain a tracking result of the first assembly.
[0017] In one embodiment of the present application, determining the tracking object of the first assembly according to the tracking result of the same assembly in the previous image frame, tracking the tracking object, and obtaining the tracking result of the current image frame includes:
[0018] If the tracking result of the same assembly is invalid for face tracking, it is determined that the tracking object of the first assembly is a head, and head tracking is performed on the first assembly to obtain the tracking result of the current image frame.
[0019] In one embodiment of the present application, performing head tracking on the first assembly to obtain a tracking result of the current image frame includes:
[0020] If the head tracking is successful, the head frame information is updated and it is determined that the face identifier is invalid and the head identifier is valid, and the tracking result of the first combination is obtained;
[0021] If the head tracking fails, it is determined that the face identifier is invalid and the head identifier is invalid, and the tracking result of the first combination is obtained.
[0022] In one embodiment of the present application, the method further includes:
[0023] If the tracking of the first assembly fails, the tracker corresponding to the first assembly is destroyed.
[0024] In one embodiment of the present application, the full-image detection of the face and the head of the current image frame is performed to obtain the detection result of the second combination of the face and the head in the current image frame, including:
[0025] Performing full-image detection of a face and a head on the current image frame to obtain a face detection frame and a head detection frame in the current image frame;
[0026] The face detection frame and the head detection frame are matched to determine the faces and heads belonging to the same person in the current image frame, and the faces and heads belonging to the same person are combined to obtain the second combination and the valid state of the face identification and the head identification of the second combination as the detection result of the current image frame.
[0027] In one embodiment of the present application, matching the face detection frame and the head detection frame to determine the face and head belonging to the same person in the current image frame includes:
[0028] Traversing the face detection frame to obtain a first center point of the currently traversed face detection frame;
[0029] Determine a set of remaining unmatched head detection frames corresponding to the currently traversed face detection frame, and a second center point of each head detection frame in the set;
[0030] A head detection frame that matches the currently traversed face detection frame is determined according to the first center point and the second center point, wherein the currently traversed face detection frame and the matched head detection frame belong to the same person.
[0031] In one embodiment of the present application, determining, according to the first center point and the second center point, a head detection frame that matches the currently traversed face detection frame includes:
[0032] Acquire a second center point that is closest to the first center point as the target second center point;
[0033] If the first center point is located in the head detection frame where the second center point of the target is located, it is determined that the currently traversed face detection frame matches the head detection frame where the second center point of the target is located.
[0034] In one embodiment of the present application, the method further includes:
[0035] If the first center point is not located in the head detection frame where the target second center point is located, determining that the currently traversed face detection frame fails to match;
[0036] Based on the outward diffusion of the attribute information of the currently traversed face detection frame, a corresponding head detection frame is determined, wherein the currently traversed face detection frame and the head detection frame determined by diffusion belong to the same person.
[0037] In one embodiment of the present application, combining a face and a head belonging to the same person to obtain a second combination of the face and the head includes:
[0038] The face detection frame and the head detection frame belonging to the same person are combined to obtain the second combination.
[0039] In one embodiment of the present application, after obtaining the second assembly, the method further includes:
[0040] Saving relevant information of the face detection frame and the head detection frame in the second combination;
[0041] The face identification and the head identification marking the second combination are valid at the same time.
[0042] In one embodiment of the present application, the method further includes:
[0043] After the face detection frame matching is completed, there are still remaining unmatched head detection frames, and the second combination is directly generated based on the remaining unmatched head detection frames.
[0044] In one embodiment of the present application, the method further includes:
[0045] Saving relevant information of the remaining unmatched head detection frames in the second combination;
[0046] The head mark of the second combination is valid and the face mark is invalid.
[0047] In one embodiment of the present application, matching the tracking result of the first assembly and the detection result of the second assembly includes:
[0048] Traversing the second combination to determine a second head frame in the second combination currently traversed;
[0049] Acquire the remaining unmatched first combinations corresponding to the currently traversed second combinations, and traverse the remaining unmatched first combinations to determine the first head frame in the currently traversed first combinations;
[0050] Obtaining the intersection-and-union ratio between the second head frame currently traversed and the first head frame;
[0051] If the intersection-and-union ratio is greater than or equal to a preset threshold, it is determined that the currently traversed second combination successfully matches the currently traversed first combination;
[0052] If the intersection-and-union ratio is less than the preset threshold, it is determined that the currently traversed second combination fails to match the currently traversed first combination, and the next remaining unmatched first combination is traversed.
[0053] In one embodiment of the present application, the method further includes:
[0054] If the second assembly currently traversed does not match any first assembly, create a corresponding tracker for the second assembly currently traversed;
[0055] The tracking result of the created tracker is determined according to the valid status of the face identifier and the head identifier in the second combination currently traversed.
[0056] In one embodiment of the present application, after determining that the currently traversed second combination successfully matches the currently traversed first combination, the method further includes:
[0057] Determine the tracker of the first assembly currently traversed as the tracker of the second assembly currently traversed;
[0058] The tracking result of the tracker of the currently traversed first assembly is determined according to the tracking result of the tracker of the currently traversed second assembly.
[0059] In one embodiment of the present application, determining the tracking object of the second assembly according to the matching result, and subsequently tracking the tracking object includes:
[0060] According to the tracking result of the pursuer corresponding to the second assembly and the valid status of the face mark and the head mark in the detection result of the second assembly, the tracking object of the second assembly is determined, and the tracking object is subsequently tracked.
[0061] In one embodiment of the present application, determining the tracking object of the second assembly according to the tracking result of the pursuer corresponding to the second assembly and the valid state of the face identifier and the head identifier in the detection result of the second assembly includes:
[0062] If the tracking result corresponding to the second assembly is that the face identification is invalid and the head identification is valid, but the face identification in the detection result of the second assembly is valid, it is determined that the tracking object of the tracker is a face, and the mode is switched to face tracking.
[0063] In one embodiment of the present application, the method further includes:
[0064] Resetting the valid status of the face identifier and the head identifier of the tracker to the face identifier is valid and the head identifier is valid;
[0065] The identification of the tracker corresponding to the second assembly is maintained unchanged.
[0066] In one embodiment of the present application, the method further includes:
[0067] In the case that the current image frame is not a full-image detection frame, a tracking object of the first assembly is determined according to the tracking result of the first assembly, and the tracking object is continuously tracked in the next image frame.
[0068] The second aspect of the present application provides a video tracking device, including:
[0069] A first acquisition module is used to acquire a first combination of a face and a head in a current image frame, and perform face tracking or head tracking on each of the first combinations to obtain a tracking result of the first combination;
[0070] A second acquisition module is used to perform full-image detection of a face and a head on the current image frame when the current image frame is a full-image detection frame, and obtain a detection result of a second combination of a face and a head in the current image frame;
[0071] A matching module, used for matching the tracking result of the first assembly with the detection result of the second assembly;
[0072] The tracking module is used to determine the tracking object of the second combination according to the matching result, and continue to track the tracking object in the next image frame.
[0073] In one embodiment of the present application, the first acquisition module is used to:
[0074] Determine a tracking result of the same assembly in a previous image frame corresponding to the first assembly;
[0075] According to the tracking result of the same combination in the previous image frame, a tracking object of the first combination is determined, and the tracking object is tracked to obtain the tracking result of the first combination.
[0076] In one embodiment of the present application, the first acquisition module is used to:
[0077] If the tracking result of the same assembly is valid for face tracking, determining that the tracking object of the first assembly is a face, and performing face tracking on the first assembly;
[0078] If the face tracking is successful, the face frame information and the head frame information are updated and it is determined that the face identifier and the head identifier are both valid, thereby obtaining the tracking result of the first combination;
[0079] If the face tracking fails, the process switches to head tracking of the first assembly to obtain a tracking result of the first assembly.
[0080] In one embodiment of the present application, the first acquisition module is used to:
[0081] If the tracking result of the same assembly is invalid for face tracking, it is determined that the tracking object of the first assembly is a head, and head tracking is performed on the first assembly to obtain the tracking result of the current image frame.
[0082] In one embodiment of the present application, the first acquisition module is used to:
[0083] If the head tracking is successful, the head frame information is updated and it is determined that the face identifier is invalid and the head identifier is valid, and the tracking result of the first combination is obtained;
[0084] If the head tracking fails, it is determined that the face identifier is invalid and the head identifier is invalid, and the tracking result of the first combination is obtained.
[0085] In one embodiment of the present application, the device is also used for:
[0086] If the tracking of the first assembly fails, the tracker corresponding to the first assembly is destroyed.
[0087] In one embodiment of the present application, the second acquisition module is used to:
[0088] Performing full-image detection of a face and a head on the current image frame to obtain a face detection frame and a head detection frame in the current image frame;
[0089] The face detection frame and the head detection frame are matched to determine the faces and heads belonging to the same person in the current image frame, and the faces and heads belonging to the same person are combined to obtain the second combination and the valid state of the face identification and the head identification of the second combination as the detection result of the current image frame.
[0090] In one embodiment of the present application, the second acquisition module is used to:
[0091] Traversing the face detection frame to obtain a first center point of the currently traversed face detection frame;
[0092] Determine a set of remaining unmatched head detection frames corresponding to the currently traversed face detection frame, and a second center point of each head detection frame in the set;
[0093] A head detection frame that matches the currently traversed face detection frame is determined according to the first center point and the second center point, wherein the currently traversed face detection frame and the matched head detection frame belong to the same person.
[0094] In one embodiment of the present application, the second acquisition module is used to:
[0095] Acquire a second center point that is closest to the first center point as the target second center point;
[0096] If the first center point is located in the head detection frame where the second center point of the target is located, it is determined that the currently traversed face detection frame matches the head detection frame where the second center point of the target is located.
[0097] In one embodiment of the present application, the device is also used for:
[0098] If the first center point is not located in the head detection frame where the target second center point is located, determining that the currently traversed face detection frame fails to match;
[0099] Based on the outward diffusion of the attribute information of the currently traversed face detection frame, a corresponding head detection frame is determined, wherein the currently traversed face detection frame and the head detection frame determined by diffusion belong to the same person.
[0100] In one embodiment of the present application, the second acquisition module is used to:
[0101] The face detection frame and the head detection frame belonging to the same person are combined to obtain the second combination.
[0102] In one embodiment of the present application, the second acquisition module is further used to:
[0103] Saving relevant information of the face detection frame and the head detection frame in the second combination;
[0104] The face identification and the head identification marking the second combination are valid at the same time.
[0105] In one embodiment of the present application, the device is also used for:
[0106] After the face detection frame matching is completed, there are still remaining unmatched head detection frames, and the second combination is directly generated based on the remaining unmatched head detection frames.
[0107] In one embodiment of the present application, the device is also used for:
[0108] Saving relevant information of the remaining unmatched head detection frames in the second combination;
[0109] The head mark of the second combination is valid and the face mark is invalid.
[0110] In one embodiment of the present application, the matching module is used to:
[0111] Traversing the second combination to determine a second head frame in the second combination currently traversed;
[0112] Acquire the remaining unmatched first combinations corresponding to the currently traversed second combinations, and traverse the remaining unmatched first combinations to determine the first head frame in the currently traversed first combinations;
[0113] Obtaining the intersection-and-union ratio between the second head frame currently traversed and the first head frame;
[0114] If the intersection-and-union ratio is greater than or equal to a preset threshold, it is determined that the currently traversed second combination successfully matches the currently traversed first combination;
[0115] If the intersection-and-union ratio is less than the preset threshold, it is determined that the currently traversed second combination fails to match the currently traversed first combination, and the next remaining unmatched first combination is traversed.
[0116] In one embodiment of the present application, the device is also used for:
[0117] If the second assembly currently traversed does not match any first assembly, create a corresponding tracker for the second assembly currently traversed;
[0118] The tracking result of the created tracker is determined according to the valid status of the face identifier and the head identifier in the second combination currently traversed.
[0119] In one embodiment of the present application, the matching module is further used for:
[0120] Determine the tracker of the first assembly currently traversed as the tracker of the second assembly currently traversed;
[0121] The tracking result of the tracker of the currently traversed first assembly is determined according to the tracking result of the tracker of the currently traversed second assembly.
[0122] In one embodiment of the present application, the tracking module is used to:
[0123] According to the tracking result of the pursuer corresponding to the second assembly and the valid status of the face mark and the head mark in the detection result of the second assembly, the tracking object of the second assembly is determined, and the tracking object is subsequently tracked.
[0124] In one embodiment of the present application, the tracking module is used to:
[0125] If the tracking result corresponding to the second assembly is that the face identification is invalid and the head identification is valid, but the face identification in the detection result of the second assembly is valid, it is determined that the tracking object of the tracker is a face, and the mode is switched to face tracking.
[0126] In one embodiment of the present application, the device is also used for:
[0127] Resetting the valid status of the face identifier and the head identifier of the tracker to the face identifier is valid and the head identifier is valid;
[0128] The identification of the tracker corresponding to the second assembly is maintained unchanged.
[0129] In one embodiment of the present application, the device is also used for:
[0130] In the case that the current image frame is not a full-image detection frame, a tracking object of the first assembly is determined according to the tracking result of the first assembly, and the tracking object is continuously tracked in the next image frame.
[0131] The third aspect of the present application provides an electronic device, including: the video tracking device provided in the second aspect of the present application.
[0132] The fourth aspect embodiment of the present application proposes an electronic device, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the video tracking method proposed in the first aspect embodiment of the present application.
[0133] The fifth aspect embodiment of the present application proposes a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by the processor of an electronic device, the electronic device can execute the method proposed in the first aspect embodiment of the present application.
[0134] The sixth aspect embodiment of the present application proposes a computer program product, including a computer program, which implements the method proposed in the first aspect embodiment of the present application when executed by a processor in a communication device.
[0135] The technical solution provided by the embodiments of the present application brings at least the following beneficial effects:
[0136] In the embodiment of the present application, a first combination of a face and a head in the current image frame is obtained, and the face tracking or head tracking is performed on the first combination to obtain a corresponding tracking result; when the current image frame is a full-image detection frame, the full-image detection is performed on the current image frame to obtain the detection result of the second combination, the tracking result of the first combination is matched with the detection result of the second combination, the tracking object of the second combination is determined based on the matching result, and the tracking object is tracked in the next image frame. In the embodiment of the present application, the face and the head are tracked as a combination, which avoids the final matching of the face and the head and reduces the amount of calculation. In order to improve the detection efficiency, the full-image detection frame is set to perform full-image detection on the image, and it is determined whether the detection result during the full-image detection matches the tracking result of the normal tracking. The tracking object is adjusted according to the matching situation, which improves the analysis efficiency while ensuring the accuracy of the tracking object and the tracking effect.
[0137] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0138] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0139] Figure 1 A schematic diagram of a flow chart of a video tracking method provided in an embodiment of the present application;
[0140] Figure 2 A flowchart of another video tracking method provided in an embodiment of the present application;
[0141] Figure 3 A flowchart of another video tracking method provided in an embodiment of the present application;
[0142] Figure 4 A schematic diagram of a process of matching a face detection frame and a head detection frame provided in an embodiment of the present application;
[0143] Figure 5 A flowchart of a matching process of a tracking result of a first assembly and a detection result of a second assembly provided in an embodiment of the present application;
[0144] Figure 6 A flowchart of another video tracking method provided in an embodiment of the present application;
[0145] Figure 7 A schematic diagram of the structure of a video tracking device provided in an embodiment of the present application;
[0146] Figure 8A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;
[0147] Fig. 9 A schematic diagram of the structure of another electronic device provided in an embodiment of the present application;
[0148] Fig.10 A schematic diagram of the structure of another electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0149] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.
[0150] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present application. The singular forms of "a" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.
[0151] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in the embodiments of the present application, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at..." or "when..." or "in response to determination".
[0152] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.
[0153] It should be noted that the video tracking method provided in any embodiment of the present application can be executed alone, or in combination with possible implementation methods in other embodiments, or in combination with any technical solution in the related technology.
[0154] The video tracking method, device and electronic device according to the embodiments of the present application are described below with reference to the accompanying drawings.
[0155] Figure 1 A schematic diagram of a video tracking method provided in an embodiment of the present application. Figure 1 As shown, the method includes but is not limited to the following steps:
[0156] S101, obtaining a first combination of a face and a head in a current image frame, and performing face tracking or head tracking on each first combination to obtain a tracking result of the first combination.
[0157] The embodiments of the present application can be applied to electronic devices with image acquisition devices such as cameras or monitoring devices, such as mobile phones, laptop computers, desktop computers, netbook computers, personal digital assistants, etc. The electronic devices can track faces and heads, such as tracking faces and heads in the portrait mode of the camera. The current image frame is the image frame at the current moment in the process of shooting by the camera or monitoring device, and the current image frame is used to identify and track faces and heads, that is, real-time face and head recognition is performed on each captured image frame.
[0158] In order to improve the tracking efficiency, in the embodiment of the present application, faces and heads are jointly tracked, that is, faces and / or heads belonging to the same person are combined to obtain a combination.
[0159] In the embodiment of the present application, the face-head combination in the current image frame is referred to as the first combination. The face-head joint tracking is performed starting from the first image frame of the video, and the tracked face and head are associated to obtain the first face-head combination in the current image frame.
[0160] In some implementations, after acquiring the first assembly in the current image frame, face tracking or head tracking can be performed on the first assembly. In this application, when face tracking or head tracking is performed on the first assembly, it is necessary to refer to the tracking result of the previous image frame. Further, based on the tracking result of the previous image frame, the tracking object of the first assembly is determined, and the tracking object is tracked to obtain the tracking result of the first assembly.
[0161] Optionally, the tracking result includes: face valid, head valid, face invalid, head invalid, etc. In some implementations, the tracking object of the first assembly in the current image frame is determined according to the tracking result of the same assembly as the first assembly in the previous image frame. If face tracking is valid in the previous image frame, the tracking object of the current image frame is face tracking. If head tracking is valid in the previous image frame, the tracking object of the current image frame is head tracking. In the current image frame, the determined tracking object continues to be tracked to obtain the tracking result of the current image frame.
[0162] Optionally, the method for tracking the first assembly may be an optical flow tracking method, a Kalman filter method, or the like.
[0163] S102, when the current image frame is a full-image detection frame, performing full-image detection of a face and a head on the current image frame to obtain a detection result of a second combination of a face and a head in the current image frame.
[0164] Determining the tracking object to track based on the tracking result of the previous image frame will cause certain errors in the current image frame during actual tracking. If real-time tracking is always based on the previous image frame, errors may continue to accumulate and the accuracy is low. Therefore, the image can be re-detected. Considering the high cost of frame-by-frame detection, a full-image detection frame is set in the embodiment of the present application. The full-image detection frame is the image that needs to be detected as a whole. In other words, if the current image frame is not a full-image detection frame, the current image frame is tracked based on the tracking result of the previous image frame; if the current image frame is a full-image detection frame, the current image frame needs to be re-detected as a whole to achieve the purpose of reducing errors and improving tracking accuracy.
[0165] When the current image frame is a full-image detection frame, obtain the position of the head and the face in the current image frame, and expand the head or face by a certain proportion to construct the corresponding head detection frame and face detection frame. Considering that the current image frame may include multiple head detection frames and face detection frames, it is necessary to distinguish each head detection frame and face detection frame.
[0166] Optionally, the corresponding head detection frame and face detection frame may be represented by the coordinates of the positions of the head detection frame and face detection frame; different head detection frames and face detection frames may also be represented by special marks. The special marks may be numbers, letters, or special symbols such as triangles and rectangles.
[0167] After determining the head detection frame and the face detection frame in the current image frame, the head detection frame and the face detection frame of the same person are associated, that is, it can be determined whether the head detection frame and the face detection frame belong to the same person. If so, the head detection frame and the face detection frame belonging to the same person are associated to form a corresponding second combination.
[0168] Further, the valid states of the face identifiers and the head identifiers in the second assembly are determined, and the valid states of the face identifiers and the head identifiers of the second assembly and the second assembly are used as the detection result of the current image frame.
[0169] Optionally, full-image detection can be performed at intervals of several frames, for example, full-image detection can be performed at intervals of 8 or 10 frames. Full-image detection can also be performed based on the frame number, for example, full-image detection can be performed on frames with a frame number of 5, such as images with frame numbers of 5, 15, 25, 35, etc. Full-image detection can avoid error accumulation in continuous tracking and improve tracking accuracy.
[0170] S103, matching the tracking result of the first assembly and the detection result of the second assembly.
[0171] The first combination is obtained by the tracker tracking the current frame image according to the tracking object of the previous frame image, and the second combination is obtained by detection when the current frame image is a full-image detection frame. Therefore, after determining the tracking result of the first combination and the detection result of the second combination, the tracking result of the first combination and the detection result of the second combination are matched, and the matching status between the first combination and the second combination is determined.
[0172] Optionally, the matching situation between the first assembly and the second assembly can be determined by the area of the overlapping area of any tracking object in the first assembly and the second assembly. The tracking state of the first assembly when tracking with the tracker includes: tracking a face and tracking a head; wherein, when tracking a face, a face frame can be obtained, and a head frame can be obtained based on a certain proportion of the face frame expansion; the detection result of the second assembly is a valid face and a valid head. Therefore, in the embodiment of the present application, the head frame in the first assembly and the second assembly can be selected to analyze the matching situation between the first assembly and the second assembly.
[0173] In some implementations, whether the match is successful can be determined based on the intersection area of the head frames in the first assembly and the second assembly. For example, if the intersection area of the head frames in the first assembly and the head frames in the second assembly is larger, it means that the same area of the head frames in the first assembly and the head frames in the second assembly is larger, and the first assembly and the second assembly are more likely to be successfully matched. Conversely, if the intersection area of the head frames in the first assembly and the head frames in the second assembly is smaller, it means that the same area of the head frames in the first assembly and the head frames in the second assembly is smaller, and the first assembly and the second assembly are more likely to fail to match.
[0174] S104, determining the tracking object of the second combination according to the matching result, and continuing to track the tracking object in the next image frame.
[0175] Optionally, when the first combination and the second combination are matched successfully, the tracking result of the currently matched first combination is used as the tracking result of the second combination, that is, the tracking result of the second combination is the tracking result of the currently matched first combination.
[0176] In some implementations, the tracking object of the second assembly can be determined according to the tracking result of the second assembly and the detection result of the second assembly. When the tracking result of the second assembly is that the face identification is invalid and the head identification is valid, but the detection result of the second assembly is that the face identification is valid, the tracking object of the second assembly is determined to be face tracking, and the face is tracked in the next image frame.
[0177] In the embodiment of the present application, a first combination of a face and a head in the current image frame is obtained, and the face tracking or head tracking is performed on the first combination to obtain a corresponding tracking result; when the current image frame is a full-image detection frame, the full-image detection is performed on the current image frame to obtain the detection result of the second combination, the tracking result of the first combination is matched with the detection result of the second combination, the tracking object of the second combination is determined based on the matching result, and the tracking object is tracked in the next image frame. In the embodiment of the present application, the face and the head are tracked as a combination, which avoids the final matching of the face and the head and reduces the amount of calculation. In order to improve the detection efficiency, the full-image detection frame is set to perform full-image detection on the image, and it is determined whether the detection result during the full-image detection matches the tracking result of the normal tracking. The tracking object is adjusted according to the matching situation, which improves the analysis efficiency while ensuring the accuracy of the tracking object and the tracking effect.
[0178] Figure 2 A schematic diagram of a video tracking method provided in an embodiment of the present application. Figure 2 As shown, the method includes but is not limited to the following steps:
[0179] S201, obtaining a first combination of a face and a head in a current image frame.
[0180] In the embodiment of the present application, the implementation method of step S201 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0181] S202, determining the tracking result of the same assembly in the previous image frame corresponding to the first assembly.
[0182] In some implementations, the tracking result of each assembly in the previous image frame is stored, so the tracking result of the same assembly as the first assembly in the previous image frame can be determined based on the historical tracking record.
[0183] Optionally, the tracking results include: valid face, valid head, invalid face, invalid head, etc.
[0184] S203, determining a tracking object of the first assembly according to the tracking result of the same assembly in the previous image frame, tracking the tracking object, and obtaining a tracking result of the first assembly.
[0185] In some implementations, if the tracking result of the same combination is valid for face tracking, the tracking object of the first combination is determined to be a face. In other words, if the tracking result of the same combination in the previous image frame is valid for face tracking, the tracking object of the first combination in the current image frame is a face.
[0186] In other implementations, if the tracking result of the same combination is invalid for face tracking, the tracking object of the first combination is determined to be a human head. In other words, if the tracking result of the same combination in the previous image frame is invalid for face tracking, the tracking object of the first combination in the current image frame is a human head.
[0187] After the tracking object of the first assembly is determined, the tracking object of the first assembly is tracked to determine the tracking result of the first assembly.
[0188] Optionally, in the case where the first assembly is face tracking, if the face tracking is successful, the face frame information and the head frame information are updated and it is determined that the face identifier and the head identifier are valid at the same time to obtain the tracking result of the first assembly; if the face tracking fails, switch to head tracking of the first assembly to obtain the tracking result of the first assembly.
[0189] Optionally, the head frame can be obtained by expanding the face frame by a certain proportion; wherein the face frame can be a rectangular area or a circular area obtained by expanding the position of the face.
[0190] As an example, assume that face tracking is performed on the first assembly at this time. When face tracking is successful, the face frame information and head frame information in the current image frame are obtained, and the face frame information and head frame information of the first assembly in the current image frame are updated; at the same time, the face identifier and head identifier in the current image frame are determined to be valid; the face valid and head valid are used as the tracking result of the first assembly. When face tracking fails, head tracking is performed on the first assembly, and the tracking result of the first assembly is determined based on the head tracking result.
[0191] Optionally, in the case where the first combination is head tracking, if the head tracking is successful, the head frame information is updated and it is determined that the face identifier is invalid and the head identifier is valid, and the tracking result of the first combination is obtained; if the head tracking fails, it is determined that the face identifier is invalid and the head identifier is invalid, and the tracking result of the first combination is obtained.
[0192] For example, assume that head tracking is performed on the first assembly at this time. When head tracking is successful, the head frame information in the current image frame is updated, and it is determined that the face identifier is invalid and the head identifier is valid at this time; the face identifier is invalid and the head identifier is valid as the tracking result of the first assembly. When head tracking fails, it is determined that the face identifier is invalid and the head identifier is invalid at this time, and the face identifier is invalid and the head identifier is invalid as the tracking result of the first assembly.
[0193] Optionally, if the tracking of the first assembly fails, the tracker corresponding to the first assembly is destroyed.
[0194] S204, when the current image frame is a full-image detection frame, performing full-image detection of the face and the head on the current image frame to obtain a detection result of a second combination of the face and the head in the current image frame.
[0195] Optionally, when the current image frame is not a full-image detection frame, the tracking object of the first assembly is determined according to the tracking result of the first assembly, and the tracking object is continued for the next image frame. That is, when the current image frame is not a full-image detection frame, if the tracking result of the first assembly is that the face identification is valid, the tracking object is a face; if the tracking object of the first assembly is that the head identification is valid, the tracking object is a head.
[0196] In the embodiment of the present application, the implementation method of step S204 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0197] S205, matching the tracking result of the first assembly and the detection result of the second assembly.
[0198] In the embodiment of the present application, the implementation method of step S205 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0199] S206, determining the tracking object of the second combination according to the matching result, and continuing to track the tracking object in the next image frame.
[0200] In the embodiment of the present application, the implementation method of step S206 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0201] In the embodiment of the present application, by obtaining the first combination of human faces and heads in the current image frame, determining the tracking result of the first combination in the previous image frame, determining the tracking object of the current image frame and tracking it according to the tracking result of the previous image frame, and obtaining the tracking result of the first combination in the current image frame; when the current image frame is a full-image detection frame, performing full-image detection on the current image frame to obtain the detection result of the second combination, matching the tracking result of the first combination with the detection result of the second combination, determining the tracking object of the second combination based on the matching result, and tracking the tracking object for the next image frame. In the embodiment of the present application, when determining the tracking result of the first combination in the current image frame, refer to the tracking result of the same combination in the previous image frame, determine the tracking object according to the tracking result of the previous image frame, and obtain the tracking result of the first combination based on the tracking object, so that the tracking result of the current image frame is more reasonable, matching the detection result of the second combination based on the more reasonable tracking result, and determining the tracking object to track the next image frame, so that the overall tracking efficiency is higher and the effect is better.
[0202] Figure 3 A schematic diagram of a video tracking method provided in an embodiment of the present application. Figure 3 As shown, the method includes but is not limited to the following steps:
[0203] S301, obtaining a first combination of a face and a head in a current image frame, and performing face tracking or head tracking on each first combination to obtain a tracking result of the first combination.
[0204] In the embodiment of the present application, the implementation method of step S301 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0205] S302, when the current image frame is a full-image detection frame, perform full-image detection of a face and a head on the current image frame to obtain a face detection frame and a head detection frame in the current image frame.
[0206] Optionally, when the current image frame is a full-image detection frame, the current image frame is detected to determine all faces and heads therein, and further determine a face detection frame and a head detection frame according to the positions of the faces and heads.
[0207] S303, matching the face detection frame and the head detection frame, determining the faces and heads belonging to the same person in the current image frame, and combining the faces and heads belonging to the same person to obtain a second combination and the valid state of the face identification and the head identification of the second combination as the detection result of the current image frame.
[0208] It is understandable that the face and head belonging to the same person should be close to each other and have more overlapping areas, so the analysis can be based on the distance between the face detection frame and the head detection frame, or based on the overlapping area between the face detection frame and the head detection frame. The smaller the distance between the face detection frame and the head detection frame, the more likely it is that the face and head belong to the same person, or the larger the overlapping area between the face detection frame and the head detection frame, the more likely it is that the face and head belong to the same task.
[0209] The face and head determined to be the same person are combined to obtain a second combination.
[0210] Optionally, after determining the second combination, the valid status of the face identifier and the head identifier of the second combination is obtained, for example, in the second combination of the face and head combination of the same person, the valid status of the face identifier and the head identifier is that the face identifier is valid and the head identifier is valid. The valid status of the face identifier and the head identifier in the second combination is used as the detection result of the current image frame.
[0211] S304, matching the tracking result of the first assembly and the detection result of the second assembly.
[0212] In the embodiment of the present application, the implementation method of step S304 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0213] S305: Determine the tracking object of the second combination according to the matching result, and continue to track the tracking object in the next image frame.
[0214] In the embodiment of the present application, the implementation method of step S305 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0215] In an embodiment of the present application, a first combination of a face and a head in the current image frame is obtained, and the corresponding tracking result is obtained; when the current image frame is a full-image detection frame, a head detection frame and a face detection frame in the current image frame are obtained, and it is determined whether the face detection frame and the head detection frame belong to the same person, thereby determining a second combination and the valid state of the face identification and the head identification in the second combination, obtaining the detection result of the second combination, matching the tracking result of the first combination with the detection result of the second combination to determine the tracking object, and tracking the next image frame. In an embodiment of the present application, a full-image detection frame is set to improve the efficiency of video tracking and avoid the redundancy of frame-by-frame detection and analysis; all head detection frames and face detection frames in the full-image detection frame are obtained, and the head detection frame and the face detection frame are matched to determine whether they belong to the same person, and the corresponding second combination is obtained, which reduces the overall complexity and calculation amount, avoids additional power consumption, and is more efficient.
[0216] Based on the above embodiment, the process of matching the face detection frame and the head detection frame is explained. Figure 4 As shown, the process of matching the face detection frame and the head detection frame includes but is not limited to the following steps:
[0217] S401, traverse the face detection frame to obtain the first center point of the currently traversed face detection frame.
[0218] After determining the face detection frame and the head detection frame in the current image frame, determine the face detection frame and the head detection frame belonging to the same person in the current image frame, that is, it is necessary to match the head detection frame and the face detection frame in the current image frame.
[0219] Each face detection frame in the current image frame is traversed to determine the first center point of the traversed face detection frame.
[0220] Optionally, the first center point may be a point at a central position or a centroid.
[0221] S402, determining a set of remaining unmatched head detection frames corresponding to the currently traversed face detection frame, and locating a second center point of each head detection frame in the set.
[0222] Optionally, when matching the corresponding head detection frame for the currently traversed face detection frame, all the head detection frames in the current image frame that have not been successfully matched are used as the head detection frame set of the face detection frame; and the second center point of each head detection frame in the head detection frame set is determined.
[0223] Optionally, the second center point may also be a point at a central position or a centroid.
[0224] Optionally, to ensure the accuracy of the match between the head detection frame and the face detection frame, the acquisition of the first center point and the second center point can be kept consistent. For example, if the first center point is the point at the center position, then the second center point also corresponds to the point at the center position; if the first center point is the center of mass, then the second center point also corresponds to the center of mass.
[0225] After obtaining the first center point and the second center point, the head detection frame that matches the currently traversed face detection frame can be determined based on the first center point of the currently traversed face detection frame and the second center point of each head detection frame in the head detection frame set.
[0226] S403, obtaining a second center point that is closest to the first center point as the target second center point.
[0227] It is understandable that the head detection frame and face detection frame belonging to the same person should be close to each other. Therefore, the second center point closest to the first center point can be obtained as the target second center point to determine whether the target head detection frame where the target second center point is located and the face detection frame corresponding to the first center point belong to the same person.
[0228] S404: If the first center point is located in the head detection frame where the target second center point is located, determine that the currently traversed face detection frame matches the head detection frame where the target second center point is located.
[0229] Optionally, if the position of the first center point is within the target head detection frame, it means that there is a large overlapping area between the face detection frame where the first center point is located and the target head detection frame, then it is determined that the face detection frame corresponding to the first center point matches the target head detection frame, that is, it is determined that the currently traversed face detection frame and the head detection frame where the target second center point is located belong to the same person.
[0230] S405: If the first center point is not located in the head detection frame where the target second center point is located, it is determined that the matching of the currently traversed face detection frame fails.
[0231] Optionally, if the first center point is not located in the head detection frame where the second center point of the target is located, it means that there is no overlapping area between the face detection frame where the first center point is located and the head detection frame where the second center point of the target is located, or there is very little overlapping area, then it is determined that the match between the face detection frame corresponding to the first center point and the head detection frame where the second center point of the target is located fails.
[0232] S406, diffusing outward based on the attribute information of the currently traversed face detection frame to determine a corresponding head detection frame, wherein the currently traversed face detection frame and the head detection frame determined by diffusion belong to the same person.
[0233] Optionally, the attribute information of the currently traversed face detection frame may be the position and size of the face detection frame.
[0234] After the currently traversed face detection frame fails to match the head detection frame where the second center point of the target is located, it diffuses outward based on the attribute information of the currently traversed face detection frame, that is, it expands outward by a certain proportion according to the position and size of the currently traversed face detection frame, and uses the expanded area as the head detection frame corresponding to the currently traversed face detection frame, and determines that the face detection frame traversed at this time matches its corresponding diffused head detection frame, that is, determines that the currently traversed face detection frame and its corresponding diffused head detection frame belong to the same person.
[0235] S407, combining the faces and heads belonging to the same person to obtain a second combination and a valid state of the face identification and the head identification of the second combination as the detection result of the current image frame.
[0236] In some implementations, the face detection frame and the head detection frame belonging to the same person are combined to obtain a second combination.
[0237] In some other implementations, relevant information of the face detection frame and the head detection frame in the second assembly is saved; and the face identification and the head identification of the second assembly are marked to be valid at the same time.
[0238] Optionally, after the face detection frame matching is completed, if there are still unmatched head detection frames remaining, the second assembly is directly generated based on the remaining unmatched head detection frames. That is, after the face detection frame matching in the current image frame is completed, there are still unmatched head detection frames remaining, and the remaining unmatched head detection frames are used to generate the second assembly. At the same time, the relevant information of the remaining unmatched head detection frames in the second assembly is saved; and the head identification of the second assembly is marked as valid and the face identification is invalid.
[0239] It can be understood that for the second combination of the face detection frame and the head detection frame belonging to the same person, the face identification and the head identification are both valid; for the second combination of the remaining unmatched head detection frames, the head identification is valid and the face identification is invalid. The state of the head identification and the face identification in the second combination is used as the detection result of the current image frame.
[0240] As an exemplary explanation, the face detection frames detected in the current image frame include face detection frame A, face detection frame B, face detection frame C, and face detection frame D. The head detection frames detected in the current image frame include head detection frame A', head detection frame B', head detection frame C', and head detection frame D'. The face detection frames are traversed, and assuming that the currently traversed frame is face detection frame A, the first center point of face detection frame A is obtained; at this time, the remaining unmatched head detection frame set corresponding to face detection frame A includes head detection frame A', head detection frame B', head detection frame C', and head detection frame D', and the second center point corresponding to each head detection frame in the head detection frame set is determined; further, the distance between the first center point of face detection frame A and the second center point of each head detection frame in the head detection frame set is calculated, and the second center point of the head detection frame with the smallest distance is selected. Point A is taken as the second center point of the target. Assuming that the head detection frame corresponding to the second center point of the target is head detection frame A', it is determined whether the first center point of face detection frame A is within head detection frame A'. If so, face matching frame A and head matching frame A' are successfully matched, and face matching frame A and head matching frame A' are combined into a second combination; if not, face matching frame A and head matching frame A' fail to match, and the head detection frame corresponding to the face detection frame is obtained by diffusion based on the attribute information of face matching frame A. Face matching frame A and the head detection frame obtained by its diffusion are combined into a second combination. Further, continue to traverse the head detection frame. For example, the next traversed face detection frame is face detection frame B, and continue to obtain the remaining unmatched head detection frame set corresponding to face detection frame B, and so on, until all face detection frames in the current image frame are traversed. After the face detection frame traversal is completed, if there are still unmatched head detection frames left, the remaining unmatched head detection frames are taken as a new second combination.
[0241] In the embodiment of the present application, the first center point currently traversed is matched with the second center point of the head detection frame. Considering that the face and head belonging to the same person are close, the target second center point closest to the first center point is selected for analysis and matching. By determining whether the first center point is in the head detection frame where the target second center point is located, it is determined whether the person belongs to the same person. This is in line with actual logic, the result is more intuitive, and the computational complexity is low. At the same time, the head detection frame that has not been successfully matched is also regarded as a new second combination. The detection result of each second combination is determined according to the effective state of the actual head and face. The full-image detection frame is increased, and the error accumulation in continuous tracking is reduced. By matching the face and head in the full-image detection, there is no need to introduce an additional matching algorithm, which reduces the overall complexity and computational complexity, avoids additional power consumption, and ensures the tracking effect.
[0242] Based on the above embodiment, the process of matching the tracking result of the first assembly with the detection result of the second assembly can be explained. Figure 5 As shown, the process of matching the tracking result of the first assembly with the detection result of the second assembly includes but is not limited to the following steps:
[0243] S501, traverse the second assembly to determine the second head frame in the currently traversed second assembly.
[0244] Optionally, since the current image frame may include more than one second combination, and the second combination may include only a head detection frame, or may include a head detection frame and a face detection frame belonging to the same person; therefore, the second combination is traversed to determine the second head frame in the currently traversed second combination, wherein the second head frame is also the head detection frame in the currently traversed second combination.
[0245] S502, obtaining the remaining unmatched first assemblies corresponding to the currently traversed second assemblies, and traversing the remaining unmatched first assemblies to determine the first head frame in the currently traversed first assemblies.
[0246] The second combination is matched with the first combination, and the remaining unmatched first combinations corresponding to the currently traversed second combination are obtained.
[0247] In some implementations, the remaining unmatched first combinations are grouped into a first set, and each first combination in the first set is traversed to determine the first head frame in the currently traversed first combination, where the first head frame is also the head detection frame in the currently traversed first combination.
[0248] S503: Obtain an intersection-over-union ratio between the second head frame currently traversed and the first head frame.
[0249] After determining the second head frame and the first head frame that are currently traversed, the intersection-and-union ratio between the second head frame and the first head frame that are currently traversed is obtained, wherein the intersection-and-union ratio is the ratio of the area of the intersection region between the head frame of the first combination and the head frame of the second combination to the area of the union region. The intersection-and-union ratio reflects the similarity between the second head frame and the first head frame. The closer the intersection-and-union ratio is to 1, the greater the similarity between the second head frame and the first head frame.
[0250] Optionally, the intersection-over-union ratio may be calculated based on the areas of the second head frame and the first head frame, or may be calculated based on the number of pixels in the second head frame and the first head frame.
[0251] S504: If the intersection-and-union ratio is greater than or equal to a preset threshold, it is determined that the currently traversed second combination successfully matches the currently traversed first combination.
[0252] Optionally, a preset threshold is set. When the intersection-over-intersection ratio between the first head frame and the second head frame is greater than or equal to the preset threshold, it indicates that the first head frame and the second head frame are more similar. The larger the intersection area between the first head frame and the second head frame, the more successful the match between the second combination currently traversed and the first combination currently traversed is. Optionally, the threshold can be set to a value between 0.5 and 1, such as 0.5 or 0.6.
[0253] S505: If the intersection-and-union ratio is less than a preset threshold, it is determined that the currently traversed second combination fails to match the currently traversed first combination, and the next remaining unmatched first combination is traversed.
[0254] Optionally, when the intersection-over-union ratio between the first head frame and the second head frame is less than a preset threshold, it means that the similarity between the first head frame and the second head frame is small, and the intersection area between the first head frame and the second head frame is smaller, so it is determined that the currently traversed second combination fails to match the currently traversed first combination.
[0255] Further, the next remaining unmatched first combination in the first set is traversed, and the currently traversed second combination is matched with the currently traversed remaining unmatched first combination, until a first combination that successfully matches the currently traversed second combination is determined.
[0256] S506: Determine the tracking object of the second combination according to the matching result, and continue to track the tracking object in the next image frame.
[0257] Optionally, the matching result includes a successful match and a failed match.
[0258] In some implementations, when the first assembly and the second assembly are successfully matched, the tracker of the currently traversed first assembly is determined as the tracker of the currently traversed second assembly; and the tracking result of the tracker of the currently traversed second assembly is determined based on the tracking result of the tracker of the currently traversed first assembly. In other words, when the first assembly and the second assembly are successfully matched, the tracker of the first assembly is used as the tracker of the second assembly, and correspondingly, the tracking result of the tracker of the first assembly is used as the tracking result of the tracker of the second assembly.
[0259] In some other implementations, if the currently traversed second assembly does not match any of the first assembly, a corresponding tracker is created for the currently traversed second assembly; and the tracking result of the created tracker is determined according to the valid status of the face identifier and the head identifier in the currently traversed second assembly. That is, when the currently traversed second assembly does not successfully match the first assembly, a tracker is created as the tracker of the second assembly; and the tracking result of the tracker is the valid status of the face identifier and the head identifier in the second assembly, for example, if the face identifier in the second assembly is invalid and the head identifier is valid, then the tracking result of the tracker of the second assembly is that the face identifier is invalid and the head identifier is valid.
[0260] Furthermore, according to the tracking result of the pursuer corresponding to the second assembly and the valid status of the face mark and the head mark in the detection result of the second assembly, the tracking object of the second assembly is determined, and the tracking object is subsequently tracked.
[0261] Optionally, if the tracking result corresponding to the second combination is that the face identification is invalid and the head identification is valid, but the face identification in the detection result of the second combination is valid, it is determined that the tracking object of the tracker of the second combination is a face, and the tracker is switched to face tracking to track the face in the next image frame.
[0262] Optionally, the valid status of the face identifier and the head identifier of the tracker can be reset to the face identifier is valid and the head identifier is valid; the identifier of the tracker corresponding to the second assembly is maintained unchanged. That is, when the tracking result corresponding to the second assembly is that the face identifier is invalid and the head identifier is valid, but the face identifier in the detection result of the second assembly is valid, the tracking result of the tracker corresponding to the second assembly is updated to the face identifier is valid and the head identifier is valid; but the tracking object of the second assembly is still a face, and the tracker is switched to face tracking to track the face in the next image frame.
[0263] For example, it is assumed that there are a first assembly a1, a first assembly b1, and a first assembly c1, and there are a second assembly a2, a second assembly b2, and a second assembly c2. The second assembly is traversed, assuming that the second assembly currently traversed is the second assembly b2, the second head frame in the second assembly b2 is determined, and if the corresponding remaining unmatched first assemblies only include the first assembly b1 and the first assembly c1, the remaining unmatched first assemblies b1 and the first assembly c1 are traversed, and if the first assembly currently traversed is the first assembly b1, the first head frame in the first assembly b1 is determined; further, the intersection-and-union ratio between the second head frame of the second assembly b2 and the first head frame of the first assembly b1 is obtained, and when the intersection-and-union ratio is greater than or equal to a preset threshold, it is determined that the second assembly b2 and the first assembly b1 are matched successfully; when the intersection-and-union ratio is less than the preset threshold, it is determined that the second assembly b2 and the first assembly b1 are matched unsuccessfully, and the second assembly b2 continues to be matched with the first assembly c1 until all remaining unmatched first assemblies are traversed.
[0264] In the embodiment of the present application, matching is performed by using the intersection-and-union ratio between the first head frame and the second head frame, which is simple to calculate and more efficient; analysis is performed using the head frames between the first combination and the second combination, which is more reference-based, avoids overly complex calculations, reduces additional power consumption, and achieves higher matching efficiency while ensuring tracking effects.
[0265] Figure 6 A schematic diagram of a video tracking method provided in an embodiment of the present application. Figure 6 As shown, the method includes but is not limited to the following steps:
[0266] S601, obtaining a first combination of a face and a head in a current image frame.
[0267] In the embodiment of the present application, the implementation method of step S601 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0268] S602, determining the tracking result of the same assembly in the previous image frame corresponding to the first assembly.
[0269] In the embodiment of the present application, the implementation method of step S602 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0270] S603, determining a tracking object of the first assembly according to the tracking result of the same assembly in the previous image frame, tracking the tracking object, and obtaining a tracking result of the first assembly.
[0271] In the embodiment of the present application, the implementation method of step S603 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0272] S604: When the current image frame is a full-image detection frame, perform full-image detection of a face and a head on the current image frame to obtain a face detection frame and a head detection frame in the current image frame.
[0273] In the embodiment of the present application, the implementation method of step S604 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0274] S605: traverse the face detection frame to obtain the first center point of the currently traversed face detection frame.
[0275] In the embodiment of the present application, the implementation method of step S605 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0276] S606: Determine a set of remaining unmatched head detection frames corresponding to the currently traversed face detection frame, and find the second center point of each head detection frame in the set.
[0277] In the embodiment of the present application, the implementation method of step S606 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0278] S607, obtaining a second center point that is closest to the first center point as the target second center point.
[0279] In the embodiment of the present application, the implementation method of step S607 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0280] S608: If the first center point is located in the head detection frame where the target second center point is located, determine that the currently traversed face detection frame matches the head detection frame where the target second center point is located.
[0281] In the embodiment of the present application, the implementation method of step S608 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0282] S609: If the first center point is not located in the head detection frame where the target second center point is located, it is determined that the currently traversed face detection frame fails to match.
[0283] In the embodiment of the present application, the implementation method of step S609 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0284] S610, diffusing outward based on the attribute information of the currently traversed face detection frame to determine a corresponding head detection frame, wherein the currently traversed face detection frame and the head detection frame determined by diffusion belong to the same person.
[0285] In the embodiment of the present application, the implementation method of step S610 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0286] S611, combining the faces and heads belonging to the same person to obtain a second combination and a valid state of the face identification and the head identification of the second combination as the detection result of the current image frame.
[0287] In the embodiment of the present application, the implementation method of step S611 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0288] S612, traverse the second assembly to determine the second head frame in the currently traversed second assembly.
[0289] In the embodiment of the present application, the implementation method of step S612 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0290] S613, obtaining the remaining unmatched first assemblies corresponding to the currently traversed second assemblies, and traversing the remaining unmatched first assemblies to determine the first head frame in the currently traversed first assemblies.
[0291] In the embodiment of the present application, the implementation method of step S613 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0292] S614, obtaining an intersection-over-union ratio between the second head frame currently traversed and the first head frame.
[0293] In the embodiment of the present application, the implementation method of step S614 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0294] S615: If the intersection-and-union ratio is greater than or equal to the preset threshold, it is determined that the currently traversed second combination successfully matches the currently traversed first combination.
[0295] In the embodiment of the present application, the implementation method of step S615 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0296] S616: If the intersection-and-union ratio is less than a preset threshold, it is determined that the currently traversed second combination fails to match the currently traversed first combination, and the next remaining unmatched first combination is traversed.
[0297] In the embodiment of the present application, the implementation method of step S616 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0298] S617, determining the tracking object of the second combination according to the matching result, and continuing to track the tracking object in the next image frame.
[0299] In the embodiment of the present application, the implementation method of step S617 can be implemented by any of the methods in the embodiments of the present disclosure, which is not limited here and will not be repeated.
[0300] In an embodiment of the present application, a first combination of a face and a head in a current image frame is obtained, the tracking result of the first combination in the previous image frame is determined, and the tracking object of the current image frame is determined and tracked according to the tracking result of the previous image frame to obtain the tracking result of the first combination of the current image frame; when the current image frame is a full-image detection frame, the head detection frame and the face detection frame in the current image frame are obtained to obtain a second combination, and the detection result of the second combination is determined, and the head detection frame in the first combination and the second combination are matched according to the intersection and union ratio, to determine whether the first combination and the second combination are matched successfully, thereby determining the tracking object and tracking the next image frame. In an embodiment of the present application, a tracking object is determined according to the tracking result of the previous image frame, and tracking is performed based on the tracking object to obtain a tracking result of the first combination, so that the tracking result of the current image frame is more reasonable; a full-image detection frame is set to improve the efficiency of video tracking and avoid the redundancy of frame-by-frame detection and analysis; all head detection frames and face detection frames in the full-image detection frame are obtained, and all head detection frames and face detection frames are matched to obtain the corresponding second combination, without introducing additional recognition algorithms and matching algorithms, and the processing efficiency is higher; further, based on the analysis of the head detection frames in the first combination and the second combination, the intersection-union ratio between the head detection frames is used to determine whether the match is successful, thereby avoiding overly complex calculations, reducing additional power consumption, and the overall tracking efficiency is higher and the tracking effect is better.
[0301] Figure 7 The schematic diagram of the structure of a video tracking device provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, the video tracking device 700 includes:
[0302] The first acquisition module 701 is used to acquire a first combination of a face and a head in the current image frame, and perform face tracking or head tracking on each first combination to obtain a tracking result of the first combination;
[0303] The second acquisition module 702 is used to perform full-image detection of a face and a head on the current image frame when the current image frame is a full-image detection frame, and obtain a detection result of a second combination of a face and a head in the current image frame;
[0304] A matching module 703, used for matching the tracking result of the first assembly with the detection result of the second assembly;
[0305] The tracking module 704 is used to determine the tracking object of the second combination according to the matching result, and continue to track the tracking object in the next image frame.
[0306] In some implementations, the first acquisition module 701 is configured to:
[0307] Determine the tracking result of the same assembly in the previous image frame corresponding to the first assembly;
[0308] According to the tracking result of the same assembly in the previous image frame, the tracking object of the first assembly is determined, and the tracking object is tracked to obtain the tracking result of the first assembly.
[0309] In some implementations, the first acquisition module 701 is configured to:
[0310] If the tracking result of the same assembly is that face tracking is valid, it is determined that the tracking object of the first assembly is a face, and face tracking is performed on the first assembly;
[0311] If the face tracking is successful, the face frame information and the head frame information are updated and it is determined that the face identifier and the head identifier are both valid, thereby obtaining the tracking result of the first combination;
[0312] If the face tracking fails, the process switches to head tracking of the first assembly to obtain the tracking result of the first assembly.
[0313] In some implementations, the first acquisition module 701 is configured to:
[0314] If the tracking result of the same combination is that face tracking is invalid, it is determined that the tracking object of the first combination is a head, and head tracking is performed on the first combination to obtain the tracking result of the current image frame.
[0315] In some implementations, the first acquisition module 701 is configured to:
[0316] If the head tracking is successful, the head frame information is updated and it is determined that the face identifier is invalid and the head identifier is valid, and the tracking result of the first combination is obtained;
[0317] If the head tracking fails, it is determined that the face identifier is invalid and the head identifier is invalid, and the tracking result of the first combination is obtained.
[0318] In some implementations, the apparatus 700 is further configured to:
[0319] If the tracking of the first assembly fails, the tracker corresponding to the first assembly is destroyed.
[0320] In some implementations, the second acquisition module 702 is configured to:
[0321] Perform full-image detection of faces and heads on the current image frame, and obtain face detection frames and head detection frames in the current image frame;
[0322] The face detection frame and the head detection frame are matched to determine the faces and heads belonging to the same person in the current image frame, and the faces and heads belonging to the same person are combined to obtain a second combination and the valid state of the face identification and the head identification of the second combination as the detection result of the current image frame.
[0323] In some implementations, the second acquisition module 702 is configured to:
[0324] Traverse the face detection frame and obtain the first center point of the currently traversed face detection frame;
[0325] Determine the remaining unmatched head detection frame set corresponding to the currently traversed face detection frame, and the second center point of each head detection frame in the set;
[0326] A head detection frame matching the currently traversed face detection frame is determined according to the first center point and the second center point, wherein the currently traversed face detection frame and the matched head detection frame belong to the same person.
[0327] In some implementations, the second acquisition module 702 is configured to:
[0328] Obtain the second center point closest to the first center point as the target second center point;
[0329] If the first center point is located in the head detection frame where the second center point of the target is located, it is determined that the currently traversed face detection frame matches the head detection frame where the second center point of the target is located.
[0330] In some implementations, the apparatus 700 is further configured to:
[0331] If the first center point is not located in the head detection frame where the target second center point is located, it is determined that the currently traversed face detection frame fails to match;
[0332] Based on the attribute information of the currently traversed face detection frame, the corresponding head detection frame is diffused outward, wherein the currently traversed face detection frame and the diffused determined head detection frame belong to the same person.
[0333] In some implementations, the second acquisition module 702 is configured to:
[0334] The face detection frame and the head detection frame belonging to the same person are combined to obtain a second combination.
[0335] In some implementations, the second acquisition module 702 is further configured to:
[0336] Save the relevant information of the face detection frame and the head detection frame in the second combination;
[0337] The face identification and head identification of the second combination are valid at the same time.
[0338] In some implementations, the apparatus 700 is further configured to:
[0339] After the face detection frame matching is completed, there are still remaining unmatched head detection frames, and the second combination is directly generated based on the remaining unmatched head detection frames.
[0340] In some implementations, the apparatus 700 is further configured to:
[0341] Saving the relevant information of the remaining unmatched head detection frames in the second combination;
[0342] The head mark of the second combination is valid and the face mark is invalid.
[0343] In some implementations, the matching module 703 is configured to:
[0344] Traversing the second assembly to determine the second head frame in the second assembly currently traversed;
[0345] Obtain the remaining unmatched first assemblies corresponding to the currently traversed second assemblies, and traverse the remaining unmatched first assemblies to determine the first head frame in the currently traversed first assemblies;
[0346] Get the intersection-and-union ratio between the second head frame currently traversed and the first head frame;
[0347] If the intersection-and-union ratio is greater than or equal to a preset threshold, it is determined that the currently traversed second combination successfully matches the currently traversed first combination;
[0348] If the intersection-and-union ratio is less than a preset threshold, it is determined that the currently traversed second combination fails to match the currently traversed first combination, and the next remaining unmatched first combination is traversed.
[0349] In some implementations, the apparatus 700 is further configured to:
[0350] If the second assembly currently traversed does not match any first assembly, create a corresponding tracker for the second assembly currently traversed;
[0351] The tracking result of the created tracker is determined according to the valid states of the face identifier and the head identifier in the second combination currently traversed.
[0352] In some implementations, the matching module 703 is further configured to:
[0353] Determine the tracker of the first assembly currently traversed as the tracker of the second assembly currently traversed;
[0354] According to the tracking result of the tracker of the currently traversed first assembly, the tracking result of the tracker of the currently traversed second assembly is determined.
[0355] In some implementations, the tracking module 704 is configured to:
[0356] According to the tracking result of the pursuer corresponding to the second assembly and the valid status of the face mark and the head mark in the detection result of the second assembly, the tracking object of the second assembly is determined, and the tracking object is subsequently tracked.
[0357] In some implementations, the tracking module 704 is configured to:
[0358] If the tracking result corresponding to the second assembly is that the face identifier is invalid and the head identifier is valid, but the face identifier in the detection result of the second assembly is valid, it is determined that the tracking object of the tracker is a face, and the mode is switched to face tracking.
[0359] In some implementations, the apparatus 700 is further configured to:
[0360] Reset the valid status of the face identification and head identification of the tracker to face identification valid and head identification valid;
[0361] The identification of the tracker corresponding to the second assembly is maintained unchanged.
[0362] In some implementations, the apparatus 700 is further configured to:
[0363] In the case that the current image frame is not a full-image detection frame, the tracking object of the first assembly is determined according to the tracking result of the first assembly, and the tracking object is continuously tracked in the next image frame.
[0364] In the embodiment of the present application, a first combination of a face and a head in the current image frame is obtained, and the face tracking or head tracking is performed on the first combination to obtain a corresponding tracking result; when the current image frame is a full-image detection frame, the full-image detection is performed on the current image frame to obtain the detection result of the second combination, the tracking result of the first combination is matched with the detection result of the second combination, the tracking object of the second combination is determined based on the matching result, and the tracking object is tracked in the next image frame. In the embodiment of the present application, the face and the head are tracked as a combination, which avoids the final matching of the face and the head and reduces the amount of calculation. In order to improve the detection efficiency, the full-image detection frame is set to perform full-image detection on the image, and it is determined whether the detection result during the full-image detection matches the tracking result of the normal tracking. The tracking object is adjusted according to the matching situation, which improves the analysis efficiency while ensuring the accuracy of the tracking object and the tracking effect.
[0365] Figure 8 FIG. 1 is a block diagram of an electronic device according to an exemplary embodiment. Figure 8 As shown, the electronic device 800 includes a video tracking device 700. The electronic device may be a mobile electronic device or a non-mobile electronic device. For example, the mobile electronic device may be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc., and the non-mobile electronic device may be a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiments of the present application.
[0366] According to an embodiment of the present application, an electronic device is also provided, including: a processor; and a memory for storing instructions executable by the processor, wherein the processor is configured to execute the instructions to implement the video tracking method as described above.
[0367] In order to implement the above embodiments, the present application also proposes a storage medium.
[0368] When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the video tracking method as described above.
[0369] In order to implement the above embodiments, the present application also provides a computer program product.
[0370] When the computer program product is executed by a processor of an electronic device, the electronic device is enabled to execute the method as described above.
[0371] Fig. 9 The present invention is a block diagram of an electronic device according to an exemplary embodiment. Fig. 9 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0372] like Fig. 9 As shown, the electronic device 900 includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a memory 906 to a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The processor 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.
[0373] The following components are connected to the I / O interface 905: a memory 906 including a hard disk, etc.; and a communication part 907 including a network interface card such as a LAN (Local Area Network) card, a modem, etc., and the communication part 907 performs communication processing via a network such as the Internet; a drive 908 is also connected to the I / O interface 905 as needed.
[0374] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 907. When the computer program is executed by the processor 901, the above-mentioned functions defined in the method of the present application are executed.
[0375] In an exemplary embodiment, a storage medium including instructions is also provided, such as a memory including instructions, and the instructions can be executed by the processor 901 of the electronic device 900 to complete the above method. Optionally, the storage medium can be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0376] In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.
[0377] Fig.10 The present invention is a structural block diagram of an electronic device according to an exemplary embodiment. Fig.10 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present application. Fig.10 As shown, the electronic device 1000 includes a processor 1001 and a memory 1002. The memory 1002 is used to store program codes, and the processor 1001 is connected to the memory 1002 and is used to read the program codes from the memory 1002 to implement the video tracking method in the above embodiment.
[0378] Optionally, the number of processors 1001 may be one or more.
[0379] Optionally, the electronic device may further include an interface 1003, and the number of the interface 1003 may be multiple. The interface 1003 may be connected to an application program and may receive data from an external device such as a sensor.
[0380] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.
[0381] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. A video tracking method, characterized in that: The method comprises: Acquire a first combination of a face and a head in the current image frame, and perform face tracking or head tracking on each of the first combinations to obtain a tracking result of the first combination; When the current image frame is a full-image detection frame, performing full-image detection of a face and a head on the current image frame to obtain a detection result of a second combination of a face and a head in the current image frame; matching the tracking result of the first assembly with the detection result of the second assembly; The tracking object of the second combination is determined according to the matching result, and the tracking object is continued to be tracked in the next image frame.
2. The method according to claim 1, characterized in that The performing face tracking or head tracking on each of the first assemblies to obtain a tracking result of the first assemblies includes: Determine a tracking result of the same assembly in a previous image frame corresponding to the first assembly; According to the tracking result of the same combination in the previous image frame, a tracking object of the first combination is determined, and the tracking object is tracked to obtain the tracking result of the first combination.
3. The method according to claim 2, characterized in that The step of determining a tracking object of the first assembly according to the tracking result of the same assembly in the previous image frame, tracking the tracking object, and obtaining the tracking result of the first assembly includes: If the tracking result of the same assembly is valid for face tracking, determining that the tracking object of the first assembly is a face, and performing face tracking on the first assembly; If the face tracking is successful, the face frame information and the head frame information are updated and it is determined that the face identifier and the head identifier are both valid, thereby obtaining the tracking result of the first combination; If the face tracking fails, the process switches to head tracking of the first assembly to obtain a tracking result of the first assembly.
4. The method according to claim 2, characterized in that: The step of determining a tracking object of the first assembly according to the tracking result of the same assembly in the previous image frame, tracking the tracking object, and obtaining the tracking result of the current image frame includes: If the tracking result of the same assembly is that face tracking is invalid, it is determined that the tracking object of the first assembly is a head, and head tracking is performed on the first assembly to obtain the tracking result of the current image frame.
5. The method according to claim 3 or 4, characterized in that: Performing head tracking on the first assembly to obtain a tracking result of the current image frame includes: If the head tracking is successful, the head frame information is updated and it is determined that the face identifier is invalid and the head identifier is valid, and the tracking result of the first combination is obtained; If the head tracking fails, it is determined that the face identifier is invalid and the head identifier is invalid, and the tracking result of the first combination is obtained.
6. The method according to claim 5, characterized in that The method further comprises: If the tracking of the first assembly fails, the tracker corresponding to the first assembly is destroyed.
7. The method according to claim 1, characterized in that The whole-image detection of the face and the head of the current image frame is performed to obtain the detection result of the second combination of the face and the head in the current image frame, including: Performing full-image detection of a face and a head on the current image frame to obtain a face detection frame and a head detection frame in the current image frame; The face detection frame and the head detection frame are matched to determine the faces and heads belonging to the same person in the current image frame, and the faces and heads belonging to the same person are combined to obtain the second combination and the valid state of the face identification and the head identification of the second combination as the detection result of the current image frame.
8. The method according to claim 7, characterized in that The matching of the face detection frame and the head detection frame to determine the face and the head belonging to the same person in the current image frame includes: Traversing the face detection frame to obtain a first center point of the currently traversed face detection frame; Determine a set of remaining unmatched head detection frames corresponding to the currently traversed face detection frame, and a second center point of each head detection frame in the set; A head detection frame that matches the currently traversed face detection frame is determined according to the first center point and the second center point, wherein the currently traversed face detection frame and the matched head detection frame belong to the same person.
9. The method according to claim 8, characterized in that The step of determining a head detection frame that matches the currently traversed face detection frame according to the first center point and the second center point includes: Acquire a second center point that is closest to the first center point as the target second center point; If the first center point is located in the head detection frame where the second center point of the target is located, it is determined that the currently traversed face detection frame matches the head detection frame where the second center point of the target is located.
10. The method according to claim 9, characterized in that The method further comprises: If the first center point is not located in the head detection frame where the target second center point is located, determining that the currently traversed face detection frame fails to match; Based on the outward diffusion of the attribute information of the currently traversed face detection frame, a corresponding head detection frame is determined, wherein the currently traversed face detection frame and the head detection frame determined by diffusion belong to the same person.
11. The method according to claim 10, characterized in that The step of combining the face and head of the same person to obtain a second combination of the face and head includes: The face detection frame and the head detection frame belonging to the same person are combined to obtain the second combination.
12. The method according to claim 11, characterized in that After obtaining the second combination, the method further comprises: Saving relevant information of the face detection frame and the head detection frame in the second combination; The face identification and the head identification marking the second combination are valid at the same time.
13. The method according to claim 11, characterized in that The method further comprises: After the face detection frame matching is completed, there are still remaining unmatched head detection frames, and the second combination is directly generated based on the remaining unmatched head detection frames.
14. The method according to claim 13, characterized in that The method further comprises: Saving relevant information of the remaining unmatched head detection frames in the second combination; The head mark of the second combination is valid and the face mark is invalid.
15. The method according to claim 7, characterized in that The matching of the tracking result of the first assembly and the detection result of the second assembly includes: Traversing the second combination to determine a second head frame in the second combination currently traversed; Acquire the remaining unmatched first combinations corresponding to the currently traversed second combinations, and traverse the remaining unmatched first combinations to determine the first head frame in the currently traversed first combinations; Obtaining the intersection-and-union ratio between the second head frame currently traversed and the first head frame; If the intersection-and-union ratio is greater than or equal to a preset threshold, it is determined that the currently traversed second combination successfully matches the currently traversed first combination; If the intersection-and-union ratio is less than the preset threshold, it is determined that the currently traversed second combination fails to match the currently traversed first combination, and the next remaining unmatched first combination is traversed.
16. The method according to claim 15, characterized in that The method further comprises: If the second assembly currently traversed does not match any first assembly, create a corresponding tracker for the second assembly currently traversed; The tracking result of the created tracker is determined according to the valid status of the face identifier and the head identifier in the second combination currently traversed.
17. The method according to claim 15, characterized in that After determining that the currently traversed second combination successfully matches the currently traversed first combination, the method further includes: Determine the tracker of the first assembly currently traversed as the tracker of the second assembly currently traversed; The tracking result of the tracker of the currently traversed first assembly is determined according to the tracking result of the tracker of the currently traversed second assembly.
18. The method according to claim 16 or 17, characterized in that Determining a tracking object of the second combination according to the matching result, and subsequently tracking the tracking object, including: According to the tracking result of the pursuer corresponding to the second assembly and the valid status of the face mark and the head mark in the detection result of the second assembly, the tracking object of the second assembly is determined, and the tracking object is subsequently tracked.
19. The method according to claim 18, characterized in that The step of determining the tracking object of the second assembly according to the tracking result of the pursuer corresponding to the second assembly and the valid status of the face mark and the head mark in the detection result of the second assembly comprises: If the tracking result corresponding to the second assembly is that the face identification is invalid and the head identification is valid, but the face identification in the detection result of the second assembly is valid, it is determined that the tracking object of the tracker is a face, and the mode is switched to face tracking.
20. The method according to claim 19, characterized in that The method further comprises: Resetting the valid status of the face identifier and the head identifier of the tracker to the face identifier is valid and the head identifier is valid; The identification of the tracker corresponding to the second assembly is maintained unchanged.
21. The method according to claim 1, characterized in that The method further comprises: In the case that the current image frame is not a full-image detection frame, a tracking object of the first assembly is determined according to the tracking result of the first assembly, and the tracking object is continuously tracked in the next image frame.
22. A video tracking device, characterized in that: include: A first acquisition module is used to acquire a first combination of a face and a head in a current image frame, and perform face tracking or head tracking on each of the first combinations to obtain a tracking result of the first combination; A second acquisition module is used to perform full-image detection of a face and a head on the current image frame when the current image frame is a full-image detection frame, and obtain a detection result of a second combination of a face and a head in the current image frame; A matching module, used for matching the tracking result of the first assembly with the detection result of the second assembly; The tracking module is used to determine the tracking object of the second combination according to the matching result, and continue to track the tracking object in the next image frame.
23. An electronic device, characterized in that: include: The device as claimed in claim 22.
24. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method as claimed in any one of claims 1 to 21.
25. A non-transitory computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the method as claimed in any one of claims 1 to 21.
26. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 21 when being executed by a processor.
Citation Information
Patent Citations
Method for rapidly detecting human face based on video
CN101350062A
Real-time video face recognition method and system based on vision tracking technology
CN107609497A
Rapid face tracking method based on face detection
CN110688940A
Method and system for multi-feature joint video tracking based on face recognition
CN111178218A
Obfect tracking system and method thereof
KR1020120068150A
