Human image tracking method and apparatus, and non-volatile storage medium and electronic device

Abstract

The present application relates to a human image tracking method and apparatus, and a non-volatile storage medium and an electronic device. The method comprises: acquiring a current image frame, and determining tracking information of the current image frame; determining a target face recognition region and a target head and shoulder recognition region from the current image frame; on the basis of a degree-of-overlap indicator between the target face recognition region and the target head and shoulder recognition region, and a first preset condition, respectively determining a first face recognition region and a first head and shoulder recognition region, and a second face recognition region and a second head and shoulder recognition region; acquiring a historical image frame corresponding to the current image frame; on the basis of a historical recognition region, determining a prediction region of a tracking object in the current image frame; matching a second region with the prediction region, so as to obtain a similarity indicator; and determining, from the second region, a target region with a similarity indicator between same and the prediction region meeting a second preset condition, and on the basis of the target region, updating tracking information of the prediction region.

Description

Portrait tracking method and device, nonvolatile storage medium and electronic device

[0001] Related applications

[0002] The present application claims priority to the Chinese patent application No. 2024112671551, filed on September 10, 2024, and entitled "Portrait tracking method and device, nonvolatile storage medium and electronic device", the contents of which are hereby incorporated by reference in its entirety. TECHNICAL FIELD

[0003] The present application relates to the technical field of computer vision, in particular to a portrait tracking method and device, a nonvolatile storage medium and an electronic device. BACKGROUND

[0004] Portrait tracking is a computer vision technology used to detect, identify and track the position and movement of people in a video stream. This technology has wide applications in security monitoring, smart home, autonomous driving, augmented reality and many other fields.

[0005] In a monitoring system, portrait tracking first identifies the portrait in the monitoring area through face detection, body detection, head-shoulder detection and other technologies, and tracks the target in real time as it moves, ensuring continuous tracking of the target's position and dynamics. Subsequently, the captured face and body images are subjected to quality filtering and optimization processing to ensure that clear and high-quality capture images are obtained. These processed images are then sent to a face recognition or body recognition system for further face and body comparison and identification.

[0006] However, the current portrait tracking technology has the problem of poor effect. SUMMARY

[0007] The present application provides a portrait tracking method and device, a nonvolatile storage medium and an electronic device.

[0008] According to an aspect of an embodiment of the present application, a portrait tracking method is provided, comprising:

[0009] obtaining a current image frame and determining tracking information of the current image frame; the current image frame includes a tracking object; the tracking information at least includes an activity track of the tracking object;

[0010] determining a target face recognition area and a target head-shoulder recognition area in the current image frame;

[0011] determining a first face recognition area and a first head-shoulder recognition area, and a second face recognition area and a second head-shoulder recognition area based on a coincidence degree index of the target face recognition area and the target head-shoulder recognition area, and a first preset condition;

[0012] obtain a historical image frame corresponding to the current image frame; the historical image frame includes a historical recognition region of the tracking object;

[0013] determine a prediction region of the tracking object in the current image frame according to the historical recognition region; the prediction region includes a face recognition region and a head-shoulder recognition region;

[0014] match the second region with the prediction region to obtain a similarity index; the second region includes the first region, the second face recognition region, and the second head-shoulder recognition region; the first region includes the first face recognition region and the first head-shoulder recognition region;

[0015] determine a target region in the second region that satisfies a second preset condition with the similarity index of the prediction region, and update tracking information of the prediction region according to the target region.

[0016] In some embodiments, the determination of the first face recognition region and the first head-shoulder recognition region, and the second face recognition region and the second head-shoulder recognition region based on the coincidence index of the target face recognition region and the target head-shoulder recognition region, and the first preset condition respectively includes:

[0017] determine, as the first face recognition region, a region in the target face recognition region that satisfies the first preset condition with the coincidence index;

[0018] determine, as the first head-shoulder recognition region, a region in the target head-shoulder recognition region that satisfies the first preset condition with the coincidence index;

[0019] determine, as the second face recognition region, a region in the target face recognition region that does not satisfy the first preset condition with the coincidence index;

[0020] determine, as the second head-shoulder recognition region, a region in the target head-shoulder recognition region that does not satisfy the first preset condition with the coincidence index.

[0021] In some embodiments, the coincidence index includes an intersection over union of the target face recognition region and the target head-shoulder recognition region; and the first preset condition includes that the intersection over union is greater than a preset threshold.

[0022] In some embodiments, the matching of the second region with the prediction region to obtain the similarity index includes:

[0023] match the first region and the second face recognition region in the second region with the face recognition region in the first prediction region, and match the target head-shoulder recognition region in the second region with the head-shoulder recognition region in the first prediction region to obtain a first matching result; the first prediction region is a prediction region in a non-loss state of tracking information;

[0024] match the second region that fails to match in the first matching result with the first target region to obtain a second matching result; the first target region includes: the second head-shoulder recognition region and the prediction region that fails to match in the first matching result;

[0025] match the first region and the second face recognition region in the second region that fails to match in the second matching result with the second target region to obtain a third matching result; the second target region includes: the prediction region that fails to match in the second matching result and the prediction region in the first matching result in a loss state of tracking information;

[0026] match the second region that fails to match in the third matching result with the second prediction region to obtain a fourth matching result; the second prediction region is a prediction region that fails to match in the third matching result, has a time length not exceeding a preset time length in a loss state of tracking information, and has an identification quality index of the head-shoulder recognition region satisfying a third preset condition; the identification quality index of the head-shoulder recognition region is used to represent the definition of the head-shoulder of the tracked object;

[0027] determine a similarity index according to the first matching result, the second matching result, the third matching result and the fourth matching result.

[0028] In some embodiments, matching the first region and the second face recognition region in the second region with the face recognition region in the first prediction region, and matching the target head-shoulder recognition region in the second region with the head-shoulder recognition region in the first prediction region to obtain the first matching result includes:

[0029] determine a first face feature of the first region and the second face recognition region in the second region, and determine a second face feature of the face recognition region;

[0030] determine a first Euclidean distance between the first face feature and the second face feature; and determine a first intersection-over-union between the first head-shoulder recognition region and the head-shoulder recognition region;

[0031] determine a first cost matrix according to the first Euclidean distance, the first intersection-over-union, a first preset weight corresponding to the first Euclidean distance, and a second preset weight corresponding to the first intersection-over-union;

[0032] determine the first matching result according to the first cost matrix and a preset matching algorithm; the preset matching algorithm at least includes a Hungarian matching algorithm.

[0033] In some embodiments, the second region that fails to match in the first matching result is matched with the first target region to obtain a second matching result, including:

[0034] A second intersection-over-union ratio between the second region that fails to match in the first matching result and the first target region is determined.

[0035] A second cost matrix is determined according to the second intersection-over-union ratio.

[0036] The second matching result is determined according to the second cost matrix and a preset matching algorithm; the preset matching algorithm at least includes a Hungarian matching algorithm.

[0037] In some embodiments, a first region in the second region that fails to match in the second matching result and the second face recognition region are matched with the second target region to obtain a third matching result, including:

[0038] Third face features of the first region in the second region that fails to match in the second matching result and the second face recognition region are determined, and fourth face features of the second target region are determined.

[0039] A second Euclidean distance between the third face features and the fourth face features is determined.

[0040] A third cost matrix is determined according to the second Euclidean distance.

[0041] The third matching result is determined according to the third cost matrix and a preset matching algorithm; the preset matching algorithm at least includes a Hungarian matching algorithm.

[0042] In some embodiments, the second region that fails to match in the third matching result is matched with the second predicted region to obtain a fourth matching result, including:

[0043] A third intersection-over-union ratio between the second region that fails to match in the third matching result and the second predicted region is determined.

[0044] A fourth cost matrix is determined according to the third intersection-over-union ratio.

[0045] The fourth matching result is determined according to the fourth cost matrix and a preset matching algorithm; the preset matching algorithm at least includes a Hungarian matching algorithm.

[0046] In some embodiments, tracking information of the predicted region is updated according to the target region, including:

[0047] If the matching success rate of the first matching result, the matching success rate of the second matching result, the matching success rate of the third matching result, or the matching success rate of the fourth matching result is greater than the preset success rate, the target region is determined as the historical recognition region corresponding to the current image frame, and the historical recognition region corresponding to the current image frame is added to the tracking information of the prediction region.

[0048] In some embodiments, after the similarity index is determined, the method further includes:

[0049] If the matching success rate of the first matching result, the matching success rate of the second matching result, the matching success rate of the third matching result, and the matching success rate of the fourth matching result are all less than the preset success rate, it is determined whether the recognition quality index of the face region and / or the recognition quality index of the head-shoulder region in the second region meet a fourth preset condition.

[0050] In a case where the recognition quality index of the face region and / or the recognition quality index of the head-shoulder region in the second region meet the fourth preset condition, tracking information corresponding to the second region is established, and the tracking information corresponding to the second region is added to a tracking information set corresponding to the current image frame. The tracking information set includes different tracking information corresponding to different tracking objects.

[0051] In some embodiments, after the similarity index is determined, the method further includes:

[0052] If the matching success rate of the first matching result, the matching success rate of the second matching result, the matching success rate of the third matching result, and the matching success rate of the fourth matching result are all less than the preset success rate, in a case where the tracking object is an uncertain object, the state of the tracking information of the prediction region is modified to an ineffective state.

[0053] If the matching success rate of the first matching result, the matching success rate of the second matching result, the matching success rate of the third matching result, and the matching success rate of the fourth matching result are all less than the preset success rate, in a case where the tracking object is a valid tracking object, it is determined whether a target time length from a first time to a zero time is a preset time length. The first time is a time when the matching success rate of the first matching result, the matching success rate of the second matching result, the matching success rate of the third matching result, and the matching success rate of the fourth matching result are all less than the preset success rate, and the zero time is a time when the current image frame is acquired.

[0054] If the target time length is the preset time length, the state of the tracking information of the prediction region is modified to a lost state, and if the target time length is less than the preset time length, the state of the tracking information of the prediction region is modified to an early lost state.

[0055] According to another aspect of the embodiments of the present application, a person image tracking device is also provided, comprising:

[0056] The first obtaining module is configured to obtain a current image frame and determine tracking information of the current image frame, wherein the current image frame comprises a tracking object, and the tracking information comprises at least a movement track of the tracking object;

[0057] The first determining module is configured to determine a target face recognition region and a target shoulder recognition region in the current image frame.

[0058] The region determining module is configured to determine a first face recognition region and a first shoulder recognition region, and a second face recognition region and a second shoulder recognition region based on a coincidence index of the target face recognition region and the target shoulder recognition region and a first preset condition.

[0059] The second obtaining module is configured to obtain a historical image frame corresponding to the current image frame, wherein the historical image frame comprises a historical recognition region of the tracking object.

[0060] The second determining module is configured to determine a predicted region of the tracking object in the current image frame based on the historical recognition region, wherein the predicted region comprises a face recognition region and a shoulder recognition region.

[0061] The matching module is configured to match the second region with the predicted region to obtain a similarity index, wherein the second region comprises the first region, the second face recognition region and the second shoulder recognition region, and the first region comprises the first face recognition region and the first shoulder recognition region.

[0062] The updating module is configured to determine a target region in the second region that satisfies a second preset condition with the similarity index of the predicted region, and update the tracking information of the predicted region based on the target region.

[0063] According to another aspect of the embodiments of the present application, a non-volatile storage medium is also provided, which comprises a stored program, wherein the program controls a device where the storage medium is located to execute the person image tracking method when the program is running.

[0064] According to another aspect of the embodiments of the present application, an electronic device is also provided, which comprises a memory and a processor, and the processor is configured to run a program stored in the memory, wherein the program executes the person image tracking method when the program is running.

[0065] According to another aspect of the embodiments of the present application, a computer program is also provided, wherein the computer program is executed by a processor to implement the person image tracking method.

[0066] According to still another aspect of the embodiments of the present application, a computer program product is also provided. The computer program product includes a non-transitory computer readable storage medium storing a computer program. The computer program, when executed by a processor, implements the above-mentioned portrait tracking method. BRIEF DESCRIPTION OF DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort based on the disclosed drawings.

[0068] FIG. 1 is a flowchart of a portrait tracking method according to an embodiment of the present application;

[0069] FIG. 2 is a flowchart of another portrait tracking method according to an embodiment of the present application;

[0070] FIG. 3 is a flowchart of tracking state jumping and falling according to an embodiment of the present application;

[0071] FIG. 4 is a flowchart of tracking state conversion according to an embodiment of the present application;

[0072] FIG. 5 is a structural diagram of a portrait tracking device according to an embodiment of the present application;

[0073] FIG. 6 is a hardware structural block diagram of a computer terminal of a portrait tracking method according to an embodiment of the present application. DETAILED DESCRIPTION

[0074] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort fall within the scope of protection of the present application.

[0075] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and in the above drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0076] Current portrait tracking technology is not effective, which can cause the following problems:

[0077] 1. Frequent switching of ID: the ID of the same person can be frequently switched, and the same target can be regarded as multiple different individuals by the system, affecting the accuracy and continuity of recognition. 2. Repeat snapshot: the system can capture multiple repeated images of the same person, increasing the complexity and resource consumption of subsequent processing, thereby reducing the overall processing efficiency and response speed. 3. Low snapshot rate: depending on the accuracy of face detection or body detection, if the face is too small or the body is too small for people far away from the camera, it is easy to miss tracking, resulting in a decrease in the snapshot rate.

[0078] Therefore, ensuring the efficiency and accuracy of portrait tracking technology is crucial for the normal operation of intelligent monitoring systems, which not only depends on the accuracy and stability of the technology itself, but also needs to fully consider various complex factors and challenges in actual application scenarios.

[0079] Current portrait tracking technology is based on face frame tracking, body frame tracking, or a combination of face and body frame tracking. However, the monitoring scene is complex and variable, and personnel are easily affected by light, obstruction, posture, etc. Each method has its own shortcomings: 1. In the case of using body tracking, when the personnel are close to the camera or the lower body is blocked, the body cannot be detected, resulting in tracking failure. 2. In the case of using face tracking, when the personnel are looking down, turning around, or blocking, the face cannot be seen, also causing the face tracking algorithm to fail. 3. In the case of combining face and body frame tracking, due to the matching method, it can also cause tracking failure. When there is a detection error in the middle of a video image frame, the face or body is not detected, resulting in ID loss.

[0080] According to the embodiment of the present application, a method embodiment of a portrait tracking method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in a different order.

[0081] FIG. 1 is a flowchart of a portrait tracking method according to an embodiment of the present application. As shown in FIG. 1, the method comprises the following steps:

[0082] In step S101, a current image frame is obtained, and tracking information of the current image frame is determined. The current image frame includes a tracking object. The tracking information at least includes an activity track of the tracking object.

[0083] For example, the current image frame can be obtained by the following method: accessing a camera, or receiving a video file, obtaining a video sequence, uniformly sampling 5 frames per second or according to actual work requirements in the video sequence to obtain a plurality of image frames, and determining the image frame currently processed as the current image frame.

[0084] It is worth noting that the tracking information of the image frame is Tracklet, which is a professional term in the field of video or object tracking and can be translated as "tracking small segment" or "track segment". Tracklet is an important component in the process of forming a complete track, which is composed of multiple continuous tracking segments belonging to the same physical target. The following is a detailed explanation of Tracklet: Tracklet refers to the short segment in the object tracking process, which connects the same object detection results in consecutive frames through data association methods. It is the basic unit of building a complete track. The detection results in Tracklet are continuous in time, i.e., they correspond to consecutive frames in the video. The detection results in Tracklet belong to the same tracking object and have the same or similar features. Multiple Tracklets can form a complete track through further data association and track combination algorithms.

[0085] In multiple object tracking (MOT), multiple targets are tracked in consecutive video frames, and each target will form a series of Tracklets, which are finally combined into a complete track.

[0086] Suppose there is a video sequence containing multiple pedestrians. In the first frame of the video, the position of each pedestrian is detected using a target detection algorithm. Then, in the subsequent frames, the pedestrians are continuously detected, and the detection results of the same pedestrian are connected through a data association method to form a Tracklet. Finally, multiple Tracklets are combined into complete tracks Tracklets, realizing the tracking of each pedestrian in the entire video sequence.

[0087] Step S102, determine the target face recognition region and the target shoulder recognition region in the current image frame;

[0088] Specifically, the intersection over union (IOU) between the detected target face recognition region and the target shoulder recognition region in the current image frame is determined to obtain a cost matrix. Using the Hungarian matching algorithm, the matching results of the target face recognition region and the target shoulder recognition region can be determined, and then the target face recognition region and the target shoulder recognition region whose coincidence index meets the first preset condition and the target face recognition region and the target shoulder recognition region whose coincidence index does not meet the first preset condition can be determined.

[0089] The coincidence index is, for example, the intersection over union (IoU). The IoU is an important measure in target detection, used to evaluate the degree of overlap between the predicted box (predicted region) and the true box (or true region). The value of IoU ranges from 0 to 1. The closer the IoU value is to 1, the higher the degree of overlap between the predicted region and the true region. The coincidence index meets the first preset condition, for example, the intersection over union is greater than 0.85.

[0090] Step S103, based on the coincidence index of the target face recognition region and the target shoulder recognition region, and the first preset condition, respectively determine the first face recognition region and the first shoulder recognition region, and the second face recognition region and the second shoulder recognition region;

[0091] Specifically, the region in the target face recognition region corresponding to the coincidence index meeting the first preset condition is determined as the first face recognition region; the region in the target shoulder recognition region corresponding to the coincidence index meeting the first preset condition is determined as the first shoulder recognition region; the region in the target face recognition region corresponding to the coincidence index not meeting the first preset condition is determined as the second face recognition region; the region in the target shoulder recognition region corresponding to the coincidence index not meeting the first preset condition is determined as the second shoulder recognition region.

[0092] Step S104, obtain the historical image frame corresponding to the current image frame; the historical image frame includes the historical recognition region of the tracked object.

[0093] For example, the historical image frame is a previous frame of the current image frame.

[0094] In step S105, a predicted region of the tracking object in the current image frame is determined according to the historical recognition region; the predicted region includes the face recognition region and the head-shoulder recognition region.

[0095] In some examples, the predicted region of the tracking object in the current image frame is determined by the following method: determining the tracking state of each tracking information in a tracking information set, and it is worth noting that the tracking information set is the Tracklets described above, wherein one tracking information corresponds to one tracking object. The tracking information in the tracking information set whose tracking state is End and NotSet is deleted, that is, the tracking object corresponding to the tracking information has not been matched for n consecutive frames before the current image frame, wherein n is a positive integer. The historical recognition region of the historical image frame in the remaining tracking information in the tracking information set is Kalman filtered, and the active position of the tracking object corresponding to the tracking information in the current image frame is predicted to obtain the predicted region.

[0096] It is to be explained that the state (tracking state) of the tracking information includes but is not limited to: 1. NotSet: the tracking ID (tracking object) is not effective. 2. Uncertain: the tracking ID is initially created, but it is not determined whether it is a valid tracking ID. 3. Confirmed: after multiple frames, it is confirmed that it is a valid tracking ID in the current image frame. 4. EarlyLost: the tracking ID disappears before the first time of pushing the image, and this mode is added to complete the pushing of the image twice. 5. Lost: the lost tracking ID. Deleted: the tracking ID has ended.

[0097] Wherein, the first time of pushing the image is the image frame which is first received and started to be processed by the tracker in the tracking process. This is the starting point of the tracking task, and the target tracking algorithm will initialize the tracking state according to the target detection result in the image, such as the position, appearance feature, etc. of the target. Subsequently, the algorithm will use these information to continue tracking the target in the subsequent frames of the video sequence. EarlyLost (early loss) refers to the target in the target tracking process, which is lost in a very short time after being tracked, which is usually caused by the target being blocked, fast motion or the limitation of the algorithm itself, etc. The target tracking algorithm needs to have a certain robustness to handle such early loss, such as through motion model prediction, historical information backtracking or redetection mechanism to reacquire and maintain the tracking of the target.

[0098] Step S106, matching the second region with the prediction region to obtain a similarity index; the second region includes the first region, the second face recognition region, and the second shoulder recognition region; the first region includes the first face recognition region and the first shoulder recognition region;

[0099] Specifically, the first face recognition region and the first shoulder recognition region can be combined to obtain the first region, and the second face recognition region and the second shoulder recognition region are added to the first region to obtain the second region.

[0100] It should be noted that the above combination (Bind) is to bind the attributes belonging to the same tracking object together. That is, the first face recognition region and the first shoulder recognition region are Bound to obtain TrackInput, and the unmatched target face recognition region and the target shoulder recognition region are also added to TrackInput.

[0101] Step S107, determining a target region in the second region that satisfies a second preset condition with the similarity index of the prediction region, and updating the tracking information of the prediction region according to the target region.

[0102] Specifically, the region in the second region that satisfies the second preset condition with the similarity index of the prediction region can be determined as the target region. Exemplarily, a cascaded matching manner is adopted, a multi-stage matching strategy is adopted to realize face tracking, and the second region is matched with the prediction region. Specifically, different matching methods can be used four times in succession. The first time is to perform the first accurate matching by combining the IoU and the face feature weight self-adaption of the face recognition region and the shoulder recognition region of the current image frame and the face recognition region and the shoulder recognition region corresponding to the ID of the tracking object tracked to the track, to ensure the correct matching rate of the clear tracking ID. Then, the supplementary matching is performed by the shoulder IOU, and finally, the third matching is performed by the face feature and the fourth matching is performed by the IOU of the shoulder recognition region to find the lost tracking ID, so as to ensure the tracking accuracy and better reduce the repetition rate of tracking.

[0103] According to the above steps, the current image frame is obtained, and tracking information of the current image frame is determined, wherein the current image frame includes a tracking object, and the tracking information at least includes an activity track of the tracking object; a target face recognition area in the current image frame is determined, a first shoulder recognition area with a coincidence index of the target face recognition area satisfying a first preset condition and a second shoulder recognition area with the coincidence index of the target face recognition area not satisfying the first preset condition are determined; the target face recognition area and the first shoulder recognition area are combined to obtain a first area, and the second shoulder recognition area is added to the first area to obtain a second area; a historical image frame corresponding to the current image frame is obtained, wherein the historical image frame includes a historical recognition area of the tracking object; a prediction area of the tracking object in the current image frame is determined according to the historical recognition area, wherein the prediction area includes a face recognition area and a shoulder recognition area; the second area is matched with the prediction area to obtain a similarity index, a target area in the second area with the similarity index of the prediction area satisfying a second preset condition is determined, and the tracking information of the prediction area is updated according to the target area, so as to reduce the influence of factors such as light, shielding and posture on portrait tracking, thereby realizing the technical effect of improving the accuracy of portrait tracking.

[0104] The steps shown in FIG. 1 are exemplarily described and explained as follows.

[0105] According to some optional embodiments of the present application, the prediction area is matched with the second area to obtain the similarity index, which can be realized by the following method.

[0106] In step S1061, the first area and the second face recognition area in the second area are matched with the face recognition area in the first prediction area, and the target shoulder recognition area in the second area is matched with the shoulder recognition area in the first prediction area to obtain a first matching result, wherein the first prediction area is a prediction area with non-lost tracking information.

[0107] In step S1062, the second area with a matching failure in the first matching result is matched with a first target area to obtain a second matching result, wherein the first target area includes the second shoulder recognition area and the prediction area with a matching failure in the first matching result.

[0108] In step S1063, the first area and the second face recognition area in the second area with a matching failure in the second matching result are matched with a second target area to obtain a third matching result, wherein the second target area includes the prediction area with a matching failure in the second matching result and the prediction area with lost tracking information in the first matching result.

[0109] In step S1064, the second region that fails to match in the third matching result is matched with a second predicted region to obtain a fourth matching result, where the second predicted region is a predicted region that fails to match in the third matching result, has a tracking information that is in a lost state for a time length not exceeding a preset time length, and has an identification quality index of the head-shoulder identification region satisfying a third preset condition, and the identification quality index of the head-shoulder identification region is used to represent the definition of the head and shoulder of the tracked object.

[0110] In step S1065, a similarity index is determined according to the first matching result, the second matching result, the third matching result, and the fourth matching result.

[0111] Specifically, step S1061 can be implemented by the following method: determining first face features of the first region in the second region and the second face identification region, and determining second face features of the face identification region; determining a first Euclidean distance between the first face features and the second face features; determining a first intersection-over-union between the first head-shoulder identification region and the head-shoulder identification region; determining a first cost matrix according to the first Euclidean distance, the first intersection-over-union, a first preset weight corresponding to the first Euclidean distance, and a second preset weight corresponding to the first intersection-over-union; and determining the first matching result according to the first cost matrix and a preset matching algorithm.

[0112] The cost matrix is a two-dimensional array, and the elements in the array represent the cost of associating the predicted region with the detected region. The cost is the intersection-over-union between the head-shoulder identification regions and / or the Euclidean distance between the face identification regions. By minimizing the total cost in the cost matrix, the best matching solution can be found, thereby achieving accurate target tracking.

[0113] Step S1062 can be implemented by the following method: matching the second region that fails to match in the first matching result with the first target region to obtain a second matching result, including: determining a second intersection-over-union between the second region that fails to match in the first matching result and the first target region; determining a second cost matrix according to the second intersection-over-union; and determining the second matching result according to the second cost matrix and a preset matching algorithm.

[0114] Step S1063 can be implemented by the following method: matching the first region in the second region that fails to match in the second matching result and the second face identification region with the second target region to obtain a third matching result, including: determining third face features of the first region in the second region that fails to match in the second matching result and the second face identification region, and determining fourth face features of the second target region; determining a second Euclidean distance between the third face features and the fourth face features; determining a third cost matrix according to the second Euclidean distance; and determining the third matching result according to the third cost matrix and a preset matching algorithm.

[0115] It is worth noting that through the third matching, the temporarily lost prediction region can be recovered.

[0116] The step S1064 can be implemented by matching the second region with failed matching in the third matching result with the second prediction region to obtain a fourth matching result, including: determining a third intersection-union ratio between the second region with failed matching in the third matching result and the second prediction region; determining a fourth cost matrix according to the third intersection-union ratio; and determining the fourth matching result according to the fourth cost matrix and a preset matching algorithm.

[0117] It is worth noting that the fourth matching can be used as a supplement to the third recovery of the temporarily lost prediction region.

[0118] The preset matching algorithm at least includes a Hungarian matching algorithm. The Hungarian matching algorithm, also known as the Hungarian algorithm, is an algorithm used to solve assignment problems in operations research and graph theory.

[0119] The basic steps of the algorithm are as follows: first, confirm the detection frame in the current frame that matches the target in the historical frame and establish a bipartite graph model. Then, match in order of the target ID. For example, first match the target 1 in the current frame that may match the target 1 in the previous frame. If a conflict occurs during the matching process, i.e., a target's potential match in the current frame is already occupied by another target, try to rematch by augmenting the path. The augmented path is a path that passes through an unmatched point and can increase the number of matches. If the number of matches cannot be increased after the augmented path, weight adjustment is needed. Repeat the above steps until all targets are matched or it is confirmed that the number of matches cannot be further increased.

[0120] In multi-target tracking, the Hungarian algorithm can effectively associate the detected targets in the current frame with the historical trajectories, thereby maintaining the identity consistency of the targets.

[0121] Further, the tracking information of the prediction region can be updated according to the target region by the following method: if the matching success rate of the first matching result is greater than a preset success rate, or the matching success rate of the second matching result is greater than the preset success rate, or the matching success rate of the third matching result is greater than the preset success rate, or the matching success rate of the fourth matching result is greater than the preset success rate, the target region is determined as the historical recognition region corresponding to the current image frame, and the historical recognition region corresponding to the current image frame is added to the tracking information of the prediction region.

[0122] For example, if the matching success rate of the first matching result is greater than 80%, or the matching success rate of the second matching result is greater than 80%, or the matching success rate of the third matching result is greater than 80%, or the matching success rate of the fourth matching result is greater than 80%, the target region is determined as the historical recognition region corresponding to the current image frame to update the tracking information of the prediction region.

[0123] Further, after determining the similarity index, the following step can also be performed: if the matching success rate of the first matching result, the matching success rate of the second matching result, the matching success rate of the third matching result, and the matching success rate of the fourth matching result are all less than the preset success rate, it is determined whether the recognition quality index of the face region and / or the recognition quality index of the head-shoulder region in the second region satisfy a fourth preset condition, and in the case that the recognition quality index of the face region and / or the recognition quality index of the head-shoulder region in the second region satisfy the fourth preset condition, the tracking information corresponding to the second region is established and added to the tracking information set corresponding to the current image frame, wherein the tracking information set includes different tracking information corresponding to different tracking objects.

[0124] On the other hand, after determining the similarity index, the following step can also be performed: if the matching success rate of the first matching result, the matching success rate of the second matching result, the matching success rate of the third matching result, and the matching success rate of the fourth matching result are all less than the preset success rate, in the case that the tracking object is an uncertain object, the state of the tracking information of the prediction region is modified to an unvalidated state; if the matching success rate of the first matching result, the matching success rate of the second matching result, the matching success rate of the third matching result, and the matching success rate of the fourth matching result are all less than the preset success rate, in the case that the tracking object is a valid tracking object, it is determined whether the target time length from the first time to the zero time is a preset time length, wherein the first time is the time when the matching success rate of the first matching result, the matching success rate of the second matching result, the matching success rate of the third matching result, and the matching success rate of the fourth matching result are all less than the preset success rate, and the zero time is the time when the current image frame is acquired; if the target time length is the preset time length, the state of the tracking information of the prediction region is modified to a lost state, and if the target time length is less than the preset time length, the state of the tracking information of the prediction region is modified to an early lost state.

[0125] The tracking object is an uncertain object, i.e., the tracking state of the tracking information is Uncertain, i.e., the tracking ID is initially created, but it is uncertain whether it is a valid tracking ID.

[0126] FIG. 2 is a flowchart of another portrait tracking method according to an embodiment of the present application, as shown in FIG. 2, the method includes the following steps:

[0127] Step S201, access the camera, or receive a video file, get a video sequence, in the video sequence, 5 frames per second or according to the actual work requirements are uniformly sampled, to obtain a plurality of image frames, the plurality of image frames are input into a single frame analysis module.

[0128] Step S202, the single frame analysis module analyzes the image frame to obtain the following information: face frame, face attribute, face feature, head and shoulder frame, body frame, and body frame quality.

[0129] Step S203, converting the single frame result detected in step S202 into a format required for tracking, the tracking required information: face frame, face quality, face feature, head and shoulder frame, body frame, and body frame quality.

[0130] Step S204, calculating the IOU between all detected head and shoulder frames and face frames to obtain a cost matrix, applying the Hungarian matching algorithm to calculate the matching result, binding the face frame and the corresponding head and shoulder body frame to obtain TrackInput. The unmatched face frame and head and shoulder body frame are also added to TrackInput.

[0131] Step S205, deleting the Tracklet in the track list Tracklets that has ended (state is Deleted or NotSet), then traversing each Tracklet in Tracklets, applying Kalman filtering, predicting the motion trajectory of each Tracklet according to the historical trajectory of each Tracklet to obtain an estimated head and shoulder frame, which is used for subsequent IOU matching.

[0132] Step S206, performing four matching processes.

[0133] Step S2061, first matching: M TrackInput with face frame, N Tracklets in Tracklets with state not being Lost, calculating the IOU value of the head and shoulder frame and the Euclidean distance of the face feature, then applying different weight values to combine the two values to calculate a cost matrix (the weight setting is according to actual debugging), generating an N*M cost matrix, applying the Hungarian matching algorithm to calculate the first matching result.

[0134] Step S2062, second matching: M TrackInput in the first matching, N TrackInput without face frame, and L Tracklets in the first matching that are not matched (total (N+L) Tracklets), then calculating the head and shoulder frame IOU value between the M TrackInput and the (N+L) Tracklets to obtain an (N+L)*M cost matrix, applying the Hungarian algorithm to calculate the second matching result.

[0135] Step S2063, third matching: M TrackInputs with face bounding box which are not matched in the second matching, N Tracklets which are not matched in the second matching and L Tracklets which are in the state of Lost in the first matching are matched, the Euclidean distance between the face features of the M TrackInputs and the (N+L) Tracklets is calculated to obtain a (N+L)*M matrix as a cost matrix, and the Hungarian algorithm is applied to calculate the third matching result. The main purpose of the third matching is to find back the temporarily lost Tracklets due to occlusion and other factors.

[0136] Step S2064, fourth matching: M Tracklets which are in the state of Lost within 5 seconds and have a head-shoulder bounding box quality greater than a quality threshold in the third matching are matched again with N TrackInputs which are not matched to calculate the IOU value of the head-shoulder bounding box, to obtain an N*M matrix as a cost matrix, and the Hungarian matching algorithm is applied to calculate the fourth matching result. The fourth matching is mainly used to complement the third matching to find back the temporarily lost Tracklets.

[0137] Step S207, FIG. 3 is a flowchart of a tracking state jump and drop-out according to an embodiment of the present application, and FIG. 4 is a flowchart of a tracking state conversion according to an embodiment of the present application. For the TrackInputs and Tracklets which are all matched in the four matchings, the TrackInputs are applied to update the corresponding Tracklets, the state of each matched Tracklet is updated according to the method shown in FIG. 4, and subsequent drop-out is performed according to the method shown in FIG. 3. The drop-out is to save the image with the best tracking ID quality according to the requirements.

[0138] The state of a Tracklet (tracking state) includes but is not limited to: 1. NotSet: the tracking ID is not effective. 2. Uncertain: the tracking ID is initially created, but it is not determined whether it is a valid tracking ID. 3. Confirmed: after multiple frames, it is confirmed that it is a valid tracking ID in the current frame. 4. EarlyLost: the tracking ID is lost before the first push image time, and this mode is added to complete the two push images. 5. Lost: the lost tracking ID. Deleted: the tracking ID has ended.

[0139] Step S208, for the TrackInput which is not matched with the Tracklets for four times, it is judged whether the size and quality of the face frame or the shoulder frame meet the quality threshold for creating a new Tracklet, and if yes, a new Tracklet is created and added to the Tracklets, and if not, it is discarded.

[0140] Step S209, for the Tracklet in the Tracklets which is not matched with the TrackInput for four times, the state is changed according to the state updating condition in FIG. 4, and the subsequent tracking is performed according to the flow in FIG. 3.

[0141] Step S210, the next frame image is input.

[0142] Through the above steps, the tracking is performed by using the self-adaptive fusion of multiple features (shoulder frame, face frame, face feature, face quality, and body quality), the advantages of both are better combined, the tracking loss caused by factors such as face occlusion, blur, and angle change when tracking only by using the face frame and face feature is reduced, and the tracking accuracy is improved.

[0143] In addition, the face tracking is implemented by using the cascade matching mode and the multi-stage matching strategy, the four different matching modes are used in succession, the first accurate matching is performed by using the input face frame and shoulder frame, the ID shoulder frame IOU, and the face feature weight adaptive combination in the first time, the clear ID matching accuracy is ensured, then the supplementary matching is performed by using the shoulder IOU, the third matching is performed by using the face feature, and the fourth matching is performed by using the shoulder frame IOU for finding the lost ID, the tracking accuracy is ensured, and the tracking repetition rate is better reduced.

[0144] In summary, the embodiment can significantly improve the accuracy and robustness of face tracking, effectively cope with challenges such as light changes, occlusions, and posture changes in complex monitoring scenes, and thus has a wide prospect in practical applications.

[0145] FIG. 5 is a structure diagram of a portrait tracking device according to an embodiment of the present application, as shown in FIG. 5, the device includes:

[0146] The first acquisition module 51 is configured to acquire a current image frame and determine tracking information of the current image frame, wherein the current image frame includes a tracking object, and the tracking information at least includes an activity track of the tracking object.

[0147] The first determination module 52 is configured to determine a target face recognition area and a target shoulder recognition area in the current image frame.

[0148] The region determination module 53 is configured to determine a first face recognition region and a first shoulder recognition region, and a second face recognition region and a second shoulder recognition region, respectively, based on the coincidence degree indicators of the target face recognition region and the target shoulder recognition region, and a first preset condition.

[0149] The second acquisition module 54 is configured to acquire a historical image frame corresponding to the current image frame, wherein the historical image frame includes a historical recognition region of the tracking object.

[0150] The second determination module 55 is configured to determine a predicted region of the tracking object in the current image frame according to the historical recognition region, wherein the predicted region includes a face recognition region and a shoulder recognition region.

[0151] The matching module 56 is configured to match the second region with the predicted region to obtain a similarity indicator, wherein the second region includes the first region, the second face recognition region, and the second shoulder recognition region, and the first region includes the first face recognition region and the first shoulder recognition region.

[0152] The updating module 57 is configured to determine a target region in the second region that satisfies a second preset condition with the similarity indicator of the predicted region, and update tracking information of the predicted region according to the target region.

[0153] In some embodiments, the region determination module 53 is further configured to perform the following steps: determining a region in the target face recognition region that corresponds to the coincidence degree indicator satisfying the first preset condition as the first face recognition region; determining a region in the target shoulder recognition region that corresponds to the coincidence degree indicator satisfying the first preset condition as the first shoulder recognition region; determining a region in the target face recognition region that corresponds to the coincidence degree indicator not satisfying the first preset condition as the second face recognition region; and determining a region in the target shoulder recognition region that corresponds to the coincidence degree indicator not satisfying the first preset condition as the second shoulder recognition region.

[0154] In some embodiments, the matching module 56 is further configured to perform the following steps: matching the first region and the second face recognition region in the second region with the face recognition region in the first predicted region, and matching the target head-shoulder recognition region in the second region with the head-shoulder recognition region in the first predicted region to obtain a first matching result, wherein the first predicted region is a predicted region in a non-lost state according to the tracking information; matching the second region that fails to match in the first matching result with the first target region to obtain a second matching result, wherein the first target region includes the second head-shoulder recognition region and the predicted region that fails to match in the first matching result; matching the first region and the second face recognition region in the second region that fails to match in the second matching result with the second target region to obtain a third matching result, wherein the second target region includes the predicted region that fails to match in the second matching result and the predicted region in a lost state according to the tracking information in the first matching result; matching the second region that fails to match in the third matching result with the second predicted region to obtain a fourth matching result, wherein the second predicted region is a predicted region that fails to match in the third matching result, has a time length not exceeding a preset time length in the lost state according to the tracking information, and has an identification quality index of the head-shoulder recognition region satisfying a third preset condition, and the identification quality index of the head-shoulder recognition region is used to represent the clarity of the head-shoulder of the tracked object; and determining a similarity index according to the first matching result, the second matching result, the third matching result, and the fourth matching result.

[0155] In some embodiments, the matching module 56 is further configured to perform the following steps: determining a first face feature of the first region and the second face recognition region in the second region, and determining a second face feature of the face recognition region; determining a first Euclidean distance between the first face feature and the second face feature; determining a first intersection-over-union between the first head-shoulder recognition region and the head-shoulder recognition region; determining a first cost matrix according to the first Euclidean distance, the first intersection-over-union, a first preset weight corresponding to the first Euclidean distance, and a second preset weight corresponding to the first intersection-over-union; and determining the first matching result according to the first cost matrix and a preset matching algorithm, wherein the preset matching algorithm at least includes a Hungarian matching algorithm.

[0156] In some embodiments, the matching module 56 is further configured to perform the following steps: determining a second intersection-over-union between the second region that fails to match in the first matching result and the first target region; determining a second cost matrix according to the second intersection-over-union; and determining the second matching result according to the second cost matrix and the preset matching algorithm, wherein the preset matching algorithm at least includes a Hungarian matching algorithm.

[0157] In some embodiments, the matching module 56 is further configured to perform the following steps: determining third facial features of the first region in the second region in which the matching fails in the second matching result and the second face recognition region, and determining fourth facial features of the second target region; determining a second Euclidean distance between the third facial features and the fourth facial features; determining a third cost matrix according to the second Euclidean distance; and determining a third matching result according to the third cost matrix and a preset matching algorithm, wherein the preset matching algorithm at least includes a Hungarian matching algorithm.

[0158] In some embodiments, the matching module 56 is further configured to perform the following steps: determining a third IoU between the second region in which the matching fails in the third matching result and the second predicted region; determining a fourth cost matrix according to the third IoU; and determining a fourth matching result according to the fourth cost matrix and a preset matching algorithm, wherein the preset matching algorithm at least includes a Hungarian matching algorithm.

[0159] In some embodiments, the updating module 57 is further configured to perform the following steps: if the matching success rate of the first matching result, the matching success rate of the second matching result, the matching success rate of the third matching result, or the matching success rate of the fourth matching result is greater than a preset success rate, determining the target region as a historical recognition region corresponding to the current image frame, and adding the historical recognition region corresponding to the current image frame to the tracking information of the predicted region.

[0160] In some embodiments, the portrait tracking device further comprises a third determining module configured to perform the following steps after determining the similarity indicators: if the matching success rate of the first matching result, the matching success rate of the second matching result, the matching success rate of the third matching result, and the matching success rate of the fourth matching result are all less than a preset success rate, determining whether the recognition quality indicators of the face regions and / or the recognition quality indicators of the shoulder regions in the second region satisfy a fourth preset condition, establishing tracking information corresponding to the second region in a case where the recognition quality indicators of the face regions and / or the recognition quality indicators of the shoulder regions in the second region satisfy the fourth preset condition, and adding the tracking information corresponding to the second region to a tracking information set corresponding to the current image frame, wherein the tracking information set comprises different tracking information corresponding to different tracking objects.

[0161] In some embodiments, the human portrait tracking device further comprises a fourth determination module configured to perform the following steps after determining the similarity index: if the matching success rates of the first matching result, the second matching result, the third matching result and the fourth matching result are all less than the preset success rate, modifying the state of the tracking information of the prediction region to the un-effective state in the case that the tracked object is an uncertain object; if the matching success rates of the first matching result, the second matching result, the third matching result and the fourth matching result are all less than the preset success rate, determining whether the target time length from the first time to the zero time is the preset time length in the case that the tracked object is an effective tracked object, wherein the first time is the time when the matching success rates of the first matching result, the second matching result, the third matching result and the fourth matching result are all less than the preset success rate, and the zero time is the time when the current image frame is acquired; if the target time length is the preset time length, modifying the state of the tracking information of the prediction region to the lost state, and if the target time length is less than the preset time length, modifying the state of the tracking information of the prediction region to the early lost state.

[0162] It should be noted that each module in FIG. 5 can be a program module (for example, a program instruction set for implementing a certain specific function) or a hardware module. For the latter, it can be in the form of, but not limited to, a processor, or the functions of the above-mentioned modules are implemented by a processor.

[0163] It should be noted that the specific implementation of the embodiment shown in FIG. 5 can refer to the related description of the embodiment shown in FIG. 1, which will not be repeated here.

[0164] FIG. 6 shows a hardware structure block diagram of a computer terminal for implementing the human portrait tracking method. As shown in FIG. 6, the computer terminal 60 can include one or more (shown in the figure as 602a, 602b, …, 602n) processors 602 (the processor 602 can include, but not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 604 for storing data, and a transmission module 606 for communication function. In addition, it can also include a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports of the BUS bus), a network interface, a power supply and / or a camera. Those skilled in the art can understand that the structure shown in FIG. 6 is only schematic, which does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 60 can also include more or fewer components than those shown in FIG. 6, or have a different configuration from that shown in FIG. 6.

[0165] It should be noted that the one or more processors 602 and / or other data processing circuitry described above can be referred to herein generically as "data processing circuitry." The data processing circuitry can be embodied as software, hardware, firmware, or any combination thereof, in whole or in part. Moreover, the data processing circuitry can be a single standalone processing module, or incorporated in whole or in part within any one of the other elements of the computer terminal 60. As referred to in embodiments of the present application, the data processing circuitry functions as a processor to control, for example, selection of the variable resistance terminal path in connection with the interface.

[0166] The memory 604 can be used to store software programs and modules for application software, such as program instructions / data storage means corresponding to the portrait tracking method of embodiments of the present application. The processor 602 can execute various functional applications and data processing, i.e., implement the portrait tracking method described above, by running the software programs and modules stored in the memory 604. The memory 604 can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 604 can further include a memory disposed remotely with respect to the processor 602, which can be connected to the computer terminal 60 through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0167] The transmission module 606 is configured to receive or send data via a network. Specific examples of the network can include a wireless network provided by a communication provider of the computer terminal 60. In one example, the transmission module 606 includes a network interface controller (NIC) that can be connected to other network devices through a base station to communicate with the Internet. In one example, the transmission module 606 can be a radio frequency (RF) module configured to communicate with the Internet in a wireless manner.

[0168] The display can be, for example, a touch screen type liquid crystal display (LCD) that can enable a user to interact with the user interface of the computer terminal 60.

[0169] It should be noted that in some alternative embodiments, the computer terminal shown in FIG. 6 can include hardware elements (including circuitry), software elements (including computer code stored on a computer readable media), or combinations of both hardware and software elements. It should be noted that FIG. 6 is only one example of a particular implementation and is intended to illustrate the types of components that can be present in the computer terminal.

[0170] It should be noted that the computer terminal shown in FIG. 6 is used to execute the portrait tracking method shown in FIG. 1, and therefore the related explanations in the execution method of the above commands also apply to the electronic device, which will not be repeated here.

[0171] The embodiments of the present application also provide a non-volatile storage medium, which comprises a stored program, wherein the program controls a device in which the storage medium is located to execute the above portrait tracking method when the program is running.

[0172] The non-volatile storage medium executes the program to perform the following functions: obtaining a current image frame, and determining tracking information of the current image frame, wherein the current image frame comprises a tracking object, and the tracking information at least comprises an activity track of the tracking object; determining a target face recognition region in the current image frame, determining a first shoulder recognition region whose coincidence degree with the target face recognition region meets a first preset condition and a second shoulder recognition region whose coincidence degree with the target face recognition region does not meet the first preset condition; combining the target face recognition region and the first shoulder recognition region to obtain a first region, and adding the second shoulder recognition region to the first region to obtain a second region; obtaining a historical image frame corresponding to the current image frame, wherein the historical image frame comprises a historical recognition region of the tracking object; determining a prediction region of the tracking object in the current image frame according to the historical recognition region, wherein the prediction region comprises a face recognition region and a shoulder recognition region; matching the second region and the prediction region to obtain a similarity index, determining a target region in the second region whose similarity index with the prediction region meets a second preset condition, and updating the tracking information of the prediction region according to the target region.

[0173] The embodiments of the present application also provide an electronic device, which comprises a memory and a processor, and the processor is configured to run a program stored in the memory, wherein the program controls the device to execute the above portrait tracking method when the program is running.

[0174] The processor is configured to run a program to perform the following functions: acquiring a current image frame, and determining tracking information of the current image frame, wherein the current image frame comprises a tracking object, and the tracking information at least comprises an activity track of the tracking object; determining a target face recognition region in the current image frame, determining a first shoulder recognition region with which the coincidence degree index of the target face recognition region meets a first preset condition and a second shoulder recognition region with which the coincidence degree index of the target face recognition region does not meet the first preset condition; combining the target face recognition region and the first shoulder recognition region to obtain a first region, and adding the second shoulder recognition region to the first region to obtain a second region; acquiring a historical image frame corresponding to the current image frame, wherein the historical image frame comprises a historical recognition region of the tracking object; determining a prediction region of the tracking object in the current image frame according to the historical recognition region, wherein the prediction region comprises a face recognition region and a shoulder recognition region; matching the second region with the prediction region to obtain a similarity index, determining a target region in the second region with which the similarity index meets a second preset condition, and updating the tracking information of the prediction region according to the target region.

[0175] The above sequence numbers of the embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0176] In the above embodiments of the present application, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0177] In the above embodiments of the present application, the collected information is information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards, necessary protection measures are taken, public order and good customs are not violated, and appropriate operation entrances are provided for the user to choose authorization or refusal.

[0178] In the several embodiments of the present application, it should be understood that the disclosed technology can be implemented in other ways. Of course, the unit described as the division is only a logic function division, and there can be other division ways in actual implementation, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, unit or module, and can be electrical or other forms.

[0179] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e. may be located in one place, or may be distributed to multiple units. Part or all of the units may be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0180] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0181] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the related art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various program code storage media.

[0182] The technical features of the above-described embodiments can be combined arbitrarily, and to make the description concise, not all possible combinations of the technical features in the above-described embodiments are described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application.

[0183] The above-described embodiments only express several implementation manners of the present application, the description is more specific and detailed, but it should not be understood as a limitation on the patent scope of the application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the protection scope of the present application. Therefore, the patent protection scope of the present application should be subject to the appended claims.

Claims

1. A portrait tracking method, comprising: obtaining a current image frame and determining tracking information of the current image frame; the current image frame comprises a tracking object; the tracking information comprises at least a moving track of the tracking object; determining a target face recognition region and a target shoulder recognition region in the current image frame; determining a first face recognition region and a first shoulder recognition region, and a second face recognition region and a second shoulder recognition region based on a coincidence index of the target face recognition region and the target shoulder recognition region, and a first preset condition; obtaining a historical image frame corresponding to the current image frame; the historical image frame comprises a historical recognition region of the tracking object; determining a predicted region of the tracking object in the current image frame according to the historical recognition region; the predicted region comprises a face recognition region and a shoulder recognition region; matching a second region with the predicted region to obtain a similarity index; the second region comprises a first region, the second face recognition region, and the second shoulder recognition region; the first region comprises the first face recognition region and the first shoulder recognition region; determining a target region in the second region that satisfies a second preset condition with the similarity index of the predicted region, and updating tracking information of the predicted region according to the target region.

2. The method of claim 1, wherein, The determining of the first face recognition region and the first shoulder recognition region, and the second face recognition region and the second shoulder recognition region based on the coincidence index of the target face recognition region and the target shoulder recognition region, and the first preset condition comprises: determining a region in the target face recognition region that corresponds to the coincidence index satisfying the first preset condition as the first face recognition region; determining a region in the target shoulder recognition region that corresponds to the coincidence index satisfying the first preset condition as the first shoulder recognition region; determining a region in the target face recognition region that corresponds to the coincidence index not satisfying the first preset condition as the second face recognition region; determining a region in the target shoulder recognition region that corresponds to the coincidence index not satisfying the first preset condition as the second shoulder recognition region.

3. The method of claim 1 or 2, wherein, The coincidence index comprises an intersection-over-union of the target face recognition region and the target shoulder recognition region; the first preset condition comprises that the intersection-over-union is greater than a preset threshold.

4. The method of claim 1, wherein, The matching of the second region with the predicted region to obtain the similarity index comprises: matching the first region and the second face recognition region in the second region with a face recognition region in a first predicted region, and matching the target shoulder recognition region in the second region with a shoulder recognition region in the first predicted region to obtain a first matching result; the first predicted region is a predicted region in a non-loss state of the tracking information. matching the second region that fails in the first matching result with a first target region to obtain a second matching result, wherein the first target region comprises the second shoulder identification region and a predicted region that fails in the first matching result; matching the first region and the second face identification region in the second region that fails in the second matching result with a second target region to obtain a third matching result, wherein the second target region comprises the predicted region that fails in the second matching result and a predicted region that has a tracking information in a lost state in the first matching result; matching the second region that fails in the third matching result with a second predicted region to obtain a fourth matching result, wherein the second predicted region is a predicted region that fails in the third matching result, has a tracking information in a lost state and has a length of time not exceeding a preset length of time, and has an identification quality index of the shoulder identification region satisfying a third preset condition, and the identification quality index of the shoulder identification region is used to represent a definition of a shoulder of the tracking object; determining the similarity index according to the first matching result, the second matching result, the third matching result and the fourth matching result.

5. The method of claim 4, wherein, matching the first region and the second face identification region in the second region with a face identification region in a first predicted region, and matching a target shoulder identification region in the second region with a shoulder identification region in the first predicted region to obtain a first matching result, comprising: determining a first face feature of the first region and the second face identification region in the second region, and determining a second face feature of the face identification region; determining a first Euclidean distance between the first face feature and the second face feature; determining a first intersection-over-union between the first shoulder identification region and the shoulder identification region; determining a first cost matrix according to the first Euclidean distance, the first intersection-over-union, a first preset weight corresponding to the first Euclidean distance and a second preset weight corresponding to the first intersection-over-union; determining the first matching result according to the first cost matrix and a preset matching algorithm, wherein the preset matching algorithm at least comprises a Hungarian matching algorithm.

6. The method of claim 4, wherein, matching the second region that fails in the first matching result with a first target region to obtain a second matching result, comprising: determining a second intersection-over-union between the second region that fails in the first matching result and the first target region; determining a second cost matrix according to the second intersection-over-union; determining the second matching result according to the second cost matrix and a preset matching algorithm, wherein the preset matching algorithm at least comprises a Hungarian matching algorithm.

7. The method of claim 4, wherein, matching the first region and the second face identification region in the second region that fails in the second matching result with a second target region to obtain a third matching result, comprising: determine third face features of the first region and the second face recognition region in the second region in which the matching fails in the second matching result, and determine fourth face features of the second target region; determine a second Euclidean distance between the third face features and the fourth face features; determine a third cost matrix according to the second Euclidean distance; determine the third matching result according to the third cost matrix and a preset matching algorithm; the preset matching algorithm at least includes a Hungarian matching algorithm.

8. The method of claim 4, wherein, match the second region in which the matching fails in the third matching result with a second predicted region to obtain a fourth matching result, including: determine a third intersection over union between the second region in which the matching fails in the third matching result and the second predicted region; determine a fourth cost matrix according to the third intersection over union; determine the fourth matching result according to the fourth cost matrix and a preset matching algorithm; the preset matching algorithm at least includes a Hungarian matching algorithm.

9. The method of claim 4, wherein, update tracking information of the predicted region according to the target region, including: if a matching success rate of the first matching result is greater than a preset success rate, or a matching success rate of the second matching result is greater than the preset success rate, or a matching success rate of the third matching result is greater than the preset success rate, or a matching success rate of the fourth matching result is greater than the preset success rate, determine the target region as a historical recognition region corresponding to the current image frame, and add the historical recognition region corresponding to the current image frame into the tracking information of the predicted region.

10. The method of claim 4, wherein, after determining the similarity index, the method further includes: if the matching success rate of the first matching result, the matching success rate of the second matching result, the matching success rate of the third matching result and the matching success rate of the fourth matching result are all less than a preset success rate, determine whether a recognition quality index of a face region and / or a recognition quality index of a head and shoulder region in the second region meet a fourth preset condition; in a case where the recognition quality index of the face region and / or the recognition quality index of the head and shoulder region in the second region meet the fourth preset condition, establish tracking information corresponding to the second region, and add the tracking information corresponding to the second region into a tracking information set corresponding to the current image frame; the tracking information set includes different tracking information corresponding to different tracking objects.

11. The method of claim 10, wherein, after determining the similarity index, the method further includes: if the matching success rate of the first matching result, the matching success rate of the second matching result, the matching success rate of the third matching result and the matching success rate of the fourth matching result are all less than a preset success rate, in a case where the tracking object is an uncertain object, modify a state of the tracking information of the predicted region to an unvalidated state; If the match success rates of the first match result, the second match result, the third match result and the fourth match result are all less than a preset success rate, and the tracking object is a valid tracking object, it is determined whether a target time length from a first time to a zero time is a preset time length, the first time being a time when the match success rates of the first match result, the second match result, the third match result and the fourth match result are all less than the preset success rate, and the zero time being a time when the current image frame is acquired; If the target time length is the preset time length, the state of the tracking information of the prediction region is modified to a lost state, and if the target time length is less than the preset time length, the state of the tracking information of the prediction region is modified to an early lost state.

12. A portrait tracking apparatus, comprising: a first acquisition module configured to acquire a current image frame and determine tracking information of the current image frame, wherein the current image frame comprises a tracking object, and the tracking information comprises at least an activity track of the tracking object; a first determination module configured to determine a target face recognition region and a target head-shoulder recognition region in the current image frame; a region determination module configured to determine a first face recognition region and a first head-shoulder recognition region, and a second face recognition region and a second head-shoulder recognition region based on a coincidence degree index of the target face recognition region and the target head-shoulder recognition region and a first preset condition; a second acquisition module configured to acquire a historical image frame corresponding to the current image frame, wherein the historical image frame comprises a historical recognition region of the tracking object; a second determination module configured to determine a prediction region of the tracking object in the current image frame according to the historical recognition region, wherein the prediction region comprises a face recognition region and a head-shoulder recognition region; a matching module configured to match the second region and the prediction region to obtain a similarity index, wherein the second region comprises a first region, the second face recognition region and the second head-shoulder recognition region, and the first region comprises the first face recognition region and the first head-shoulder recognition region; an updating module configured to determine a target region in the second region that satisfies a second preset condition with the similarity index of the prediction region, and update the tracking information of the prediction region according to the target region.

13. A non-volatile storage medium comprising a stored program, wherein, The program controls a device in which the nonvolatile storage medium is located to perform the portrait tracking method in any one of claims 1 to 11 when the program is running.

14. An electronic device comprising: A memory and a processor, wherein the processor is configured to run a program stored in the memory, and the program performs the steps of the portrait tracking method in any one of claims 1 to 11 when the program is running.

15. A computer program product comprising a computer program, wherein the computer program performs the steps of the portrait tracking method in any one of claims 1 to 11 when the computer program is executed by a processor.

Images