Target tracking method, computer device and storage apparatus

Abstract

The application discloses a target tracking method, a computer device and a storage device. The method comprises the following steps: detecting at least two parts of a current video frame of a preset video to obtain a plurality of part detection boxes; determining a set of associated detection boxes of a target body from the plurality of part detection boxes, wherein the set of associated detection boxes comprises at least two part detection boxes, and each part detection box in the set of associated detection boxes belongs to the same target body and belongs to different part categories; and tracking the target body in other video frames based on the set of associated detection boxes of the target body, wherein the other video frames are video frames located after the current video frame in the preset video. The above scheme can improve the accuracy of target tracking.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a target tracking method, a computer device and a storage device. BACKGROUND

[0002] With the development of the field of computer vision, target tracking is widely used in the fields of intelligent video monitoring, automatic driving, unmanned supermarket, passenger flow statistics, intelligent transportation, etc.

[0003] Target tracking and capture can determine the position, size and complete motion trajectory of one or more independent moving targets in a video frame sequence. However, in the process of target tracking, due to high pedestrian density, target congestion, mutual occlusion and overlap between targets often occur, and the tracking effect of the target is poor, and the tracking accuracy is low. SUMMARY

[0004] The technical problem solved by the present application is to provide a target tracking method, a computer device and a storage device, which can improve the accuracy of target tracking.

[0005] In order to solve the above problems, the first aspect of the present application provides a target tracking method, which comprises: detecting at least two parts of a current video frame of a preset video to obtain a plurality of part detection boxes; determining a set of associated detection boxes of a target body from the plurality of part detection boxes, wherein the set of associated detection boxes includes at least two part detection boxes, and each part detection box in the set of associated detection boxes belongs to the same target body and belongs to different part categories; tracking the target body in other video frames based on the set of associated detection boxes of the target body, wherein the other video frames are video frames located after the current video frame in the preset video.

[0006] In order to solve the above problems, the second aspect of the present application provides a computer device, which comprises a memory and a processor coupled to each other, the memory stores program data, and the processor is used to execute the program data to realize any step of the above target tracking method.

[0007] In order to solve the above problems, the third aspect of the present application provides a storage device, which stores program data capable of being run by a processor, and the program data is used to realize any step of the above target tracking method.

[0008] The above scheme detects at least two parts of the current video frame of a preset video to obtain multiple part detection frames. From the multiple part detection frames, a set of associated detection frames of the target object is determined, wherein the associated detection frame set includes at least two part detection frames, and each part detection frame in the associated detection frame set belongs to the same target object but belongs to different part categories. Based on the associated detection frame set of the target object, the target object is tracked in other video frames, which are video frames in the preset video that follow the current video frame. Because the target object is tracked using the association relationship between at least two parts, if one part is occluded, the other part can still be used to track the target object, which can improve the accuracy of target tracking.

[0009] In addition, in the current video frame, the part detection frames of different part categories belonging to the same target body are associated, and the target body is tracked using the associated detection frame set obtained by association. This can reduce the separate matching of each part of the target body between video frames, thereby reducing the time spent on tracking the target body. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions of this application, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be derived from these drawings without inventive effort. Among them:

[0011] Figure 1 This is a flowchart of the first embodiment of the target tracking method of the present application;

[0012] Figure 2 This is an example schematic diagram of an embodiment of a part detection frame in a current video frame of the present application;

[0013] Figure 3 This application Figure 1 A flow chart of an embodiment of step S12;

[0014] Figure 4 This is an example schematic diagram of another embodiment of a part detection frame in the current video frame of the present application;

[0015] Figure 5 This is an example schematic diagram of an embodiment of a set of associated detection frames in a current video frame of the present application;

[0016] Figure 6 This application Figure 1 A flow chart of an embodiment of step S13;

[0017] Figure 7 This application Figure 4Flowchart of one embodiment of step S131 in the method;

[0018] Figure 8 is an example schematic diagram of the detection frame set and the tracking frame set of one embodiment of the present application;

[0019] Figure 9 is an example schematic diagram of the first part detection frame and the first part tracking frame of one embodiment of the present application;

[0020] Figure 10 is an example schematic diagram of the second part detection frame and the second part tracking frame of one embodiment of the present application;

[0021] Figure 11 is an example schematic diagram of the target tracking method of one embodiment of the present application Figure 1 Flowchart of another embodiment of step S13 in the method;

[0022] Figure 12 is a flowchart of the target tracking method of a second embodiment of the present application;

[0023] Figure 13 is a flowchart of an application example of the target tracking method of one embodiment of the present application;

[0024] Figure 14 is an example schematic diagram of an application example of the part detection frame in the target tracking process of one embodiment of the present application;

[0025] Figure 15 is an example schematic diagram of the target tracking method of one embodiment of the present application Figure 12 Flowchart of one embodiment of step S23 in the method;

[0026] Figure 16 is a flowchart of the target tracking method of a third embodiment of the present application;

[0027] Figure 17 is a structural schematic diagram of the target tracking device of a first embodiment of the present application

[0028] Figure 18 is a structural schematic diagram of the target tracking device of a second embodiment of the present application;

[0029] Figure 19 is a structural schematic diagram of the computer device of one embodiment of the present application;

[0030] Figure 20 is a structural schematic diagram of the storage device of one embodiment of the present application. DETAILED DESCRIPTION

[0031] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0032] The terms "first", "second" in the present application are only for descriptive purpose, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "multiple" is at least two, for example, two, three, etc., unless otherwise explicitly and specifically limited. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0033] In the present application, referring to "embodiments" means that the specific features, structures or properties described in combination with the embodiments can be included in at least one embodiment of the present application. The phrase appears at various places in the specification does not necessarily refer to the same embodiment, nor is it independent or alternative to other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0034] The present application provides the following embodiments, which will be specifically described below.

[0035] Please refer to Figure 1 , Figure 1 is a flowchart of a first embodiment of the object tracking method of the present application. The method can include the following steps:

[0036] S11: detecting at least two parts of a current video frame of a preset video to obtain a plurality of part detection boxes.

[0037] The video of the target body can be obtained by the camera device in real time to obtain the preset video, so as to track one or more target bodies by using the preset video. In addition, the preset video can be a monitoring video obtained by shooting one or more target bodies in a preset scene, for example, a certain shopping mall, the entrance / exit of a certain company, a street place, etc. can be shot to obtain the preset video. The present application does not limit this.

[0038] A video frame of a preset video may contain one or more objects. The object can be any object capable of relative motion, such as a person, motor vehicle, non-motor vehicle, or animal. The object can be composed of multiple parts. Parts can be divided into regions or other methods. For example, if the object is a person, the body can be divided into the face, head and shoulders, torso, limbs, upper body, lower body, and other body parts.

[0039] The detection box mentioned in this application includes the location of the part. The detection box can be represented by coordinates, length, and width. For example, the detection box is represented as (upper left vertex coordinates, length, width). For another example, the detection box is represented as (upper left vertex coordinates, lower right vertex coordinates).

[0040] Detection of at least two parts in a video frame can be achieved using the same or different object detection networks. For example, networks such as YOLOv3 (You Only Look Once), Faster R-CNN, and FSAF can be used. For example, the YOLOv3 network can detect parts of different sizes and obtain detection boxes corresponding to multiple parts.

[0041] In some embodiments, the at least two parts include a first part, a second part, and a third part that belong to different part categories. The at least two parts may include the same region or different regions of the target body.

[0042] In some embodiments, the at least two parts include a first part and a third part belonging to different part categories. When one or more objects in a video frame are distant objects, the third part detection frame can be detected even when the first part is occluded, and the distant object can be tracked using the first and third parts.

[0043] In some embodiments, the multiple part detection frames include several first part detection frames corresponding to the first part, several second part detection frames corresponding to the second part, and several third part detection frames corresponding to the third part. The sizes of the part detection frames of different categories can be the same or different. For example, the size of the detection frames can be smaller than the size of the second part detection frame, and the size of the third part detection frame can be smaller than the size of the third part detection frame.

[0044] In some embodiments, see Figure 2Taking a human as an example, the first part is the face, the second part is the head and shoulders, and the third part is the body. The head and shoulders portion of the second part can include the head and shoulders of the human body, and the third part can include the entire human body, such as the face, head and shoulders, torso, limbs, upper body, lower body, and other parts. By detecting the first, second, and third parts of the current image frame, a number of first part detection frames, a number of second part detection frames, and a number of third part detection frames can be obtained, respectively.

[0045] S12: Determine an associated detection frame set of the target body from the multiple part detection frames, wherein the associated detection frame set includes at least two part detection frames, and each part detection frame in the associated detection frame set belongs to the same target body and belongs to different part categories.

[0046] Among multiple part detection frames, part detection frames of different part categories may belong to the same target body. The part detection frames of the same part category can be regarded as a category of detection frames, and matched between part detection frames of different part categories to determine part detection frames of different categories belonging to the same target body. At least two matched part detection frames can be associated to form an associated detection frame set.

[0047] In this way, the part detection frames corresponding to multiple parts of the target object in the current video frame can be associated, that is, the multiple parts of the target object can be associated within the frame.

[0048] In some embodiments, the associated detection frame set may include a first part detection frame, a second part detection frame, and a third part detection frame. Furthermore, if the part detection frames corresponding to some parts of the target cannot be detected due to reasons such as occlusion or head rotation, the associated detection frame set may also include a single part detection frame, although this application does not impose any restrictions on this.

[0049] S13: Tracking the target in other video frames based on the associated detection frame set of the target, where the other video frames are video frames in the preset video that are located after the current video frame.

[0050] In the video frames after the current video frame in the preset video, that is, in other video frames, the same method can also be used to obtain the associated detection frame sets of the target objects in other video frames.

[0051] In some implementations, the associated detection frame set of the target object in the current video frame may be matched with the associated detection frame set of the target object in other video frames to track the target object in the other video frames.

[0052] In this embodiment, at least two parts are detected in the current video frame of a preset video to obtain multiple part detection frames. From the multiple part detection frames, a set of associated detection frames of the target object is determined, wherein the associated detection frame set includes at least two part detection frames, and each part detection frame in the associated detection frame set belongs to the same target object but belongs to different part categories. Based on the associated detection frame set of the target object, the target object is tracked in other video frames, where the other video frames are video frames in the preset video that follow the current video frame. Because the target object is tracked using the association relationship between at least two parts, if one part is occluded, the other parts can still be used to track the target object, thereby improving the accuracy of target tracking.

[0053] In addition, in the current video frame, the part detection frames of different part categories belonging to the same target body are associated, and the target body is tracked using the associated detection frame set obtained by association. This can reduce the separate matching of each part of the target body between video frames, thereby reducing the time spent on tracking the target body.

[0054] In some embodiments, see Figure 3 In the above step S12, determining a set of associated detection frames of the target body from the multiple part detection frames may include the following steps:

[0055] S121: Selecting a to-be-associated detection frame from a plurality of part detection frames, wherein all part detection frames except the to-be-associated detection frame are used as other detection frames.

[0056] Any one of the multiple part detection frames can be used as a detection frame to be associated, and the detection frames of other types of parts can be used as other detection frames.

[0057] S122: Select at least one other detection frame that matches the detection frame to be associated as the target other detection frame.

[0058] If several first part detection frames are used as detection frames to be associated, and several second part detection frames or several third part detection frames are used as other detection frames, the second part detection frame and / or third part detection frame that matches the first part detection frame can be selected as the target other detection frame.

[0059] If the second part detection frame is used as the detection frame to be associated, the first part detection frame and / or the third part detection frame that matches the second part detection frame can be selected as other target detection frames for matching.

[0060] In some embodiments, for example, where at least two parts include a first part, a second part, and a third part belonging to different part categories, and the multiple part detection frames include several first part detection frames, several second part detection frames, and several third part detection frames, matching of the first part detection frame, the second part detection frame, and the third part detection frame for parts of different categories can be obtained by matching the two part detection frames.

[0061] In some embodiments, the first part detection frame can be used as the detection frame to be associated, and among several second part detection frames, the target second part detection frame that matches the detection frame to be associated is selected; among several third part detection frames, the target third part detection frame that matches the target second part detection frame is selected; and the target second part detection frame and / or the target third part detection frame are used as other target detection frames.

[0062] In some embodiments, the degree of overlap between each other detection frame and the detection frame to be associated can be obtained, and other detection frames whose degree of overlap meets the first position requirement are selected as target other detection frames. The degree of overlap can be expressed by the intersection-over-union (IOU), which can be expressed as the overlap rate of the detection frame to be associated with other detection frames, that is, the ratio of their intersection to their union. The first position requirement may include an intersection-over-union greater than a first overlap threshold, and the first overlap threshold for matching between multiple groups of two part detection frames may be the same or different. For example, the first overlap threshold for matching between the first part detection frame and the second part detection frame is different from the first overlap threshold for matching between the first part detection frame and the third part detection frame. This application does not impose any restrictions on this.

[0063] As an example, you can first perform an intersection-and-union match on several first-part detection frames and several second-part detection frames, select the second-part detection frames that match each first-part detection frame, then perform an intersection-and-union match on several second-part detection frames and several third-part detection frames, select the third-part detection frames that match each second-part detection frame. In this way, you can obtain matching first-part detection frames, second-part detection frames, and third-part detection frames. If the first part of the target is blocked and the first-part detection frame is not detected, you can obtain matching second-part detection frames and third-part detection frames. If the third-part detection frame is not detected, you can obtain matching first-part detection frames and second-part detection frames.

[0064] In some implementations, the positional relationships of preset points corresponding to other detection frames may also be obtained, and the detection frames whose positional relationships of preset points meet a second position requirement may be selected as target detection frames. The positional relationships of preset points corresponding to the other detection frames are the positional relationships between preset points in the other detection frames and preset points in the detection frame to be associated. The preset points may include the center coordinates of the detection frames, and the second position requirement may include whether the positional relationship is a containment relationship or a contained relationship.

[0065] As an example, if the center coordinates of the first part detection frame (the detection frame to be associated) are within the second part detection frame (the other detection frame), that is, if the containment relationship is satisfied, then the first part detection frame and the second part detection frame can be considered matched. If there are multiple other detection frames that meet the second position requirement, the Euclidean distance between the detection frame to be associated and each of the multiple other detection frames can be obtained in sequence, and the other detection frame with the smallest Euclidean distance value can be used as the target detection frame for extraction.

[0066] See also Figure 4 As an example, taking the target object as a person, the first part is the face, the second part is the head and shoulders, and the third part is the body. By detecting the first part, the second part, and the third part of the human body in the current image frame, a plurality of first part detection frames, a plurality of second part detection frames, and a plurality of third part detection frames can be obtained respectively. The first part detection frame, the second part detection frame, and the third part detection frame have different sizes. In some application scenarios, different styles (e.g., different colors) of detection frames can be used to represent part detection frames of different part categories, that is, part detection frames of the same part can be represented by detection frames of the same style.

[0067] S123: Associating the detection frame to be associated with other detection frames of the target to obtain a set of associated detection frames of the target body.

[0068] After selecting the target other detection frames that match the detection frame to be associated, it can be indicated that the detection frame to be associated and the target other detection frames belong to the same target body. The detection frame to be associated can be associated with the target other detection frames to obtain a set of associated detection frames of the target body.

[0069] See also Figure 5 As an example, taking a human as the target object, at least two part detection frames in the current video frame can be associated to form an associated detection frame set. In some application scenarios, the current video frame may contain multiple associated detection frame sets for the target object. Detection frames of different styles (e.g., different colors) can be used to represent the associated frame sets of different target objects. That is, part detection frames of the same target object can be represented by detection frames of the same style.

[0070] In this embodiment, by associating the part detection frames of multiple parts within a video frame, and because the second part detection is not easily obscured, the second part detection frame is matched with the first and third part detection frames to select a set of associated detection frames for the target object. Compared to matching the three part detection frames separately, the solution of this application can reduce the number of pairwise matches between the first, second, and third part detection frames, that is, it can reduce the time spent on matching within the frame, thereby reducing the algorithm complexity and solving many head turning or occlusion problems.

[0071] In addition, associating part detections of multiple part categories to form a set of associated detection frames of the target body can effectively utilize the association relationship between the parts and improve the ability of the associated detection frame set to express the characteristics of the target body.

[0072] In some embodiments, see Figure 6 The above step S13, tracking the target in other video frames based on the associated detection frame set of the target, and determining the associated detection frame set of the target from multiple part detection frames, may include the following steps:

[0073] S131: Determine a target tracking frame set that matches the associated detection frame set from a plurality of tracking frame sets; wherein the target bodies corresponding to the target tracking frame set and the associated detection frame set belong to the same target body.

[0074] The detection frame set includes at least one of a first part detection frame, a second part detection frame, and a third part detection frame associated with the target. The tracking frame set includes at least one of a first part tracking frame corresponding to the associated first part, a second part tracking frame corresponding to the second part, and a third part tracking frame corresponding to the third part. That is, the tracking frame set includes a set of associated detection frames obtained by detecting at least two parts of historical video frames, where the historical video frames include video frames preceding the current video frame in a preset video. Thus, the tracking frame set includes a set of historical associated detection frames of the target in multiple historical video frames.

[0075] By matching the associated detection frame set with several tracking frame sets, a target tracking frame set that belongs to the same target body as the associated detection frame is determined.

[0076] In some embodiments, see Figure 7 The above step S131 may include the following steps:

[0077] S1311: taking each tracking frame set as a tracking frame set to be processed; wherein the tracking frame set to be processed includes a part tracking frame corresponding to at least one part.

[0078] When matching the association bounding box set with the plurality of tracking box sets, each tracking box set can be taken as a to-be-processed tracking box set, and the tracking box set includes at least a part tracking box corresponding to a part.

[0079] S1312: Based on the to-be-processed tracking box set and the association bounding box set, a part tracking box and a part bounding box belonging to the same part category are determined.

[0080] The part bounding box and the part tracking box belonging to the same part category can be determined from the to-be-processed tracking box set and the association bounding box.

[0081] Please refer to Figure 8 As an example, the association bounding box set includes a first part bounding box and a second part bounding box. The tracking box set can include part tracking boxes of multiple video frames, and the latest part tracking box in the tracking box set is matched. The tracking box set can include a first part tracking box, a second part tracking box, and a third part tracking box.

[0082] Please refer to Figures 9 and 10 As an example, the first part bounding box and the first part tracking box belonging to the same first part can be determined. The second part bounding box and the second part tracking box belonging to the same second part can be determined.

[0083] S1313: The part bounding box and the part tracking box belonging to the same part category are matched respectively to select a target tracking box set matched with the association bounding box set.

[0084] The part bounding box and the part tracking box belonging to the same part category can be matched respectively by using the overlap degree between the part bounding box and the part tracking box to select a target tracking box set matched with the association bounding box set. The overlap degree can be an intersection-over-union ratio. If the intersection-over-union ratio of the part tracking box and the part bounding box is greater than a second overlap threshold, it can be determined that the part tracking box and the part bounding box are matched. The second overlap threshold can be set according to the part category to be matched, which is not limited in the present application.

[0085] In some embodiments, if the part tracking box of the same part category includes a plurality of part tracking boxes, the overlap degree of each part tracking box and the part bounding box can be obtained; and based on the statistical value of the overlap degree of each part, a target tracking box set matched with the association bounding box set is selected. The statistical value includes an average value, a total value, etc., which is not limited in the present application.

[0086] Please refer to Figures 9 and 10As an example, the intersection-of-union (IoU) ratio between the first part detection frame and the first part tracking frame can be obtained to obtain a first IoU; and the intersection-of-union ratio (IoU) ratio between the second part detection frame and the second part tracking frame can be obtained to obtain a second IoU. Thus, an average value of the first IoU and the second IoU is obtained, and then, based on the average value, the associated detection frame set and the tracking frame set can be matched. If the average value is greater than a preset average value, it can be determined that the associated detection frame set and the tracking frame set match.

[0087] In some embodiments, if multiple tracking frame sets match the associated detection frame set, the Euclidean distances between the associated detection frame and the multiple tracking frame sets can be obtained, and the tracking frame set with the smallest Euclidean distance can be used as the target tracking frame set that matches the associated detection frame set. Alternatively, if multiple tracking frame sets match the associated detection frame set, the tracking frame set that matches later can be used as the target tracking frame set that matches the associated detection frame set. This application does not impose any restrictions on this.

[0088] In some embodiments, see Figure 11 , at least two parts can be detected on the K-th video frame to execute the above steps S11 to S13 (i.e., iterative update) in this embodiment to obtain n tracking frame sets. Execute the above steps S11 to S12 on the k+1-th video frame (e.g., the current video frame) to associate at least two part detection frames within the frame to obtain a set of associated detection frames for m target bodies. At this time, a set of tracking frames in a valid state for n target bodies can also be included. Wherein, n and m are positive integers greater than or equal to 2. The valid state indicates that the tracking of the tracking frame set is normal, and there is no target loss or disappearance. Please refer to the following description of this valid state.

[0089] Based on a set of m associated detection frames for a target object and a set of n tracking frames for a target object, the average value of the intersection-and-union ratios between each associated detection frame set and each tracking frame set can be obtained. The average value of the intersection-and-union ratios can be described as the degree of overlap, resulting in a two-dimensional matrix of dimension m*n. Each element of the m*n two-dimensional overlap matrix M represents the degree of overlap between the associated detection frame set and the tracking frame set. A matching algorithm can be used to obtain the best matching result between the associated detection frame set and the tracking frame set in the m*n two-dimensional overlap matrix M. For example, matching algorithms include the Hungarian Algorithm and the Kuhn-Munkres Algorithm.

[0090] This application uses the Hungarian algorithm for Hungarian matching as an example, but is not limited thereto. Among m associated detection frame sets and n tracking frame sets, if the degree of overlap (e.g., average intersection-over-union) between the associated detection frame set and the tracking frame set is greater than a second overlap threshold, then it can be determined that the associated detection frame set and the tracking frame set are successfully matched; otherwise, it is determined that the associated detection frame set and the tracking frame set are unmatched.

[0091] S132: Track the target object in other video frames based on the associated detection frame set and the target tracking frame set of the target object.

[0092] See also Figure 11 If the associated detection frame set has a matching tracking frame set (target tracking frame set), that is, the associated detection frame set and the target tracking frame set of the target object successfully match, then the associated detection frame set can be added to the target tracking frame set, so that the part detection frames in the associated detection frame set are used as the part tracking frames of the corresponding part category in the target tracking frame set. This process can store the coordinate information of each part detection frame in the historical coordinate information of the tracking frame of the corresponding part category in the target tracking frame set. In addition, the tracking state of the target tracking frame set can be determined to be valid.

[0093] In some embodiments, each tracking frame set corresponds to a target body, and each tracking frame set is provided with an identifier (Identity Document, referred to as ID). A tracking state can be set for each identifier. If the tracking frame set matches an associated detection frame set in the current video frame, the tracking state of the tracking frame set is determined to be a valid state, and the associated detection frame set and the tracking frame set share the same identifier.

[0094] In some embodiments, if the associated detection frame set and the tracking frame set of the target body fail to match, that is, the tracking frame set does not match the associated detection frame set in the current video, the tracking state of the tracking frame set is determined to be a lost state.

[0095] In some embodiments, if the associated detection frame set and the tracking frame set of the target body fail to match, that is, the associated detection frame set does not match the tracking frame set in the current video, in order to avoid false detection of the target, a to-be-tracked frame set can be created for the associated detection frame set.

[0096] In some embodiments, after creating a to-be-tracked frame set for an associated detection frame set, during subsequent target tracking, the detected detection frame sets of the other frames can be matched with the to-be-tracked frame set in other video frames. If the to-be-tracked frame set is successfully matched in a first number of consecutive other video frames, the target body corresponding to the to-be-tracked frame set is created as a new tracking body frame set. Alternatively, if the to-be-tracked frame set is successfully matched within a preset first time period, the target body corresponding to the to-be-tracked frame set is created as a new tracking body frame set. Furthermore, an identifier can be assigned to the new tracking body set to identify the target body corresponding to the tracking body set.

[0097] In some embodiments, after the tracking state of the tracking frame set is determined to be a lost state, in the subsequent tracking process of the target, if the tracking state of the tracking frame set is a lost state in a second number of other consecutive video frames, that is, in the second number of video frames, for example, the number of consecutive lost video frames is greater than the second number (30 frames), the tracking frame set does not match the associated detection frame set, then the target body corresponding to the tracking frame set is deleted, and the identifier of the target body can also be deleted.

[0098] As an example, taking the tracking target as a human target, the first part is the face, the second part is the head and shoulders, and the third part is the body. When the face frame or head and shoulders frame of the tracking target disappears, it is still possible to rely on the human body frame in the associated detection frame set and the tracking frame set to complete the matching, so that when the face frame and head and shoulders frame of the target body appear again, the original identifier ID of the target body is still maintained.

[0099] In this embodiment, during the target tracking process, by matching the smallest unit (part) of the associated detection frame set with the tracking frame set, problems such as the target turning its head or being blocked during the tracking process can be solved, and the accuracy of target tracking can be improved.

[0100] In some embodiments, see Figure 12 and Figure 13 , Figure 12 This is a flow chart of the second embodiment of the target tracking method of the present application. The method may include the following steps:

[0101] S21: Determine a target tracking frame set that matches the associated detection frame set from a plurality of tracking frame sets; wherein the target bodies corresponding to the target tracking frame set and the associated detection frame set belong to the same target body.

[0102] The tracking frame set includes a set of associated detection frames obtained by detecting at least two parts of a historical video frame, and the historical video frame includes a video frame before the current video frame in a preset video.

[0103] The following embodiments of the application take the first part as the face part, the second part as the head-shoulder part, and the third part as the body part as examples for illustration.

[0104] S22: Track the target object in other video frames based on the set of associated bounding boxes of the target object and the set of target tracking boxes.

[0105] The specific implementation process of steps S21 to S22 in this embodiment can refer to the implementation process of steps S131 to S132 in the above-mentioned embodiments, which will not be repeated here.

[0106] In some embodiments, during the tracking of the target, a snapshot image of the target object can be obtained. After step S22 or the above-mentioned step S13, the following step S23 can be included.

[0107] In actual use scenarios, multiple repeated snapshots are often caused by some targets lingering in the detection area. Especially in environments such as reflection, occlusion, backlight, and the like, the feature matching is difficult and cannot be de-duplicated. Therefore, in order to reduce the occurrence of this phenomenon from the root, the application can determine whether to obtain a snapshot image of the target object before step S23.

[0108] The set of associated bounding boxes includes at least two part bounding boxes, wherein the at least two part bounding boxes include a to-be-associated bounding box and at least one other part bounding box matched with the to-be-associated bounding box.

[0109] It can be determined whether the movement displacement of the to-be-associated bounding box crosses the preset mixing line. If the movement displacement of the to-be-associated bounding box crosses the preset mixing line, a snapshot image of the target object corresponding to the to-be-associated bounding box is obtained. The to-be-associated bounding box can be at least one of the first part bounding box, the second part bounding box, and the third part bounding box. The preset mixing line can be a line set in a preset region of a preset video, which is not limited by the application.

[0110] Please refer to Figure 14 , the application takes the to-be-associated bounding box as the second part bounding box (head-shoulder part bounding box) as an example. Since the face bounding box will disappear when the target turns its head or turns its back, and the body bounding box will not be detected when there are many people and it is crowded, by judging that the motion trajectory of the head-shoulder part bounding box crosses the mixing line, a snapshot image of the target object corresponding to the head-shoulder part bounding box is obtained. The accuracy of the snapshot can be improved.

[0111] S23: Determine a snapshot image for the target object based on the set of target tracking boxes.

[0112] The set of target tracking boxes includes at least one of the following: a first part image sequence corresponding to the first part, a second part image sequence corresponding to the second part, and a third part image sequence corresponding to the third part.

[0113] The captured images include a first part captured image corresponding to the first part and / or a third part captured image corresponding to the third part.

[0114] In some embodiments, a corresponding part image sequence can be obtained based on a part tracking frame sequence in a target tracking frame set. The target tracking frame set includes at least one of a first part tracking frame, a second part tracking frame, and a third part tracking frame in multiple video frames. The corresponding part image can be obtained based on the part detection frame in each video frame.

[0115] In some embodiments, see Figure 15 The above step S23, determining a captured image for the target object based on the target tracking frame set, may include the following steps:

[0116] S231: Obtain a first evaluation value and a second evaluation value of each first part image in the first part image sequence; and select a captured image of the first part of the target object based on the first evaluation value and the second evaluation value.

[0117] A first neural network model can be used to obtain a first evaluation value for each first part image in a first part image sequence, wherein the first evaluation value is a quality evaluation value, a completeness evaluation value, etc. This application uses the first evaluation value as an example for explanation, and the quality evaluation value can represent the image quality of the first part image, and the quality evaluation value is proportional to the image quality. A second neural network model is used to obtain a second evaluation value for each first part image in a first part image sequence, wherein the second evaluation value is an angle evaluation value, a rotation evaluation value, etc. This application uses the second evaluation value as an example for explanation, and the angle evaluation value can represent the angle evaluation of the first part (face) in the first part image, and the angle range can include 0° to 360° or -180° to 180°. This application is not limited to this.

[0118] Based on the first evaluation value and the second evaluation value of each first-part image, an image of the first part of the target object is selected as the first-part snapshot image. For example, if the first evaluation value of the first part image meets a first evaluation requirement (e.g., the first evaluation value is greater than a first preset evaluation threshold), and the second evaluation value meets a second evaluation requirement (e.g., the second evaluation value is within a preset angle range), the first part image is selected as the first-part snapshot image.

[0119] In some embodiments, a first weight can be set for the first evaluation value, and a second weight can be set for the second evaluation value. The total evaluation value of each first part image is obtained by multiplying the first evaluation value by the first weight and the product of the second evaluation value by the second weight, so as to select the first part snapshot image based on the total evaluation value.

[0120] In some embodiments, the training process of the first neural network model and the second neural network model using the sample images is consistent with the above-mentioned use process, and the present application will not be repeated here.

[0121] S232: Obtain a third evaluation value of each third part image in the third part image sequence; based on the third evaluation value, select a third part snapshot image of the target object.

[0122] The third evaluation value of each third part image in the third part image sequence can be obtained by using the third audit network model; wherein the third evaluation value is a quality evaluation value, a completeness degree, etc., and the present application takes the third evaluation value as a quality evaluation value as an example for description.

[0123] Based on the third evaluation value of each third part image, the third part image with the highest third evaluation value can be selected as the third part snapshot image of the target object.

[0124] In some embodiments, the training process of the third neural network model using the sample images is consistent with the above-mentioned use process, and the present application will not be repeated here.

[0125] In some embodiments, please refer to Figure 16 and Figure 13 , Figure 16 is a flowchart of the third embodiment of the target tracking method of the present application. The method can include the following steps:

[0126] S31: Determine a snapshot image for the target object based on the target tracking frame set.

[0127] Among the target tracking frame set, at least one of the following is included: a first part image sequence corresponding to the first part, a second part image sequence corresponding to the second part, and a third part image sequence corresponding to the third part; the snapshot image includes a first part snapshot image corresponding to the first part and / or a third part snapshot image corresponding to the third part.

[0128] This embodiment takes the first part as the face part, the second part as the head and shoulder part, and the third part as the body part as an example for description.

[0129] The specific implementation process of step S31 in this embodiment can refer to the implementation process of step S23 in the above-mentioned embodiments, and the present application will not be repeated here.

[0130] In some embodiments, after determining the snapshot image for the target object based on the target tracking frame set, the following step S32 can be performed.

[0131] S32: Based on the first part snapshot image and / or the third part snapshot image, perform crowd flow statistics on the target object.

[0132] Since the capture is based on multiple part tracking frames, some parts of the target body are blocked, and the corresponding part detection frame may not be detected, or the images of the first part image sequence or the third part image sequence are not clear, etc., it is possible that the first part capture image or the third part capture image is not selected.

[0133] In the following description of this embodiment, the first part captured image is a facial captured image, and the third part captured image is a body captured image.

[0134] The first case: if only the first part snapshot image is included, or if the first part snapshot image and the third part snapshot image are included, and the first evaluation value is higher than the third evaluation value, the passenger flow statistics of the target object can be performed based on the first part snapshot image.

[0135] Specifically, a first feature similarity between the first part captured image and a first image in a preset first image library of the library object can be obtained. The preset first image library includes first image models of multiple library objects, each of which is a three-dimensional model constructed from a sequence of first part images of the tracking frame set of the library object.

[0136] In some embodiments, each first image model in a preset first image library can be mapped according to the angle evaluation value to obtain a first library image of each library target object. For example, multiple three-dimensional first image models can be mapped to a two-dimensional image (first library image) at the same angle as the first part snapshot image. This allows comparison of multiple images at the same angle, thereby improving image comparison accuracy.

[0137] A first part recognition model (e.g., a face recognition model) is used to extract image features from the first part snapshot image and each of the first library images, thereby obtaining a cosine similarity between the features of the first part snapshot image and the image features of each of the first library images. This cosine similarity can be used as the first feature similarity. The greater the first feature similarity, the more similar the first part snapshot image is to the first library images.

[0138] Based on the first feature similarity, it is determined whether the target body corresponding to the first part snapshot image is repeated with the target body in the gallery. If the first feature similarity is greater than the first similarity threshold, it is determined that the target body corresponding to the first part snapshot image is repeated with the target body in the gallery.

[0139] In some application scenarios, if it is determined that the object corresponding to the first part snapshot image is the same as multiple library objects, it can be determined that the object corresponding to the first part snapshot image is the same as the library object with the greatest similarity to the first feature.

[0140] In the above embodiment, by utilizing the three-dimensional first image model to deduplicate the captured images of the first part, it is possible to map the two-dimensional first image library images (face library images) at the same angle based on the captured images of the first part at different angles (such as facial captured images), thereby greatly improving the accuracy of deduplication of the first part. It can effectively avoid the problem of failure in first part matching due to a large difference between the angle of the captured image and the angle of the image in the preset first image library, resulting in the inability to correctly deduplicate the target object, thereby improving the accuracy of passenger flow statistics.

[0141] In addition, the first image model is obtained by constructing a three-dimensional model based on the first part image sequence of the tracking frame set of the target body in the gallery. The first part image sequence of the first image model contains more timing information and more first part images at different angles. Compared with a single first part image, the present application can model a more accurate and more realistic three-dimensional facial model (first image model).

[0142] During the tracking process, the target may lower its head at a high angle, turn its head at a high yaw angle, or block its face, making it impossible for the camera to capture the face image. This means that the first captured image or the first part image sequence cannot be obtained. Furthermore, on lower-end devices, factors such as backlighting, image blur, and excessive image noise may result in poor quality of the 3D reconstructed face model (first image model), making it impossible to complete normal facial feature matching and target deduplication. The following method can be used to deduplicate targets.

[0143] The second case: if only the third part snapshot image is included, or if the first part snapshot image and the third part snapshot image are included, and the first evaluation value is lower than the third evaluation value, the passenger flow statistics of the target object can be performed based on the third part snapshot image.

[0144] Specifically, the second feature similarity between the third part snapshot image and the third image library in the preset third image library of the library target object may be obtained.

[0145] The preset third image library includes a plurality of third library images of the library target object, and the third library images include a third part snapshot image of the library target object or a third part image sequence of a tracking frame set.

[0146] In the process of obtaining the second feature similarity, the image features of the third part snapshot image and each third gallery image can be extracted respectively; thereby obtaining the cosine similarity between the features of the third part snapshot image and the image features of each third gallery image, and the cosine similarity can be used as the second feature similarity. The larger the value of the cosine similarity, the higher the second feature similarity, and the higher the similarity between the third part snapshot image and the third gallery image.

[0147] Based on the second feature similarity, it is determined whether the target body corresponding to the third part snapshot image is repeated with the target body in the gallery. If the second feature similarity is greater than the second similarity threshold, it is determined that the target body corresponding to the third part snapshot image is repeated with the target body in the gallery.

[0148] In some application scenarios, if it is determined that the target object is repeated with multiple gallery target objects, it may be determined that the target object is repeated with the gallery target object having the greatest similarity with the first feature.

[0149] The third case: if the first part snapshot image and the third part snapshot image are included, and the first evaluation value is lower or higher than the third evaluation value, the passenger flow statistics of the target object can be performed based on the first part snapshot image and the third part snapshot image.

[0150] A first feature similarity is obtained between the first part snapshot image and a first library image in a preset first image library of the library object, and a second feature similarity is obtained between the third part snapshot image and a third library image in a preset third image library of the library object. Based on the first feature similarity and the second feature similarity, a determination is made as to whether the object and the library object are duplicates. If the first feature similarity is greater than a third similarity threshold or the second feature similarity is greater than a fourth similarity threshold, the object is determined to be a duplicate of the library object. The third and fourth similarity thresholds may be the same or different.

[0151] In some embodiments, if it is determined that the target object is a duplicate of the library target object, that is, the target object already exists in the preset first image library and / or the preset third image library, indicating that the passenger flow count of the target object has been performed before, the passenger flow count of the target object will not be performed.

[0152] In some embodiments, if it is determined that the target object is not repeated with the library target object, that is, the target object does not exist in the preset first image library and the preset third image library, indicating that the passenger flow count of the target object has not been performed before, the passenger flow count of the target object is performed.

[0153] Optionally, the target object can also be used as a subsequent gallery target. A preset neural network can be used to construct a corresponding first image model for the image sequence of the first part of the target object. For example, taking the first part as the face, a three-dimensional face model can be constructed for the image sequence of the first part. The three-dimensional face model can be stored in a point cloud data format as a three-dimensional first image model. The first image model of the target object can then be added to the preset first image library so that the target object can be used as a gallery target for subsequent deduplication.

[0154] In some embodiments, taking the first part of the face as an example, a first-part image sequence (a facial image sequence) can be input into a preset neural network. Key point information of the face in the facial image sequence can be extracted using the preset neural network. Face alignment is performed based on the key point information. The extracted key point information is then used to generate a three-dimensional face model, also known as the first image model. Because the first-part image sequence is a complete sequence of images of the tracking target and includes multiple angles of the face, the reconstructed three-dimensional face model more closely matches the contours of the real face.

[0155] In addition, even if the first part of the target body is blocked, the first image model can be reconstructed based on a single or multiple first part image sequences, and the complete face can still be reconstructed. Compared with the two-dimensional face pictures as the library images of the library targets, the three-dimensional face model more completely expresses the real existence of the target body, and the three-dimensional face model also contains more facial information.

[0156] Optionally, the captured images of the third part of the target body (body image data) can be added to the preset third image library. In some application scenarios, the image sequence of the third part of the target body (body image data) can be added to the preset third image library so that the target body can be used as the library target body for subsequent deduplication.

[0157] In this embodiment, a three-dimensional first image model is used to map the angle evaluation values ​​to obtain multiple first gallery images, and then features of the multiple first gallery images and the first part snapshot images are extracted respectively for deduplication. This can avoid the problem of face deduplication failure caused by a large difference between the angle of the first part snapshot image and the angle of the first gallery image in the preset first image library.

[0158] In addition, when it is impossible to deduplicate the target object using the first part snapshot image, the third part snapshot image and the third library image in the preset third image library can be used for deduplication, thereby increasing the accuracy of target object deduplication.

[0159] Furthermore, by combining the first and third body part snapshot images to deduplicate the target, the third body part snapshot images (body snapshots) can be used for deduplication when there is no first body part image sequence (face snapshots) or the camera imaging quality is low, resulting in blurred facial images. When both the first and third body part snapshot images are present, the snapshot with a higher quality score can be used for deduplication. This approach increases the algorithm's applicability and accuracy, improves the accuracy of target deduplication, avoids counting the same target multiple times, and thus improves the accuracy of passenger flow statistics.

[0160] For the above embodiment, this application also provides a target tracking device. Figure 17 , Figure 17 is a structural schematic diagram of a first embodiment of a target tracking device.

[0161] The target tracking device 40 comprises a detection module 41, an association module 42 and a tracking module 43. The detection module 41, the association module 42 and the tracking module 43 are connected.

[0162] The detection module 41 is configured to detect at least two parts of a current video frame of a preset video to obtain a plurality of part detection boxes.

[0163] The association module 42 is configured to determine a set of associated detection boxes of a target body from the plurality of part detection boxes, wherein the set of associated detection boxes comprises at least two part detection boxes, and each part detection box in the set of associated detection boxes belongs to the same target body and belongs to a different part category.

[0164] The tracking module 43 is configured to track the target body in other video frames based on the set of associated detection boxes of the target body, wherein the other video frames are video frames after the current video frame in the preset video.

[0165] The specific implementation of this embodiment can refer to the implementation process of the above-mentioned embodiments, which will not be repeated here.

[0166] Please refer to Figure 18 , Figure 18 is a structural schematic diagram of a second embodiment of a target tracking device. The target tracking device 40 comprises a detection module 41, an association module 42, a tracking module 43, a snapshot module 44 and a passenger flow statistical module 45.

[0167] Different from the above-mentioned embodiments, the target tracking device 40 in this embodiment further comprises a snapshot module 44 and / or a passenger flow statistical module 45.

[0168] The tracking module 43 is configured to determine a target tracking box set matched with the set of associated detection boxes from a plurality of tracking box sets; and track the target body in other video frames based on the set of associated detection boxes of the target body and the target tracking box set. The tracking box set comprises the set of associated detection boxes obtained by detecting at least two parts of a historical video frame, and the historical video frame comprises a video frame before the current video frame in the preset video; and the target bodies corresponding to the set of associated detection boxes and the target tracking box set belong to the same target body.

[0169] The capture module 44 is configured to determine a captured image for the target object based on the target tracking frame set. The target tracking frame set includes at least one of: a first-part image sequence corresponding to a first part, a second-part image sequence corresponding to a second part, and a third-part image sequence corresponding to a third part; and the captured image includes a first-part captured image corresponding to the first part and / or a third-part captured image corresponding to the third part.

[0170] The passenger flow statistics module 45 is used to perform passenger flow statistics on the target object based on the first part snapshot image and / or the third part snapshot image.

[0171] The specific implementation of this embodiment can refer to the implementation process of the above embodiment, which will not be repeated here.

[0172] For the above embodiment, this application provides a computer device, see Figure 19 , Figure 19 1 is a schematic diagram of the structure of an embodiment of a computer device of the present application. The computer device 50 includes a memory 51 and a processor 52, wherein the memory 51 and the processor 52 are coupled to each other, the memory 51 stores program data, and the processor 52 is used to execute the program data to implement the steps of any embodiment of the target tracking method described above.

[0173] In this embodiment, the processor 52 may also be referred to as a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip having signal processing capabilities. The processor 52 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The general-purpose processor may be a microprocessor, or the processor 52 may be any conventional processor.

[0174] The specific implementation of this embodiment can refer to the implementation process of the above embodiment, which will not be repeated here.

[0175] The method of the above embodiment can be implemented in the form of a computer program, so this application proposes a storage device, see Figure 20 , Figure 20 The storage device 60 stores program data 61 that can be run by a processor. The program data 61 can be executed by the processor to implement the steps of any embodiment of the target tracking method described above.

[0176] The specific implementation of this embodiment can refer to the implementation process of the above embodiment, which will not be repeated here.

[0177] The storage device 60 of this embodiment can be a medium that can store program data 61, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or it can also be a server that stores the program data 61. The server can send the stored program data 61 to other devices for execution, or it can also execute the stored program data 61 by itself.

[0178] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation methods described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0179] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0180] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0181] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage device, which is a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for enabling an electronic device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the various embodiments of the present application.

[0182] Obviously, those skilled in the art should understand that the modules or steps of the present application described above can be implemented using a general-purpose computing device. They can be concentrated on a single computing device or distributed across a network consisting of multiple computing devices. Alternatively, they can be implemented using program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0183] The above description is merely an embodiment of the present application and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A target tracking method, characterized in that: The method comprises: Performing detection on at least two parts of a current video frame of a preset video to obtain multiple part detection frames; Determining a set of associated detection frames of the target object from the multiple part detection frames, wherein the set of associated detection frames includes at least two part detection frames, and each of the part detection frames in the set of associated detection frames belongs to the same target object and belongs to different part categories; Tracking the target in other video frames based on the associated detection frame set of the target, where the other video frames are video frames in the preset video that are located after the current video frame; and tracking the target using the part detection frames of other parts when no part detection frame of one part can be detected; Determining a captured image for the target body based on the target tracking frame set; wherein the captured image includes a first part captured image corresponding to the first part and / or a third part captured image corresponding to the third part; Performing passenger flow statistics on the target object based on the first part snapshot image and / or the third part snapshot image includes: In the case where the first part snapshot image is included, obtaining a first feature similarity between the first part snapshot image and a first image in a preset first image library of a library object, and determining whether the object corresponding to the first part snapshot image is the same as the library object based on the first feature similarity; if the object is the same as the library object, not counting the number of passengers for the object; Among them, the first gallery image of each gallery target body is obtained by mapping each first image model in the preset first image library according to the angle evaluation value of the first part capture image, and the first image model is a three-dimensional model constructed based on the first part image sequence of the tracking frame set of the gallery target body.

2. The method according to claim 1, characterized in that Determining a set of associated detection frames of the target object from the multiple part detection frames includes: Selecting a to-be-associated detection frame from the plurality of part detection frames, wherein the part detection frames other than the to-be-associated detection frame are all used as other detection frames; selecting at least one of the other detection frames that matches the detection frame to be associated as the target other detection frame; The to-be-associated detection frame is associated with other detection frames of the target to obtain a set of associated detection frames of the target body.

3. The method according to claim 2, characterized in that The selecting at least one other detection frame that matches the detection frame to be associated as the target other detection frame includes: Obtaining the degree of overlap between each of the other detection frames and the detection frame to be associated, and selecting the other detection frame whose degree of overlap meets the first position requirement as the target other detection frame; or, Obtain the preset point position relationship corresponding to each of the other detection frames, and select the other detection frame whose preset point position relationship meets the second position requirement as the target other detection frame, wherein the preset point position relationship corresponding to the other detection frame is the position relationship between the preset position point in the other detection frame and the preset position point in the detection frame to be associated.

4. The method according to claim 2, characterized in that The at least two parts include a first part, a second part, and a third part belonging to different part categories, and the multiple part detection frames include a plurality of first part detection frames corresponding to the first part, a plurality of second part detection frames corresponding to the second part, and a plurality of third part detection frames corresponding to the third part; The selecting a detection frame to be associated from the plurality of part detection frames, and selecting at least one other detection frame that matches the detection frame to be associated as a target other detection frame, include: Using the first part detection frame as the detection frame to be associated; The selecting at least one other detection frame that matches the detection frame to be associated includes: Selecting a target second part detection frame that matches the detection frame to be associated from the plurality of second part detection frames; Selecting a target third part detection frame that matches the target second part detection frame from among the plurality of third part detection frames; The target second part detection frame and / or the target third part detection frame are used as the target other detection frames.

5. The method according to claim 1, characterized in that Tracking the target object in other video frames based on the associated detection frame set of the target object includes: Determining, from a plurality of tracking frame sets, a target tracking frame set that matches the associated detection frame set; wherein the tracking frame set includes a set of associated detection frames obtained by detecting the at least two parts on historical video frames, the historical video frames including video frames before the current video frame in the preset video; and the target bodies corresponding to the target tracking frame set and the associated detection frame set belong to the same target body; The target object in the other video frames is tracked based on the associated detection frame set of the target object and the target tracking frame set.

6. The method according to claim 5, characterized in that The determining, from the plurality of tracking frame sets, a target tracking frame set that matches the associated detection frame set comprises: Each of the tracking frame sets is used as a tracking frame set to be processed; wherein the tracking frame set to be processed includes a part tracking frame corresponding to at least one part; Determining part tracking frames and part detection frames belonging to the same part category based on the set of tracking frames to be processed and the set of associated detection frames; The part detection frames and part tracking frames belonging to the same part category are matched respectively to select a target tracking frame set that matches the associated detection frame set.

7. The method according to claim 6, characterized in that If there are multiple part tracking frames of the same part category, matching the part detection frames and part tracking frames belonging to the same part to select a target tracking frame set that matches the associated detection frame set includes: Obtaining the degree of overlap between the part tracking frame corresponding to each part and the part detection frame; Based on the statistical values ​​of the overlapping degrees corresponding to the parts, a target tracking frame set that matches the associated detection frame set is selected.

8. The method according to claim 5, characterized in that Tracking the target object in the other video frames based on the associated detection frame set and the target tracking frame set of the target object includes: If the associated detection frame set of the target object successfully matches the target tracking frame set, the associated detection frame set is added to the target tracking frame set, wherein the tracking state of the target tracking frame set is determined to be a valid state; If the associated detection frame set of the target body fails to match the tracking frame set, a to-be-tracked frame set is created for the associated detection frame set; and / or, the tracking state of the tracking frame set is determined to be a lost state.

9. The method according to claim 8, characterized in that After creating a to-be-tracked frame set for the associated detection frame set, the method further includes: In the other video frames, matching the detected other frame detection frame sets with the to-be-tracked frame set; If the to-be-tracked frame set is successfully matched in a first number of other consecutive video frames, the target body corresponding to the to-be-tracked frame set is created as a new tracking body frame set; and / or, After determining the tracking state of the tracking frame set as a lost state, the method further includes: If the tracking status of the tracking frame set in a second number of consecutive other video frames is all in the lost state, the target body corresponding to the tracking frame set is deleted.

10. The method according to claim 5, characterized in that The target tracking frame set includes at least one of: a first part image sequence corresponding to the first part, a second part image sequence corresponding to the second part, and a third part image sequence corresponding to the third part.

11. The method according to claim 10, characterized in that The determining of a captured image for the target object based on the target tracking frame set includes: Obtaining a first evaluation value and a second evaluation value of each first part image in the first part image sequence; selecting a snapshot image of the first part of the target body based on the first evaluation value and the second evaluation value; and / or, Acquire a third evaluation value of each third part image in the third part image sequence; and select the captured image of the third part of the target object based on the third evaluation value.

12. The method according to claim 11, characterized in that In a case where the captured image includes a captured image of a first part, the first evaluation value is higher than the third evaluation value; or The performing passenger flow statistics on the target object based on the first part snapshot image and / or the third part snapshot image includes: In a case where the third part snapshot image is included, obtaining a second feature similarity between the third part snapshot image and a third library image in a preset third image library of the library object, and determining whether the object corresponding to the third part snapshot image is the same as the library object based on the second feature similarity, wherein the first evaluation value is lower than the third evaluation value; If the target object is repeated with the target object in the gallery, the passenger flow count for the target object will not be performed.

13. The method according to claim 12, characterized in that The second evaluation value includes an angle evaluation value; The obtaining of the first feature similarity between the first part snapshot image and a first image in a preset first image library of the library target object includes: Mapping each first image model of the preset first image library according to the angle evaluation value to obtain a first library image of each library target object; respectively extracting image features of the first part snapshot image and each of the first gallery images; Obtaining a cosine similarity between a feature of the first part snapshot image and an image feature of each of the first gallery images as the first feature similarity; The obtaining of the second feature similarity between the third part snapshot image and a third image in a preset third image library of the library target object includes: Extracting image features of the third part snapshot image and each of the third gallery images respectively; wherein the third gallery images include the third part snapshot image of the gallery target or a third part image sequence of the tracking frame set; The cosine similarity between the feature of the third part snapshot image and the image feature of each of the third gallery images is obtained as the second feature similarity.

14. The method according to claim 12, characterized in that The method further comprises: If the target object is not repeated with the target object in the gallery, counting the passenger flow of the target object; and A corresponding first image model is constructed for the image sequence of the first part of the target body, and the first image model is added to the preset first image library, and / or, the captured image of the third part of the target body or the image sequence of the third part is added to the preset third image library.

15. The method according to claim 9, characterized in that The at least two part detection frames in the associated detection frame set include a detection frame to be associated and at least one other part detection frame matching the detection frame to be associated; Before determining a captured image for the target object based on the target tracking frame set, the method includes: If the movement displacement of the detection frame to be associated crosses the preset mixing line, a captured image of the target object corresponding to the detection frame to be associated is obtained.

16. The method according to claim 1, wherein The at least two parts include a first part, a second part and a third part belonging to different part categories, the first part is a face part, the second part is a head and shoulder part, and the third part is a body part.

17. A computer device, characterized in that: The method comprises a memory and a processor coupled to each other, wherein the memory stores program data, and the processor is configured to execute the program data to implement the steps of the method according to any one of claims 1 to 16.

18. A storage device, characterized in that: Program data capable of being executed by a processor is stored, and the program data is used to implement the steps of the method according to any one of claims 1 to 16.

Images