Target trajectory recognition method and device, storage medium and program product

Abstract

The invention provides a target track identification method and device, a storage medium and a program product, and the method comprises the steps: carrying out the track prediction in a current video frame according to the historical track data of a known target, and obtaining the prediction position of the known target and a corresponding prediction box; performing target detection and attribute identification on the current video frame to obtain a detection position of a detection target, a detection frame corresponding to the detection position and attribute data of the detection target in the current video frame; and determining a first intersection-to-union ratio of a first detection frame and a first prediction frame which are intersected in the current video frame, performing target matching identification on the known target according to the first intersection-to-union ratio and attribute data of a first detection target of the first detection frame in the current video frame, and updating historical track data of the known target under the condition that the known target is identified. According to the embodiment, the accuracy of detection target identification can be improved, the situation of track association errors caused by target identification errors is reduced, and the accuracy of track prediction is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a target trajectory recognition method, device, storage medium and program product. Background Art

[0002] With the continuous development of deep learning and machine vision technology, AI-based target recognition and tracking technology is playing an increasingly important role in video scene monitoring, intelligent transportation, and intelligent driving.

[0003] Target recognition and tracking technology includes two parts: target recognition and trajectory tracking prediction. Trajectory tracking recognition in related technologies usually predicts the subsequent trajectory of the target based on the existing trajectory of the target, and corrects the predicted trajectory based on the identified position of the target. The applicant found that this type of trajectory prediction method is relatively simple, and it is easy to have target recognition errors leading to trajectory association errors, and the trajectory prediction accuracy is not high.

[0004] Application Contents

[0005] The present application provides a target trajectory recognition method, device, storage medium and program product to solve the problem in the related art that the trajectory prediction method is relatively simple, it is easy to have target recognition errors leading to trajectory association errors, and the trajectory prediction accuracy is not high.

[0006] In a first aspect, an embodiment of the present application provides a method for identifying a target trajectory, comprising:

[0007] Obtain the current video frame in the video data to be detected, and perform trajectory prediction in the current video frame based on the historical trajectory data of the known target to obtain the predicted position of the known target, where one predicted position corresponds to one prediction box;

[0008] Perform target detection on the current video frame to obtain the detection position of the detection target and the detection frame corresponding to the detection position, and perform attribute recognition based on the detection frame of the detection target to obtain attribute data of the detection target in the current video frame. The detection target may be one or more;

[0009] Determine an intersection-and-union ratio between a first detection frame and a first prediction frame in a current video frame to obtain a first intersection-and-union ratio, wherein the first detection frame is a detection frame in the current video frame that intersects with the prediction frame, and the first prediction frame is a prediction frame in the current video frame that intersects with the first detection frame;

[0010] According to the first intersection-union ratio and the attribute data of the first detection target in the current video frame, target matching and recognition are performed on the known target, and when the known target is recognized in the current video frame, the historical trajectory data of the known target is updated, and the first detection target is the detection target corresponding to the first detection frame.

[0011] Optionally, target matching and recognition are performed on the known target according to the first intersection-over-union ratio and the attribute data of the first detection target in the current video frame, and when the known target is recognized in the current video frame, historical trajectory data of the known target is updated, including:

[0012] When the first intersection-over-union ratio is greater than or equal to the first threshold, performing attribute jump detection on the known target according to the attribute data of the first detection target in the current video frame and the attribute data of the known target in the previous video frame;

[0013] According to the attribute jump of the known target, the known target is matched and recognized, and the historical trajectory data of the known target is updated when the known target is recognized in the current video frame.

[0014] Optionally, according to the attribute jump of the known target, target matching recognition is performed on the known target, and when the known target is recognized in the current video frame, the historical trajectory data of the known target is updated, including:

[0015] When a change in the orientation attribute of a known target is detected, it is determined that the known target is not recognized in the current video frame, the trajectory state of the known target is marked as a lost state, and the historical trajectory data of the known target is not updated;

[0016] When it is detected that there is no orientation attribute jump and appearance attribute jump of the known target, it is determined that the known target is recognized in the current video frame, the predicted position of the known target in the current video frame is corrected to the detection position of the first detection target, and the historical trajectory data of the known target is updated.

[0017] Optionally, according to the attribute jump of the known target, target matching recognition is performed on the known target, and when the known target is recognized in the current video frame, the historical trajectory data of the known target is updated, including:

[0018] When it is detected that there is no orientation attribute jump of the known target but there is an appearance attribute jump, attribute data matching is performed on other detected targets other than the first detected target in the current video frame according to the attribute data of the known target in the previous video frame;

[0019] When the attribute data of other detection targets are matched, the known target is matched and identified according to the detection position of the second detection target. When the known target is identified in the current video frame, the historical trajectory data of the known target is updated. The second detection target is the other detection target in the current video frame that matches the attribute data of the known target in the previous video frame.

[0020] Optionally, target matching and recognition are performed on the known target according to the detection frame of the second detection target, and when the known target is recognized in the current video frame, historical trajectory data of the known target is updated, including:

[0021] Determine an intersection-and-union ratio of a detection frame of a second detection target in a current video frame and a prediction frame of a known target to obtain a second intersection-and-union ratio;

[0022] When the second intersection-over-union ratio is less than the second threshold, it is determined that the known target is not recognized in the current video frame, the trajectory state of the known target is marked as a lost state, the historical trajectory data of the known target is not updated, and the second threshold is less than the first threshold;

[0023] When the second intersection-over-union ratio is greater than or equal to the second threshold, it is determined that a known target is recognized in the current video frame, the predicted position of the known target in the current video frame is corrected to the detection position of the second detection target, and the historical trajectory data of the known target is updated.

[0024] Optionally, the attribute data includes an orientation attribute and an appearance attribute, and attribute jump detection is performed on the first detection target according to the attribute data of the first detection target in the current video frame and the attribute data of the known target in the previous video frame, including:

[0025] When it is detected that the orientation attribute of the first detection target in the current video frame is opposite to the orientation attribute of the known target in the previous video frame, it is determined that there is an orientation attribute jump of the known target;

[0026] When it is determined that the appearance attribute of the first detection target in the current video frame and the appearance attribute of the known target in the previous video frame meet a preset jump condition, it is determined that there is an appearance attribute jump of the known target.

[0027] Optionally, after performing target matching and recognition on the known target according to the first intersection-over-union ratio and the attribute data of the first detection target in the current video frame, the method further includes:

[0028] In the case where a known target is identified in the current video frame, if the trajectory state of the known target in the previous video frame is a lost state, and the duration between the video frame in which the known target was last identified and the current video frame is less than or equal to the preset duration, the trajectory state of the known target is marked as a tracking state, and the historical trajectory data of the known target is updated according to the position of the known target in the current video frame;

[0029] When a known target is not recognized in the current video frame, if the trajectory state of the known target in the previous video frame is a lost state, and the duration between the video frame in which the known target was last recognized and the current video frame is greater than the preset duration, the trajectory state of the known target is marked as a disconnected state, and the historical trajectory data of the known target is not updated.

[0030] In a second aspect, an embodiment of the present application provides a target trajectory recognition device, including:

[0031] An acquisition module, used for acquiring a current video frame in the video data to be detected;

[0032] The prediction module is used to perform trajectory prediction in the current video frame based on the historical trajectory data of the known target to obtain the predicted position of the known target, where one predicted position corresponds to one prediction box;

[0033] A detection module is used to detect a target in the current video frame, obtain a detection position of the detection target and a detection frame corresponding to the detection position, and perform attribute recognition based on the detection frame of the detection target to obtain attribute data of the detection target in the current video frame. The detection target may be one or more;

[0034] A determination module is used to determine an intersection-and-union ratio between a first detection frame and a first prediction frame in a current video frame to obtain a first intersection-and-union ratio, wherein the first detection frame is a detection frame in the current video frame that intersects with the prediction frame, and the first prediction frame is a prediction frame in the current video frame that intersects with the detection frame;

[0035] The recognition module is used to perform target matching and recognition on the known target according to the first intersection-union ratio and the attribute data of the first detection target in the current video frame, and update the historical trajectory data of the known target when the known target is recognized in the current video frame. The first detection target is the detection target corresponding to the first detection frame.

[0036] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned target trajectory recognition method when executing the computer program.

[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned target trajectory recognition method are implemented.

[0038] In a fifth aspect, an embodiment of the present application provides a computer program product, characterized in that the computer program product includes a computer program, which, when executed by a processor, implements the steps of the above-mentioned target trajectory recognition method.

[0039] In a scheme provided by the above-mentioned target trajectory recognition method, device, storage medium and program product, a current video frame in the video data to be detected is obtained, and trajectory prediction is performed in the current video frame according to the historical trajectory data of the known target to obtain the predicted position of the known target, and one predicted position corresponds to a prediction box; target detection is performed on the current video frame to obtain the detection position of the detection target and the detection box corresponding to the detection position, and attribute recognition is performed based on the detection box of the detection target to obtain attribute data of the detection target in the current video frame, and the detection target is one or more; the intersection and union ratio of the first detection box and the first prediction box in the current video frame is determined to obtain the first intersection and union ratio, the first detection box is the detection box that intersects with the prediction box in the current video frame, and the first prediction box is the prediction box that intersects with the detection box in the current video frame; target matching recognition is performed on the known target according to the first intersection and union ratio and the attribute data of the first detection target in the current video frame, and when the known target is recognized in the current video frame, the historical trajectory data of the known target is updated, and the first detection target is the detection target corresponding to the first detection box. In this embodiment, a prediction frame of a known target in a current video frame is determined, and target detection is performed on the current video frame to obtain a detection frame and attribute data of the detected target. Then, the known target is identified based on a first intersection-over-union ratio of an intersecting first prediction frame and a first detection frame, and attribute data of the detection frame and the first detected target of the prediction frame. When a known target is identified, historical trajectory data of the known target is updated, which can improve the accuracy of target recognition, reduce the situation where trajectory association errors are caused by target recognition errors, and improve the accuracy of trajectory prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0041] Figure 1 is a structural schematic diagram of a target trajectory recognition system in one embodiment of the present application;

[0042] Figure 2 This is a flow chart of a target trajectory recognition method in one embodiment of the present application;

[0043] Figure 3 is a schematic diagram of the trajectory change of a known target in an embodiment of the present application;

[0044] Figure 4 is a schematic diagram of a prediction box and a detection box in a certain video frame in an embodiment of the present application;

[0045] Figure 5It is a schematic diagram of determining the intersection and union ratio in one embodiment of the present application;

[0046] Figure 6 yes Figure 2 A schematic diagram of an implementation process of step S40;

[0047] Figure 7 yes Figure 6 A schematic diagram of an implementation process of step S41;

[0048] Figure 8 yes Figure 6 A schematic diagram of an implementation flow of step S42;

[0049] Fig. 9 is another flow chart of a target trajectory recognition method in one embodiment of the present application;

[0050] Fig.10 yes Figure 1 A structural schematic diagram of a target trajectory recognition device;

[0051] Fig.11 It is a structural schematic diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or their collections. It should also be understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0054] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0055] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0056] It should be understood that the size of the serial numbers of the steps in the following embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0057] In order to illustrate the technical solution of the present application, a specific embodiment is provided below for illustration.

[0058] It should be understood that the trajectory tracking and recognition in the related art usually predicts the subsequent trajectory of the tracking target based on the existing trajectory of the tracking target, and corrects the predicted trajectory according to the identified tracking target position. This type of trajectory prediction method is relatively simple, and it is easy for target recognition errors to lead to trajectory association errors, and the trajectory prediction accuracy is not high. For example, in a complex scene with a dense crowd, there are multiple tracking targets in the video, and there are mutual occlusions and truncations between different tracking targets. In this case, the positions of different tracking targets identified overlap, and it is easy for the positions of different tracking targets to conflict with each other, resulting in tracking target position recognition errors and trajectory association errors; or, the tracking target disappears at a certain position, and another target appears from the disappearing position. In the related technology, it is easy to identify two different tracking targets as the same target and associate the trajectories of the two, resulting in trajectory association errors and low trajectory prediction accuracy.

[0059] In view of the above problems, the target trajectory recognition method provided in the embodiment of the present application can be applied in the following aspects: Figure 1 In the target trajectory recognition system shown, the target trajectory recognition system includes a target trajectory recognition device and a terminal device, wherein the terminal device communicates with the target trajectory recognition device via a network or a cable.

[0060] In the actual application process, the terminal device collects the video data to be detected and sends it to the target trajectory recognition device, and the video data to be detected includes multiple video frames. The target trajectory recognition device obtains the current video frame in the video data to be detected, and performs trajectory prediction in the current video frame according to the historical trajectory data of the known target, and obtains a predicted position of the known target, and one predicted position corresponds to a prediction frame. Perform target detection on the current video frame, obtain the detection position of the detection target and the detection frame corresponding to the detection position, and perform attribute recognition based on the detection frame of the detection target to obtain the attribute data of the detection target in the current video frame, and the detection target is one or more; determine the intersection and union ratio of the first detection frame and the first prediction frame in the current video frame, and obtain the first intersection and union ratio. The first detection frame is the detection frame that intersects with the prediction frame in the current video frame, and the first prediction frame is the prediction frame that intersects with the detection frame in the current video frame; according to the first intersection and union ratio and the attribute data of the first detection target in the current video frame, target matching recognition is performed on the known target, and when the known target is recognized in the current video frame, the historical trajectory data of the known target is updated, and the first detection target is the detection target corresponding to the first detection frame.

[0061] In this embodiment, a prediction frame of a known target in a current video frame is determined, and target detection is performed on the current video frame to obtain a detection frame and attribute data of the detected target. Then, the known target is identified based on a first intersection-over-union ratio of an intersecting first prediction frame and a first detection frame, and the attribute data of the detection frame and the first detected target of the prediction frame. When a known target is identified, the historical trajectory data of the known target is updated to correct and update the predicted position of the known target. This can improve the accuracy of target identification, reduce the situation where trajectory association errors are caused by target identification errors, and improve the accuracy of trajectory prediction.

[0062] The terminal device can be various sensors that collect the video data to be detected, such as image sensors, radar sensors, etc., or intelligent devices that can collect the video data to be detected through various sensors, such as vehicles, various personal computers, laptops, smart phones, tablet computers and portable wearable devices. The target trajectory recognition device can be a server, which can be implemented as an independent server or a server cluster composed of multiple servers; it can also include but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices.

[0063] In one embodiment, if Figure 2 As shown, a target trajectory recognition method is provided, and the method is applied in Figure 1 The target trajectory recognition device in the example is used as an example to illustrate, and includes the following steps:

[0064] S10: Obtain a current video frame in the video data to be detected, and perform trajectory prediction in the current video frame according to historical trajectory data of a known target to obtain a predicted position of the known target, where one predicted position corresponds to one prediction box.

[0065] The target trajectory recognition device obtains the video data to be detected from the terminal device, and performs target trajectory tracking and recognition on the video data to be detected. During the target trajectory tracking and recognition process, the target trajectory recognition device obtains the current video frame in the video data to be detected, and any video frame in the video data to be detected can be used as the current video frame.

[0066] Wherein, if the target trajectory recognition device does not record the historical trajectory data of the known target in the video data to be detected, that is, the current video frame is the first frame of the video data to be detected, then the target trajectory recognition device performs target detection on the current video frame, uses the obtained detection position as the recorded trajectory data of the detected target, and performs trajectory tracking and recognition on the next video frame. Wherein, the historical trajectory data of the known target is a set of position data of the known target stored in the form of a time series, indicating the trajectory changes of the known target in each current video frame of the video data to be detected. The historical trajectory data of the known target includes the position of the known target in each video frame of the video data to be detected.

[0067] like Figure 3 As shown, it shows the trajectory change (i.e., position change) of a known target in three consecutive video frames, that is, it shows the process of obtaining historical trajectory data based on the trajectory of a known target. Figure 3 The left part in the middle represents the detection position (such as a black dot) and the detection frame (such as a solid line frame) of the known target (human figure) in the first video frame; Figure 3 The middle part indicates that the position of the known target in the second video frame is predicted and identified based on the position of the known target in the first video frame (i.e., historical trajectory data); Figure 3 The middle right part indicates that the position of the known target in the third video frame is predicted and identified and corrected according to the position of the known target in the first video frame and the second video (ie, historical trajectory data).

[0068] If the target trajectory recognition device records the historical trajectory data of the known target in the video data to be detected, that is, the current video frame is not the first frame of the video data to be detected, the target trajectory recognition device predicts the trajectory (i.e., predicted position) of the known target in the current video frame based on the historical trajectory data of the known target using a prediction algorithm, and obtains a predicted position of the known target in the current video frame, and the predicted position corresponds to a prediction frame. Among them, there are one or more known targets, one known target corresponds to one predicted position, and one predicted position corresponds to one prediction frame.

[0069] For example, after the target trajectory recognition device identifies the position of the known target in a certain video frame, it stores the position of the known target in a certain video frame to obtain historical trajectory data; adopts the Kalman filter algorithm, uses the historical trajectory data of the known target, and performs trajectory tracking prediction based on the dynamic model on the trajectory of the known target in the current video frame to obtain the predicted position of the known target in the current video frame and the prediction box corresponding to the predicted position. For another example, the historical trajectory data of the known target before the current video frame and the current video frame can be used as model inputs, and a pre-trained trajectory prediction model (such as a recurrent neural network model) is used to predict the trajectory of the known target in the current video frame, and the trajectory prediction model outputs the predicted position of the known target in the current video frame and the prediction box corresponding to the predicted position.

[0070] The predicted position of the known target may be represented by the center position coordinates of a rectangular prediction box of the known target in the labeled video frame.

[0071] S20: Perform target detection on the current video frame to obtain a detection position of the detection target and a detection frame corresponding to the detection position, and perform attribute recognition based on the detection frame of the detection target to obtain attribute data of the detection target in the current video frame.

[0072] After obtaining the current video frame in the video data to be detected, the target trajectory recognition device also performs target detection on the current video frame to obtain the detection position of the detection target and the detection frame corresponding to the detection position, and performs attribute recognition based on the detection frame of the detection target to obtain the attribute data of the detection target in the current video frame. There are one or more detection targets.

[0073] Among them, the target trajectory recognition device uses a target detection algorithm to perform target detection on the current video frame to obtain the detection position of the detection target in the current video frame. The detection position is represented by a rectangular detection frame, and the detection position is represented by the coordinates of the center position of the detection frame. After obtaining the detection position and the detection frame, the area where the detection frame of the detection target is located in the current video frame is cropped and segmented to obtain the detection frame area, and a pre-trained target attribute recognition model is used to perform target attribute recognition on the frame area to obtain the attribute data of the detection target in the current video frame. Among them, the attribute data includes at least one of the orientation attribute (such as body orientation, facial orientation, etc.) and appearance attribute of the detection target. The appearance attributes include the appearance color of the detection target, the size of the detection target, etc.

[0074] S30: Determine an intersection-over-union ratio between a first detection frame and a first prediction frame in the current video frame to obtain a first intersection-over-union ratio.

[0075] The target trajectory recognition device determines the intersection-and-union ratio of the first detection frame and the first prediction frame in the current video frame to obtain the first intersection-and-union ratio, where the first detection frame is the detection frame that intersects with the prediction frame in the current video frame, and the first prediction frame is the prediction frame that intersects with the first detection frame in the current video frame. That is, the target trajectory recognition device determines the intersecting detection frame and prediction frame in the current video frame, that is, the detection frame and prediction frame with overlapping areas, and records the two intersecting detection frames and prediction frames as the first detection frame and the first prediction frame, respectively, and determines the intersection-and-union ratio of the first detection frame and the first prediction frame in the current video frame to obtain the first intersection-and-union ratio.

[0076] For example, the historical trajectory data of a known target is Figure 3 As shown in , the current video frame is the fourth video frame. Figure 4 As shown, according to the historical trajectory data of the known target (such as Figure 3 ), predict the prediction box of the known target in the current video frame as Figure 4 The predicted position is the center of the dotted box; target detection is performed on the current video frame, and the detection box is detected as Figure 4 The detection position is the center position of the leftmost solid line frame in . Figure 4 It can be seen that the detection frame does not overlap with the pre-stored frame but intersects, so the detection frame is used as the first detection frame, and the prediction frame is recorded as the first prediction frame, and the intersection and union ratio of the first detection frame and the first prediction frame in the current video frame is determined to obtain the first intersection and union ratio.

[0077] Among them, the intersection-in-union ratio is the ratio of the intersection of two data to the union of two data. That is, the first intersection-in-union ratio is the ratio of the intersection of the first detection frame and the first prediction frame to the union of the first detection frame and the first prediction frame. That is, the first intersection-in-union ratio = (intersection with the first detection frame and the first prediction frame) / (union with the first detection frame and the first prediction frame).

[0078] Take two rectangular boxes as an example, Figure 5 As shown, the intersection of two rectangular boxes represents the overlapping area of ​​the two rectangular boxes, such as Figure 5 The left figure in the figure is shown; the union of two rectangular boxes represents the complete area including the two rectangular boxes, such as Figure 5 The middle figure shows the intersection of two rectangular boxes and the union of two rectangular boxes, such as Figure 5 on the right side of the screen.

[0079] S40: performing target matching and recognition on a known target according to the first intersection-over-union ratio and the attribute data of the first detection target in the current video frame, and updating the historical trajectory data of the known target when the known target is recognized in the current video frame.

[0080] The target trajectory recognition device performs target matching recognition on the known target according to the first intersection-and-union ratio and the attribute data of the first detection target in the current video frame, and updates the historical trajectory data of the known target when the known target is recognized in the current video frame. The first detection target is the detection target corresponding to the first detection frame.

[0081] For example, the attribute data of the known target in the previous video frame of the current video frame can be obtained, and the attribute data of the known target in the previous video frame can be compared with the attribute data of the first detection target in the current video frame. If the attribute data of the known target in the previous video frame is consistent with the attribute data of the first detection target in the current video frame, and the first intersection and union ratio is greater than or equal to a certain threshold, it means that the attributes of the two are consistent, and the first detection frame is relatively close to the predicted frame of the known target, then it is determined that the first detection target corresponding to the first detection frame matches the known target (including position matching and attribute matching). That is, it is determined that the known target is detected in the current video frame, and the predicted position of the known target is not used to update the historical trajectory data, but the detection position of the first detection target is used as the position of the known target in the current video frame to update the historical trajectory data of the known target, that is, the predicted position of the known target in the current video frame is corrected to the detection position of the first detection target in the current video frame, and updated to the historical trajectory data of the known target.

[0082] If the first intersection-over-union ratio is less than a certain threshold, it means that the difference between the detection position of the first detection target in the current video frame and the predicted position of the known target is too large, and it is determined that the first detection target corresponding to the first detection frame does not match the known target. To reduce the situation of target recognition errors, it can be determined that the known target is not recognized in the current video frame, the known target trajectory is lost, the trajectory state of the known target is marked as lost, and the historical trajectory data of the known target is not updated; if the attribute data of the known target in the previous video frame is inconsistent with the attribute data of the first detection target in the current video frame, to avoid the situation of target association errors, it is determined that the known target is not recognized in the current video frame, the trajectory state of the known target is marked as lost, and the historical trajectory data of the known target is not updated. This can reduce the situation of trajectory recognition errors caused by target association errors, reduce the accuracy of trajectory recognition input of the next video frame, and improve the accuracy of trajectory recognition.

[0083] In this embodiment, a prediction frame of a known target in a current video frame is determined, and target detection is performed on the current video frame to obtain a detection frame and attribute data of the detected target. Then, the known target is identified based on a first intersection-over-union ratio of an intersecting first prediction frame and a first detection frame, and the attribute data of the detection frame and the first detected target of the prediction frame. When a known target is identified, the historical trajectory data of the known target is updated to correct and update the predicted position of the known target. This can improve the accuracy of target identification, reduce the situation where trajectory association errors are caused by target identification errors, and improve the accuracy of trajectory prediction.

[0084] In one embodiment, if Figure 6 As shown, in step S40, target matching and recognition are performed on the known target according to the first intersection-over-union ratio and the attribute data of the first detection target in the current video frame, and when the known target is recognized in the current video frame, the historical trajectory data of the known target is updated, which specifically includes the following steps:

[0085] S41: When the first intersection-over-union ratio is greater than or equal to the first threshold, attribute jump detection is performed on the known target according to the attribute data of the first detection target in the current video frame and the attribute data of the known target in the previous video frame.

[0086] After determining the first intersection-and-union ratio of the first detection frame and the first prediction frame, it is determined whether the first intersection-and-union ratio is less than a first threshold value, which is a pre-calibrated intersection-and-union ratio value, and the value range of the first threshold value can be 0.6 to 0.7. When the first intersection-and-union ratio is greater than or equal to the first threshold value, that is, when the detection position of the first detection target corresponding to the first detection frame in the current video frame is close to the predicted position of the known target in the current video frame, the target trajectory recognition device performs attribute jump detection on the known target based on the attribute data of the first detection target in the current video frame and the attribute data of the known target in the previous video frame, and obtains the attribute jump situation of the known target.

[0087] Among them, when the first intersection-over-union ratio is less than the first threshold, that is, the detection position of the first detection target corresponding to the first detection frame in the current video frame has a large error with the predicted position of the known target in the current video frame, it is determined that the known target is not recognized in the current video frame, and the trajectory state of the known target is marked as a lost state. The historical trajectory data of the known target is not updated, and the original historical trajectory data of the known target is directly used to predict the trajectory of the next video frame. This can reduce the situation of trajectory association errors caused by target recognition errors, improve the accuracy of the target trajectory data, and improve the accuracy of subsequent trajectory predictions.

[0088] S42: According to the attribute jump of the known target, target matching and recognition are performed on the known target, and when the known target is recognized in the current video frame, the historical trajectory data of the known target is updated.

[0089] The target trajectory recognition device performs target matching recognition on the known target according to the attribute jump of the known target, and updates the historical trajectory data of the known target when the known target is recognized in the current video frame.

[0090] For example, when it is detected that there is an attribute jump of a known target, such as when the attribute data of the detected target in the current video frame is inconsistent with the attribute data of the known target in the previous video frame, it is determined that the known target is not detected and identified in the current video frame, that is, the trajectory of the known target is lost, then the target trajectory recognition device marks the trajectory state of the known target as a lost state, and does not update the historical trajectory data of the known target, thereby reducing the situation where trajectory association errors are caused by target recognition errors. When it is detected that there is no attribute jump of the first detected target, that is, it is determined that the first detected target matches the known target, it can be determined that the known target is detected and identified in the current video frame, and the predicted position of the known target in the current video frame is corrected to the detected position of the first detected target, and updated to the historical trajectory data of the known target, that is, the detected position of the first detected target in the current video frame is used as the position of the known target in the current video frame, and updated to the historical trajectory data of the known target, so as to improve the accuracy of the target trajectory in the current video frame, so as to improve the accuracy of subsequent trajectory predictions.

[0091] Among them, if the orientation attribute of the first detected target in the current video frame is inconsistent with the orientation attribute of the known target in the previous video frame, or if the appearance attribute of the first detected target in the current video frame is inconsistent with the appearance attribute of the known target in the previous video frame, it can be determined that there is an attribute jump in the known target; if the orientation attribute and appearance attribute of the first detected target in the current video frame are consistent with the orientation attribute and appearance attribute of the known target in the previous video frame of the current video frame, that is, the attributes of the detected target match the known target, it can be determined that there is no attribute jump in the known target between the two previous video frames.

[0092] In this embodiment, when the first intersection-over-union ratio is greater than or equal to the first threshold, attribute jump detection is performed on the known target based on the attribute data of the first detected target in the current video frame and the attribute data of the known target in the previous video frame; target matching recognition is performed on the known target based on the attribute jump of the known target, and when the known target is recognized in the current video frame, the historical trajectory data of the known target is updated. Attribute jump detection can identify abnormal changes of targets between consecutive video frames. By judging abnormal changes, it can reduce the situation of trajectory association errors caused by target detection errors. The introduction of attribute jump detection in the trajectory update process improves the target recognition ability and adaptability in complex scenes, and improves the accuracy and robustness of trajectory tracking recognition.

[0093] In one embodiment, the attribute data includes orientation attributes and appearance attributes. Figure 7 As shown, in step S41, attribute jump detection is performed on the first detection target according to the attribute data of the first detection target in the current video frame and the attribute data of the known target in the previous video frame, which specifically includes the following steps:

[0094] S411: When it is detected that the orientation attribute of the first detection target in the current video frame is opposite to the orientation attribute of the known target in the previous video frame, it is determined that there is an orientation attribute jump of the known target.

[0095] In this embodiment, the attribute data includes an orientation attribute. Taking the detection target as a human-shaped target as an example, the orientation attribute of the detection target may include at least one of a facial orientation and a body orientation.

[0096] The target trajectory recognition device needs to determine whether the orientation attribute of the first detected target in the current video frame is opposite to the orientation attribute of the known target in the previous video frame. In the case where it is detected that the orientation attribute of the first detected target in the current video frame is opposite to the orientation attribute of the known target in the previous video frame, the target trajectory recognition device determines that there is an orientation attribute jump of the known target. In the case where it is detected that the orientation attribute of the first detected target in the current video frame is not opposite to the orientation attribute of the known target in the previous video frame, the target trajectory recognition device determines that there is no orientation attribute jump of the known target.

[0097] Among them, opposite orientation attributes refer to that the facial orientations (or body orientations) of the detected target and the known target in the two previous and next video frames are in two opposite directions; non-opposite orientation attributes refer to that the facial orientations (or body orientations) of the first detected target and the known target in the two previous and next video frames are in two identical or similar directions.

[0098] For example, if the facial orientation of the first detected target in the current video frame is to the left, and the facial orientation of the known target in the previous video frame is to the right, then the orientation attribute of the first detected target in the current video frame is determined to be opposite to the orientation attribute of the known target in the previous video frame, and it is determined that there is an orientation attribute jump of the known target. If the facial orientation of the first detected target in the current video frame is to the left, and the facial orientation of the known target in the previous video frame is to the left or close to the left, then the orientation attribute of the first detected target in the current video frame is determined to be not opposite to the orientation attribute of the known target in the previous video frame, and it is determined that there is no orientation attribute jump of the known target.

[0099] S412: When it is determined that the appearance attribute of the first detection target in the current video frame and the appearance attribute of the known target in the previous video frame meet a preset jump condition, it is determined that there is an appearance attribute jump of the known target.

[0100] In this embodiment, the attribute data of the detection target also includes appearance attributes. Among them, the appearance attributes may include attributes such as shape information, target type, and color. Taking the detection target as a human-shaped target as an example, the appearance attributes of the detection target include attributes such as human shape integrity, shapes of different parts (hair style, face shape, top type, bottom type), colors of different parts, such as hair color, top color and bottom color, and gender. Top types may include long sleeves, short sleeves, coats, etc., and bottom types may include skirts, pants (including trousers and shorts), etc.

[0101] The target trajectory recognition device can determine whether the change of the target appearance attribute meets the preset jump condition based on the appearance attribute of the first detection target in the current video frame and the appearance attribute of the known target in the previous video frame; when the attribute change between the appearance attribute of the first detection target in the current video frame and the appearance attribute of the known target in the previous video frame meets the preset jump condition, the target trajectory recognition device determines that there is an appearance attribute jump of the known target.

[0102] For example, the preset jump condition may include at least one of inconsistent shape information, inconsistent target type, and inconsistent color. Taking the detection target as a human-shaped target as an example, inconsistent shape information includes inconsistent human shape integrity, inconsistent shapes of different parts, such as inconsistent hairstyle, face shape, top shape, and bottom shape; inconsistent target type includes inconsistent gender; inconsistent color includes inconsistent colors of different parts, such as inconsistent hair color, top color, and bottom color.

[0103] Among them, color inconsistency refers to the large difference in color of the same part, which does not belong to the similar color system. For example, if the hair color is black and yellow, it is determined that the color difference of the hair is too large and does not belong to the similar color system, and the color of the known target is determined to be inconsistent, and the appearance attribute jump of the known target is determined; if the hair color is black and brown, it is determined that the color difference of the hair is small and belongs to the similar color system, and the color of the known target is determined to be inconsistent, and the appearance attribute jump of the known target is determined, which reduces the impact of color recognition errors on judgment errors and improves the accuracy of appearance attribute jump detection.

[0104] In this embodiment, when it is detected that the orientation attribute of the first detection target in the current video frame is opposite to the orientation attribute of the known target in the previous video frame, it is determined that the known target has an orientation attribute jump; when it is determined that the appearance attribute of the first detection target in the current video frame meets the preset jump condition with the appearance attribute of the known target in the previous video frame, it is determined that the known target has an appearance attribute jump. When multiple targets appear in the scene, by comprehensively analyzing the changes in the orientation and appearance attributes of the targets, different detection targets can be effectively distinguished, the possibility of target confusion and false association is reduced, and the accuracy and robustness of target recognition and trajectory tracking are enhanced, which is particularly suitable for multi-target tracking tasks in dynamic scenes and complex environments.

[0105] In one embodiment, step S42, i.e., performing target matching recognition on the known target according to the attribute jump of the known target, and updating the historical trajectory data of the known target when the known target is recognized in the current video frame, specifically includes the following steps:

[0106] S4201: When a change in the orientation attribute of a known target is detected, it is determined that the known target is not recognized in the current video frame, the trajectory state of the known target is marked as a lost state, and the historical trajectory data of the known target is not updated.

[0107] When a known target is detected to have a change in orientation attribute, it means that the target orientation of the two previous and next video frames has changed too much, and there may be a target association recognition error. At this time, the target trajectory recognition device can determine that the known target is not detected and recognized, and the known target trajectory is lost. The trajectory state of the known target is marked as a lost state, and the historical trajectory data of the known target is not updated. This can reduce the situation of trajectory association errors caused by target recognition errors, improve the accuracy of the target trajectory data, and improve the accuracy of subsequent trajectory predictions.

[0108] S4201: When it is detected that there is no orientation attribute jump and appearance attribute jump of the known target, determine that the known target is recognized in the current video frame, correct the predicted position of the known target in the current video frame to the detection position of the first detection target, and update the historical trajectory data of the known target.

[0109] When it is detected that there is no change in the orientation attribute and appearance attribute of the known target, that is, the attribute data of the first detected target in the current video frame is consistent or similar to the attribute data of the known target in the previous video frame, indicating that the orientation and appearance of the target in the previous and subsequent video frames have little or no change, and the first detected target matches the known target position and attributes. At this time, the target trajectory recognition device can determine that the known target is detected and recognized, that is, the first detected target is a known target, and correct the predicted position of the known target in the current video frame to the detected position of the first detected target, and update it to the historical trajectory data of the known target. That is, the detected position of the first detected target in the current video frame is used as the position of the known target in the current video frame, updated to the historical trajectory data of the known target, and the trajectory state of the known target is kept in the tracking state, which can improve the accuracy of the target trajectory data and the accuracy of subsequent trajectory predictions.

[0110] In this embodiment, when it is detected that a known target has a direction attribute jump, it is determined that the known target is not recognized in the current video frame, the trajectory state of the known target is marked as a lost state, and the historical trajectory data of the known target is not updated; when it is detected that the known target does not have a direction attribute jump and an appearance attribute jump, it is determined that the known target is recognized in the current video frame, the predicted position of the known target in the current video frame is corrected to the detection position of the first detected target, and the historical trajectory data of the known target is updated, which can improve the accuracy of the target trajectory data and the accuracy of subsequent trajectory predictions.

[0111] In one embodiment, if Figure 8 As shown, in step S42, target matching and recognition are performed on the known target according to the attribute jump of the known target, and when the known target is recognized in the current video frame, the historical trajectory data of the known target is updated, which specifically includes the following steps:

[0112] S421: When it is detected that there is no orientation attribute jump of the known target but there is an appearance attribute jump, attribute data matching is performed on other detected targets except the first detected target in the current video frame according to the attribute data of the known target in the previous video frame.

[0113] It should be understood that since the recognition of the target's appearance is easily affected by the lighting environment, display effect and recognition accuracy, recognition errors or association errors may occur. To ensure the accuracy of target recognition detection, when it is detected that there is no orientation attribute jump in the known target but there is an appearance attribute jump, the target trajectory recognition device can match the attribute data of other detected targets in the current video frame except the first detected target based on the attribute data of the known target in the previous video frame.

[0114] Among them, according to the attribute data of the known target in the previous video frame, the attribute data of other detection targets in the current video frame except the first detection target are matched, and the similarity between the attribute data of the known target in the previous video frame and the attribute data of other detection targets in the current video frame can be calculated. If the similarity between the two is greater than or equal to the preset similarity (such as 80%), it is determined that the attribute data of the known target in the previous video frame matches the attribute data of the other detection targets in the current video frame, and the attribute data of the other detection targets in the current video frame are matched. If the similarity between the two is less than the preset similarity (such as 80%), it is determined that the attribute data of the known target in the previous video frame does not match the attribute data of the other detection targets in the current video frame. In the case that the attribute data of the known target in the previous video frame does not match the attribute data of all other detection targets in the current video frame, it is determined that the attribute data of other detection targets in the current video frame are not matched.

[0115] S422: When the attribute data of other detection targets are matched, target matching and recognition are performed on the known target according to the detection frame of the second detection target. When the known target is recognized in the current video frame, the historical trajectory data of the known target is updated.

[0116] In the case of matching the attribute data of other detection targets, the target trajectory recognition device performs target matching recognition on the known target according to the detection frame of the second detection target, and updates the historical trajectory data of the known target when the known target is recognized in the current video frame. The second detection target is the other detection target in the current video frame that matches the attribute data of the known target in the previous video frame.

[0117] That is, when the attribute data of other detection targets are matched, the target trajectory recognition device determines the other detection targets in the current video frame that are matched, and records the other detection targets that are matched as second detection targets, and performs target matching and recognition on the known targets based on the detection frame of the second detection target in the current video frame, and when the known targets are recognized in the current video frame, the historical trajectory data of the known targets are updated.

[0118] For example, the predicted position of the known target in the current video frame can be directly corrected to the detected position of the second detected target in the current video frame (that is, the center position of the detection box), that is, the detected position of the second detected target in the current video frame is directly used as the position of the known target in the current video frame to update the historical trajectory data of the known target, and the trajectory state of the known target is kept in the tracking state.

[0119] In this embodiment, when it is detected that there is no orientation attribute jump of a known target, but there is an appearance attribute jump, attribute data matching is performed on other detection targets other than the first detection target in the current video frame based on the attribute data of the known target in the previous video frame; when the attribute data of other detection targets are matched, target matching and identification is performed on the known target based on the detection frame of the second detection target, and when the known target is identified in the current video frame, the historical trajectory data of the known target is updated, wherein the second detection target is the other detection target in the current video frame that matches the attribute data of the known target in the previous video frame. When it is detected that the appearance of the target has jumped but the orientation has not changed, it can be identified through attribute data matching whether it is a state change of the same target, and the attribute matching mechanism can be used to relocate the target when the appearance of the target changes, thereby improving the accuracy and robustness of target tracking and identification.

[0120] In one embodiment, step S422, i.e., performing target matching recognition on the known target according to the detection frame of the second detection target, and updating the historical trajectory data of the known target when the known target is recognized in the current video frame, specifically includes the following steps:

[0121] S4221: Determine an intersection-and-union ratio between a detection frame of a second detection target in a current video frame and a prediction frame of a known target to obtain a second intersection-and-union ratio.

[0122] In the case of matching the attribute data of other detection targets, the target trajectory recognition device records the other detection targets that are matched as the second detection targets, and determines the intersection-and-union ratio of the detection frame of the second detection target in the current video frame and the prediction frame of the known target in the current video frame to obtain the second intersection-and-union ratio. That is, the second intersection-and-union ratio is the ratio of the intersection of the detection frame of the second detection target in the current video frame and the prediction frame of the known target in the current video frame to the union of the detection frame of the second detection target in the current video frame and the prediction frame of the known target in the current video frame.

[0123] S4222: When the second intersection-over-union ratio is less than the second threshold, it is determined that the known target is not recognized in the current video frame, the trajectory state of the known target is marked as a lost state, and the historical trajectory data of the known target is not updated.

[0124] When the second intersection-over-union ratio is less than the second threshold, the target trajectory recognition device determines that the second detected target does not match the known target, and the known target is not recognized in the current video frame, marks the trajectory state of the known target as a lost state, and does not update the historical trajectory data of the known target.

[0125] S4223: When the second intersection-over-union ratio is greater than or equal to the second threshold, determine that a known target is recognized in the current video frame, correct the predicted position of the known target in the current video frame to the detection position of the second detection target, and update the historical trajectory data of the known target.

[0126] When the second intersection-and-union ratio is greater than or equal to the second threshold, the target trajectory recognition device determines that the second detected target matches the known target, recognizes the known target in the current video frame, corrects the predicted position of the known target in the current video frame to the detected position of the second detected target, and updates the historical trajectory data of the known target.

[0127] Among them, the second threshold is smaller than the first threshold. Since it has been determined that the first intersection-and-union ratio is greater than the first threshold, that is, there are other detected targets that are relatively close to the known target, and the close positions of different targets may result in the target being blocked or blocked, resulting in a smaller intersection-and-union ratio between the detection frame and the prediction frame. At this time, the threshold requirement for updating the second intersection-and-union ratio of the detection frame and the prediction frame can be lowered to ensure that the known target can be identified. In other embodiments, the second threshold may also be the same as the first threshold.

[0128] In this embodiment, the intersection-and-union ratio of the detection frame of the second detection target in the current video frame and the prediction frame of the known target is determined to obtain a second intersection-and-union ratio; when the second intersection-and-union ratio is less than the second threshold value, it is determined that the known target is not recognized in the current video frame, the trajectory state of the known target is marked as a lost state, and the historical trajectory data of the known target is not updated. The second threshold value is less than the first threshold value, which can avoid erroneous updates of trajectory data due to detection errors or mismatches, and reduce the possibility of misdetected targets or interfering objects being mistakenly added to the trajectory data; when the second intersection-and-union ratio is greater than or equal to the second threshold value, it is determined that the known target is recognized in the current video frame, the predicted position of the known target in the current video frame is corrected to the detection position of the second detection target, and updated to the historical trajectory data of the known target, to ensure the accuracy and reliability of the trajectory data, and improve the accuracy of subsequent target trajectory prediction.

[0129] In one embodiment, if Fig. 9 As shown, after step S40, that is, after target matching and recognition is performed on the known target according to the first intersection-over-union ratio and the attribute data of the first detection target in the current video frame, the method further specifically includes the following steps:

[0130] S50: When a known target is identified in the current video frame, if the trajectory state of the known target in the previous video frame is a lost state, and the duration between the video frame in which the known target was last identified and the current video frame is less than or equal to a preset duration, the trajectory state of the known target is marked as a tracking state, and the historical trajectory data of the known target is updated according to the position of the known target in the current video frame.

[0131] In this embodiment, after target matching and identification of the known target is performed based on the first intersection-and-union ratio and the attribute data of the first detection target in the current video frame, when the known target is identified in the current video frame, if the trajectory state of the known target in the previous video frame is a lost state, and the duration between the video frame in which the known target was last identified and the current video frame is less than or equal to the preset duration, the trajectory state of the known target is marked as a tracking state, and the historical trajectory data of the known target is updated according to the position of the known target in the current video frame. In some video frames, in order to identify the known target, that is, after the known target trajectory is temporarily lost, the possible position of the known target is predicted through the historical trajectory data, and attempts to identify the target trajectory in subsequent video frames are continued, thereby enhancing the target's ability to recover after a short loss, such as being able to re-identify and continue to track the target after occlusion or detection interference, thereby improving the performance of target trajectory recognition and tracking.

[0132] Among them, the preset time length can be 3 seconds to ensure rapid identification and tracking of the target trajectory.

[0133] Among them, when the known target is not recognized in the current video frame, the trajectory state of the known target is marked as a lost state. The target trajectory recognition device can continue to recognize the trajectory of the known target in subsequent video frames based on the historical trajectory data of the known target within a certain period of time (such as 1 to 1.2 seconds), so as to recognize the target trajectory in subsequent video frames as soon as possible, effectively avoiding the interruption of target tracking caused by short-term loss, and significantly improving the low missed detection rate and strong trajectory continuity.

[0134] S60: When the known target is not recognized in the current video frame, if the trajectory state of the known target in the previous video frame is a lost state, and the duration between the video frame in which the known target was last recognized and the current video frame is greater than a preset duration, the trajectory state of the known target is marked as a disconnected state, and the historical trajectory data of the known target is not updated.

[0135] In the case where no known target is identified in the current video frame, if the trajectory state of the known target in the previous video frame is a lost state, and the duration between the video frame in which the known target was last identified and the current video frame is greater than the preset duration, the trajectory state of the known target is marked as a disconnected state, and the historical trajectory data of the known target is not updated to avoid subsequent target association errors.

[0136] In this embodiment, when a known target is identified in the current video frame, if the track state of the known target in the previous video frame is a lost state, and the duration between the video frame in which the known target was identified last time and the current video frame is less than or equal to the preset duration, the track state of the known target is marked as a tracking state, and the historical track data of the known target is updated according to the position of the known target in the current video frame; when a known target is not identified in the current video frame, if the track state of the known target in the previous video frame is a lost state, and the duration between the video frame in which the known target was identified last time and the current video frame is greater than the preset duration, the track state of the known target is marked as a disconnected state, and the historical track data of the known target is not updated. By setting a reasonable duration window, the track disconnection state of the target is delayed to ensure that it is marked as a disconnected state only in the case of continuous loss, avoiding premature disconnection of the target track due to short-term detection failure or intermittent loss, ensuring that there is still an opportunity to resume tracking within a short time after the target disappears, and improving the robustness of the algorithm.

[0137] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0138] In one embodiment, a target trajectory recognition device is provided, which corresponds one-to-one to the target trajectory recognition method in the above embodiment. Fig.10 As shown, the target trajectory recognition device includes an acquisition module 801, a prediction module 102, a detection module 103, a determination module 104 and an identification module 105. The functional modules are described in detail as follows:

[0139] The acquisition module 101 is used to acquire the current video frame in the video data to be detected;

[0140] A prediction module 102 is used to perform trajectory prediction in the current video frame based on the historical trajectory data of the known target to obtain a predicted position of the known target, where one predicted position corresponds to one prediction box;

[0141] The detection module 103 is used to perform target detection on the current video frame, obtain the detection position of the detection target and the detection frame corresponding to the detection position, and perform attribute recognition based on the detection frame of the detection target to obtain attribute data of the detection target in the current video frame. The detection target can be one or more.

[0142] A determination module 104 is used to determine an intersection-and-union ratio between a first detection frame and a first prediction frame in a current video frame to obtain a first intersection-and-union ratio, wherein the first detection frame is a detection frame in the current video frame that intersects with the prediction frame, and the first prediction frame is a prediction frame in the current video frame that intersects with the detection frame;

[0143] The recognition module 105 is used to perform target matching and recognition on the known target based on the first intersection-union ratio and the attribute data of the first detection target in the current video frame, and update the historical trajectory data of the known target when the known target is recognized in the current video frame. The first detection target is the detection target corresponding to the first detection frame.

[0144] Optionally, the identification module 105 is specifically used for:

[0145] When the first intersection-over-union ratio is greater than or equal to the first threshold, performing attribute jump detection on the known target according to the attribute data of the first detection target in the current video frame and the attribute data of the known target in the previous video frame;

[0146] According to the attribute jump of the known target, the known target is matched and recognized, and the historical trajectory data of the known target is updated when the known target is recognized in the current video frame.

[0147] Optionally, the identification module 105 is further configured to:

[0148] When a change in the orientation attribute of a known target is detected, it is determined that the known target is not recognized in the current video frame, the trajectory state of the known target is marked as a lost state, and the historical trajectory data of the known target is not updated;

[0149] When it is detected that there is no orientation attribute jump and appearance attribute jump of the known target, it is determined that the known target is recognized in the current video frame, the predicted position of the known target in the current video frame is corrected to the detection position of the first detection target, and the historical trajectory data of the known target is updated.

[0150] Optionally, the identification module 105 is further configured to:

[0151] When it is detected that there is no orientation attribute jump of the known target but there is an appearance attribute jump, attribute data matching is performed on other detected targets other than the first detected target in the current video frame according to the attribute data of the known target in the previous video frame;

[0152] When the attribute data of other detection targets are matched, the known target is matched and identified according to the detection position of the second detection target. When the known target is identified in the current video frame, the historical trajectory data of the known target is updated. The second detection target is the other detection target in the current video frame that matches the attribute data of the known target in the previous video frame.

[0153] Optionally, the identification module 105 is further configured to:

[0154] Determine an intersection-and-union ratio of a detection frame of a second detection target in a current video frame and a prediction frame of a known target to obtain a second intersection-and-union ratio;

[0155] When the second intersection-over-union ratio is less than the second threshold, it is determined that the known target is not recognized in the current video frame, the trajectory state of the known target is marked as a lost state, the historical trajectory data of the known target is not updated, and the second threshold is less than the first threshold;

[0156] When the second intersection-over-union ratio is greater than or equal to the second threshold, it is determined that a known target is recognized in the current video frame, the predicted position of the known target in the current video frame is corrected to the detection position of the second detection target, and the historical trajectory data of the known target is updated.

[0157] Optionally, the attribute data includes orientation attributes and appearance attributes, and the recognition module 105 is further configured to:

[0158] When it is detected that the orientation attribute of the first detection target in the current video frame is opposite to the orientation attribute of the known target in the previous video frame, it is determined that there is an orientation attribute jump of the known target;

[0159] When it is determined that the appearance attribute of the first detection target in the current video frame and the appearance attribute of the known target in the previous video frame meet a preset jump condition, it is determined that there is an appearance attribute jump of the known target.

[0160] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0161] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0162] The present application also provides an electronic device, such as Fig.11As shown, the electronic device includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps in any of the above-mentioned method embodiments when executing the computer program, or implements the functions of the modules / units in the above-mentioned device embodiments when executing the computer program.

[0163] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the electronic device.

[0164] Those skilled in the art will understand that Fig.11 These are merely examples of the electronic device and do not constitute a limitation of the electronic device. The electronic device may include more or fewer components than those shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0165] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0166] The memory may be an internal storage unit of the electronic device, such as a hard disk or memory of the electronic device. The memory may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Furthermore, the memory may also include both an internal storage unit of the electronic device and an external storage device.

[0167] An embodiment of the present application further provides a readable storage medium, wherein the readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0168] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0169] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the camera / terminal device, recording medium, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0170] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0171] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0172] In the embodiments provided in the present application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are merely schematic, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0173] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0174] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A target trajectory recognition method, characterized in that: include: Obtaining a current video frame in the video data to be detected, and performing trajectory prediction in the current video frame according to historical trajectory data of a known target to obtain a predicted position of the known target, wherein one predicted position corresponds to one prediction box; Performing target detection on the current video frame to obtain a detection position of the detection target and a detection frame corresponding to the detection position, and performing attribute recognition based on the detection frame of the detection target to obtain attribute data of the detection target in the current video frame, where the detection target is one or more; Determine an intersection-and-union ratio between a first detection frame and a first prediction frame in a current video frame to obtain a first intersection-and-union ratio, wherein the first detection frame is the detection frame in the current video frame that intersects with the prediction frame, and the first prediction frame is the prediction frame in the current video frame that intersects with the first detection frame; According to the first intersection-to-union ratio and the attribute data of the first detection target in the current video frame, the known target is matched and recognized, and when the known target is recognized in the current video frame, the historical trajectory data of the known target is updated, and the first detection target is the detection target corresponding to the first detection frame.

2. The target trajectory recognition method according to claim 1, characterized in that: The target matching and recognition is performed on the known target according to the first intersection-over-union ratio and the attribute data of the first detection target in the current video frame, and when the known target is recognized in the current video frame, the historical trajectory data of the known target is updated, including: When the first intersection-over-union ratio is greater than or equal to a first threshold, performing attribute jump detection on the known target according to the attribute data of the first detection target in the current video frame and the attribute data of the known target in the previous video frame; According to the attribute jump of the known target, target matching and recognition are performed on the known target, and when the known target is recognized in the current video frame, the historical trajectory data of the known target is updated.

3. The target trajectory recognition method according to claim 2, characterized in that: The step of performing target matching and recognition on the known target according to the attribute jump of the known target, and updating the historical trajectory data of the known target when the known target is recognized in the current video frame, includes: In the case where it is detected that the known target has a change in the orientation attribute, determining that the known target is not recognized in the current video frame, marking the trajectory state of the known target as a lost state, and not updating the historical trajectory data of the known target; When it is detected that the known target does not have the orientation attribute jump and the appearance attribute jump, it is determined that the known target is recognized in the current video frame, the predicted position of the known target in the current video frame is corrected to the detection position of the first detection target, and the historical trajectory data of the known target is updated.

4. The target trajectory recognition method according to claim 2, characterized in that: The step of performing target matching and recognition on the known target according to the attribute jump of the known target, and updating the historical trajectory data of the known target when the known target is recognized in the current video frame, includes: When it is detected that the known target does not have a change in orientation attribute but has a change in appearance attribute, matching attribute data of other detected targets other than the first detected target in the current video frame according to the attribute data of the known target in the previous video frame; When the attribute data of other detection targets are matched, the known target is matched and identified according to the detection frame of the second detection target, and the historical trajectory data of the known target is updated when the known target is identified in the current video frame. The second detection target is other detection targets in the current video frame that match the attribute data of the known target in the previous video frame.

5. The target trajectory recognition method according to claim 4, characterized in that: The performing target matching and recognition on the known target according to the detection frame of the second detection target, and updating the historical trajectory data of the known target when the known target is recognized in the current video frame, includes: Determine an intersection-and-union ratio of a detection frame of the second detection target in the current video frame and a prediction frame of the known target to obtain a second intersection-and-union ratio; When the second intersection-over-union ratio is less than a second threshold, determining that the known target is not recognized in the current video frame, marking the trajectory state of the known target as a lost state, not updating the historical trajectory data of the known target, and the second threshold is less than the first threshold; When the second intersection-over-union ratio is greater than or equal to the second threshold, it is determined that the known target is identified in the current video frame, the predicted position of the known target in the current video frame is corrected to the detection position of the second detection target, and the historical trajectory data of the known target is updated.

6. The target trajectory recognition method according to claim 2, characterized in that: The attribute data includes orientation attributes and appearance attributes, and the attribute jump detection of the known target is performed according to the attribute data of the first detection target in the current video frame and the attribute data of the known target in the previous video frame, including: When it is detected that the orientation attribute of the first detection target in the current video frame is opposite to the orientation attribute of the known target in the previous video frame, determining that there is an orientation attribute jump of the known target; When it is determined that the appearance attribute of the first detection target in the current video frame and the appearance attribute of the known target in the previous video frame meet a preset jump condition, it is determined that there is an appearance attribute jump of the known target.

7. The target trajectory recognition method according to any one of claims 1 to 6, characterized in that: After performing target matching and recognition on the known target according to the first intersection-over-union ratio and the attribute data of the first detection target in the current video frame, the method further includes: In the case where the known target is identified in the current video frame, if the trajectory state of the known target in the previous video frame is a lost state, and the duration between the video frame in which the known target was last identified and the current video frame is less than or equal to a preset duration, the trajectory state of the known target is marked as a tracking state, and according to the position of the known target in the current video frame, the historical trajectory data of the known target is updated; In the case that the known target is not identified in the current video frame, if the trajectory state of the known target in the previous video frame is the lost state, and the duration between the video frame in which the known target was last identified and the current video frame is greater than the preset duration, the trajectory state of the known target is marked as a disconnected state, and the historical trajectory data of the known target is not updated.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the target trajectory recognition method according to any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the target trajectory recognition method according to any one of claims 1 to 7 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the steps of the target trajectory recognition method according to any one of claims 1 to 7.

Images