Target tracking method and anomaly detection method

Abstract

The invention provides a target tracking method and an anomaly detection method, and relates to the field of financial science and technology or other related fields. The method comprises the following steps: acquiring a target video, wherein the target video comprises a target object; according to the target video and a target object, obtaining a bounding box of the target object in the target video and motion information of the target object; according to the bounding box corresponding to each frame in the target video and the motion information of the target object, a prediction track of the target object is obtained, and the prediction track comprises a prediction track bounding box; according to at least two matching results of the bounding boxes of at least two categories belonging to the same frame and the predicted trajectory bounding box, obtaining an actual trajectory bounding box of the target object of the frame; and generating an actual track of the target object according to the matched actual track bounding box of each frame. According to the method, the security of the internal area of the bank is improved.

Description

Technical Field

[0001] The present application relates to the field of financial technology or other related fields, and in particular to a target tracking method and an anomaly detection method. Background Art

[0002] In recent years, with the rapid development of banking business, data centers, as the core of business, have been increasing in size, and security management requirements have become increasingly stringent. The data center can be divided into multiple functional areas such as the main computer room, network room, air conditioning room, power distribution room, office auxiliary room, and cylinder room. The types of equipment in each area are different, and due to security management needs, different team employees and external visitors are given different regional permissions. Although the internal areas of the data center have been isolated by fingerprint access control, when there are multiple groups of external personnel working or different teams of employees are maintaining at the same time, they often follow each other through different areas, posing a certain threat to the security of the data center and violating the security management regulations.

[0003] Therefore, how to improve the security of the internal area of ​​the bank is an urgent problem to be solved. Summary of the invention

[0004] The present application provides a target tracking method and an anomaly detection method for improving the security of the internal area of ​​a bank.

[0005] In a first aspect, the present application provides a target tracking method, comprising:

[0006] Acquire a target video, wherein the target video includes a target object;

[0007] According to the target video and the target object, obtaining a bounding box of the target object in the target video and motion information of the target object, wherein the bounding box includes at least two categories;

[0008] Obtaining a predicted trajectory of the target object according to the bounding box corresponding to each frame in the target video and the motion information of the target object, wherein the predicted trajectory includes a predicted trajectory bounding box;

[0009] According to at least two matching results between the bounding boxes of at least two categories belonging to the same frame and the predicted trajectory bounding box, an actual trajectory bounding box of the target object in the frame is obtained, and the matching result between the bounding box and the predicted trajectory bounding box is determined according to the geometric features and overlapping areas of the two;

[0010] The actual trajectory of the target object is generated according to the matched actual trajectory bounding box of each frame.

[0011] In a second aspect, the present application provides an anomaly detection method, comprising:

[0012] Get the active authorization area of ​​the target object;

[0013] Determine whether the target object enters an unauthorized area outside the activity authorized area according to an actual trajectory of the target object in the target video and the activity authorized area, wherein the actual trajectory is obtained by using any method as described in the first aspect;

[0014] If so, an exception warning message is output.

[0015] In a third aspect, the present application provides a target tracking device, comprising:

[0016] An acquisition module, used for acquiring a target video, wherein the target video includes a target object;

[0017] A first processing module, configured to obtain a bounding box of the target object in the target video and motion information of the target object according to the target video and the target object, wherein the bounding box includes at least two categories;

[0018] A second processing module, configured to obtain a predicted trajectory of the target object according to the bounding box corresponding to each frame in the target video and the motion information of the target object, wherein the predicted trajectory includes a predicted trajectory bounding box;

[0019] A third processing module, configured to obtain an actual trajectory bounding box of the target object in the frame according to at least two matching results between the bounding boxes of at least two categories belonging to the same frame and the predicted trajectory bounding box, wherein the matching result between the bounding box and the predicted trajectory bounding box is determined according to the geometric features and overlapping areas of the two.

[0020] The control module is used to generate the actual trajectory of the target object according to the matched actual trajectory bounding box of each frame.

[0021] In a fourth aspect, the present application provides an abnormality detection device, comprising:

[0022] An acquisition module, used to acquire the active authorization area of ​​the target object;

[0023] a processing module, configured to determine whether the target object enters an unauthorized area outside the activity authorization area according to an actual trajectory of the target object in the target video and the activity authorization area, wherein the actual trajectory is obtained by using any method as described in the first aspect;

[0024] The output module is used to output abnormal warning information if so.

[0025] In a fifth aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0026] The memory stores computer-executable instructions;

[0027] The processor executes the computer-executable instructions stored in the memory to implement the method as described in the first aspect or the second aspect.

[0028] In a sixth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method as described in the first aspect or the second aspect.

[0029] In a seventh aspect, the present application provides a computer program product, which, when executed by a processor, is used to implement the method as described in the first aspect or the second aspect.

[0030] The target tracking method and anomaly detection method provided by the present application obtain a target video, and obtain the bounding box of the target object in the target video and the motion information of the target object according to the target video and the target object. According to the bounding box corresponding to each frame in the target video, the predicted trajectory of the target object is obtained, and the actual trajectory bounding box of the target object of the frame is obtained according to at least two matching results of the bounding boxes of at least two categories belonging to the same frame and the predicted trajectory bounding box. According to the matched actual trajectory bounding box of each frame, the actual trajectory of the target object is generated, thereby improving the accuracy of target tracking. Further, by obtaining the active authorized area of ​​the target object, according to the actual trajectory of the target object in the target video obtained by the target tracking method and the active authorized area, it is determined whether the target object enters the unauthorized area outside the active authorized area. If so, an abnormal warning message is output, thereby improving the security of specific areas in the bank. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0032] Figure 1 A flowchart of an anomaly detection method provided in an embodiment of the present application;

[0033] Figure 2 A flowchart of a target tracking method provided in an embodiment of the present application;

[0034] Figure 3 A flowchart of another target tracking method provided in an embodiment of the present application;

[0035] Figure 4 A flowchart of another target tracking method provided in an embodiment of the present application;

[0036] Figure 5 A flowchart of another target tracking method provided in an embodiment of the present application;

[0037] Figure 6 A schematic diagram of a buffer zone provided in an embodiment of the present application;

[0038] Figure 7 A flowchart of another target tracking method provided in an embodiment of the present application;

[0039] Figure 8 A flowchart of another target tracking method provided in an embodiment of the present application;

[0040] Fig. 9 A schematic diagram of the structure of a target tracking device provided in an embodiment of the present application;

[0041] Fig.10 A schematic diagram of the structure of an abnormality detection device provided in an embodiment of the present application;

[0042] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0043] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0044] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0045] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0046] In addition, this application involves conducting big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.), and using artificial intelligence technology to make automated decisions, and making technical solutions that have a significant impact on personal rights and interests based on the results of automated decisions. The application provides users with corresponding operation entrances for them to choose to agree or reject the results of automated decisions; if the user chooses to reject, the expert decision-making process will be entered.

[0047] It should be noted that the target tracking method and anomaly detection method provided in the present application can be used in the field of financial technology or other related fields, and can also be used in any field other than the field of financial technology or other related fields. The application field of the target tracking method and anomaly detection method in the present application is not limited.

[0048] In recent years, with the rapid development of banking business, data centers, as the core of business, have been increasing in size, and security management requirements have become increasingly stringent. The data center can be divided into multiple functional areas such as the main computer room, network room, air conditioning room, power distribution room, office auxiliary room, and cylinder room. The types of equipment in each area are different, and due to security management needs, different team employees and external visitors are given different regional permissions. Although the internal areas of the data center have been isolated by fingerprint access control, when there are multiple groups of external personnel working or different teams of employees are maintaining at the same time, they often follow each other through different areas, posing a certain threat to the security of the data center and violating the security management regulations.

[0049] Therefore, how to improve the security of the internal area of ​​the bank is an urgent problem to be solved.

[0050] At present, the internal area of ​​the bank can be fully monitored through surveillance video, and all personnel entering the internal area of ​​the bank can be tracked to monitor whether there are abnormal personnel entering the prohibited area. However, the traditional multi-target tracking method has the problem of track loss due to occlusion, resulting in low recognition accuracy.

[0051] In view of this, the present application provides an anomaly detection method and a target tracking method, by acquiring a target video, and acquiring a bounding box of a target object in the target video and motion information of the target object according to the target video and the target object. According to the bounding boxes corresponding to each frame in the target video, a predicted trajectory of the target object is obtained, and according to at least two matching results of bounding boxes of at least two categories belonging to the same frame and the predicted trajectory bounding box, an actual trajectory bounding box of the target object of the frame is obtained, and according to the matched actual trajectory bounding box of each frame, an actual trajectory of the target object is generated, thereby improving the accuracy of target tracking.

[0052] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0053] Figure 1 The following is a flow chart of an abnormality detection method provided in an embodiment of the present application. Figure 1 As shown, the method may include:

[0054] S101: Acquire the activity authorization area of ​​the target object.

[0055] The target object is an individual whose activity range needs to be monitored. Taking the banking scenario as an example, the target object can be a bank user, bank staff, or other individuals whose activity range needs to be monitored. The activity authorization area refers to the area where the target object is allowed to move. For example, when handling business, bank users are allowed to move in areas such as the business handling hall and self-service area, while bank staff are allowed to move in corresponding work areas such as the cash area and non-cash business area according to their job functions.

[0056] Specifically, the pre-set activity authorization area information can be obtained from the bank's regional management system. The regional management system stores information such as the coordinate range or regional identification of the activity authorization area corresponding to different types of target objects (such as users of different business types, staff of different positions), and the activity authorization area of ​​the target object can be obtained by querying the system.

[0057] S102: Determine whether the target object enters an unauthorized area outside the active authorized area according to the actual track of the target object in the target video and the active authorized area.

[0058] Among them, the actual trajectory is the actual motion trajectory of the target object in the target video. The actual trajectory of the target object in the target video can be obtained by the target tracking method provided in the subsequent embodiments. Please refer to the content of the subsequent embodiments. The unauthorized area is the area outside the activity authorization area of ​​the target object. In the banking scenario, for the target object being an ordinary user, for example, it can be the bank's computer room, back-end office area and other areas where the target object is prohibited from entering.

[0059] If it is detected that the actual trajectory of the target object in the target video includes a trajectory located in the unauthorized area, it is determined that the target object has entered the unauthorized area outside the active authorized area; if it is not detected that the actual trajectory of the target object in the target video includes a trajectory located in the unauthorized area, it is determined that the target object has not entered the unauthorized area outside the active authorized area.

[0060] S103: If yes, output abnormal warning information.

[0061] Abnormal warning information is a message used to notify relevant personnel (such as security personnel, bank managers) that the target object has entered an unauthorized area. The message content may include the characteristics of the target object (such as appearance characteristics, staff member work number, and even the identity of the target object identified through technologies such as face recognition, etc.), the location and time of entry into the unauthorized area, and other information.

[0062] When it is determined that the target object has entered an unauthorized area outside the authorized area, an abnormal warning message will be sent to the bank's security monitoring system. After receiving the message, the security monitoring system will notify the security personnel or management personnel by means of pop-up windows, sound alarms, etc. Alternatively, an abnormal warning message can be directly sent to the terminal equipment of the bank's security personnel or management personnel.

[0063] The method provided in the embodiment of the present application obtains the active authorized area of ​​the target object, and determines whether the target object enters the unauthorized area outside the active authorized area based on the actual trajectory of the target object in the target video obtained by the target tracking method and the active authorized area. If so, an abnormal warning message is output, thereby improving the security of specific areas in the bank.

[0064] Next, Figure 1 The target tracking method described in the embodiment is introduced. Figure 2 A flow chart of a target tracking method provided in an embodiment of the present application. Figure 2 As shown, the method may include:

[0065] S201, obtaining a target video.

[0066] The target video includes the target object. The target video is a video stream captured by surveillance cameras installed in various areas of the bank (such as the lobby, corridor, vault entrance, etc.), which contains the activity images of the target object (such as bank customers, staff, etc.).

[0067] Optionally, the target video can be obtained from the bank's video surveillance storage system. The bank's surveillance cameras store the captured video data in a dedicated storage system, which is connected to the storage system via a network, and retrieves and obtains the target video from the storage system based on pre-set camera identification or surveillance area identification information.

[0068] Optionally, target videos sent from other monitoring management modules may also be received. For example, a bank may have a dedicated video screening module that first performs a preliminary screening of a large number of monitoring videos to determine videos that may contain target objects.

[0069] S202. According to the target video and the target object, obtain a bounding box of the target object in the target video and motion information of the target object.

[0070] The bounding box includes at least two categories. The categories of the bounding box can be determined according to actual needs, for example, there can be two categories, three categories, etc., and this application does not limit this. Exemplarily, the categories of the bounding box can be divided according to the confidence of the bounding box. The motion information of the target object is information that characterizes the motion trend of the target object.

[0071] One possible implementation method is to detect each frame in the video sequence of the target video through a target detection algorithm (such as a target detection algorithm based on deep learning, for example, using YOLO as a detector) to determine the position of the target object and the motion information of the target object, thereby obtaining a bounding box of the target object and the motion information of the target object.

[0072] Another possible implementation method is to first pre-process the target video (such as grayscale, noise reduction, etc.), then use traditional image processing algorithms (such as feature matching-based algorithms) to obtain the bounding box of the target object, and then predict the motion information of the target object according to the above-mentioned target detection algorithm.

[0073] S203: Obtain a predicted trajectory of the target object according to the bounding boxes corresponding to each frame in the target video.

[0074] The predicted trajectory includes a predicted trajectory bounding box. The predicted trajectory is a prediction of the target object's motion path in the video by processing the bounding box information of the target object in each frame of the target video. The predicted trajectory bounding box is composed of a series of predicted trajectory bounding boxes, which represent the possible position range of the target object in future frames.

[0075] Specifically, a prediction method based on a motion model can be used, such as the Kalman Filter (KF) algorithm, the Extended Kalman Filter (EKF) algorithm, etc. The motion model of the target object is established based on the position, size, speed and other information of the bounding boxes of the previous frames in the target video, and then the model is used to predict the predicted trajectory bounding box of the target object in the next frame, and finally all the predicted trajectory bounding boxes are combined to obtain the predicted trajectory of the target object.

[0076] For example, the predicted trajectory bounding box of the target object in the next frame can be predicted based on the bounding box corresponding to each frame. Specifically, the next possible position range, i.e., the predicted trajectory bounding box, can be predicted based on the position change pattern of the target object between adjacent frames (such as displacement direction, speed change, etc.). Then, all predicted trajectory bounding boxes are aggregated to obtain the predicted trajectory of the target object.

[0077] S204 , obtaining an actual trajectory bounding box of the target object of the frame according to at least two matching results between the bounding boxes of at least two categories belonging to the same frame and the predicted trajectory bounding box.

[0078] Among them, the actual trajectory bounding box is a bounding box that more accurately represents the actual position of the target object in the frame after comprehensively considering the matching of different categories of bounding boxes in the same frame with the predicted trajectory bounding box. The matching result can be determined, for example, by algorithms such as Intersection over Union (IoU) and Zero-centered IoU (ZIoU). Among them, the ZIoU algorithm is used to measure the geometric features (such as shape, direction, etc.) and overlapping area relationship between two bounding boxes (bounding box and predicted trajectory bounding box) to determine whether the match is successful. Taking the matching result determined by the ZIoU algorithm as an example, the matching result of the bounding box and the predicted trajectory bounding box is determined based on the geometric features and overlapping area of ​​the two.

[0079] Specifically, the bounding boxes of each category belonging to the same frame can be matched with the predicted trajectory bounding box in turn according to at least two categories of bounding boxes. That is, for example, if there are three categories of bounding boxes, the bounding box of category 1 can be matched with the predicted trajectory bounding box by the ZIoU algorithm first, and the predicted trajectory bounding box that is not matched in the first ZIoU match can be matched with the bounding box of category 2 for a second time, and then the predicted trajectory bounding box that is not matched in the second ZIoU match can be matched with the bounding box of category 3 for a third time. Based on the successfully matched bounding boxes in the three matching results, the actual trajectory bounding box of the target object in the frame is generated.

[0080] S205 : Generate an actual trajectory of the target object according to the matched actual trajectory bounding box of each frame.

[0081] The actual trajectory is the real motion trajectory of the target object in the target video composed of the actual trajectory bounding box in each frame, which can accurately reflect the actual activity path of the target object in the bank scene.

[0082] Specifically, the center position (or other points representing the position) of the actual trajectory bounding box of each frame can be connected in the order of the video frames to form the actual trajectory of the target object. Alternatively, the coordinate range information of the actual trajectory bounding box of each frame can be recorded in time sequence, and these records constitute the actual trajectory of the target object. In this way, the movement process of the target object in the video can be more comprehensively reflected, including the changes in its shape and size.

[0083] The method provided by the embodiment of the present application obtains a target video, and obtains the bounding box of the target object in the target video and the motion information of the target object according to the target video and the target object. According to the bounding box corresponding to each frame in the target video, the predicted trajectory of the target object is obtained, and according to at least two matching results of the bounding box of at least two categories belonging to the same frame and the predicted trajectory bounding box, the actual trajectory bounding box of the target object of the frame is obtained, and according to the matched actual trajectory bounding box of each frame, the actual trajectory of the target object is generated, thereby improving the accuracy of target tracking.

[0084] Next, taking the example that the bounding box includes a high-score bounding box and a low-score bounding box, how to obtain the bounding box of the target object in the target video according to the target video and the target object in the aforementioned step S202 is described in detail. Figure 3 A flowchart of another target tracking method provided in an embodiment of the present application is shown below. Figure 3 As shown, the aforementioned step S202 may specifically include:

[0085] S301. According to a target video and a target object, a bounding box to be classified of the target object in the target video is obtained.

[0086] If the confidence of the bounding box to be classified is greater than or equal to the first preset confidence threshold, step S302 is executed; if the confidence of the bounding box to be classified is less than or equal to the second preset confidence threshold, step S303 is executed.

[0087] The bounding boxes to be classified are rectangular boxes preliminarily determined in the target video to mark the location and range of the target object. These bounding boxes have not yet been classified according to the confidence level and are the basis for subsequent classification into high-score bounding boxes or low-score bounding boxes.

[0088] Specifically, a target detection algorithm can be used to detect each frame in the target video. For example, a target detection algorithm based on a convolutional neural network (CNN) is used to input each frame of the target video into a pre-trained network model. The network model extracts image features through operations such as convolutional layers and pooling layers, and then outputs the possible locations of the target object through the final detection layer. These locations are represented in the form of rectangular boxes, which are the bounding boxes to be classified.

[0089] Confidence is a numerical value output by the target detection algorithm, which indicates the degree of reliability of the bounding box to be classified to accurately mark the target object. The higher the value, the more likely it is that the bounding box will accurately frame the target object. The first preset confidence threshold is pre-set and is used to distinguish high-scoring bounding boxes from other bounding boxes. The first preset confidence threshold can be determined according to actual needs. For example, it can be set to 0.8, indicating that the bounding boxes to be classified with a confidence greater than or equal to 0.8 are considered to be high-scoring bounding boxes. Correspondingly, the second preset confidence threshold is also pre-set and is used to determine the low-scoring bounding boxes. The second preset confidence threshold can also be determined according to actual needs. For example, it can be set to 0.3, indicating that the bounding boxes to be classified with a confidence less than or equal to 0.3 are considered to be low-scoring bounding boxes.

[0090] S302: The bounding box to be classified is a high-score bounding box.

[0091] S303: The bounding box to be classified is a low-score bounding box.

[0092] The method provided in the embodiment of the present application obtains a bounding box to be classified of a target object in a target video according to a target video and a target object, and divides the bounding box to be classified into a high-score bounding box and a low-score bounding box according to the confidence of the bounding box to be classified, so as to further improve the matching accuracy through the matching process of the two types of bounding boxes with the predicted trajectory bounding box, thereby improving the accuracy of target tracking.

[0093] exist Figure 3 In the case shown, the following describes in detail how to obtain the actual trajectory bounding box of the target object of the frame according to at least two matching results of the bounding boxes of at least two categories belonging to the same frame and the predicted trajectory bounding box in the aforementioned step S204. Figure 4 A flowchart of another target tracking method provided in an embodiment of the present application is shown below. Figure 4 As shown, the aforementioned step S204 may specifically include:

[0094] S401 : Match a high-score bounding box and a predicted trajectory bounding box belonging to the same frame to determine a first high-score bounding box and a first predicted trajectory bounding box.

[0095] The first high-score bounding box is a high-score bounding box that successfully matches at least one predicted trajectory bounding box, and the second predicted trajectory bounding box is a predicted trajectory bounding box that fails to match the high-score bounding box.

[0096] Get all high-score bounding boxes and predicted trajectory bounding boxes in the same frame. For each high-score bounding box, calculate its matching degree with each predicted trajectory bounding box through the ZIoU algorithm. If the matching degree exceeds the preset matching threshold (for example, 0.5), it is considered that the high-score bounding box successfully matches the corresponding predicted trajectory bounding box, and this high-score bounding box is marked as the first high-score bounding box; and those predicted trajectory bounding boxes that have not successfully matched all high-score bounding boxes are marked as the first predicted trajectory bounding boxes. This process can be achieved by looping through all high-score bounding boxes and predicted trajectory bounding boxes and marking them according to the calculation results.

[0097] S402 : Match the low-score bounding box and the first predicted trajectory bounding box belonging to the same frame to determine a first low-score bounding box.

[0098] The first low-score bounding box is a low-score bounding box that successfully matches at least one first predicted trajectory bounding box.

[0099] Get all low-scoring bounding boxes and the determined first prediction trajectory bounding box in the same frame. For each low-scoring bounding box, use the ZIoU algorithm to calculate its matching degree with each first prediction trajectory bounding box. If the matching degree is greater than the preset matching threshold (for example, 0.3), mark the low-scoring bounding box as the first low-scoring bounding box. This operation is completed by traversing and calculating all low-scoring bounding boxes and the first prediction trajectory bounding box, and marking the first low-scoring bounding box according to the calculation result.

[0100] S403 : Obtain an actual trajectory bounding box of the target object of the frame according to the first high-score bounding box and the first low-score bounding box.

[0101] A possible implementation method is to fuse the first high-score bounding box and the first low-score bounding box based on their position and size information. For example, a weighted average of the first high-score bounding box and the first low-score bounding box is calculated. If the weight of the first high-score bounding box is set to 0.8 and the weight of the first low-score bounding box is set to 0.2, the weighted average of the center coordinates and width and height of the two bounding boxes is calculated respectively, and the actual trajectory bounding box is constructed based on these weighted average center coordinates and width and height.

[0102] Another possible implementation method is to screen and adjust the first high-score bounding box and the first low-score bounding box. First, determine the relative position relationship between the first low-score bounding box and the first high-score bounding box. If the first low-score bounding box is close to the first high-score bounding box within a certain range (for example, the center coordinate distance is less than a certain threshold), the first high-score bounding box is used as the main one, and the first high-score bounding box is fine-tuned (for example, the first high-score bounding box is appropriately enlarged or reduced according to part of the information of the first low-score bounding box) to obtain the actual track bounding box; if the relative position deviation between the first low-score bounding box and the first high-score bounding box is large, the first high-score bounding box is directly used as the actual track bounding box.

[0103] The method provided in the embodiment of the present application determines a first high-score bounding box and a first predicted trajectory bounding box by matching a high-score bounding box and a predicted trajectory bounding box belonging to the same frame, matches a low-score bounding box and a first predicted trajectory bounding box belonging to the same frame to determine a first low-score bounding box, and obtains an actual trajectory bounding box of the target object of the frame according to the first high-score bounding box and the first low-score bounding box. The method further improves the accuracy of matching through the process of twice matching two types of bounding boxes and predicted trajectory bounding boxes, thereby improving the accuracy of target tracking.

[0104] In one possible implementation, the first predicted trajectory bounding box that fails to match the low-score bounding box can also be supplemented with a match to further improve the accuracy and comprehensiveness of the match. Specifically, the second predicted trajectory bounding box can be matched again with the bounding boxes belonging to other frames within the first time period, and the second predicted trajectory bounding box is the first predicted trajectory bounding box that fails to match the low-score bounding box. Among them, the first time period can be set according to actual needs, for example, it can be set to 30 frames. At this time, the second predicted trajectory bounding box will continue to be retained for a time period of 30 frames to be matched again with the bounding boxes belonging to other frames within the first time period. If the match is successful, the actual trajectory bounding box of the target object belonging to other frames is obtained based on the bounding box that matches the second predicted trajectory bounding box. If the match fails, the second predicted trajectory bounding box is discarded.

[0105] Furthermore, the target tracking method provided in the present application can further improve the matching accuracy through the Minimum Perimeter Distance - Intersection over Union (MPDIoU) algorithm. Figure 5 A flowchart of another target tracking method provided in an embodiment of the present application is shown below. Figure 5 As shown, the method may also include:

[0106] S501: Generate a bounding box buffer of a bounding box and a bounding box buffer of a predicted trajectory bounding box.

[0107] The bounding box buffer refers to the area formed by expanding a certain range outward from the original bounding box. The size of this area can be set according to actual needs, for example, the area is expanded according to a certain ratio (such as 10%) based on the side length of the bounding box. Its purpose is to consider the possible position offset of the target object during the matching process and increase the fault tolerance of the matching. The bounding box buffer is similar to the bounding box buffer. It is an area formed by expanding a certain range outward from the predicted trajectory bounding box, which is used to improve the accuracy during the matching process.

[0108] For example, Figure 6 A schematic diagram of a buffer zone provided in an embodiment of the present application. Figure 6 As shown, the buffer zone weight calculation formula is as follows: Formula (1)

[0109] (1)

[0110] in, is the weight of the buffer, is the width of the original detection box or the predicted trajectory bounding box, is the length of the original detection box or the predicted trajectory bounding box, is the width of the bounding box buffer or the bounding box buffer, The length of the bounding box buffer or the bounding box buffer.

[0111] S502 , performing matching according to the bounding box, the bounding box buffer, the predicted trajectory bounding box, and the bounding box buffer to obtain an actual trajectory bounding box of the target object.

[0112] The MPDIoU algorithm is used to calculate the matching relationship between the bounding box (including its buffer) and the predicted trajectory bounding box (including its buffer). For each bounding box and its buffer, the MPDIoU value with each predicted trajectory bounding box and its buffer is calculated. If the MPDIoU value is greater than the preset matching threshold (for example, 0.6), the match is considered successful. Because the buffer is proportional to the original detection box and the predicted trajectory bounding box, MPDIoU does not change their position center, scale, and shape, but expands their matching space.

[0113] Exemplarily, the MPDIoU algorithm can be used to calculate the matching relationship between the bounding box (including its buffer) and the predicted trajectory bounding box (including its buffer) by the following formula (2):

[0114] (2)

[0115] in, is the MPDIoU value, is the intersection of the bounding box (including its buffer) and the predicted trajectory bounding box (including its buffer), is the union of the bounding box (including its buffer) and the predicted trajectory bounding box (including its buffer), and are the minimum perimeter distances between the bounding box and the predicted trajectory bounding box, respectively, and are calculated The vertices of the required bounding box are calculated with The vertices of the required bounding box are different, and the calculation The vertices of the predicted trajectory bounding box are calculated The vertices of the desired bounding box of the predicted trajectory are different.

[0116] The part that is ultimately used to construct the actual trajectory bounding box is determined from the successfully matched bounding boxes (including high-score bounding boxes and low-score bounding boxes). For example, the number of successfully matched bounding boxes, confidence, and other factors can be comprehensively considered. If multiple high-score bounding boxes are successfully matched, the one with the highest confidence can be selected. For low-score bounding boxes, they can be filtered based on their relative positional relationship with the high-score bounding boxes. Then, the actual trajectory bounding box of the target object is constructed based on the filtered bounding boxes. The construction method can be the weighted average mentioned above or adjustment based on the positional relationship.

[0117] The method provided in the embodiment of the present application generates a bounding box buffer of a bounding box and a bounding box buffer of a predicted trajectory bounding box, and performs matching according to the bounding box, the bounding box buffer, the predicted trajectory bounding box, and the bounding box buffer to obtain the actual trajectory bounding box of the target object, so as to calculate the matching relationship between the bounding box (including its buffer) and the predicted trajectory bounding box (including its buffer) through the MPDIoU algorithm, thereby reducing the matching error caused by the possible position offset of the target object, increasing the fault tolerance of the matching, thereby improving the accuracy of the matching, and improving the accuracy of the target tracking.

[0118] Next, how to obtain the bounding box of the target object in the target video in the aforementioned step 202 is described in detail. Figure 7 A flowchart of another target tracking method provided in an embodiment of the present application is shown below. Figure 7 As shown, the aforementioned step S202 may specifically include:

[0119] S701: Obtain a candidate bounding box of a target object in a target video.

[0120] Candidate bounding boxes refer to bounding boxes that are initially detected in the target video and may contain target objects. These bounding boxes are obtained by a certain target detection algorithm, but have not been further screened and processed, and contain multiple possible location ranges of target objects.

[0121] In this step, the method for obtaining the candidate bounding box of the target object in the target video is the same as the method for obtaining the bounding box of the target object in the target video according to the target video and the target object in the aforementioned step S202, and will not be repeated here.

[0122] S702: Determine redundant bounding boxes in the candidate bounding boxes according to the confidence scores of the candidate bounding boxes.

[0123] The confidence score is a value output by the object detection algorithm for each candidate bounding box, indicating the probability that the candidate bounding box accurately contains the target object. Redundant bounding boxes are those bounding boxes that overlap with other candidate bounding boxes and have low confidence among multiple candidate bounding boxes. They contribute little to accurately marking the target object and may even cause interference.

[0124] Specifically, the confidence score of each candidate bounding box can be obtained, and then the non-maximum suppression (NMS) algorithm can be used to determine the redundant bounding boxes. The NMS algorithm determines the degree of overlap between candidate bounding boxes based on IoU. For each candidate bounding box, calculate its IoU value with other candidate bounding boxes. If the IoU value of two candidate bounding boxes is greater than a preset threshold (for example, 0.5), and the confidence score of one of the candidate bounding boxes is low, then the candidate bounding box with a low confidence score is determined to be a redundant bounding box. This process requires pairwise comparison of all candidate bounding boxes to find all redundant bounding boxes.

[0125] S703: Remove redundant bounding boxes from the candidate bounding boxes to obtain a bounding box.

[0126] The bounding boxes remaining after filtering out redundant bounding boxes from the candidate bounding boxes are used as the bounding boxes mentioned in the above embodiment.

[0127] The method provided in the embodiment of the present application, when obtaining the bounding box of the target object in the target video, first obtains the candidate bounding box of the target object in the target video, and uses the NMS algorithm to determine the redundant bounding boxes in the candidate bounding boxes according to the confidence scores of the candidate bounding boxes, and removes the redundant bounding boxes from the candidate bounding boxes to obtain the bounding box, so as to remove the redundant bounding boxes that do not need to be matched, thereby improving the speed and efficiency of subsequent matching, and further improving the efficiency of target tracking.

[0128] Next, how to obtain the predicted trajectory of the target object according to the bounding boxes corresponding to each frame in the target video in the aforementioned step S203 is described in detail. Figure 8 A flowchart of another target tracking method provided in an embodiment of the present application is shown below. Figure 8 As shown, the aforementioned step S203 may specifically include:

[0129] S801 , predicting and obtaining a predicted trajectory bounding box of the target object in the next frame according to the bounding box corresponding to each frame and the motion information of the target object.

[0130] Motion information refers to the motion characteristics of the target object in the video, such as the target object's speed (including horizontal speed and vertical speed), acceleration, direction of movement, etc. This information can be obtained by analyzing the position changes of the target object's bounding box in the previous frame.

[0131] Specifically, the position information of the target object (such as the center coordinates of the bounding box) can be extracted from the bounding boxes corresponding to each frame that has been acquired. Then the motion information of the target object is calculated, for example, the velocity vector of the target object is obtained by calculating the difference between the center coordinates of the bounding boxes of adjacent frames. For acceleration, the rate of change of the velocity vector between multiple frames can be further calculated. Taking the EFK algorithm for prediction as an example, the EFK algorithm is based on the motion model of the target object, which can be linear or nonlinear. In this scenario, the motion information such as the position and velocity of the target object is used as the state variable. According to the state variables of the previous frame (i.e., the position, velocity and other information of the target object) and the pre-set system dynamic equation (describing the change law of the motion state of the target object over time), the state of the target object in the next frame is predicted by the EFK algorithm. Finally, the predicted trajectory bounding box of the target object in the next frame is constructed based on the predicted state variables (such as position information).

[0132] S802: Obtain a predicted trajectory of the target object according to all predicted trajectory bounding boxes.

[0133] In this step, the predicted trajectory bounding boxes are combined in the order of the video frames to form the predicted trajectory of the target object. Specifically, the center coordinates of each predicted trajectory bounding box (or other feature points that can represent its position) can be connected in time order. In this way, a trajectory representing the predicted motion path of the target object in the video is obtained, which can reflect the possible movement direction and position change trend of the target object in the future.

[0134] Fig. 9 A schematic diagram of the structure of a target tracking device provided in an embodiment of the present application. Fig. 9 As shown, the target tracking device may include: an acquisition module 11, a first processing module 12, a second processing module 13, a third processing module 14, and a control module 15.

[0135] The acquisition module 11 is used to acquire a target video, where the target video includes a target object.

[0136] The first processing module 12 is used to obtain a bounding box of the target object in the target video and motion information of the target object according to the target video and the target object, wherein the bounding box includes at least two categories.

[0137] The second processing module 13 is used to obtain a predicted trajectory of the target object according to the bounding box corresponding to each frame in the target video and the motion information of the target object, wherein the predicted trajectory includes a predicted trajectory bounding box.

[0138] The third processing module 14 is used to obtain the actual trajectory bounding box of the target object in the frame according to at least two matching results of the bounding box of at least two categories belonging to the same frame with the predicted trajectory bounding box, and the matching result of the bounding box and the predicted trajectory bounding box is determined according to the geometric features and overlapping areas of the two.

[0139] The control module 15 is used to generate the actual trajectory of the target object according to the matched actual trajectory bounding box of each frame.

[0140] Optionally, when the bounding box includes a high-scoring bounding box and a low-scoring bounding box, the first processing module 12 is specifically configured to obtain a bounding box to be classified of the target object in the target video according to the target video and the target object. If the confidence of the bounding box to be classified is greater than or equal to a first preset confidence threshold, the bounding box to be classified is a high-scoring bounding box. If the confidence of the bounding box to be classified is less than or equal to a second preset confidence threshold, the bounding box to be classified is a low-scoring bounding box.

[0141] Optionally, the third processing module 14 is specifically used to match the high-score bounding box and the predicted trajectory bounding box belonging to the same frame to determine the first high-score bounding box and the first predicted trajectory bounding box. The low-score bounding box and the first predicted trajectory bounding box belonging to the same frame are matched to determine the first low-score bounding box. According to the first high-score bounding box and the first low-score bounding box, the actual trajectory bounding box of the target object in the frame is obtained. Among them, the first high-score bounding box is a high-score bounding box that successfully matches at least one of the predicted trajectory bounding boxes, and the first predicted trajectory bounding box is a predicted trajectory bounding box that fails to match the high-score bounding box. The first low-score bounding box is a low-score bounding box that successfully matches at least one of the first predicted trajectory bounding boxes.

[0142] Optionally, the third processing module 14 is further used to rematch the second predicted trajectory bounding box with the bounding box belonging to other frames within the first time period. If the match is successful, the actual trajectory bounding box of the target object belonging to the other frame is obtained according to the bounding box matching the second predicted trajectory bounding box. The second predicted trajectory bounding box is the first predicted trajectory bounding box that failed to match the low-scoring bounding box.

[0143] Optionally, the third processing module 14 is further configured to generate a bounding box buffer of the bounding box and a bounding box buffer of the predicted trajectory bounding box. Matching is performed based on the bounding box, the bounding box buffer, the predicted trajectory bounding box, and the bounding box buffer to obtain the actual trajectory bounding box of the target object.

[0144] Optionally, the first processing module 12 is specifically configured to obtain a candidate bounding box of the target object in the target video, determine a redundant bounding box in the candidate bounding box according to a confidence score of the candidate bounding box, and remove the redundant bounding box from the candidate bounding box to obtain the bounding box.

[0145] Optionally, the second processing module 13 is specifically configured to predict and obtain a predicted trajectory bounding box of the target object in the next frame according to the bounding box corresponding to each frame and the motion information of the target object, and obtain the predicted trajectory of the target object according to all the predicted trajectory bounding boxes.

[0146] The target tracking device provided in the embodiment of the present application can execute the target tracking method in the above method embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0147] Fig.10 This is a schematic diagram of the structure of an abnormality detection device provided in an embodiment of the present application. Fig.10 As shown, the abnormality detection device may include: an acquisition module 21, a processing module 22, and an output module 23.

[0148] The acquisition module 21 is used to acquire the activity authorization area of ​​the target object.

[0149] The processing module 22 is used to determine whether the target object enters the unauthorized area outside the activity authorization area according to the actual trajectory of the target object in the target video and the activity authorization area, and the actual trajectory is obtained by using any of the aforementioned target tracking methods.

[0150] The output module 23 is used to output abnormal warning information if yes.

[0151] The anomaly detection device provided in the embodiment of the present application can execute the anomaly detection method in the above method embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0152] Fig.11 The structure diagram of an electronic device provided in an embodiment of the present application. The electronic device is used to execute the target tracking method or anomaly detection method mentioned above, for example, it can be the electronic device with data processing function mentioned above. Fig.11As shown, the electronic device 1100 may include: at least one processor 1101 , a memory 1102 , and a communication interface 1103 .

[0153] The memory 1102 is used to store programs. Specifically, the programs may include program codes, and the program codes include computer operation instructions.

[0154] The memory 1102 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0155] The processor 1101 is used to execute the computer-executable instructions stored in the memory 1102 to implement the method described in the above method embodiment. The processor 1101 may be a CPU, or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0156] The processor 1101 can communicate and interact with external devices through the communication interface 1103. The external device can be, for example, the terminal device of the security personnel mentioned above. In specific implementation, if the communication interface 1103, the memory 1102 and the processor 1101 are implemented independently, the communication interface 1103, the memory 1102 and the processor 1101 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc., but it does not mean that there is only one bus or one type of bus.

[0157] Optionally, in a specific implementation, if the communication interface 1103, the memory 1102 and the processor 1101 are integrated on a chip, the communication interface 1103, the memory 1102 and the processor 1101 can communicate through an internal interface.

[0158] The present application also provides a computer-readable storage medium, which may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes. Specifically, the computer-readable storage medium stores program instructions, and the program instructions are used for the methods in the above embodiments.

[0159] The present application also provides a program product, which includes an execution instruction, which is stored in a readable storage medium. At least one processor of a computing device can read the execution instruction from the readable storage medium, and at least one processor executes the execution instruction so that the computing device implements the method in the above embodiment.

[0160] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0161] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0162] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0163] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0164] If the integrated unit / module is implemented in the form of hardware, the hardware may be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. Unless otherwise specified, the processor may be any appropriate hardware processor, such as a CPU, a GPU, an FPGA, a DSP, an ASIC, etc. Unless otherwise specified, the storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory (RRAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), an enhanced dynamic random access memory (EDRAM), a high-bandwidth memory (HBM), a hybrid memory cube (HMC), etc.

[0165] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0166] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0167] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0168] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A target tracking method, characterized in that: include: Acquire a target video, wherein the target video includes a target object; According to the target video and the target object, obtaining a bounding box of the target object in the target video and motion information of the target object, wherein the bounding box includes at least two categories; Obtaining a predicted trajectory of the target object according to the bounding box corresponding to each frame in the target video and the motion information of the target object, wherein the predicted trajectory includes a predicted trajectory bounding box; According to at least two matching results between the bounding boxes of at least two categories belonging to the same frame and the predicted trajectory bounding box, an actual trajectory bounding box of the target object in the frame is obtained, and the matching result between the bounding box and the predicted trajectory bounding box is determined according to the geometric features and overlapping areas of the two; The actual trajectory of the target object is generated according to the matched actual trajectory bounding box of each frame.

2. The method according to claim 1, characterized in that The bounding box includes a high-score bounding box and a low-score bounding box, and acquiring the bounding box of the target object in the target video according to the target video and the target object includes: According to the target video and the target object, obtaining a bounding box to be classified of the target object in the target video; If the confidence of the bounding box to be classified is greater than or equal to a first preset confidence threshold, the bounding box to be classified is a high-scoring bounding box; If the confidence of the bounding box to be classified is less than or equal to a second preset confidence threshold, the bounding box to be classified is a low-scoring bounding box.

3. The method according to claim 2, characterized in that The obtaining, according to at least two matching results between the bounding boxes of at least two categories belonging to the same frame and the predicted trajectory bounding box, an actual trajectory bounding box of the target object of the frame comprises: Matching the high-score bounding box and the predicted trajectory bounding box belonging to the same frame to determine a first high-score bounding box and a first predicted trajectory bounding box, wherein the first high-score bounding box is a high-score bounding box that successfully matches at least one of the predicted trajectory bounding boxes, and the first predicted trajectory bounding box is a predicted trajectory bounding box that fails to match the high-score bounding box; Matching the low-score bounding box and the first predicted trajectory bounding box belonging to the same frame to determine a first low-score bounding box, wherein the first low-score bounding box is a low-score bounding box that successfully matches at least one of the first predicted trajectory bounding boxes; An actual trajectory bounding box of the target object in the frame is obtained according to the first high-score bounding box and the first low-score bounding box.

4. The method according to claim 3, characterized in that The method further comprises: In a first time period, rematching a second predicted trajectory bounding box with a bounding box belonging to another frame, wherein the second predicted trajectory bounding box is the first predicted trajectory bounding box that fails to match the low-scoring bounding box; If the match is successful, the actual trajectory bounding box of the target object belonging to the other frame is obtained according to the boundary box that matches the second predicted trajectory bounding box.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: generating a bounding box buffer of the bounding box and a bounding box buffer of the predicted trajectory bounding box; Matching is performed according to the bounding box, the bounding box buffer, the predicted trajectory bounding box, and the bounding box buffer to obtain an actual trajectory bounding box of the target object.

6. The method according to claim 5, characterized in that The obtaining a bounding box of the target object in the target video includes: Obtain a candidate bounding box of the target object in the target video; Determining a redundant bounding box in the candidate bounding box according to the confidence score of the candidate bounding box; The redundant bounding box is removed from the candidate bounding boxes to obtain the bounding box.

7. The method according to claim 5, characterized in that The step of obtaining a predicted trajectory of the target object according to the bounding box corresponding to each frame in the target video and the motion information of the target object includes: According to the bounding box corresponding to each frame and the motion information of the target object, a predicted trajectory bounding box of the target object in the next frame is predicted and obtained; The predicted trajectory of the target object is obtained according to all the predicted trajectory bounding boxes.

8. An anomaly detection method, characterized in that: The method comprises: Get the active authorization area of ​​the target object; Determining whether the target object enters an unauthorized area outside the active authorized area according to an actual trajectory of the target object in the target video and the active authorized area, wherein the actual trajectory is obtained by using any method as claimed in claim 1; If so, an exception warning message is output.

9. A target tracking device, characterized in that: include: An acquisition module, used for acquiring a target video, wherein the target video includes a target object; A first processing module, configured to obtain a bounding box of the target object in the target video and motion information of the target object according to the target video and the target object, wherein the bounding box includes at least two categories; A second processing module, configured to obtain a predicted trajectory of the target object according to the bounding box corresponding to each frame in the target video and the motion information of the target object, wherein the predicted trajectory includes a predicted trajectory bounding box; A third processing module, configured to obtain an actual trajectory bounding box of the target object in the frame according to at least two matching results between the bounding boxes of at least two categories belonging to the same frame and the predicted trajectory bounding box, wherein the matching result between the bounding box and the predicted trajectory bounding box is determined according to the geometric features and overlapping areas of the two. The control module is used to generate the actual trajectory of the target object according to the matched actual trajectory bounding box of each frame.

10. An abnormality detection device, characterized in that: include: An acquisition module, used to acquire the active authorization area of ​​the target object; a processing module, configured to determine whether the target object enters an unauthorized area outside the activity authorization area according to an actual trajectory of the target object in the target video and the activity authorization area, wherein the actual trajectory is obtained by using any method according to claims 1 to 7; The output module is used to output abnormal warning information if so.

11. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 8 when executed by a processor.

13. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 8 when being executed by a processor.

Images