Object tracking method, device, equipment and system

By combining scene information and current status information, pedestrian tracking is used to use prediction models to solve the problem of Kalman filter predicting position deviation in a fixed area, and the accuracy and effectiveness of pedestrian tracking are improved.

CN118887254APending Publication Date: 2024-11-01SANY AUTOMOBILE HOISTING MACHINERY
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410917965.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-09
Publication Date
2024-11-01

AI Technical Summary

Technical Problem

In the case of fixed areas or clear movement modes, there is a deviation in Kalman filtering to predict positions, which affects the tracking effect.

Method used

Combining the scene information of the tracking area, the current status information and tracking information are obtained, the prediction position is determined through the prediction model, and matching and update to improve the accuracy of the tracking object position prediction.

Benefits of technology

More accurately predicting the movement trajectory of the tracked object in a fixed area improves the tracking effect and reduces the deviation of position prediction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118887254A_ABST
    Figure CN118887254A_ABST
Patent Text Reader

Abstract

The invention provides an object tracking method, device, equipment and system, and the method comprises the steps: obtaining the current state information and tracking information of a tracked object, the current state information comprises a current position and a current speed, the tracking information comprises scene information, and the scene information comprises the spatial data information of a tracking region; determining a predicted position of the tracked object after a preset time interval according to the current state information and a predicted value of the influence of the tracking information on the motion state of the tracked object; and tracking the tracking object according to the predicted position. The object tracking method provided by the invention combines the scene information of the tracking area, can more accurately predict the possible moving track of the tracking object in the fixed area, can more accurately estimate the speed information of the tracking object at different positions compared with the statistical characteristics depending on random walk, and improves the tracking accuracy. Therefore, the position of the tracking object is predicted more accurately, and the tracking effect is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of pedestrian tracking, and in particular to an object tracking method, device, equipment and system. Background Art

[0002] Pedestrian tracking refers to the use of computer vision and image processing technology to identify and track pedestrians in video images or real-time video streams to understand the behavior of target pedestrians. Pedestrian tracking is an important research direction in computer vision and is widely used in security monitoring, intelligent transportation, human-computer interaction, and competitive sports. The accuracy and real-time performance of pedestrian tracking are crucial. Summary of the invention

[0003] In view of this, the embodiments of the present application are committed to providing an object tracking method, device, equipment and system, which can combine the scene information of the tracking area to more accurately predict the position of the tracked object, thereby improving the object tracking effect.

[0004] According to a first aspect of an embodiment of the present application, there is provided an object tracking method, comprising:

[0005] Acquire current state information and tracking information of the tracked object, wherein the current state information includes a current position and a current speed, and the tracking information includes scene information, and the scene information includes spatial data information of the tracking area;

[0006] Determining a predicted position of the tracked object after a preset time interval according to the current state information and a predicted value of the impact of the tracking information on the motion state of the tracked object;

[0007] The tracked object is tracked according to the predicted position.

[0008] Optionally, the tracking information also includes walking information of the tracked object, and the walking information includes target path nodes and node travel order of the tracked object in the tracking area.

[0009] Optionally, the tracking area includes at least one sub-area, and the scene information further includes attribute information of the sub-area, where the attribute information is used to characterize regional characteristics of the sub-area.

[0010] Optionally, determining the predicted position of the tracked object after a preset time interval according to the current state information and the predicted value of the impact of the tracking information on the motion state of the tracked object includes:

[0011] Determining speed information of the tracked object within a preset time interval according to the current state information and a predicted value of the impact of the tracking information on the motion state of the tracked object;

[0012] The predicted position of the tracked object after the preset time interval is determined according to the current state information and the speed information.

[0013] Optionally, determining the predicted position of the tracked object after a preset time interval according to the current state information and the predicted value of the impact of the tracking information on the motion state of the tracked object includes:

[0014] The current state information and the tracking information are input into a pre-trained position prediction model, and the position prediction model determines the predicted position of the tracked object after a preset time interval based on the predicted value of the impact of the current state information and the tracking information on the motion state of the tracked object.

[0015] Optionally, the current state information includes state information of a first detection frame corresponding to the tracked object in the current frame image;

[0016] The step of determining the predicted position of the tracked object after a preset time interval according to the current state information and the predicted value of the impact of the tracking information on the motion state of the tracked object includes:

[0017] Determine, according to the current state information and the predicted value of the impact of the tracking information on the motion state of the tracked object, a predicted position of a first predicted frame corresponding to the tracked object in a next frame image, wherein the next frame image is a frame image after the current frame image in the video stream;

[0018] Tracking the tracked object according to the predicted position includes:

[0019] Obtaining a detection frame in the next frame image;

[0020] Performing a first match between the detection frame in the next frame image and the first prediction frame to determine a matching result, wherein the first matching includes at least one of cosine distance matching, Mahalanobis distance matching, Euclidean distance matching, stay matching, gait matching, and key point matching;

[0021] According to the matching result, the tracking result of the tracked object is updated, and the tracking result includes the location information of the tracked object.

[0022] Optionally, the tracking area includes at least one sub-area, and the method further includes:

[0023] Determine whether the tracked object enters the first sub-area according to the current position of the tracked object and the spatial data information of the preset first sub-area, and generate alarm information when it is determined that the tracked object enters the first sub-area, the first sub-area being a preset sub-area that the tracked object is prohibited from entering;

[0024] and / or,

[0025] When the tracked object enters the second sub-area, a first feature of the tracked object is obtained, and when the first feature does not meet a preset first feature condition, it is determined that the tracked object cannot enter a third sub-area located in the second sub-area, and an alarm message is generated, wherein the first feature condition is determined based on an access condition of the third sub-area.

[0026] According to a second aspect of an embodiment of the present application, there is provided an object tracking device, comprising:

[0027] A first unit is used to obtain current state information and tracking information of the tracked object, wherein the current state information includes a current position and a current speed, and the tracking information includes scene information, and the scene information includes spatial data information of the tracking area;

[0028] A second unit is used to determine a predicted position of the tracked object after a preset time interval according to the current state information and a predicted value of the influence of the tracking information on the motion state of the tracked object;

[0029] The third unit is used to track the tracked object according to the predicted position.

[0030] According to a third aspect of an embodiment of the present application, there is provided an object tracking device, including a memory and a processor;

[0031] The memory is connected to the processor and is used to store programs;

[0032] The processor is used to implement the object tracking method as described in any one of the first aspects of the embodiments of the present application by running the program in the memory.

[0033] A fourth aspect of the application embodiment provides an object tracking system, including:

[0034] A monitoring device and an object tracking device as described in the third aspect of the embodiment of the present application, wherein:

[0035] The monitoring device is used to obtain video data and / or image data in the monitoring area.

[0036] The object tracking method provided by the present application first obtains the current state information and tracking information of the tracked object, the current state information includes the current position and the current speed, the tracking information includes the scene information, and the scene information includes the spatial data information of the tracking area; secondly, according to the current state information and the predicted value of the impact of the tracking information on the motion state of the tracked object, the predicted position of the tracked object after a preset time interval is determined; finally, according to the predicted position, the tracked object is tracked. The object tracking method provided by the present application combines the scene information of the tracking area, and can more accurately predict the possible movement trajectory of the tracked object in a fixed area. Compared with relying on the statistical characteristics of random walks, it can more accurately estimate the speed information of the tracked object at different positions, and then more accurately predict the position of the tracked object, thereby improving the tracking effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0038] Figure 1 A schematic diagram of the structure of an object tracking system provided in an embodiment of the present application.

[0039] Figure 2 A flowchart of an object tracking method provided in an embodiment of the present application.

[0040] Figure 3 A flowchart of another object tracking method provided in an embodiment of the present application.

[0041] Figure 4 A schematic diagram of the structure of an object tracking device provided in an embodiment of the present application.

[0042] Figure 5 A schematic diagram of the structure of an object tracking device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] The technical solution of the embodiment of the present application is applicable to various scenarios where object tracking is required and the tracking area is relatively fixed, such as construction sites, factories, exhibition halls, etc. The technical solution provided by the embodiment of the present application can be combined with the scene information of the tracking area to more accurately predict the position of the tracked object, thereby improving the object tracking effect.

[0044] The technical solution provided in the embodiments of the present application can be exemplarily applied to hardware devices such as processors, electronic devices, servers (including cloud servers), or packaged into software programs to be run. When the hardware device executes the processing of the technical solution of the embodiments of the present application, or the above-mentioned software program is run, the automatic splitting of the target task and the automatic calling of the application program interface required for the task can be achieved to complete the purpose of the target task. The embodiments of the present application only exemplarily introduce the specific processing of the technical solution of the present application, and do not limit the specific implementation form of the technical solution of the present application. Any technical implementation form that can execute the processing of the technical solution of the present application can be adopted by the embodiments of the present application.

[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0046] Before introducing this application solution, the relevant technologies are first introduced:

[0047] Pedestrian tracking refers to the use of computer vision and image processing technology to identify and track pedestrians in video images or real-time video streams to understand the behavior of target pedestrians. Pedestrian tracking is an important research direction in computer vision and is widely used in security monitoring, intelligent transportation, human-computer interaction, and competitive sports. The accuracy and real-time performance of pedestrian tracking are crucial.

[0048] In the prior art, the Deep SORT model is usually used for pedestrian tracking. The Deep SORT model uses Kalman filtering to predict the possible position of the target pedestrian in the current frame based on the state information of the target pedestrian in the previous frame. However, when predicting the position of the pedestrian, the Kalman filter models and estimates the movement of the target pedestrian based on the theoretical basis of random walk. Although this method performs well in dealing with random walk problems, for non-random walking pedestrians, especially when the tracking area is relatively fixed or the pedestrian's movement pattern is relatively clear, the Kalman filter may not be able to fully utilize the known information, resulting in deviations in the predicted position, which in turn affects the tracking effect.

[0049] In view of this, the embodiments of the present application are committed to providing an object tracking method, device, equipment and system, which can more accurately predict the position of the tracked object and improve the object tracking effect, which are described in detail one by one in the following embodiments.

[0050] Exemplary Systems

[0051] For ease of understanding, the present application embodiment first introduces the object tracking system. Figure 1 , Figure 1 A schematic diagram of the structure of an object tracking system provided in an embodiment of the present application.

[0052] like Figure 1 As shown, the object tracking system provided in this embodiment includes an object tracking device 110 and at least one monitoring device 120 , and the object tracking device 110 is communicatively connected with the monitoring device 120 .

[0053] The monitoring device 120 is used to obtain video data and / or image data in the monitoring area, and send the obtained data information to the object tracking device 110 through the communication network to provide necessary data support for object tracking.

[0054] The object tracking device 110 is used to execute the object tracking method provided in the following embodiments of the present application according to the data information sent by the monitoring device 120 to track the target pedestrian. Specifically, after the object tracking device 110 obtains the tracking information and obtains the video data and / or image data in the monitoring area sent by the monitoring device 120, it obtains the current state information of the tracked object according to the data information sent by the monitoring device 120, the tracking information includes scene information, the scene information includes spatial data information of the tracking area, and the current state information includes the current position and the current speed; then, according to the current state information and the predicted value of the impact of the tracking information on the motion state of the tracked object, the predicted position of the tracked object after a preset time interval is determined; finally, according to the predicted position, the tracked object is tracked. As for other specific implementations of the object tracking method, they will be specifically expanded in the subsequent content and will not be described in detail here.

[0055] The monitoring device 120 can be a network camera, an infrared camera, a PTZ camera, or other device that can obtain video data or image data in the monitoring area, or any combination of the above devices, and this application does not limit this. As for the number of monitoring devices to be installed and the location of the installation, it is determined according to the specific monitoring needs and scenarios. Optionally, when setting up the monitoring equipment, the location of the monitoring equipment is determined based on the area, importance, and personnel flow of the monitoring area, and comprehensively considers factors such as the monitoring range, field of view, and light conditions to ensure that the monitoring equipment can capture effective images and data. Exemplarily, sufficient monitoring points are set in important areas to ensure coverage without blind spots. For key areas or dangerous areas, the number and density of monitoring equipment are increased. For privacy areas (such as locker rooms, rest areas, etc.), avoid setting up monitoring equipment.

[0056] The object tracking device 110 can be various types of processing devices. The specific type is determined according to the scale, application scenario and performance requirements of the object tracking system. This application does not limit this. For example, the object tracking device 110 can be an independent server, a server cluster composed of multiple physical servers, or a cloud server that provides cloud computing services, etc.

[0057] The above-mentioned communication connection can be a wireless communication connection or a wired communication connection or any combination thereof, and can be a direct communication connection or an indirect communication connection or any combination thereof, which is not limited in the present application.

[0058] Exemplary Methods

[0059] Figure 2 A flowchart of an object tracking method provided in an embodiment of the present application. Figure 2 As shown, the object tracking method provided in this embodiment includes steps S201-S203:

[0060] S201. Acquire current state information and tracking information of a tracked object, wherein the current state information includes a current position and a current speed, and the tracking information includes scene information, and the scene information includes spatial data information of a tracking area.

[0061] The tracking object can be understood as an individual pedestrian whose position identification and trajectory tracking are required to be performed continuously. In a pedestrian tracking scenario, there may be multiple pedestrians at the same time, and the tracking object is a specific individual among these pedestrians. In this scenario, multiple pedestrians in the scene can be tracked simultaneously, and each pedestrian is a specific tracking object. A unique identifier can be assigned to each tracking object to distinguish and identify different pedestrian individuals. The identifier can be a number, a string, or other unique symbol in other forms.

[0062] The current state information can be understood as the specific state data of the tracked object at the current moment, which can characterize the actual situation and state change of the tracked object at the current moment. The current state information includes the current position and current speed, which are used to describe the current motion state of the tracked object, and may also include other supplementary information such as the acceleration and posture of the tracked object at the current moment, which is used to more accurately describe the current state of the tracked object.

[0063] The current position may be understood as the specific position of the tracked object in the tracking scene, which is usually expressed in pixel coordinates or relative coordinates relative to a reference point.

[0064] The current speed can be understood as the movement speed and direction of the tracked object at the current moment. The specific implementation method of obtaining the current state information of the tracked object can be implemented with reference to the relevant prior art, and the present application does not limit this. Exemplarily, the first detection frame corresponding to the tracked object is extracted from the current video frame using a target detection algorithm (such as YOLO, SSD, etc.) and image processing technology, and then the position information of the first detection frame in the current video frame is determined by identifying the pixel coordinates of each vertex of the first detection frame, and finally, through coordinate conversion, the position information of the first detection frame in the current video is converted into the current position information of the tracked object corresponding to the first detection frame in the actual tracking scene. Exemplarily, the movement speed of the tracked object is determined by analyzing the changes in the position information of the tracked object in the current frame and a preset number of consecutive frames before the current frame.

[0065] When tracking an object, in addition to the direct observation data of the tracked object (such as the current state information, appearance features, etc. of the tracked object), there is other data information that can improve the tracking performance and position prediction accuracy, such as scene information, pedestrian target walking information (such as final destination information, target travel node, travel order, etc.), walking rule information, pedestrian walking preference information (such as whether to comply with walking rules, preferred path selection, etc.), etc. These data information are tracking information.

[0066] The tracking information includes scene information of the tracking area. The scene information can be understood as data and information describing the specific environment and spatial layout of the tracking area, describing the specific environment and features of the tracking area, such as terrain, building layout, road network, etc.

[0067] The scene information includes spatial data information of the tracking area, i.e., data information related to the spatial relationship, size, position and other attributes in the tracking area, which provide three-dimensional or two-dimensional geometric information of the tracking area. For example, when the tracking area is a shopping mall, the spatial data information of the tracking area can provide information such as the location of each store in the shopping mall, the layout of corridors and stairs, and the location of the shopping mall entrances and exits; when the tracking area is a construction site, the spatial data information of the tracking area can provide information such as the layout of buildings and facilities in the construction site, the placement of construction equipment and materials, terrain and landform features, and the placement of safety facilities and signs.

[0068] It should be noted that the specific method of obtaining the scene information of the tracking area can be implemented with reference to the relevant prior art, and the present application does not limit this. Optionally, the scene information is pre-collected and set on the controller or server side, that is, before executing the object tracking method proposed in the present application, the scene information of the tracking area is pre-acquired and stored in the internal storage space or cloud of the object tracking device of the object tracking system, and the object tracking device can obtain the scene information of the tracking area in the internal storage space or cloud. Exemplarily, a drone equipped with a camera or a laser radar can be used in advance to perform a high-altitude scan of the tracking area to quickly and efficiently collect spatial data information of the tracking area. Exemplarily, a mobile terminal device can be used in advance to collect digital data coordinates at the ground level. Exemplarily, a three-dimensional tracking area model can be constructed based on the spatial data information collected by the drone and / or the digital data coordinates collected by the mobile terminal device, and the model can show the spatial layout and characteristics of the tracking area. Optionally, when executing the object tracking method proposed in the present application, the spatial data information of the tracking area can be collected in real time by sensors deployed in the tracking area, such as cameras, radars, infrared sensors, etc., and the collected spatial data information of the tracking area is sent to the object tracking device.

[0069] S202: Determine a predicted position of the tracked object after a preset time interval according to the current state information and a predicted value of the impact of the tracking information on the motion state of the tracked object.

[0070] During the object tracking process, different tracking information will have a direct or indirect impact on the motion state of the tracked object. For example, different terrains (such as flat ground, uphill, downhill, etc.) and road conditions (such as smooth, rugged, slippery, etc.) in the tracking area will affect the travel speed of the tracked object. Obstacles in the tracking area will cause the tracked object to change its travel path or slow down to avoid collision. Visibility (such as light conditions, line of sight obstruction, etc.), navigation signs (such as signs, maps, etc.), turns, etc. in the tracking area will affect the travel speed of the tracked object and affect the tracked object's choice of travel path. Traffic facilities such as traffic lights and zebra crossings in the tracking area will affect the travel path and speed of the tracked object.

[0071] The predicted value of the impact of the tracking information on the motion state of the tracked object can be understood as a predicted value of the change in the motion state of the tracked object within a future preset time interval based on the impact of the tracking information. The predicted value comprehensively considers various factors contained in the tracking information that may affect the motion state of the tracked object.

[0072] When predicting the position information of the tracked object after a preset time interval, the tracking information provides contextual information about the tracking situation, helping the object tracking system to better predict the pedestrian's possible route, stop point, and obstacle avoidance behaviors, that is, helping the object tracking system determine the predicted value of the tracking information on the motion state of the tracked object. By comprehensively considering the tracking information, the tracking system can more accurately predict the impact value of the tracking information on the motion state of the tracked object, and then obtain a more accurate predicted position after the preset time interval. Exemplarily, when there is an obstacle in a certain area of ​​the tracking area, the object tracking system can predict that the tracked object is unlikely to pass through the area directly, but will choose to detour, which will affect the motion state of the tracked object; when a certain area in the tracking area is an open space (such as a large square, etc.), the object tracking system can predict that the tracked object may speed up in the area, which will affect the motion state of the tracked object; when a certain area in the tracking area is a narrow corridor or staircase, the object tracking system can predict that the tracked object may slow down or queue up in the area, which will affect the motion state of the tracked object. In the above examples, the specific value of the predicted value of the impact on the motion state of the tracked object is determined according to the actual tracking information.

[0073] Optionally, the predicted position of the tracked object after a preset time interval is determined based on the predicted value of the influence of the current state information and the tracking information on the motion state of the tracked object, including: determining the predicted position of the tracked object after a preset time interval based on the predicted value of the influence of the current state information and the spatial data information of the tracking area on the motion state of the tracked object.

[0074] As an optional implementation method, before executing step S202, it is necessary to first execute the first strategy to determine the predicted value of the impact of the tracking information on the motion state of the tracked object based on the acquired tracking information, and then execute the second strategy to determine the predicted position of the tracked object after a preset time interval based on the predicted value of the impact of the tracking information on the motion state of the tracked object and the current state information of the tracked object.

[0075] The first strategy and the second strategy can be implemented in multiple ways. Optionally, both the first strategy and the second strategy can be implemented based on a model-based processing concept: the first strategy includes inputting the acquired tracking information into a pre-trained first model, and the first model determines a predicted value of the impact of the tracking information on the motion state of the tracked object based on the input tracking information; the second strategy includes inputting the predicted value of the impact of the tracking information on the motion state of the tracked object and the current state information of the tracked object into a pre-trained second model, and the second model determines the predicted position of the tracked object after a preset time interval based on the predicted value of the impact of the input tracking information on the motion state of the tracked object and the current state information of the tracked object.

[0076] Wherein, the first model and the second model are both pre-trained models, and the first model can be obtained by training the first training sample for motion state influence prediction, the first training sample includes multiple sets of data, each set of data includes sample tracking information and annotated predicted values ​​of the influence of the tracking information on the motion state of the tracked object, and the trained first model can determine the predicted value of the influence of the tracking information on the motion state of the tracked object according to the input tracking information; the second model can be obtained by training the second training sample for first position prediction, the second training sample includes multiple sets of data, each set of data includes sample current state information, sample annotated predicted values ​​of the influence of the tracking information on the motion state of the tracked object, and annotated predicted positions, and the trained second model can determine the predicted position of the tracked object after a preset time interval according to the predicted value of the influence of the input tracking information on the motion state of the tracked object and the current state information of the tracked object. The first training sample and the second training sample can use existing public data sets, or they can create and annotate data sets by themselves.

[0077] As another optional implementation, after obtaining the tracking information and the current state information of the tracked object, the tracking information and the current state information are input into a pre-trained position prediction model, and the position prediction model determines the predicted position of the tracked object after a preset time interval based on the predicted value of the impact of the tracking information on the motion state of the tracked object and the current state information of the tracked object. This will be explained in subsequent embodiments and will not be elaborated in detail here.

[0078] According to the above content, tracking information containing scene information plays many roles in position prediction in object tracking. It not only provides spatial constraints and guidance, but also helps with path planning and behavior pattern analysis. When predicting the position of the tracked object, making full use of tracking information can significantly improve the accuracy and reliability of the position prediction of the tracked object.

[0079] S203: Track the tracking object according to the predicted position.

[0080] Specifically, after determining the predicted position of the tracked object after the preset time interval, the predicted position is analyzed to determine the actual position of the tracked object after the preset time interval, thereby achieving tracking of the tracked object.

[0081] Optionally, analyzing the predicted position may be matching the predicted position after a preset time interval with actual detection information after the preset time interval, thereby determining the actual position of the tracked object after the preset time interval.

[0082] Next, this step will be described in detail by taking the scene of object tracking by video surveillance as an example.

[0083] In this scenario, the current state information of the tracked object includes state information of a first detection frame corresponding to the tracked object in the current frame image.

[0084] Determine the predicted position of the tracked object after a preset time interval based on the predicted value of the influence of the current state information and the tracking information on the motion state of the tracked object, including: determine the predicted position of a first predicted frame corresponding to the tracked object in a next frame image based on the predicted value of the influence of the current state information and the tracking information on the motion state of the tracked object, the next frame image being a frame image after the current frame image in the video stream.

[0085] The step of tracking the tracked object according to the predicted position comprises steps A1-A3:

[0086] A1. Obtain a detection frame in the next frame image.

[0087] After determining the predicted position of the tracked object in the next frame image (i.e., the position of the first predicted frame), a detection frame is obtained from the next frame image according to the predicted position. Optionally, obtaining the detection frame from the next frame image according to the predicted position includes: obtaining the detection frame within a preset distance range around the predicted position, the obtained detection frame is a detection frame that may contain the tracked object, that is, the detection frame in the next frame image is a detection frame within a preset distance range around the predicted position of the first predicted frame in the next frame image.

[0088] Optionally, a detection box that may contain the tracked object is identified in the next frame of the image through a target detection algorithm (such as YOLO, SSD, etc.) and image processing technology. Optionally, the detection box in the next frame of the image obtained contains the location information of the identified target (such as the coordinates of the upper left corner and the width and height of the box) as well as possible category labels and confidence scores.

[0089] A2. Perform a first match between the detection frame in the next frame image and the first prediction frame to determine a matching result, wherein the first matching includes at least one of cosine distance matching, Mahalanobis distance matching, Euclidean distance matching, stay matching, gait matching, and key point matching.

[0090] A first match is performed between the detection frame identified in the next frame image in step A1 and the first prediction frame corresponding to the tracked object in the next frame image to obtain a matching result of the first match.

[0091] The first matching can adopt a variety of methods, which can be any one of cosine distance matching, Mahalanobis distance matching, Euclidean distance matching, dwell matching, gait matching and key point matching, or a combination of any two or more of the above matching methods to improve the accuracy and robustness of the first matching. It can also include other related matching methods of the prior art. In practical applications, a suitable matching method can be selected according to actual conditions, and this application does not limit this.

[0092] Cosine distance matching: Matching is performed by calculating the cosine similarity between the detection box and the prediction box.

[0093] Mahalanobis distance matching: Taking into account the relationship between various features, matching is performed by calculating the Mahalanobis distance between the detection box and the prediction box.

[0094] Euclidean distance matching: Directly calculate the spatial distance between the detection box and the prediction box for matching.

[0095] Stay Matching: When the tracked object disappears in the occluded area (such as the lounge, locker room, etc.), the in-situ matching mechanism will be performed to wait for the tracked object to reappear. Optionally, when the regional attributes of the occluded area entered by the tracked object include attribute information that can characterize the occlusion situation, such as the occluded range of the occluded area, the entrance and exit positions of the occluded area, and the occluded time, the time or position of the tracked object's reappearance can be inferred based on the regional attributes of the occluded area, and stay matching will be performed near the time or position of the tracked object's reappearance, waiting for the tracked object to reappear.

[0096] Gait matching: Matching is performed by analyzing the walking gait characteristics of the tracked object and other targets. For example, in scenes such as factories and construction sites, the tracked object and other targets are generally dressed in a highly similar manner. Gait matching can be used to assist in confirming the target's identity by analyzing the walking pattern, thereby increasing the accuracy and stability of tracking.

[0097] Key point matching: Matching is performed by extracting the key points of the tracked object (such as human joints, corner points of specific objects, etc.). It is suitable for tracking objects with obvious feature points. For example, in the scene of a construction site, the limbs and torso of the workers, as well as the safety helmets and labor protection supplies worn by the workers, can be used as key points for matching.

[0098] The matching result of the first matching includes whether a detection box in the detection box of the next frame image successfully matches the first prediction box. Optionally, if the match is successful, the successfully matched detection box is determined as the detection box of the tracked object in the next frame image. If the match fails, it indicates that no detection box matching the first prediction box is determined in the detection box of the next frame image, and a target loss processing mechanism can be initiated, such as reinitializing the tracker or expanding the search area.

[0099] When the first matching adopts a single matching method, the matching result of the adopted single matching method is the matching result of the first matching.

[0100] When the first matching adopts a combination of multiple matching methods, the matching result of the first matching is determined according to the matching result of each matching method. Optionally, the matching scores of each matching method are weighted and summed to obtain a comprehensive matching score, and the matching result of the first matching is determined according to the comprehensive matching score.

[0101] A3. Update the tracking result of the tracked object according to the matching result, where the tracking result includes the location information of the tracked object.

[0102] Optionally, if the matching result indicates that a detection frame in the next frame image successfully matches the first prediction frame, then the successfully matched detection frame is determined as the detection frame of the tracked object in the next frame image, and the position information of the detection frame is used to update the position information of the tracked object. Other state information of the detection frame can also be used to update the state information of the tracked object, such as speed, acceleration, direction of movement, etc. At the same time, the confidence of the tracked object can also be updated according to the confidence of the matching result.

[0103] Optionally, if the matching result indicates that there is no detection box matching the first prediction box in the next frame image, a target loss processing mechanism is initiated, such as re-initializing the tracker or expanding the search range to re-match.

[0104] The object tracking method provided by the present application first obtains the current state information and tracking information of the tracked object, the current state information includes the current position and the current speed, the tracking information includes the scene information, and the scene information includes the spatial data information of the tracking area; secondly, according to the current state information and the predicted value of the impact of the tracking information on the motion state of the tracked object, the predicted position of the tracked object after a preset time interval is determined; finally, according to the predicted position, the tracked object is tracked. The object tracking method provided by the present application combines the scene information of the tracking area, and can more accurately predict the possible movement trajectory of the tracked object in a fixed area. Compared with relying on the statistical characteristics of random walks, it can more accurately estimate the speed information of the tracked object at different positions, and then more accurately predict the position of the tracked object, thereby improving the tracking effect.

[0105] As an optional implementation manner, the tracking area includes at least one sub-area, and the scene information further includes attribute information of the sub-area, and the attribute information is used to characterize the regional characteristics of the sub-area.

[0106] In a large or complex tracking environment, the functions and characteristics of different areas may be very different. Optionally, in order to better understand and adapt to the complex environment in which the tracked object is active, the tracking area can be divided into one or more sub-areas, thereby helping the system to more accurately analyze and predict the behavior of the tracked object in different sub-areas.

[0107] For example, in an engineering construction scenario, the tracking area can be divided into multiple sub-areas such as the working area (such as the working area under the crane tower, the working area on the crane tower, etc.), the material stacking area, the changing room, the worker rest area, the dangerous area (such as the crane operating radius area, the heavy object lifting area, etc.), and the worker check-in area according to actual needs and scenario characteristics.

[0108] Exemplarily, after constructing a three-dimensional tracking area model based on the spatial data information collected by the drone and / or the digital data coordinates collected by the mobile terminal device, different sub-areas can be divided in the tracking area model according to pre-set sub-area division rules (such as division by function, division by risk wind turbine, etc.), and corresponding attribute information can be set for each sub-area.

[0109] The attribute information of the sub-area refers to a series of data and information corresponding to each sub-area collected in order to more accurately describe and understand the characteristics of different sub-areas in the tracking area. These attribute information specifically characterize the regional characteristics of the sub-area, such as functional characteristics, environmental characteristics and other characteristics that may affect the movement of the target pedestrian. The attribute information of the sub-area may include but is not limited to the size, shape, functional use, security level, occlusion, specific environmental conditions and preset specific attributes (such as specified stay time, entry authority) of the sub-area. By analyzing and utilizing the attribute information of the sub-area, the object tracking system can more accurately predict and locate the movement trajectory and behavioral intention of the tracked object in different sub-areas, thereby improving the accuracy and efficiency of object tracking.

[0110] Exemplarily, the attribute information of a sub-area of ​​the tracking area includes dangerous area, dangerous level 2, persons with identity information A or B are allowed to enter, special safety protection equipment must be worn, etc. Exemplarily, the attribute information of a sub-area of ​​the tracking area includes locker room, camera blocking range, entrance and exit position of the blocking area, etc. Exemplarily, the attribute information of a sub-area of ​​the tracking area includes lunch break lounge, specified rest time, entrance and exit position of the lounge, etc.

[0111] As an optional implementation manner, the tracking information further includes walking information of the tracked object, and the walking information includes target path nodes and node travel order of the tracked object in the tracking area.

[0112] The walking information may be understood as pre-acquired information related to the expected walking of the tracked object in the tracking area, including target path nodes and node travel order of the tracked object in the tracking area.

[0113] The target path nodes can be understood as the target key position points that the tracking object passes through during walking, such as doorways, channel intersections, near important equipment or areas, etc. These nodes are important markers of the target movement trajectory. Optionally, when the tracking area is divided into multiple sub-areas, the target path nodes can also be one or more sub-areas. The node marching order can be understood as the order in which the tracking object visits the target path nodes during walking.

[0114] Optionally, the walking information may also include a specific target travel path of the tracked object in the tracking area. Optionally, the walking information may also include information such as the stay time of the tracked object at a specific target path node.

[0115] By analyzing the walking information of the tracked object, the path selection of the tracked object at the next moment can be more accurately predicted, and the movement of the tracked object at the next moment can be more accurately predicted.

[0116] In some application scenarios where the tracking area is relatively fixed and the movement pattern of the tracked object is relatively clear, the movement of the tracked object often follows a certain pattern or rule. For example, on a factory production line, workers may need to follow a fixed route for inspection or material handling every day; in an office building, employees usually follow a fixed path from the office to the meeting room, rest area, etc.

[0117] The walking information may be walking information of a common pattern shown by the tracked object in historical movement. By summarizing and analyzing movement data of a relatively fixed movement pattern of the tracked object in historical time, the walking information of the tracked object may be acquired in advance.

[0118] The walking information may also be walking information of the movement mode of the tracked object specified by certain specific requirements or rules. For example, in a factory or construction site, for safety or efficiency reasons, it may be stipulated that employees must follow a specific movement mode. By analyzing the specific requirements or rules, the walking information of the tracked object can be determined. Exemplarily, in an application scenario of a factory or construction site, when the tracked object enters the tracking area, the tracked object will be authenticated to determine the identity information of the tracked object, and then the work task of the tracked object will be determined based on the identity information of the tracked object and the work ticket information of the on-site work task, and the walking information of the tracked object will be determined based on the work task.

[0119] As an optional implementation, step S202 "determining the predicted position of the tracked object after a preset time interval according to the current state information and the predicted value of the impact of the tracking information on the motion state of the tracked object" includes steps B1-B2:

[0120] B1. Determine speed information of the tracked object within a preset time interval according to the current state information and a predicted value of the impact of the tracking information on the motion state of the tracked object.

[0121] The speed information of the tracked object within the preset time interval includes the speed information of the tracked object at each moment within the preset time interval. The speed information includes speed magnitude and speed direction.

[0122] According to the foregoing content, the current state information includes the current position information and current speed information of the tracked object. The predicted value of the impact of the tracking information on the motion state of the tracked object can reflect the change of the motion state of the tracked object within a preset time interval as auxiliary information. The speed change includes a change in speed magnitude and a change in speed direction. Therefore, by analyzing the current state information and the predicted value of the impact of the tracking information on the motion state of the tracked object, the speed information of the tracked object within the preset time interval can be determined.

[0123] As an optional implementation method, after determining the current state information and tracking information of the tracked object, a walking fluid equation of the tracked object is established based on the current state information and tracking information of the tracked object. The walking fluid equation can automatically determine the predicted value of the impact of the tracking information on the motion state of the tracked object, and determine the speed information of the tracked object within a preset time interval based on the current state information of the tracked object and the predicted value of the impact of the tracking information on the motion state of the tracked object.

[0124] As an optional implementation method, after determining the tracking information of the tracked object, a walking fluid equation of the tracked object is pre-established based on the tracking information of the tracked object, and the walking fluid equation can automatically determine the predicted value of the impact of the tracking information on the motion state of the tracked object. After determining the current state information of the tracked object, the current state information is input into the walking fluid equation, and the walking fluid equation determines the speed information of the tracked object within a preset time interval.

[0125] B2. Determine a predicted position of the tracked object after the preset time interval according to the current state information and the speed information.

[0126] After determining the current state information of the tracked object and the speed information of the tracked object within the preset time interval, the current state information and the speed information of the tracked object within the preset time interval are analyzed to determine the predicted position of the tracked object after the preset time interval.

[0127] As an optional implementation, a digital coordinate integration model is used to determine the predicted position of the tracked object after a preset time interval based on the current state information of the tracked object and the speed information of the tracked object within the preset time interval. Specifically, the integration step length is calculated in space based on the speed information of the tracked object at the current moment or the average speed information within the preset historical time period, and then the speed integral value within the preset time interval is calculated based on the integration step length, and the predicted position of the tracked object after the preset time interval is determined based on the speed integral value and the current state information.

[0128] As an optional implementation method, step S202 is implemented based on a model-based processing approach.

[0129] In this implementation, step S202 "determining the predicted position of the tracked object after a preset time interval based on the predicted value of the influence of the current state information and the tracking information on the motion state of the tracked object" includes: inputting the current state information and the tracking information of the tracked object into a pre-trained position prediction model, and the position prediction model performs a position prediction analysis based on the predicted value of the influence of the current state information of the tracked object and the tracking information on the motion state of the tracked object, to determine the predicted position of the tracked object after a preset time interval.

[0130] Optionally, the position prediction model performs position prediction analysis based on current state information of the tracked object and the predicted value of the impact of the tracking information on the motion state of the tracked object, and before determining the predicted position of the tracked object after a preset time interval, the predicted value of the impact of the tracking information on the motion state of the tracked object is first determined based on the input tracking information.

[0131] Specifically, a position prediction model is pre-trained in an embodiment of the present application. The position prediction model can be obtained by performing position prediction training on a third training sample. The third training sample includes multiple groups of data, each group of data includes sample current state information, sample tracking information, and annotated positions. The third training sample can use an existing public data set, or it can create and annotate a data set by itself. The position prediction training aims to enable the position prediction model to correctly predict the predicted position of the tracked object after a preset time interval. The trained position prediction model can perform position prediction analysis based on the input current state information and tracking information of the tracked object, and output the predicted position of the tracked object after a preset time interval.

[0132] The first speech recognition model can adopt any model for speech recognition and can be implemented based on a variety of deep learning neural network structures, such as CNN (Convolutional Neural Networks) network structure, RNN (Recurrent Neural Networks) network structure, CRNN (Convolutional Recurrent Neural Network) network structure, etc. This application does not limit this.

[0133] The position prediction model can adopt any model for position prediction or target tracking, which can be implemented based on machine learning or deep learning algorithms, such as linear regression models, support vector machines (SVM), random forests, gradient boosting trees (GBM), neural networks (including multi-layer perceptrons, convolutional neural networks CNN, long short-term memory networks LSTM, etc.) or algorithm models specific to target tracking such as Kalman filters, particle filters, etc., or hybrid models based on the combination of physical models and deep learning can also be used. This application does not limit this, and a suitable model or algorithm can be selected according to actual application scenarios and needs.

[0134] As an optional implementation, Figure 3 As shown, the method further includes step S301:

[0135] S301, determining whether the tracked object enters the first sub-area according to the current position of the tracked object and spatial data information of a preset first sub-area, and generating an alarm message when it is determined that the tracked object enters the first sub-area, the first sub-area being a preset sub-area that the tracked object is prohibited from entering; and / or, when the tracked object enters a second sub-area, acquiring a first feature of the tracked object, and when the first feature does not meet a preset first feature condition, determining that the tracked object cannot enter a third sub-area located in the second sub-area, and generating an alarm message, the first feature condition being determined based on an access condition of the third sub-area.

[0136] The first sub-area can be understood as a preset sub-area that the tracked object is prohibited from entering, that is, a sub-area that the tracked object has no permission to enter. The setting of the first sub-area is usually based on some important considerations, such as security, privacy, and protection of specific resources, etc., which are not limited in this application. For example, in a construction scene involving a tower crane operating at high altitude,

[0137] Specifically, the object tracking system will compare the current position of the tracked object with the spatial data information of the preset first sub-area. This comparison process mainly uses some algorithms, such as calculating the spatial distance between the two, to determine whether the tracked object has physically entered the first sub-area. If the object tracking system determines that the tracked object has entered the first sub-area, an alarm message will be generated immediately. The alarm message can be in a variety of forms such as text, sound, or photoelectric, to remind relevant personnel to pay attention and take corresponding countermeasures.

[0138] When the tracked object enters the second sub-area, the object tracking system obtains the first feature of the tracked object. The first feature may include but is not limited to shape, color, speed, direction, etc., depending on the application scenario and requirements. The first feature may be obtained through various sensors, such as cameras, radars, etc.

[0139] After obtaining the first feature, the object tracking system will compare it with the preset first feature condition. If the first feature does not meet the first preset condition, the object tracking system will determine that the tracked object cannot enter the third sub-area located in the second sub-area, and generate an alarm message.

[0140] The setting of the first characteristic condition is determined based on the access condition of the third sub-area. In other words, the third sub-area is set with special requirements or restrictions, and only the tracked objects that meet these requirements or restrictions are allowed to enter. These access conditions may involve many factors such as safety, efficiency, and management rules. Therefore, by comparing the first feature with the first characteristic condition, the object tracking system can effectively control and manage the tracked objects entering the third sub-area, thereby ensuring the normal operation and order of the area.

[0141] According to the foregoing content, it can be known that in the object tracking method provided in the embodiment of the present application, the spatial data information of the tracking area included in the scene information may be three-dimensional spatial data, which can provide three-dimensional geometric information of the tracking area. Through the three-dimensional spatial data of the tracking area, the object tracking system can more accurately grasp the state changes of the tracked object in the vertical direction, thereby realizing the vertical tracking of the tracked object. Therefore, as an optional implementation method, when the spatial data information of the tracking area in the scene information is three-dimensional spatial data, the object tracking method provided in the embodiment of the present application is suitable for application scenarios that require vertical tracking, such as high-altitude work scenes such as tower crane construction sites, scenes involving precise vertical positioning, etc.

[0142] Exemplary Devices

[0143] Corresponding to the above-mentioned object tracking method, an embodiment of the present application also provides an object tracking device. Figure 4is a schematic diagram of the structure of an object tracking device provided in an embodiment of the present application, such as Figure 4 As shown, the object tracking device provided in the embodiment of the present application includes:

[0144] The first unit 401 is used to obtain current state information and tracking information of the tracked object, where the current state information includes a current position and a current speed, and the tracking information includes scene information, and the scene information includes spatial data information of the tracking area;

[0145] The second unit 402 is configured to determine a predicted position of the tracked object after a preset time interval according to the current state information and a predicted value of the impact of the tracking information on the motion state of the tracked object;

[0146] The third unit 403 is configured to track the tracked object according to the predicted position.

[0147] The object tracking device provided by the present application first obtains the current state information and tracking information of the tracked object through the first unit 401, the current state information includes the current position and the current speed, the tracking information includes the scene information, and the scene information includes the spatial data information of the tracking area; then the second unit 402 determines the predicted position of the tracked object after a preset time interval according to the current state information and the predicted value of the impact of the tracking information on the motion state of the tracked object; finally, the tracked object is tracked according to the predicted position through the third unit 403. The object tracking device provided by the present application combines the scene information of the tracking area, and can more accurately predict the possible movement trajectory of the tracked object in a fixed area. Compared with relying on the statistical characteristics of random walks, it can more accurately estimate the speed information of the tracked object at different positions, and then more accurately predict the position of the tracked object, thereby improving the tracking effect.

[0148] Optionally, the tracking information also includes walking information of the tracked object, and the walking information includes target path nodes and node travel order of the tracked object in the tracking area.

[0149] Optionally, the tracking area includes at least one sub-area, and the scene information further includes attribute information of the sub-area, where the attribute information is used to characterize regional characteristics of the sub-area.

[0150] Optionally, the second unit 402 may be specifically used for:

[0151] Determining speed information of the tracked object within a preset time interval according to the current state information and a predicted value of the impact of the tracking information on the motion state of the tracked object;

[0152] The predicted position of the tracked object after the preset time interval is determined according to the current state information and the speed information.

[0153] Optionally, the second unit 402 may be specifically used for:

[0154] The current state information and the tracking information are input into a pre-trained position prediction model, and the position prediction model determines the predicted position of the tracked object after a preset time interval based on the predicted value of the impact of the current state information and the tracking information on the motion state of the tracked object.

[0155] Optionally, the current state information includes state information of a first detection frame corresponding to the tracked object in the current frame image;

[0156] The second unit 402 can be specifically used for:

[0157] Determine, according to the current state information and the predicted value of the impact of the tracking information on the motion state of the tracked object, a predicted position of a first predicted frame corresponding to the tracked object in a next frame image, wherein the next frame image is a frame image after the current frame image in the video stream;

[0158] The third unit 403 can be specifically used for:

[0159] Obtaining a detection frame in the next frame image;

[0160] Performing a first match between the detection frame in the next frame image and the first prediction frame to determine a matching result, wherein the first matching includes at least one of cosine distance matching, Mahalanobis distance matching, Euclidean distance matching, stay matching, gait matching, and key point matching;

[0161] According to the matching result, the tracking result of the tracked object is updated, and the tracking result includes the location information of the tracked object.

[0162] Optionally, the tracking area includes at least one sub-area, and the device further includes:

[0163] a fourth unit, configured to determine whether the tracked object enters the first sub-area according to the current position of the tracked object and the spatial data information of the preset first sub-area, and generate alarm information when it is determined that the tracked object enters the first sub-area, wherein the first sub-area is a preset sub-area that the tracked object is prohibited from entering;

[0164] and / or,

[0165] The fifth unit is used to obtain a first feature of the tracked object when the tracked object enters the second sub-area, and when the first feature does not meet a preset first feature condition, determine that the tracked object cannot enter a third sub-area located in the second sub-area, and generate an alarm message, wherein the first feature condition is determined based on the access condition of the third sub-area.

[0166] The object tracking device provided in this embodiment belongs to the same application concept as the object tracking method provided in the above embodiments of this application, can execute the object tracking method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of executing the object tracking method. For technical details not fully described in this embodiment, please refer to the specific processing content of the object tracking method provided in the above embodiments of this application, and will not be repeated here.

[0167] The functions implemented by the above first unit 401, second unit 402 and third unit 403 may be implemented by the same or different processors, respectively, and are not limited in the embodiment of the present application.

[0168] It should be understood that the units in the above devices can be implemented in the form of a processor calling software. For example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit of the device, wherein the processor can be a general-purpose processor, such as a CPU or a microprocessor, etc., and the memory can be a memory in the device or a memory outside the device. Alternatively, the units in the device can be implemented in the form of hardware circuits, and the functions of some or all units can be realized by designing the hardware circuits. The hardware circuit can be understood as one or more processors; for example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are realized by designing the logical relationship of the components in the circuit; for another example, in another implementation, the hardware circuit can be implemented by PLD, taking FPGA as an example, which can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by the configuration file, so as to realize the functions of some or all of the above units. All units of the above devices can be implemented in the form of a processor calling software, or in the form of hardware circuits, or in part by a processor calling software, and the remaining part is implemented in the form of hardware circuits.

[0169] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and run instructions, such as a CPU, a microprocessor, a GPU, or a DSP; in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0170] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0171] In addition, all or part of the units in the above device can be integrated together, or can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a SOC. The SOC may include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device. The type of the at least one processor may be different, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0172] Exemplary Electronic Devices

[0173] Another embodiment of the present application also provides an object tracking device, see Figure 5 As shown, the device includes:

[0174] Memory 200 and processor 210;

[0175] The memory 200 is connected to the processor 210 and is used to store programs;

[0176] The processor 210 is used to implement the object tracking method disclosed in any of the above embodiments by running the program stored in the memory 200.

[0177] Specifically, the object tracking device may further include: a bus, a communication interface 220 , an input device 230 and an output device 240 .

[0178] The processor 210, the memory 200, the communication interface 220, the input device 230 and the output device 240 are connected to each other via a bus.

[0179] A bus may include a pathway that transfers information between components of a computer system.

[0180] The processor 210 may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the scheme of the present invention. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0181] The processor 210 may include a main processor, and may also include a baseband chip, a modem, and the like.

[0182] The memory 200 stores a program for executing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include a program code, and the program code includes a computer operation instruction. More specifically, the memory 200 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk storage, a flash, and the like.

[0183] The input device 230 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0184] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0185] The communication interface 220 may include any transceiver or the like to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0186] The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement each step of any object tracking method provided in the above embodiments of the present application.

[0187] An embodiment of the present application also proposes a chip, which includes a processor and a data interface. The processor reads and runs a program stored in a memory through the data interface to execute the object tracking method introduced in any of the above embodiments. The specific processing process and its beneficial effects can be found in the introduction to the above-mentioned embodiment of the object tracking method.

[0188] Exemplary computer program products and storage media

[0189] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the object tracking method according to various embodiments of the present application described in any of the above embodiments of this specification.

[0190] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0191] In addition, the embodiment of the present application may also be a storage medium, on which a computer program is stored, and the computer program is executed by a processor to execute the steps of the object tracking method according to various embodiments of the present application described in any of the above embodiments of this specification, and specifically the following steps may be implemented:

[0192] S201. Acquire current state information and tracking information of a tracked object, wherein the current state information includes a current position and a current speed, and the tracking information includes scene information, and the scene information includes spatial data information of a tracking area.

[0193] S202: Determine a predicted position of the tracked object after a preset time interval according to the current state information and a predicted value of the impact of the tracking information on the motion state of the tracked object.

[0194] S203: Track the tracking object according to the predicted position.

[0195] Exemplary Systems

[0196] Optionally, the embodiment of the present application further provides an object tracking system, including:

[0197] A monitoring device and an object tracking device as described in the aforementioned embodiments, wherein the monitoring device is used to obtain video data and / or image data within a monitoring area, and the monitoring device is communicatively connected with the object tracking device.

[0198] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0199] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0200] The steps in the methods of each embodiment of the present application can be adjusted in order, combined and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0201] The modules and sub-modules in the devices and terminals of the various embodiments of the present application can be combined, divided and deleted according to actual needs.

[0202] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the terminal embodiments described above are only schematic, for example, the division of modules or submodules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0203] The modules or submodules described as separate components may or may not be physically separated, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place, or they may be distributed on multiple network modules or submodules. Some or all of the modules or submodules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0204] In addition, each functional module or submodule in each embodiment of the present application may be integrated into one processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into one module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or in the form of software functional modules or submodules.

[0205] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0206] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly by hardware, software units executed by a processor, or a combination of the two. The software units may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0207] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0208] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An object tracking method, characterized in that: include: Acquire current state information and tracking information of the tracked object, wherein the current state information includes a current position and a current speed, and the tracking information includes scene information, and the scene information includes spatial data information of the tracking area; Determining a predicted position of the tracked object after a preset time interval according to the current state information and a predicted value of the impact of the tracking information on the motion state of the tracked object; The tracked object is tracked according to the predicted position.

2. The method according to claim 1, characterized in that The tracking information also includes walking information of the tracked object, and the walking information includes target path nodes and node travel order of the tracked object in the tracking area.

3. The method according to claim 1, characterized in that The tracking area includes at least one sub-area, and the scene information further includes attribute information of the sub-area, where the attribute information is used to characterize the regional characteristics of the sub-area.

4. The method according to claim 1, characterized in that: The step of determining the predicted position of the tracked object after a preset time interval according to the current state information and the predicted value of the impact of the tracking information on the motion state of the tracked object includes: Determining speed information of the tracked object within a preset time interval according to the current state information and a predicted value of the impact of the tracking information on the motion state of the tracked object; The predicted position of the tracked object after the preset time interval is determined according to the current state information and the speed information.

5. The method according to claim 1, characterized in that The step of determining the predicted position of the tracked object after a preset time interval according to the current state information and the predicted value of the impact of the tracking information on the motion state of the tracked object includes: The current state information and the tracking information are input into a pre-trained position prediction model, and the position prediction model determines the predicted position of the tracked object after a preset time interval based on the predicted value of the impact of the current state information and the tracking information on the motion state of the tracked object.

6. The method according to claim 1, characterized in that The current state information includes state information of a first detection frame corresponding to the tracked object in the current frame image; The step of determining the predicted position of the tracked object after a preset time interval according to the current state information and the predicted value of the impact of the tracking information on the motion state of the tracked object includes: Determine, according to the current state information and the predicted value of the impact of the tracking information on the motion state of the tracked object, a predicted position of a first predicted frame corresponding to the tracked object in a next frame image, wherein the next frame image is a frame image after the current frame image in the video stream; Tracking the tracked object according to the predicted position includes: Obtaining a detection frame in the next frame image; Performing a first match between the detection frame in the next frame image and the first prediction frame to determine a matching result, wherein the first matching includes at least one of cosine distance matching, Mahalanobis distance matching, Euclidean distance matching, stay matching, gait matching, and key point matching; According to the matching result, the tracking result of the tracked object is updated, and the tracking result includes the location information of the tracked object.

7. The method according to any one of claims 1 to 6, characterized in that: The tracking area includes at least one sub-area, and the method further includes: Determine whether the tracked object enters the first sub-area according to the current position of the tracked object and the spatial data information of the preset first sub-area, and generate alarm information when it is determined that the tracked object enters the first sub-area, the first sub-area being a preset sub-area that the tracked object is prohibited from entering; and / or, When the tracked object enters the second sub-area, a first feature of the tracked object is obtained, and when the first feature does not meet a preset first feature condition, it is determined that the tracked object cannot enter a third sub-area located in the second sub-area, and an alarm message is generated, wherein the first feature condition is determined based on an access condition of the third sub-area.

8. An object tracking device, characterized in that: include: A first unit is used to obtain current state information and tracking information of the tracked object, wherein the current state information includes a current position and a current speed, and the tracking information includes scene information, and the scene information includes spatial data information of the tracking area; A second unit is used to determine a predicted position of the tracked object after a preset time interval according to the current state information and a predicted value of the influence of the tracking information on the motion state of the tracked object; The third unit is used to track the tracked object according to the predicted position.

9. An object tracking device, characterized in that: including memory and processor; The memory is connected to the processor and is used to store programs; The processor is used to implement the object tracking method according to any one of claims 1 to 7 by running the program in the memory.

10. An object tracking system, characterized in that: include: A monitoring device and an object tracking device as claimed in claim 9, wherein: The monitoring device is used to obtain video data and / or image data in the monitoring area.

Citation Information

Cited By

  • Unmanned aerial vehicle public security inspection method, system and device

    CN120148073A