Multi-target displacement tracking method and system based on security video

Abstract

The application relates to the technical field of image processing, in particular to a multi-target displacement tracking method and system based on a security video. The method comprises the following steps: preprocessing video frames of a security video to generate corresponding multi-target video frame data; segmenting a moving target in adjacent target video frame data to obtain different moving targets; calculating the heterogeneity of any one of the moving targets, and predicting the position of the moving target according to the heterogeneity to generate a corresponding position prediction result; and giving a warning according to the difference between the position prediction result and the actual position of the moving target. The application solves the problem that the same prediction model leads to errors in target displacement tracking of the prediction model in the prior art.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to a method and system for multi-target displacement tracking based on security video. Background Technology

[0002] In modern logistics and warehousing management, warehouse security video surveillance systems are crucial tools for ensuring safety and improving operational efficiency. Traditional warehouse monitoring primarily relies on manual inspections and post-event video analysis, which suffers from low efficiency and an inability to respond to anomalies in real time. With the development of computer vision technology, multi-target displacement tracking technology is increasingly being applied to warehouse security. By tracking the real-time displacement of multiple targets (such as workers, forklifts, and goods) in video, the behavior of personnel and equipment within the warehouse can be dynamically monitored. However, the warehouse environment presents new challenges due to the diverse types of targets, frequent movement, changing lighting, and environmental complexity.

[0003] In warehouse security video surveillance, it is necessary to track the displacement of multiple targets (such as staff, forklifts, etc.) in real time to ensure safety and efficiency. Chinese patent application CN113920161A discloses a multi-target tracking method with a fusion mechanism, which describes how target tracking is further achieved by combining the previous frame image and the current frame image into a motion image pair and inputting it into a position prediction model.

[0004] However, since different targets in the warehouse move in different ways—workers walk slowly while forklifts move quickly—using the same prediction model would lead to inaccurate predictions, which in turn would cause errors in target displacement tracking. Summary of the Invention

[0005] To address the issue of errors in target displacement tracking caused by using the same prediction model, this application analyzes adjacent video frame data to identify the same moving target in different video frames. Based on the difference in instantaneous velocity of the same moving target in different frames, the heterogeneity of the moving target is calculated. Furthermore, the position of the moving target is predicted based on the heterogeneity of the moving target, thereby completing the displacement tracking of the target.

[0006] Firstly, this application provides a multi-target displacement tracking method based on security video, employing the following technical solution:

[0007] A multi-target displacement tracking method based on security video includes the following steps:

[0008] Preprocess the video frames of the security video to generate data for multiple target video frames;

[0009] Segment the moving targets in adjacent target video frame data to obtain different moving targets;

[0010] Calculate the heterogeneity of any one of the moving targets, and predict the position of the moving target based on the heterogeneity to generate the corresponding position prediction result;

[0011] A warning is issued based on the difference between the predicted position and the actual position of the moving target;

[0012] The calculation of the heterogeneity of any one of the moving targets includes: obtaining the inter-frame displacement of the same moving target in adjacent frames, and taking the ratio of the inter-frame displacement to the inter-frame time interval as the instantaneous velocity of the moving target in the frame. The greater the degree of change of the instantaneous velocity, the greater the heterogeneity of the moving target, and vice versa.

[0013] Optionally, the video frames of the security video are preprocessed, including:

[0014] Convert the video frames of the collected security video into grayscale images;

[0015] The grayscale image is denoised using a Gaussian filter and the image contrast is enhanced using histogram equalization.

[0016] Optionally, the moving targets in adjacent target video frame data are segmented to obtain different moving targets, including:

[0017] For the acquired continuous frame security video, starting from the second video frame, the absolute value of the difference between the gray value of all pixels in the next frame and the gray value of the corresponding pixel in the previous frame is subtracted from the gray value of the corresponding pixel in the previous frame to generate the corresponding difference image.

[0018] Edge detection algorithms are used to extract the edges of each moving target in the difference image to obtain different moving targets;

[0019] For moving targets in consecutive frames, calculate the similarity between any two moving targets in two adjacent frames;

[0020] For any first moving target in any frame, obtain the maximum similarity between the first moving target and all moving targets in the previous frame;

[0021] If the maximum similarity value is greater than a preset threshold, then the second moving target corresponding to all moving targets in the previous frame is obtained, and the first moving target and the second moving target are regarded as the same moving target.

[0022] Optionally, in the differential image, the region with a gray value that is not 0 is the region where the moving target is located.

[0023] Optionally, the similarity between any two moving targets in two adjacent frames is calculated, including:

[0024] Get the The first frame The first movement target and the first The first frame The grayscale difference between individual moving targets;

[0025] Calculate the first The first frame The first movement target and the first The first frame The Euclidean distance between the centers of mass of the moving targets;

[0026] Based on the grayscale difference and the Euclidean distance, the similarity between any two moving targets in two adjacent frames is calculated.

[0027] Optionally, the position of the moving target is predicted based on the heterogeneity, generating a corresponding position prediction result, including:

[0028] Obtain the position coordinates of any moving target in all frames it is in, and sort the position coordinates according to the order of the frames it is in;

[0029] Based on the heterogeneity of the moving target, the position coordinates of the moving target in the next frame are predicted, and the corresponding position prediction result is generated.

[0030] Optionally, an early warning is issued based on the difference between the predicted position and the actual position of the moving target, including:

[0031] Obtain the difference between the predicted position and the actual position of any moving target;

[0032] If the cumulative difference among all video frames collected within a preset continuous time period exceeds a preset distance, an early warning will be issued to the staff.

[0033] Secondly, this application provides a multi-target displacement tracking system based on security video, which adopts the following technical solution:

[0034] A multi-target displacement tracking system based on security video includes:

[0035] The preprocessing module is used to preprocess the video frames of security videos to generate data for multiple target video frames.

[0036] A target segmentation model is used to segment moving targets in adjacent target video frame data to obtain different moving targets;

[0037] The position prediction module is used to calculate the heterogeneity of any one of the moving targets, and predict the position of the moving target based on the heterogeneity, generating a corresponding position prediction result.

[0038] The early warning module is used to issue an early warning based on the difference between the predicted position and the actual position of the moving target;

[0039] The calculation of the heterogeneity of any one of the moving targets includes: obtaining the inter-frame displacement of the same moving target in adjacent frames, and taking the ratio of the inter-frame displacement to the inter-frame time interval as the instantaneous velocity of the moving target in the frame. The greater the degree of change of the instantaneous velocity, the greater the heterogeneity of the moving target, and vice versa.

[0040] This application has the following technical effects:

[0041] 1. By calculating the similarity of all moving targets in adjacent frames, the same target in consecutive video frames can be identified, making the identification results more accurate.

[0042] 2. This application determines the motion state of a moving target by analyzing the heterogeneity of any moving target;

[0043] 3. This application constructs different prediction models based on the heterogeneity of the moving target to predict its position, making the prediction results more accurate and further improving the accuracy of tracking and early warning results. Attached Figure Description

[0044] Figure 1 This is a flowchart illustrating the multi-target displacement tracking method based on security video according to this application. Detailed Implementation

[0045] This application discloses a multi-target displacement tracking method based on security video, referring to... Figure 1 ,include:

[0046] S1: Preprocess the video frames of the security video to generate corresponding target video frame data.

[0047] It should be noted that insufficient lighting or excessive dust in warehouse environments often degrades image quality, affecting tracking performance. Therefore, this application improves image quality and reduces noise interference by preprocessing the video footage.

[0048] In one embodiment, the video frames of the acquired security video are converted into grayscale images to reduce complexity; a Gaussian filter is used to denoise the grayscale images, and histogram equalization is applied to enhance the image contrast to generate data corresponding to multiple target video frames.

[0049] This completes the preprocessing of the video frames in the security video.

[0050] S2: Segment the moving targets in adjacent target video frame data to obtain different moving targets.

[0051] It should be noted that there are different moving targets in the collected warehouse video. For the purpose of analysis, the background area does not move. Therefore, this application uses the frame difference method to obtain the moving targets so as to further segment the targets.

[0052] In one embodiment, for the acquired continuous frame security video, starting from the second video frame, the absolute value of the difference between the grayscale values ​​of all pixels in the subsequent frame and the grayscale values ​​of the corresponding pixels in the previous frame is subtracted from the grayscale values ​​of all pixels in the subsequent frame to generate a difference image. In the difference image, the regions with non-zero grayscale values ​​represent the areas where the moving target is located.

[0053] Furthermore, edge detection algorithms are used to extract the edges of each moving target in the difference image, thereby obtaining different moving targets. For moving targets in consecutive frames, the similarity between any two moving targets in two adjacent frames is calculated, and the formula is as follows:

[0054]

[0055] In the formula, Indicates the first The first frame The first movement target and the first The first frame The similarity between moving targets; among which... These represent the frame number of the video frame, ; Indicates the ordinal number of the moving target in the video frame; Indicates the first The first frame The average gray level of a moving target; Indicates the first The first frame The average gray level of a moving target; Indicates the first The first frame The first movement target and the first The first frame The Euclidean distance between the centers of mass of the moving targets; Represented by natural constant An exponential function with base 1.

[0056] in, Indicates the first The first frame The first movement target and the first The first frame The larger the grayscale difference of a moving target, the greater the difference; conversely, the smaller the grayscale difference, the smaller the difference. Since the maximum grayscale difference is 255, we use 255 as the denominator for normalization. At the same time, in consecutive frames, the closer the displacement between two targets (i.e., the smaller the Euclidean distance between their centroids), the greater their similarity.

[0057] As an example, for any first moving target in any frame, the maximum similarity between the first moving target and all moving targets in the previous frame is obtained. If the maximum similarity is greater than a preset threshold, the corresponding second moving target among all moving targets in the previous frame is obtained, and the first moving target and the second moving target are considered as the same moving target. Here, the first moving target is any selected moving target in the current frame, and the second moving target is the moving target whose maximum similarity with the first moving target in the previous frame is greater than the preset threshold. The preset threshold is a pre-defined similarity threshold standard; that is, if the maximum similarity between the first moving target and any moving target in the previous frame is greater than this similarity threshold standard, the moving target is determined to be the second moving target. The similarity threshold standard can be set to 0.7.

[0058] This completes the analysis of multiple video frame data and reveals different moving targets.

[0059] S3: Calculate the heterogeneity of any moving target, predict the position of the moving target based on the heterogeneity, and generate the corresponding position prediction result.

[0060] It should be noted that since different targets move at different speeds, using the same model for prediction when tracking them will lead to inaccurate prediction results. Therefore, this application analyzes the motion characteristics of each moving target and selects different prediction coefficients based on the motion characteristics of different moving targets, thereby making the prediction results more accurate.

[0061] In one embodiment, for the same moving target, its inter-frame displacement in adjacent frames is obtained (i.e., the Euclidean distance between the centroids of the same moving target in adjacent frames); the ratio of the inter-frame displacement to the inter-frame time interval is used as the instantaneous velocity of the moving target in that frame.

[0062] As an example, the formula for calculating the heterogeneity of any moving target is as follows:

[0063]

[0064] In the formula, Indicates the first Heterogeneity of individual moving targets; This indicates that the video frame contains the first... Total number of frames for each moving target; Indicates the first The frame number ordinal number of each moving target; Indicates the first The moving target in the first The instantaneous velocity of a frame; Indicates the first The average speed of a moving target; Indicates the first The maximum instantaneous velocity of a moving target; Indicates the first The standard deviation of the instantaneous velocity of a moving target in its frame; This represents the standard normalization function.

[0065] Heterogeneity represents the stability of the moving target during its motion. The greater the heterogeneity, the worse the stability of the moving target during its motion; conversely, the smaller the heterogeneity, the stronger the stability of the moving target during its motion. This represents the difference between the speed and average speed of the moving target in different video frames. The larger the value, the greater the difference and the greater the heterogeneity of the moving target. Conversely, the smaller the value, the smaller the difference and the less heterogeneous the moving target. Indicates the first The degree of instantaneous velocity change of a moving target in its frame is determined by the value of the value. The larger the value, the greater the degree of instantaneous velocity change and the greater the heterogeneity of the moving target. Conversely, the smaller the value, the smaller the heterogeneity of the moving target.

[0066] At this point, the heterogeneity calculation for any moving target is complete.

[0067] Furthermore, for any moving target, the greater its heterogeneity, the greater its changes in the most recent consecutive frames. Therefore, when predicting it, it is necessary to increase the prediction weight of its most recent video frame and decrease the prediction weight of its more distant video frames.

[0068] As an example, for any moving target, obtain the position coordinates of the moving target in all its frames, sort the position coordinates according to the order of the frames, and predict the position coordinates of the moving target in the next frame based on the heterogeneity of the moving target. The prediction formula is as follows:

[0069]

[0070]

[0071] In the formula, Indicates the first The moving target in the first The predicted value of the frame's position coordinates; Indicates the first The moving target in the first The predicted value of the frame's position coordinates; Indicates the first The moving target in the first The actual values ​​of the frame's position coordinates; Indicates the first Heterogeneity of individual moving targets.

[0072] For any moving target, the greater its heterogeneity, the greater the change in the target over the most recent video frames, thus requiring a higher weighting for those frames. This heterogeneity is directly used as the smoothing coefficient in the exponential smoothing method; a larger value results in a greater prediction weight for the most recent video frames.

[0073] This completes the position prediction for any moving target, generating and obtaining the corresponding position prediction result for that moving target.

[0074] S4: Issue an early warning based on the difference between the predicted position of a moving target and its actual position;

[0075] In one embodiment, the difference between the predicted position of any moving target and its actual position is obtained; if the cumulative difference in all video frames collected within a preset continuous time period exceeds a preset distance, an early warning is issued to the staff, thereby completing multi-target position tracking.

[0076] Among them, the preset continuous time period is the time period for the cumulative difference in all video frames to be collected, which can be 2 seconds; the preset distance is the preset distance range standard, which can be 5 meters.

[0077] For example, when monitoring a logistics warehouse where forklifts and workers are active, the system needs to track their locations in real time and issue alerts. The scenario is set as follows: Monitoring area: inside the logistics warehouse; Monitoring objects: forklifts and workers; Video frame acquisition frequency: 30 frames per second; Preset parameters: Preset continuous time period: 2 seconds; Preset distance: 5 meters.

[0078] Specifically, the system analyzes each frame of video and calculates the predicted position of each moving target (such as a forklift or worker) in the current frame through the preprocessing, target segmentation and position prediction steps mentioned above. At the same time, the system obtains the actual position of each moving target through actual video frame analysis.

[0079] As an example, for each moving target, the system calculates the difference between its predicted position and its actual position. Suppose that at a certain moment, the system predicts the forklift's position coordinates in the next frame to be (X1, Y1), while the actual position coordinates are (X2, Y2). The difference can be obtained by calculating the Euclidean distance between the two points. Within a preset continuous time period (2 seconds), the system collects 60 frames of video (assuming 30 frames per second). For each moving target, the system calculates the difference between the predicted position and the actual position in each of these 60 frames and accumulates these differences to obtain the cumulative difference. If the cumulative difference for a moving target exceeds a preset distance (5 meters) within 2 seconds, the system will trigger an alert. For example, suppose that within 2 seconds, the forklift's cumulative difference is 6 meters (exceeding the preset 5 meters), the system will immediately issue an alert to the staff, indicating that the forklift's movement trajectory is abnormal and may pose a safety hazard.

[0080] This application also discloses a multi-target displacement tracking system based on security video, including:

[0081] The preprocessing module is used to preprocess the video frames of security videos to generate data for multiple target video frames.

[0082] The target segmentation model is used to segment moving targets in adjacent target video frame data to obtain different moving targets;

[0083] The position prediction module is used to calculate the heterogeneity of any moving target, predict the position of the moving target based on the heterogeneity, and generate the corresponding position prediction result.

[0084] The early warning module is used to issue early warnings based on the difference between the predicted position of a moving target and its actual position.

[0085] The calculation of the heterogeneity of any moving target includes: obtaining the inter-frame displacement of the same moving target in adjacent frames, and taking the ratio of the inter-frame displacement to the inter-frame time interval as the instantaneous velocity of the moving target in the frame. The greater the change in instantaneous velocity, the greater the heterogeneity of the moving target, and vice versa.

[0086] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and will not be described in detail here.

[0087] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A method for multi-target displacement tracking based on security video, characterized in that, The method comprises the following steps: Preprocessing video frames of a security video to generate corresponding target video frame data; Segmenting moving targets in adjacent target video frame data to obtain different moving targets; Calculating the heterogeneity of any one of the moving targets and predicting the position of the moving target according to the heterogeneity to generate a corresponding position prediction result, wherein the heterogeneity represents the stability of the moving target in the movement process, the greater the heterogeneity, the poorer the stability of the moving target in the movement process, and vice versa, the smaller the heterogeneity, the stronger the stability of the moving target in the movement process, for any one moving target, the greater the heterogeneity corresponding to the moving target, the greater the change amount of the moving target in the recent continuous frames, and when the moving target is predicted, the prediction weight of the recent video frame of the moving target is increased and the prediction weight of the video frame far away from the moving target is reduced; Warning according to the difference between the position prediction result of the moving target and the actual position; The calculation of the heterogeneity of any one of the moving targets comprises: obtaining the inter-frame displacement of the same moving target, taking the ratio of the inter-frame displacement and the inter-frame time interval as the instantaneous speed of the moving target in the frame, and the greater the change degree of the instantaneous speed, the greater the heterogeneity of the moving target, and vice versa, the smaller the heterogeneity of the moving target.

2. The method of claim 1, wherein, The preprocessing of the video frames of the security video comprises: Converting the video frames of the collected security video into gray scale images; Using a Gaussian filter to denoise the gray scale images and using a histogram equalization technique to enhance the contrast of the images.

3. The method of claim 1, wherein, Segmenting moving targets in adjacent target video frame data to obtain different moving targets comprises: For the obtained continuous frame security video, starting from the second video frame, the absolute value of the difference between the gray scale value of all pixel points of the subsequent frame image and the gray scale value of the pixel points at the corresponding position of the previous frame image is generated to generate a corresponding difference image; Using an edge detection algorithm to extract the edges of each moving target in the difference image to obtain different moving targets; For continuous frame moving targets, calculating the similarity between any two moving targets in adjacent two frames; For any one first moving target in any frame, obtaining the maximum value of the similarity between the first moving target and all moving targets in the previous frame; If the maximum value of the similarity is greater than a preset threshold, obtaining the corresponding second moving target among all moving targets in the previous frame, and regarding the first moving target and the second moving target as the same moving target.

4. The method of claim 3, wherein, In the difference image, the area with a gray scale value of 0 is the area where the moving target is located.

5. The method of claim 3, wherein, Calculating the similarity between any two moving targets in adjacent two frames comprises: Get the The first frame The first movement target and the first The first frame The grayscale difference between individual moving targets; computing a first euclidean distance between the centroid of the first moving object in the first frame and the centroid of the first moving object in the second frame moving object in the second frame Based on the maximum gray scale difference and the Euclidean distance, the similarity between any two moving targets in adjacent two frames is calculated.

6. The method of claim 1, wherein, According to the heterogeneity, the position of the moving target is predicted to generate a corresponding position prediction result, comprising: Obtain the position coordinates of any moving target in all frames it is in, and sort the position coordinates according to the order of the frames it is in; Based on the heterogeneity of the moving target, the position coordinates of the moving target in the next frame are predicted, and the corresponding position prediction result is generated.

7. The method of claim 1, wherein, Early warning is issued based on the difference between the predicted position and the actual position of the moving target, including: Obtain the difference between the predicted position and the actual position of any moving target; If the cumulative difference among all video frames collected within a preset continuous time period exceeds a preset distance, an early warning will be issued to the staff.

8. A multi-target displacement tracking system based on security video, characterized in that, include: The preprocessing module is used to preprocess the video frames of security videos to generate data for multiple target video frames. A target segmentation model is used to segment moving targets in adjacent target video frame data to obtain different moving targets; The position prediction module is used to calculate the heterogeneity of any moving target and predict the position of the moving target based on the heterogeneity, generating a corresponding position prediction result. Here, the heterogeneity represents the stability of the moving target during the motion process. The greater the heterogeneity, the worse the stability of the moving target during the motion process. Conversely, the smaller the heterogeneity, the stronger the stability of the moving target during the motion process. For any moving target, the greater the heterogeneity corresponding to the moving target, the greater the change of the moving target in the most recent consecutive frames. Therefore, when predicting the moving target, the prediction weight of the most recent video frame of the moving target is increased, and the prediction weight of video frames that are farther away from the moving target is decreased. The early warning module is used to issue an early warning based on the difference between the predicted position and the actual position of the moving target; The calculation of the heterogeneity of any one of the moving targets includes: obtaining the inter-frame displacement of the same moving target in adjacent frames, and taking the ratio of the inter-frame displacement to the inter-frame time interval as the instantaneous velocity of the moving target in the frame. The greater the degree of change of the instantaneous velocity, the greater the heterogeneity of the moving target, and vice versa.

Images