A Cross-Domain Tracking Method and System for Campus Security Based on Massive Video Data

Abstract

This disclosure provides a method and system for cross-domain tracking of campus security based on massive video data, relating to the field of image detection and recognition technology. The method includes acquiring real-time campus surveillance video and uploading it to an edge embedded device for video stream decoding; acquiring video streams containing moving targets; converting these video streams into video frame images and inputting them into a YOLO network and a deepsort network for discrimination and tracking, assigning each pedestrian a unique number; storing video frame images of pedestrians identifying the same target separately to establish a historical pedestrian sequence image library; using a re-identification model to compare the features of the image sequence of the pedestrian to be found with historical pedestrian sequence images, outputting sequences with high similarity; filtering the output sequences with high similarity by time region; and drawing the trajectory route of the target pedestrian based on a map of the campus surveillance area. This disclosure reduces the time cost of the model reading and comparing massive amounts of images.

Description

Technical Field

[0001] This disclosure relates to the field of image detection and recognition technology, specifically to a cross-domain tracking method and system for campus security based on massive amounts of video. Background Technology

[0002] The statements in this section are merely background information relating to this disclosure and do not necessarily constitute prior art.

[0003] With the ever-expanding demand for public security and the increasingly sophisticated video surveillance systems—a primary means of security—the number of cameras is growing year by year, especially in high-security locations, businesses, and institutions such as schools and airports. In recent years, various issues related to campus safety and the movement of people within schools have become increasingly important. However, traditional campus security relies solely on manual monitoring and analysis of massive amounts of video footage, which is inefficient and consumes significant manpower and resources. With the development of artificial intelligence (AI) technology, excellent algorithms and models for target detection, personnel identification, and tracking within videos are emerging, making it possible to solve campus safety problems through AI.

[0004] The inventors discovered that most mainstream pedestrian re-identification methods are based on searching and comparing single-frame images. First, each frame from each camera is processed using a YOLO model. Then, all pedestrian slices in that frame are output and stored in a historical image library in a gallery directory. Next, all similar images of a specified person in the historical image library are searched. However, for more complex scenes, issues such as pedestrian occlusion, variations in lighting intensity from different cameras, differences in camera resolution, and variations in viewpoint and pose often lead to insufficient pedestrian information or significant discrepancies in matching, resulting in misidentification. Summary of the Invention

[0005] To address the aforementioned issues, this disclosure proposes a cross-domain tracking method and system for campus security based on massive video streams. This method embeds artificial intelligence technologies into mobile devices or deploys them on edge devices to preprocess massive video streams, detect and track multiple frames of moving targets in the video stream, and plot the target's trajectory, thereby achieving accurate identification of moving targets.

[0006] According to some embodiments, the present disclosure adopts the following technical solutions:

[0007] Cross-domain tracking methods for campus security based on massive amounts of video data include:

[0008] The campus is divided into monitoring zones, real-time monitoring videos of the campus are acquired and uploaded to edge embedded devices for video stream decoding;

[0009] The decoded video stream is preprocessed to obtain a video stream with moving targets;

[0010] The video stream with moving targets is converted into video frame images. A video frame image is captured at the same time interval and input into the YOLO network for local pedestrian detection. Then, the recognition result and the coordinates of the pedestrian in the slice of the video frame image are input into the deepsort network for discrimination and tracking, and the pedestrian is assigned a number. Video frame images of pedestrians that are identified as the same target are stored separately to build a historical pedestrian sequence image library.

[0011] The re-identification model is used to compare the features of the image sequence of the pedestrian to be found with the historical pedestrian sequence image. A temporal and spatial correlation attention mechanism module is introduced. For each pair of nodes in the feature map, an association result is constructed. The association results are stacked into a feature matrix in the form of pairwise relation terms. The sequence with high similarity is output. The sequence with high similarity is filtered by time region. The trajectory route of the target pedestrian is drawn according to the campus monitoring area map.

[0012] According to some embodiments, the present disclosure adopts the following technical solutions:

[0013] A campus security cross-domain tracking system based on massive video data includes:

[0014] The data acquisition module is used to divide the campus monitoring area, acquire real-time monitoring video of the campus, and upload it to the edge embedded device for video stream decoding.

[0015] The motion video detection module is used to preprocess the decoded video stream to obtain a video stream with moving targets;

[0016] The local moving target monitoring and tracking module is used to convert video streams with moving targets into video frame images. It captures a video frame image at the same time interval and inputs it into the YOLO network for local pedestrian detection. Then, it inputs the recognition result and the coordinates of the pedestrian slice in the video frame image into the deepsort network for discrimination and tracking, and assigns a pedestrian number. The video frame pedestrian images that are identified as the same target are stored separately to establish a historical pedestrian sequence image library.

[0017] The re-identification module is used to compare the features of the image sequence of the pedestrian to be found with the historical pedestrian sequence images using the re-identification model. It introduces a temporal and spatial correlation attention mechanism module, and constructs an association result between each pair of nodes in the feature map. The association results are stacked into a feature matrix in the form of pairwise relation terms, and the sequence with high similarity is output. The sequence with high similarity is filtered by time region, and the trajectory route of the target pedestrian is drawn according to the campus monitoring area map.

[0018] According to some embodiments, the present disclosure adopts the following technical solutions:

[0019] A non-transitory computer-readable storage medium is provided for storing computer instructions, which, when executed by a processor, implement the aforementioned cross-domain tracking method for campus security based on massive video data.

[0020] According to some embodiments, the present disclosure adopts the following technical solutions:

[0021] An electronic device includes a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the aforementioned cross-domain tracking method for campus security based on massive video.

[0022] Compared with the prior art, the beneficial effects of this disclosure are as follows:

[0023] This disclosure provides a cross-domain tracking method for campus security based on massive video streams. It preprocesses the massive real-time video streams and uses a low-cost embedded device with hardware video decoding to decode the real-time video streams, reducing the computational cost of video decoding. Furthermore, considering the campus setting, not all cameras are used when pedestrians are passing by. Therefore, to avoid the significant waste of computational power in processing large amounts of similar video streams without pedestrians, a low-complexity differential algorithm is used to detect motion frames in the decoded video streams, greatly reducing the cost of analyzing and processing massive amounts of real-time video.

[0024] This disclosure uses YOLO to detect pedestrians under the same camera, and then uses the low-complexity deepsort algorithm to track the same pedestrian under a single camera. This increases the temporal and spatial correlation of pedestrian images and greatly reduces false identifications caused by factors such as similar appearance or lighting. Furthermore, the same trajectory of the same person under a single camera is stored separately. When searching the historical image database, it is not necessary to send all images to the recognition model for feature reading and comparison; only a few images of the same trajectory need to be sent, greatly reducing the time cost of running the model to read and compare massive amounts of images.

[0025] This disclosure post-processes the results of the input pedestrian re-identification model. In order to avoid misidentification and misfollowing of pedestrians with similar appearances, an effective strategy is proposed in the spatiotemporal aspect of the monitoring area. This strategy excludes objects that are identified as the same target but have a large spatial span in a short period of time, which effectively reduces the false recognition rate and makes the target trajectory more reasonable and smooth. Attached Figure Description

[0026] The accompanying drawings, which form part of this disclosure, are used to provide a further understanding of this disclosure. The illustrative embodiments of this disclosure and their descriptions are used to explain this disclosure and do not constitute an undue limitation of this disclosure.

[0027] Figure 1 This is a schematic diagram of the structure of a campus security cross-domain tracking system for massive video data in this embodiment of the present disclosure;

[0028] Figure 2 This is a flowchart of the motion target detection algorithm based on background subtraction and inter-frame difference in this embodiment of the present disclosure; Detailed Implementation

[0029] The present disclosure will be further described below with reference to the accompanying drawings and embodiments.

[0030] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains.

[0031] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this disclosure. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0032] Example 1

[0033] One embodiment of this disclosure provides a cross-domain tracking method for campus security based on massive video data, including:

[0034] Step 1: Divide the campus monitoring area, acquire real-time campus monitoring video, and upload it to the edge embedded device for video stream decoding;

[0035] Step 2: Preprocess the decoded video stream to obtain a video stream with moving targets;

[0036] Step 3: Convert the video stream with moving targets into video frame images. Capture a video frame image at the same time interval and input it into the YOLO network for local pedestrian detection. Then, input the recognition result and the coordinates of the pedestrian in the slice of the video frame image into the deepsort network for discrimination and tracking, and assign a number to the pedestrian. Video frame images of pedestrians that are identified as the same target are stored separately to build a historical pedestrian sequence image library.

[0037] Step 4: Use the re-identification model to compare the features of the image sequence of the pedestrian to be found with the historical pedestrian sequence images. Introduce the temporal and spatial correlation attention mechanism module, and construct an association result between each pair of nodes in the feature map. Stack the association results into a feature matrix in the form of pairwise relation terms, output the sequence with high similarity, filter the output sequence with high similarity by time region, and draw the trajectory route of the target pedestrian based on the campus monitoring area map.

[0038] As one example, most mainstream pedestrian re-identification methods are based on searching and comparing single-frame images. First, each frame from each camera is processed using the YOLO model. Then, all pedestrian slices in that frame are output and stored in a historical image library in the gallery directory. Next, all similar images of a specified person in the historical image library are searched. However, for more complex scenarios, issues such as occlusion, variations in lighting intensity from different cameras, different camera resolutions, and changes in viewpoint and pose often lead to insufficient pedestrian information or significant discrepancies in matching, resulting in misidentification. Therefore, video-based pedestrian re-identification models can effectively combine the temporal features of the same pedestrian target and utilize inter-frame enhanced feature associations to effectively solve the aforementioned problems.

[0039] This disclosure uses the PSTA model, which takes a sequence of the same person as input and then analyzes and outputs sequences with similar appearances. Test results show that the model's accuracy in real-world scenarios is significantly lower than its accuracy on publicly available datasets, with a high false recognition rate for pedestrians with similar appearances. Furthermore, due to the model's large size, its real-time performance is completely insufficient for processing massive amounts of campus video. Therefore, the inventors propose an innovative improvement: a cross-domain tracking method for campus security based on massive amounts of video. The factors affecting real-time performance mainly include video decoding speed, the number of videos the model processes, and the number of model parameters. The specific implementation process is as follows:

[0040] Firstly, regarding video decoding speed, current popular video analysis methods rely on software decoding performed on servers. However, software decoding consumes significant computing resources, severely impacting subsequent server analysis. Therefore, this patent utilizes embedded edge mobile devices with hardware decoding capabilities for decoding massive amounts of video data. This effectively alleviates the problem of insufficient computing power on servers during the decoding phase, preventing real-time performance issues. Furthermore, since hardware decoding devices are far less expensive than servers, this method also significantly reduces the engineering costs of equipment deployment.

[0041] Step 1: Divide the campus monitoring area, acquire real-time campus monitoring video and upload it to the edge embedded device for video stream decoding;

[0042] The process of dividing the campus monitoring area includes: dividing the campus monitoring area into several sub-areas and numbering the sub-areas sequentially; associating the monitoring equipment in each sub-area with the corresponding sub-area and marking the orientation of the monitoring equipment; entering and saving the sub-area number and monitoring equipment number information; and drawing a map of the campus monitoring area after the numbering is completed. The monitoring equipment is the installed cameras.

[0043] Specifically, the implementation steps include the following:

[0044] S1: Campus area and monitoring equipment numbering. Divide the campus monitoring areas and number them. The sub-areas are named Z1, Z2, Z3, Z4... respectively. Based on the sub-areas, name the cameras in each sub-area and indicate the direction of each camera. The cameras are named Z1_C1_S, Z1_C2_E, Z2_C1_W... respectively, where the last letter indicates the direction of the camera.

[0045] Connect the numbered cameras to the embedded device with hardware video decoding, input the monitoring video for video decoding; associate the video streaming media address with the camera and establish the corresponding mapping relationship.

[0046] Furthermore, regarding the number of videos processed by the model, since there are thousands of surveillance videos in a campus setting, each video needs to be analyzed in real time. This places high demands on the performance and quantity of servers, necessitating a moving target detection method to filter videos containing moving targets. Currently, mainstream moving target detection methods fall into two categories: traditional computer vision-based methods, including frame differencing and optical flow; and deep learning-based methods. Optical flow and deep learning-based methods have high computational complexity and cannot meet the requirements of real-time video analysis. Frame differencing, including inter-frame differencing and background subtraction, has lower complexity, but it is prone to problems such as holes in moving targets, and is significantly affected by the surrounding environment. Background subtraction requires dynamic background updates, which are also resource-intensive. In outdoor campus scenes, interfering moving targets mainly include wind-blown plants and trees, and flashing lights. Therefore, this patent designs an improved background subtraction algorithm to detect videos containing moving targets in such scenarios.

[0047] Step 2: Preprocess the decoded video stream to obtain a video stream with moving targets;

[0048] Preprocessing involves filtering the motion video stream. The decoded video stream is preprocessed using a low-complexity background subtraction inter-frame difference algorithm to determine whether there are moving targets in the video frames. If there are moving targets, local pedestrian detection, recognition, and tracking are then performed.

[0049] Specifically, the video stream from each camera is first converted to grayscale, the background distribution is obtained by inter-frame difference of consecutive frames, and then the background of each frame is removed by constructing a mask to obtain the moving target in the foreground.

[0050] First, each frame in the video stream is converted to grayscale. Then, the difference between the grayscale images of the current frame and the previous frame is calculated, and the result is binarized. Next, the binarized result is processed according to a periodically updated mask, which can effectively eliminate the influence of periodic movement and flickering objects on moving target detection in outdoor scenes. A threshold is applied to the white pixel density of the binarized result after mask processing. If the density exceeds a specified threshold, a moving target is determined to exist in the current frame; otherwise, no moving target is found. Finally, only when a moving target appears in multiple consecutive frames is the video stream determined to have a moving target, and the video stream is then fed into the subsequent analysis model.

[0051] The mask construction process includes: First, the binarized results of the difference between two frames are accumulated. For a moving target walking in a single direction for a short period of time, the accumulated value of the pixel at the same position is limited. However, for a moving target with periodicity or a flickering object, the accumulated value of the pixel at the same position is relatively large. Therefore, by periodically binarizing the accumulated binarized results and updating the mask, it is possible to distinguish whether the problem is caused by a real foreground moving target or by a periodic background.

[0052] Step 3: Convert the video stream with moving targets into video frame images. Capture a video frame image at the same time interval and input it into the YOLO network for local pedestrian detection. Then, input the recognition result and the coordinates of the pedestrian in the slice of the video frame image into the deepsort network for discrimination and tracking, and assign a number to the pedestrian. Store the video frame pedestrian images that are identified as the same target separately to build a historical pedestrian sequence image library.

[0053] A local pedestrian detection and tracking model under a single camera was constructed and trained. To ensure real-time performance, the relatively lightweight pedestrian detection model YOLOv5s was selected as the pedestrian detection model and trained using the general object detection dataset COCO. The low-complexity multi-object tracking model deepsort was used to discriminate and track pedestrians in each frame and trained using the general multi-object tracking dataset MOT16.

[0054] For video frames containing moving targets, pedestrians are detected in each frame. At equal time intervals, one image is captured for pedestrian detection. The detection results and the coordinates of the pedestrians in the image frames are then fed into the deepsort network for discrimination and tracking. Pedestrians with the same target are assigned a pedestrian number until the target disappears from the monitored area.

[0055] Specifically, in the deepsort network, the input includes a pedestrian slice, its coordinates in the original video frame, and the pedestrian's confidence level. First, the network calculates the intersection-over-union (IoU) ratio by comparing the current slice's bounding box in the original video frame with the bounding boxes of all pedestrian slices from the previous frame. If the IoU is high, the slice is considered a candidate for tracking; otherwise, it is not. Then, for each subsequent frame, a pedestrian slice undergoes feature extraction using a pre-trained model. It is prioritized for feature matching with candidate slices; if no match is found, it is considered a secondary candidate. If no match is found after a certain number of matching attempts, the slice is discarded.

[0056] Finally, output the tracking number corresponding to the pedestrian.

[0057] Considering the massive amount of input video and the algorithmic logic of the deepsort network, we use a method of feeding one frame every 10 frames into the YOLO and deepsort networks for recognition and tracking, which greatly alleviates the pressure on real-time performance and cost caused by massive amounts of video.

[0058] The trajectory sequence of each pedestrian is stored. A time directory is created based on the date of the day, and the trajectories of all cameras on that day are stored in this time directory. For example, 2023 / 7 / 31 means that this directory contains the trajectories of all pedestrians in the campus monitoring area on July 31, 2023. Then, a camera directory is created under the time directory based on the camera number, and the pedestrian trajectories appearing in the corresponding camera are stored in this camera directory. For example, Z1_C1_S means that this directory contains the trajectories of all pedestrians monitored by the first south-facing camera in the Z1 area. Next, a trajectory directory is created under the camera directory, and the image frames of the same pedestrian with the same pedestrian number output by the deepsort network under the same camera are saved in the same trajectory directory. For example, tracklet1_7:30:10_7:31:50 means that the first trajectory under this camera contains all image frames of the same pedestrian from 7:30:10 to 7:31:50.

[0059] Specifically, since there are a large number of redundant images with high spatial similarity in the sequence of the same pedestrian, in order to ensure that more effective images can be stored, it is necessary to periodically delete redundant images from the sequence directory. When the number of images in a directory does not change within 5 minutes and the number of images in the directory is greater than 8, 8 images are selected at corresponding intervals according to the number of images in the trajectory directory, and then the other images are deleted.

[0060] Step 4: Use the re-identification model to compare the features of the image sequence of the pedestrian to be found with the historical pedestrian sequence images, output the sequence with high similarity, filter the output sequence with high similarity by time region, and draw the trajectory route of the target pedestrian based on the campus monitoring area map.

[0061] A pedestrian re-identification model was constructed and trained. Due to the complexity of campus scenes, issues such as pedestrian occlusion, varying lighting intensity under different cameras, different camera resolutions, and variations in viewpoint and pose often lead to insufficient pedestrian information or significant discrepancies in matching, resulting in misidentification. Therefore, incorporating temporal features as a feature dimension for fusion can effectively solve these problems. Thus, the PSTA model, a pedestrian re-identification model incorporating temporal features, was adopted. The input to this model is a time-series image of the same person. By comparing the fused target sequence features with sequence features in a sequence image database, the model identifies sequences with high similarity and outputs the correct sequence.

[0062] Furthermore, the pedestrian sequence to be searched and all image sequences under the specified date directory are input into the PSTA model for re-identification. The output is the directory name of the sequence with high similarity and the directory name of the previous level camera, forming an array output corresponding to the pedestrian time period and region.

[0063] Specifically, the input pedestrian sequences to be searched should ideally include those with minimal occlusion of the target person, ensuring that as many features of the target as possible are captured. The input sequence image library does not contain all images of the sequence; instead, eight images are input at corresponding intervals based on the number of images in the sequence directory. This ensures the model can capture the temporal features of the pedestrian, extracting more multi-dimensional features and improving recognition accuracy. The PSTA model is used here, which can capture the temporal features of the same person, and it is trained using the training set from the Mars dataset.

[0064] Specifically, in the PSTA model, the data processing flow is as follows: Input eight frames of images at specified intervals representing the same pedestrian and the same trajectory. The model fuses the information between frames to obtain a feature map with fused temporal features. This map also contains spatial information about the same person. The feature extraction module then extracts the final spatiotemporal features belonging to that person. All trajectory segments from the historical image library are also fed into the model (each trajectory segment is fed eight frames at specified intervals), and compared with the input trajectory features to obtain the closest specified trajectory.

[0065] However, regarding the model's parameters, due to the introduction of temporal features from the video sequences, this model needs to extract correlations between these temporal features and combine them with spatial features of the images. Therefore, compared to traditional image-based person re-identification models, the number of parameters is significantly larger. Specifically, to more meticulously focus on the spatial correlations within the video sequences and distinguish between pedestrians with high appearance similarity, this model introduces a temporal and spatial correlation attention mechanism module. Furthermore, it constructs a correlation result for each pair of nodes in the feature map, stacking these correlation results into a feature matrix in the form of pairwise relational terms. The stacking process is as follows:

[0066] r i =γ(r i,1 ···,r i,N ,r 1,i ···r N,i ),

[0067] Where r i,j The output of the channel fusion of the node in the i-th row and j-th column of the feature map can be represented as follows:

[0068] r i,j =θ(x i ) T φ(x j ),

[0069] Where θ, φ, and γ are learnable embedding functions.

[0070] The feature matrix is ​​learned using learnable weights, ultimately outputting an image based on a temporal and spatial correlation attention mechanism. This part involves a large number of parameters. Considering that in a campus scenario, images with high appearance similarity are relatively few, a sacrifice in recognition accuracy is made to improve real-time performance. The stacked matrix of pairwise relation terms is changed to a single-relation term stacking. The modified stacking process is as follows.

[0071] ri=γ(r i,1 ···,r i,N ),

[0072] This reduces the number of parameters in the entire model by nearly one-third, thereby improving real-time performance. There is a slight decrease in accuracy; analysis of the experimental results reveals that the model's ability to distinguish highly similar appearances has declined. Therefore, this patent incorporates a post-processing module into the model's output. Since the same person is unlikely to make significant spatial leaps in a short period, the model's output is filtered and restricted in both time and space, greatly reducing misidentification due to similar appearances.

[0073] Furthermore, time zones are filtered. Based on the array of time periods and corresponding areas for the specified pedestrians obtained from the re-identification, areas with large gaps between adjacent time periods are excluded, effectively reducing the false identification rate of two people with high similarity over long distances. Then, the time and location relationships between the excluded time period and corresponding array and the divided campus monitoring areas are used to draw the trajectory map of the specified person.

[0074] Example 2

[0075] One embodiment of this disclosure provides a campus security cross-domain tracking system based on massive video data, including:

[0076] The data acquisition module is used to divide the campus monitoring area, acquire real-time monitoring video of the campus, and upload it to the edge embedded device for video stream decoding.

[0077] The motion video detection module is used to preprocess the decoded video stream to obtain a video stream with moving targets;

[0078] The local moving target monitoring and tracking module is used to convert video streams with moving targets into video frame images. It captures a video frame image at the same time interval and inputs it into the YOLO network for local pedestrian detection. Then, it inputs the recognition result and the coordinates of the pedestrian slice in the video frame image into the deepsort network for discrimination and tracking, and assigns a pedestrian number. The video frame pedestrian images that are identified as the same target are stored separately to establish a historical pedestrian sequence image library.

[0079] The re-identification module is used to compare the features of the image sequence of the pedestrian to be found with the historical pedestrian sequence images using the re-identification model. It introduces a temporal and spatial correlation attention mechanism module, and constructs an association result between each pair of nodes in the feature map. The association results are stacked into a feature matrix in the form of pairwise relation terms, and the sequence with high similarity is output. The sequence with high similarity is filtered by time region, and the trajectory route of the target pedestrian is drawn according to the campus monitoring area map.

[0080] Example 3

[0081] One embodiment of this disclosure provides a non-transitory computer-readable storage medium for storing computer instructions. When these computer instructions are executed by a processor, they implement the aforementioned cross-domain tracking method for campus security based on massive video data.

[0082] Example 4

[0083] One embodiment of this disclosure provides an electronic device, including a processor, a memory, and a computer program; wherein the processor is connected to the memory, the computer program is stored in the memory, and when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the aforementioned cross-domain tracking method for campus security based on massive video.

[0084] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0085] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0086] While the specific embodiments of this disclosure have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of this disclosure. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of this disclosure are still within the scope of protection of this disclosure.

Claims

1. A campus security cross-domain tracking method based on massive videos, characterized in that, The method comprises the following steps: Divide the campus monitoring area, obtain real-time monitoring video of the campus, and upload the video to an edge embedded device for video stream decoding; Preprocess the decoded video stream to obtain a video stream with a moving target, including: Use background subtraction interframe difference algorithm on the decoded video stream to determine whether there is movement in the video stream, first perform grayscale processing on the video stream, obtain the distribution of the background through interframe difference of consecutive frames, remove the background of each frame through mask construction, and obtain the video stream of the running target in the foreground; The mask construction process includes: accumulating the binarization results of each two frames of difference, periodically binarizing the accumulated binarization results, updating the mask, and distinguishing whether it is caused by real foreground moving target or periodic background; Convert the video stream with the moving target into a video frame image, capture a video frame image at the same time interval, input the image into a YOLO network for local pedestrian detection, then input the recognition result and the coordinates of the slice of the pedestrian in the video frame image into a deepsort network for identification and tracking, and assign a number to the pedestrian, separate and store the video frame images of the same target recognized as pedestrians, establish a historical pedestrian sequence image library, and periodically delete redundant images in the sequence directory; Use a re-identification model to compare the features of the image sequence to be searched and the historical pedestrian sequence images, introduce a time and space correlation attention mechanism module, and construct a correlation result between each pair of nodes in the feature map, stack the correlation result into a feature matrix in the form of a pair relationship item, output the sequence with high similarity, and filter the time region of the sequence with high similarity to draw the target pedestrian's trajectory route according to the campus monitoring area map. The output of the re-identification model processing enters a post-processing module, which filters and limits the output of the re-identification model in time and space. 2.The mass video-based campus security cross-domain tracking method according to claim 1, wherein, Dividing the campus monitoring area includes: dividing the campus monitoring area into a plurality of sub-areas, sequentially numbering the sub-areas, sequentially corresponding the monitoring devices of each sub-area with the corresponding sub-areas, marking the direction of the monitoring devices, recording and saving the sub-area number and monitoring device number information, and drawing a campus monitoring area map after numbering. 3.The mass video-based campus security cross-domain tracking method according to claim 1, wherein, Connect the numbered monitoring devices with edge embedded devices with video hardware decoding, upload real-time monitoring video of the campus to the edge embedded devices for video decoding, set the video stream media address and the corresponding monitoring devices, and construct a mapping relationship. 4.The mass video-based campus security cross-domain tracking method according to claim 1, wherein, The local pedestrian detection includes: detecting pedestrians in each frame of the video frame image with a moving target, capturing an image at the same time interval for pedestrian detection, inputting the detection result and the coordinates of the pedestrian in the image frame into the deepsort network for identification and tracking, and assigning a number to the pedestrian of the same target until the target disappears in the monitoring area. 5.The mass video-based campus security cross-domain tracking method according to claim 1, wherein, The video frame pedestrian images identified as the same target are stored separately, and a historical pedestrian sequence picture library is established, including: constructing the picture library according to time, establishing a time directory according to the date of the day, and storing the trajectories of all monitoring devices on the day under the time directory; then establishing a camera directory under the time directory according to the number of the monitoring device, and storing the pedestrian trajectories appearing in the corresponding camera under the camera directory; and further establishing a trajectory directory under each camera directory, and saving the image frames of the same pedestrian number under the same camera in the same trajectory directory. 6.The mass video-based campus security cross-domain tracking method according to claim 1, wherein, The pedestrian sequence to be searched and all image sequences in the specified date directory are input into the PSTA model for re-identification, and the directory name of the sequence with higher similarity and the camera directory name of the upper level are output, forming an array output corresponding to the time period and the area of the pedestrian.

7. A campus security cross-domain tracking system based on mass videos, adopting the campus security cross-domain tracking method based on mass videos according to any one of claims 1-6, characterized in that, It comprises: A data acquisition module is configured to divide the campus monitoring area, acquire real-time monitoring videos of the campus, and upload the monitoring videos to an edge embedded device for video stream decoding. A motion video detection module is configured to preprocess the decoded video stream and acquire video stream with motion targets. A local motion target monitoring and tracking module is configured to convert the video stream with motion targets into video frame images, capture a video frame image at the same time interval, input the video frame image into a YOLO network for local pedestrian detection, then input the detection result and the coordinates of the slice of the pedestrian in the video frame image into a deepsort network for discrimination and tracking, and assign a number to the pedestrian, and store the video frame pedestrian images identified as the same target separately to establish a historical pedestrian sequence picture library. A re-identification module is configured to compare the image sequence of the pedestrian to be searched and the historical pedestrian sequence picture by using a re-identification model, introduce a time and space correlation attention mechanism module, construct a correlation result between each pair of nodes in a feature map, stack the correlation result into a feature matrix in the form of a pair relationship item, output the sequence with high similarity, and perform time and area screening on the sequence with high similarity to draw a trajectory route of the target pedestrian according to a campus monitoring area map.

8. A non-transitory computer-readable storage medium, comprising: The non-transitory computer readable storage medium is configured to store computer instructions, and the computer instructions are executed by a processor to implement the campus safety cross-domain tracking method based on massive videos according to any one of claims 1-6.

9. An electronic device, comprising: It comprises: A processor, a memory, and a computer program; the processor is connected with the memory, and the computer program is stored in the memory; when the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to implement the campus safety cross-domain tracking method based on massive videos according to any one of claims 1-6.

Images