Rail transit boundary intrusion scene personnel cross-mirror tracking and early warning method and system

Abstract

The application provides a rail transit boundary intrusion scene personnel cross-mirror tracking and early warning method and system, and belongs to the technical field of safe operation of rail transit. First, a target image of an intruding personnel in a concerned area is obtained based on a rail transit perimeter image. Then, a rail transit boundary intrusion scene personnel cross-mirror tracking and identification algorithm is used to track and identify the illegal intruding personnel across the monitoring area. Finally, the harm degree and walking direction of the illegal intruding personnel are judged and early warning is performed. Compared with other methods, the rail transit boundary intrusion scene personnel cross-mirror tracking and identification algorithm is proposed, the intruding personnel categories are distinguished according to the rail transit boundary intrusion demand area, only the illegal intruding personnel is tracked, the harmfulness of the intruding personnel is judged, the walking direction is predicted, and early warning is performed. The calculation resource consumption of the system is reduced, the cross-mirror tracking accuracy is improved, and the application demand of rail transit boundary intrusion detection is more suitable.

Description

Technical Field

[0001] The present invention relates to the technical field of safe operation of rail transit, and in particular to a video-based cross-camera tracking and early warning method and system for personnel intrusion scenarios in rail transit. Background Art

[0002] With increasing train speeds and the expansion of the railway network, operational safety has become increasingly important. Railway lines are long and the security situation along them complex. Relying solely on human defense cannot meet the basic needs of intrusion detection. If a person intrudes within the rail transit perimeter, it could seriously endanger operational safety and impact the operational efficiency of the entire railway network. Improving the ability to detect and address potential safety hazards along the railway line is crucial for its safe operation. Currently, the primary method for identifying suspicious individuals is manual screening, which is time-consuming and labor-intensive, and ineffective in effectively tracking suspicious individuals, identifying terrorists, and resolving potential risks. Traditional algorithms for detecting perimeter intrusions based on video images are easily affected by factors such as lighting, and are unable to implement cross-camera tracking of suspicious individuals. Cross-camera target tracking meets the security needs of railway perimeters, providing alerts and tracking for suspicious individuals who repeatedly enter the perimeter.

[0003] Rail transit video surveillance is an important way to detect illegal intrusions within rail transit perimeters. The problem of cross-regional target tracking was first proposed by Cai Q et al. at the ICPR conference in 1996. They applied a multivariate Gaussian model to find the most likely matching human objects between frames captured by multiple cameras. In 2006, N. Gheissari et al. first proposed the concept of pedestrian re-identification at CVPR, using a spatiotemporal segmentation algorithm for foreground detection and combining normalized color and salient edge histograms to generate invariant labels. Since then, research on cross-border targets has continued to develop. Cross-camera tracking can be divided into traditional cross-camera target tracking methods based on manual features and cross-camera tracking algorithms based on deep learning.

[0004] Cross-region object tracking methods based on handcrafted features require defining a feature description method before extracting image features. Image similarity ranking is then determined using a distance metric. Handcrafted features commonly used for image description include color features (such as RGB, LAB, and HSV) and texture features (such as LBP and SIFT), which are represented in a matrix format. Common distance metrics include Euclidean distance, Mahalanobis distance, and cosine distance.

[0005] Deep learning-based person re-identification methods can automatically extract superior pedestrian image features and achieve superior similarity metrics. Depending on the type of loss, they can be categorized as representation-based and metric-based. After global features from single-frame images encountered performance bottlenecks, researchers have introduced local and sequence features to further advance person re-identification research.

[0006] Therefore, how to apply existing advanced technologies to the problem of rail transit intrusion, meet the security maintenance needs of rail transit perimeters, and realize the functions of alarming and tracking suspicious persons entering the perimeter is still a problem to be solved. Summary of the Invention

[0007] The purpose of the present invention is to provide a method and system for cross-camera tracking and early warning of personnel in rail transit intrusion scenarios, so as to solve at least one technical problem existing in the above-mentioned background technology.

[0008] In order to achieve the above object, the present invention adopts the following technical solutions:

[0009] In one aspect, the present invention provides a method for cross-camera tracking and early warning of personnel in rail transit boundary intrusion scenarios, comprising:

[0010] Obtain video images of each section of the rail transit perimeter and extract the time and monitoring geographic location information of the video;

[0011] The improved YOLOv7 algorithm is used to perform real-time intruder target detection on the focus area of ​​the video image. If an intruder target is detected, a detection information box is marked on the video image for each intruder target, and an image of each intruder target with the information box as the boundary is output. The improved YOLOv7 algorithm is different from other YOLOv7 algorithms in terms of training set, and uses the collected rail transit pedestrian intrusion dataset.

[0012] Generate the monitoring geographic location tag and intrusion time tag of the intrusion target image based on the extracted video information;

[0013] Use the rail transit perimeter staff feature classification detector to detect and identify local features of personnel, and distinguish between normal personnel and illegal intruders;

[0014] Using the cross-camera tracking and identification algorithm for rail transit intrusion scenarios, we can track and identify illegal intruders across monitoring areas, and confirm the movement routes of the same pedestrian in surveillance video images collected in different areas.

[0015] Based on the results of the cross-camera tracking and recognition algorithm for people in rail transit intrusion scenarios, the degree of harm caused by illegal intruders and the walking direction warning are judged.

[0016] Preferably, a cross-camera tracking and identification algorithm for rail transit intrusion scenes is used to track and identify illegal intruders across monitoring areas, and the movement route of the same pedestrian is confirmed in the surveillance video images collected in different areas, including:

[0017] If the pedestrian is illegally intruding, the cross-camera tracking and recognition algorithm for rail transit intrusion scenes is used to extract the feature values ​​of the intruder dataset image;

[0018] The image features of the illegally intruding pedestrian to be retrieved are used as the benchmark, and the other images in the dataset are used as the candidate set to calculate the cosine distance with the benchmark. When the feature distance is less than the set threshold, there is a similar target, and the feature distance is sorted. The top ten images are selected as the output result;

[0019] The cosine feature distance between the images is used to determine whether two pedestrian images are the same target, and to determine whether the illegally intruding pedestrian target appears in different monitoring areas, or whether it has invaded the railway perimeter in the past.

[0020] Preferably, according to the results of the cross-camera tracking and recognition algorithm for the person intruding in the rail transit boundary scene, the degree of harm caused by the illegal intruder and the warning of the walking direction are judged, including:

[0021] Based on the tracking results and the corresponding image's monitoring geographic location tag and intrusion time tag, the illegal intruder is tracked and alarmed. If there is no similar matching target, the target's first intrusion alarm is issued;

[0022] When there are similar targets and the intrusion interval is less than the set time threshold, the rail transit intrusion scene personnel warning method is used to predict the illegal intruder's travel trajectory. Combined with the monitoring network topology, the next intrusion area of ​​the target is predicted and an early warning is issued;

[0023] Otherwise, if the illegal intruder invades the perimeter for multiple periods of time, a multiple intrusion alarm will be issued and the intrusion target status and time and space information will be output.

[0024] Preferably, each intrusion target is marked with a detection information frame, and each intrusion target image with the information frame as the boundary is intercepted and scaled, and the target image is detected and expanded after the point feature values ​​are expanded:

[0025] f(x, y) = W × F,

[0026] Where W = w(i) × w(j), i and j represent the horizontal and vertical distances of the point relative to the feature point, respectively.

[0027] F represents the eigenvalue matrix of the 4×4 range around the point,

[0028] The expanded image undergoes four convolution operations with a kernel size of 5*5 and a stride of 1, and finally outputs the intrusion target image.

[0029] Preferably, the intrusion target image is normalized by BN after passing through a 6*6 convolution kernel, activated by a ReLU activation function, and then subjected to a maximum pooling of 3*3 with a step size of 2 to complete the initialization of the image features; the convolution block is composed of 1*1, 3*1, 1*3 and 1*1 convolution kernels, and the network passes through 4 convolution units, each of which has 2, 3, 5 and 2 convolution blocks respectively; wherein, the input and output of the previous convolution block are connected through a residual connection as the input of the next convolution block; the output of the first convolution unit is added to the SE channel attention mechanism operation, and then input to the second convolution unit operation; the difference between each convolution unit is the number of channels, the first convolution unit has 64 channels, the second convolution unit has 128 channels, the third convolution unit has 256 channels, and the fourth convolution unit has 1024 channels.

[0030] Preferably, trajectory prediction uses LSTM as the backbone network, and the hidden state output is:

[0031]

[0032] Where σ represents the sigmoid function, z t represents the time series vector of coordinates, f t represents the forget gate, C t-1 Indicates the upper state, i t represents the update gate, W represents the weight parameter, and b represents the bias parameter;

[0033] s t =relu(W s ·H s +b s )

[0034] Among them H s Indicates the total number of other intruders within the range of the predicted point u*v;

[0035] h' t-1 =h t-1 +W p H p

[0036] Among them H p Represents the hidden state of the past p LSTM units.

[0037] In a second aspect, the present invention provides a cross-camera tracking and early warning system for personnel intrusion in rail transit scenarios, comprising:

[0038] An acquisition module is configured to acquire video images of each section of the rail transit perimeter and extract time and monitoring geographic position information of the video;

[0039] An extraction module is configured to use an improved YOLOv7 algorithm to perform real-time intrusion personnel target detection on a part of the attention region in the video image, label a detection information box on each intrusion target on the video image if an intrusion target is detected, and output an image of each intrusion target with the information box as a boundary; wherein the improved YOLOv7 algorithm is distinguished from other YOLOv7 algorithms on a training set, and uses a collected rail transit boundary intrusion pedestrian data set;

[0040] A labeling module is configured to generate a monitoring geographic position label and an intrusion time label of the image of the intrusion target according to the extracted video information;

[0041] A detection module is configured to use a rail transit perimeter staff feature classification detector to detect and identify the local features of the personnel, and distinguish between normal staff and illegal intrusion personnel;

[0042] A tracking module is configured to use a rail transit boundary intrusion scene personnel cross-mirror tracking and identification algorithm to track and identify the illegal intrusion personnel across the monitoring areas, and confirm the action route of the same pedestrian in the monitoring video collection images in different areas;

[0043] A warning module is configured to determine the degree of harm and walking direction warning of the illegal intrusion personnel according to the result of the rail transit boundary intrusion scene personnel cross-mirror tracking and identification algorithm.

[0044] In a third aspect, the present application provides a non-transitory computer readable storage medium for storing computer instructions, which, when executed by a processor, implement the rail transit boundary intrusion scene personnel cross-mirror tracking and warning method as described above.

[0045] In a fourth aspect, the present application provides a computer program product comprising a computer program, which, when running on one or more processors, is used to implement the rail transit boundary intrusion scene personnel cross-mirror tracking and warning method as described above.

[0046] In a fifth aspect, the present application provides an electronic device comprising a processor, a memory and a computer program; wherein the processor is connected to the memory, and the computer program is stored in the memory; when the electronic device is running, the processor executes the computer program stored in the memory, so that the electronic device executes the instructions for implementing the rail transit boundary intrusion scene personnel cross-mirror tracking and warning method as described above.

[0047] The beneficial effects of the present invention are as follows: in response to the problem of personnel intrusion at the perimeter of rail transit, combined with the actual application scenarios of rail transit intrusion, a cross-camera target tracking model based on deep learning is constructed, different personnel categories are proposed according to the requirements of rail transit intrusion scenarios, and only illegal intruders are tracked; the system's computing resource consumption is reduced, the accuracy of re-identification is improved, and it is more suitable for on-site application needs; according to the cross-camera tracking results, the harm of intruders to the safety of rail transit operations is judged, graded warnings are issued, and the areas that intruders in multiple areas are about to invade are predicted, so that the intrusion behavior is analyzed and judged in advance and corresponding measures are taken.

[0048] Additional advantages of the present invention will be more clearly given in the following description or learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0050] Figure 1 This is a flow chart of the video-based cross-camera tracking and early warning method for personnel in rail transit intrusion scenarios described in the present invention.

[0051] Figure 2 This is a diagram showing the output results of rail transit intrusion target detection according to an embodiment of the present invention.

[0052] Figure 3 This is a schematic diagram of the visualization results of the cross-camera tracking and recognition algorithm for people intrusion scenarios in rail transit according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and are not to be construed as limiting the present invention.

[0054] Those skilled in the art will understand that unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this invention belongs.

[0055] It should also be understood that terms, such as those defined in commonly used dictionaries, should be understood to have a meaning consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless as defined herein.

[0056] Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a," "an," "said," and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the stated features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, and / or groups thereof.

[0057] In the description of this specification, reference to the terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art may combine and integrate different embodiments or examples described in this specification, as well as features of different embodiments or examples, unless otherwise inconsistent.

[0058] To facilitate understanding of the present invention, the present invention is further explained below with reference to specific embodiments in conjunction with the accompanying drawings, and the specific embodiments do not constitute a limitation on the embodiments of the present invention.

[0059] Those skilled in the art should understand that the drawings are merely schematic diagrams of embodiments, and the components in the drawings are not necessarily necessary for implementing the present invention.

[0060] Example 1

[0061] In this embodiment 1, a cross-camera tracking and early warning system for people intruding in rail transit scenarios is provided, comprising:

[0062] The acquisition module is used to obtain video images of each section of the rail transit perimeter and extract the time and monitoring geographical location information of the video;

[0063] The extraction module uses the improved YOLOv7 algorithm to perform real-time intruder target detection on the focus area of ​​the video image. If an intruder target is detected, a detection information box is marked on the video image for each intruder target, and an image of each intruder target with the information box as the boundary is output. The improved YOLOv7 algorithm is different from other YOLOv7 algorithms in terms of training set, and uses the collected rail transit pedestrian intrusion dataset.

[0064] A labeling module is used to generate a monitoring geographic location tag and an intrusion time tag of the intrusion target image based on the extracted video information;

[0065] The detection module is configured to detect and identify the local features of the personnel by using the rail transit perimeter staff feature classification detector, and distinguish the normal staff from the illegal intruding personnel.

[0066] The tracking module is configured to track and identify the illegal intruding personnel across the monitoring areas by using the rail transit intrusion scene personnel cross-camera tracking and identification algorithm, and confirm the action route of the same person in the monitoring video collection images in different areas.

[0067] The warning module is configured to determine the harm degree and walking direction warning of the illegal intruding personnel according to the result of the rail transit intrusion scene personnel cross-camera tracking and identification algorithm.

[0068] In the embodiment 1, the system is used to realize a rail transit intrusion scene personnel cross-camera tracking and warning method, which includes the following steps:

[0069] The video images of each section of the rail transit perimeter are acquired, and the time and monitoring geographic location information of the video are extracted;

[0070] The improved yolov7 algorithm is used to detect the intruding personnel targets in the concerned area of the video images in real time, and if the intruding targets are detected, the detection information box is marked on each intruding target in the video images, and the image of each intruding target with the information box as the boundary is output; wherein, the improved YOLOv7 algorithm is different from other YOLOv7 algorithms on the training set, and uses the collected rail transit intrusion pedestrian data set;

[0071] According to the extracted video information, the monitoring geographic location label and the intrusion time label of the image of the intruding target are generated;

[0072] The detection module is configured to detect and identify the local features of the personnel by using the rail transit perimeter staff feature classification detector, and distinguish the normal staff from the illegal intruding personnel;

[0073] The tracking module is configured to track and identify the illegal intruding personnel across the monitoring areas by using the rail transit intrusion scene personnel cross-camera tracking and identification algorithm, and confirm the action route of the same person in the monitoring video collection images in different areas.

[0074] According to the result of the rail transit intrusion scene personnel cross-camera tracking and identification algorithm, the harm degree and walking direction warning of the illegal intruding personnel are determined.

[0075] The tracking module is configured to track and identify the illegal intruding personnel across the monitoring areas by using the rail transit intrusion scene personnel cross-camera tracking and identification algorithm, and confirm the action route of the same person in the monitoring video collection images in different areas.

[0076] If it is an illegal intruding pedestrian, the track crossing mirror tracking and recognition algorithm is used to extract the feature value of the intruding pedestrian data set image;

[0077] Taking the image feature of the illegal intruding pedestrian to be searched as a reference, the cosine distance between the reference and other images in the data set is calculated, and when the feature distance is less than the set threshold 0.5, there is a similar target, and the feature distance is sorted, and the top ten images are selected as the output result;

[0078] The cosine feature distance between the two images is used to determine whether the two images of the pedestrian are the same target, and whether the illegal intruding pedestrian target appears in different monitoring areas or whether it has intruded the railway perimeter in the past time.

[0079] According to the results of the track crossing mirror tracking and recognition algorithm, the harm degree and walking direction warning of the illegal intruding personnel are judged, including:

[0080] Based on the tracking results and the corresponding image monitoring geographic location label and intrusion time label, the illegal intruding personnel are tracked and alarmed, and when there is no similar matching target, the target first intrusion alarm is given;

[0081] When there is a similar target and the intrusion interval is less than the set time threshold 10 min, the track crossing mirror tracking and recognition algorithm is used to predict the illegal intruding personnel track, combine the monitoring network topology, predict the next intrusion area of the target, and give an early warning;

[0082] Otherwise, the illegal intruding personnel intrude the perimeter for multiple periods, and the target multiple intrusion alarm is given, and the intrusion target state and time space information are output.

[0083] Preferably, each intrusion target is labeled with detection information box, and each intrusion target image with the information box as the boundary is intercepted, scaled, and expanded in size after detecting the target image expansion point feature value:

[0084] f(x, y) = W x F,

[0085] Where W = w(i) x w(j), i and j represent the horizontal and vertical distances of the point relative to the feature point,

[0086] F represents the feature value matrix of the 4x4 range around the point,

[0087] The expanded image is subjected to four convolution kernel operations with a size of 5*5 and a step of 1, and finally the intrusion target image is output.

[0088] The intrusion target image is normalized after passing through a 6*6 convolution kernel, activated by the ReLU activation function, and then subjected to maximum pooling of 3*3 with a stride of 2 to complete the initialization of the image features; the convolution block is composed of 1*1, 3*1, 1*3 and 1*1 convolution kernels, and the network passes through 4 convolution units, each of which has 2, 3, 5 and 2 convolution blocks respectively; among them, the input and output of the previous convolution block are connected through residual connections as the input of the next convolution block; the output of the first convolution unit is added to the SE channel attention mechanism operation and then input into the second convolution unit operation; the difference between each convolution unit is the number of channels, the first convolution unit has 64 channels, the second convolution unit has 128 channels, the third convolution unit has 256 channels, and the fourth convolution unit has 1024 channels.

[0089] Preferably, trajectory prediction uses LSTM as the backbone network, and the hidden state output is:

[0090]

[0091] Where σ represents the sigmoid function, z t represents the time series vector of coordinates, f t represents the forget gate, C t-1 Indicates the upper state, i t represents the update gate, W represents the weight parameter, and b represents the bias parameter;

[0092] s t =relu(W s ·H s +b s )

[0093] Among them H s Indicates the total number of other intruders within the range of the predicted point u*v;

[0094] h' t-1 =h t-1 +W p H p

[0095] Among them H p Represents the hidden state of the past p LSTM units.

[0096] Example 2

[0097] like Figure 1 As shown, in this embodiment 2, a method for cross-camera tracking and early warning of personnel in track intrusion scenarios based on video is provided, comprising the following steps:

[0098] Step 1: Obtain video images of each section of the rail transit perimeter and extract basic information such as the time and monitoring location of the video;

[0099] Step 2: Use the improved YOLOv7 algorithm to perform real-time intruder target detection on the area of ​​interest in the video image. Once an intruder target is detected, a detection information box is marked on the video image for each intruder target, and an image of each intruder target with the information box as the boundary is output;

[0100] Step 3: Generate the monitoring geographic location tag and intrusion time tag of the intrusion target image based on the extracted video information;

[0101] Step 4: The rail transit perimeter worker feature classification detector detects and identifies local features such as safety clothing, helmets, or railway work clothes, and distinguishes between normal workers and illegal intruders. Normal workers are legally on the track and will not trigger an intrusion alarm.

[0102] Step 5: Use the cross-camera tracking and identification algorithm for rail transit intrusion scenarios to track and identify illegal intruders across monitoring areas, and confirm the movement routes of the same pedestrian in the surveillance video images collected in different areas;

[0103] The specific classification method of step 5 mainly includes the following steps:

[0104] Step 5.1: If the pedestrian is illegally intruding, use the cross-camera tracking and recognition algorithm for rail transit intrusion scenes to extract the feature values ​​of the intruder dataset image;

[0105] In step 5.2, the image features of the illegally intruding pedestrian to be retrieved are used as the benchmark, and the other images in the dataset are used as the candidate set to calculate the cosine distance with the benchmark. If the feature distance is less than the set threshold of 0.5, there is a similar target, and the feature distance is sorted. The top ten images are selected as the output results.

[0106] In step 5.3, the cosine feature distance between the images is used to determine whether the two pedestrian images are of the same target, whether the intruding pedestrian target appears in different monitoring areas, or whether it has invaded the railway perimeter in the past.

[0107] Step 6: Based on the results of the cross-camera tracking and recognition algorithm for people in the rail transit intrusion scene, the degree of harm caused by the illegal intruder and the walking direction warning are determined.

[0108] The step 6 mainly includes the following steps:

[0109] Step 6.1: Based on the tracking results of step 5 and the monitoring geographic location tag and intrusion time tag of the corresponding image, the illegal intruder is tracked and alarmed. If there is no similar matching target, the target is alarmed for the first intrusion;

[0110] Step 6.2: When there are similar targets and the intrusion interval is less than 10 minutes, the rail transit intrusion scenario personnel warning method is used to predict the illegal intruder's travel trajectory. Combined with the monitoring network topology, the next intrusion area of ​​the target is predicted and an early warning is issued.

[0111] Step 6.3: Otherwise, if the illegal intruder invades the perimeter in multiple time periods, a multiple intrusion alarm of the target is issued, and the intrusion target status and time and space information are output.

[0112] The improved YOLOv7 algorithm in step 1 is different from other YOLOv7 algorithms in terms of training set, and uses the collected rail transit intrusion pedestrian dataset.

[0113] Each intrusion target is marked with a detection information box, and each intrusion target image with the information box as the boundary is intercepted and scaled to 64*128 dimensions. The detection target image 100m away is expanded to 80*160 dimensions after the point feature values ​​are expanded: f(x, y) = W×F,

[0114] Where W = w(i) × w(j), i and j represent the horizontal and vertical distances of the point relative to the feature point, respectively.

[0115] F represents the eigenvalue matrix of the 4×4 range around the point,

[0116] The enlarged image undergoes four convolution operations with a kernel size of 5*5 and a stride of 1, and finally outputs a 64*128 intrusion target image.

[0117] The image is normalized by BN after passing through a 6*6 convolution kernel, activated by the ReLU activation function, and then subjected to maximum pooling of 3*3 with a step size of 2 to complete the initialization of the image features; the convolution block is composed of 1*1, 3*1, 1*3 and 1*1 convolution kernels, and the network passes through 4 convolution units, each of which has 2, 3, 5 and 2 convolution blocks respectively; the input and output of the previous convolution block are connected through residual connections as the input of the next convolution block; the output of the first convolution unit is added to the SE channel attention mechanism operation, and then input into the second convolution unit operation; the difference between each convolution unit is the number of channels, the first convolution unit has 64 channels, the second convolution unit has 128 channels, the third convolution unit has 256 channels, and the fourth convolution unit has 1024 channels.

[0118] Trajectory prediction uses LSTM as the backbone network, and the hidden state output

[0119] h t =σ(Wo·[h' t-1 ,z t ,s t ]+bo)*tanh(f t*C t-1 +i t *tanh(W c ·[h' t-1 ,z t ,s t ]+b c ) t )

[0120] Where σ represents the sigmoid function, z t represents the time series vector of coordinates, f t represents the forget gate, C t-1 Indicates the upper state, i t represents the update gate, W represents the weight parameter, and b represents the bias parameter;

[0121] s t =relu(W s ·H s +b s )

[0122] Among them H s Indicates the total number of other intruders within the range of the predicted point u*v;

[0123] h' t-1 =h t-1 +W p H p

[0124] Among them H p Represents the hidden state of the past p LSTM units.

[0125] In this embodiment 2, a video-based cross-camera tracking and early warning method for personnel in rail transit intrusion scenarios is applied to rail transit intrusion scenarios to solve the perimeter intrusion detection and tracking and hazard alarm prediction problems in rail transit scenarios.

[0126] Taking a rail transit perimeter scene as an example, a video-based rail transit intrusion scene cross-camera tracking and warning is carried out. Video images of each section of the rail transit perimeter are obtained, and the improved yolov7 algorithm is used to perform real-time intruder target detection on the focus area in the video image. The image of each intruder target with the information box as the boundary is output. The intruder target detection is as follows: Figure 2 shown.

[0127] The intrusion target image is annotated with the monitoring geographic location tag and intrusion time tag. The image naming method is as follows, taking 0001_c1_2021_0514_156720.jpg as an example:

[0128] 1) 0001 represents each person’s ID tag number;

[0129] 2) c1 represents the camera ID;

[0130] 3) 2021_0514_156720 indicates the time of intrusion detection.

[0131] The camera ID can be used to match the camera's corresponding geographic location information.

[0132] In this rail transit perimeter scenario, the experimental time is set to the skylight operation time specified in the railway bureau plan, and wearing yellow vests is used as work clothes on the road. At this time, it is reasonable for the staff to work on the road. The rail transit perimeter staff feature classification detector is used to distinguish the types of intruders. The classification results are as follows: Figure 2 As shown, only illegal intruders are tracked across the camera.

[0133] The cross-camera tracking and recognition algorithm for rail transit intrusion scenes is used to extract pedestrian features from the intrusion target image. The extracted feature vectors are shown in Table 2. The newly detected intrusion target image to be retrieved is used as the query image set, and the historical images in the stored dataset are used as the candidate set. The query set is used as the standard, and the cosine similarity metric is used to calculate the feature distance between the query set and the candidate set pedestrian images. When the feature distance is less than the threshold of 0.5, the similar target results are sorted by feature distance. The visualization results are shown in Figure 2. Figure 3 As shown in the figure, the same intrusion detection ID is assigned to the matched similar targets and the label value is modified.

[0134] Table 1 shows the accuracy comparison results of the patented cross-camera tracking and recognition algorithm for people intruding in rail transit scenarios and other algorithms.

[0135] Table 1 Comparison of method accuracy

[0136]

[0137] Table 2 Some feature extraction results

[0138]

[0139] Based on the tracking results of the cross-camera tracking and recognition algorithm for rail transit intrusion scenes and the monitoring geographic location tag and intrusion time tag of the corresponding image, illegal intruders are tracked and alarmed. When there is no similar matching target, the target's first intrusion alarm is issued, and the status and time-space information of the intrusion target are output;

[0140] When there are similar targets, the time difference between the intruder and the historical intrusion image is calculated based on the label attributes of the intruder (geographic location (camera ID), intrusion time). If the intrusion time interval is less than 10 minutes, the rail transit intrusion scene personnel warning method is used to predict the travel trajectory of the illegal intruder. Combined with the monitoring network topology and the physical location relationship of the camera, it is determined which areas the predicted trajectory will pass through, and the next intrusion area of ​​the target is predicted to issue an early warning. If the intrusion time interval is greater than 10 minutes, the harmfulness of the illegal intruder is determined as a multi-period intrusion perimeter, and multiple target intrusion alarms are issued, and the current status and time and space information of the intruder target are output.

[0141] Example 3

[0142] Embodiment 3 of the present invention provides a non-transitory computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, a method for cross-camera tracking and early warning of personnel in a rail transit intrusion scene is implemented. The method includes:

[0143] Obtain video images of each section of the rail transit perimeter and extract the time and monitoring geographic location information of the video;

[0144] The improved YOLOv7 algorithm is used to perform real-time intruder target detection on the focus area of ​​the video image. If an intruder target is detected, a detection information box is marked on the video image for each intruder target, and an image of each intruder target with the information box as the boundary is output. The improved YOLOv7 algorithm is different from other YOLOv7 algorithms in terms of training set, and uses the collected rail transit pedestrian intrusion dataset.

[0145] Generate the monitoring geographic location tag and intrusion time tag of the intrusion target image based on the extracted video information;

[0146] Use the rail transit perimeter staff feature classification detector to detect and identify local features of personnel, and distinguish between normal personnel and illegal intruders;

[0147] Using the cross-camera tracking and identification algorithm for rail transit intrusion scenarios, we can track and identify illegal intruders across monitoring areas, and confirm the movement routes of the same pedestrian in surveillance video images collected in different areas.

[0148] Based on the results of the cross-camera tracking and recognition algorithm for people in rail transit intrusion scenarios, the degree of harm caused by illegal intruders and the walking direction warning are judged.

[0149] Example 4

[0150] Embodiment 4 of the present invention provides a computer program (product), including a computer program, which, when executed on one or more processors, is used to implement a method for cross-camera tracking and early warning of personnel in rail transit intrusion scenarios, the method comprising:

[0151] Obtain video images of each section of the rail transit perimeter and extract the time and monitoring geographic location information of the video;

[0152] The improved YOLOv7 algorithm is used to perform real-time intruder target detection on the focus area of ​​the video image. If an intruder target is detected, a detection information box is marked on the video image for each intruder target, and an image of each intruder target with the information box as the boundary is output. The improved YOLOv7 algorithm is different from other YOLOv7 algorithms in terms of training set, and uses the collected rail transit pedestrian intrusion dataset.

[0153] Generate the monitoring geographic location tag and intrusion time tag of the intrusion target image based on the extracted video information;

[0154] Use the rail transit perimeter staff feature classification detector to detect and identify local features of personnel, and distinguish between normal personnel and illegal intruders;

[0155] Using the cross-camera tracking and identification algorithm for rail transit intrusion scenarios, we can track and identify illegal intruders across monitoring areas, and confirm the movement routes of the same pedestrian in surveillance video images collected in different areas.

[0156] Based on the results of the cross-camera tracking and recognition algorithm for people in rail transit intrusion scenarios, the degree of harm caused by illegal intruders and the walking direction warning are judged.

[0157] Example 5

[0158] Embodiment 5 of the present invention provides an electronic device, including: a processor, a memory, and a computer program; wherein the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory to cause the electronic device to execute instructions for implementing a method for cross-camera tracking and early warning of personnel in rail transit intrusion scenarios, the method comprising:

[0159] Obtain video images of each section of the rail transit perimeter and extract the time and monitoring geographic location information of the video;

[0160] The improved YOLOv7 algorithm is used to perform real-time intruder target detection on the focus area of ​​the video image. If an intruder target is detected, a detection information box is marked on the video image for each intruder target, and an image of each intruder target with the information box as the boundary is output. The improved YOLOv7 algorithm is different from other YOLOv7 algorithms in terms of training set, and uses the collected rail transit pedestrian intrusion dataset.

[0161] Generate the monitoring geographic location tag and intrusion time tag of the intrusion target image based on the extracted video information;

[0162] Use the rail transit perimeter staff feature classification detector to detect and identify local features of personnel, and distinguish between normal personnel and illegal intruders;

[0163] Using the cross-camera tracking and identification algorithm for rail transit intrusion scenarios, we can track and identify illegal intruders across monitoring areas, and confirm the movement routes of the same pedestrian in surveillance video images collected in different areas.

[0164] Based on the results of the cross-camera tracking and recognition algorithm for people in rail transit intrusion scenarios, the degree of harm caused by illegal intruders and the walking direction warning are judged.

[0165] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0166] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0167] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0168] These computer program instructions can also be loaded onto a computer or other programmable data processing device, and a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide the functions for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0169] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solutions disclosed in the present invention without the need for creative work should be included in the scope of protection of the present invention.

Claims

1. A cross-camera tracking and early warning method for personnel intrusion in rail transit, characterized by: include: Obtain video images of each section of the rail transit perimeter and extract the time and monitoring geographic location information of the video; The improved YOLOv7 algorithm is used to perform real-time intruder target detection on the focus area of ​​the video image. If an intruder target is detected, a detection information box is marked on the video image for each intruder target, and an image of each intruder target with the information box as the boundary is output. The improved YOLOv7 algorithm is different from other YOLOv7 algorithms in terms of training set, and uses the collected rail transit pedestrian intrusion dataset. Generate the monitoring geographic location tag and intrusion time tag of the intrusion target image based on the extracted video information; Use the rail transit perimeter staff feature classification detector to detect and identify local features of personnel, and distinguish between normal personnel and illegal intruders; Using the cross-camera tracking and identification algorithm for rail transit intrusion scenarios, we can track and identify illegal intruders across monitoring areas, and confirm the movement routes of the same pedestrian in surveillance video images collected in different areas. Based on the results of the cross-camera tracking and recognition algorithm for people in rail transit intrusion scenes, the severity of the damage caused by illegal intruders and the direction of their movement are judged and warned; Among them, the cross-camera tracking and identification algorithm for rail transit intrusion scenes is used to track and identify illegal intruders across monitoring areas, and the movement routes of the same pedestrians are confirmed in the surveillance video images collected in different areas, including: If the pedestrian is illegally intruding, the cross-camera tracking and recognition algorithm for rail transit intrusion scenes is used to extract the feature values ​​of the intruder dataset image; The image features of the illegally intruding pedestrian to be retrieved are used as the benchmark, and the other images in the dataset are used as the candidate set to calculate the cosine distance with the benchmark. When the feature distance is less than the set threshold, there is a similar target, and the feature distance is sorted. The top ten images are selected as the output result; The cosine feature distance between the two images is used to determine whether the two pedestrian images are the same target, and to determine whether the illegally intruding pedestrian target appears in different monitoring areas, or whether it has invaded the railway perimeter in the past; Each intrusion target is marked with a detection information box, and each intrusion target image with the information box as the boundary is intercepted and scaled. The detection target image is expanded after the point feature values ​​are expanded: f(x, y) = W × F, Where W = w(i) × w(j), i and j represent the horizontal and vertical distances of the point relative to the feature point, respectively. F represents the eigenvalue matrix of the 4×4 range around the point, The expanded image undergoes four convolution operations with a kernel size of 5*5 and a stride of 1, and finally outputs the intrusion target image.

2. The method for cross-border tracking and early warning of personnel in rail transit boundary intrusion scenarios according to claim 1 is characterized in that: Based on the results of the cross-camera tracking and recognition algorithm for people in rail transit intrusion scenarios, the system can determine the degree of harm caused by illegal intruders and their walking direction, including: Based on the tracking results and the corresponding image's monitoring geographic location tag and intrusion time tag, the illegal intruder is tracked and alarmed. If there is no similar matching target, the target's first intrusion alarm is issued; When there are similar targets and the intrusion interval is less than the set time threshold, the rail transit intrusion scene personnel warning method is used to predict the illegal intruder's travel trajectory. Combined with the monitoring network topology, the next intrusion area of ​​the target is predicted and an early warning is issued; Otherwise, if the illegal intruder invades the perimeter for multiple periods of time, a multiple intrusion alarm will be issued and the intrusion target status and time and space information will be output.

3. The method for cross-border tracking and early warning of personnel in rail transit boundary intrusion scenarios according to claim 2 is characterized in that: The intrusion target image is normalized after passing through a 6*6 convolution kernel, activated by the ReLU activation function, and then subjected to maximum pooling of 3*3 with a stride of 2 to complete the initialization of the image features; the convolution block is composed of 1*1, 3*1, 1*3 and 1*1 convolution kernels, and the network passes through 4 convolution units, each of which has 2, 3, 5 and 2 convolution blocks respectively; among them, the input and output of the previous convolution block are connected through residual connections as the input of the next convolution block; the output of the first convolution unit is added to the SE channel attention mechanism operation and then input into the second convolution unit operation; the difference between each convolution unit is the number of channels, the first convolution unit has 64 channels, the second convolution unit has 128 channels, the third convolution unit has 256 channels, and the fourth convolution unit has 1024 channels.

4. The method for cross-border tracking and early warning of personnel in rail transit boundary intrusion scenarios according to claim 3 is characterized in that: Trajectory prediction uses LSTM as the backbone network, and the hidden layer state output is: h t =σ(W o ·[h' t-1 ,z t ,s t ]+b o )*tanh(f t *C t-1 +i t *tanh(W c ·[h' t-1 ,z t ,s t ]+b c ) t ) Where σ represents the sigmoid function, z t Represents the time series vector of coordinates, f t represents the forget gate, C t-1 Indicates the upper state, i t represents the update gate, W represents the weight parameter, and b represents the bias parameter; s t =relu(W s ·H s +b s ) Among them H s Indicates the total number of other intruders within the range of the predicted point u*v; h' t-1 =h t-1 +W p H p Among them H p Represents the hidden state of the past p LSTM units.

5. A cross-camera tracking and warning system for people intruding in rail transit, characterized by: include: The acquisition module is used to obtain video images of each section of the rail transit perimeter and extract the time and monitoring geographical location information of the video; The extraction module uses the improved YOLOv7 algorithm to perform real-time intruder target detection on the focus area of ​​the video image. If an intruder target is detected, a detection information box is marked on the video image for each intruder target, and an image of each intruder target with the information box as the boundary is output. The improved YOLOv7 algorithm is different from other YOLOv7 algorithms in terms of training set, and uses the collected rail transit pedestrian intrusion dataset. A labeling module is used to generate a monitoring geographic location tag and an intrusion time tag of the intrusion target image based on the extracted video information; The detection module is used to detect and identify local features of personnel using a rail transit perimeter personnel feature classification detector to distinguish between normal personnel and illegal intruders; The tracking module is used to track and identify illegal intruders across monitoring areas using a cross-camera tracking and identification algorithm for rail transit intrusion scenarios, and to confirm the movement routes of the same pedestrian in surveillance video images collected in different areas; The early warning module is used to determine the degree of harm caused by illegal intruders and their walking direction based on the results of the cross-camera tracking and recognition algorithm for rail transit intrusion scenes; Among them, the cross-camera tracking and identification algorithm for rail transit intrusion scenes is used to track and identify illegal intruders across monitoring areas, and the movement routes of the same pedestrians are confirmed in the surveillance video images collected in different areas, including: If the pedestrian is illegally intruding, the cross-camera tracking and recognition algorithm for rail transit intrusion scenes is used to extract the feature values ​​of the intruder dataset image; The image features of the illegally intruding pedestrian to be retrieved are used as the benchmark, and the other images in the dataset are used as the candidate set to calculate the cosine distance with the benchmark. When the feature distance is less than the set threshold, there is a similar target, and the feature distance is sorted. The top ten images are selected as the output result; The cosine feature distance between the two images is used to determine whether the two pedestrian images are the same target, and to determine whether the illegally intruding pedestrian target appears in different monitoring areas, or whether it has invaded the railway perimeter in the past; Each intrusion target is marked with a detection information box, and each intrusion target image with the information box as the boundary is intercepted and scaled. The detection target image is expanded after the point feature values ​​are expanded: f(x, y) = W × F, Where W = w(i) × w(j), i and j represent the horizontal and vertical distances of the point relative to the feature point, respectively. F represents the eigenvalue matrix of the 4×4 range around the point, The expanded image undergoes four convolution operations with a kernel size of 5*5 and a stride of 1, and finally outputs the intrusion target image.

6. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by the processor, the method for cross-camera tracking and early warning of personnel in rail transit intrusion scenarios as described in any one of claims 1 to 4 is implemented.

7. A computer program product, characterized in that It includes a computer program, which, when running on one or more processors, is used to implement the cross-camera tracking and early warning method for personnel in rail transit intrusion scenarios as described in any one of claims 1 to 4.

8. An electronic device, characterized in that: include: A processor, a memory and a computer program; wherein the processor is connected to the memory, and the computer program is stored in the memory. When the electronic device is running, the processor executes the computer program stored in the memory to enable the electronic device to execute instructions for implementing the cross-mirror tracking and early warning method for personnel in rail transit intrusion scenarios as described in any one of claims 1 to 4.

Images