Railway intrusion pedestrian detection method, system, device, medium and program

Abstract

The invention relates to a railway intrusion pedestrian detection method, system, device, medium and program, and the method comprises the steps: obtaining a detection frame in a railway track pedestrian target frame image and a prediction frame in a cross-frame prediction result of the target frame image, and carrying out the confidence interval filtering of the detection frame in the target frame image, and performing association matching on the first detection result and a prediction frame in the cross-frame prediction result to obtain a first matching result, obtaining a tracking result of a detection frame corresponding to the first detection result according to the first matching result, predicting a next frame track of the target frame in the tracking result, screening out a target track meeting a preset condition, and outputting the target track. And performing association matching on the detection frame in the second detection result and the tracking result of the target trajectory to obtain a second matching result, and performing reset filtering on the confidence coefficient of the second detection result according to the second matching result to obtain a final detection result. According to the invention, the railway intrusion pedestrian detection efficiency and the accuracy of pedestrian intrusion detection are improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, system, device, medium and program for detecting pedestrian intrusion on a railway. Background Art

[0002] Safe operation is the first goal of rail transit. Foreign objects intruding into railway limits is the main factor causing frequent railway accidents, which poses a great threat to operational safety. Among them, illegal pedestrians on the road is still the main cause of casualties in railway accidents. Illegal pedestrians on the road seriously threaten railway transportation safety and also cause huge losses to the national economy and people's property safety.

[0003] At present, there are two types of detection methods for foreign objects in railway clearances: contact and non-contact. The contact detection methods that are widely used include dual-grid monitoring technology and fiber grating technology. Among them, the protection net technology determines whether there is a foreign object by detecting whether the protection net is impacted. It can accurately detect sudden foreign objects such as falling rocks that fall on the protection net, but it cannot judge other intruding foreign objects. At the same time, the large-scale installation of the protection net requires regular maintenance and cannot meet the detection of diversified foreign object intrusion clearance situations on the railway. Non-contact detection methods include infrared, millimeter wave radar, laser and video analysis. Among them, the intrusion target recognition detection method based on video analysis can obtain relatively intuitive detection results, so it is widely used by the railway safety system.

[0004] However, the existing technology has the problems of low efficiency in detecting pedestrian intrusion on railways and low accuracy in detecting pedestrian intrusion. Summary of the invention

[0005] The present application provides a railway intrusion pedestrian detection method, system, device, medium and program to solve the problems of low railway intrusion pedestrian detection efficiency and low accuracy of detecting pedestrian intrusion.

[0006] In a first aspect, the present application provides a method for detecting pedestrian intrusion on a railway, comprising:

[0007] Obtaining a detection frame in a target frame image of a railway track pedestrian video sequence and a prediction frame in a cross-frame prediction result of the target frame image;

[0008] Using a single-frame target detection algorithm to perform confidence interval filtering on the detection frame in the target frame image to obtain a first detection result and a second detection result;

[0009] Associatively matching the first detection result with the prediction box in the cross-frame prediction result to obtain a first matching result;

[0010] Obtain the tracking result of the detection box corresponding to the first detection result according to the first matching result;

[0011] Use a preset linear Kalman model to predict the trajectory of the next frame of the target frame in the tracking result, and filter out the target trajectories that meet the preset conditions;

[0012] Associate and match the detection boxes in the second detection result with the tracking results of the target trajectories to obtain a second matching result;

[0013] Reset and filter the confidence level of the second detection result according to the second matching result to obtain the final detection result.

[0014] In some embodiments, the confidence interval filtering of the detection boxes in the target frame image by using a single-frame target detection algorithm to obtain the first detection result and the second detection result includes:

[0015] Perform normalization processing on the target frame image to obtain a normalized image;

[0016] Use a preset YOLO model to calculate the confidence level of the detection boxes in the normalized image to obtain an initial detection result;

[0017] Select the detection boxes with a confidence level greater than the upper limit of the preset confidence level threshold interval from the initial detection result as the first detection result;

[0018] Select the detection boxes with a confidence level less than the lower limit of the preset confidence level threshold interval from the initial detection result as the second detection result.

[0019] In some embodiments, the associating and matching the first detection result with the prediction boxes in the cross-frame prediction result to obtain a first matching result includes:

[0020] Calculate the first IOU value between several detection boxes in the first detection result and the prediction boxes in the cross-frame prediction result one by one;

[0021] Calculate the first association loss value between the first detection result and the cross-frame prediction result one by one according to the first IOU value;

[0022] Use the first association loss value to construct a first matrix;

[0023] Remove the minimum value of each row in the first matrix to obtain a first standard matrix;

[0024] Match the first standard matrix with the prediction boxes in the cross-frame prediction result:

[0025] When the first association loss value in the first standard matrix is less than a preset association loss threshold, it is determined that the obtained first matching result is a successful match between the prediction box in the first detection result and the cross-frame prediction result;

[0026] When the first association loss value in the first standard matrix is greater than or equal to the association loss threshold, it is determined that the obtained first matching result is a failed match between the prediction box in the first detection result and the cross-frame prediction result.

[0027] In some embodiments, obtaining the tracking result of the detection box corresponding to the first detection result according to the first matching result includes:

[0028] If the first matching result is a successful match between the prediction box in the first detection result and the cross-frame prediction result, the preset single-object tracking model parameters are updated using the first detection result;

[0029] The first detection result is used as the tracking result of the single-object tracking model in the target frame;

[0030] If the first matching result is a failed match between the prediction box in the first detection result and the cross-frame prediction result, the tracking result of the target frame is generated using the target frame image and the iterative parameters of the single-object tracking model.

[0031] In some embodiments, predicting the trajectory of the next frame of the target frame in the tracking result using a preset linear Kalman model and screening out the target trajectories that meet the preset conditions includes:

[0032] Calculate the state prediction matrix and covariance matrix of the trajectory of the next frame of the target frame in the tracking result;

[0033] Calculate the prediction result of the trajectory of the next frame of the target frame in the tracking result according to the state prediction matrix and the covariance matrix;

[0034] Calculate the squared value of the Mahalanobis distance between the tracking result and the prediction result;

[0035] If the squared value of the Mahalanobis distance is greater than a preset squared Mahalanobis distance threshold, the linear Kalman model does not update the state parameters;

[0036] If the squared value of the Mahalanobis distance is less than or equal to the squared Mahalanobis distance threshold, calculate the Kalman gain coefficient of the trajectory of the next frame of the target frame in the tracking result;

[0037] Update the state parameters of the linear Kalman model using the tracking result and the Kalman gain coefficient;

[0038] Count the number of matching failures and the number of trajectory anomalies of the next frame trajectory of the target frame in the state parameters of the linear Kalman model;

[0039] Filter out the target trajectories that meet the preset conditions according to the number of matching failures and the number of trajectory anomalies.

[0040] In some embodiments, the resetting and filtering the confidence of the second detection result according to the second matching result to obtain the final detection result includes:

[0041] If the second detection result fails to match the tracking result of the target trajectory, update the confidence of the second detection result using a preset reset constant;

[0042] If the second detection result matches the tracking result of the target trajectory, update the confidence of the second detection result using the second association loss value;

[0043] If the confidence of the updated second detection result is less than or equal to the upper limit of the confidence threshold interval, delete the detection box in the second detection result;

[0044] If the confidence of the updated second detection result is greater than the upper limit of the confidence threshold interval, obtain the final detection result.

[0045] In a second aspect, the present application provides a railway intrusion pedestrian detection system, including:

[0046] A data acquisition module for acquiring the detection box in the target frame image of the railway track pedestrian video sequence and the prediction box in the cross-frame prediction result of the target frame image;

[0047] An image filtering module for filtering the confidence interval of the detection box in the target frame image using a single-frame target detection algorithm to obtain a first detection result and a second detection result;

[0048] A first association matching module for associating and matching the first detection result with the prediction box in the cross-frame prediction result to obtain a first matching result;

[0049] A tracking result acquisition module for obtaining the tracking result of the detection box corresponding to the first detection result according to the first matching result;

[0050] A trajectory screening module for predicting the next frame trajectory of the target frame in the tracking result using a preset linear Kalman model and screening out the target trajectories that meet the preset conditions;

[0051] A second correlation matching module, configured to perform correlation matching between the detection boxes in the second detection result and the tracking result of the target trajectory to obtain a second matching result;

[0052] A reset filtering module, configured to perform reset filtering on the confidence level of the second detection result according to the second matching result to obtain a final detection result.

[0053] In a third aspect, the present application provides a computer device, including a memory, a processor, and a computer program stored on the memory, where the processor executes the computer program to implement the steps of the method described in the above aspect.

[0054] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the method described in the above aspect.

[0055] In a fifth aspect, the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the steps of the method described in the above aspect.

[0056] A railway intrusion pedestrian detection method, system, device, medium, and program provided by the present application obtain detection boxes in a target frame image of a railway track pedestrian video sequence and prediction boxes in a cross-frame prediction result of the target frame image, use a single-frame target detection algorithm to perform confidence interval filtering on the detection boxes in the target frame image to obtain a first detection result and a second detection result, perform correlation matching between the first detection result and the prediction boxes in the cross-frame prediction result to obtain a first matching result, obtain a tracking result of the detection box corresponding to the first detection result according to the first matching result, use a preset linear Kalman model to predict the next frame trajectory of the target frame in the tracking result, screen out target trajectories that meet the preset conditions, perform correlation matching between the detection boxes in the second detection result and the tracking result of the target trajectory to obtain a second matching result, and perform reset filtering on the confidence level of the second detection result according to the second matching result to obtain a final detection result. Thereby, the problems of low efficiency in railway intrusion pedestrian detection and low accuracy in detecting pedestrian intrusion are solved.

[0057] 1. Based on the single-object tracking method PrDIMP, existing trajectory tracking and new trajectory establishment are realized. Through the online learning mechanism, it has strong target discrimination ability and adaptability to appearance changes, and also has good initialization characteristics, which can meet the requirements of railway pedestrian target tracking.

[0058] 2. Taking the tracking results of each trajectory in each frame as the observation values, a Kalman model is constructed or updated for each trajectory respectively to achieve cross-frame prediction and unevenness evaluation of the trajectory. And the trajectory screening result of the current frame is comprehensively analyzed by using three types of state feature information, namely, the spatial relationship information of the trajectory, the trajectory unevenness information, and the matching information between the detection result and the trajectory prediction result, so as to obtain the trajectory information that can be used to enhance the target detection effect. Description of the Drawings

[0059] In the following, the present application will be described in more detail based on embodiments and with reference to the drawings:

[0060] Figure 1 It is a schematic flowchart of a railway intrusion pedestrian detection method provided by an embodiment of the present application;

[0061] Figure 2 It is an algorithm framework structure of a railway intrusion pedestrian detection method provided by an embodiment of the present application;

[0062] Figure 3 It is a schematic diagram of the functional structure of a trajectory prediction algorithm for a railway intrusion pedestrian detection method provided by an embodiment of the present application;

[0063] Figure 4 It is a schematic diagram of the functional structure of a trajectory screening algorithm for a railway intrusion pedestrian detection method provided by an embodiment of the present application;

[0064] Figure 5 It is the experimental result of the tracking performance of the multi-object tracking public dataset MOT17 for a railway intrusion pedestrian detection experiment provided by an embodiment of the present application;

[0065] Figure 6 It is the experimental result of the railway dataset for a railway intrusion pedestrian detection experiment provided by an embodiment of the present application;

[0066] Figure 7 It is the experimental confidence threshold parameter table for a railway intrusion pedestrian detection experiment provided by an embodiment of the present application;

[0067] Figure 8 It is the experimental result of the logarithmically averaged miss detection rate for different scenarios of the railway dataset in a railway intrusion pedestrian detection experiment provided by an embodiment of the present application;

[0068] Figure 9 It is a schematic diagram of the visualization result for a railway intrusion pedestrian detection experiment provided by an embodiment of the present application;

[0069] Figure 10 It is a schematic diagram of the functional modules of a railway intrusion pedestrian detection system provided by an embodiment of the present application;

[0070] Figure 11Schematic structural diagram of an electronic device for detecting railway-invading pedestrians provided in an embodiment of the present application.

[0071] In the accompanying drawings, the same components are denoted by the same reference numerals, and the drawings are not drawn to actual scale. Detailed implementation manners

[0072] In order to enable those skilled in the art of the present technology to better understand the technical solutions of the present application, and to fully understand how the present application uses technical means to solve technical problems and the implementation process of achieving corresponding technical effects and to implement accordingly, the following will, in conjunction with the accompanying drawings in the embodiments of the present application, clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The embodiments of the present application and each feature in the embodiments can be combined with each other on the premise of not conflicting, and the formed technical solutions are all within the protection scope of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present application.

[0073] It should be noted that the terms "first", "second", etc. in the description and claims of the present application and the above accompanying drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0074] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0075] Embodiment 1

[0076] Figure 1 Flowchart of a method for detecting railway-invading pedestrians provided in an embodiment of the present application Figure 2 Algorithm framework structure of a method for detecting railway-invading pedestrians provided in an embodiment of the present application. As Figure 1 and Figure 2 shown, a method for detecting railway-invading pedestrians includes:

[0077] S1. Obtain the detection boxes in the target frame image of the railway track pedestrian video sequence and the prediction boxes in the cross-frame prediction result of the target frame image.

[0078] In the embodiment of the present invention, the detection box in the target frame image of the railway track pedestrian video sequence refers to the target pedestrian recognized by monitoring. The historical frame of the target frame image generates the cross-frame prediction result of the target frame image through a preset linear Kalman model, preparing a data basis for subsequent correlation matching with the detection result. Among them, the prediction box in the cross-frame prediction result refers to the position prediction result of the target pedestrian at the target frame by the historical frame of the target frame image. The prediction of the linear Kalman model includes three items: state prediction, covariance prediction, and Kalman gain coefficient calculation. The position prediction result of the target pedestrian at the target frame is calculated according to the state prediction matrix and covariance prediction matrix of the historical frame, and the prediction box represents the position prediction result.

[0079] S2. Use the single-frame object detection algorithm to filter the detection boxes in the target frame image to obtain the first detection result and the second detection result.

[0080] In the embodiment of the present invention, the use of the single-frame object detection algorithm to filter the detection boxes in the target frame image to obtain the first detection result and the second detection result includes:

[0081] Perform normalization processing on the target frame image to obtain a normalized image;

[0082] Use a preset YOLO model to calculate the confidence of the detection boxes in the normalized image to obtain an initial detection result;

[0083] Select the detection boxes with a confidence greater than the upper limit of the preset confidence threshold interval from the initial detection result as the first detection result;

[0084] Select the detection boxes with a confidence less than the lower limit of the preset confidence threshold interval from the initial detection result as the second detection result.

[0085] Specifically, first adjust the target frame image to the input size required by the preset YOLO model, and then scale the pixel values of the adjusted target frame image from the initial range of 0 to 255 to between 0 and 1, and finally obtain a normalized image. The specific calculation formula is as follows:

[0086]

[0087] Among them, nor represents the pixel value of the scaled target frame image, and ori represents the initial pixel value of the target frame image.

[0088] Filtering the target frame image using a single-frame object detection algorithm can significantly improve the accuracy and practicality of detection. By screening the detection results with a confidence level greater than a preset high-confidence threshold, it is ensured that only the target results with high confidence are retained, reducing false detections and thus improving the reliability of detection. At the same time, screening the detection results with a confidence level less than a preset low-confidence threshold helps to eliminate potential false detections with low confidence, further improving the accuracy and clarity of the results. This hierarchical filtering mechanism not only enhances the quality of the detection results but also contributes to subsequent intrusion detection and analysis.

[0089] S3. Associate and match the first detection result with the prediction boxes in the cross-frame prediction result to obtain a first matching result.

[0090] In the embodiment of the present invention, the step of associating and matching the first detection result with the prediction boxes in the cross-frame prediction result to obtain a first matching result includes:

[0091] Calculate the first IOU value between several detection boxes in the first detection result and the prediction boxes in the cross-frame prediction result one by one;

[0092] Calculate the first association loss value between the first detection result and the cross-frame prediction result one by one according to the first IOU value;

[0093] Construct a first matrix using the first association loss value;

[0094] Remove the minimum value in each row of the first matrix to obtain a first standard matrix;

[0095] Match the first standard matrix with the prediction boxes in the cross-frame prediction result:

[0096] When the first association loss value in the first standard matrix is less than a preset association loss threshold, determine that the obtained first matching result is a successful match between the first detection result and the prediction boxes in the cross-frame prediction result;

[0097] When the first association loss value in the first standard matrix is greater than or equal to the association loss threshold, determine that the obtained first matching result is a failed match between the first detection result and the prediction boxes in the cross-frame prediction result.

[0098] Specifically, the Hungarian algorithm performs marking and covering operations on the first standard matrix, covers the rows and columns of the first standard matrix, finds that the number of lines of the minimum cover is equal to the dimension of the first standard matrix, adjusts the first standard matrix, and repeats the covering step to finally find the first matching result with the minimum distance.

[0099] Specifically, let the first detection result be D h, the cross-frame prediction result is f t , each containing U1 detection boxes and V1 prediction boxes, and the first detection result D is calculated using formula (1) h (i) (i ∈ [1, U1]) and the cross-frame prediction result f t (j) (j ∈ [1, V1]) the first IOU value iou between them i,j .

[0100]

[0101] Among them, iou i,j represents the first IOU value, D h (i) represents the first detection result, f t (j) represents the cross-frame prediction result.

[0102] From the first IOU result, the first association loss value of D h (i) and f t (j) is:[[]]

[0103] cost i,j = 1 - iou i,j (2)

[0104] Among them, cost i,j represents the first association loss value of D h (i) and f t (j), iou i,j represents the first IOU value.

[0105] Based on the two-dimensional matrix assignment algorithm, the first detection result is associated and matched with the cross-frame prediction result to obtain the first matching result, as shown in formula (3):

[0106]

[0107] Among them, X * represents the set of the first matching result including x i,j , x i,j ∈ {0, 1}, when the first detection result successfully matches the prediction box in the cross-frame prediction result, x i,j = 1, when the first detection result fails to match the prediction box in the cross-frame prediction result, x i,j = 0, cost i,j represents the first association loss value of D h (i) and f t (j), U1 represents there are U1 detection boxes, V1 represents there are V1 prediction boxes, i ∈ [1, U1] represents the i-th detection box, and j ∈ [1, V1] represents the j-th prediction box.

[0108] By calculating the IOU value between the detection box and the prediction result, and calculating the association loss based on this value, a matrix is constructed, and the matrix is processed to remove the minimum value and perform matching, which can effectively improve the accuracy of the detection result, and can accurately associate and match the first detection result with the cross-frame prediction result, improving the accuracy and reliability of the target tracking in the target frame image of the detection system.

[0109] S4. Obtain the tracking result of the detection box corresponding to the first detection result according to the first matching result.

[0110] In the embodiment of the present invention, the obtaining the tracking result of the detection box corresponding to the first detection result according to the first matching result includes:

[0111] If the first matching result is that the first detection result matches successfully with the prediction box in the cross-frame prediction result, update the parameters of the preset single-object tracking model by using the first detection result;

[0112] Use the first detection result as the tracking result of the single-object tracking model in the target frame;

[0113] If the first matching result is that the first detection result does not match the prediction box in the cross-frame prediction result, generate the tracking result of the target frame by using the target frame image and the iterative parameters of the single-object tracking model.

[0114] Specifically, when the first matching result is that the first detection result does not match the prediction box in the cross-frame prediction result, the first detection result with a failed match is regarded as a new target, and the PrDIMP algorithm is used to construct a single-object tracking model for the new target, and the first detection result is used as the starting point of the new trajectory. The detection boxes and prediction boxes with successful matches are constructed into a matching pair set X1, such as (D h (i), f t (j)) ∈ X1 indicates that D h (i) and f t (j) match successfully, and together with the corresponding first association loss value, they form the matching information M1 for completing the tracking of the target frame.

[0115] The PrDIMP algorithm is an online discriminative single-object tracking algorithm, which has strong target discrimination ability and appearance change adaptation ability, and also has good initialization characteristics, meeting the requirements of pedestrian tracking of railway track targets. Through staged matching and updating steps, it can flexibly handle the changes of targets and the establishment of new targets during the target tracking process, thereby improving the accuracy and reliability of the tracking.

[0116] Figure 3Schematic diagram of the functional structure of the trajectory prediction algorithm for a railway intrusion pedestrian detection method provided by an embodiment of the present application. According to Figure 3 The process of Figure 3 specifically describes screening out appropriate trajectories according to the Mahalanobis distance to update the parameters of the linear Kalman model.

[0117] S5. Use a preset linear Kalman model to predict the trajectory of the next frame of the target frame in the tracking result, and screen out target trajectories that meet the preset conditions.

[0118] In the embodiment of the present invention, the step of using a preset linear Kalman model to predict the trajectory of the next frame of the target frame in the tracking result and screening out target trajectories that meet the preset conditions includes:

[0119] Calculate the state prediction matrix and covariance matrix of the trajectory of the next frame of the target frame in the tracking result;

[0120] Calculate the prediction result of the trajectory of the next frame of the target frame in the tracking result according to the state prediction matrix and the covariance matrix;

[0121] Calculate the squared value of the Mahalanobis distance between the tracking result and the prediction result;

[0122] If the squared value of the Mahalanobis distance is greater than the preset squared Mahalanobis distance threshold, the linear Kalman model does not update the state parameters;

[0123] If the squared value of the Mahalanobis distance is less than or equal to the squared Mahalanobis distance threshold, calculate the Kalman gain coefficient of the trajectory of the next frame of the target frame in the tracking result;

[0124] Use the tracking result and the Kalman gain coefficient to update the state parameters of the linear Kalman model;

[0125] Statistically count the number of matching failures and the number of trajectory anomalies of the trajectory of the next frame of the target frame in the state parameters of the linear Kalman model;

[0126] Screen out target trajectories that meet the preset conditions according to the number of matching failures and the number of trajectory anomalies.

[0127] By predicting the trajectory of the next frame of the tracking target through a linear Kalman model and combining the Mahalanobis distance to detect the difference between the prediction and the actual result, it is decided whether to update the state parameters of the model, improving the accuracy and reliability of the prediction. At the same time, it avoids invalid updates of the model when the prediction result is not credible, thereby optimizing the overall performance of the tracking system.

[0128] Specifically, the step of screening out target trajectories that meet the preset conditions according to the number of matching failures and the number of trajectory anomalies includes:

[0129] If the number of abnormal trajectories is greater than or equal to the abnormal trajectory number threshold, delete the trajectory of the next frame of the target frame in the tracking result;

[0130] If the number of matching failures is greater than or equal to the matching failure number threshold, delete the trajectory of the next frame of the target frame in the tracking result;

[0131] If the number of matching failures is less than the preset matching failure number threshold, determine whether the number of abnormal trajectories is less than the preset abnormal trajectory number threshold;

[0132] If the number of abnormal trajectories is less than the abnormal trajectory number threshold, mark the trajectory of the next frame of the target frame in the tracking result as the target trajectory.

[0133] Specifically, by analyzing the number of matching failures and the number of abnormal trajectories, the system can dynamically adjust the Kalman gain coefficient and optimize the update of state parameters, ensuring that the finally output target trajectory has high quality and consistency. By setting the thresholds for the number of matching failures and the number of abnormal trajectories, abnormal trajectories can be automatically processed and deleted, reducing the need for manual intervention and improving the automation level and robustness of the system.

[0134] Specifically, the state parameters of the linear Kalman model are:

[0135]

[0136] Among them, x, y, r, and h respectively represent the center point coordinates of the tracking box, the aspect ratio of the target box, and the pixel value of the height of the target box. represents the change speed of the x, y, r, h parameters.

[0137] Taking the cross-frame prediction process from the trajectory j of the (t - 1)-th frame to the t-th frame and the update of the corresponding linear Kalman model at the t-th frame as an example, the prediction of the linear Kalman model corresponding to the trajectory j includes three items: state prediction, covariance prediction, and Kalman gain coefficient calculation. The calculation formulas are shown in Formulas (5), (6), and (7). After completing the state prediction and covariance prediction of the trajectory j at the t-th frame, use Formula (8) to obtain the trajectory prediction result f t (j) of the trajectory j at the t-th frame.

[0138]

[0139]

[0140]

[0141]

[0142] Among them, t and j represent parameters, and belong to the linear Kalman model corresponding to the j-th trajectory in the t-th frame. represents the state prediction matrix of the t-th frame, and A represents the state transition matrix. represents the state prediction matrix of the (t - 1)-th frame. represents the covariance matrix of the t-th frame, P t-1 (j) represents the covariance matrix of the (t - 1)-th frame, Q represents the noise matrix of the system, K t (j) represents the Kalman gain, H represents the measurement matrix, and R represents the observation noise matrix, f t (j) represents the trajectory prediction result of the t-th frame.

[0143] In this embodiment, the tracking result T t (j) of the trajectory j in the t-th frame and the prediction result f t (j) of this trajectory from the (t - 1)-th frame to the t-th frame are used to solve the squared value of the Mahalanobis distance to measure the non-smooth feature of the trajectory in the t-th frame, as shown in Equation (9).

[0144]

[0145] Among them, D t (j) represents the squared value of the Mahalanobis distance of the linear Kalman model corresponding to the trajectory j in the t-th frame, S t (j) is the covariance matrix of the observation space of the linear Kalman model corresponding to the trajectory j at the t-th frame moment, T t (j) represents the tracking result of the trajectory j in the t-th frame, and t and j represent parameters belonging to the linear Kalman model corresponding to the j-th trajectory in the t-th frame. represents the state prediction matrix of the t-th frame, and H represents the measurement matrix.

[0146] If D t (j) is greater than the preset squared Mahalanobis distance threshold, the state parameters of the linear Kalman model are not updated. If D t (j) is less than or equal to the preset squared Mahalanobis distance threshold, then T t (j), that is, the tracking result of the trajectory j in the t-th frame, is used as the observed value to update the state parameters and covariance matrix of the linear Kalman model. The state parameter update and covariance matrix update are as shown in Equations (10) and (11).

[0147]

[0148]

[0149] Among them, I is the identity matrix, and t and j represent parameters belonging to the linear Kalman model corresponding to the j-th trajectory in the t-th frame. represents the state prediction matrix of the t-th frame, Kt (j) represents the Kalman gain, T t (j) represents the tracking result of trajectory j at the t-th frame, H represents the measurement matrix, represents the covariance matrix.

[0150] Screening high-confidence trajectories helps to eliminate outliers and noise, improve the stability and robustness of the tracking system, not only can improve the tracking accuracy, but also optimize the use of resources, enabling the system to perform long-term tracking tasks more reliably.

[0151] Figure 4 It is a schematic functional structure diagram of the trajectory screening algorithm for a railway intrusion pedestrian detection method provided by an embodiment of the present application. As Figure 4 shown, it is considered credible only when the confidence of the detection information is greater than the preset confidence threshold when the trajectory is established or the cross-frame prediction result within 3 frames after the trajectory is established successfully matches the first detection result no less than twice. Combining the screening of noise trajectories and distorted trajectories completes the screening of all trajectories and the confirmation of the target trajectory, and updates the trajectory state parameters during this process.

[0152] S6. Associate and match the detection boxes in the second detection result with the tracking result of the target trajectory to obtain a second matching result.

[0153] In the embodiment of the present invention, associating and matching the detection boxes in the second detection result with the tracking result of the target trajectory to obtain a second matching result includes:

[0154] Calculate the second IOU value between several detection boxes in the second detection result and the tracking result of the target trajectory one by one;

[0155] Calculate the second association loss value between the second detection result and the tracking result of the target trajectory one by one according to the second IOU value;

[0156] Construct a second matrix using the second association loss value;

[0157] Remove the minimum value in each row of the second matrix to obtain a second standard matrix;

[0158] Match the second standard matrix with the tracking result of the target trajectory:

[0159] When the second association loss value in the second standard matrix is less than the association loss threshold, it is determined that the obtained second matching result is a successful match between the second detection result and the tracking result of the target trajectory;

[0160] When the second associated loss value in the second standard matrix is greater than or equal to the associated loss threshold, it is determined that the obtained second matching result fails to match the tracking result of the target trajectory with the second detection result.

[0161] Specifically, associating and matching the detection boxes in the second detection result with the tracking result of the target trajectory can significantly improve the accuracy and efficiency of target detection and tracking, provide more reliable and accurate target information for the system, find more reliable matching results in a complex environment, effectively handle the uncertainty in the tracking process, and enhance the robustness of the system in different situations.

[0162] S7. Reset and filter the confidence level of the second detection result according to the second matching result to obtain the final detection result.

[0163] In the embodiment of the present invention, the resetting and filtering the confidence level of the second detection result according to the second matching result to obtain the final detection result includes:

[0164] If the second detection result fails to match the tracking result of the target trajectory, update the confidence level of the second detection result using a preset reset constant;

[0165] If the second detection result successfully matches the tracking result of the target trajectory, update the confidence level of the second detection result using the second associated loss value;

[0166] If the confidence level of the updated second detection result is less than or equal to the upper limit of the confidence threshold interval, delete the detection box in the second detection result;

[0167] If the confidence level of the updated second detection result is greater than the upper limit of the confidence threshold interval, obtain the final detection result.

[0168] Specifically, obtain the second detection result D l and the tracking result T of the target trajectory in the t-th frame tc , assuming that the second detection result and the tracking result of the target trajectory in the t-th frame respectively include U2 detection boxes and V2 tracking boxes, calculate the IOU value iou of the corresponding target boxes of each D l (a) (a ∈ [1, U2]) and T tc (b) (b ∈ [1, V2]) ab and the associated loss value cost ab , and use the matching algorithm and processing means in the trajectory tracking and establishment stage to obtain the finally successfully matched information set X2. For example, (D l (a), T tc (b)) ∈ X2 represents (D l(a) is matched with T tc (b), and the matching information M2 is formed with the corresponding matching loss.

[0169] During the confidence reset process, let Confa (a ∈ [1, U2]) be the confidence of the a-th detection box D l (a). If (D l (a), T tc (b)) ∈ X2, then Confa is reset using Equation (12):

[0170]

[0171] where Confa (a ∈ [1, U2]) is the confidence of the a-th detection box D l (a), and cost ab represents the association loss between the second detection result and the tracking result of the target trajectory in the t-th frame.

[0172] If D l (a) fails to be successfully matched with any trajectory, then its confidence value is reset using the following formula:

[0173] Confa a = Confa a - down(13)

[0174] where down represents a constant greater than 0 and less than 1, and Confa (a ∈ [1, U2]) is the confidence of the a-th detection box D l (a).

[0175] By resetting and filtering the confidence of the second detection result according to the second matching result, the final detection result can more accurately reflect the actual situation, which helps to eliminate noise and errors, improve the reliability and accuracy of detection, reduce the false negatives caused by the incorrect suppression of the detection result when the trajectory does not exist, thereby ensuring that the final result is more credible and optimizing the performance of the system.

[0176] Figure 5 This is the experimental result of the multi-object tracking performance of the publicly available dataset MOT17 for railway intrusion pedestrian detection experiments provided by the embodiments of this application. Experiments were conducted on the training set sequences of the MOT17 dataset. The target trajectories were constructed and screened by the algorithm as the tracking results for evaluation, and compared with the advanced multi-object tracking algorithm DeepSORT. Both used the detection results of the SDP detector provided by MOT17 official as the detection information. The experimental results are as Figure 5 shown.

[0177] Figure 6 This is the experimental result of the railway dataset for railway intrusion pedestrian detection experiments provided by the embodiments of this application. Figure 7This is a table of experimental confidence threshold parameters for a railway intrusion pedestrian detection experiment provided by an embodiment of the present application. From Figure 6 It can be seen that after combining the trajectory collaborative detection algorithm, the overall algorithm, while maintaining high detection accuracy compared to the original single-frame algorithm, has the recall rate increased by 2.3%, 3.6%, and 4.5% respectively, and the logarithmically averaged miss detection rate decreased by 1.3%, 3.5%, and 4.0% respectively. This indicates that the algorithm of this embodiment can significantly reduce the miss reporting of pedestrian detection in railway scenarios and improve the detection effect.

[0178] Figure 8 This is the experimental result of the logarithmically averaged miss detection rate of the railway dataset for a railway intrusion pedestrian detection experiment provided by an embodiment of the present application. Figure 8 It shows the logarithmically averaged miss detection rate of Yolov3 and Yolov5x with better detection performance and the combination with the trajectory collaborative detection algorithm respectively in different scenarios of the railway dataset. Figure 9 This is a schematic diagram of the visualization result of a railway intrusion pedestrian detection experiment provided by an embodiment of the present application. From top to bottom, it corresponds to Scenario 1 to Scenario 5 in sequence, and from left to right, they are the detection effects of Yolov3, the combination of Yolov3 and the trajectory collaborative detection algorithm, Yolov5x, and the combination of Yolov5x and the trajectory collaborative detection algorithm.

[0179] From Figure 8 it can be known that the algorithm of this embodiment has improved the detection effect in all actual railway scenarios compared to the original single-frame object detection algorithm, and the degree of improvement in the detection effect after combining the trajectory collaborative detection algorithm for the same detection algorithm is different in different scenarios. For different scenarios, the logarithmically averaged miss detection rate of Yolov3 and Yolov5x after combining the trajectory collaborative detection algorithm has decreased by 0.4 - 5.3% and 2.3 - 8.8% respectively.

[0180] Embodiment 2

[0181] As Figure 10 shown, this is a functional module diagram of a railway intrusion pedestrian detection system 100 provided by this embodiment.

[0182] The railway intrusion pedestrian detection system 100 described in the present invention can be installed in an electronic device. According to the functions achieved, the railway intrusion pedestrian detection system 100 may include a data acquisition module 101, an image filtering module 102, a first association matching module 103, a tracking result acquisition module 104, a trajectory screening module 105, a second association matching module 106, and a reset filtering module 107. The modules described in the present invention can also be referred to as units, which refer to a series of computer program segments that can be executed by the processor of an electronic device and can complete fixed functions, and are stored in the memory of the electronic device.

[0183] In this embodiment, the functions of each module / unit are as follows:

[0184] The data acquisition module 101 is used to obtain the detection boxes in the target frame image of the railway track pedestrian video sequence and the prediction boxes in the cross-frame prediction results of the target frame image;

[0185] The image filtering module 102 is used to filter the confidence intervals of the detection boxes in the target frame image by using a single-frame object detection algorithm to obtain a first detection result and a second detection result;

[0186] The first association matching module 103 is used to associate and match the first detection result with the prediction boxes in the cross-frame prediction results to obtain a first matching result;

[0187] The tracking result acquisition module 104 is used to obtain the tracking result of the detection box corresponding to the first detection result according to the first matching result;

[0188] The trajectory screening module 105 is used to predict the trajectory of the next frame of the target frame in the tracking result by using a preset linear Kalman model, and screen out the target trajectories that meet the preset conditions;

[0189] The second association matching module 106 is used to associate and match the detection boxes in the second detection result with the tracking results of the target trajectories to obtain a second matching result;

[0190] The reset filtering module 107 is used to reset and filter the confidence of the second detection result according to the second matching result to obtain a final detection result.

[0191] Embodiment 3

[0192] Figure 11 FIG. is a schematic structural diagram of an electronic device for detecting railway intrusion pedestrians provided by an embodiment of the present application.

[0193] On the basis of the above embodiment, this embodiment provides a computer device, including a memory, a processor, and a computer program stored on the memory. The processor executes the computer program to implement the steps of the method described in the above embodiment.

[0194] In some embodiments of this embodiment, a computer-readable storage medium is provided, on which a computer program is stored. The computer program, when executed by a processor, implements the steps of the method described in the above embodiment.

[0195] In some embodiments of this embodiment, a computer program product is provided, including a computer program. The computer program, when executed by a processor, implements the steps of the method described in the above embodiment.

[0196] The processor may include, but is not limited to, for example, one or more processors or microprocessors, etc. Each processor may be implemented by an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components, and is used to execute the methods in the above embodiments.

[0197] The computer-readable storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof. The computer-readable storage medium may include, but is not limited to, for example, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (such as hard disks, floppy disks, solid state drives, removable disks, CD ROMs, DVD ROMs, Blu-ray discs, etc.).

[0198] The computer-readable storage medium may also store at least one computer-executable program, which is, for example, computer-readable instructions. The computer-readable storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. The computer-readable storage medium may include, for example, read-only memory (ROM), hard disks, flash memory, etc. For example, the non-transitory computer-readable storage medium may be connected to a computing device such as a computer. Then, when the computing device runs the computer-readable instructions stored on the computer-readable storage medium, the various methods described above may be performed.

[0199] In addition, the computer device may also include (but is not limited to) a data bus, an input / output (I / O) bus, a display, and input / output devices (such as a keyboard, a mouse, a speaker, etc.).

[0200] The processor may communicate with the communication interface of an external device via the I / O bus through a wired or wireless network.

[0201] In one embodiment, the at least one computer-executable instruction may also be compiled into or comprise a software product / computer program product, and when one or more computer-executable instructions are run by a processor, each function and / or step of the method in the embodiments described in the present technology is performed.

[0202] In the embodiments provided in the present application, it should be understood that the disclosed systems and methods may also be implemented in other ways. The system embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architectures, functions, and operations of systems, methods, and computer program products according to multiple embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the above-mentioned module, program segment, or part of code includes one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

[0203] It should be noted that in the present application, the terms "including", "comprising", or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device including a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. Without further limitation, an element limited by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article, or device including the element.

[0204] Although the disclosed embodiments of the present application are as above, the above content is only an embodiment adopted for the convenience of understanding the present application and is not used to limit the present application. Any person skilled in the art within the technical field to which the present application pertains may make any modifications and changes in the form of implementation and details without departing from the spirit and scope disclosed by the present application. However, the scope of patent protection of the present application shall still be subject to the scope defined by the appended claims.

Claims

1. A method for detecting railway intrusion into pedestrians, characterized in that, Including: Obtaining the detection boxes in the target frame image of the railway track pedestrian video sequence and the prediction boxes in the cross-frame prediction result of the target frame image; Using a single-frame object detection algorithm to perform confidence interval filtering on the detection boxes in the target frame image to obtain a first detection result and a second detection result; Associating and matching the first detection result with the prediction boxes in the cross-frame prediction result to obtain a first matching result; Obtaining the tracking result of the detection box corresponding to the first detection result according to the first matching result; Using a preset linear Kalman model to predict the next frame trajectory of the target frame in the tracking result, and screening out the target trajectories that meet the preset conditions; Associating and matching the detection boxes in the second detection result with the tracking result of the target trajectory to obtain a second matching result; Resetting and filtering the confidence of the second detection result according to the second matching result to obtain a final detection result.

2. The method according to claim 1, wherein The step of using a single-frame object detection algorithm to perform confidence interval filtering on the detection boxes in the target frame image to obtain a first detection result and a second detection result includes: Performing normalization processing on the target frame image to obtain a normalized image; Using a preset YOLO model to calculate the confidence of the detection boxes in the normalized image to obtain an initial detection result; Selecting the detection boxes with confidence greater than the upper limit of the preset confidence threshold interval from the initial detection result as the first detection result; Selecting the detection boxes with confidence less than the lower limit of the preset confidence threshold interval from the initial detection result as the second detection result.

3. The method according to claim 1, wherein The step of associating and matching the first detection result with the prediction boxes in the cross-frame prediction result to obtain a first matching result includes: Calculating the first IOU values between several detection boxes in the first detection result and the prediction boxes in the cross-frame prediction result one by one; Calculating the first association loss values between the first detection result and the cross-frame prediction result one by one according to the first IOU values; Using the first association loss values to construct a first matrix; Removing the minimum value in each row of the first matrix to obtain a first standard matrix; Matching the first standard matrix with the prediction boxes in the cross-frame prediction result; When the first association loss value in the first standard matrix is less than the preset association loss threshold, determining that the obtained first matching result is a successful match between the first detection result and the prediction boxes in the cross-frame prediction result; When the first association loss value in the first standard matrix is greater than or equal to the association loss threshold, determining that the obtained first matching result is a failed match between the first detection result and the prediction boxes in the cross-frame prediction result.

4. The method according to claim 1, wherein The step of obtaining the tracking result of the detection box corresponding to the first detection result according to the first matching result includes: If the first matching result is a successful match between the first detection result and the prediction boxes in the cross-frame prediction result, then using the first detection result to update the parameters of the preset single-object tracking model; Taking the first detection result as the tracking result of the single-object tracking model in the target frame. If the first matching result indicates that the prediction box in the first detection result and the cross-frame prediction result fails to match, the tracking result of the target frame is generated using the target frame image and the iterative parameters of the single-object tracking model.

5. The method according to claim 1, wherein The predicting the next-frame trajectory of the target frame in the tracking result using the preset linear Kalman model and screening out the target trajectories that meet the preset conditions includes: Calculating the state prediction matrix and covariance matrix of the next-frame trajectory of the target frame in the tracking result; Calculating the prediction result of the next-frame trajectory of the target frame in the tracking result according to the state prediction matrix and the covariance matrix; Calculating the squared value of the Mahalanobis distance between the tracking result and the prediction result; If the squared value of the Mahalanobis distance is greater than the preset squared Mahalanobis distance threshold, the linear Kalman model does not update the state parameters; If the squared value of the Mahalanobis distance is less than or equal to the squared Mahalanobis distance threshold, calculating the Kalman gain coefficient of the next-frame trajectory of the target frame in the tracking result; Updating the state parameters of the linear Kalman model using the tracking result and the Kalman gain coefficient; Counting the number of matching failures and the number of trajectory anomalies of the next-frame trajectory of the target frame in the state parameters of the linear Kalman model; Screening out the target trajectories that meet the preset conditions according to the number of matching failures and the number of trajectory anomalies.

6. The method according to claim 1, wherein The resetting and filtering the confidence level of the second detection result according to the second matching result to obtain the final detection result includes: If the second detection result fails to match the tracking result of the target trajectory, updating the confidence level of the second detection result using a preset reset constant; If the second detection result matches the tracking result of the target trajectory, updating the confidence level of the second detection result using the second association loss value; If the confidence level of the updated second detection result is less than or equal to the upper limit of the confidence level threshold interval, deleting the detection box in the second detection result; If the confidence level of the updated second detection result is greater than the upper limit of the confidence level threshold interval, obtaining the final detection result.

7. A railway pedestrian intrusion detection system, characterized in that, The system includes: A data acquisition module for acquiring the detection box in the target frame image of the railway track pedestrian video sequence and the prediction box in the cross-frame prediction result of the target frame image; An image filtering module for filtering the confidence level interval of the detection box in the target frame image using a single-frame object detection algorithm to obtain a first detection result and a second detection result; A first association matching module for associating and matching the first detection result with the prediction box in the cross-frame prediction result to obtain a first matching result; A tracking result acquisition module for obtaining the tracking result of the detection box corresponding to the first detection result according to the first matching result; A trajectory screening module for predicting the next-frame trajectory of the target frame in the tracking result using a preset linear Kalman model and screening out the target trajectories that meet the preset conditions; The second correlation matching module is used to perform correlation matching between the detection boxes in the second detection result and the tracking result of the target trajectory to obtain a second matching result; The reset filtering module is used to reset and filter the confidence of the second detection result according to the second matching result to obtain a final detection result.

8. A computer device, comprising a memory, a processor, and a computer program stored on the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images