Track intrusion monitoring method

Through weakly supervised abnormal feature detection and multi-scale orbital abnormal positioning model, the problems of low orbit intrusion detection accuracy and high resource consumption in the existing technology are solved, efficient monitoring of diverse foreign objects is achieved, and detection accuracy and system reliability are improved. sex.

CN119007098AActive Publication Date: 2024-11-22BEIJING JIAOTONG UNIV

Patent Information

Application Number
CN202410965295.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-18
Publication Date
2024-11-22
Estimated Expiration
2044-07-18

AI Technical Summary

Technical Problem

The existing vision-based orbit intrusion detection technology has problems such as low detection accuracy, high resource consumption, poor model generalization and high data labeling complexity. Especially when facing foreign object invasion with diversity and morphological changes, it is difficult to achieve real-time and efficient. monitor.

Method used

Weakly supervised anomaly feature detection and multi-scale orbital anomaly positioning model are used to pre-train the X3D model through Kinetics-400, combined with Non-Local and multi-scale time domain attention mechanisms, video feature extraction and abnormal target positioning are performed to reduce model parameters Scale and improve computing efficiency.

Benefits of technology

It realizes efficient monitoring of track intrusion, reduces the complexity of data labeling, improves the application scope and real-time processing capabilities of the model, and improves detection accuracy and system reliability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007098A_ABST
    Figure CN119007098A_ABST
Patent Text Reader

Abstract

The invention provides a track intrusion monitoring method. The method comprises the following steps: acquiring monitoring video data of a track through monitoring equipment mounted beside the track, and preprocessing the monitoring video data to obtain input video data; performing weak supervision abnormal feature detection on the input video data to obtain abnormal data; and carrying out abnormal target positioning on the abnormal data to obtain a track intrusion detection result. According to the embodiment of the invention, a weak supervision mode and an unsupervised mode are adopted aiming at the high dependence of an existing mode on large-scale annotation data, the complexity of data annotation is reduced, and the application range of an algorithm model is widened. Through an elaborately designed model lightweight strategy, the model parameter scale and the calculation burden are reduced, and the application feasibility and efficiency in a mobile deployment scene with limited calculation resources are greatly improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of track monitoring, and in particular to a track intrusion monitoring method. Background Art

[0002] With the rapid development of China's high-speed rail network, by the beginning of 2024, the total mileage of railways in China has exceeded 150,000 kilometers, and the operating length of high-speed railways has exceeded 40,000 kilometers, highlighting its global leadership. This huge transportation network plays a core role in promoting social and economic development and regional integration, but its expansion also puts higher demands on safety performance. How to balance transportation efficiency and safety has become an urgent issue.

[0003] The high-speed railway system is complex, with multiple and multi-layered safety considerations, among which environmental factors are particularly critical. Extreme climate and geological disasters pose a threat to the natural environment, while foreign body intrusion in the external environment (such as falling objects from highway-railway bridges and rockfalls from mountains) becomes difficult to monitor due to its suddenness. In view of this, the development of a real-time and efficient foreign body intrusion monitoring system, coupled with a rapid response mechanism, is crucial to improving the railway's ability to respond to environmental and human safety risks.

[0004] my country's railway monitoring system has demonstrated strong effectiveness, relying on its wide-area coverage and continuous operation advantages to effectively monitor security threats along the line. However, faced with the exponential growth of surveillance video data, the traditional model that relies on human review faces problems of low efficiency and easy omissions, especially with the current situation of a surge in the number of railway monitoring nodes.

[0005] The integration of machine vision and deep learning technology provides an innovative solution for this. Through automated and efficient detection and in-depth data analysis capabilities, it not only improves monitoring efficiency, but also enhances adaptability to complex environments and security protection levels. At present, the existing vision-based rail intrusion detection methods include traditional vision-based methods and deep learning-based methods.

[0006] Track foreign body intrusion detection technology based on traditional vision covers a variety of methods, including but not limited to motion difference or background modeling methods, weighted classification methods combining scene priors and motion patterns, and feature description and multi-scale analysis methods. The disadvantages of these methods include: low detection accuracy and single application scenario.

[0007] The detection technology of foreign body intrusion on the track based on visual deep learning has shown a diversified development trend in domestic and foreign research. The core branches of this field can be summarized into two categories: detection technology based on 2D images and detection technology based on 3D vision. For the detection field based on 2D images, it is further subdivided into two directions: static image analysis and video sequence analysis according to the different characteristics of the processed data. In static image processing, the research focuses on applying image segmentation or target detection algorithms, such as Faster R-CNN or YOLO series, to accurately depict and locate the shape and position of foreign bodies on the track. In video sequence analysis, the research uses technologies such as optical flow, background difference, and video anomaly detection. Through in-depth analysis of continuous video frames, it effectively captures transient foreign body intrusion events that occur in the track area. On the other hand, the detection field based on 3D vision covers two major parts: point cloud processing technology and stereo vision technology. In point cloud processing, researchers combine 3D lidar and other three-dimensional sensing equipment to collect point cloud data, and use deep learning architectures such as PointNet or PointCNN designed for three-dimensional point clouds to achieve high-precision positioning of foreign bodies in the three-dimensional space of the track. Stereo vision technology uses binocular or multi-camera systems to build a stereo vision model, calculates parallax to generate depth images, and then combines deep learning technology to accurately identify foreign object intrusion phenomena embedded in stereo images.

[0008] The shortcomings of the existing track foreign body intrusion detection methods based on vision-based deep learning technology include: the diversity of foreign body types and morphological changes require the model to be highly generalized, but comprehensive coverage is difficult in practice; the high resource consumption of deep learning models not only increases hardware costs, but may also restrict real-time processing efficiency; the lack of sufficient and accurate labeled data limits the training effect of the model; in addition, balancing detection accuracy with reducing false positives and false negatives, such as misidentification of natural objects or non-threatening objects, is still a difficulty in practice. These problems together constitute the main obstacles in the implementation of the technology. Summary of the invention

[0009] An embodiment of the present invention provides a track intrusion monitoring method to achieve effective monitoring of track intrusion.

[0010] In order to achieve the above object, the present invention adopts the following technical scheme.

[0011] A track intrusion monitoring method, comprising:

[0012] The monitoring video data of the track is collected by the monitoring equipment installed beside the track, and the monitoring video data is pre-processed to obtain the input video data;

[0013] Performing weakly supervised abnormal feature detection on the input video data to obtain abnormal data;

[0014] Abnormal target positioning is performed on the abnormal data to obtain track intrusion detection results.

[0015] Preferably, the monitoring video data of the track is collected by a monitoring device installed beside the track, and the monitoring video data is preprocessed to obtain input video data, including:

[0016] Track intrusion videos are collected by surveillance cameras installed beside the track. Half of the positive samples and half of the negative samples are selected from the collected surveillance video data images. The positive samples are video frames under normal conditions, and the negative samples are video frames under abnormal conditions. The selected surveillance video data are preprocessed, and the preprocessing includes image mask cropping and far-point information magnification processes. The preprocessed surveillance video data is used as input video data.

[0017] Preferably, the selected surveillance video data is preprocessed, and the preprocessing includes image mask cropping and far point information amplification process, including:

[0018] The track spacing between tracks is set to 1435mm, and the track area detection range is set to 2500mm from the center of the track to both sides. The track line is obtained through the track detection algorithm, and the middle line of the track line is drawn. The track lines on both sides are translated to the outside by the ratio of 2500 / (1435 / 2) to form a limit line. The mask is obtained according to the limit line, and other local areas are cropped according to the region of interest ROI. The local image of the far point area of ​​the track is cropped separately and merged with the uncropped complete image into a set of complementary image samples to obtain the final input image video.

[0019] Preferably, the performing weakly supervised abnormal feature detection on the input video data to obtain abnormal data includes:

[0020] The X3D model is pre-trained using Kinetics-400, and features are extracted from the input video data using the trained X3D model to obtain feature vectors. The feature vectors are enhanced using Non-Local and multi-scale temporal attention mechanisms to obtain optimized feature vectors. The optimized feature vectors are scored using a classifier, and K scores with the highest absolute values ​​are respectively selected from the scores of normal and abnormal video frames, and corresponding K feature vectors are extracted according to the frame indexes corresponding to the K highest scores.

[0021] The amplitude score loss function, amplitude feature loss function and time smoothing loss function are calculated according to the selected K scores and the corresponding feature vectors. The calculation method of the amplitude score loss function is as follows:

[0022]

[0023] Where x represents a normal or abnormal video frame, y represents the label of the video where the video frame is located, and f represents the model operation;

[0024] Take the L2 norm of the K video frame scores of the normal class and the abnormal class and sum them up:

[0025]

[0026] Where N is the number of video frames, Ω k (X) is a set of K video frames, ||f(x i )|| 2 represents the L2 norm of the score of the i-th video frame;

[0027] Subtract the sum of the feature norms of the normal K video frames from the sum of the feature norms of the abnormal K video frames, as shown in the following formula:

[0028] d(X + , X - ) = g k (X + )-g k (X - )

[0029] Among them, g k (X + ) represents the L2 norm and g of the normal video frame score. k (X - ) represents the L2 norm sum of the abnormal video frame scores, which represents the degree of distinction between normal and abnormal video frames;

[0030] The calculation method of the amplitude feature loss function is as follows:

[0031] l s =max(0,md(X i , X j ))

[0032] m is the threshold defined by the implementation to ensure that the loss function is a positive number; d(X i , X j ) is the function representing the degree of distinction between normal and abnormal video frames mentioned above;

[0033] By adding a time smoothing loss function, the scores between adjacent normal frames and abnormal frames in a video are close after multiple trainings. The calculation method of the time smoothing loss function is as follows;

[0034]

[0035] Among them, λ is a pre-set hyperparameter, i represents the index order of the current frame in the video, and f(x i ) represents the score of the i-th video frame, f(x i+1 ) represents the score of the i+1th video frame, and N represents the number of video frames;

[0036] The specific formula of the total loss function is as follows:

[0037]

[0038] The video frames whose total loss function values ​​exceed the set threshold are judged as abnormal video frames, and the video frames whose total loss function values ​​do not exceed the set threshold are judged as normal video frames.

[0039] Preferably, the abnormal target positioning of the abnormal data to obtain the track intrusion detection result includes:

[0040] Construct a multi-scale track anomaly localization model architecture based on parameter sharing mechanism. The multi-scale track anomaly localization model architecture divides the encoder and decoder structure into three progressive levels. Two layers of down / up sampling units are configured in the first two levels, and a conventional up / down sampling module and an up / down sampling module that is dynamically enabled as needed are integrated in the third level.

[0041] The video frames judged as abnormal are input into the multi-scale track anomaly positioning model architecture. The encoder and decoder structure in the multi-scale track anomaly positioning model architecture includes five consecutive downsampling or upsampling units. Each sampling module integrates a spatial position encoding mechanism. It is a downsampling operation in the encoding stage and is converted to an upsampling operation in the decoding stage to capture local to global spatial information. The point convolution or deconvolution layer enhances the expressiveness of the feature map. All convolution operations are supplemented by batch normalization steps. The core calculation link of each module adopts an activation function, and the corresponding number of sampling layers are configured according to the input images of different sizes.

[0042] An effective channel attention ECA mechanism is set in the multi-scale track anomaly localization model architecture. The multi-scale VAE architecture contains latent variables of multiple scales. Each latent variable of each scale has a corresponding mean and variance. The total KL divergence can be defined as follows:

[0043]

[0044] Here, z l represents the latent variable of the lth scale level, L is the total number of scale levels, q(z l |x, z<l) is a given observation data x and a low-level latent variable z <lWhen , the approximate posterior distribution of the high-level latent variable z, and p(z l |z <l ) is the corresponding prior distribution;

[0045] The multi-scale track anomaly localization model architecture receives abnormal video frames screened out by the anomaly detection algorithm as input, and uses a deep neural network architecture to predict the background image that constitutes the video background. By comparing the abnormal video frame with the background image at the pixel level, a difference map between the abnormal video frame and the background image, namely the foreground mask, is calculated. The foreground mask highlights the abnormal activity areas with large expected changes relative to the background. The activations at different positions on the foreground mask correspond to multiple independent abnormal targets in the video.

[0046] It can be seen from the technical solutions provided by the above embodiments of the present invention that the embodiments of the present invention adopt weak supervision and unsupervised methods to reduce the complexity of data annotation and improve the scope of application of the algorithm model in view of the high dependence of the existing methods on large-scale labeled data. In view of the dilemma that the existing methods cannot process in real time and resources cannot be used efficiently, the present invention greatly reduces the scale of model parameters and the computational burden through a carefully designed model lightweight strategy, greatly improving the feasibility and efficiency of application in mobile deployment scenarios with limited computing resources.

[0047] Additional aspects and advantages of the present invention will be given in part in the following description, which will become obvious from the following description, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0049] Figure 1 A processing flow chart of a track intrusion monitoring method provided by an embodiment of the present invention;

[0050] Figure 2 A track intrusion detection data sample provided by an embodiment of the present invention;

[0051] Figure 3 An example of input data composition provided by an embodiment of the present invention;

[0052] Figure 4 An example of image mask cropping provided by an embodiment of the present invention;

[0053] Figure 5A far-point cropping and global retention of an image provided by an embodiment of the present invention (the far-point image is an image obtained by cropping the upper left corner of the original image according to a certain size, and the global image is an image obtained by scaling the original image to the same size as the far-point image;

[0054] Figure 6 A track video anomaly detection model framework based on weak supervision is provided in an embodiment of the present invention;

[0055] Figure 7 A Non-Local module structure diagram provided for an embodiment of the present invention;

[0056] Figure 8 A multi-scale temporal attention module provided by an embodiment of the present invention;

[0057] Fig. 9 A diagram of the architecture of a multi-scale foreground segmentation algorithm based on parameter sharing provided by an embodiment of the present invention;

[0058] Fig.10 An improved downsampling block and upsampling block provided by an embodiment of the present invention;

[0059] Fig.11 A display of track anomaly detection results provided by an embodiment of the present invention;

[0060] Fig.12 A sample display of foreground segmentation results provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0062] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or coupling. The term "and / or" used herein includes any unit and all combinations of one or more associated listed items.

[0063] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless defined as herein.

[0064] To facilitate understanding of the embodiments of the present invention, several specific embodiments will be further explained below with reference to the accompanying drawings, and each embodiment does not constitute a limitation on the embodiments of the present invention.

[0065] The embodiment of the present invention provides a method for monitoring track intrusion, which is based on weakly supervised video anomaly detection and unsupervised foreground segmentation technology, aiming to overcome the limitations of existing methods. In view of the contradiction between the accuracy and breadth of anomaly detection, the present invention solves the problem of track intrusion detection by combining video anomaly detection and foreground segmentation, improves the ability to identify unseen or atypical intrusion behaviors, and improves the reliability and practicality of the system.

[0066] The processing flow chart of a track intrusion monitoring method provided by an embodiment of the present invention is as follows: Figure 1 As shown, the processing steps include the following:

[0067] Step S10: data collection.

[0068] The surveillance camera installed beside the track collects the track intrusion video. The collected surveillance video data image is as follows: Figure 2 As shown, the video resolution is 2560*1440 and the frame rate is 25 frames per second.

[0069] Step S20: Data input setting.

[0070] The first is the input setting of the model. In the video anomaly detection based on weak supervision, half of the positive samples (video frames under normal conditions) and half of the negative samples (video frames under abnormal conditions) are selected from the collected surveillance video data images, and the selected surveillance video data is used as the input video data.

[0071] The above design strategy is intended to ensure that the model is fully exposed to abnormal cases during the learning process to accurately identify various abnormal phenomena, while ensuring that normal behaviors are sufficiently representative to prevent the model from misidentifying normal situations as abnormalities. Figure 3 shown.

[0072] Step S30: pre-process the input video data.

[0073] The preprocessing of the video in the embodiment of the present invention includes two processes: image mask cropping and far point information magnification. The specific principles and processes of the two preprocessing processes are described in detail below.

[0074] The first is the image mask cropping process. The embodiment of the present invention masks the range outside the track limit with a mask to reduce the impact of non-detection areas on the results in model detection. In the specific process, the embodiment of the present invention knows that the track spacing between the tracks is 1435mm, and the general track area detection range is 2500mm from the center of the track to both sides. First, the track line is obtained through the track detection algorithm, and then the middle line of the track line is drawn. The track lines on both sides are translated to the outside by the ratio of 2500 / (1435 / 2) to form a limit line, and then the mask is drawn according to the limit line. Finally, according to the ROI (region of interest) area, other local areas are cropped to obtain the final input image video. It is worth noting that by implementing this operation, the abnormal features that were originally tightly coupled with the position can be converted into position-independent abnormal representations, so that the model can still adapt to abnormality detection tasks in various scenarios in the absence of obvious location information clues.

[0075] Figure 4 This is a schematic diagram of the data after masking and cropping. The original image is 2560*1440. After masking and cropping, the image becomes 1790*660. A large number of useless areas are removed, which helps feature extraction to extract more useful information. The input image becomes smaller, which can also improve the running speed of the model to a certain extent.

[0076] The second is the process of amplifying the far-point information. The embodiment of the present invention proposes a strategy of cropping the local image of the far-point area of ​​the track separately and merging it with the uncropped complete image into a set of complementary image samples, the former highlighting the detailed features of the distant target in the current frame, and the latter reflecting the general features of the overall scene.

[0077] The deliberate retention of distant point information is actually equivalent to constructing an additional abstract scene perspective. Despite the cropping operation, from a logical level, this preprocessing step does not weaken the detection accuracy of the model. On the contrary, considering and integrating distant point detail information and global scene information for anomaly detection at the same time significantly improves the detection efficiency of abnormal events.

[0078] Figure 5 This is a schematic diagram of the image after far-point cropping and global retention. The original image is a 1770*660 image, which is processed into two 512*189 images.

[0079] Step S40: weakly supervised video abnormal feature detection.

[0080] Weakly supervised video anomaly detection technology is one step in the phased processing strategy for track foreign object intrusion videos. First, the system classifies the pre-processed video data into two categories, identifying it as "normal" or "abnormal". When the video scene is judged to meet the expected "normal" standard, the detection process is terminated without further processing, thereby optimizing computational efficiency. On the contrary, if the video is marked as "abnormal", it triggers the next stage of analysis - foreground segmentation and abnormal target information location.

[0081] Figure 6 It is a schematic diagram of the overall framework of a weakly supervised learning detection model for track foreign object intrusion based on video surveillance provided by an embodiment of the present invention, including video input settings, a video preprocessing module, a feature extractor pre-trained using a large video data set, an attention improvement module (including a Non-Local attention module for establishing global dependencies and a multi-scale time domain attention module for establishing local time dependencies), a linear classifier module, a TOP-K selection module, and a loss function (including an amplitude score loss function, an amplitude feature loss function, and a time smoothing term loss function).

[0082] The embodiment of the present invention uses the Xception3D (X3D for short) model to extract video features. X3D is a deep neural network model for video understanding proposed by Facebook AI. It is designed to improve the performance of tasks such as video classification, action recognition, and video generation. The core idea of ​​the X3D model is to design an efficient three-dimensional convolutional structure to process the spatiotemporal characteristics of video data.

[0083] The embodiment of the present invention uses Kinetics-400 (K400 for short), a large-scale action recognition benchmark dataset, to pre-train the X3D model, and then transfers it to a weakly supervised learning detection model for track foreign object intrusion based on video surveillance. The K400 dataset covers 400 diverse action categories, such as sports activities, daily life behaviors, and performing arts, each of which includes approximately 400 independent video clips, contributing a total of approximately 240,000 unique video samples with a cumulative duration of more than 320 hours. Pre-training the X3D model on this dataset aims to extract more universal video feature representations, which helps to improve the model's transfer learning performance in various video understanding tasks, not only significantly reducing the time and computing resources required for training from scratch for specific tasks, but also enhancing the model's adaptability to different visual scenes and target tasks.

[0084] The Non-Local Attention Mechanism is a neural network structure used to model long-distance dependencies. It was first proposed by Facebook AI. Its structure is as follows: Figure 7As shown in Figure 1. This network structure is designed to address the limitations of traditional convolutional neural networks in dealing with long-distance dependencies. The core idea is to introduce non-local operations that enable the network to establish connections between any two locations, thereby capturing global contextual information. This operation is based on a key observation: there are rich correlations between pixels in an image.

[0085] Although the Non-Local attention mechanism is effective in strengthening the global dependency between features, local correlation in the temporal dimension is also indispensable in the anomaly detection task. To this end, the embodiment of the present invention introduces a pyramid structure solution, which uses one-dimensional dilated convolution technology to deeply explore the multi-scale expression characteristics of video clips in the temporal dimension. It is worth mentioning that the operation mechanism of this dilated convolution is designed to capture the local temporal correlation under different receptive fields. Its intuitive representation can be seen in Figure 8 Schematic diagram of the multi-scale temporal attention module shown.

[0086] The feature extractor extracts features from the preprocessed video data to obtain feature vectors. After the feature vectors are enhanced by the Non-Local and multi-scale temporal attention mechanisms, optimized feature vectors are obtained. These optimized feature vectors will be passed through the classifier to generate scores corresponding to the video frames. Furthermore, the K scores with the highest absolute values ​​are selected from the scores of normal and abnormal video frames, and the amplitude score loss (Amplitude ScoresLoss) is calculated accordingly. Thereafter, the corresponding feature vectors are extracted according to the frame indexes corresponding to the K highest scores, in order to calculate the feature loss (Amplitude Features Loss) of the K significant features.

[0087] Based on the Top-K selection, the embodiment of the present invention calculates the loss of the selected K scores and the corresponding features. The prototype of the score loss function is the cross entropy loss function, but the input is only the scores of K normal and abnormal video frames, which is specifically expressed as follows.

[0088]

[0089] Where x represents a normal or abnormal video frame, y represents the label of the video where the video frame is located, and represents the abstract representation of the weakly supervised video anomaly detection model operation. The weakly supervised video anomaly detection model is as follows Figure 6 shown.

[0090] In order to increase the degree of distinction between normal and abnormal videos in the loss function, TOP-K is also used to select the features with the largest difference. The feature is generally a vector. The feature can be converted into a positive number through the L2 norm, which can represent the score corresponding to the feature. The following formula represents the L2 norm of the scores of K video frames in the video and the summation.

[0091]

[0092] Next, the sum of the feature norms of the normal K video frames in the normal video is subtracted from the sum of the feature norms of the abnormal K video frames, as shown in the following formula.

[0093] d(X + , X - ) = g k (X + )-g k (X - )

[0094] Finally, the feature amplitude loss function is defined as follows, where m is the threshold defined by the implementation to ensure that the loss function is a positive number.

[0095] l s =max(0,md(X i , X j ))

[0096] In addition, normal frames and abnormal frames in a video often appear continuously, that is, the adjacent frames of a normal frame or abnormal frame are also normal frames and abnormal frames. Therefore, the embodiment of the present invention adds a temporal smoothness loss function (SmoothnessConstraint Loss, SCL), so that after multiple trainings, the scores between adjacent normal frames and abnormal frames in a video are relatively close.

[0097]

[0098] Among them, λ is a pre-set hyperparameter that limits the value of the temporal smoothing loss function to a reasonable range, and i represents the index order of the current frame in the video.

[0099] Finally, the loss function consists of three parts: amplitude score loss function, amplitude feature loss function and time smoothing loss function. The specific formula is shown below.

[0100]

[0101] The video frames whose total loss function values ​​exceed the set threshold are judged as abnormal video frames, and the video frames whose total loss function values ​​do not exceed the set threshold are judged as normal video frames.

[0102] Step S50: foreground segmentation and abnormal target positioning.

[0103] After weakly supervised video anomaly detection, foreground segmentation and abnormal target localization follow closely, and their operation is based on the video frame image data that has been identified as "abnormal" in the previous step. The core input of this process is the video image sequence that has been initially screened and determined to be abnormal. The purpose is to produce two key outputs through in-depth analysis of these images: one is the background component information of the corresponding image, and the other is the mask image information used to identify the prominent abnormal objects in the video frame. This mask image is crucial for accurately locating the geometric boundaries of abnormal targets. This step aims to accurately locate the location of foreign body intrusion in the video frame, extract key abnormal area information, and provide accurate basis for timely intervention and response. Through this hierarchical processing mechanism, it ensures a rapid response to potential threats and avoids unnecessary waste of resources.

[0104] The start condition of this subsequent processing step is clear, that is, only when the weakly supervised video anomaly localization step determines that the video content violates the normal mode and confirms its abnormal attributes, the foreground segmentation and abnormal target localization process is activated to achieve accurate definition of the spatial position of the abnormal entity. On the contrary, if the previous analysis concludes that the video content is in line with the norm (that is, the "normal" category), this link is skipped directly, avoiding unnecessary resource consumption and ensuring the efficient and accurate operation of the entire detection system. This strategy not only reflects the rational allocation of computing resources, but also deeply demonstrates the hierarchical processing logic from macro anomaly identification to micro target precise positioning.

[0105] Fig. 9 The architectural design of a multi-scale track anomaly localization model based on a parameter sharing mechanism is presented. The uniqueness of this model lies in that it divides the encoder and decoder structures into three progressive levels. In the first two levels, two layers of down / up sampling units are configured. Through this series of operations, low-dimensional variance and mean feature vectors are extracted and generated to construct a latent representation space. The third level integrates a layer of conventional up / down sampling modules and a layer of up / down sampling modules that are dynamically enabled as needed to generate higher-dimensional latent variables. These latent variables at different levels each capture image features of different scales, from fine-grained to coarse-grained.

[0106] By adopting this multi-level variational autoencoder architecture, the model effectively integrates the potential image feature information at all scales. It is worth noting that although the model actually exhibits the characteristics of three independent VAEs (Variational Autoencoders) operating in parallel, thanks to the parameter sharing design, its overall parameter volume is close to that of a single top-level complexity VAE, thus achieving efficient parameter reuse between the VAE structures between the low-level and high-level layers.

[0107] The encoder and decoder structures both consist of five consecutive downsampling or upsampling units. The design of this series of units is as follows: Fig.10 As shown, each process module integrates multiple key components. Specifically, each sampling module integrates a spatial position encoding mechanism to capture local to global spatial information; then, a deep separable convolution is used (a downsampling operation in the encoding stage, and an upsampling operation in the decoding stage) to achieve efficient feature extraction and reconstruction; the subsequent point convolution or deconvolution layer further enhances the expressiveness of the feature map; at the same time, all convolution operations are supplemented by batch normalization steps to stabilize the training process and optimize model convergence; finally, the core calculation link of each module uses an activation function to enhance the effect of nonlinear transformation and the learning ability of the network. In addition, in order to enhance the model's attention to the distinction of each convolution channel feature and enhance the robustness to illumination noise, the embodiment of the present invention further introduces the ECA (Efficient Channel Attention) mechanism into the network structure.

[0108] The multi-scale VAE architecture contains latent variables of multiple scales (or levels), each of which has a corresponding mean and variance, and they all affect the KL divergence loss. In the objective function of the variational autoencoder (VAE), the KL divergence is an indicator used to measure the difference between the approximate posterior distribution learned by the model and the prior distribution. For a multi-scale hierarchical VAE, the total KL divergence can be defined as follows:

[0109]

[0110] Here, z l represents the latent variable of the lth scale level, L is the total number of scale levels, q(z l |x, z<l) is a given observation data x and a low-level latent variable z <l When , the approximate posterior distribution of the high-level latent variable z, and p(z l |z <l ) is the corresponding prior distribution.

[0111] The multi-scale track anomaly localization model architecture receives as input anomaly video frames filtered by an anomaly detection algorithm, and then uses a deep neural network architecture to predict static image features that constitute the video background. This prediction process involves the reconstruction of background image information to capture normal visual patterns in the scene. By comparing the input anomaly video frame with the background image generated by the model at a pixel level, a difference map between the two can be calculated, the so-called foreground mask. This mask highlights areas with large expected changes relative to the background, which are often associated with potential abnormal activities. It is worth noting that activations at different positions on the mask correspond to multiple independent anomaly targets in the video, thus enabling parallel recognition and localization of multiple anomaly types.

[0112] In order to enhance the adaptability and generalization ability of the model to input images of different sizes, the embodiment of the present invention adopts a flexible strategy when designing the model, that is, configuring the corresponding number of sampling layers according to the input images of different sizes. Specifically, for images of smaller sizes (for example, less than or equal to 500x500 pixels), considering the effective use of computing resources and preventing overfitting, it is selected to set 5 layers of sampling blocks, and the number of convolution channels is configured as [3, 64, 160, 160, 32, 16]. This channel number allocation can ensure that the model fully captures image features with a limited number of parameters, thereby achieving efficient and accurate feature extraction.

[0113] For larger images, since they contain more spatial information and potential details, one more sampling block is added to 6 layers, and the corresponding number of convolution channels is set to [3, 64, 160, 160, 160, 32, 16]. This design allows the model to maintain a low computational complexity when processing large-size images, and also to more deeply mine and integrate image features, further improving the performance and robustness of the model in complex scenes.

[0114] In summary, by flexibly using a multi-layer sampling block structure and combining it with fine-tuning the number of convolution channels for images of different sizes, the model can effectively enhance the adaptability and generalization performance of input images of various sizes while ensuring learning efficiency, thereby showing better recognition and processing effects in practical applications.

[0115] In summary, compared with the current advanced methods, the embodiments of the present invention innovatively integrate weak supervision and unsupervised learning strategies to deal with the intelligent identification of track intrusion abnormalities and the precise positioning of abnormal targets, which significantly simplifies the data set construction and annotation process and effectively alleviates the annotation complexity problem. More importantly, the present invention demonstrates excellent generalization ability and can autonomously discover potential threats in abnormal instances that are not clearly labeled, greatly broadening the detection scope and applicability.

[0116] In addition, the present invention significantly reduces the model parameter scale and computational burden through a carefully designed model lightweight strategy, greatly improving the feasibility and efficiency of application in mobile deployment scenarios with limited computing resources. This optimization not only promotes the efficient use of resources, but also paves the way for the miniaturization and portable deployment of real-time monitoring systems.

[0117] In terms of performance indicators, the present invention achieves a dual improvement in detection accuracy and missed detection rate, ensuring high accuracy and reliability in monitoring track intrusion events, and contributing a more accurate, efficient and adaptable solution to the field of rail transit safety monitoring.

[0118] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0119] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present invention can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention or certain parts of the embodiments.

[0120] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0121] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for monitoring track intrusion, It is characterized in that include: The monitoring video data of the track is collected by the monitoring equipment installed beside the track, and the monitoring video data is pre-processed to obtain the input video data; Performing weakly supervised abnormal feature detection on the input video data to obtain abnormal data; Abnormal target positioning is performed on the abnormal data to obtain track intrusion detection results.

2. The method according to claim 1, It is characterized in that The monitoring video data of the track is collected by the monitoring equipment installed beside the track, and the monitoring video data is preprocessed to obtain the input video data, including: Track intrusion videos are collected by surveillance cameras installed beside the track. Half of the positive samples and half of the negative samples are selected from the collected surveillance video data images. The positive samples are video frames under normal conditions, and the negative samples are video frames under abnormal conditions. The selected surveillance video data are preprocessed, and the preprocessing includes image mask cropping and far-point information magnification processes. The preprocessed surveillance video data is used as input video data.

3. The method according to claim 2, It is characterized in that The selected surveillance video data is preprocessed, and the preprocessing includes image mask cropping and far point information amplification process, including: The track spacing between tracks is set to 1435mm, and the track area detection range is set to 2500mm from the center of the track to both sides. The track line is obtained through the track detection algorithm, and the middle line of the track line is drawn. The track lines on both sides are translated to the outside by the ratio of 2500 / (1435 / 2) to form a limit line. The mask is obtained according to the limit line, and other local areas are cropped according to the region of interest ROI. The local image of the far point area of ​​the track is cropped separately and merged with the uncropped complete image into a set of complementary image samples to obtain the final input image video.

4. The method according to claim 2 or 3, It is characterized in that The step of performing weakly supervised abnormal feature detection on the input video data to obtain abnormal data includes: The X3D model is pre-trained using Kinetics-400, and features are extracted from the input video data using the trained X3D model to obtain feature vectors. The feature vectors are enhanced using Non-Local and multi-scale temporal attention mechanisms to obtain optimized feature vectors. The optimized feature vectors are scored using a classifier, and K scores with the highest absolute values ​​are respectively selected from the scores of normal and abnormal video frames, and corresponding K feature vectors are extracted according to the frame indexes corresponding to the K highest scores. The amplitude score loss function, amplitude feature loss function and time smoothing loss function are calculated according to the selected K scores and the corresponding feature vectors. The calculation method of the amplitude score loss function is as follows: Where x represents a normal or abnormal video frame, y represents the label of the video where the video frame is located, and f represents the model operation; Take the L2 norm of the K video frame scores of the normal class and the abnormal class and sum them up: Where N is the number of video frames, Ω k (X) is a set of K video frames, ||f(x i )|| 2 represents the L2 norm of the score of the i-th video frame; Subtract the sum of the feature norms of the normal K video frames from the sum of the feature norms of the abnormal K video frames, as shown in the following formula: d(X + ,X - )=g k (X + )-g k (X - ) Among them, g k (X + ) represents the L2 norm and g of the normal video frame score. k (X - ) represents the L2 norm sum of the abnormal video frame scores, which represents the degree of distinction between normal and abnormal video frames; The calculation method of the amplitude feature loss function is as follows: l s =max(0,m-d(X i ,X j )) m is the threshold defined by the implementation to ensure that the loss function is a positive number; d(X i , X j ) is the function representing the degree of distinction between normal and abnormal video frames mentioned above; By adding a time smoothing loss function, the scores between adjacent normal frames and abnormal frames in a video are close after multiple trainings. The calculation method of the time smoothing loss function is as follows; Among them, λ is a pre-set hyperparameter, i represents the index order of the current frame in the video, and f(x i ) represents the score of the i-th video frame, f(x i+1 ) represents the score of the i+1th video frame, and N represents the number of video frames; The specific formula of the total loss function is as follows: The video frames whose total loss function values ​​exceed the set threshold are judged as abnormal video frames, and the video frames whose total loss function values ​​do not exceed the set threshold are judged as normal video frames.

5. The method according to claim 4, It is characterized in that The abnormal target positioning of the abnormal data and obtaining the track intrusion detection result includes: Construct a multi-scale track anomaly localization model architecture based on parameter sharing mechanism. The multi-scale track anomaly localization model architecture divides the encoder and decoder structure into three progressive levels. Two layers of down / up sampling units are configured in the first two levels, and a conventional up / down sampling module and an up / down sampling module that is dynamically enabled as needed are integrated in the third level. The video frames judged as abnormal are input into the multi-scale track anomaly positioning model architecture. The encoder and decoder structure in the multi-scale track anomaly positioning model architecture includes five consecutive downsampling or upsampling units. Each sampling module integrates a spatial position encoding mechanism. It is a downsampling operation in the encoding stage and is converted to an upsampling operation in the decoding stage to capture local to global spatial information. The point convolution or deconvolution layer enhances the expressiveness of the feature map. All convolution operations are supplemented by batch normalization steps. The core calculation link of each module adopts an activation function, and the corresponding number of sampling layers are configured according to the input images of different sizes. An effective channel attention ECA mechanism is set in the multi-scale track anomaly localization model architecture. The multi-scale VAE architecture contains latent variables of multiple scales. Each latent variable of each scale has a corresponding mean and variance. The total KL divergence can be defined as follows: Here, z l represents the latent variable of the lth scale level, L is the total number of scale levels, q(z l |X, z<l) is a given observation data x and low-level latent variable z <l When , the approximate posterior distribution of the high-level latent variable z, and p(z l |z <l ) is the corresponding prior distribution; The multi-scale track anomaly localization model architecture receives abnormal video frames screened out by the anomaly detection algorithm as input, and uses a deep neural network architecture to predict the background image that constitutes the video background. By comparing the abnormal video frame with the background image at the pixel level, a difference map between the abnormal video frame and the background image, namely the foreground mask, is calculated. The foreground mask highlights the abnormal activity areas with large expected changes relative to the background. The activations at different positions on the foreground mask correspond to multiple independent abnormal targets in the video.

Citation Information

Patent Citations

  • Anomaly detection method and device, computer equipment and medium

    CN112766429A

  • Weak supervision video violence detection method based on time domain enhancement and comparative learning

    CN118015507A

  • Hyperspectral abnormal target detection method based on multi-scale analysis and variational auto-encoder

    CN118279747A

  • KR20240063363A

Cited By

  • Data processing method based on edge AI calculation and storage integrated chip and related equipment

    CN119762897A

  • Weak supervision video anomaly detection method and system based on multi-head spectrum residual gating

    CN121963060A