A method and system for airport crossing safety management based on video capture

Abstract

This application proposes a method and system for airport crossing safety management based on video capture. The method uses video surveillance equipment to capture crossing image data in real time, pre-processes the data and performs dynamic target recognition processing to generate the motion trajectories of the first and second objects, and assigns an independent ID to each target. The method outputs a dynamic detention warning signal through trajectory filtering and abnormal stationary target recognition, and screens and matches the trajectory of the second object to identify trajectory anomalies and trigger alarm signals and associated response actions. This technical solution solves the problem that crossing safety management relies on manual monitoring and is unable to monitor multi-target interactions and complex behaviors in real time, effectively improving the safety monitoring capabilities and management efficiency of airport crossings.

Description

Technical Field

[0001] The present invention relates to the field of road crossing safety management, and in particular to a method and system for airport road crossing safety management based on video capture. Background Art

[0002] In modern airport operations, crossing safety management is crucial for ensuring smooth operations and passenger safety. As primary access points for vehicles and personnel, a safety incident at an airport crossing can severely impact the airport's operations and even threaten the lives of passengers and staff. To prevent unauthorized access by tailgating vehicles, unauthorized personnel entering restricted areas, and other potential safety hazards, enhanced monitoring and management of crossing areas is essential. Therefore, establishing an effective airport crossing safety management system capable of real-time monitoring, identification, and early warning of potential security threats is crucial for improving the airport's overall safety and security capabilities.

[0003] In existing technologies, the safety management of airport crossings primarily relies on an inspection method that combines video surveillance systems with manual inspections, with manual inspections being the primary method. By installing surveillance cameras, staff can monitor the crossing area to promptly detect abnormal behavior or potential safety hazards. However, surveillance footage typically requires real-time manual review, and the crossing's wide monitoring area and frequent vehicle and personnel movements make it difficult to achieve full coverage and real-time response due to limited human resources. In addition, due to the limited number of staff and the gradually aging age structure of inspectors, the energy and reaction speed of manual inspections are affected. Parts of airport crossings have blind spots and blind spots, making safety inspections susceptible to human factors and making it difficult to achieve real-time analysis and accurate judgment of multi-target behaviors. This management model, which relies primarily on manpower, is prone to delayed warnings or missed reports when faced with complex interactive behaviors, affecting the overall safety management level of the crossing.

[0004] In summary, there are still many problems in the existing technology for airport crossing safety management, especially in the dynamic monitoring of multiple targets and the identification of abnormal behaviors in the crossing area. Although the existing monitoring system can cover the crossing to a certain extent, due to the lack of accurate identification and interaction analysis of multiple targets, the monitoring system cannot handle the complex behavior patterns between multiple targets at the same time. In particular, when there are behaviors such as multi-target interaction and target retention, it is often difficult to identify and issue early warning signals in a timely manner. In addition, during manual inspections, due to limited monitoring personnel and the inability to monitor the monitoring screen around the clock, it is impossible to detect tailgating violations in the blind spots of large vehicles (such as long lanes), which increases safety hazards. Therefore, the existing technology cannot provide comprehensive, accurate and efficient safety management of the airport crossing area, and a solution is urgently needed to solve the problem of dynamic monitoring of multiple targets and identification of abnormal behaviors. Summary of the Invention

[0005] In view of the above actual situation, this application proposes an airport crossing safety management method and system based on video capture to solve the problems existing in the existing technology of being unable to simultaneously track the interactions of multiple targets, complex behavior pattern analysis and monitoring blind spots, especially the safety hazards of being unable to effectively identify multi-target dynamic behaviors, tailgating violations and target retention behaviors.

[0006] A method for airport crossing safety management based on video capture, the method comprising the following steps:

[0007] S1, obtaining real-time data to be processed, wherein the real-time data to be processed is video crossing image data captured in real time by a video surveillance device;

[0008] S2, preprocessing the real-time data to be processed to obtain real-time preprocessed data;

[0009] S3 performs recognition processing on the real-time pre-processed data for the first object and the second object, generates dynamic data of the first object and the second object, models the dynamic data to generate respective motion trajectories, and assigns an independent ID to each object;

[0010] S4, performing trajectory filtering processing on the movement trajectory of the first object under the unique ID, identifying abnormal stationary targets and outputting a dynamic retention warning signal;

[0011] S5, when outputting the dynamic detention warning signal, screening and matching analysis are performed on the movement trajectory of the second object, the trajectory anomaly is identified and the abnormal behavior alarm signal and the associated response action are triggered.

[0012] Furthermore, the step S2 includes the following sub-steps:

[0013] S201, performing a frame extraction operation on the real-time data to be processed, decomposing the continuous video picture into a discrete frame sequence and performing normalization processing to obtain discrete normalized data, performing Gaussian filtering on the discrete normalized data to remove noise, and generating denoised frame data;

[0014] S202, performing background modeling and foreground segmentation processing on the denoised frame data, extracting the motion foreground area using a convolutional neural network segmentation algorithm to generate foreground segmentation data, adding a timestamp to each frame of the foreground segmentation data, and generating real-time preprocessed data.

[0015] Furthermore, the first object recognition process in step S3 includes the following sub-steps:

[0016] S301, performing spatiotemporal domain separation on the foreground segmentation data in the real-time preprocessed data, filtering the motion features of the foreground area using an adaptive spatiotemporal filtering algorithm to generate motion pattern data, wherein the spatiotemporal domain separation is performed by combining a convolutional neural network and LSTM;

[0017] S302, applying a partitioning processing algorithm based on tensor decomposition based on the motion pattern data to dynamically construct a target motion attribute model within the region, thereby generating first object dynamic data; the partitioning processing algorithm is implemented by local maximum clustering and adaptive thresholding algorithm;

[0018] S303: Perform motion trajectory modeling on the first object's dynamic data using an optimized Kalman filter algorithm based on time-frequency joint analysis to generate a movement trajectory of the first object and assign it an independent ID. The trajectory modeling adopts a fusion of weighted mean filtering and a multi-dimensional Gaussian model.

[0019] Furthermore, the recognition process of the second object in step S3 includes the following sub-steps:

[0020] S311, generating a local saliency map for the foreground segmentation data in the real-time preprocessed data, and expanding and contracting the object region using a boundary tracking method based on a spatial-temporal composite optimization algorithm to extract second object candidate region data;

[0021] S312, performing hierarchical feature extraction on the second object candidate region data in combination with a deep autoencoder neural network, generating second object dynamic data using a dynamic flow field model and a motion clustering algorithm, wherein the feature extraction is jointly processed using a graph convolutional network and a motion prediction algorithm;

[0022] S313: Apply a multi-scale convolutional neural network based on spatiotemporal correlation analysis to the dynamic data of the second object to perform time series modeling, generate a movement trajectory of the second object, and assign an independent ID to it.

[0023] Furthermore, the S4 step includes the following sub-steps:

[0024] S401, performing a time series analysis on the movement trajectory of the first object with the unique ID, calculating the static duration in each time period, and comparing it with a preset baseline motion model to obtain static behavior deviation data;

[0025] S402: Accumulate the stationary behavior deviation data under the unique ID, calculate the cumulative deviation values ​​for multiple time periods, and generate a deviation change rate based on the cumulative deviation changes in adjacent time periods. The deviation change rate is generated by differential analysis and a weighted moving average algorithm.

[0026] S403: constructing a spatiotemporal change trend model of the target's stationary behavior based on the accumulated deviation value and the deviation change rate under the unique ID, wherein the model is modeled using a spatiotemporal convolutional network to obtain a change trend of the target's stationary behavior;

[0027] S404: Perform causal reasoning analysis on the spatiotemporal change trend model under the unique ID to generate target abnormal behavior determination data, and perform comprehensive analysis in combination with a multi-objective decision network to output a dynamic detention warning signal.

[0028] Furthermore, the step S5 includes the following sub-steps:

[0029] S501, when a dynamic detention warning signal is output, the movement trajectory of the second object is screened based on the independent ID of the second object and the spatiotemporal constraint model. The spatiotemporal constraint model uses dynamic spatiotemporal window and spatial neighborhood cluster analysis to screen out the difference dataset between trajectory anomalies and normal behavior trajectories to generate a valid target dataset;

[0030] S502, performing similarity comparison on the valid target dataset using an adaptive trajectory similarity matching algorithm, comparing the valid target dataset with a preset normal action trajectory template, and generating a matching analysis result;

[0031] S503: Process the matching results based on the spatiotemporal deviation fusion analysis model, identify when the deviation from the preset trajectory exceeds a threshold, output an abnormal behavior alarm signal, and trigger an associated response action.

[0032] Furthermore, the temporal modeling process in S313 includes cross-frame motion feature extraction, spatiotemporal correlation modeling, trajectory generation, and ID allocation.

[0033] Furthermore, the time series analysis in step S401 adopts a dynamic time warping algorithm.

[0034] Furthermore, the similarity comparison in step S502 includes feature extraction processing and similarity calculation processing. The similarity calculation processing is performed using an adaptive trajectory similarity matching algorithm. The adaptive trajectory similarity matching is calculated using a dynamic time warping algorithm. The spatiotemporal deviation fusion analysis model in step S503 includes deviation value extraction processing and spatiotemporal fusion calculation processing.

[0035] In addition, the present application also proposes an airport crossing safety management system based on video capture, characterized in that the system includes:

[0036] An acquisition unit is used to acquire real-time data to be processed, wherein the real-time data to be processed is video crossing image data captured in real time by a video surveillance device;

[0037] A preprocessing unit, used for preprocessing the real-time data to be processed to obtain real-time preprocessed data;

[0038] an identification unit, configured to perform identification processing on the first object and the second object based on the real-time preprocessed data, generate dynamic data of the first object and the second object, perform modeling based on the dynamic data to generate respective motion trajectories, and assign an independent ID to each object;

[0039] a detention detection unit, configured to perform trajectory filtering processing on the movement trajectory of the first object under the unique ID, identify abnormal stationary targets and output a dynamic detention warning signal;

[0040] The anomaly detection unit is used to screen and match the movement trajectory of the second object when outputting the dynamic detention warning signal, identify the trajectory anomaly and trigger the abnormal behavior alarm signal and associated response action.

[0041] The present application proposes a method and system for airport crossing safety management based on video capture, which realizes real-time monitoring and behavior trajectory analysis of multiple targets, can effectively identify abnormal stationary and tailgating violations, and solves the problems of blind spots in monitoring and limited personnel by triggering dynamic detention warnings and abnormal behavior alarm signals, thereby improving the real-time, accuracy and response efficiency of crossing safety management. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram of the method flow of a video capture-based airport crossing safety management method proposed in this application;

[0043] Figure 2 This is a schematic diagram of the process of pre-processing data in the airport crossing safety management method based on video capture proposed in this application;

[0044] Figure 3 This is a flowchart of the first object recognition process in the airport crossing safety management method based on video capture proposed in this application;

[0045] Figure 4 This is a flow chart of the second object recognition process in the airport crossing safety management method based on video capture proposed in this application;

[0046] Figure 5 This is a flow chart of outputting dynamic detention warning signals in a video capture-based airport crossing safety management method proposed in this application;

[0047] Figure 6 This is a flow chart of triggering abnormal behavior alarm signals and associated response actions in a video capture-based airport crossing safety management method proposed in this application;

[0048] Figure 7 A schematic diagram of the structure of a system for reversely predicting production parameters based on tar content provided in an embodiment of the present application; DETAILED DESCRIPTION

[0049] The following will clearly and completely describe the simulation technology route in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0050] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0051] The features and performance of the present invention are further described in detail below with reference to the embodiments. Figure 1 As shown, a road crossing safety management method based on video capture comprises the following steps:

[0052] S1, obtaining real-time data to be processed, wherein the real-time data to be processed is video crossing image data captured in real time by a video surveillance device;

[0053] In this embodiment, this step involves acquiring real-time data to be processed, specifically capturing crossing scene data in real time using video surveillance equipment. The data to be processed originates from cameras or video surveillance devices installed near the crossing. The video surveillance equipment continuously captures video images within the crossing area, converts the captured image data into electronic signals, and transmits them to a processing unit in real time. The video surveillance equipment provides comprehensive coverage within the crossing area, fully capturing dynamic conditions within the crossing and ensuring comprehensive scene data capture.

[0054] In some embodiments, the video surveillance equipment utilizes a high-definition digital video capture device with high resolution and real-time processing capabilities. This device is capable of capturing real-time image information of the road crossing at a specific frame rate, which can be adjusted based on demand, typically reaching 30 frames per second or higher. The video surveillance equipment then processes the continuously captured image signals according to a specific video coding standard, such as H.264 or H.265, and then compresses and transmits them to a data processing platform.

[0055] The video crossing image data contains not only static background information about the crossing area, but also various dynamically changing information. This information is transmitted to the processed data stream via video surveillance equipment, forming the core data source for this step. The processed data includes each video frame and the timestamp information for each frame. The timestamp identifies the moment each frame was captured, ensuring the continuity and accuracy of the data in the time series.

[0056] In this embodiment, to ensure data accuracy and real-time performance, the video surveillance equipment utilizes photoelectric sensing technology for image acquisition. The sensor senses light signals from the access area and converts them into electrical signals. This electrical signal is decoded and formatted by the image processing system, ultimately generating digital image data. In some embodiments, the video surveillance equipment utilizes a CMOS or CCD image sensor. These sensors have high photosensitivity and can maintain high image quality even in low-light environments, ensuring clear images under varying lighting conditions.

[0057] The data to be processed is not limited to static image data but also includes motion information extracted from each frame. During video capture, the device can extract information about moving targets within the crossing area through real-time difference analysis or motion detection algorithms, providing more detailed dynamic data. This data provides the basis for subsequent processing such as target recognition and behavior analysis.

[0058] In some implementations, the real-time data to be processed undergoes preliminary processing in a data buffer and storage unit to ensure that no data is lost during the processing process. Video surveillance equipment has a certain amount of memory capacity for storing captured video frames and caching them for a set duration. This process helps ensure the continuity of the data flow and avoids data omissions or loss during high-speed acquisition.

[0059] By capturing real-time video footage of crossings, the processed data not only possesses real-time capabilities but also includes complete temporal and spatial information, providing a reliable basis for subsequent image analysis, target recognition, and behavioral assessment. The data processing process, based on the real-time capture, transmission, and processing of video signals, ensures the system's ability to monitor crossing conditions in real time and dynamically respond.

[0060] S2, preprocessing the real-time data to be processed to obtain real-time preprocessed data;

[0061] For details, please refer to the attached Figure 2 As shown, this step includes the following sub-steps:

[0062] S201, performing a frame extraction operation on the real-time data to be processed, decomposing the continuous video picture into a discrete frame sequence and performing normalization processing to obtain discrete normalized data, performing Gaussian filtering on the discrete normalized data to remove noise, and generating denoised frame data;

[0063] In some embodiments, the frame extraction operation is implemented using a sampling algorithm that extracts each frame from a continuous video stream at a set frame rate. The frame rate is set based on the actual application scenario and processing requirements; in this embodiment, it is 30 frames per second or higher. The extracted image data for each frame is stored in a two-dimensional matrix format, where each matrix element corresponds to a pixel value at a specific point in the image.

[0064] Normalization is performed on the extracted discrete frame data to convert the pixel values ​​of each frame into a uniform numerical range, thereby improving the stability and accuracy of subsequent processing. In an embodiment, through normalization, the pixel values ​​of the image will be adjusted to the range of [0, 1] to ensure that the image is not biased during processing due to differences in pixel value ranges.

[0065] In this embodiment, the normalized discrete data is further subjected to Gaussian filtering to remove noise from the image. Noise often originates from environmental interference or sensor limitations, which can affect the accuracy of target detection and image analysis. Specifically, the following two-dimensional Gaussian kernel function is used to perform convolution processing on the image: Where (x, y) is the position within the filter kernel, and σ is the standard deviation of the Gaussian kernel, which controls the smoothness of the filter. A Gaussian filter reduces the impact of noise on image details by performing a weighted average of each pixel and its neighborhood. During implementation, the filter size and standard deviation σ are adjusted based on the type and intensity of noise in the image.

[0066] In this embodiment, Gaussian filtering is implemented by the following convolution formula: Among them I x,y is the pixel value at position (x, y) in the original image, G(i, j) is the filter weight at position (i, j) in the Gaussian kernel, and k is the radius of the filter. Through this convolution operation, noise in the image is effectively suppressed, and the resulting denoised frame data has a good smoothing effect, which helps improve the accuracy of subsequent image analysis tasks.

[0067] Through frame extraction, normalization, and Gaussian filtering, the resulting denoised frame data is clearer and less noisy than the original image. The denoised image data provides high-quality input data for subsequent processing such as target recognition and behavioral analysis. In some embodiments, the denoised data can be further subjected to other image enhancement processes, such as edge detection or brightness adjustment, to further improve image quality.

[0068] S202, performing background modeling and foreground segmentation processing on the denoised frame data, extracting the motion foreground area using a convolutional neural network segmentation algorithm to generate foreground segmentation data, adding a timestamp to each frame of the foreground segmentation data, and generating real-time preprocessed data.

[0069] In this embodiment, this step performs background modeling and foreground segmentation on the denoised frame data. The purpose of background modeling is to distinguish between the static background and dynamic foreground in the image, and then extract moving objects or changing regions, which represent the moving targets in the scene. In some embodiments, background modeling is implemented using statistical modeling methods, comparing the value of each pixel in the image with the background model to determine whether it belongs to the background.

[0070] Background modeling uses a segmentation algorithm based on a convolutional neural network (CNN). In this embodiment, the convolutional neural network automatically learns features from the denoised frame data through deep learning, and performs pixel-level classification to distinguish between foreground and background. The core structure of the convolutional neural network includes multiple convolution layers, activation function layers, pooling layers, etc., and each layer learns different spatial hierarchical features in the image. Through multi-layer convolution operations, the convolutional neural network can effectively extract local features in the image and map the features to the output space through the fully connected layer to achieve separation of foreground and background.

[0071] In some embodiments, the training process of a convolutional neural network is based on a labeled dataset and optimized using a backpropagation algorithm. The goal of the training is to minimize the difference between the predicted label and the true label, and the cross-entropy loss function is usually used to measure the accuracy of the output. The specific loss function is as follows: Where N is the total number of pixels in the image, y i is the true label, indicating whether the pixel is foreground (1 for foreground, 0 for background), The foreground probability predicted by the network.

[0072] After network training is complete, the resulting convolutional neural network is applied to each denoised image frame to extract the moving foreground region. The convolutional neural network distinguishes the dynamic portion (i.e., foreground region) from the static portion (i.e., background region) in each frame through pixel-by-pixel classification, generating foreground segmentation data. The foreground region typically represents the target object in the scene, whose motion information is tracked and analyzed in subsequent processing steps.

[0073] In some embodiments, to further enhance foreground segmentation accuracy, spatiotemporal constraints are employed to optimize foreground segmentation results. By introducing temporal continuity constraints, the foreground segmentation algorithm can identify the consistency of moving objects across multiple image frames, reducing mis-segmentation caused by illumination variations or temporary interference. In some embodiments, foreground segmentation data undergoes temporal smoothing to further remove mis-segmented areas caused by motion blur or occasional noise.

[0074] In addition, each frame of foreground segmentation data is timestamped to record the moment the image was captured. This timestamp provides timing information for each frame, facilitating subsequent object tracking and behavioral analysis. Timestamps are typically automatically generated by video surveillance equipment and stored with each frame. In some implementations, the accuracy of the timestamp matches the video frame rate, ensuring precise temporal alignment of the data.

[0075] The generated real-time pre-processed data includes denoising, segmentation, and timestamp information, including the foreground regions of the image and their temporal characteristics. This data provides accurate and structured input data for subsequent target recognition, target tracking, and behavior analysis.

[0076] S3 performs recognition processing on the real-time pre-processed data for the first object and the second object, generates dynamic data of the first object and the second object, models the dynamic data to generate respective motion trajectories, and assigns an independent ID to each object;

[0077] Specifically, the data to be processed is the dynamic part that has been filtered, that is, there are moving areas in the image, but the motion patterns within these dynamic areas are diverse. In order to better identify and track the different characteristics of these objects in the complex airport crossing safety management environment, the present application designs a first object and a second object. In this embodiment, the first object represents a vehicle with a relatively regular and predictable motion pattern, such as a ground transportation vehicle, specifically including but not limited to passenger service vehicles (shuttle buses, boarding vehicles), cargo and baggage handling vehicles (luggage trailers, cargo lifting platform vehicles), aircraft service vehicles (food trucks, fuel trucks, water trucks, power supply trucks), ground support vehicles (tractors, guide vehicles), and other special vehicles (apron bird-repelling vehicles, tractors); the second object represents dynamic and irregular targets, such as people and animals. Therefore, the dynamic foreground part is processed differently to ensure accurate identification and efficient tracking of different types of targets, which helps to improve the safety and response efficiency of the system.

[0078] For details, please refer to the attached Figure 3 As shown, the first object recognition process includes the following sub-steps:

[0079] S301, performing spatiotemporal domain separation on the foreground segmentation data in the real-time preprocessed data, filtering the motion features of the foreground area using an adaptive spatiotemporal filtering algorithm to generate motion pattern data, wherein the spatiotemporal domain separation is performed by combining a convolutional neural network and LSTM;

[0080] In some embodiments, this step performs spatiotemporal separation on the foreground segmentation data in the real-time preprocessed data, thereby filtering and extracting dynamic data that meets the target requirements to generate motion pattern data. In this embodiment, the spatiotemporal separation is performed by combining a convolutional neural network with a long short-term memory network to effectively distinguish different motion patterns and filter out irrelevant dynamic features, thereby optimizing subsequent target recognition and trajectory modeling.

[0081] In some embodiments, foreground segmentation data is pixel-by-pixel classified using a convolutional neural network, separating the dynamic portion from the static background. At this point, the foreground data still contains a variety of dynamic patterns. To this end, the spatiotemporal domain separation algorithm extracts motion pattern data by analyzing the spatiotemporal features of the foreground region. The basic idea behind the spatiotemporal domain separation process is to combine the spatial features extracted by the CNN with the time series features processed by the LSTM to accurately filter the motion pattern data from both the temporal and spatial dimensions.

[0082] In this embodiment, the convolutional neural network is responsible for extracting spatial features from the image. These features represent the local information in the foreground area. Through a series of convolution operations, the CNN decomposes the input foreground image into multiple feature maps, each of which represents the characteristic expression of the image in a certain dimension. Let the input image be I(x, y, t), where x and y are spatial coordinates and t is the time dimension. The convolutional neural network obtains the feature map F(x, y, t) through the convolution operation. This feature map contains the spatial motion pattern characteristics: Where K(m,n) is the convolution kernel, and m,n are the spatial positions of the convolution kernel.

[0083] The extracted spatial features will be used as the input of LSTM to model time series information. The role of the LSTM network is to capture the temporal changes of the motion pattern in the foreground area and identify the motion behavior of the target in different time frames. The core principle of LSTM is to control the information flow through the gating mechanism (input gate, forget gate and output gate) to model the temporal dependency in the foreground area. Let the input of LSTM be x t , whose output is h t , the state transition equation of LSTM is as follows: t =σ(W f ·[h t-1 ,x t ]+b f ), i t=σ(W i ·[h t-1 ,x t ]+b i ), o t =σ(W o ·[h t-1 ,x t ]+b o ), h t =o t tanh(C t ), where f t For the forget gate, i t is the input gate, is the candidate memory unit, C t is the cell state, o t is the output gate, h t is the output of LSTM. After being processed by the LSTM network, the motion pattern data at each time point can be obtained.

[0084] In some embodiments, the motion pattern data obtained from LSTM is further refined by an adaptive spatiotemporal filtering algorithm. The function of this filtering algorithm is to denoise the motion pattern, remove irrelevant features, and retain those dynamic features related to the target recognition task. The adaptive spatiotemporal filtering algorithm is based on the motion characteristics of the foreground area and its temporal variation law, and updates the filtering effect in real time by dynamically adjusting the filter parameters. The motion pattern data is set to M(x, y, t), and the filtering algorithm uses an adaptive filter to smooth the motion pattern data, thereby improving the accuracy of target recognition. Its mathematical model is: M filtered (x,y,t)=α·M(x,y,t)+(1-α)·M filtered (x, y, t-1), where α is an adaptive weight parameter representing the weighted ratio of the motion pattern data of the current frame to the previous frame. By taking a weighted average frame by frame, the filter can dynamically adjust its response to spatiotemporal changes, thereby extracting motion pattern data that meets the requirements of target recognition.

[0085] After processing through the aforementioned spatiotemporal domain separation and adaptive spatiotemporal filtering algorithm, the dynamic features of the foreground area are effectively extracted and optimized, generating motion pattern data. This data reflects the behavioral characteristics of moving objects in the foreground area, including the target's motion state over different time periods. This motion pattern data not only removes noise but also better highlights the target's motion patterns, providing accurate input data for subsequent target recognition and trajectory modeling.

[0086] S302, applying a partitioning processing algorithm based on tensor decomposition based on the motion pattern data to dynamically construct a target motion attribute model within the region, thereby generating first object dynamic data; the partitioning processing algorithm is implemented by local maximum clustering and adaptive thresholding algorithm;

[0087] In some embodiments, this step dynamically constructs a target motion attribute model within the region based on the motion pattern data by applying a partitioning algorithm based on tensor decomposition, thereby generating first object dynamic data. Specifically, by analyzing and clustering the local features of the motion pattern data, moving targets within the foreground region are identified and their dynamic behavior characteristics are extracted. The partitioning algorithm can transform complex motion pattern data into more structured information, facilitating subsequent target recognition and trajectory modeling.

[0088] In this embodiment, the tensor decomposition method is used to efficiently process multidimensional data structures, thereby providing an effective means for modeling target motion characteristics. The motion pattern data is set to M(x, y, t), which represents the dynamic characteristics at position (x, y) and time t. Since the motion pattern data exhibits the characteristics of spatiotemporal interweaving, using tensors to represent these data can effectively capture the correlation in spatial and temporal dimensions. The tensor representation is: Where M(x,y,t) is the motion pattern data represented by the tensor, a ijk is the decomposition coefficient of the tensor, u i (x), v j (y), w k (t) represents the basis vectors at spatial position x, y and time t respectively. By decomposing the tensor, the feature matrix of each dimension is obtained, thereby revealing the correlation between different dimensions.

[0089] In some embodiments, the feature matrix obtained after tensor decomposition requires further partitioning. To accurately extract the target's dynamic characteristics, a local maximum clustering algorithm and an adaptive threshold algorithm are used to cluster spatial regions, thereby identifying moving targets within the region. In this embodiment, the local maximum clustering algorithm automatically selects regions with significant motion pattern changes based on the local extreme value characteristics of the motion pattern data, and further optimizes the clustering accuracy of the target by setting an adaptive threshold.

[0090] The mathematical model of local maximum clustering finds the area with the largest change in motion pattern by searching for local extreme points of motion pattern data in a continuous region of time and space. Assume that the local area of ​​motion pattern data is R(x,y,t), and its local maximum point P(x0,y0,t0) satisfies the following conditions: M(x0,y0,t0)>M(x,y,t) where N(x0,y0,t0) represents the local neighborhood. This condition means that the point P(x0,y0,t0) is a local maximum point, that is, it has the maximum value in its neighborhood.

[0091] The local maximum clustering algorithm can extract target regions with strong motion characteristics from the spatiotemporal region and generate candidate target region data. An adaptive threshold algorithm is then used to filter out regions that do not meet the target recognition requirements. The adaptive threshold algorithm dynamically adjusts based on the target's motion intensity and background noise, accurately identifying the target region. The dynamic threshold is set to T(x,y,t), which is adaptively adjusted based on the local characteristics of the motion pattern data: T(x,y,t) = α·max(M(x,y,t)) + (1-α)·min(M(x,y,t)), where α is the adjustment coefficient that controls the weighting of the maximum and minimum values. Dynamically adjusting the threshold ensures effective identification of moving targets under varying background conditions.

[0092] Through the above-mentioned partitioning and clustering algorithm, the target's dynamic behavior characteristics can be extracted from the motion pattern data, and first object dynamic data can be generated. The result of the partitioning process is a motion attribute model of the target within a spatiotemporal region. This model reflects the laws of the target's motion in space and time. The mathematical representation of this model is: Where D1(x, y, t) represents the dynamic data of the first object, b ijk is the new tensor decomposition coefficient obtained by local maximum clustering and adaptive threshold algorithm, u i ′(x), v′ j (y), w′ k (t) is the processed spatial and temporal feature matrix.

[0093] S303 , using an optimized Kalman filter algorithm based on time-frequency joint analysis to perform motion trajectory modeling on the first object's dynamic data, generating a motion trajectory of the first object and assigning a unique ID to the first object. The trajectory modeling adopts a fusion of weighted mean filtering and a multi-dimensional Gaussian model.

[0094] In this embodiment, the Kalman filter algorithm is optimized as a joint time-frequency analysis method to more accurately track the changes of dynamic targets when processing their motion data. The first object's dynamic data is set as D1(x, y, t), representing the dynamic changes of the target in the spatiotemporal dimensions. The goal of motion trajectory modeling is to smooth the target's true motion trajectory by filtering the dynamic data.

[0095] The Kalman filter consists of two main components: state update and covariance prediction. In some embodiments, state update is optimized by jointly analyzing time and frequency information. Let the state vector be x(t), which contains the position and velocity information of the target at time t: Where x(t) and y(t) are the position coordinates of the target, and is the target's velocity component. The Kalman filter's state update equation is: x(t) = A·x(t-1) + B·u(t), where A is the state transition matrix, representing the evolution of the target state; B is the control matrix; and u(t) is the control input. In this embodiment, the control input u(t) includes frequency domain information to capture changes in the target's motion frequency.

[0096] In some embodiments, in order to enhance the stability and accuracy of the Kalman filter algorithm, a weighted mean filter is fused with a multidimensional Gaussian model. The weighted mean filter method smoothes the target's trajectory based on the target's relative position and velocity information in the spatiotemporal domain, thereby reducing the error caused by noise. The weighting factor of the weighted mean filter is set to w i (t), which is calculated as follows: where x i (t) is the target state vector at time i, w i (t) is the weight coefficient related to the motion feature at the current time t. This coefficient is obtained by the spatiotemporal distance metric to ensure that the filter can adapt to the motion state of the target.

[0097] The multidimensional Gaussian model is used to describe the probability distribution of target motion. Assuming the target's trajectory follows a Gaussian distribution, the distribution of the target state vector x(t) is expressed as: N(x(t); μ(t), Σ(t)), where μ(t) is the target's expected state vector at time t, and Σ(t) is the covariance matrix of the target state. The Gaussian model can model the target trajectory, helping filtering algorithms better predict the target's future motion state.

[0098] Based on the optimized Kalman filter algorithm described above, the target state vector x(t) is obtained, representing the target's trajectory at different time points. This filtered trajectory information is used to model the target's trajectory and assign a unique ID to each target. Each target's ID is assigned based on its unique characteristics along the trajectory, ensuring accurate differentiation between targets during subsequent trajectory analysis.

[0099] In this embodiment, trajectory modeling is performed by performing time series modeling on the filtered target state vector x(t) to construct the target's continuous motion trajectory. The target ID is assigned based on the uniqueness of its motion trajectory. By comparing the similarity of different target trajectories, a matching algorithm based on trajectory features is used to assign a unique identifier to each target. Specifically, assuming that the trajectory of target Ti is x i (t), the trajectory of the target Tj is x j (t), if the trajectory similarity of two targets meets a certain threshold, they are considered to belong to the same target, otherwise they are assigned different IDs.

[0100] Please continue to refer to the attached Figure 4 As shown, the second object recognition process includes the following sub-steps:

[0101] S311, generating a local saliency map for the foreground segmentation data in the real-time preprocessed data, and expanding and contracting the object region using a boundary tracking method based on a spatial-temporal composite optimization algorithm to extract second object candidate region data, wherein the boundary tracking is achieved through joint optimization of an image pyramid and a wavelet transform;

[0102] In some embodiments, after the foreground segmentation data is processed by a convolutional neural network, the dynamic area is distinguished from the static background. In order to more accurately extract the second object from these dynamic areas, a local saliency map is generated for the foreground segmentation data. The generation of the saliency map is based on the combination of image gradient calculation in the spatial domain and temporal information. Specifically, the saliency map is constructed using the following formula: Where S(x,y,t) represents the saliency value at the temporal and spatial location (x,y,t), I(x,y,t) is the pixel intensity at location (x,y) and time t, i,j,k are the image translation and time delay parameters, and n is the sliding window size. By generating a saliency map, we can extract the foreground region with significant changes, providing a basis for subsequent object tracking.

[0103] In this embodiment, a boundary tracking method based on a spatial-temporal composite optimization algorithm is further employed to expand and contract the target region, utilizing a spatiotemporal expansion and contraction strategy. Specifically, boundary tracking utilizes a combined optimization strategy of image pyramid and wavelet transform. The image pyramid constructs a multi-scale image hierarchy, effectively reducing image resolution and enabling rough region localization. The wavelet transform extracts detailed image information through multi-scale analysis, thereby determining the target region's boundaries.

[0104] In some embodiments, an image pyramid is generated by dividing the image into multiple scale layers, with the resolution of each layer being 1 / 2 of the previous layer. That is: Among them Ip (x, y) represents the pixel value of the p-th layer pyramid, G p is the Gaussian filter kernel, I p-1 (i, j) is the image of the previous layer of the pyramid. Objects are detected and located at different scales by processing the image pyramid layer by layer.

[0105] Wavelet transform is used to extract details from boundary areas. Wavelet transform decomposes the image into different frequency sub-bands and analyzes the changes in image edges. For each scale of wavelet transform, its transformation is defined as: Where W(x,y) is the wavelet transform result of the image at position (x,y), and h(m,n) is the wavelet filter kernel. By performing a wavelet transform on the boundary of each layer, subtle changes in the boundary are captured, thereby tracking the boundary of the target in each layer of the image pyramid.

[0106] Combining image pyramids with wavelet transforms, a spatial-temporal composite optimization algorithm tracks the boundaries of foreground segmentation data, further expanding and contracting the target region, ultimately extracting candidate regions for the second object. This method, through the coordinated processing of spatiotemporal information, effectively avoids excessive expansion or contraction of the target region, ensuring the accuracy of the candidate regions.

[0107] S312, performing hierarchical feature extraction on the second object candidate region data in combination with a deep autoencoder neural network, generating second object dynamic data using a dynamic flow field model and a motion clustering algorithm, wherein the feature extraction is jointly processed using a graph convolutional network and a motion prediction algorithm;

[0108] In some embodiments, hierarchical feature extraction is performed on the second object candidate area data extracted from the previous step through a deep autoencoding neural network. The deep autoencoding network can automatically learn the feature representation of the candidate area and map it to a low-dimensional feature space. Specifically, the deep autoencoding network consists of an encoder and a decoder. The encoder maps the input data to a low-dimensional latent space representation, and the decoder restores it back to the original input data. In this embodiment, the input data is the image data of the second object candidate area, the encoder performs feature compression on the data, and the decoder reconstructs it. The specific encoding process is represented by the following mathematical formula: h = f(Wx + b), where x is the input second object candidate area data, W and b are the weight matrix and bias of the encoder, respectively, h is the low-dimensional feature representation obtained by the encoder, and f(·) is the activation function. The decoding process is completed by the following formula: in is the reconstructed candidate region data, W′ and b′ are the weight matrix and bias of the decoder, and g(·) is the decoding function. Through the hierarchical learning of the encoder and decoder, high-level features of the second object can be extracted, thus better supporting the subsequent dynamic data generation.

[0109] In this embodiment, the output feature representation of the deep autoencoder neural network will be used as input and further combined with the graph convolutional network to extract local structural information. The graph convolutional network can process the local structural information in the image through the graph structure to capture the relationship between different regions. For each node v i , that is, a pixel or local area in the image, the graph convolution operation is calculated by the following formula: in Represents node v i In the feature representation of the kth layer, N(i) is the node v i The set of neighbor nodes, c ij is the normalization coefficient, W(k) and b(k) are the weight and bias of the kth layer respectively, and σ(·) is the activation function. Through the multi-layer propagation of the graph convolutional network, the local structural information of the second object can be extracted in the spatial domain.

[0110] In one embodiment, the output features of the graph convolutional network are combined with a motion prediction algorithm to generate dynamic data of the second object. The motion prediction algorithm predicts the target's motion trajectory and state changes based on the feature information of the candidate region of the second object. Specifically, the motion features are modeled by constructing a dynamic flow field model, and the object's motion trajectory is predicted using the following formula: in is the predicted state at time t+1, y(t) is the target state at time t, u(t) is the control input (for example, the acceleration or direction of the target), and A and B are the state transition matrix and control matrix. The dynamic flow field model can account for the motion of the second object in the spatiotemporal domain and generate more accurate dynamic data.

[0111] In some embodiments, the dynamic data of the second object is clustered based on a motion clustering algorithm. The motion clustering algorithm clusters similar dynamic behaviors based on the target's motion characteristics, thereby identifying the motion pattern of the second object. Clustering the motion data of the second object enables classification of the target's behavior pattern. The clustering result is generated using the following formula: Where C is the clustering result, x i is the i-th data point, c k The kth cluster center is used. In this way, the motion characteristics of the second object are effectively extracted, which provides an important basis for the subsequent target behavior analysis.

[0112] Through the above steps, the generated second object dynamic data includes its motion state, motion trajectory and classification information. In other words, the second object dynamic data includes two parts: time series motion state data and clustered motion pattern. Specifically expressed as: Where y(t) is the state at the original moment, is the predicted state, and C is the motion pattern clustering result.

[0113] S313, applying a multi-scale convolutional neural network based on spatiotemporal correlation analysis to the dynamic data of the second object to perform time series modeling, generate a motion trajectory of the second object, and assign a unique ID to the second object;

[0114] Temporal modeling of the second object's dynamic data utilizes a combination of multidimensional convolution and graph analysis to accurately predict the trajectory and assign an ID to the second object by processing cross-frame motion features. In some embodiments, the temporal modeling process includes cross-frame motion feature extraction, spatiotemporal correlation modeling, and trajectory generation and ID assignment.

[0115] In some embodiments, cross-frame motion feature extraction is performed using a multidimensional convolutional neural network. The input dynamic data contains state information of the second object at multiple time steps. The convolutional neural network extracts motion features between different time steps. These features can reflect the temporal changes and motion patterns of the second object. This operation can effectively extract cross-frame information from the dynamic data and capture the dynamic behavior of the second object.

[0116] In some embodiments, spatiotemporal correlation modeling is achieved by combining a multiscale convolutional neural network with graph analysis. The multiscale convolutional neural network is used to extract motion features at different time scales. Through convolution operations at different scales, the motion pattern of the second object at different times is obtained. Graph analysis combines spatial and temporal information, models the relationship between space and time, and considers the interaction between objects. This allows the complex dynamic behavior of the second object to be captured in the spatiotemporal dimension, providing complete spatiotemporal correlation information for subsequent trajectory modeling.

[0117] In some embodiments, the results of the time series modeling are integrated using a weighted average method to generate a movement trajectory of the second object. The movement trajectory reflects the actual movement pattern of the second object at different time periods. The trajectory of each object is fitted based on the weighted average of the spatiotemporal characteristics to generate a movement trajectory that conforms to the actual movement patterns. Based on the movement trajectory, in some embodiments, each object can be assigned a unique ID to ensure that the second object can be uniquely identified and tracked during subsequent monitoring and analysis.

[0118] S4, performing trajectory filtering processing on the motion trajectory of the first object under the unique ID, identifying abnormal stationary targets and outputting a dynamic retention warning signal;

[0119] For details, please refer to the attached Figure 5 As shown, the step includes the following sub-steps:

[0120] S401, performing a time series analysis on the motion trajectory of the first object with the unique ID, calculating the static duration in each time period, and comparing it with a preset baseline motion model to obtain static behavior deviation data;

[0121] In some embodiments, time series analysis utilizes a dynamic time warping algorithm. This algorithm addresses the time series matching problem caused by misaligned time axes by measuring the similarity between targets at different time points. This algorithm can calculate the differences in target movement at different time points and, based on these differences, assess the target's stationary state during the monitoring period. If the target's movement differences within a certain period are very small, the target is considered stationary. The stationary duration is calculated by summing the time the target was stationary during the monitoring period and is used for subsequent analysis of the target's behavior.

[0122] In some embodiments, a preset baseline motion model represents the expected motion pattern of a target under normal conditions. This model analyzes the motion trajectories of a large number of normal targets to derive the target's normal motion characteristics, such as velocity and acceleration. Under this baseline model, the target should maintain a certain motion state for a certain period of time and should not remain stationary for excessive periods of time. By comparing the baseline model with the target's actual stationary duration, stationary behavior deviation data can be derived. If the target's stationary duration exceeds the normal range in the baseline model, it indicates that the target is exhibiting abnormal stationary behavior.

[0123] In some embodiments, the static behavior deviation data reflects the difference between the target's static behavior and a normal motion model. The target's static behavior deviation is calculated by calculating the difference between the target's actual static duration and the static duration in the baseline model. This deviation data can provide a basis for subsequent abnormal behavior determination, helping to identify whether the target has safety hazards or abnormal behavior.

[0124] S402: Accumulate the stationary behavior deviation data under the unique ID, calculate the cumulative deviation values ​​for multiple time periods, and generate a deviation change rate based on the cumulative deviation changes in adjacent time periods. The deviation change rate is generated by differential analysis and a weighted moving average algorithm.

[0125] In this embodiment, the stationary behavior deviation data ΔT static(t) Perform cumulative processing in multiple time periods to obtain the cumulative deviation value in each time period. Set time period t i The corresponding static behavior deviation data is ΔT static (t i ), by accumulating these data, we can get the cumulative deviation C for each time period. static (t): Among them C static (t) is the cumulative deviation value at time t, representing the cumulative deviation of the target's stationary behavior from the initial time to the current time. The cumulative deviation value reflects the total deviation of the target's stationary behavior and provides a measure of the difference between the target's movement pattern and normal behavior for subsequent analysis.

[0126] In some embodiments, the deviation change rate is used to quantify the deviation change between adjacent time periods. static (t) is generated by differential analysis and weighted moving average algorithm. Set two adjacent time periods t i-1 and t i The cumulative deviation values ​​are C static (t i-1 ) and C static (t i ), then the deviation change rate R static (t) is: Deviation change rate R static (t) By performing differential calculation on the cumulative deviations of adjacent time periods, the changing trend of the target's static behavior is obtained. In this embodiment, the rate of change is smoothed using a weighted sliding average algorithm to reduce the impact of noise interference on the change rate calculation result. The calculation formula for the weighted sliding average is: where R smooth (t) is the deviation change rate after smoothing, w k is the weighting coefficient, representing the weight of the rate of change at the previous moment, and N is the length of the weighted window. The weighted moving average algorithm uses the weighted window to average the rate of change, avoiding the influence of outliers on the calculation result and obtaining a smoothed rate of change.

[0127] In this embodiment, the cumulative deviation and the rate of change of the deviation are combined to detect abnormalities in the target's stationary behavior. By accumulating the stationary behavior deviation data and generating the rate of change of the deviation, the target's stationary behavior evolution trend can be analyzed and any signs of prolonged or abnormal stationary behavior can be identified.

[0128] S403: constructing a spatiotemporal change trend model of the target's stationary behavior based on the accumulated deviation value and the deviation change rate under the unique ID, wherein the model is modeled using a spatiotemporal convolutional network to obtain a change trend of the target's stationary behavior;

[0129] In some embodiments, a spatiotemporal convolutional network combines the learning of spatial and temporal features to effectively capture the changing patterns of target behavior across both spatial and temporal dimensions. By inputting cumulative deviation values ​​and deviation change rate data, the network can simultaneously analyze the target's stationary distribution in space and its temporal variation patterns, thereby extracting the spatiotemporal characteristics of the target's stationary behavior. Specifically, the spatiotemporal convolutional network structure consists of multiple convolutional layers, each of which progressively extracts the spatial and temporal characteristics of the target's stationary behavior.

[0130] In this embodiment, the input data of the spatiotemporal convolutional network includes the target's stationary deviation (accumulated deviation value) and the deviation's changing trend (deviation change rate) over different time periods. Through the convolution operation, the network can jointly analyze these two types of information in the spatiotemporal domain, gradually extracting the spatiotemporal characteristics of the target's stationary behavior. Each layer of convolution operation learns the spatial and temporal dependencies in the data to better capture the evolution of the target's stationary behavior.

[0131] The output of the spatiotemporal convolutional network is the temporal and spatial variation trends of the target's stationary behavior. This output represents the target's stationary behavior pattern in both time and space, as well as the changing trends of this behavior pattern. Through a multi-layer convolutional structure, the network gradually extracts and optimizes these features, accurately revealing the changes in the target's behavior over time and at different locations.

[0132] In some embodiments, the output of the spatiotemporal trend model provides a dynamic, spatiotemporal view of the target's static behavior. Based on these outputs, it is further determined whether the target is in an abnormal static state. For example, if the model output reflects a sudden change in the target's behavior or an unusual static pattern, the target is considered to have abnormal static behavior. The results of this model will serve as the basis for subsequent anomaly detection, helping to promptly identify potential safety hazards.

[0133] In this embodiment, the spatiotemporal change trend model obtained by the spatiotemporal convolutional network can capture the evolution law of the target's static behavior in both spatial and temporal dimensions, and effectively judge whether it is abnormal behavior.

[0134] S404: Perform causal reasoning analysis on the spatiotemporal change trend model under the unique ID to generate target abnormal behavior determination data, and perform comprehensive analysis in combination with a multi-objective decision network to output a dynamic detention warning signal.

[0135] In this embodiment, causal reasoning analysis uses the spatiotemporal trend model T trend (t) extracts the static behavior pattern data and derives the causal relationship. This analysis is done by constructing a causal graph model G, where the nodes of the graph represent static behavior states and the edges represent the causal relationships between behaviors.

[0136] Setting a static behavior bias is the behavioral deviation of the target in time period t. In this embodiment, the causal reasoning model of the target behavior is assumed to be: Among them i is the parent node of node i and represents the causal factor of that node. By optimizing the parameters in the causal graph through maximum likelihood estimation, we obtain the causal inference results of the target behavior. Based on the behavioral patterns output by causal inference, we can determine whether the target exhibits abnormal stationary behavior.

[0137] In some implementations, a multi-objective decision-making network is used to comprehensively analyze target abnormal behavior data. This network integrates behavioral information from multiple targets, enabling it to make informed decisions in complex environments. For each target, the network first comprehensively assesses its behavioral deviations based on the output of a spatiotemporal trend model and causal inference results.

[0138] The core of the multi-objective decision network is to optimize the decision weights of each target behavior to ensure that the network output can comprehensively consider the dynamic behavior information of multiple targets. target (i) is the decision weight of target i, is the final behavioral deviation of target i, and the decision rule of the target is: Where N is the number of targets, W target (j) is the weight of target j. By performing weighted summation on the behavior data of multiple targets, a comprehensive abnormal behavior assessment of each target is finally generated.

[0139] In this embodiment, the comprehensive abnormal behavior assessment of the target The preset abnormal behavior threshold Compare. The target is considered to have abnormal static behavior. This judgment result will further generate a dynamic detention warning signal.

[0140] S5, when outputting the dynamic detention warning signal, screening and matching analysis are performed on the movement trajectory of the second object, the trajectory anomaly is identified and the abnormal behavior alarm signal and the associated response action are triggered.

[0141] Specifically, in the safety management of the crossing, the stationary behavior of the first object is analyzed, and its stationary duration, deviation value, rate of change and other data are calculated to confirm whether there is any abnormal detention. When a detention warning signal is detected, the system uses it as a preliminary warning of potential risks, indicating that there may be problems in the area. However, this detention signal does not directly indicate a safety threat and requires further observation. Next, the system screens and matches the behavior trajectory of the second object to determine whether it is tailgating or engaging in other abnormal behaviors. When the behavior of the second object deviates from the preset normal pattern, or when non-compliant behavior is detected, the system triggers an alarm signal and associated response actions. This process is handled in stages, first prompting potential risk signals, and then responding when actual threats arise, thereby improving the accuracy and timeliness of processing, avoiding false alarms and effectively improving the level of safety management. Please refer to the attached Figure 6 As shown, this step includes the following sub-steps:

[0142] S501, when a dynamic detention warning signal is output, the movement trajectory of the second object is screened based on the independent ID of the second object and the spatiotemporal constraint model. The spatiotemporal constraint model uses dynamic spatiotemporal window and spatial neighborhood cluster analysis to screen out the difference dataset between trajectory anomalies and normal behavior trajectories to generate a valid target dataset;

[0143] In this embodiment, the spatiotemporal constraint model is combined with the dynamic behavior information of the second object to construct a multidimensional space model that can capture the target trajectory change characteristics. i (t)=(x i (t),y i (t)) is the position data of the second object at time t, where x i (t) and y i (t) represents the coordinate in two-dimensional space. By introducing the spatiotemporal window mechanism, local feature modeling of the target trajectory is performed. The spatiotemporal window is defined as W(t) = [t-ξt, t+ξt], where ξt is the time window size. The movement data of the target in the time interval [t-ξt, t+ξt] is extracted through the window, and the spatiotemporal characteristics are analyzed. In some embodiments, the size of the spatiotemporal window is dynamically adjusted according to the speed and behavioral characteristics of the target trajectory. In this embodiment, the application of the dynamic spatiotemporal window is based on the speed and direction information of the target within a given time period.

[0144] In the spatiotemporal window, the target trajectory data X i (t) will be clustered according to its position change characteristics to filter out possible abnormal points in the trajectory. The abnormal point judgment formula is set as: ΔX(t)=||X i (t)-X norm (t)||, where X norm(t) is the estimated point of the normal behavior trajectory. The standard of the abnormal point is that the distance between the trajectory point and the normal trajectory exceeds a certain set threshold, that is, ΔX(t)>ε. When the target trajectory point X i (t) and the normal trajectory X norm When the distance of (t) exceeds the threshold ε, it is determined to be an abnormal point.

[0145] In some embodiments, spatial neighborhood clustering analysis further identifies abnormalities in trajectory data by clustering target trajectories within a spatiotemporal window. The clustering method is based on the spatial distribution of the target trajectory and divides the trajectory points into multiple clusters using a clustering algorithm. In this embodiment, a density-based clustering algorithm is used for spatial neighborhood clustering analysis. The spatial distance metric of the target trajectory is set to: The DBSCAN algorithm calculates the distance d(X i (t),X j (t)), clustering closely spaced trajectory points into the same cluster and determining whether there are noise points or outliers. By setting an appropriate neighborhood radius ε and the minimum number of points MinPts, outliers in the trajectory can be effectively identified.

[0146] In this embodiment, the trajectory data filtered by the dynamic spatiotemporal window and spatial neighborhood clustering analysis will generate a valid target dataset. val id is a valid target data set, which contains the filtered target trajectory points X i (t), these trajectory points represent the difference dataset between the target’s normal behavior trajectory and abnormal behavior trajectory. The abnormal behavior characteristics of the target are determined by comparing the deviation between the abnormal points and the normal trajectory.

[0147] In some embodiments, the generation of a valid target dataset is based on the following condition: if the deviation of a target trajectory point within the spatiotemporal window exceeds a threshold, and the point is identified as an outlier by the clustering algorithm, then the trajectory point is considered part of the valid target data. Through multiple iterations of screening and optimization, it is possible to ensure that the dataset only contains trajectory points with significant behavioral deviations.

[0148] S502, performing similarity comparison on the valid target dataset using an adaptive trajectory similarity matching algorithm, comparing the valid target dataset with a preset normal action trajectory template, and generating a matching analysis result;

[0149] In some embodiments, the similarity comparison includes feature extraction and similarity calculation. Specifically, the valid target dataset is compared with a preset normal motion trajectory template to generate a matching analysis result.

[0150] In this embodiment, the feature extraction process is performed by extracting spatiotemporal trajectory features from the valid target data set. Assume that the time series data of the target trajectory is T i ={X i (t1),X i (t2),…,X i (t n )}, where X i (t) represents the position of the target at time t, and n is the length of the trajectory. Feature extraction is performed on the trajectory data, including position, velocity, and acceleration information.

[0151] Set the velocity of the trajectory to: Where Δt is the time interval, X i (t+Δt) and X i (t) represents the target position at time t and t+Δt respectively. The acceleration is calculated using the following formula: In some implementations, feature extraction may also include information about the direction and angle of the target trajectory and the smoothness of the motion. These features can effectively describe the behavioral characteristics of the target, and we will not elaborate on them here. On this basis, a high-dimensional feature vector of the target is constructed based on the trajectory features at each time point.

[0152] In this embodiment, the similarity calculation process is performed using an adaptive trajectory similarity matching algorithm, which aims to identify whether there is abnormal behavior by measuring the similarity between the valid target data set and the normal action trajectory template. norm ={X norm (t1),X norm (t2),…,X norm (t m )}, where m is the length of the template trajectory.

[0153] Adaptive trajectory similarity matching is calculated by the dynamic time warping algorithm to measure the similarity between two trajectories. The DTW algorithm finds the best matching path by minimizing the cumulative distance between trajectories. Let D(i,j) be the trajectory point X i (t) and template point X norm The distance between (t) is expressed as Euclidean distance: D(i,j)=||X i (t)-X norm (t)||, the goal of DTW is to calculate the minimum cumulative distance through dynamic programming, and then compare the overall similarity between the target trajectory and the template trajectory. The matching result of DTW is represented by the following optimized path P: In some implementations, the DTW algorithm introduces a weighting strategy to assign different weights to trajectory features at different times, thereby better reflecting the difference between the target trajectory and the normal behavior trajectory. Set the weighting factor to w(t), and modify the distance metric formula to: D(i,j) = w(t)||X i (t)-X norm (t)||, where w(t) is adaptively adjusted according to the dynamic changes of the target trajectory in order to highlight the trajectory characteristics of the key period.

[0154] In this embodiment, the similarity between the target trajectory and the normal behavior trajectory template is calculated by the DTW algorithm, and a threshold is used to determine whether there is abnormal behavior. The similarity metric is set as: Where S is the similarity value of the trajectory matching, and D(i, j) is the distance between the trajectory points. A smaller similarity value S indicates that the target trajectory is more similar to the normal trajectory; a larger similarity value S indicates that the target trajectory is more different from the normal trajectory.

[0155] When the similarity value exceeds a preset threshold α, the target trajectory is determined to have deviated, and a matching analysis result is generated, indicating that the target may be exhibiting abnormal behavior. This step can accurately identify targets that have deviated from their normal trajectory and provide data support for subsequent abnormal behavior determination.

[0156] S503: Process the matching results based on the spatiotemporal deviation fusion analysis model, identify targets whose trajectory deviation exceeds a threshold, output abnormal behavior alarm signals, and trigger associated response actions.

[0157] In some embodiments, the spatiotemporal deviation fusion analysis model includes a deviation value extraction process and a spatiotemporal fusion calculation process.

[0158] In this embodiment, the deviation value extraction process is performed by calculating the difference between the target trajectory and the normal trajectory from the matching results. The similarity measurement value between the target trajectory and the normal trajectory is set as follows: where X i (t) and X norm (t) are the positions of the target trajectory and the normal trajectory at time t. Thus, the difference between the target trajectory and the normal trajectory is measured. On this basis, by calculating the deviation value between each target trajectory point and the template trajectory point, a set of deviation data sets D is formed. diff , indicating the degree to which the target trajectory deviates from the normal trajectory. In this embodiment, the spatiotemporal fusion calculation process is used to further process the deviation data, considering the combined influence of time and space factors to identify abnormal targets. For this purpose, the spatiotemporal deviation fusion algorithm is used. Assume that the spatial position of the target trajectory at time t is X i (t), its deviation value in the time dimension is Δt i, the spatial deviation value is Δs i The spatiotemporal deviation fusion calculation is performed using the following formula: Δ t,s (t) = α·Δt i +β·Δs i , where Δ t,s (t) is the combined deviation, and α and β are weighting factors representing the weights of temporal and spatial deviations in the spatiotemporal deviation fusion. In some embodiments, the weights α and β are dynamically adjusted based on the actual application scenario to highlight behavioral characteristics under specific spatiotemporal conditions. This approach effectively integrates the spatiotemporal deviations of the target trajectory, further quantifying the overall difference between the trajectory and the normal trajectory.

[0159] In this embodiment, after the spatiotemporal deviation fusion calculation, a deviation threshold ζ is further introduced to set a threshold interval. t,s When (t) exceeds this threshold, it means that the target trajectory deviates significantly from the normal trajectory, thus identifying the abnormal behavior target. The specific judgment formula is as follows: If Abnorma l ityScore>ζ, the target behavior is determined to be abnormal and an abnormal behavior alarm signal is generated.

[0160] In this embodiment, when an abnormal behavior alarm signal is generated, the response action will be determined based on the temporal and spatial characteristics of the target trajectory and preset rules. Response actions may include but are not limited to activating safety equipment, issuing an alarm, and recording trajectory information.

[0161] Based on the description of the embodiment of the airport crossing safety management method based on video capture above, the embodiment of the present application also discloses an airport crossing safety management system based on video capture. The airport crossing safety management system based on video capture can be a computer program (including program code) that runs the above-mentioned airport crossing safety management method based on video capture. Figure 7 As shown, the airport crossing safety management system based on video capture can run the following units:

[0162] An acquisition unit 110 is configured to acquire real-time data to be processed, wherein the real-time data to be processed is video crossing image data captured in real time by a video surveillance device;

[0163] The preprocessing unit 120 is used to preprocess the real-time data to be processed to obtain real-time preprocessed data;

[0164] The recognition unit 130 is configured to perform recognition processing on the real-time pre-processed data for the first object and the second object, generate dynamic data of the first object and the second object, perform modeling based on the dynamic data to generate respective motion trajectories, and assign an independent ID to each object;

[0165] a retention detection unit 140 for performing trajectory filtering processing on the movement trajectory of the first object under the unique ID, identifying abnormal stationary targets and outputting a dynamic retention warning signal;

[0166] The anomaly detection unit 150 is used to screen and match the movement trajectory of the second object when outputting the dynamic retention warning signal, identify the trajectory anomaly and trigger the abnormal behavior alarm signal and associated response action.

[0167] The foregoing description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein and should not be construed as excluding other embodiments. Rather, the present invention can be used in various other combinations, modifications, and environments and can be modified within the scope of the concept described herein through the above teachings or techniques or knowledge in the relevant field. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention are intended to be protected by the appended claims.

Claims

1. A method for airport crossing safety management based on video capture, characterized in that: The method comprises the following steps: S1, obtaining real-time data to be processed, which is the video crossing image data captured in real time by the video surveillance equipment; S2, preprocessing the real-time data to be processed to obtain real-time preprocessed data; S3, performing recognition processing of the first object and the second object on the real-time pre-processed data, the recognition processing of the first object includes: performing spatiotemporal separation on the foreground segmentation data in the real-time pre-processed data, filtering the motion features of the foreground area through an adaptive spatiotemporal filtering algorithm, and generating motion pattern data; applying a partitioning processing algorithm based on tensor decomposition based on the motion pattern data, dynamically constructing a target motion attribute model in the area, and generating dynamic data of the first object; performing motion trajectory modeling on the dynamic data of the first object using an optimized Kalman filtering algorithm based on time-frequency joint analysis, generating a movement trajectory of the first object, and assigning an independent ID to it; the recognition processing of the second object includes: extracting candidate area data of the second object; performing hierarchical feature extraction on the candidate area data of the second object in combination with a deep autoencoder neural network, generating dynamic data of the second object through a dynamic flow field model and a motion clustering algorithm, and jointly processing the feature extraction through a graph convolutional network and a motion prediction algorithm; applying a multi-scale convolutional neural network based on spatiotemporal correlation analysis to the dynamic data of the second object for time series modeling, generating a movement trajectory of the second object, and assigning an independent ID to it; S4, performing trajectory filtering processing on the movement trajectory of the first object under the independent ID, identifying abnormal stationary targets and outputting a dynamic retention warning signal; S5, when outputting the dynamic detention warning signal, screening and matching analysis are performed on the movement trajectory of the second object, identifying trajectory anomalies and triggering abnormal behavior alarm signals and associated response actions; the first object is a ground transportation vehicle, and the second object is a person and an animal; S4 includes: performing time series analysis on the movement trajectory of the first object under the independent ID, calculating the static duration in each time period, and comparing it with the preset benchmark motion model to obtain static behavior deviation data; performing cumulative processing on the static behavior deviation data under the independent ID, calculating the cumulative deviation values ​​of multiple time periods, and generating the deviation change rate according to the cumulative deviation changes of adjacent time periods; constructing a spatiotemporal change trend model of the target static behavior based on the cumulative deviation values ​​and the deviation change rate under the independent ID, and modeling the spatiotemporal change trend model through a spatiotemporal convolutional network to obtain the change trend of the target static behavior; performing causal reasoning analysis on the spatiotemporal change trend model under the independent ID to generate target abnormal behavior judgment data, and combining multiple targets The decision network performs a comprehensive analysis and outputs a dynamic detention warning signal; S5 includes: when the dynamic detention warning signal is output, the movement trajectory of the second object is screened based on the independent ID of the second object and the spatiotemporal constraint model, and the spatiotemporal constraint model screens out the difference data sets between the trajectory anomalies and the normal behavior trajectory through dynamic spatiotemporal window and spatial neighborhood clustering analysis to generate a valid target data set; the effective target data set is compared with the preset normal action trajectory template using an adaptive trajectory similarity matching algorithm to generate a matching analysis result; the matching result is processed based on the spatiotemporal deviation fusion analysis model, and the deviation from the preset trajectory exceeds the threshold, and an abnormal behavior alarm signal is output, and an associated response action is triggered.

2. The method for airport crossing safety management based on video capture according to claim 1, characterized in that: S2 includes the following sub-steps: S201, performing a frame extraction operation on the real-time data to be processed, decomposing the continuous video picture into a discrete frame sequence and performing normalization processing to obtain discrete normalized data, performing Gaussian filtering on the discrete normalized data to remove noise, and generating denoised frame data; S202, performing background modeling and foreground segmentation processing on the denoised frame data, extracting the motion foreground area using a convolutional neural network segmentation algorithm to generate foreground segmentation data, adding a timestamp to each frame of the foreground segmentation data, and generating real-time preprocessed data.

3. The method for airport crossing safety management based on video capture according to claim 2, characterized in that: Temporal modeling includes cross-frame motion feature extraction, spatiotemporal correlation modeling, trajectory generation and ID assignment.

4. The method for airport crossing safety management based on video capture according to claim 3 is characterized in that: The dynamic time warping algorithm was used for time series analysis.

5. The method for airport crossing safety management based on video capture according to claim 4 is characterized in that: The similarity comparison includes feature extraction processing and similarity calculation processing. The similarity calculation processing is performed using an adaptive trajectory similarity matching algorithm, and the adaptive trajectory similarity matching is calculated using a dynamic time warping algorithm. The spatiotemporal deviation fusion analysis model includes deviation value extraction processing and spatiotemporal fusion calculation processing.

6. An airport crossing safety management system based on video capture, used to implement the airport crossing safety management method based on video capture according to any one of claims 1 to 5, characterized in that: The system comprises: An acquisition unit is used to acquire real-time data to be processed, wherein the real-time data to be processed is video crossing image data captured in real time by a video surveillance device; A preprocessing unit, used for preprocessing the real-time data to be processed to obtain real-time preprocessed data; an identification unit, configured to perform identification processing on the first object and the second object based on the real-time preprocessed data, generate dynamic data of the first object and the second object, perform modeling based on the dynamic data to generate respective motion trajectories, and assign an independent ID to each object; a detention detection unit, configured to perform trajectory filtering processing on the movement trajectory of the first object under the independent ID, identify abnormal stationary targets and output a dynamic detention warning signal; The anomaly detection unit is used to screen and match the movement trajectory of the second object when outputting the dynamic detention warning signal, identify the trajectory anomaly and trigger the abnormal behavior alarm signal and associated response action.

Images