Pedestrian trajectory abnormal behavior detection method, system, device and storage medium

Abstract

The application discloses a kind of abnormal behavior detection methods, systems, equipment and storage medium of pedestrian trajectory, the method includes: by TraDeS model respectively extracting each section pedestrian trajectory video every frame pedestrian foot position coordinates, to generate time series trajectory data, by perspective conversion matrix to time series trajectory data coordinate conversion, obtain pedestrian space-time trajectory data;Pedestrian space-time trajectory data is normalized in space, and it is mapped to the pixel space in trajectory blank image, to generate trajectory gray scale image;According to time information and pedestrian information, trajectory gray scale image is color-coded, and color gradient trajectory image is obtained;There is no overlap in color gradient trajectory image track point, and color gradient trajectory image is input into ConvNeXt network model, to output trajectory abnormal behavior detection result.The application accurately identifies complex covert behavior by introducing space-time trajectory mapping, color coding technology and network model.

Description

Technical Field

[0001] This invention belongs to the field of public safety monitoring technology, and in particular relates to a method, system, device and storage medium for detecting abnormal behavior of pedestrian trajectories. Background Technology

[0002] Current surveillance systems typically rely on visual feature-based analysis methods to identify unusual behavior. These methods effectively address unusual events with obvious visual characteristics, such as fights and stampedes, by analyzing postures, movements, and crowd interaction patterns. However, many covert unusual behaviors, such as stealthy stalking, lack obvious external features and are often difficult to detect accurately using traditional visual analysis methods. Such covert behaviors can pose a serious threat to public safety and increase the probability of potential criminal activity.

[0003] Under the current technological background, research on abnormal behavior detection has gradually developed, and a series of methods have emerged, such as detection techniques based on statistical models, machine learning, and deep learning. These methods have certain advantages in dealing with obvious abnormal behavior, but there are still many technical bottlenecks when dealing with hidden abnormal behavior. For example: (1) Reliance on large sample training: Existing deep learning models perform well in abnormal behavior detection, but they usually require a large number of labeled samples for training. However, actual cases of hidden abnormal behavior, such as hidden tailing, are very rare, which leads to a significant drop in model performance in small sample scenarios. (2) Limited data augmentation effect: In order to solve the problem of sample scarcity, researchers often use data augmentation techniques to generate more samples, but due to the limitations of the original data quality, the augmented data often cannot effectively improve the detection ability of the model. This makes the existing technology still face the problem of insufficient data when dealing with hidden abnormal behavior, affecting the generalization ability of the model. (3) Feature design relies on expert knowledge: Many abnormal behavior detection methods rely on features designed manually by experts. These features often need to be adjusted for different scenarios and cannot be adapted to all monitoring scenarios once and for all. Such a design process is not only time-consuming, but may also lead to the loss of key information, limiting the detection accuracy of the model.

[0004] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this invention is to provide a method, system, device, and storage medium for detecting abnormal behavior in pedestrian trajectories, aiming to solve the technical problem of how to accurately identify complex and covert behaviors.

[0006] To achieve the above objectives, the present invention provides a method for detecting abnormal behavior in pedestrian trajectories, the method comprising:

[0007] Pedestrian surveillance video is segmented based on time windows to obtain multiple segments of pedestrian trajectory video;

[0008] The TraDeS model is used to extract the foot position coordinates of each frame in each segment of pedestrian trajectory video to generate time series trajectory data. The time series trajectory data is then transformed by a perspective transformation matrix to obtain the spatiotemporal trajectory data of pedestrians.

[0009] The pedestrian spatiotemporal trajectory data is spatially normalized, and the spatially normalized pedestrian spatiotemporal trajectory data is mapped to the pixel space of the trajectory blank image to generate a trajectory grayscale image.

[0010] The trajectory grayscale image is color-coded based on time information and pedestrian information to obtain a color gradient trajectory image;

[0011] Detect whether there is overlap between trajectory points in the color gradient trajectory image;

[0012] If not, the color gradient trajectory image is input into the ConvNeXt network model so that the ConvNeXt network model outputs the trajectory abnormal behavior detection result.

[0013] Optionally, the step of segmenting the pedestrian surveillance video based on a time window to obtain multiple segments of pedestrian trajectory video includes:

[0014] Determine the frame rate and duration of the pedestrian surveillance video;

[0015] The number of frames in the short video segment is determined based on the frame rate and the duration.

[0016] The pedestrian monitoring video is segmented using a time window based on the number of frames in the short video segment to obtain multiple pedestrian trajectory video segments.

[0017] Optionally, the step of performing coordinate transformation on the time series trajectory data using a perspective transformation matrix to obtain pedestrian spatiotemporal trajectory data includes:

[0018] The original source point coordinates and the real source point coordinates of each trajectory image in multiple pedestrian trajectory videos were determined respectively;

[0019] Calculate the perspective transformation matrix based on the original source point coordinates and the actual source point coordinates;

[0020] The perspective transformation matrix is ​​used to transform each trajectory image in multiple pedestrian trajectory videos, and the time series trajectory data is transformed by a set of coordinate transformation equations based on the transformed trajectory images to obtain pedestrian spatiotemporal trajectory data.

[0021] Optionally, the step of spatially normalizing the pedestrian spatiotemporal trajectory data includes:

[0022] Determine the horizontal width and vertical height of the walking monitoring area, and select the minimum and maximum coordinates of the pedestrian's footsteps from the pedestrian spatiotemporal trajectory data based on the walking monitoring area;

[0023] A normalized coordinate formula is constructed based on the horizontal width, the vertical height, the minimum coordinates of the pedestrian's foot position, and the maximum coordinates of the pedestrian's foot position;

[0024] The pedestrian spatiotemporal trajectory data is spatially normalized using the normalized coordinate formula.

[0025] Optionally, the step of color encoding the trajectory grayscale image based on time information and pedestrian information to obtain a color gradient trajectory image includes:

[0026] Time information is encoded into the hue and saturation channels, and pedestrian information is encoded into the luminance channel;

[0027] The color of each pedestrian trajectory pixel in the grayscale trajectory image is color-coded based on the hue channel, the saturation channel and the luminance channel to obtain a color gradient trajectory image.

[0028] Optionally, after the step of detecting whether there is overlap between trajectory points in the color gradient trajectory image, the method further includes:

[0029] If there is overlap, a new trajectory grayscale image of the same dimension is created based on the trajectory grayscale image, and the HSV value of the overlapping trajectory points in the color gradient trajectory image is determined.

[0030] The HSV values ​​of the overlapping trajectory points are updated using the HSV update formula;

[0031] The HSV values ​​of the non-overlapping trajectory points and the updated HSV values ​​of the overlapping trajectory points are mapped to the new trajectory grayscale image to obtain a color gradient trajectory image.

[0032] Optionally, the step of inputting the color gradient trajectory image into the ConvNeXt network model so that the ConvNeXt network model outputs trajectory abnormal behavior detection results includes:

[0033] The color gradient trajectory image is input into the ConvNeXt network model to obtain the confidence score of abnormal trajectory behavior;

[0034] Based on the confidence level of the trajectory anomaly behavior, the ConvNeXt network model outputs the trajectory anomaly behavior detection result.

[0035] Furthermore, to achieve the above objectives, the present invention also proposes an abnormal behavior detection system for pedestrian trajectories, the abnormal behavior detection system for pedestrian trajectories comprising:

[0036] The segmentation module is used to segment pedestrian surveillance videos based on time windows to obtain multiple segments of pedestrian trajectory videos;

[0037] The conversion module is used to extract the foot position coordinates of each frame in each segment of pedestrian trajectory video using the TraDeS model to generate time series trajectory data, and to perform coordinate transformation on the time series trajectory data using a perspective transformation matrix to obtain pedestrian spatiotemporal trajectory data.

[0038] The mapping module is used to spatially normalize the pedestrian spatiotemporal trajectory data and map the spatially normalized pedestrian spatiotemporal trajectory data into the pixel space of the trajectory blank image to generate a trajectory grayscale image.

[0039] The encoding module is used to color encode the trajectory grayscale image based on time information and pedestrian information to obtain a color gradient trajectory image;

[0040] The detection module is used to detect whether there is overlap between trajectory points in the color gradient trajectory image;

[0041] The detection module is further configured to, if not, input the color gradient trajectory image into the ConvNeXt network model so that the ConvNeXt network model outputs the trajectory abnormal behavior detection result.

[0042] Furthermore, to achieve the above objectives, the present invention also proposes an abnormal behavior detection device for pedestrian trajectories. The device includes: a memory, a processor, and an abnormal behavior detection program for pedestrian trajectories stored in the memory and executable on the processor. The abnormal behavior detection program for pedestrian trajectories is configured to implement the steps of the abnormal behavior detection method for pedestrian trajectories as described above.

[0043] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing an abnormal behavior detection program for pedestrian trajectories, wherein when the abnormal behavior detection program for pedestrian trajectories is executed by a processor, the steps of the abnormal behavior detection method for pedestrian trajectories described above are implemented.

[0044] This invention first segments pedestrian surveillance videos based on time windows to obtain multiple pedestrian trajectory video segments. Then, it extracts the footstep coordinates of each frame in each pedestrian trajectory video segment using the TraDeS model to generate time-series trajectory data. A perspective transformation matrix is ​​then used to transform the coordinates of the time-series trajectory data to obtain spatiotemporal trajectory data. Next, the spatiotemporal trajectory data is spatially normalized and mapped to the pixel space of a trajectory blank image to generate a grayscale trajectory image. Finally, the grayscale trajectory image is color-coded based on time and pedestrian information to obtain a color gradient trajectory image. The system detects whether trajectory points in the color gradient trajectory image overlap. If not, the color gradient trajectory image is input into a ConvNeXt network model, which outputs the trajectory abnormal behavior detection result. This invention, by introducing spatiotemporal trajectory mapping and color coding techniques, combined with a ConvNeXt network model, can more accurately identify complex and hidden behaviors, reducing false alarms and false negatives. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the structure of an abnormal behavior detection device for pedestrian trajectory in the hardware operating environment involved in the embodiments of the present invention;

[0046] Figure 2 This is a flowchart illustrating the first embodiment of the abnormal behavior detection method for pedestrian trajectories of the present invention;

[0047] Figure 3 This is a schematic diagram of trajectory grayscale image construction in the first embodiment of the pedestrian trajectory abnormal behavior detection method of the present invention;

[0048] Figure 4 This is a schematic diagram of color gradient trajectory image construction in the first embodiment of the abnormal behavior detection method for pedestrian trajectory of the present invention;

[0049] Figure 5 This is a color gradient trajectory map of different types of pedestrians in the first embodiment of the abnormal behavior detection method for pedestrian trajectories of the present invention;

[0050] Figure 6 This is a schematic diagram of the ConvNeXt network model of the first embodiment of the abnormal behavior detection method for pedestrian trajectory of the present invention;

[0051] Figure 7 This is a structural block diagram of the first embodiment of the pedestrian trajectory abnormal behavior detection system of the present invention.

[0052] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0053] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0054] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of an abnormal behavior detection device for pedestrian trajectory in the hardware operating environment involved in the embodiments of the present invention.

[0055] like Figure 1 As shown, the pedestrian trajectory abnormal behavior detection device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk storage device. The memory 1005 may also optionally be a storage system independent of the aforementioned processor 1001.

[0056] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the abnormal behavior detection device for pedestrian trajectories, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0057] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a pedestrian trajectory abnormal behavior detection program.

[0058] exist Figure 1In the pedestrian trajectory abnormal behavior detection device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the pedestrian trajectory abnormal behavior detection device of the present invention can be set in the pedestrian trajectory abnormal behavior detection device. The pedestrian trajectory abnormal behavior detection device calls the pedestrian trajectory abnormal behavior detection program stored in the memory 1005 through the processor 1001 and executes the pedestrian trajectory abnormal behavior detection method provided in the embodiment of the present invention.

[0059] This invention provides a method for detecting abnormal behavior in pedestrian trajectories, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the abnormal behavior detection method for pedestrian trajectories of the present invention.

[0060] In this embodiment, the abnormal behavior detection method for pedestrian trajectories includes the following steps:

[0061] Step S10: Segment the pedestrian monitoring video based on the time window to obtain multiple pedestrian trajectory video segments.

[0062] It is easy to understand that the executing entity of this embodiment can be an abnormal behavior detection system for pedestrian trajectories with functions such as data processing, network communication and program execution, or other computer devices with similar functions. This embodiment does not limit it.

[0063] Furthermore, the frame rate and duration of the pedestrian monitoring video are determined; the number of frames in a short video segment is determined based on the frame rate and the duration; and the pedestrian monitoring video is segmented into time windows based on the number of frames in the short video segment to obtain multiple pedestrian trajectory videos.

[0064] It should be noted that pedestrian surveillance video covers the area where pedestrians are walking. When the segments of pedestrian surveillance video are too long, directly processing the entire video may cause computational bottlenecks. The video segmentation strategy based on time windows can divide the video into frames according to a set time interval (24 seconds).

[0065] If there are significant changes in the time points of pedestrian surveillance video (such as when an event occurs), a dynamic segmentation strategy can be set to adjust based on changes in background or pedestrian density.

[0066] In the specific implementation, the frame rate and duration information of the pedestrian monitoring video are first read to determine the number of frames contained in the short video segment within 24 seconds (i.e., the number of frames in the short video segment). Then, the pedestrian monitoring video is extracted frame by frame at preset time intervals, and multiple independent short video segments (i.e., multiple pedestrian trajectory video segments) are generated sequentially, ensuring that each short video segment is saved in the same chronological order as the original video (i.e., the pedestrian monitoring video). This effectively reduces the computational complexity of video processing and improves the efficiency of trajectory analysis and behavior detection without affecting the integrity of the video content.

[0067] Step S20: Extract the footstep coordinates of each frame in each segment of pedestrian trajectory video using the TraDeS model to generate time series trajectory data, and perform coordinate transformation on the time series trajectory data using a perspective transformation matrix to obtain pedestrian spatiotemporal trajectory data.

[0068] It should be understood that the TraDeS model (online multi-object tracker) is used to track all pedestrians appearing in the video, record the coordinates of the pedestrians' feet in each frame, and form time-series trajectory data.

[0069] The time-series trajectory data includes the coordinates (i.e., trajectory points) of the foot positions of all pedestrians in each frame corresponding to the time sequence.

[0070] Furthermore, the process of obtaining pedestrian spatiotemporal trajectory data by performing coordinate transformation on the time series trajectory data through perspective transformation matrix is ​​as follows: the original source point coordinates and the real source point coordinates of each trajectory image in multiple pedestrian trajectory video segments are determined respectively; the perspective transformation matrix is ​​calculated based on the original source point coordinates and the real source point coordinates; the perspective transformation matrix is ​​used to transform each trajectory image in multiple pedestrian trajectory video segments; and based on the transformed trajectory images, the time series trajectory data is transformed through a set of coordinate transformation equations to obtain pedestrian spatiotemporal trajectory data.

[0071] In the specific implementation, four key feature points are selected from each trajectory image in multiple pedestrian trajectory videos as the original source point coordinates. These four key feature points can be the coordinates of the four corner points of each trajectory image, and the corresponding real source point coordinates in the real scene are defined to determine the rectangular area required after perspective transformation.

[0072] It should also be noted that the selection of the original source point and the real source point should be reasonably arranged according to the specific application scenario to ensure the best transformation effect.

[0073] The general form for calculating the perspective transformation matrix H is:

[0074]

[0075] For perspective transformation, the four points (x1,y1), (x2,y2), (x3,y3), (x4,y4) of the real source point can be mapped to the four points (x1′,y1′), (x2′,y2′), (x3′,y3′), (x4′,y4′) of the original source point. 11 ,h 12 ,...,h 33 These are the elements of the perspective transformation matrix. Using these corresponding points, the perspective transformation matrix can be calculated through the following steps:

[0076] (1) Setting up equations: Establish a system of equations (i.e., a coordinate transformation equation system) to describe the transformation relationship of the points. The transformation of each point can be expressed in the following form:

[0077]

[0078] (2) Expanding the equations: Expanding the above equations, we can obtain the following system of equations:

[0079] `

[0080] x = h 11 x+h 12 x+h 13

[0081] `

[0082] y = h 21 x+h 22 x+h 23

[0083] 1 = h 31 x+h 32 x+h 33

[0084] (3) Constructing the matrix: Combining these equations together, we can obtain a matrix equation Ah = 0, where A is a matrix containing the coefficients of all equations, and h is the column vector of the perspective transformation matrix:

[0085]

[0086] (4) Solve the equation: By solving the eigenvalue problem of the above matrix or by using the least squares method, the parameters of the perspective transformation matrix H can be obtained.

[0087] In this embodiment, the calculated perspective transformation matrix H is used to transform each trajectory image in multiple pedestrian trajectory videos. Specifically, the cv2.warpPerspective(image,H,(width,height)) function can be called to convert each trajectory image in multiple pedestrian trajectory videos into the target shape (i.e., the desired rectangle), thereby eliminating distortion.

[0088] In the specific implementation, it is also necessary to transform the time series trajectory data corresponding to each transformed trajectory image through a set of coordinate transformation equations, and sort the coordinate points of each trajectory after coordinate transformation according to the time order to obtain the spatiotemporal trajectory data of pedestrians.

[0089] It should also be understood that the transformed image and trajectory should be visually inspected to ensure there is no obvious distortion. If necessary, the original source point and the true source point should be reselected, and a new perspective transformation matrix should be calculated to optimize the effect.

[0090] Step S30: Spatial normalize the pedestrian spatiotemporal trajectory data, and map the spatially normalized pedestrian spatiotemporal trajectory data into the pixel space of the trajectory blank image to generate a trajectory grayscale image.

[0091] Furthermore, the spatial normalization process for pedestrian spatiotemporal trajectory data involves determining the horizontal width and vertical height of the walking monitoring area, and selecting the minimum and maximum coordinates of pedestrian footsteps from the pedestrian spatiotemporal trajectory data based on the walking monitoring area; constructing a normalized coordinate formula based on the horizontal width, vertical height, minimum and maximum coordinates of pedestrian footsteps; and then spatially normalizing the pedestrian spatiotemporal trajectory data using the normalized coordinate formula.

[0092] In practical implementation, spatial normalization of pedestrian spatiotemporal trajectory data is crucial. By scaling the coordinates to a consistent range (usually [0,1]), the accurate representation of the trajectory shape is ensured, thereby ensuring that all trajectory information can be represented at the same scale regardless of the actual size of the monitored scene.

[0093] The pedestrian's walking range is determined by the minimum and maximum coordinates of the monitored area, using the formula:

[0094] d x =x max -x min

[0095] d y =y max -y min

[0096] The minimum coordinates of the monitored area (i.e., the minimum coordinates of the pedestrian's footsteps) are (x min y min The maximum coordinates (i.e., the maximum coordinates of the pedestrian's footsteps) are (x max y max ), d x d represents the horizontal width (or span) of the monitored area. y This indicates the vertical height (or span) of the monitored area.

[0097] The following normalized coordinate formula ensures that the generated trajectory image is scaled uniformly.

[0098]

[0099] Assuming the pedestrian's trajectory coordinates in the spatiotemporal trajectory data are (x, y), and the normalized coordinates are (normalized...) x normalized y Using the above formula, the coordinates of all trajectory points in the pedestrian spatiotemporal trajectory data are scaled to a uniform range (usually [0,1]). (Refer to...) Figure 3 , Figure 3 This is a schematic diagram of trajectory grayscale image construction in the first embodiment of the pedestrian trajectory abnormal behavior detection method of the present invention. Figure 3 In the figure, 'a' represents the trajectory map formed by the spatially normalized spatiotemporal trajectory data of pedestrians.

[0100] In this embodiment, reference Figure 3 In step 'b', the spatially normalized pedestrian spatiotemporal trajectory data is mapped to the pixel space of the image (i.e., the pixel space of the trajectory blank image). By using appropriate scaling and offset, the physical coordinates of the spatially normalized pedestrian spatiotemporal trajectory data are converted into the corresponding pixel coordinates within the trajectory blank image, ensuring that the mapped coordinates accurately reflect the object's position in the trajectory blank image.

[0101] In the specific implementation, the normalized coordinates (i.e., the spatially normalized pedestrian spatiotemporal trajectory data) are converted into pixel coordinates in the trajectory blank image. Given that the trajectory blank image is of size N×N, the pixel coordinates are calculated as follows:

[0102] pixel x =normalized x ×N

[0103] pixel y =normalized y ×N

[0104] pixel x The x-coordinate of a pixel in the trajectory blank image, pixel y y is the vertical coordinate of the pixel in the blank image of the trajectory.

[0105] refer to Figure 3 In the algorithm, 'c' assigns a value to each pixel in the grid based on the number of trajectory points that fall on each pixel. Specifically, the more trajectory points, the longer the object stays in that grid (corresponding to a small area in reality), and therefore the higher the pixel value assigned to that grid. This assignment mechanism effectively captures the temporal attributes of an object within a specific area.

[0106] Convert the processed trajectory data into Figure 3 The image shown in d is an example. The grayscale value of each pixel reflects the number of trajectory points within the corresponding pixel, forming an image that displays the motion pattern and temporal attributes of the object (i.e., a trajectory grayscale image).

[0107] Step S40: Color-encode the trajectory grayscale image according to the time information and pedestrian information to obtain a color gradient trajectory image.

[0108] Furthermore, time information is encoded into the hue and saturation channels, and pedestrian information is encoded into the luminance channel; based on the hue, saturation, and luminance channels, the pixels of each pedestrian trajectory in the trajectory grayscale image are color-coded to obtain a color gradient trajectory image.

[0109] In this embodiment, reference Figure 4 , Figure 4 This is a schematic diagram of color gradient trajectory image construction in the first embodiment of the pedestrian trajectory abnormal behavior detection method of the present invention, with time information t1-t. n The image uses different colors corresponding to different times, mapping time information to color changes to enhance the temporal characteristics of the image. This allows the trajectory image to not only display spatial motion but also reflect information in the temporal dimension.

[0110] The coordinates of each trajectory point are assigned a corresponding color value on the image according to the normalized position, forming a visualized trajectory image for subsequent analysis and recognition.

[0111] Temporal information is encoded into the hue and saturation channels of the image, and pedestrian information is encoded into the value channel, ensuring that the color of all trajectory points is within a predetermined range.

[0112] The specific steps are as follows:

[0113] (1) Define initial saturation and brightness

[0114] Set the initial saturation S0 and initial brightness V0. Different initial brightness values ​​represent information about different pedestrians, so as to distinguish different pedestrians in the image.

[0115] (2) Calculate hue and saturation

[0116] Time information is encoded into the hue and saturation channels to determine a given time t. i The color value is calculated as follows:

[0117]

[0118] S i =S0+t i *mod*p

[0119] Among them, H i Represents the current time t i The corresponding hue value, S i Represents the current time t i The corresponding saturation value, p represents a fixed time interval, and S0 represents the initial saturation.

[0120] (3) Encoding pedestrian information

[0121] Pedestrian information is encoded into the luminance (Value) channel using the following formula:

[0122] V = V0 + ΔV*P

[0123] Where V0 represents the initial brightness value, P is the pedestrian identifier (such as pedestrian number), and ΔV is the change in brightness, so that different pedestrians have different brightness values.

[0124] Assign a corresponding HSV color value to each location coordinate based on time information to generate a color gradient trajectory image.

[0125] Step S50: Detect whether there is overlap between trajectory points in the color gradient trajectory image.

[0126] Step S60: If not, input the color gradient trajectory image into the ConvNeXt network model so that the ConvNeXt network model outputs the trajectory abnormal behavior detection result.

[0127] Furthermore, the system detects whether there is overlap between trajectory points in the color gradient trajectory image. The method for detecting whether there is overlap is to detect each pixel in the color gradient trajectory image. If the HSV value of a certain pixel is non-zero, it indicates that there is trajectory overlap at that position. If there is overlap, a new trajectory grayscale image of the same dimension is created based on the trajectory grayscale image, and the HSV values ​​of the overlapping trajectory points in the color gradient trajectory image are determined. The HSV values ​​of the overlapping trajectory points are updated using the HSV update formula. The HSV values ​​of the non-overlapping trajectory points and the updated HSV values ​​of the overlapping trajectory points are mapped to the new trajectory grayscale image to obtain the color gradient trajectory image.

[0128] In the specific implementation, when overlap is detected, a new trajectory grayscale image with the same dimensions as the original trajectory grayscale image is created; overlap point processing: for each overlapping trajectory point, its HSV value is updated using the HSV update formula:

[0129] Assume H old ,S old Vold It is the HSV value of the original overlapping points, H new ,S new V new It is the updated HSV value at a certain time t. i The color value is calculated using the following method:

[0130]

[0131] S new =S0+t i modp

[0132] V new =V old

[0133] p represents a fixed time interval, S0 represents the initial saturation, and the brightness V new To maintain consistency in pedestrian trajectories.

[0134] The updated HSV values ​​are used in the new trajectory grayscale image to distinguish information about overlapping times and ensure the integrity of the time series.

[0135] refer to Figure 5 , Figure 5 To illustrate the abnormal behavior detection method for pedestrian trajectories of the first embodiment of the present invention, different types of pedestrian color gradient trajectory maps are plotted, for example... Figure 5 In (a) of the data, the trajectories of "acquaintance pairs" are usually parallel and close to each other. Figure 5 The trajectories of (b) the "hidden follower pairs" show an overlapping trend, while Figure 5 In (c), the trajectories of the "passersby" intersect and vary in distance, which helps to identify the hidden tailing phenomenon.

[0136] The time-series information of the trajectory is mapped onto the image through color changes. This not only preserves the spatial trajectory of pedestrians, but also clearly shows the dynamic changes of behavior over time through color gradients, effectively enhancing the ability to capture hidden abnormal behaviors.

[0137] It should also be noted that the reference Figure 6 , Figure 6 This is a schematic diagram of the ConvNeXt network model in the first embodiment of the pedestrian trajectory abnormal behavior detection method of the present invention. The last fully connected layer of the ConvNeXt network model (usually a layer used for multi-class classification) is replaced with a fully connected layer suitable for binary classification.

[0138] The ConvNeXt model, without relying on human-made features, can improve its ability to identify hidden abnormal behaviors by automatically learning the spatial and temporal features of different trajectories, thereby enhancing the model's generalization performance in different scenarios.

[0139] Furthermore, the color gradient trajectory image is input into the ConvNeXt network model to obtain the trajectory abnormal behavior confidence score; based on the trajectory abnormal behavior confidence score, the ConvNeXt network model outputs the trajectory abnormal behavior detection result.

[0140] In the implementation, color-coded pedestrian trajectory images are input to the ConvNeXt network model. To match the input dimension of ConvNeXt, the images need to be resized to a specific size (e.g., 224×224 pixels). A fully connected layer for binary classification is configured so that the number of output nodes is the same as the number of pedestrian categories. The model outputs a confidence score for abnormal trajectory behavior based on the features of the input image, thereby identifying whether there is any covert tailing behavior.

[0141] Based on the confidence score (e.g., 0.5) of the abnormal trajectory output by the model, it is determined whether there is covert tailing behavior. If the confidence score exceeds a set threshold, it is identified as covert tailing, triggering an alarm or logging. The identification results are fed back to the monitoring system, and security personnel are notified for further investigation if necessary.

[0142] In this embodiment, pedestrian surveillance video is first segmented based on a time window to obtain multiple pedestrian trajectory video segments. Then, the TraDeS model is used to extract the footstep coordinates of each frame in each pedestrian trajectory video segment to generate time-series trajectory data. A perspective transformation matrix is ​​then used to transform the coordinates of the time-series trajectory data to obtain pedestrian spatiotemporal trajectory data. Next, the spatiotemporal trajectory data is spatially normalized and mapped to the pixel space of a trajectory blank image to generate a trajectory grayscale image. Finally, the trajectory grayscale image is color-coded according to time and pedestrian information to obtain a color gradient trajectory image. The system detects whether trajectory points in the color gradient trajectory image overlap. If not, the color gradient trajectory image is input into the ConvNeXt network model, allowing the ConvNeXt network model to output the trajectory abnormal behavior detection result. This embodiment, by introducing spatiotemporal trajectory mapping and color coding techniques, combined with the ConvNeXt network model, can more accurately identify complex and hidden behaviors, reducing false alarms and false negatives.

[0143] Reference Figure 7 , Figure 7 This is a structural block diagram of the first embodiment of the pedestrian trajectory abnormal behavior detection system of the present invention.

[0144] like Figure 7 As shown, the pedestrian trajectory abnormal behavior detection system proposed in this embodiment of the invention includes:

[0145] The segmentation module 7001 is used to segment pedestrian monitoring videos based on time windows to obtain multiple segments of pedestrian trajectory videos;

[0146] The conversion module 7002 is used to extract the foot position coordinates of each frame of the pedestrian trajectory video in each segment of the pedestrian trajectory video through the TraDeS model to generate time series trajectory data, and to perform coordinate transformation on the time series trajectory data through the perspective transformation matrix to obtain pedestrian spatiotemporal trajectory data.

[0147] The mapping module 7003 is used to perform spatial normalization on the pedestrian spatiotemporal trajectory data and map the spatially normalized pedestrian spatiotemporal trajectory data to the pixel space of the trajectory blank image to generate a trajectory grayscale image.

[0148] Encoding module 7004 is used to color encode the trajectory grayscale image according to time information and pedestrian information to obtain a color gradient trajectory image;

[0149] Detection module 7005 is used to detect whether there is overlap between trajectory points in the color gradient trajectory image;

[0150] The detection module 7005 is further configured to, if not, input the color gradient trajectory image into the ConvNeXt network model so that the ConvNeXt network model outputs the trajectory abnormal behavior detection result.

[0151] Other embodiments or specific implementations of the pedestrian trajectory abnormal behavior detection system of the present invention can be referred to the above-described method embodiments, and will not be repeated here.

[0152] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0153] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0154] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0155] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A method for detecting abnormal behavior of a pedestrian trajectory, characterized in that, The abnormal behavior detection method for pedestrian trajectories includes the following steps: Pedestrian surveillance video is segmented based on time windows to obtain multiple segments of pedestrian trajectory video; The TraDeS model is used to extract the foot position coordinates of each frame in each segment of pedestrian trajectory video to generate time series trajectory data. The time series trajectory data is then transformed by a perspective transformation matrix to obtain the spatiotemporal trajectory data of pedestrians. The pedestrian spatiotemporal trajectory data is spatially normalized, and the spatially normalized pedestrian spatiotemporal trajectory data is mapped to the pixel space of the trajectory blank image to generate a trajectory grayscale image. The trajectory grayscale image is color-coded based on time information and pedestrian information to obtain a color gradient trajectory image; Detect whether there is overlap between trajectory points in the color gradient trajectory image; If not, the color gradient trajectory image is input into the ConvNeXt network model so that the ConvNeXt network model outputs the trajectory abnormal behavior detection result; The step of color encoding the trajectory grayscale image based on time information and pedestrian information to obtain a color gradient trajectory image includes: Time information is encoded into the hue and saturation channels, and pedestrian information is encoded into the luminance channel; The color of each pedestrian trajectory pixel in the trajectory grayscale image is color-coded based on the hue channel, the saturation channel and the brightness channel to obtain a color gradient trajectory image; After the step of detecting whether there is overlap between trajectory points in the color gradient trajectory image, the method further includes: If there is overlap, a new trajectory grayscale image of the same dimension is created based on the trajectory grayscale image, and the HSV value of the overlapping trajectory points in the color gradient trajectory image is determined. The HSV values ​​of the overlapping trajectory points are updated using the HSV update formula; The HSV values ​​of the non-overlapping trajectory points and the updated HSV values ​​of the overlapping trajectory points are mapped to the new trajectory grayscale image to obtain a color gradient trajectory image.

2. The method of claim 1, wherein, The step of segmenting pedestrian surveillance video based on a time window to obtain multiple segments of pedestrian trajectory video includes: Determine the frame rate and duration of the pedestrian surveillance video; The number of frames in the short video segment is determined based on the frame rate and the duration. The pedestrian monitoring video is segmented using a time window based on the number of frames in the short video segment to obtain multiple pedestrian trajectory video segments.

3. The method as described in claim 1, characterized in that, The step of performing coordinate transformation on the time series trajectory data using a perspective transformation matrix to obtain pedestrian spatiotemporal trajectory data includes: The original source point coordinates and the real source point coordinates of each trajectory image in multiple pedestrian trajectory videos were determined respectively; Calculate the perspective transformation matrix based on the original source point coordinates and the actual source point coordinates; The perspective transformation matrix is ​​used to transform each trajectory image in multiple pedestrian trajectory videos, and the time series trajectory data is transformed by a set of coordinate transformation equations based on the transformed trajectory images to obtain pedestrian spatiotemporal trajectory data.

4. The method as described in claim 1, characterized in that, The step of spatially normalizing the pedestrian spatiotemporal trajectory data includes: Determine the horizontal width and vertical height of the walking monitoring area, and select the minimum and maximum coordinates of the pedestrian's footsteps from the pedestrian spatiotemporal trajectory data based on the walking monitoring area; A normalized coordinate formula is constructed based on the horizontal width, the vertical height, the minimum coordinates of the pedestrian's foot position, and the maximum coordinates of the pedestrian's foot position; The pedestrian spatiotemporal trajectory data is spatially normalized using the normalized coordinate formula.

5. The method as described in claim 1, characterized in that, The step of inputting the color gradient trajectory image into the ConvNeXt network model so that the ConvNeXt network model outputs trajectory abnormal behavior detection results includes: The color gradient trajectory image is input into the ConvNeXt network model to obtain the confidence score of abnormal trajectory behavior; Based on the confidence level of the trajectory anomaly behavior, the ConvNeXt network model outputs the trajectory anomaly behavior detection result.

6. An abnormal behavior detection system for pedestrian trajectories, characterized in that, The abnormal behavior detection system for pedestrian trajectories includes: The segmentation module is used to segment pedestrian surveillance videos based on time windows to obtain multiple segments of pedestrian trajectory videos; The conversion module is used to extract the foot position coordinates of each frame in each segment of pedestrian trajectory video using the TraDeS model to generate time series trajectory data, and to perform coordinate transformation on the time series trajectory data using a perspective transformation matrix to obtain pedestrian spatiotemporal trajectory data. The mapping module is used to spatially normalize the pedestrian spatiotemporal trajectory data and map the spatially normalized pedestrian spatiotemporal trajectory data into the pixel space of the trajectory blank image to generate a trajectory grayscale image. The encoding module is used to encode time information into the hue channel and saturation channel, and to encode pedestrian information into the luminance channel; based on the hue channel, the saturation channel and the luminance channel, the pixel points of each pedestrian trajectory in the trajectory grayscale image are color encoded to obtain a color gradient trajectory image; The detection module is used to detect whether there is overlap between trajectory points in the color gradient trajectory image; The detection module is also used to input the color gradient trajectory image into the ConvNeXt network model if there is none, so that the ConvNeXt network model outputs the trajectory abnormal behavior detection result. The detection module is further configured to, if there is overlap, create a new trajectory grayscale image of the same dimension based on the trajectory grayscale image, and determine the HSV values ​​of the overlapping trajectory points in the color gradient trajectory image; update the HSV values ​​of the overlapping trajectory points using the HSV update formula; and map the HSV values ​​of the non-overlapping trajectory points and the updated HSV values ​​of the overlapping trajectory points to the new trajectory grayscale image to obtain the color gradient trajectory image.

7. An abnormal behavior detection device for pedestrian trajectories, characterized in that, The device includes: a memory, a processor, and an abnormal behavior detection program for pedestrian trajectories stored in the memory and executable on the processor, the abnormal behavior detection program for pedestrian trajectories being configured to implement the steps of the abnormal behavior detection method for pedestrian trajectories as described in any one of claims 1 to 5.

8. A storage medium, characterized in that, The storage medium stores an abnormal behavior detection program for pedestrian trajectories, and when the abnormal behavior detection program for pedestrian trajectories is executed by the processor, it implements the steps of the abnormal behavior detection method for pedestrian trajectories as described in any one of claims 1 to 5.

Images