Beidou video multi-behavior analysis early warning system and terminal

Abstract

The invention relates to the technical field of video monitoring, and discloses a Beidou video multi-behavior analysis early warning system and terminal, and a video acquisition module, which are used for constructing a time-space synchronous video monitoring frame, and preprocessing an acquired video stream by adopting a self-adaptive image enhancement algorithm to obtain a preprocessed video; the multi-modal feature fusion module is used for extracting pedestrian local space-time features from the preprocessed video by using a 3D-CNN network, acquiring human body feature coordinates in combination with an HRNet model, and mapping Beidou positioning camera coordinates and the human body feature coordinates to a unified geographic coordinate system to obtain a multi-modal feature set; the behavior recognition module is used for modeling an action sequence of a pedestrian limb based on an ST-GCN network, capturing a long-term dependency relationship in combination with a Transform encoder, and determining an abnormal behavior and a corresponding grade; the early warning module is used for triggering a corresponding early warning mechanism according to the abnormal behavior level and pushing abnormal information; the early warning efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of video surveillance technology, specifically to a BeiDou video multi-behavior analysis and early warning system and terminal. Background Technology

[0002] Existing video behavior analysis systems mostly rely on data from a single camera, resulting in problems such as low spatiotemporal correlation accuracy, poor generalization of behavior recognition algorithms, and delayed response to anomaly warnings. In complex scenarios, it is difficult to accurately correlate video streams with geographical location and time information, and the ability to extract refined features and analyze behavior of specific pedestrians is insufficient. The timeliness and linkage of abnormal behavior warnings also need to be improved, failing to meet the needs of multi-scenario, high-precision, and intelligent security monitoring. Summary of the Invention

[0003] The purpose of this invention is to solve the above-mentioned problems by designing a Beidou video multi-behavior analysis and early warning system and terminal.

[0004] The first aspect of this invention provides a BeiDou video multi-behavior analysis and early warning system, the system comprising: The video acquisition module is used to construct a time-space synchronized video surveillance framework by utilizing the positioning, timing and short message communication functions of the Beidou satellite. It uses an adaptive image enhancement algorithm to preprocess the acquired video stream to obtain the preprocessed video. The multimodal feature fusion module is used to extract local spatiotemporal features of pedestrians from preprocessed videos using a 3D-CNN network, and to obtain human body feature coordinates by combining the HRNet model. The camera coordinates of Beidou positioning and human body feature coordinates are mapped to a unified geographic coordinate system to obtain a multimodal feature set. The behavior recognition module is used to model the action sequence of pedestrian limbs based on the ST-GCN network, and capture long-term dependencies in combination with the Transformer encoder. When human behavior deviates from the baseline, it is identified as abnormal behavior and its corresponding level. The early warning module is used to trigger corresponding early warning mechanisms based on the level of abnormal behavior and push abnormal information via BeiDou short message service.

[0005] Optionally, in a first implementation of the first aspect of the present invention, the video acquisition module includes: The parsing submodule is used to acquire BeiDou satellite signals in real time through the BeiDou satellite receiving antenna, parse out positioning data and timing information, and bind this information with the camera device ID to establish a basic spatiotemporal identifier; A submodule is constructed to build a spatiotemporal synchronization framework based on the bound spatiotemporal identifier, and to periodically send device spatiotemporal calibration requests to the system server through the BeiDou short message communication function; The splitting submodule is used to start the camera to capture video streams, split the original video stream according to the frame sequence, and attach the corresponding BeiDou timestamp and camera geographic coordinates to each video frame; The preprocessing submodule is used to preprocess each frame of video using an adaptive image enhancement algorithm, reassemble the processed video frames into a continuous video stream, and output the preprocessed video containing spatiotemporal identifiers.

[0006] Optionally, in a second implementation of the first aspect of the present invention, the preprocessing submodule includes: The lighting conditions of video frames are determined by analyzing the brightness histogram. For overly bright frames, an adaptive threshold segmentation contrast compression algorithm is used to reduce the pixel values ​​in the highlight areas. For excessively dark frames, the multi-scale Retinex algorithm is used to decompose the image illumination and reflection components, and the reflection component is enhanced to improve details; For frames with normal lighting, Gaussian filtering is used for noise reduction.

[0007] Optionally, in a third implementation of the first aspect of the present invention, the multimodal feature fusion module includes: The input submodule is used to receive the preprocessed video, extract video clips containing pedestrians, and use them as input to the 3D-CNN network. The extraction submodule is used by the 3D-CNN network to process video clips layer by layer. It slides the three-dimensional convolution kernel across multiple scales to extract the spatiotemporal features of pedestrian limbs and outputs a local spatiotemporal feature map. The capture submodule is used to input keyframes from video clips into the HRNet model. It extracts features through parallel high-resolution and low-resolution convolutional branches. The high-resolution branch preserves human detail features, while the low-resolution branch captures global features. Information is exchanged between branches through cross-scale connections. The transformation submodule is used to convert the pixel coordinates of the human feature nodes output by the HRNet model into three-dimensional coordinates in the camera coordinate system by combining the camera intrinsic parameters corresponding to the keyframes. A submodule is created to establish a transformation matrix between the camera coordinate system and the geographic coordinate system based on the camera's geographic coordinates and extrinsic parameters. The association submodule is used to substitute the 3D coordinates of the camera coordinate system of the human feature node into the transformation matrix to obtain the human feature coordinates in the geographic coordinate system, and associate them with the local spatiotemporal features extracted by 3D-CNN to form a multimodal feature set that integrates spatiotemporal information.

[0008] Optionally, in a fourth implementation of the first aspect of the present invention, the extraction submodule includes: The first convolutional layer of the 3D-CNN network initializes a three-dimensional convolutional kernel, which contains a time dimension, a height dimension, and a width dimension, and randomly assigns initial weight parameters to each convolutional kernel; The 3D convolutional kernel performs sliding calculations on the video clip. It moves position by position in the spatial dimension with a preset step size, and slides frame by frame in the temporal dimension with the same step size. At each position, the convolutional kernel performs element-wise multiplication with the video data of the corresponding region and sums the results to obtain the feature value of the current position. After the first convolutional layer completes the sliding computation, an initial spatiotemporal feature map is generated. The spatiotemporal feature map is then input into the ReLU function, and negative features are filtered out through nonlinear transformation. The activated feature map is fed into the next convolutional layer, and the sliding calculation operation is repeated using 3D convolutional kernels of different sizes to extract richer local spatiotemporal features. After being processed layer by layer through multiple convolutional layers, each layer reduces the dimensionality of the feature map through pooling operations. The maximum value is selected as the feature representative of the current window according to a fixed window in both time and space dimensions. Finally, the output is a local spatiotemporal feature map of the pedestrian limb after multiple layers of abstraction.

[0009] Optionally, in a fifth implementation of the first aspect of the present invention, the behavior recognition module includes: The arrangement submodule is used to extract the geographic coordinates of key human body nodes from the multimodal feature set and arrange them in chronological order to form a sequence of limb movements. The convolutional submodule is used to input the action sequence into the ST-GCN network, construct the spatiotemporal graph structure, learn features of the spatiotemporal graph through spatiotemporal convolutional layers, capture the correlation features of different limb nodes within the same time frame, capture the motion trend of the same node between different time frames, and output the spatiotemporal feature sequence of limb actions. The encoding submodule is used to input the spatiotemporal feature sequence into the Transformer encoder, perform position encoding on the spatiotemporal feature sequence, and add position information in the time dimension; The nonlinear transformation submodule is used to calculate the association weight of each feature with all other features through a multi-head self-attention mechanism, capture behavioral dependencies over a long time span, and perform nonlinear transformation on the attention-weighted features through a feedforward neural network. The comparison submodule is used to compare the behavior feature vectors output by the Transformer encoder after long-term dependency modeling with the preset normal behavior baseline features and calculate the feature deviation. The judgment submodule is used to determine abnormal behavior when the deviation exceeds a set threshold, and classify the abnormality level according to the magnitude of the deviation.

[0010] Optionally, in a sixth implementation of the first aspect of the present invention, the spatiotemporal graph structure uses key human body nodes as nodes of the graph, establishes spatial edges based on the human body structure, and establishes temporal edges based on the position changes of the same node in adjacent time frames.

[0011] Optionally, in a seventh implementation of the first aspect of the present invention, the early warning module includes: Receive abnormal behavior information, call the preset early warning rule library according to the abnormality level and trigger the corresponding early warning mechanism; The warning message text is encoded using the BeiDou short message module, encapsulated into short message data according to the BeiDou communication protocol, and sent to the preset monitoring center terminal and the mobile device of the person in charge. In the event of a high-level anomaly, a linkage control command is sent to the surrounding associated smart devices, based on the triggering of the medium-level early warning mechanism. The system receives delivery status information from BeiDou short messages and execution feedback information from intelligent devices, and stores the warning trigger time, execution result, and feedback status in the system log.

[0012] Optionally, in the eighth implementation of the first aspect of the present invention, when a low-level anomaly occurs, a start command is sent to the sound and light alarm associated with the on-site camera to control the alarm to emit flashing lights and a prompt sound. When a medium-level anomaly occurs, a warning message text containing anomaly details is generated.

[0013] A second aspect of the present invention provides a BeiDou video multi-behavior analysis and early warning terminal, including a memory and a processor, wherein the memory stores a computer program that can be loaded and executed by the processor to provide a BeiDou video multi-behavior analysis and early warning system.

[0014] The technical solution provided by this invention utilizes the positioning, timing, and short message communication functions of the BeiDou satellite system to construct a spatiotemporally synchronized video surveillance framework. An adaptive image enhancement algorithm is used to preprocess the acquired video stream, resulting in preprocessed video. A 3D-CNN network is used to extract local spatiotemporal features of pedestrians from the preprocessed video, and an HRNet model is combined to obtain human body feature coordinates. The camera coordinates based on BeiDou positioning and the human body feature coordinates are mapped to a unified geographic coordinate system to obtain a multimodal feature set. Based on an ST-GCN network, a sequence of pedestrian limb movements is modeled, and a Transformer encoder is used to capture long-term dependencies. When a dependency is detected... When human behavior deviates from the baseline, it is identified as abnormal behavior and its corresponding level. Based on the level of abnormal behavior, a corresponding early warning mechanism is triggered, and abnormal information is pushed via BeiDou short message service. This invention utilizes BeiDou to achieve nanosecond-level timing and centimeter-level positioning, combined with adaptive enhancement to solve the problem of spatiotemporal synchronization. It integrates 3D-CNN and HRNet, unifies the geographic coordinate system, achieves multi-dimensional feature extraction, strengthens behavior description, and captures long-term behavioral dependencies through ST-GCN and Transformer, reducing false positives and false negatives, improving recognition accuracy. It features tiered early warning linkage, BeiDou short message service to ensure information transmission, and linkage with intelligent devices to form a closed loop, improving early warning and emergency response efficiency. Attached Figure Description

[0015] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0016] Figure 1 This is a schematic diagram of the structure of the BeiDou video multi-behavior analysis and early warning system provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of the video acquisition module provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of the multimodal feature fusion module provided in an embodiment of the present invention. Detailed Implementation

[0017] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” or “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0018] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 A schematic diagram of the structure of the BeiDou video multi-behavior analysis and early warning system provided in this embodiment of the invention. The system includes: The video acquisition module is used to construct a time-space synchronized video surveillance framework by utilizing the positioning, timing and short message communication functions of the Beidou satellite. It uses an adaptive image enhancement algorithm to preprocess the acquired video stream to obtain the preprocessed video. The multimodal feature fusion module is used to extract local spatiotemporal features of pedestrians from preprocessed videos using a 3D-CNN network, and to obtain human body feature coordinates by combining the HRNet model. The camera coordinates of Beidou positioning and human body feature coordinates are mapped to a unified geographic coordinate system to obtain a multimodal feature set. The behavior recognition module is used to model the action sequence of pedestrian limbs based on the ST-GCN network, and capture long-term dependencies in combination with the Transformer encoder. When human behavior deviates from the baseline, it is identified as abnormal behavior and its corresponding level. The early warning module is used to trigger corresponding early warning mechanisms based on the level of abnormal behavior and push abnormal information via BeiDou short message service.

[0019] In this embodiment, please refer to Figure 2 The video capture module includes: The parsing submodule is used to acquire BeiDou satellite signals in real time through the BeiDou satellite receiving antenna, parse out positioning data and timing information, and bind this information with the camera device ID to establish a basic spatiotemporal identifier; A submodule is constructed to build a spatiotemporal synchronization framework based on the bound spatiotemporal identifier, and to periodically send device spatiotemporal calibration requests to the system server through the BeiDou short message communication function; The splitting submodule is used to start the camera to capture video streams, split the original video stream according to the frame sequence, and attach the corresponding BeiDou timestamp and camera geographic coordinates to each video frame; The preprocessing submodule is used to preprocess each frame of video using an adaptive image enhancement algorithm, reassemble the processed video frames into a continuous video stream, and output the preprocessed video containing spatiotemporal identifiers.

[0020] In this embodiment, the system equips each video acquisition terminal with a high-sensitivity BeiDou satellite receiving antenna. The antenna continuously receives radio frequency signals emitted by multiple BeiDou satellites and transmits them to the BeiDou signal processing chip built into the terminal. The chip demodulates and decodes the signals to extract positioning data with centimeter-level accuracy and timing information with nanosecond-level accuracy. At the same time, the terminal reads its own preset unique camera device ID and encapsulates the positioning data, timing information and device ID into a set of associated data through a custom data association algorithm. This data is stored in the terminal's local cache to form the basic spatiotemporal identifier of the camera, ensuring that the spatiotemporal information of each device uniquely corresponds to its own identity. The system server pre-sets a spatiotemporal calibration cycle, and each video acquisition terminal triggers the calibration process according to the cycle. The terminal retrieves its own basic spatiotemporal identifier from its local cache and generates a calibration request data packet containing the device ID, current positioning data, and current time synchronization information. Through the terminal's BeiDou short message communication module, the calibration request data packet is converted according to the BeiDou short message communication protocol and sent to the system server. After receiving the calibration requests from all terminals, the server compares the time synchronization information of each terminal with the server's reference time and compares the positioning data of each terminal with the preset device installation location reference value. It calculates the time deviation value and position deviation value of each terminal and then feeds back the calibration instruction to the corresponding terminal through BeiDou short message. The terminal adjusts its local spatiotemporal parameters according to the calibration instruction to achieve a unified spatiotemporal reference for all devices in the network and build a stable spatiotemporal synchronization framework. After the spatiotemporal synchronization framework is constructed, the system sends a video acquisition start command to each video acquisition terminal. The terminal activates the associated camera and acquires the live video stream according to the preset resolution and frame rate. The video processing unit built into the terminal splits the acquired raw video stream into frame sequences, breaking down the continuous video stream into independent video frames according to time order. At the same time, the system obtains the timestamp and geographic coordinates after BeiDou calibration from the local machine in real time, and writes the timestamp and geographic coordinates into the metadata field of each video frame through a data embedding algorithm. After the information is appended, the terminal reorders the processed video frames according to the original frame sequence to form a video stream containing spatiotemporal information.

[0021] In this embodiment, the preprocessing submodule includes: The lighting conditions of video frames are determined by analyzing luminance histograms. First, the luminance value distribution of all pixels in the video frame is statistically analyzed to generate a luminance histogram. The horizontal axis represents the luminance level from 0 to 255, and the vertical axis represents the number of pixels at the corresponding luminance level. The luminance ranges where pixels are concentrated in the histogram are calculated. When the proportion of pixels in the high luminance range, such as 200-255, exceeds a preset threshold, such as 30%, it is determined to be an overly bright frame. When the proportion of pixels in the low luminance range, such as 0-50, exceeds a preset threshold, such as 30%, it is determined to be an overly dark frame. All other cases are determined to be frames with normal lighting. For overly bright frames, an adaptive threshold segmentation contrast compression algorithm is used. First, the image is divided into highlight regions and non-highlight regions through local threshold segmentation. The local threshold is determined by calculating the average brightness of the neighborhood around each pixel. When the brightness of a pixel exceeds a preset proportion of the average brightness of its neighborhood, it is marked as a highlight region. Contrast compression processing is performed separately on the highlight regions. Piecewise linear transformation is used to compress the brightness value of the highlight regions to a preset normal brightness range proportionally, while keeping the brightness value of the non-highlight regions unchanged to avoid darkening the overall image. For excessively dark frames, a multi-scale Retinex algorithm is used to decompose the illumination and reflection components of the image. First, multiple Gaussian filter kernels of different scales are constructed, and Gaussian blurring is applied to the excessively dark frames to obtain multi-scale blurred images. These blurred images approximately represent the illumination components at different scales. The original excessively dark frame and the blurred image at each scale are then subjected to logarithmic difference operation to obtain the estimated value of the reflection component at the corresponding scale. The reflection component contains the image's detail information. The multi-scale reflection components are weighted and averaged to obtain the final reflection component. Finally, the logarithmic reflection component is transformed back to the original domain through exponential transformation to achieve detail enhancement. For normal lighting frames, Gaussian filtering is used for noise reduction. First, based on the noise characteristics of normal lighting frames, an appropriate Gaussian filter kernel size, such as 3×3 or 5×5, is selected. The standard deviation of the filter kernel is set according to the image noise intensity. The Gaussian filter kernel is then slid across each pixel position in the video frame. The output value of each pixel is the sum of the products of all pixel values ​​in its neighborhood and the corresponding filter kernel weight. Through this weighted averaging operation, isolated noise points caused by sensor noise in the image are smoothed, while preserving the edge and detail information of the image to the maximum extent.

[0022] In this embodiment, please refer to Figure 3 The multimodal feature fusion module includes: The input submodule is used to receive the preprocessed video, extract video clips containing pedestrians, and use them as input to the 3D-CNN network. The extraction submodule is used by the 3D-CNN network to process video clips layer by layer. It slides the three-dimensional convolution kernel across multiple scales to extract the spatiotemporal features of pedestrian limbs and outputs a local spatiotemporal feature map. The capture submodule is used to input keyframes from video clips into the HRNet model. It extracts features through parallel high-resolution and low-resolution convolutional branches. The high-resolution branch preserves human detail features, while the low-resolution branch captures global features. Information is exchanged between branches through cross-scale connections. The transformation submodule is used to convert the pixel coordinates of the human feature nodes output by the HRNet model into three-dimensional coordinates in the camera coordinate system by combining the camera intrinsic parameters corresponding to the keyframes. A submodule is created to establish a transformation matrix between the camera coordinate system and the geographic coordinate system based on the camera's geographic coordinates and extrinsic parameters. The association submodule is used to substitute the 3D coordinates of the camera coordinate system of the human feature node into the transformation matrix to obtain the human feature coordinates in the geographic coordinate system, and associate them with the local spatiotemporal features extracted by 3D-CNN to form a multimodal feature set that integrates spatiotemporal information.

[0023] In this embodiment, the extraction submodule includes: The first convolutional layer of the 3D-CNN network initializes a three-dimensional convolutional kernel, which contains a time dimension, a height dimension, and a width dimension, and randomly assigns initial weight parameters to each convolutional kernel; The 3D convolutional kernel performs sliding calculations on the video clip. It moves position by position in the spatial dimension with a preset step size, and slides frame by frame in the temporal dimension with the same step size. At each position, the convolutional kernel performs element-wise multiplication with the video data of the corresponding region and sums the results to obtain the feature value of the current position. After the first convolutional layer completes the sliding computation, an initial spatiotemporal feature map is generated. The spatiotemporal feature map is then input into the ReLU function, and negative features are filtered out through nonlinear transformation. The activated feature map is fed into the next convolutional layer, and the sliding calculation operation is repeated using 3D convolutional kernels of different sizes to extract richer local spatiotemporal features. After being processed layer by layer through multiple convolutional layers, each layer reduces the dimensionality of the feature map through pooling operations. The maximum value is selected as the feature representative of the current window according to a fixed window in both time and space dimensions. Finally, the output is a local spatiotemporal feature map of the pedestrian limb after multiple layers of abstraction.

[0024] In this embodiment, the behavior recognition module includes: The arrangement submodule is used to extract the geographic coordinates of key human body nodes from the multimodal feature set and arrange them in chronological order to form a sequence of limb movements. The convolutional submodule is used to input the action sequence into the ST-GCN network, construct the spatiotemporal graph structure, learn features of the spatiotemporal graph through spatiotemporal convolutional layers, capture the correlation features of different limb nodes within the same time frame, capture the motion trend of the same node between different time frames, and output the spatiotemporal feature sequence of limb actions. The encoding submodule is used to input the spatiotemporal feature sequence into the Transformer encoder, perform position encoding on the spatiotemporal feature sequence, and add position information in the time dimension; The nonlinear transformation submodule is used to calculate the association weight of each feature with all other features through a multi-head self-attention mechanism, capture behavioral dependencies over a long time span, and perform nonlinear transformation on the attention-weighted features through a feedforward neural network. The comparison submodule is used to compare the behavior feature vectors output by the Transformer encoder after long-term dependency modeling with the preset normal behavior baseline features and calculate the feature deviation. The judgment submodule is used to determine abnormal behavior when the deviation exceeds a set threshold, and classify the abnormality level according to the magnitude of the deviation.

[0025] In this embodiment, the spatiotemporal graph structure uses key human body nodes as nodes, establishes spatial edges based on the human body structure, and establishes temporal edges based on the position changes of the same node in adjacent time frames.

[0026] In this embodiment, the early warning module receives abnormal behavior information transmitted by the behavior recognition module. This information includes the specific type of abnormal behavior, such as loitering, climbing, conflict, etc.; the abnormality level determined by the system as low, medium, or high; the precise timestamp of the behavior occurrence provided by BeiDou time synchronization; and the corresponding geographical coordinates (latitude, longitude, and altitude) provided by BeiDou positioning, ensuring that all key information is fully recorded. Based on the received anomaly level, the system automatically calls the preset warning rule library, which stores the response strategies corresponding to different levels of anomalies: low-level anomalies are associated with local sound and light alarm rules, specifying the alarm's activation parameters such as light flashing frequency, alert volume and type; medium-level anomalies are associated with remote information push rules, including a list of information receiving terminals, push content format and priority; high-level anomalies are associated with linkage control rules, covering the types of smart devices to be linked, device IDs and specific control command templates. When the low-level abnormality warning mechanism is triggered, the system matches the corresponding on-site camera device ID based on the geographical coordinates of the abnormal behavior, then finds the bound sound and light alarm through the device association table, and sends a command data packet containing the start signal to it. The command specifies the interval of the flashing light, such as once every 0.5 seconds, and the duration of the alarm sound, such as continuous looping. After receiving the command, the alarm immediately executes it, emitting flashing lights and alarm sounds to warn people in the surrounding area. When the medium-level anomaly warning mechanism is triggered, the system first generates a warning information text according to a preset template. The text content includes the type of abnormal behavior, the time of occurrence, the precise geographical location, the anomaly level, and the on-site camera number. Then, the text is transmitted to the Beidou short message module. The module performs ASCII encoding conversion on the text and adds a message header containing the sender ID, receiver ID, message length, and check code according to the format requirements of the Beidou short message communication protocol. The message is then encapsulated into short message data that conforms to the transmission standard. The encapsulated short message data is sent sequentially to the monitoring center terminals (such as the monitoring platform server) and the mobile devices of the person in charge (such as terminals that support the BeiDou short message function) in the early warning rule base through the BeiDou satellite communication link. During the transmission process, the signal strength is monitored in real time to ensure the stability of data transmission. If the first transmission fails, the retransmission mechanism is automatically triggered, and the maximum number of retries is 3. When the high-level anomaly warning mechanism is triggered, the system, in addition to fully executing the medium-level anomaly warning mechanism, will initiate an additional linkage control process: based on the location of the anomaly, the system will use the geographic information system (GIS) to retrieve related intelligent devices within a 500-meter radius, such as access controllers, security robots, and lighting systems, extract the communication addresses and control protocols of these devices, generate targeted control commands according to the command templates in the rule base, and send them to each device through the wireless communication module. The system continuously monitors the delivery status information returned by the Beidou short message module, and at the same time receives the command execution results fed back by various linked intelligent devices. The system records the trigger time of the abnormal warning, the results of all execution steps, including the activation status of the audible and visual alarm, the delivery status of the short message, the response status of the intelligent devices, and the final feedback status, in chronological order into the system log database. The log format includes a unique identifier ID, event type, detailed content, and recording time to ensure that the entire warning process is traceable.

[0027] Based on the same inventive concept, embodiments of the present invention provide a Beidou video multi-behavior analysis and early warning terminal, including a memory and a processor, wherein the memory stores a computer program that can be loaded and executed by the processor for any of the above-mentioned Beidou video multi-behavior analysis and early warning systems.

[0028] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the various steps of the deep learning-based paper-cutting design pattern modification and generation method provided in the above embodiments.

[0029] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described equipment or apparatus / unit can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0030] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0031] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A BeiDou video multi-behavior analysis and early warning system, characterized in that, The system includes: The video acquisition module is used to construct a time-space synchronized video surveillance framework by utilizing the positioning, timing and short message communication functions of the Beidou satellite. It uses an adaptive image enhancement algorithm to preprocess the acquired video stream to obtain the preprocessed video. The multimodal feature fusion module is used to extract local spatiotemporal features of pedestrians from preprocessed videos using a 3D-CNN network, and to obtain human body feature coordinates by combining the HRNet model. The camera coordinates of Beidou positioning and human body feature coordinates are mapped to a unified geographic coordinate system to obtain a multimodal feature set. The behavior recognition module is used to model the action sequence of pedestrian limbs based on the ST-GCN network, and capture long-term dependencies in combination with the Transformer encoder. When human behavior deviates from the baseline, it is identified as abnormal behavior and its corresponding level. The early warning module is used to trigger corresponding early warning mechanisms based on the level of abnormal behavior and push abnormal information via BeiDou short message service.

2. The Beidou video multi-behavior analysis and early warning system as described in claim 1, characterized in that, The video acquisition module includes: The parsing submodule is used to acquire BeiDou satellite signals in real time through the BeiDou satellite receiving antenna, parse out positioning data and timing information, and bind this information with the camera device ID to establish a basic spatiotemporal identifier; A submodule is constructed to build a spatiotemporal synchronization framework based on the bound spatiotemporal identifier, and to periodically send device spatiotemporal calibration requests to the system server through the BeiDou short message communication function; The splitting submodule is used to start the camera to capture video streams, split the original video stream according to the frame sequence, and attach the corresponding BeiDou timestamp and camera geographic coordinates to each video frame; The preprocessing submodule is used to preprocess each frame of video using an adaptive image enhancement algorithm, reassemble the processed video frames into a continuous video stream, and output the preprocessed video containing spatiotemporal identifiers.

3. The Beidou video multi-behavior analysis and early warning system as described in claim 2, characterized in that, The preprocessing submodule includes: The lighting conditions of video frames are determined by analyzing the brightness histogram. For overly bright frames, an adaptive threshold segmentation contrast compression algorithm is used to reduce the pixel values ​​in the highlight areas. For excessively dark frames, the multi-scale Retinex algorithm is used to decompose the image illumination and reflection components, and the reflection component is enhanced to improve details; For frames with normal lighting, Gaussian filtering is used for noise reduction.

4. The BeiDou video multi-behavior analysis and early warning system as described in claim 1, characterized in that, The multimodal feature fusion module includes: The input submodule is used to receive the preprocessed video, extract video clips containing pedestrians, and use them as input to the 3D-CNN network. The extraction submodule is used by the 3D-CNN network to process video clips layer by layer. It slides the three-dimensional convolution kernel across multiple scales to extract the spatiotemporal features of pedestrian limbs and outputs a local spatiotemporal feature map. The capture submodule is used to input keyframes from video clips into the HRNet model. It extracts features through parallel high-resolution and low-resolution convolutional branches. The high-resolution branch preserves human detail features, while the low-resolution branch captures global features. Information is exchanged between branches through cross-scale connections. The transformation submodule is used to convert the pixel coordinates of the human feature nodes output by the HRNet model into three-dimensional coordinates in the camera coordinate system by combining the camera intrinsic parameters corresponding to the keyframes. A submodule is created to establish a transformation matrix between the camera coordinate system and the geographic coordinate system based on the camera's geographic coordinates and extrinsic parameters. The association submodule is used to substitute the 3D coordinates of the camera coordinate system of the human feature node into the transformation matrix to obtain the human feature coordinates in the geographic coordinate system, and associate them with the local spatiotemporal features extracted by 3D-CNN to form a multimodal feature set that integrates spatiotemporal information.

5. The BeiDou video multi-behavior analysis and early warning system as described in claim 4, characterized in that, The extraction submodule includes: The first convolutional layer of the 3D-CNN network initializes a three-dimensional convolutional kernel, which contains a time dimension, a height dimension, and a width dimension, and randomly assigns initial weight parameters to each convolutional kernel; The 3D convolutional kernel performs sliding calculations on the video clip. It moves position by position in the spatial dimension with a preset step size, and slides frame by frame in the temporal dimension with the same step size. At each position, the convolutional kernel performs element-wise multiplication with the video data of the corresponding region and sums the results to obtain the feature value of the current position. After the first convolutional layer completes the sliding computation, an initial spatiotemporal feature map is generated. The spatiotemporal feature map is then input into the ReLU function, and negative features are filtered out through nonlinear transformation. The activated feature map is fed into the next convolutional layer, and the sliding calculation operation is repeated using 3D convolutional kernels of different sizes to extract richer local spatiotemporal features. After being processed layer by layer through multiple convolutional layers, each layer reduces the dimensionality of the feature map through pooling operations. The maximum value is selected as the feature representative of the current window according to a fixed window in both time and space dimensions. Finally, the output is a local spatiotemporal feature map of the pedestrian limb after multiple layers of abstraction.

6. The BeiDou video multi-behavior analysis and early warning system as described in claim 1, characterized in that, The behavior recognition module includes: The arrangement submodule is used to extract the geographic coordinates of key human body nodes from the multimodal feature set and arrange them in chronological order to form a sequence of limb movements. The convolutional submodule is used to input the action sequence into the ST-GCN network, construct the spatiotemporal graph structure, learn features of the spatiotemporal graph through spatiotemporal convolutional layers, capture the correlation features of different limb nodes within the same time frame, capture the motion trend of the same node between different time frames, and output the spatiotemporal feature sequence of limb actions. The encoding submodule is used to input the spatiotemporal feature sequence into the Transformer encoder, perform position encoding on the spatiotemporal feature sequence, and add position information in the time dimension; The nonlinear transformation submodule is used to calculate the association weight of each feature with all other features through a multi-head self-attention mechanism, capture behavioral dependencies over a long time span, and perform nonlinear transformation on the attention-weighted features through a feedforward neural network. The comparison submodule is used to compare the behavior feature vectors output by the Transformer encoder after long-term dependency modeling with the preset normal behavior baseline features and calculate the feature deviation. The judgment submodule is used to determine abnormal behavior when the deviation exceeds a set threshold, and classify the abnormality level according to the magnitude of the deviation.

7. A Beidou video multi-behavior analysis and early warning system as described in claim 6, characterized in that, In the spatiotemporal graph structure, key human body nodes are used as nodes in the graph. Spatial edges are established based on the human body structure, and temporal edges are established based on the positional changes of the same node in adjacent time frames.

8. The BeiDou video multi-behavior analysis and early warning system as described in claim 1, characterized in that, The early warning module includes: Receive abnormal behavior information, call the preset early warning rule library according to the abnormality level and trigger the corresponding early warning mechanism; The warning message text is encoded using the BeiDou short message module, encapsulated into short message data according to the BeiDou communication protocol, and sent to the preset monitoring center terminal and the mobile device of the person in charge. In the event of a high-level anomaly, a linkage control command is sent to the surrounding associated smart devices, based on the triggering of the medium-level early warning mechanism. The system receives delivery status information from BeiDou short messages and execution feedback information from intelligent devices, and stores the warning trigger time, execution result, and feedback status in the system log.

9. A Beidou video multi-behavior analysis and early warning system as described in claim 8, characterized in that, In the event of a low-level anomaly, a start command is sent to the audible and visual alarm associated with the on-site camera, controlling the alarm to emit flashing lights and a warning sound; When a medium-level anomaly occurs, a warning message text containing anomaly details is generated.

10. A Beidou video multi-behavior analysis and early warning terminal, characterized in that, It includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executed as described in any one of claims 1 to 9 for a Beidou video multi-behavior analysis and early warning system.

Images