Campus security control method and system based on AI video technology

Through the campus safety control method based on AI video technology, the existing campus video surveillance system is solved, accurate target identification and behavior analysis is achieved, and the intelligence level and automation of campus safety control is improved. level.

CN118887604AInactive Publication Date: 2024-11-01SHENZHEN HUABAIAN INTELLIGENT TECH CO LTD
View PDF 0 Cites 4 Cited by

Patent Information

Application Number
CN202410905084.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-08
Publication Date
2024-11-01
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing campus video surveillance system is inefficient, with false alarms and omissions, and it is impossible to realize automated abnormal event detection and early warning. It lacks a complete video analysis process, fine-grained target recognition, personalized behavior analysis and intelligent security early warning. Mechanism, it is difficult to form a comprehensive campus safety control system.

Method used

Adopt campus safety control methods based on AI video technology, visual load extraction and feature enhancement by obtaining campus monitoring data, dynamic target cutting and target classification, accurate identification of vehicles and pedestrians, and analysis of human structure strokes and motion situations. Obtain the movement status data of pedestrians and vehicles, define danger based on preset indicators, carry out unit warning integration, and implement campus safety control.

Benefits of technology

Improve the accuracy and reliability of the analysis results, reduce the risks of false alarms and missed reports, and realize intelligent campus safety warnings, such as automatic control of roadblocks and adjusting traffic lights, improving the efficiency and practicality of campus safety control, forming Complete closed loop of campus safety control.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118887604A_ABST
    Figure CN118887604A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of safety management and control, in particular to a campus safety management and control method and system based on an AI video technology. The method comprises the following steps: collecting campus monitoring data in real time, and carrying out feature extraction, target detection and recognition on video data to obtain motion state data of pedestrians and vehicles. Gait feature analysis is carried out on the pedestrian motion state, the stability and posture change of the pedestrian motion state are evaluated, and pedestrian motion state data are obtained. And aiming at a vehicle motion state, estimating a vehicle speed and reconstructing a vehicle track and a driving behavior to obtain vehicle motion state data. Safety evaluation is carried out on the motion states of the pedestrians and the vehicles according to preset safety indexes, and an alarm event is generated through unit early warning fusion; according to the invention, through integrating video analysis, target identification and behavior analysis technologies, intelligent perception and early warning of campus monitoring data are realized, and the intelligent level of campus security management and control is effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of security management and control technology, and in particular to a campus security management and control method and system based on AI video technology. Background Art

[0002] In the early days, campuses began to introduce video surveillance systems to strengthen security control, but they still mainly relied on manual analysis and processing of surveillance videos. This single video surveillance system is inefficient, has a large number of false alarms and omissions, and cannot truly meet the needs of campus security. Since it is impossible to achieve automated abnormal event detection and early warning, a large amount of manpower is required for real-time monitoring and post-event analysis, which is costly and prone to negligence. In addition, these video surveillance systems also lack integration with other security subsystems such as access control and alarms, and their security control capabilities are limited, making it difficult to form a comprehensive campus security control system. With the development of deep learning technology, AI-based video analysis systems have begun to be applied to campus security control. This system can automatically detect some abnormal behaviors, such as fighting and suspicious activities, which improves the efficiency of security control to a certain extent. However, due to the low accuracy rate, the problems of false alarms and omissions have not been fundamentally solved. In addition, existing methods usually lack a complete video analysis process, fine-grained target recognition, personalized behavior analysis, and intelligent security early warning mechanism key technologies, making it difficult to achieve comprehensive, accurate, and intelligent campus security control. Summary of the invention

[0003] Based on this, it is necessary to provide a campus security management method and system based on AI video technology to solve at least one of the above technical problems.

[0004] To achieve the above purpose, the campus safety management and control method based on AI video technology includes the following steps:

[0005] Step S1: acquiring campus monitoring data; performing visual load extraction on the campus monitoring data to obtain a frame image; performing feature enhancement on the frame image to generate an enhanced frame image;

[0006] Step S2: Perform dynamic target cutting on the enhanced frame image to generate a moving target image; perform target classification and identification on the moving target image to generate a vehicle target image and a pedestrian target image;

[0007] Step S3: Perform human body structure outlining on the pedestrian target image to obtain a human body outlining image; perform human motion situation analysis on the human body outlining image to obtain pedestrian motion state data;

[0008] Step S4: dynamically estimating the vehicle speed of the vehicle target image to obtain vehicle speed data; performing vehicle state deduction on the vehicle target image based on the vehicle speed data to generate vehicle motion state data;

[0009] Step S5: Based on the preset pedestrian safety motion state indicators, the pedestrian motion state data is defined as dangerous, and pedestrian dangerous motion data is generated; based on the preset pedestrian safety motion state indicators, the pedestrian motion state data is defined as dangerous, and vehicle dangerous motion data is obtained; the pedestrian dangerous motion data and vehicle dangerous motion data are integrated into a unit warning to implement campus safety management and control.

[0010] The present invention lays a good foundation for subsequent target detection and behavior analysis from the pre-processing steps of acquiring campus surveillance video data to feature enhancement. The accuracy and reliability of the analysis results can be effectively improved. Through dynamic target cutting and target classification, accurate identification of vehicles and pedestrians is achieved, which lays the foundation for the next step of personalized behavior analysis and can greatly improve the pertinence and effectiveness of the analysis results. Through human body structure tracing and motion state analysis, personalized motion state data of pedestrians can be obtained, and personalized motion state data of vehicles can be obtained through vehicle speed estimation and vehicle state deduction, which provides valuable input for subsequent safety assessment and early warning. Based on the preset pedestrian and vehicle safety motion state indicators, specific danger assessment of individual behavior can be carried out, which can greatly improve the accuracy of campus safety early warning and reduce the risk of false alarms and missed alarms. Through the fusion analysis of pedestrian and vehicle dangerous motion data, more intelligent campus safety early warning can be achieved, such as automatic control of roadblocks and adjustment of traffic lights, which can greatly improve the efficiency and practicality of campus safety management and control. Taking full advantage of the advantages of AI video technology, the key technologies of complete visual data processing, fine-grained target recognition, personalized behavior analysis, targeted safety assessment and intelligent early warning mechanism are adopted, which can effectively improve the overall performance of campus safety management and control. Therefore, the present invention realizes intelligent perception and early warning of campus monitoring data by integrating video analysis, target recognition, and behavior analysis technologies, effectively improving the intelligent level of campus security management and control.

[0011] In this specification, a campus safety management and control system based on AI video technology is provided, which is used to execute a campus safety management and control method based on AI video technology as described above. The campus safety management and control system based on AI video technology includes:

[0012] The image acquisition module is used to acquire campus monitoring data; perform visual load extraction on the campus monitoring data to obtain a frame image; perform feature enhancement on the frame image to generate an enhanced frame image;

[0013] The image recognition module is used to perform dynamic target cutting on the enhanced frame image to generate a moving target image; perform target classification and identification on the moving target image to generate a vehicle target image and a pedestrian target image;

[0014] The pedestrian analysis module is used to perform human body structure outlining on the pedestrian target image to obtain a human body outlining image; perform human motion situation analysis on the human body outlining image to obtain pedestrian motion state data;

[0015] The vehicle analysis module is used to dynamically estimate the vehicle speed of the vehicle target image to obtain the vehicle speed data; based on the vehicle speed data, the vehicle state is deduced for the vehicle target image to generate the vehicle motion state data;

[0016] The safety management and control module is used to define the dangerousness of pedestrian movement status data based on preset pedestrian safety movement status indicators to generate pedestrian dangerous movement data; define the dangerousness of pedestrian movement status data based on preset pedestrian safety movement status indicators to obtain vehicle dangerous movement data; and perform unit warning fusion of pedestrian dangerous movement data and vehicle dangerous movement data to implement campus safety management and control.

[0017] The benefit of the present invention lies in the real-time collection and intelligent analysis of campus monitoring data, which improves the immediacy and intelligence of security monitoring. Through multimodal feature extraction and fusion, accurate detection and identification of targets (pedestrians, vehicles) are achieved, and the accuracy of security monitoring is enhanced. The motion state of pedestrians and vehicles is carefully analyzed, including gait, speed, and trajectory, which can better evaluate and predict potential safety hazards, make intelligent judgments on monitoring data based on preset safety indicators, and automatically generate unit warning events, thereby improving the automation level of campus safety management and control. Real-time monitoring data, target recognition, motion state analysis, and safety warning links are organically combined to form a complete set of campus safety management and control closed loops, which improves the overall safety management efficiency. The campus safety management and control method based on AI video technology makes full use of the advantages of computer vision and pattern recognition, and has obvious advantages in improving the immediacy, accuracy, and automation level of campus safety monitoring. Therefore, the present invention realizes intelligent perception and early warning of campus monitoring data by integrating video analysis, target recognition, and behavior analysis technology, effectively improving the intelligent level of campus safety management and control. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram of the steps of a campus safety control method based on AI video technology;

[0019] Figure 2 for Figure 1 Detailed implementation steps of step S2 in the flowchart;

[0020] Figure 3 for Figure 1 Detailed implementation steps of step S3 in FIG.

[0021] Figure 4 for Figure 1Detailed implementation steps of step S4 in FIG.

[0022] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0023] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0024] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0025] It should be understood that, although the terms "first" and "second" are used here to describe each unit, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiment, the first unit can be referred to as the second unit, and similarly the second unit can be referred to as the first unit. The term "and / or" used here includes any and all combinations of one or more of the listed associated items.

[0026] To achieve this, please refer to Figures 1 to 4 , a campus safety management and control method based on AI video technology, comprising the following steps:

[0027] Step S1: acquiring campus monitoring data; performing visual load extraction on the campus monitoring data to obtain a frame image; performing feature enhancement on the frame image to generate an enhanced frame image;

[0028] Step S2: Perform dynamic target cutting on the enhanced frame image to generate a moving target image; perform target classification and identification on the moving target image to generate a vehicle target image and a pedestrian target image;

[0029] Step S3: Perform human body structure outlining on the pedestrian target image to obtain a human body outlining image; perform human motion situation analysis on the human body outlining image to obtain pedestrian motion state data;

[0030] Step S4: dynamically estimating the vehicle speed of the vehicle target image to obtain vehicle speed data; performing vehicle state deduction on the vehicle target image based on the vehicle speed data to generate vehicle motion state data;

[0031] Step S5: Based on the preset pedestrian safety motion state indicators, the pedestrian motion state data is defined as dangerous, and pedestrian dangerous motion data is generated; based on the preset pedestrian safety motion state indicators, the pedestrian motion state data is defined as dangerous, and vehicle dangerous motion data is obtained; the pedestrian dangerous motion data and vehicle dangerous motion data are integrated into a unit warning to implement campus safety management and control.

[0032] The present invention lays a good foundation for subsequent target detection and behavior analysis from the pre-processing steps of acquiring campus surveillance video data to feature enhancement. The accuracy and reliability of the analysis results can be effectively improved. Through dynamic target cutting and target classification, accurate identification of vehicles and pedestrians is achieved, which lays the foundation for the next step of personalized behavior analysis and can greatly improve the pertinence and effectiveness of the analysis results. Through human body structure tracing and motion state analysis, personalized motion state data of pedestrians can be obtained, and personalized motion state data of vehicles can be obtained through vehicle speed estimation and vehicle state deduction, which provides valuable input for subsequent safety assessment and early warning. Based on the preset pedestrian and vehicle safety motion state indicators, specific danger assessment of individual behavior can be carried out, which can greatly improve the accuracy of campus safety early warning and reduce the risk of false alarms and missed alarms. Through the fusion analysis of pedestrian and vehicle dangerous motion data, more intelligent campus safety early warning can be achieved, such as automatic control of roadblocks and adjustment of traffic lights, which can greatly improve the efficiency and practicality of campus safety management and control. Taking full advantage of the advantages of AI video technology, the key technologies of complete visual data processing, fine-grained target recognition, personalized behavior analysis, targeted safety assessment and intelligent early warning mechanism are adopted, which can effectively improve the overall performance of campus safety management and control. Therefore, the present invention realizes intelligent perception and early warning of campus monitoring data by integrating video analysis, target recognition, and behavior analysis technologies, effectively improving the intelligent level of campus security management and control.

[0033] In the embodiment of the present invention, reference Figure 1 The above is a schematic diagram of the steps of the campus safety management and control method based on AI video technology of the present invention. In this example, the campus safety management and control method based on AI video technology includes the following steps:

[0034] Step S1: acquiring campus monitoring data; performing visual load extraction on the campus monitoring data to obtain a frame image; performing feature enhancement on the frame image to generate an enhanced frame image;

[0035] In the embodiment of the present invention, high-definition digital surveillance cameras are used and deployed in multiple key locations on campus to collect video data streams covering the entire campus environment. The cameras used use a resolution of 1920×1080, a frame rate of 30 frames per second, and an H.264 video encoding format, with an output capacity of about 10Mbps. These surveillance cameras are connected via a wired or wireless network to transmit the collected video data to the campus central control server in real time. The collected raw video data stream is input into a specially designed visual load extraction module. The module first decodes the video and extracts the RGB color image of each frame. Then, through the target detection algorithm (such as Faster R-CNN, YOLO), the pedestrians, vehicles, and buildings in each frame image are located and segmented, and a frame image containing the target of interest is generated. The extracted frame image is subjected to feature enhancement processing to improve the performance of subsequent processing. Specifically, an image enhancement model based on a convolutional neural network, such as SRCNN and ESRGAN, is used to perform super-resolution, denoising, and color correction operations on the frame image to generate an enhanced frame image with richer texture details and more vivid colors. The enhanced image resolution reaches 2048×1536, providing high-quality visual input for subsequent target recognition and scene analysis.

[0036] Step S2: Perform dynamic target cutting on the enhanced frame image to generate a moving target image; perform target classification and identification on the moving target image to generate a vehicle target image and a pedestrian target image;

[0037] In an embodiment of the present invention, the enhanced frame image is input into a real-time target detection model based on a convolutional neural network (such as Mask R-CNN) for processing. The model can accurately locate all moving targets in the image and generate corresponding target bounding boxes and segmentation masks. By applying these masks to the original frame image, independent moving target images can be extracted. These moving targets include pedestrians, vehicles, and bicycle dynamic objects. For the extracted moving target images, a pre-trained deep learning classification model (such as ResNet, VGG) is applied for target recognition. The classification model has been trained with large-scale vehicle and pedestrian data sets and can accurately distinguish between vehicle targets and pedestrian targets in the image. Through the output labels of the model, the moving targets can be divided into two categories: vehicle target images and pedestrian target images. The classification model adopts an architecture of five convolutional layers and three fully connected layers. The input image size is 224×224 pixels. The model is pre-trained on the ImageNet data set, and the Top-1 accuracy rate reaches more than 93%. In actual tests in campus scenes, the classification accuracy of vehicles and pedestrians exceeded 90%. This classification model can be flexibly deployed on edge devices (such as cameras) to achieve real-time dynamic target recognition.

[0038] Step S3: Perform human body structure outlining on the pedestrian target image to obtain a human body outlining image; perform human motion situation analysis on the human body outlining image to obtain pedestrian motion state data;

[0039] In an embodiment of the present invention, the extracted pedestrian target image is input into a human posture estimation model based on OpenPose for processing. The model adopts a multi-layer stacked encoding-decoding convolutional neural network architecture, which can accurately detect and depict the positions of 18 key joints of the human body, including the head, shoulders, elbows, wrists, trunk, hips, knees, and ankles. By connecting these joints, a complete human body outline image is generated, showing the skeletal structure and motion posture of the pedestrian. The generated human body outline image is processed by a pedestrian motion analysis algorithm based on a time series. The algorithm first extracts the motion trajectory of each joint point in the time series and calculates the speed and acceleration dynamic characteristics. Then, through a machine learning model (such as a hidden Markov model and a long short-term memory network), these features are matched with predefined walking, running, emergency stop, and turning motion patterns, and the pedestrian motion state data is output. The data includes the current motion state of the pedestrian (stationary, walking, running), motion speed and acceleration parameters, reflecting the dynamic motion situation of the pedestrian.

[0040] Step S4: dynamically estimating the vehicle speed of the vehicle target image to obtain vehicle speed data; performing vehicle state deduction on the vehicle target image based on the vehicle speed data to generate vehicle motion state data;

[0041] In an embodiment of the present invention, the extracted vehicle target image is input into a vehicle speed estimation model based on the optical flow method for processing. The model first uses the Lucas-Kanade optical flow algorithm to calculate the motion vector field of the vehicle on the image plane from continuous vehicle target images. Then, according to the internal and external parameters of the camera, these two-dimensional motion vectors are converted into the real motion speed of the vehicle in three-dimensional space. Based on the obtained vehicle speed data, the prediction-control model is applied to deduce the future motion state of the vehicle target. The model adopts a three-degree-of-freedom vehicle motion model based on dynamics, inputs the current vehicle speed, acceleration, and steering angle parameters, and predicts the position, speed, and acceleration kinematics of the vehicle in the future. Through time series simulation, vehicle motion state data can be generated, including current vehicle speed, acceleration, driving direction, and predicted trajectory information, reflecting the real-time motion situation of the vehicle.

[0042] Step S5: Based on the preset pedestrian safety motion state indicators, the pedestrian motion state data is defined as dangerous, and pedestrian dangerous motion data is generated; based on the preset pedestrian safety motion state indicators, the pedestrian motion state data is defined as dangerous, and vehicle dangerous motion data is obtained; the pedestrian dangerous motion data and vehicle dangerous motion data are integrated into a unit warning to implement campus safety management and control.

[0043] In the embodiment of the present invention, for pedestrian motion state data, predefined pedestrian safety motion indicators are applied to evaluate the danger thereof. The main indicators include: "running" state with a walking speed exceeding 5m / s, "sudden stop" state with a head and trunk tilt angle exceeding 20 degrees, and "staying" state with a static state of more than 5 seconds in the middle of the road. These danger indicators are applied to pedestrian action sequence analysis to generate "pedestrian dangerous motion data" to identify pedestrian behaviors with potential safety hazards in campus scenes. For vehicle motion state data, predefined vehicle safety motion indicators are applied to evaluate the danger thereof. The main indicators include: "speeding" state with a speed exceeding 40km / h, "sudden acceleration" state with an acceleration exceeding 3m / s^2, and "sharp turn" state with a steering angular velocity exceeding 15° / s. These danger indicators are applied to vehicle motion situation analysis to generate vehicle dangerous motion data to identify vehicle behaviors with potential safety hazards in campus scenes. Pedestrian dangerous motion data and vehicle dangerous motion data are subjected to spatiotemporal correlation analysis to identify dangerous units with collision conflicts. For these dangerous units, the movement status of pedestrians and vehicles is comprehensively considered to trigger audio and video warning signals, and the warning information is transmitted to the campus security control center, where management personnel intervene and respond in real time.

[0044] Preferably, step S1 comprises the following steps:

[0045] Step S11: Collect campus monitoring data in real time;

[0046] Step S12: decoding the campus surveillance data to obtain a continuous video frame sequence;

[0047] Step S13: extracting video frames from a continuous video frame sequence to generate frame images;

[0048] Step S14: extracting features from the frame image to obtain image feature data;

[0049] Step S15: Perform feature sharpening and enhancement on the frame image according to the image feature data to generate an enhanced frame image.

[0050] The benefit of the present invention is that it can timely obtain the latest video information of the internal environment of the campus, provide basic data for subsequent intelligent analysis, convert the real-time collected video data into a processable continuous frame image sequence, and lay the foundation for downstream target detection and recognition. Extracting low-level visual features of color, texture, and shape from frame images provides an important basis for subsequent image enhancement and target analysis. Performing feature enhancement processing on the original frame image improves the contrast and clarity of the image, which helps to improve the performance of subsequent algorithms. Together, a complete video acquisition, preprocessing, and feature extraction process is constructed, laying a solid foundation for subsequent higher-level target detection, recognition, and behavior analysis. Through these preliminary processing links, the quality and reliability of the input data can be ensured, thereby improving the overall performance of the entire campus safety management and control system.

[0051] In an embodiment of the present invention, the surveillance camera equipment deployed on campus collects high-definition (1080P) video data streams at a collection frequency of 30fps. The collected video data is transmitted to the central server in real time via a wired network, and the server provides a dedicated video collection interface to realize real-time data collection of each monitoring point inside the campus. The collected data includes video encoding information, timestamp, and camera ID metadata, which provide basic data support for subsequent video processing. The collected raw video data is stored in H.264 encoding format, and video decoding processing is required first. Using the VideoCapture class in the OpenCV library, the video data is read frame by frame and decoded to obtain a continuous RGBA format video frame sequence. For a video with a resolution of 1080P, 30 frames of image data can be obtained per second. For the decoded continuous video frame sequence, key frames are extracted at a certain time interval according to application requirements. For example, 1 frame is extracted as a key frame every 1 second to generate static image data with a resolution of 1080P. For the extracted frame image data, an image feature extraction network based on deep learning is applied to automatically learn and extract texture, edge, and regional visual features in the image. The ResNet-50 model is used as the feature extraction backbone to output a 4096-dimensional image feature vector. Using the extracted image feature data, an image enhancement model based on a generative adversarial network (GAN) is applied to sharpen and enhance the original frame image. The GAN model consists of two parts: a generator and a discriminator. The generator is responsible for synthesizing the enhanced image based on the input features, while the discriminator evaluates the realism of the generated image.

[0052] Preferably, step S2 comprises the following steps:

[0053] Step S21: performing moving target detection on the enhanced frame image to obtain moving target data;

[0054] Step S22: performing moving target cutting on the enhanced frame image using the moving target data to generate a moving target image;

[0055] Step S23: performing rapid scene recognition on the moving target image to obtain image target recognition data;

[0056] Step S24: classify the moving target image according to the image target recognition data to generate a vehicle target image and a pedestrian target image.

[0057] The benefit of the present invention is that by analyzing the enhanced frame image, the mobile target inside the campus is detected, basic data is provided for subsequent target identification and classification, and the mobile target area is accurately segmented from the original image, eliminating the interference of irrelevant background information, which is conducive to improving the accuracy of the subsequent classification algorithm. Using deep learning advanced visual recognition technology, the cut mobile target image is quickly identified to accurately determine the category of the target, such as vehicles and pedestrians. According to the recognition results, the mobile target is further classified into two categories: vehicles and pedestrians, providing a more detailed behavior analysis and early warning function for the campus safety management and control system. Combining the key technologies of target detection, segmentation, recognition and classification, a complete computer vision processing flow is constructed. Through the processing of these links, the key target information inside the campus can be quickly and accurately extracted from the massive video surveillance data, laying a reliable foundation for subsequent behavior analysis and anomaly detection. The benefit of these processing steps is that the intelligence level and application value of the campus safety management and control system have been greatly improved.

[0058] As an example of the present invention, refer to Figure 2 As shown, in this example, step S2 includes:

[0059] Step S21: performing moving target detection on the enhanced frame image to obtain moving target data;

[0060] In the embodiment of the present invention, the RPN network is used to generate multiple candidate target region proposals for the input image. These proposals cover mobile targets of various scales and aspect ratios in the image. The proposals generated by the RPN are sent to the target classification and regression network. The network extracts high-level semantic features based on ResNet-101 and uses a fully connected layer to perform target category prediction and bounding box regression. By setting an appropriate confidence threshold, the bounding box coordinates, category labels, and confidence scores of the mobile targets in the image can be accurately output to form mobile target data.

[0061] Step S22: performing moving target cutting on the enhanced frame image using the moving target data to generate a moving target image;

[0062] In the embodiment of the present invention, the target bounding box coordinate information output by the Faster R-CNN model is converted into pixel coordinates in the image. Then, an OpenCV-based precise pixel-level segmentation algorithm is used to extract and cut out the target area along the outline of the bounding box, and the change of the target position in consecutive frames is used to more accurately determine the actual range of the moving target, avoiding cutting errors caused by occlusion or motion blur. At the same time, a multi-scale cutting strategy based on an image pyramid is adopted to adapt to moving targets of different sizes and distances.

[0063] Step S23: performing rapid scene recognition on the moving target image to obtain image target recognition data;

[0064] In an embodiment of the present invention, for a moving target image, an optical flow estimation model based on deep learning is used to extract dynamic information of the target. The model adopts a PWC-Net network structure, which can efficiently estimate the motion vector field of the target with low computational overhead to form target optical flow data. Specifically, the PWC-Net model uses image pyramid feature extraction and a matching strategy based on cost volume to accurately capture target motion information of different scales and positions, and output the motion speed and direction of each pixel. The optical flow calculation module can generate high-quality target motion data at a speed of 60FPS per frame for a moving target image with a resolution of 1080P under GPU acceleration. The target optical flow data is input into a dynamic texture feature learning model based on convolutional LSTM to extract the dynamic characteristics of the spatiotemporal texture of the target. Through the encoder-decoder structure, the model can learn rich dynamic texture representations from the motion vector field of the target, including the deformation, scaling, and rotational spatiotemporal variation characteristics of the target. The output target dynamic texture feature data can fully characterize the dynamic semantic information of the target, providing an important supplement for subsequent multimodal fusion. The dynamic texture analysis module can process 1080P resolution moving target image data at a speed of 50FPS per frame. For moving target images, a convolutional neural network model based on ResNet-101 is used to extract the geometric appearance features of the target. The model can learn high-level semantic representations from the edge, texture, and shape visual features of the target and generate the target geometric feature vector. In the training process of the feature extraction model, hard mining and triplet loss techniques are used to maximize the geometric feature differences of targets of different categories. The target dynamic texture feature data and target geometric feature vector are combined. The features are cascaded and fused through fully-connected layers to generate a unified multimodal semantic representation. The fusion strategy fully considers the two key aspects of the target's motion dynamics and appearance semantics, and can more comprehensively characterize the semantic features of the target. Based on multimodal semantics, an end-to-end deep learning model is used to perform semantic scene recognition on moving targets. The model consists of a feature extraction backbone based on ResNet-101 and a temporal feature encoder based on LSTM. Through supervised training on a large-scale semantic speed scene dataset, the model can effectively conduct a comprehensive analysis of the target's appearance, motion, and semantics, and output an accurate semantic speed scene recognition model. The trained semantic speed scene recognition model is used to perform end-to-end target recognition on moving target images, outputting the category label, confidence, and corresponding semantic speed scene information of each moving target to form image target recognition data.

[0065] Step S24: classify the moving target image according to the image target recognition data to generate a vehicle target image and a pedestrian target image.

[0066] In the embodiment of the present invention, the category label information in the image target recognition data is extracted and divided into two main categories: "vehicle" and "pedestrian". For the target identified as the "vehicle" class, the corresponding moving target image area is cropped to form a "vehicle target image" data set. Similarly, for the target identified as the "pedestrian" class, the corresponding moving target image area is extracted to generate a "pedestrian target image" data set.

[0067] Preferably, step S23 includes the following steps:

[0068] Step S231: performing optical flow calculation on the moving target image to obtain target optical flow data;

[0069] Step S232: performing dynamic texture analysis on the target optical flow data to obtain target dynamic texture feature data;

[0070] Step S233: extracting geometric features from the moving target image to obtain a target geometric feature vector;

[0071] Step S234: performing multimodal cascade fusion on the target dynamic texture feature data and the target geometric feature vector to generate multimodal semantics;

[0072] Step S235: performing end-to-end semantic quick scene recognition modeling on multimodal semantics to obtain a semantic quick scene recognition model;

[0073] Step S236: Perform image recognition on the moving target image using the semantic speed scene recognition model to generate image target recognition data.

[0074] The benefit of the present invention is that by analyzing the dynamic information of the mobile target image, such as the target's motion trajectory and texture change characteristics, more abundant and accurate target description features can be extracted, providing important input for subsequent target recognition. Combined with the shape, scale and geometric attribute characteristics of the target, the identifiable information of the target can be further enriched to make up for the shortcomings of dynamic feature analysis. The dynamic and geometric features of the target are organically combined to form a more comprehensive and semantic target description, which improves the accuracy and robustness of target recognition. By building an end-to-end deep learning recognition model based on multimodal features, the intrinsic connection between features can be better explored and the recognition performance can be further optimized. Using the trained semantic speed scene recognition model, the mobile target can be accurately classified and recognized, which can provide reliable input data for subsequent behavior analysis and early warning. The advantages of cutting-edge technologies in computer vision, pattern recognition and deep learning are fully utilized to build a highly optimized target recognition process. Through multimodal feature extraction and end-to-end modeling, the accuracy and generalization performance of target recognition are greatly improved, bringing significant technical advantages to the campus safety management and control system.

[0075] In an embodiment of the present invention, for a moving target image, an optical flow estimation model based on deep learning is used to extract dynamic information of the target. The model adopts a PWC-Net network structure, which can efficiently estimate the motion vector field of the target with low computational overhead to form target optical flow data. Specifically, the PWC-Net model uses image pyramid feature extraction and a matching strategy based on cost volume to accurately capture target motion information of different scales and positions, and output the motion speed and direction of each pixel. The optical flow calculation module can generate high-quality target motion data at a speed of 60FPS per frame for a moving target image with a resolution of 1080P under GPU acceleration. The target optical flow data is input into a dynamic texture feature learning model based on convolutional LSTM to extract the dynamic characteristics of the spatiotemporal texture of the target. Through the encoder-decoder structure, the model can learn rich dynamic texture representations from the motion vector field of the target, including the deformation, scaling, and rotational spatiotemporal variation characteristics of the target. The output target dynamic texture feature data can fully characterize the dynamic semantic information of the target, providing an important supplement for subsequent multimodal fusion. The dynamic texture analysis module can process 1080P resolution moving target image data at a speed of 50FPS per frame. For moving target images, a convolutional neural network model based on ResNet-101 is used to extract the geometric appearance features of the target. The model can learn high-level semantic representations from the edge, texture, and shape visual features of the target and generate the target geometric feature vector. In the training process of the feature extraction model, hard mining and triplet loss techniques are used to maximize the geometric feature differences of targets of different categories. The target dynamic texture feature data and target geometric feature vector are combined. The features are cascaded and fused through fully-connected layers to generate a unified multimodal semantic representation. The fusion strategy fully considers the two key aspects of the target's motion dynamics and appearance semantics, and can more comprehensively characterize the semantic features of the target. Based on multimodal semantics, an end-to-end deep learning model is used to perform semantic scene recognition on moving targets. The model consists of a feature extraction backbone based on ResNet-101 and a temporal feature encoder based on LSTM. Through supervised training on a large-scale semantic speed scene dataset, the model can effectively conduct a comprehensive analysis of the target's appearance, motion, and semantics, and output an accurate semantic speed scene recognition model. The trained semantic speed scene recognition model is used to perform end-to-end target recognition on moving target images, outputting the category label, confidence, and corresponding semantic speed scene information of each moving target to form image target recognition data.

[0076] Preferably, step S3 comprises the following steps:

[0077] Step S31: Estimating the human body posture of the pedestrian target image to obtain human body posture data;

[0078] Step S32: performing body outlining on the pedestrian target image based on the human posture data to obtain a human body outlining image;

[0079] Step S33: performing inter-frame motion tracking on the human body stroke image to obtain pedestrian motion data;

[0080] Step S34: Perform gait-specific analysis on the pedestrian motion data to obtain pedestrian motion state data.

[0081] The benefit of the present invention lies in that by performing human posture analysis on pedestrian target images, the key joint point information of the target can be accurately extracted, laying the foundation for subsequent human body outlining and motion analysis. Using human posture data, the human body contour of the pedestrian target is accurately depicted, highlighting the structural characteristics of the human body, which is conducive to improving the accuracy of subsequent motion analysis. Based on the human body outlining image, the inter-frame motion tracking technology is used to obtain the motion trajectory information of the pedestrian in the time series, providing an important basis for analyzing the motion state of the pedestrian. By performing gait-specific analysis on the pedestrian motion data, the walking mode and speed dynamic characteristics of the pedestrian can be extracted, and the motion state information of the pedestrian can be further enriched and refined. The key technologies of human posture estimation and motion tracking in computer vision are fully utilized to systematically extract the dynamic behavior characteristics of the pedestrian from the static image information. These feature information provides accurate input data for subsequent behavior analysis and anomaly detection, greatly improving the perception and understanding ability of the campus safety management and control system for pedestrian behavior. It can analyze the activity status of pedestrians inside the campus more comprehensively and meticulously, and provide strong technical support for timely discovery and prevention of campus safety accidents. The benefit of these steps is that they fully tap into the behavioral information in video surveillance data, bringing significant technical advantages to campus security management.

[0082] As an example of the present invention, refer to Figure 3 As shown, in this example, step S3 includes:

[0083] Step S31: Estimating the human body posture of the pedestrian target image to obtain human body posture data;

[0084] In an embodiment of the present invention, the OpenPose algorithm based on deep learning is used to detect the key points of the human body on the input image, and the two-dimensional coordinate information of 17 key points of the human body (top of the head, eyes, ears, shoulders, elbows, wrists, hips, knees, ankles) is obtained. The detected coordinate data of the key points of the human body are post-processed to calculate the angle information of each joint of the human body. The angles of the shoulder joint, elbow joint, wrist joint, hip joint, knee joint and ankle joint are calculated according to the coordinates of adjacent key points using trigonometric formulas to form a human posture feature vector containing 13 joint angles. The obtained human posture feature vector is stored in the database as an input feature for subsequent behavior analysis and anomaly detection functions.

[0085] Step S32: performing body outlining on the pedestrian target image based on the human posture data to obtain a human body outlining image;

[0086] In an embodiment of the present invention, a human body contour extraction algorithm based on OpenCV is used to draw the precise contour lines of the human body according to the human body key point data obtained in the previous step. The algorithm first converts the image into the HSV color space, and uses the hue (H) and saturation (S) channel information to extract the binary mask of the human body area. Then, the human body contour edge is detected in the mask area by the Canny edge detection algorithm, and finally the edge is simplified and smoothed by the Ramer-Douglas-Peucker algorithm to generate a fine stroke effect of the human body, and Gaussian blur filtering and edge enhancement technology are used on the stroke image. For example, Gaussian blur removes noise and subtle textures to make the human body contour lines clearer and smoother; edge enhancement highlights the human body edge, making the contour lines more uniform in thickness, and the overall effect is more beautiful and eye-catching. The processed human body stroke image is fused with the original pedestrian target image to generate a final stroke image containing human body contour information.

[0087] Step S33: performing inter-frame motion tracking on the human body stroke image to obtain pedestrian motion data;

[0088] In the embodiment of the present invention, an individualized gait modeling method based on HMM (hidden Markov model) is adopted, and a unique gait model of each pedestrian is trained using pedestrian motion data (including position, direction, and step frequency), and a gait feature extraction module based on deep learning is introduced. A pre-trained 3D convolutional neural network is used to input the motion trajectory data of each pedestrian, and high-dimensional features containing gait information can be automatically learned. Using the individualized gait model, the motion trajectory data of each pedestrian is analyzed frame by frame. The observation probability at each moment is calculated, and the average observation probability of the entire trajectory is accumulated as a stability measure of the pedestrian's gait. The higher the observation probability, the more the pedestrian's movement conforms to his personal characteristics and the more stable the gait. Each pedestrian trajectory is divided into several segments, and the gait stability score of each segment is calculated respectively. The OpenPose human posture estimation model is used to extract the coordinates of key points from the pedestrian image data of each time frame. These key point data are used to calculate posture features such as trunk angle and limb angle, and their time series are analyzed. By calculating the change in posture features between adjacent frames, we can obtain the time series data of pedestrian posture changes. The pedestrian gait stability data and posture change data are comprehensively analyzed through machine learning algorithms. The multi-classification model based on random forest can effectively identify the movement state of pedestrians, such as normal walking, panic escape, and staggering and falling. The pedestrian movement state data finally output not only includes the type label of the movement state, but also contains rich semantic information such as abnormal probability and change trend.

[0089] Step S34: Perform gait-specific analysis on the pedestrian motion data to obtain pedestrian motion state data.

[0090] In an embodiment of the present invention, a gait recognition model based on deep learning is used to analyze pedestrian motion data. A pre-trained 3D convolutional neural network model is used, and the input includes time series features of position coordinates and motion direction, so that the gait type of pedestrians, such as normal walking, fast walking, slow walking, sudden stop, and running, can be accurately identified. By analyzing the key point data of the human body in each time frame, a variety of gait parameters such as stride, step frequency, and body sway angle can be obtained. The gait recognition results and gait parameter data are integrated to form comprehensive motion state data including gait type, stride, step frequency, and body sway angle information.

[0091] Preferably, step S34 includes the following steps:

[0092] Step S341: performing individualized gait model modeling on pedestrian motion data to obtain a pedestrian gait model;

[0093] Step S342: performing gait stability analysis on pedestrian motion data according to the pedestrian gait model to generate pedestrian gait stability data;

[0094] Step S343: performing posture change analysis on the pedestrian motion data to generate pedestrian posture change data;

[0095] Step S344: Perform motion status analysis on the pedestrian's gait stability data and the pedestrian's posture change data to obtain pedestrian motion status data.

[0096] The benefit of the present invention lies in that the gait characteristics of pedestrians are deeply analyzed and modeled, and a personalized gait model for each pedestrian can be established, providing a more accurate reference basis for subsequent gait stability analysis. By comparing and analyzing the pedestrian motion data with its corresponding individualized gait model, the gait stability of pedestrians can be accurately evaluated, abnormal gait characteristics can be identified, and important clues can be provided for behavior analysis and early warning. A detailed posture change analysis of pedestrian motion data can extract the subtle change characteristics of pedestrian movements, further enrich the description of pedestrian motion status, and lay the foundation for subsequent motion situation analysis. A comprehensive analysis of the gait stability and posture change characteristics of pedestrians can more comprehensively and accurately judge the current motion state of pedestrians, such as normal walking, running, and falling, providing a more detailed behavior recognition basis for the campus safety management and control system, reflecting the in-depth analysis and modeling of pedestrian motion status, and giving full play to the advantages of computer vision, pattern recognition and artificial intelligence technology in behavior analysis. Through the establishment of personalized gait models, the evaluation of gait stability, and the refined analysis of posture changes, the campus safety management system can more accurately understand the movement characteristics of pedestrians, detect abnormal behaviors in a timely manner, and improve the accuracy and effectiveness of campus safety management. Through multi-dimensional motion analysis, the campus safety management system can more comprehensively perceive and understand the behavior of pedestrians, provide more reliable intelligent support for campus safety management, and thus greatly improve the intelligent level of campus safety management.

[0097] In the embodiment of the present invention, an individualized gait modeling method based on HMM (hidden Markov model) is adopted, and a unique gait model of each pedestrian is trained using pedestrian motion data (including position, direction, and step frequency), and a gait feature extraction module based on deep learning is introduced. A pre-trained 3D convolutional neural network is used to input the motion trajectory data of each pedestrian, and high-dimensional features containing gait information can be automatically learned. Using the individualized gait model, the motion trajectory data of each pedestrian is analyzed frame by frame. The observation probability at each moment is calculated, and the average observation probability of the entire trajectory is accumulated as a stability measure of the pedestrian's gait. The higher the observation probability, the more the pedestrian's movement conforms to his personal characteristics and the more stable the gait. Each pedestrian trajectory is divided into several segments, and the gait stability score of each segment is calculated respectively. The OpenPose human posture estimation model is used to extract the coordinates of key points from the pedestrian image data of each time frame. These key point data are used to calculate posture features such as trunk angle and limb angle, and their time series are analyzed. By calculating the change in posture features between adjacent frames, we can obtain the time series data of pedestrian posture changes. The pedestrian gait stability data and posture change data are comprehensively analyzed through machine learning algorithms. The multi-classification model based on random forest can effectively identify the movement state of pedestrians, such as normal walking, panic escape, and staggering and falling. The pedestrian movement state data finally output not only includes the type label of the movement state, but also contains rich semantic information such as abnormal probability and change trend.

[0098] Preferably, step S4 comprises the following steps:

[0099] Step S41: comparing the vehicle positions between frames of the vehicle target image to obtain the vehicle position change data between frames;

[0100] Step S42: Estimating the vehicle speed based on the inter-frame vehicle position change data to obtain vehicle speed data;

[0101] Step S43: reconstructing the vehicle dynamic trace based on the vehicle speed data and the vehicle position change data between frames to obtain the vehicle motion mode data;

[0102] Step S44: performing motion state fusion on the vehicle speed data and the vehicle motion mode data to generate vehicle motion state data.

[0103] The benefit of the present invention lies in that by comparing and analyzing the positions of vehicle targets in continuous video frames, the position change data of the vehicle in the time series can be obtained, providing basic data for subsequent vehicle speed estimation and motion trajectory reconstruction. Based on the inter-frame vehicle position change data, combined with time information, the real-time speed of the vehicle can be calculated, providing key vehicle dynamic characteristics for the campus safety management and control system. Combining the vehicle speed data with the inter-frame position change data, the complete motion trajectory and dynamic mode of the vehicle in time and space can be reconstructed, such as straight driving, sharp turns, and sudden braking, laying the foundation for behavior analysis and anomaly detection. By conducting in-depth analysis and fusion of vehicle speed and motion mode data, more comprehensive and detailed vehicle current motion state information can be obtained, such as normal driving, speeding, and illegal turns, providing accurate decision-making basis for the campus safety management and control system, making full use of computer vision and intelligent analysis technology, extracting the dynamic behavior characteristics of the vehicle from the video surveillance data, and establishing a comprehensive vehicle motion state perception capability for the campus safety management and control system. Through in-depth analysis and integration of vehicle position changes, speed, and trajectory data, the vehicle's movement pattern and current status can be accurately identified, providing strong support for timely detection and prevention of vehicle-related safety accidents on campus. Through comprehensive perception and analysis of vehicle dynamic characteristics, the campus safety management and control system can better identify and predict potential safety hazards of vehicles, provide more intelligent and precise technical support for campus traffic management and security, and greatly improve the overall level of campus safety management.

[0104] As an example of the present invention, refer to Figure 4 As shown, in this example, step S4 includes:

[0105] Step S41: comparing the vehicle positions between frames of the vehicle target image to obtain the vehicle position change data between frames;

[0106] In an embodiment of the present invention, a vehicle detection algorithm based on deep learning, such as Mask R-CNN, is used to perform vehicle target detection on each frame of video image. The detected vehicle position coordinates (such as center point coordinates) will be used as input features. The vehicle position coordinates of adjacent frames are compared and calculated to obtain the displacement change of the vehicle between frames. The displacement increments of the vehicle on the x-axis and y-axis are calculated as a measure of the vehicle position change. The vehicle's movement direction angle is calculated, that is, the angle between the vehicle's movement direction and the horizontal line. By analyzing the direction angle changes between adjacent frames, the vehicle's steering information can be obtained. The calculated vehicle position change and direction angle changes are integrated into the vehicle position change data between frames.

[0107] Step S42: Estimating the vehicle speed based on the inter-frame vehicle position change data to obtain vehicle speed data;

[0108] In the embodiment of the present invention, based on the inter-frame vehicle position change data, the instantaneous speed of the vehicle is calculated through the existing displacement-time relationship, and a smoothing method based on Kalman filtering is adopted. The displacement data of several consecutive frames are used as the input of the filter to remove the measurement noise and obtain a smoother speed curve. At the same time, the analytical relationship between the vehicle speed and position change is established by using Newton's motion equations and vehicle dynamics characteristics. The vehicle speed data finally output includes not only the instantaneous speed value, but also the speed curve after smoothing, and the estimation result based on the kinematic model.

[0109] Step S43: reconstructing the vehicle dynamic trace based on the vehicle speed data and the vehicle position change data between frames to obtain the vehicle motion mode data;

[0110] In an embodiment of the present invention, the inter-frame vehicle position change data is used to reconstruct the continuous trajectory curve of the vehicle in time and space by interpolation calculation. A smooth and smooth trajectory curve can be obtained by using a cubic spline interpolation algorithm, which effectively suppresses the influence of measurement noise. Based on the reconstructed trajectory curve, it is discretely sampled to generate a series of vehicle position coordinate data. These position coordinate data can be directly mapped to a plane coordinate system to draw a motion trajectory image of the vehicle. The vehicle speed data and the vehicle motion trajectory image are used as input features, and a driving behavior model is constructed by a supervised learning method based on deep learning. A large amount of vehicle speed data and vehicle motion trajectory images are used to train a deep learning model that can accurately classify different driving behaviors through a back propagation algorithm. The driving behavior model finally obtained can automatically classify new vehicle speed data and trajectory images, and output the corresponding driving behavior category. The driving behavior model is input into a virtual driving environment based on physical simulation. The environment can simulate the kinematic characteristics of the vehicle, and generate the corresponding vehicle trajectory and speed curve according to the input driving behavior model. Through multiple simulation runs, a large amount of reproducing driving behavior data is obtained, including the vehicle's position trajectory, speed curve, and acceleration change. The reproduced driving behavior data, including position trajectory, speed curve and acceleration change, are input into the rule-based motion pattern recognition algorithm. The algorithm divides the vehicle's motion state into different types of acceleration, uniform speed, deceleration and steering according to the pre-defined dynamic characteristic threshold. The vehicle's motion state is divided into three basic modes: acceleration, uniform speed and deceleration according to the three different types. When the vehicle speed increases and the acceleration is greater than the positive threshold, it is determined to be in acceleration mode; when the vehicle speed remains basically unchanged and the acceleration is close to zero, it is determined to be in uniform speed mode; when the vehicle speed decreases and the acceleration is less than the negative threshold, it is determined to be in deceleration mode. By analyzing the curvature change of the vehicle trajectory, when the trajectory curvature exceeds a certain threshold, it can be determined that the vehicle is in a steering state.

[0111] Step S44: performing motion state fusion on the vehicle speed data and the vehicle motion mode data to generate vehicle motion state data.

[0112] In an embodiment of the present invention, the vehicle speed data and the vehicle motion mode data are time-aligned and spatially registered. Ensure that the two types of data are consistent in time and space. A rule-based data fusion method is used to fuse the vehicle speed data and the motion mode data. For example, when the vehicle speed is 0-20km / h and is in acceleration mode, it is determined to be a low-speed acceleration state; when the vehicle speed is 20-60km / h and is in uniform speed mode, it is determined to be a medium-speed uniform speed state; when the vehicle speed is above 60km / h and is in deceleration mode, it is determined to be a high-speed deceleration state. The vehicle speed data and motion mode data are mapped to predefined vehicle motion state categories, such as low-speed acceleration, medium-speed uniform speed, and high-speed deceleration. The vehicle motion state data finally output includes not only the motion state category at the current moment, but also richer timing information such as state duration and state transition time point.

[0113] Preferably, step S43 includes the following steps:

[0114] Step S431: reconstructing the vehicle trajectory based on the vehicle position change data between frames to obtain a vehicle motion trajectory image;

[0115] Step S432: constructing a driving behavior model based on the vehicle speed data and the vehicle motion trajectory image to obtain a driving behavior model;

[0116] Step S433: reproducing the driving behavior of the driving behavior model to generate reproduced driving behavior data;

[0117] Step S434: Perform motion pattern evaluation on the reproduced driving behavior data to obtain vehicle motion pattern data.

[0118] The benefit of the present invention is that by analyzing and processing the inter-frame vehicle position change data, the complete motion trajectory of the vehicle in time and space can be reconstructed, and the vehicle's motion state and path can be intuitively presented in the form of an image. This provides basic data support for subsequent driving behavior analysis and evaluation. Combined with the vehicle motion trajectory image and real-time vehicle speed data, a personalized driving behavior model for each vehicle can be established. This model can more accurately describe and predict the specific behavioral characteristics of the vehicle driver, and provide a more refined decision-making basis for the campus safety management and control system. By simulating and reproducing the constructed driving behavior model, simulated driving behavior data that is highly consistent with the actual observation data can be generated. These data can not only be used to verify the accuracy of the model, but also provide richer test samples for abnormal behavior detection and scene prediction. Comparing and analyzing the reproduced driving behavior data with the actual observation data can more accurately evaluate the vehicle's motion mode, such as normal driving, sudden braking, and dangerous turns. This fine-grained motion pattern recognition result can provide more accurate decision support for the campus safety management and control system. Through in-depth modeling and analysis of vehicle driving behavior, the campus safety management system can better identify and predict abnormal vehicle behavior, provide more intelligent and precise technical support for campus traffic safety management, and greatly improve the overall level of campus safety management.

[0119] In an embodiment of the present invention, the inter-frame vehicle position change data is used to reconstruct the continuous trajectory curve of the vehicle in time and space by interpolation calculation. A smooth and smooth trajectory curve can be obtained by using a cubic spline interpolation algorithm, which effectively suppresses the influence of measurement noise. Based on the reconstructed trajectory curve, it is discretely sampled to generate a series of vehicle position coordinate data. These position coordinate data can be directly mapped to a plane coordinate system to draw a motion trajectory image of the vehicle. The vehicle speed data and the vehicle motion trajectory image are used as input features, and a driving behavior model is constructed by a supervised learning method based on deep learning. A large amount of vehicle speed data and vehicle motion trajectory images are used to train a deep learning model that can accurately classify different driving behaviors through a back propagation algorithm. The driving behavior model finally obtained can automatically classify new vehicle speed data and trajectory images, and output the corresponding driving behavior category. The driving behavior model is input into a virtual driving environment based on physical simulation. The environment can simulate the kinematic characteristics of the vehicle, and generate the corresponding vehicle trajectory and speed curve according to the input driving behavior model. Through multiple simulation runs, a large amount of reproducing driving behavior data is obtained, including the vehicle's position trajectory, speed curve, and acceleration change. The reproduced driving behavior data, including position trajectory, speed curve and acceleration change, are input into the rule-based motion pattern recognition algorithm. The algorithm divides the vehicle's motion state into different types of acceleration, uniform speed, deceleration and steering according to the pre-defined dynamic characteristic threshold. The vehicle's motion state is divided into three basic modes: acceleration, uniform speed and deceleration according to the three different types. When the vehicle speed increases and the acceleration is greater than the positive threshold, it is determined to be in acceleration mode; when the vehicle speed remains basically unchanged and the acceleration is close to zero, it is determined to be in uniform speed mode; when the vehicle speed decreases and the acceleration is less than the negative threshold, it is determined to be in deceleration mode. By analyzing the curvature change of the vehicle trajectory, when the trajectory curvature exceeds a certain threshold, it can be determined that the vehicle is in a steering state.

[0120] Preferably, step S5 comprises the following steps:

[0121] Step S51: defining the safety of pedestrian motion state data based on a preset pedestrian safety motion state index, and defining the pedestrian motion state data as pedestrian dangerous motion data when the pedestrian motion state data is lower than the preset pedestrian safety motion state index;

[0122] Step S52: defining the safety of the vehicle motion state data based on a preset vehicle safety motion state index, and defining the vehicle motion state data as dangerous vehicle motion data when the vehicle motion state data is lower than the preset vehicle safety motion state index;

[0123] Step S53: performing unit warning fusion on pedestrian dangerous movement data and vehicle dangerous movement data to generate a unit warning event;

[0124] Step S54: System alarm is issued according to the unit warning event to implement campus safety management and control.

[0125] The benefit of the present invention is that by setting predefined pedestrian and vehicle safety motion state indicators, potential dangerous motion states can be identified in time, providing a basis for subsequent safety management and control, and the real-time safety status of pedestrians and vehicles on campus can be captured more accurately. Compared with the traditional rule-based simple threshold judgment, it can better adapt to the complex and changeable campus environment. By integrating pedestrian dangerous motion data and vehicle dangerous motion data, the safety hazards occurring on campus can be more comprehensively perceived, and targeted unit warning events can be generated, which can avoid isolated warnings for only a single element, thereby improving the accuracy and effectiveness of the warning. According to the generated unit warning events, an alarm can be issued to the campus safety management and control system in time, and corresponding emergency measures such as emergency evacuation and emergency parking can be taken, which can greatly improve the safety inside the campus and reduce the probability of safety accidents. Computer vision and pattern recognition AI technology are used to intelligently analyze and warn the real-time motion status of pedestrians and vehicles on campus, providing more advanced and intelligent technical support for the campus safety management and control system. Through the precise definition of the movement status of pedestrians and vehicles, the integration of unit warning events, and the integrated processing of system alarms and controls, the real-time security monitoring and prevention capabilities within the campus can be greatly enhanced, creating a safer and more reliable campus environment for teachers and students. By analyzing the real-time movement status of pedestrians and vehicles through multi-dimensional fusion and comparing it with preset safety indicators in real time, potential safety hazards can be identified more intelligently and accurately.

[0126] In an embodiment of the present invention, the movement trajectory and speed data of pedestrians on campus are collected to establish a reference index for the safe movement state of pedestrians. When it is detected that the pedestrian's stride does not exceed 0.6 meters, the body inclination angle exceeds 15 degrees, and the moving speed is less than 1.0 meters per second, its data is marked as pedestrian dangerous movement data. The speed, acceleration and steering angle change dynamic characteristic data of vehicles on campus are collected to establish a reference index for the safe movement state of vehicles. When the vehicle speed is higher than 20 kilometers per hour and the steering angle speed exceeds 20 degrees per second, its data is marked as vehicle dangerous movement data. Pedestrian dangerous movement data and vehicle dangerous movement data are compared in real time. Once a dangerous movement state of any pedestrian or vehicle is detected in a certain space-time area, the coordinate position, movement trajectory, and speed information of the area are immediately integrated into a unit warning event. Whether a pedestrian or a vehicle triggers any warning condition, a warning event will be immediately generated for alarm. Once the system generates a unit warning event, an alarm message is immediately issued through the campus broadcasting system, LED display screen and mobile phone APP at the same time, and the warning data is pushed to the campus security center at the same time. Security personnel can quickly dispatch patrol vehicles and security personnel to conduct on-site control and guidance based on the specific location and conditions of the warning event.

[0127] In this specification, a campus safety management and control system based on AI video technology is provided, which is used to execute a campus safety management and control method based on AI video technology as described above. The campus safety management and control system based on AI video technology includes:

[0128] The image acquisition module is used to acquire campus monitoring data; perform visual load extraction on the campus monitoring data to obtain a frame image; perform feature enhancement on the frame image to generate an enhanced frame image;

[0129] The image recognition module is used to perform dynamic target cutting on the enhanced frame image to generate a moving target image; perform target classification and identification on the moving target image to generate a vehicle target image and a pedestrian target image;

[0130] The pedestrian analysis module is used to perform human body structure outlining on the pedestrian target image to obtain a human body outlining image; perform human motion situation analysis on the human body outlining image to obtain pedestrian motion state data;

[0131] The vehicle analysis module is used to dynamically estimate the vehicle speed of the vehicle target image to obtain the vehicle speed data; based on the vehicle speed data, the vehicle state is deduced for the vehicle target image to generate the vehicle motion state data;

[0132] The safety management and control module is used to define the dangerousness of pedestrian movement status data based on preset pedestrian safety movement status indicators to generate pedestrian dangerous movement data; define the dangerousness of pedestrian movement status data based on preset pedestrian safety movement status indicators to obtain vehicle dangerous movement data; and perform unit warning fusion of pedestrian dangerous movement data and vehicle dangerous movement data to implement campus safety management and control.

[0133] The benefit of the present invention lies in the real-time collection and intelligent analysis of campus monitoring data, which improves the immediacy and intelligence of security monitoring. Through multimodal feature extraction and fusion, accurate detection and identification of targets (pedestrians, vehicles) are achieved, and the accuracy of security monitoring is enhanced. The motion state of pedestrians and vehicles is carefully analyzed, including gait, speed, and trajectory, which can better evaluate and predict potential safety hazards, make intelligent judgments on monitoring data based on preset safety indicators, and automatically generate unit warning events, thereby improving the automation level of campus safety management and control. Real-time monitoring data, target recognition, motion state analysis, and safety warning links are organically combined to form a complete set of campus safety management and control closed loops, which improves the overall safety management efficiency. The campus safety management and control method based on AI video technology makes full use of the advantages of computer vision and pattern recognition, and has obvious advantages in improving the immediacy, accuracy, and automation level of campus safety monitoring. Therefore, the present invention realizes intelligent perception and early warning of campus monitoring data by integrating video analysis, target recognition, and behavior analysis technology, effectively improving the intelligent level of campus safety management and control.

[0134] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points of view, and the scope of the present invention is limited by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the same elements of the application documents are included in the present invention.

[0135] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A campus safety management and control method based on AI video technology, characterized in that: The following steps are involved: Step S1: Acquire campus monitoring data; perform visual load extraction on the campus monitoring data to obtain a frame image; Perform feature enhancement on the frame image to generate an enhanced frame image; Step S2: Perform dynamic target cutting on the enhanced frame image to generate a moving target image; Perform target classification and identification on the moving target image to generate vehicle target image and pedestrian target image; Step S3: tracing the human body structure of the pedestrian target image to obtain a human body tracing image; Perform human motion situation analysis on human outline images to obtain pedestrian motion status data; Step S4: dynamically estimating the vehicle speed of the vehicle target image to obtain vehicle speed data; Perform vehicle state deduction on the vehicle target image based on the vehicle speed data to generate vehicle motion state data; Step S5: defining the danger of pedestrian motion state data based on preset pedestrian safety motion state indicators to generate pedestrian dangerous motion data; Based on the preset pedestrian safety movement state index, the pedestrian movement state data is defined as dangerous, and the vehicle dangerous movement data is obtained; The pedestrian dangerous movement data and vehicle dangerous movement data are integrated into unit warnings to implement campus safety management and control.

2. The campus safety management and control method based on AI video technology according to claim 1 is characterized in that: Step S1 includes the following steps: Step S11: Collect campus monitoring data in real time; Step S12: decoding the campus surveillance data to obtain a continuous video frame sequence; Step S13: extracting video frames from a continuous video frame sequence to generate frame images; Step S14: extracting features from the frame image to obtain image feature data; Step S15: Perform feature sharpening and enhancement on the frame image according to the image feature data to generate an enhanced frame image.

3. The campus safety management and control method based on AI video technology according to claim 1 is characterized in that: Step S2 includes the following steps: Step S21: performing moving target detection on the enhanced frame image to obtain moving target data; Step S22: performing moving target cutting on the enhanced frame image using the moving target data to generate a moving target image; Step S23: performing rapid scene recognition on the moving target image to obtain image target recognition data; Step S24: classify the moving target image according to the image target recognition data to generate a vehicle target image and a pedestrian target image.

4. The campus safety management and control method based on AI video technology according to claim 3 is characterized in that: Step S23 includes the following steps: Step S231: performing optical flow calculation on the moving target image to obtain target optical flow data; Step S232: performing dynamic texture analysis on the target optical flow data to obtain target dynamic texture feature data; Step S233: extracting geometric features from the moving target image to obtain a target geometric feature vector; Step S234: performing multimodal cascade fusion on the target dynamic texture feature data and the target geometric feature vector to generate multimodal semantics; Step S235: performing end-to-end semantic quick scene recognition modeling on multimodal semantics to obtain a semantic quick scene recognition model; Step S236: Perform image recognition on the moving target image using the semantic speed scene recognition model to generate image target recognition data.

5. The campus safety management and control method based on AI video technology according to claim 1 is characterized in that: Step S3 includes the following steps: Step S31: Estimating the human body posture of the pedestrian target image to obtain human body posture data; Step S32: performing body outlining on the pedestrian target image based on the human posture data to obtain a human body outlining image; Step S33: performing inter-frame motion tracking on the human body stroke image to obtain pedestrian motion data; Step S34: Perform gait-specific analysis on the pedestrian motion data to obtain pedestrian motion state data.

6. The campus safety management and control method based on AI video technology according to claim 5 is characterized in that: Step S34 includes the following steps: Step S341: performing individualized gait model modeling on pedestrian motion data to obtain a pedestrian gait model; Step S342: performing gait stability analysis on pedestrian motion data according to the pedestrian gait model to generate pedestrian gait stability data; Step S343: performing posture change analysis on the pedestrian motion data to generate pedestrian posture change data; Step S344: Perform motion status analysis on the pedestrian's gait stability data and the pedestrian's posture change data to obtain pedestrian motion status data.

7. The campus safety management and control method based on AI video technology according to claim 1 is characterized in that: Step S4 includes the following steps: Step S41: comparing the vehicle positions between frames of the vehicle target image to obtain the vehicle position change data between frames; Step S42: Estimating the vehicle speed based on the inter-frame vehicle position change data to obtain vehicle speed data; Step S43: reconstructing the vehicle dynamic trace based on the vehicle speed data and the vehicle position change data between frames to obtain the vehicle motion mode data; Step S44: performing motion state fusion on the vehicle speed data and the vehicle motion mode data to generate vehicle motion state data.

8. The campus safety management and control method based on AI video technology according to claim 7 is characterized in that: Step S43 includes the following steps: Step S431: reconstructing the vehicle trajectory based on the vehicle position change data between frames to obtain a vehicle motion trajectory image; Step S432: constructing a driving behavior model based on the vehicle speed data and the vehicle motion trajectory image to obtain a driving behavior model; Step S433: reproducing the driving behavior of the driving behavior model to generate reproduced driving behavior data; Step S434: Perform motion pattern evaluation on the reproduced driving behavior data to obtain vehicle motion pattern data.

9. The campus safety management and control method based on AI video technology according to claim 1 is characterized in that: Step S5 includes the following steps: Step S51: defining the safety of pedestrian motion state data based on a preset pedestrian safety motion state index, and defining the pedestrian motion state data as pedestrian dangerous motion data when the pedestrian motion state data is lower than the preset pedestrian safety motion state index; Step S52: defining the safety of the vehicle motion state data based on a preset vehicle safety motion state index, and defining the vehicle motion state data as dangerous vehicle motion data when the vehicle motion state data is lower than the preset vehicle safety motion state index; Step S53: performing unit warning fusion on pedestrian dangerous movement data and vehicle dangerous movement data to generate a unit warning event; Step S54: System alarm is issued according to the unit warning event to implement campus safety management and control.

10. A campus safety management and control system based on AI video technology, characterized in that: Used to execute the campus safety management method based on AI video technology as claimed in claim 1, the campus safety management system based on AI video technology includes: The image acquisition module is used to acquire campus monitoring data; perform visual load extraction on the campus monitoring data to obtain a frame image; perform feature enhancement on the frame image to generate an enhanced frame image; The image recognition module is used to perform dynamic target cutting on the enhanced frame image to generate a moving target image; perform target classification and identification on the moving target image to generate a vehicle target image and a pedestrian target image; The pedestrian analysis module is used to perform human body structure outlining on the pedestrian target image to obtain a human body outlining image; perform human motion situation analysis on the human body outlining image to obtain pedestrian motion state data; The vehicle analysis module is used to dynamically estimate the vehicle speed of the vehicle target image to obtain the vehicle speed data; based on the vehicle speed data, the vehicle state is deduced for the vehicle target image to generate the vehicle motion state data; The safety management and control module is used to define the dangerousness of pedestrian movement status data based on preset pedestrian safety movement status indicators to generate pedestrian dangerous movement data; define the dangerousness of pedestrian movement status data based on preset pedestrian safety movement status indicators to obtain vehicle dangerous movement data; and perform unit warning fusion of pedestrian dangerous movement data and vehicle dangerous movement data to implement campus safety management and control.

Citation Information

Cited By

  • Motion state evaluation method and device, electronic equipment and storage medium

    CN119417864A

  • Smart parking lot safety management system and method

    CN120199101A

  • High-risk operator state intelligent analysis system based on AI large model

    CN121438182A

  • Action-oriented artificial intelligence industrial safety monitoring system and methods

    TWI912039B