Video monitoring data processing method, device, equipment, storage medium and product

Abstract

This application provides a video surveillance data processing method, apparatus, device, storage medium, and product, comprising: receiving multi-source data of a target area sent by a data acquisition terminal via a 5G network, wherein the multi-source data includes video data, image data, and sensor data; preprocessing the video data and image data to obtain video image data; inputting the video image data into a pre-trained AI visual analysis model to identify objects in the target area and detect the motion trajectory and behavior patterns of the objects to obtain security early warning response data; performing fusion decision based on sensor data and video image data to obtain sensor data fusion decision results; and generating an alarm and triggering emergency response measures in the target area based on the security early warning response data and sensor data fusion decision results. This method aims to improve the efficiency of video data processing.

Description

Technical Field

[0001] This application relates to the field of video surveillance technology, and in particular to a video surveillance data processing method, apparatus, equipment, storage medium, and product. Background Technology

[0002] Today, with the increasing demand for security and protection, video surveillance is widely used in many places such as commercial areas, and is the key to ensuring public and business safety and maintaining order in these places.

[0003] Traditional video surveillance platforms rely on long-term human monitoring and visual identification of anomalies. However, when faced with massive amounts of video data, manual operation is slow to respond and prone to false alarms, while the efficiency of processing video data is also low. Summary of the Invention

[0004] This application provides video surveillance data processing methods, apparatus, devices, storage media, and products to improve the efficiency of video data processing.

[0005] In a first aspect, embodiments of this application provide a video surveillance data processing method applied to a computer device, comprising: receiving multi-source data of a target area sent by a data acquisition terminal via a 5G network, wherein the multi-source data includes video data, image data, and sensor data; preprocessing the video data and image data to obtain video image data; inputting the video image data into a pre-trained AI visual analysis model to identify objects in the target area and detect the motion trajectory and behavior patterns of the objects to obtain security warning response data; performing fusion decision based on sensor data and video image data to obtain sensor data fusion decision results; and generating an alarm based on the security warning response data and sensor data fusion decision results, and triggering emergency response measures in the target area.

[0006] In one possible implementation, the method further includes: during transmission via a 5G network, real-time monitoring and rapid reception of encrypted multi-source data from the target area, wherein the 5G network has a built-in channel encryption mechanism; using a preset verification algorithm to perform integrity verification on the encrypted multi-source data to determine whether the encrypted multi-source data is abnormal; if the encrypted multi-source data is not abnormal, obtaining a private key from the encrypted storage unit and using the private key to decrypt the encrypted multi-source data to obtain the original data; sending the original data to a data anonymization processing engine, so that the data anonymization processing engine identifies personal information according to a pre-loaded information feature library and uses a preset hash function to encrypt and convert the personal information to obtain the processed original data; sending the processed original data to an access control system, so that the access control system checks the access level according to a preset access rule library to determine whether the access verification matches; if the access verification matches, obtaining the multi-source data within the access permissions.

[0007] In one possible implementation, the video image data includes structured video and structured information. Accordingly, the video data and image data are preprocessed to obtain video image data, including: decoding the video data and image data to obtain a raw frame sequence; using a target detection algorithm to identify monitored objects in the video data and image data for the raw frame sequence; continuously tracking the monitored objects and correcting their position and feature data based on their motion trajectory; filtering the tracked monitored objects according to a preset priority rule to obtain target objects; performing attribute analysis on the target objects to identify their target detail data and obtain attribute analysis results; rendering the video data and image data based on the attribute analysis results to obtain structured video; and extracting attributes from the structured video to obtain structured information.

[0008] In one possible implementation, the structured information includes pedestrian structured information and vehicle structured information; the pedestrian structured information includes the pedestrian's gender, age, orientation, and clothing characteristics; the clothing characteristics include whether the pedestrian is carrying a bag, a backpack, a shoulder bag, wearing glasses, and wearing a hat; the vehicle structured information includes the vehicle's brand, model, color, type, and direction.

[0009] In one possible implementation, the sensor data includes thermal imaging data and sound data; accordingly, a fusion decision is made based on the sensor data and video image data to obtain a sensor data fusion decision result, including: determining a target object in a target area based on the video image data; detecting the heat distribution of the target object based on the thermal imaging data; identifying the sound pattern of the target object based on the sound data; and obtaining the sensor data fusion decision result based on the heat distribution and the sound pattern, wherein the sensor data fusion decision result includes the identification result of a person or animal hidden in the shadow, the identification result of an object with abnormal temperature, and the sound identification result of an abnormal situation.

[0010] In one possible implementation, an alarm is generated based on the fusion decision results of security early warning response data and sensor data, and emergency response measures are triggered in the target area. This includes: classifying the fusion decision results of security early warning response data and sensor data according to preset types and sources to obtain data points; performing real-time monitoring and analysis on each data point according to preset rules to determine whether the feature quantities of each data point continuously meet preset abnormal conditions within a continuous time window; if the feature quantities of each data point continuously meet the preset abnormal conditions within a continuous time window, re-encapsulating each data point according to a preset format to obtain abnormal information; and sending the abnormal information to a cloud-native alarm system so that the cloud-native alarm system can trigger emergency response measures based on the abnormal information.

[0011] Secondly, embodiments of this application provide a video surveillance data processing apparatus, applied to computer equipment, comprising:

[0012] The data receiving module is used to receive multi-source data of the target area sent by the acquisition terminal through the 5G network. The multi-source data includes video data, image data and sensor data.

[0013] The preprocessing module is used to preprocess video data and image data to obtain video image data.

[0014] The object recognition module is used to input video image data into a pre-trained AI visual analysis model, identify objects in the target area, detect the motion trajectory and behavior patterns of the objects, and obtain safety warning response data.

[0015] The fusion decision module is used to make fusion decisions based on sensor data and video image data to obtain sensor data fusion decision results.

[0016] The early warning module is used to generate alarms and trigger emergency response measures in the target area based on the fusion decision results of safety early warning response data and sensor data.

[0017] Thirdly, embodiments of this application provide a computer device, including: a memory and a processor;

[0018] The memory stores instructions that the computer executes;

[0019] The processor executes computer execution instructions stored in memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0020] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0021] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0022] The video surveillance data processing method, apparatus, device, storage medium, and product provided in this application transmit multi-source data through a 5G network. By utilizing the low latency and high bandwidth characteristics of the 5G network, real-time transmission and processing of multi-source data are achieved, greatly improving real-time monitoring capabilities and response speed. In addition, by using a deep learning-based AI visual analysis model to identify and detect video image data, high-precision identification and detection are achieved using AI technology. This solves the problems of slow response speed and false alarms in traditional video surveillance platforms, improving the processing efficiency of video data and the accuracy of event detection. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0024] Figure 1 A schematic diagram of a video surveillance data processing method provided in an embodiment of this application;

[0025] Figure 2 A flowchart illustrating the video surveillance data processing method provided in this application embodiment;

[0026] Figure 2a A flowchart of the video structured analysis process provided in this application embodiment;

[0027] Figure 2b This is a decoding diagram provided for an embodiment of this application;

[0028] Figure 3 This is a schematic diagram of the structure of the video surveillance data processing device provided in the embodiments of this application;

[0029] Figure 4 A schematic diagram of the structure of a computer device provided in an embodiment of this application.

[0030] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0031] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0032] To clearly understand the technical solution of this application, the existing technical solutions will first be described in detail. In today's context of ever-increasing security demands, video surveillance is widely used in commercial areas and many other locations, serving as crucial for ensuring public and business safety and maintaining order. Traditional video surveillance platforms often rely on manual operation, resulting in slow response times and a high risk of false alarms, while also being inefficient at processing large amounts of video data. While intelligent analysis algorithms can reduce false alarm rates and improve data processing efficiency, they also face challenges such as high computer resource requirements, data privacy and security issues, and poor algorithm interpretability.

[0033] To address the aforementioned technical challenges, the inventors recognized that the low latency and high bandwidth of 5G networks enable real-time transmission and processing of video data, thereby improving the real-time monitoring capabilities and response speed of surveillance systems. Furthermore, the application of AI visual analysis algorithms can reduce false alarm rates while enhancing the accuracy of event detection. 5G and AI visual analysis technologies not only improve the intelligence level of surveillance systems but also reduce human intervention, thereby increasing the overall efficiency and security of surveillance systems.

[0034] Based on the above-mentioned inventive discovery, the inventor has proposed the technical solution of this application.

[0035] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0036] Figure 1 This is a schematic diagram illustrating a video surveillance data processing method provided in an embodiment of this application. Figure 1 As shown, the scenario includes: computer equipment 101, data acquisition terminal 102, and cloud-native alarm system 103.

[0037] Specifically, the acquisition terminal 102 transmits the multi-source data collected in the target area to the computer device 101 via a 5G network. The computer device 101 preprocesses the video and image data in the multi-source data to obtain video image data, and inputs the video image data into a pre-trained AI visual analysis model to identify objects in the target area and detect the motion trajectory and behavior patterns of the objects, thereby obtaining safety warning response information. Based on the sensor data and video image data in the multi-source data, a fusion decision is made to obtain the sensor data fusion decision result. Based on the safety warning response data and the sensor data fusion decision result, an alarm is generated and sent to the cloud-native alarm system 103. The cloud-native alarm system 103 triggers emergency response measures in the target area based on the warning.

[0038] Figure 2 This is a flowchart illustrating the video surveillance data processing method provided in an embodiment of this application. Figure 2 As shown, the method includes:

[0039] S201: Receive multi-source data of the target area sent by the acquisition terminal via the 5G network. The multi-source data includes video data, image data, and sensor data.

[0040] Specifically, multi-source data is captured in real time by monitoring equipment such as cameras and different types of sensors deployed in the target area, and the multi-source data is transmitted through a 5G network.

[0041] S202: Preprocess the video data and image data to obtain video image data.

[0042] The video image data includes structured video and structured information.

[0043] Specifically, Figure 2a The video structured analysis business process diagram provided in the embodiments of this application is as follows: Figure 2a As shown, a video structured analysis system is used to preprocess video and image data to obtain video image information. This system is developed using a data stream processing framework, and multiple plugins are combined into a pipeline to complete the overall video structuring function. Each step is completed by a plugin, which executes in parallel using multi-threading. Data transmission between plugins is achieved through a blocking message queue. Accordingly, the video structured analysis method includes:

[0044] Sa1: Decodes video and image data to obtain the original frame sequence.

[0045] The first plugin in the video structured analysis system is the decoding plugin. This plugin can parse RTSP / RTMP video streams, local video files of various encoding formats, and JPEG images.

[0046] Specifically, Figure 2b This is a decoding diagram provided for an embodiment of this application. Figure 2b As shown, the CPU performs software decoding. After the host obtains video and image data, it performs FFmpeg demultiplexing to output H.264 or H.265 video frames. At the device's data stream processing end, the data is decoded by a video decoder such as MLU. The decoded data, such as a YUV format image, is sent to the CPU host. The CPU performs preprocessing, mainly including color conversion, resizing, subtracting the mean and dividing the variance. Then, the original frame sequence of the output BGR format image is sent to the data stream processing end.

[0047] Sa2: For the original frame sequence, a target detection algorithm is used to identify the monitored objects in video and image data.

[0048] Among them, the YOLOv3 network, which has a relatively balanced performance and accuracy, was selected as the object detection algorithm. After being trained on a self-collected dataset, the network reduced the output classification to two classes: pedestrians and vehicles. A resolution of 608×608 was chosen as the input size of the model.

[0049] Specifically, for any video stream's original frame sequence, parallel inference is performed by accumulating batches. For example, if the batch size is set to 4, each inference operation processes these 4 frames simultaneously on 4 MLU (Machine Learning Unit) cores, which is equivalent to each core processing one frame at a time.

[0050] Sa3: Continuously tracks the monitored object and corrects the position and characteristic data of the monitored object based on the movement trajectory of the monitored object.

[0051] The tracking plugin uses feature matching for tracking and sets the batch size to 4.

[0052] Specifically, the features of the detected object are extracted and provided to the feature match module. Through Kalman filtering, Hungarian matching and other methods, tracking is achieved, and a unique identifier is assigned to the successfully tracked target.

[0053] Sa4: Based on preset priority rules, filter the monitored objects in the tracking to obtain the target object.

[0054] Among them, the preset priority rules support multiple "AND" relationship strategy conditions, including target size, target change trend, target position, and whether the target box is on the edge.

[0055] Specifically, for the monitored object, attribute analysis continues after the target meets all strategy conditions. To more accurately identify vehicle attributes and license plate information, filtering strategies are set for different scenarios. For example, if the target in the video is too small, it is difficult to obtain the correct license plate; therefore, the target size needs to be filtered. Attribute and license plate analysis is performed based on a set threshold. Only when the target size is trending upwards, and this trend exceeds the set threshold, will the monitored object be further selected to obtain the target object.

[0056] Sa5: Perform attribute analysis on the target object, identify the target object's detailed data, and obtain the attribute analysis results of the target object.

[0057] The targets include pedestrians, license plates, and vehicles.

[0058] Specifically, the AWMT Caffe framework model was used for pedestrian attribute analysis. Caffe is an open-source framework widely used in the field of deep learning. Its backbone is the ResNet50 classification network, which supports multi-label classification. After training with PA-100K and a self-collected dataset, the average accuracy was 76.32.

[0059] Specifically, the vehicle attribute analysis uses a GoogleNet backbone network. One model is trained on the compcars dataset to identify the brand and model of the vehicle, while the other model is trained on a self-collected dataset for color, type, and orientation.

[0060] Specifically, license plate recognition comprises three networks, used for license plate detection, license plate correction (affine transformation), and license plate recognition, respectively.

[0061] Sa6: Based on the attribute analysis results, the video and image data are rendered to obtain structured video.

[0062] Specifically, structured video is output according to the original video frame rate through FFmpeg (Multimedia Processing Toolset) software encoding and MLU hardware encoding.

[0063] Sa7: Extract attributes from structured videos to obtain structured information.

[0064] Commonly used fields for attribute extraction include: camera information, frame number, pedestrian structured information, and vehicle structured information.

[0065] Specifically, pedestrian structured information includes a person's gender, age, orientation, and clothing characteristics. Clothing characteristics include whether the person is carrying a bag, a backpack, a shoulder bag, wearing glasses, or wearing a hat. Vehicle structured information includes the vehicle's brand, model, color, type, and orientation.

[0066] Specifically, structured information output can be achieved through TCP messages or message queue middleware, etc.

[0067] S203: Input video image data into a pre-trained AI visual analysis model to identify objects in the target area and detect the motion trajectory and behavior patterns of the objects to obtain safety warning response data.

[0068] Specifically, video image data is input into a deep learning-based AI visual analysis model to identify objects in the target area and detect their motion trajectories and behavioral patterns to obtain safety warning response information, which includes pedestrian fall warnings, crowd gathering warnings, vehicle illegal parking warnings, and vehicle collision warnings.

[0069] Specifically, through AI visual analysis algorithms, a deep learning-based AI visual analysis model composed of multi-layer convolutional neural networks and / or recurrent neural networks is constructed to accurately identify and detect people, vehicles, objects, etc. in the target area after preliminary processing, and analyze the movement trajectory and behavior patterns of the objects, such as abnormal behavior detection such as running, falling, gathering, etc., thereby providing real-time safety warnings and responses.

[0070] Deep learning is a type of machine learning that uses neural networks. Deep learning neural networks consist of multiple layers of software modules that work collaboratively within a computer. They utilize mathematical calculations to automatically process different aspects of image data and gradually form a comprehensive understanding of the image.

[0071] Convolutional Neural Networks (CNNs) utilize a labeling system to classify visual data and understand the entire image. They analyze the image as pixels and assign a label value to each pixel. This value is input to perform a mathematical operation called "convolution" and to make predictions about the image. Just as humans try to identify objects in the distance, CNNs first identify outlines and simple shapes, then fill in other details such as color, internal shape, and texture. Finally, it iterates through the prediction process multiple times to improve accuracy.

[0072] Recurrent Neural Networks (RNNs) are similar to CNNs, but they can process a series of images to find relationships between them. While CNNs are used to analyze single images, RNNs can analyze videos and understand the relationships between images.

[0073] S204: Make a fusion decision based on sensor data and video image data to obtain the sensor data fusion decision result.

[0074] The sensor data includes thermal imaging data and sound data.

[0075] Specifically, the steps for making fusion decisions based on sensor data and video image data to obtain the sensor data fusion decision results include Sb1~Sb4:

[0076] Sb1: Determine the target object in the target area based on the video image data.

[0077] Sb2: Detects the heat distribution of the target object based on thermal imaging data.

[0078] Sb3: Identify the sound pattern of the target object based on sound data.

[0079] Sb4: Based on heat distribution and sound patterns, sensor data fusion decision results are obtained, including the identification results of people or animals hidden in shadows, the identification results of objects with abnormal temperatures, and the sound identification results of abnormal situations.

[0080] For example, for video data, visual information of objects is captured, such as people and vehicles, and different objects are identified through object detection algorithms. For thermal imaging data, the heat distribution of objects is detected, which can identify people or animals hidden in shadows, or distinguish objects with abnormal temperatures. For sound data, sound patterns are analyzed, such as identifying the sound of breaking glass or specific shouts, to determine whether an abnormal situation has occurred.

[0081] S205: Based on the fusion decision results of safety early warning response data and sensor data, generate an alarm and trigger emergency response measures in the target area.

[0082] Specifically, an alarm is generated and emergency response measures are triggered in the target area when the fusion decision results of safety early warning response information and sensor data continuously meet the pre-created abnormal alarm rules.

[0083] In summary, transmitting multi-source data via 5G networks, leveraging their low latency and high bandwidth to achieve real-time transmission and processing, significantly improves real-time monitoring capabilities and response speed. Furthermore, using deep learning-based AI visual analysis models to identify and detect video image data, and employing AI technology to achieve high-precision identification and detection, can solve the problems of slow response speed and false alarms in traditional video surveillance platforms, thereby improving the efficiency of video data processing and the accuracy of event detection.

[0084] In another embodiment provided in this application, during transmission over a 5G network, the 5G network provides encrypted transmission and data anonymization technologies, including:

[0085] S301: During transmission over a 5G network, it monitors and rapidly receives encrypted multi-source data from the target area in real time, with the 5G network having a built-in channel encryption mechanism.

[0086] Specifically, by leveraging the high-speed, low-latency data transmission channel of the 5G network, its built-in channel encryption mechanism can be fully utilized to monitor and rapidly receive encrypted multi-source data from the target area in real time and without interruption. This encrypted multi-source data covers various types of information, including video data, image data, and sensor data.

[0087] S302: Use a preset verification algorithm to perform integrity verification on encrypted multi-source data in order to determine whether the encrypted multi-source data is abnormal.

[0088] Specifically, after the data is received, a preset verification algorithm is used to perform integrity verification on the encrypted multi-source data. This algorithm can be a hash verification, which compares elements such as the data sequence and checksum to determine whether there are any anomalies such as data loss, tampering, or errors in the encrypted multi-source data.

[0089] S303: If there is no anomaly in the encrypted multi-source data, the private key is obtained from the encrypted storage unit, and the private key is used to decrypt the encrypted multi-source data to obtain the original data.

[0090] Specifically, if the verification result indicates that the encrypted multi-source data is not abnormal, the corresponding private key is accurately retrieved from the encrypted storage unit. This encrypted storage unit employs high-strength storage encryption technology to ensure the security of the stored private key. Using the obtained private key, a mature decryption algorithm is used to decrypt the encrypted multi-source data to obtain the original data.

[0091] S304: Send the raw data to the data anonymization processing engine so that the data anonymization processing engine can identify personal information according to the pre-loaded information feature library, and use a preset hash function to encrypt and transform the personal information to obtain the processed raw data.

[0092] Specifically, the decrypted raw data is sent to a data anonymization processing engine. This engine is pre-loaded with an information feature library, which covers various common personal information feature patterns, such as textual, numerical, and combined features of typical personal identification information like names, ID numbers, and contact information. Based on this feature library, the data anonymization processing engine accurately identifies the personal information hidden in the raw data and then uses a preset hash function to encrypt and transform this personal information. The hash function maps personal information to a fixed-length hash value, thus concealing the original personal information while preserving some degree of data usability. The final processed raw data effectively protects personal privacy information.

[0093] S305: The processed raw data is sent to the access control system so that the access control system can check the access level according to the preset access rule base to determine whether the access verification matches.

[0094] Specifically, the processed raw data is transmitted to the access control system, which operates based on a pre-defined access rule base. This base meticulously formulates detailed access level rules based on multiple dimensions, including different user roles, business needs, and data sensitivity. The access control system strictly adheres to these rules, verifying the access levels of the incoming data and carefully checking for matching permissions to ensure that only authorized users can access the corresponding data, thus further strengthening data security control at the access level.

[0095] S306: If the permission verification matches, then multi-source data within the access permissions is obtained.

[0096] Specifically, if the above permission verification process is completed and the permission verification is found to be matched, then the user can successfully obtain multi-source data within their access permissions.

[0097] In summary, thanks to the built-in channel encryption mechanism of 5G networks, multi-source data collected from the target area is rigorously protected from the start of transmission, effectively preventing the risk of malicious external interception and information theft. On the other hand, converting personal information into a hash function with a difficult-to-recognize character sequence can conceal personal privacy information while ensuring data availability, cleverly avoiding the risk of privacy leaks.

[0098] In another embodiment provided in this application, an alarm is generated based on the fusion decision results of security warning response data and sensor data, and emergency response measures in the target area are triggered, specifically including:

[0099] S401: Based on the preset type and source, classify the fusion decision results of safety early warning response data and sensor data to obtain each data point.

[0100] For example, complex data can be divided into specific data points according to the type of object involved in the data, such as people or vehicles, or according to the type of device from which the data originates, such as cameras in different locations or various sensors.

[0101] S402: According to preset rules, perform real-time monitoring and analysis on each data point to determine whether the characteristic quantity of each data point continuously meets the preset abnormal conditions within a continuous time window.

[0102] Specifically, real-time monitoring and analysis are conducted on each data point according to preset rules. These preset rules are established based on research into the characteristics of normal and abnormal data. By continuously observing and evaluating the characteristic quantities of data points, it is determined whether they consistently meet preset abnormal conditions within a continuous time window. For example, when monitoring personnel behavior data, if the movement speed, activity range, and other characteristic quantities of personnel continuously exceed the normal range for a period of time, it may meet the abnormal conditions.

[0103] S403: If the feature quantity of each data point continuously meets the preset abnormal conditions within a continuous time window, then each data point is repackaged according to the preset format to obtain abnormal information.

[0104] Specifically, if the feature values ​​of each data point continuously meet the preset anomaly conditions within a continuous time window, then these data points are repackaged according to a preset format. The preset format may include key information such as the data point's identifier, feature value, and timestamp, which are then integrated into a complete anomaly information package.

[0105] S404: Send the abnormal information to the cloud-native alarm system so that the cloud-native alarm system can trigger emergency response measures based on the abnormal information.

[0106] Specifically, the packaged anomaly information is sent to the cloud-native alarm system, which will automatically trigger corresponding emergency response measures based on the received anomaly information. For example, when fire-related anomalies are detected, the cloud-native alarm system can quickly activate fire-fighting equipment and notify relevant personnel to evacuate, among other emergency measures.

[0107] In summary, classifying data according to preset types and sources enables subsequent processing to quickly identify the corresponding data, making the entire data processing workflow more organized and clear. Real-time monitoring and analysis of each data point according to established rules allows for the identification and unified management of anomalies. Once an anomaly is detected, data points meeting the criteria are packaged into anomaly information according to a preset format, facilitating subsequent identification and unified management. This anomaly information is then transmitted to a cloud-native alarm system, triggering emergency response measures. This achieves efficient integration from data anomaly detection to emergency handling, comprehensively ensuring system stability.

[0108] Figure 3 This is a schematic diagram of the structure of the video surveillance data processing device provided in an embodiment of this application. Figure 3 As shown, the device includes: a data receiving module 301, a preprocessing module 302, an object recognition module 303, a fusion decision module 304, and an early warning module 305.

[0109] The data receiving module 301 is used to receive multi-source data of the target area sent by the acquisition terminal through the 5G network, wherein the multi-source data includes video data, image data and sensor data.

[0110] The preprocessing module 302 is used to preprocess video data and image data to obtain video image data.

[0111] The object recognition module 303 is used to input video image data into a pre-trained AI visual analysis model, identify objects in the target area, detect the motion trajectory and behavior patterns of the objects, and obtain safety warning response data.

[0112] The fusion decision module 304 is used to make fusion decisions based on sensor data and video image data to obtain sensor data fusion decision results.

[0113] The early warning module 305 is used to generate an alarm and trigger emergency response measures in the target area based on the fusion decision results of safety early warning response data and sensor data.

[0114] In one possible implementation, the device further includes a transmission module for real-time monitoring and rapid reception of encrypted multi-source data from the target area during transmission over a 5G network, wherein the 5G network has a built-in channel encryption mechanism; performing integrity verification on the encrypted multi-source data using a preset verification algorithm to determine whether the encrypted multi-source data is abnormal; if the encrypted multi-source data is not abnormal, retrieving the private key from the encrypted storage unit and using the private key to decrypt the encrypted multi-source data to obtain the original data; sending the original data to a data anonymization processing engine, which identifies personal information according to a pre-loaded information feature library and encrypts and transforms the personal information using a preset hash function to obtain the processed original data; and sending the processed original data to an access control system, which checks the access level against a preset access rule library to determine whether the access verification matches; if the access verification matches, obtaining the multi-source data within the access permissions is obtained.

[0115] In one possible implementation, the preprocessing module 302 is specifically used to decode video data and image data to obtain an original frame sequence; for the original frame sequence, a target detection algorithm is used to identify the monitored objects in the video data and image data; the monitored objects are continuously tracked, and the position and feature data of the monitored objects are corrected according to the motion trajectory of the monitored objects; the monitored objects in the tracking are filtered according to a preset priority rule to obtain the target objects; the target objects are subjected to attribute analysis to identify the target object's target detail data and obtain the attribute analysis results of the target objects; the video data and image data are rendered according to the attribute analysis results to obtain a structured video; and attribute extraction is performed in the structured video to obtain structured information.

[0116] In one possible implementation, the structured information in the preprocessing module 302 includes pedestrian structured information and vehicle structured information; the pedestrian structured information includes the pedestrian's gender, age, orientation, and clothing characteristics; the clothing characteristics include whether the pedestrian is carrying a bag, a backpack, a shoulder bag, wearing glasses, and wearing a hat; the vehicle structured information includes the vehicle's brand, model, color, type, and orientation.

[0117] In one possible implementation, the fusion decision module 304 is specifically used to determine the target object in the target area based on video image data; detect the heat distribution of the target object based on thermal imaging data; identify the sound pattern of the target object based on sound data; and obtain the sensor data fusion decision result based on the heat distribution and sound pattern, wherein the sensor data fusion decision result includes the identification result of people or animals hidden in shadows, the identification result of objects with abnormal temperatures, and the sound identification result of abnormal situations.

[0118] In one possible implementation, the early warning module 305 is specifically used to classify the safety early warning response data and sensor data fusion decision results according to preset types and sources to obtain each data point; according to preset rules, it performs real-time monitoring and analysis on each data point to determine whether the feature quantity of each data point continuously meets the preset abnormal conditions within a continuous time window; if the feature quantity of each data point continuously meets the preset abnormal conditions within a continuous time window, it re-encapsulates each data point according to a preset format to obtain abnormal information; and sends the abnormal information to the cloud-native alarm system so that the cloud-native alarm system can trigger emergency response measures based on the abnormal information.

[0119] The video surveillance data processing device provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0120] Figure 4 A schematic diagram of the structure of a computer device provided in an embodiment of this application. For example... Figure 4 As shown, the computer device provided in this embodiment includes at least one processor 401 and a memory 402. Optionally, the computer device further includes a communication component 403. The processor 401, memory 402, and communication component 403 are connected via a bus 404.

[0121] In a specific implementation, at least one processor 401 executes computer execution instructions stored in memory 402, causing at least one processor 401 to perform the above-described method.

[0122] The specific implementation process of processor 401 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0123] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0124] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0125] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0126] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0127] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0128] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0129] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0130] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0131] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0132] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0133] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0134] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0135] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A video surveillance data processing method, characterized in that, Applied to computer equipment, including: The system receives multi-source data of the target area sent by the acquisition terminal via a 5G network, wherein the multi-source data includes video data, image data, and sensor data. The video data and the image data are preprocessed to obtain video image data; The video image data is input into a pre-trained AI visual analysis model to identify objects in the target area and detect the motion trajectory and behavior patterns of the objects to obtain safety warning response data. A fusion decision is made based on the sensor data and the video image data to obtain the sensor data fusion decision result; Based on the fusion decision results of the security early warning response data and the sensor data, an alarm is generated, and emergency response measures are triggered in the target area.

2. The method according to claim 1, characterized in that, Also includes: During transmission via the 5G network, encrypted multi-source data of the target area is monitored in real time and received rapidly, wherein the 5G network has a built-in channel encryption mechanism; The encrypted multi-source data is subjected to integrity verification using a preset verification algorithm to determine whether the encrypted multi-source data is abnormal. If the encrypted multi-source data is not abnormal, the private key is obtained from the encrypted storage unit, and the private key is used to decrypt the encrypted multi-source data to obtain the original data. The raw data is sent to the data anonymization processing engine, which identifies personal information according to the pre-loaded information feature library and uses a preset hash function to encrypt and convert the personal information to obtain the processed raw data. The processed raw data is sent to the access control system so that the access control system can check the access level according to the preset access rule library to determine whether the access verification matches. If the permission verification matches, then multi-source data within the access permissions is obtained.

3. The method according to claim 1, characterized in that, The video image data includes structured video and structured information; Accordingly, the preprocessing of the video data and the image data to obtain video image data includes: The video data and the image data are decoded to obtain the original frame sequence; For the original frame sequence, a target detection algorithm is used to identify the monitored objects in the video data and the image data; The monitored object is continuously tracked, and its position and feature data are corrected based on its movement trajectory. Based on preset priority rules, the monitored objects in the tracking are filtered to obtain the target objects; Perform attribute analysis on the target object to identify the target object's detailed data and obtain the attribute analysis results of the target object; Based on the attribute analysis results, the video data and the image data are rendered to obtain the structured video; The structured information is obtained by extracting attributes from the structured video.

4. The method according to claim 3, characterized in that, The structured information includes pedestrian structured information and vehicle structured information; The structured information of pedestrians includes the pedestrian's gender, age, orientation, and clothing characteristics; The clothing features include whether the wearer carries a handbag, a backpack, a shoulder bag, wears glasses, and wears a hat; The structured vehicle information includes the vehicle's brand, model, color, type, and direction of travel.

5. The method according to claim 1, characterized in that, The sensor data includes thermal imaging data and sound data; Accordingly, the step of making a fusion decision based on the sensor data and the video image data to obtain the sensor data fusion decision result includes: Based on the video image data, the target object in the target area is determined; Detect the heat distribution of the target object based on the thermal imaging data; Identify the sound pattern of the target object based on the sound data; Based on the heat distribution and the sound pattern, a sensor data fusion decision result is obtained, wherein the sensor data fusion decision result includes the identification result of a person or animal hidden in the shadow, the identification result of an object with abnormal temperature, and the sound identification result of an abnormal situation.

6. The method according to claim 1, characterized in that, The step of generating an alarm and triggering emergency response measures in the target area based on the fusion decision results of the security warning response data and the sensor data includes: Based on preset types and sources, the safety early warning response data and the sensor data fusion decision results are classified to obtain each data point; According to preset rules, each data point is monitored and analyzed in real time to determine whether the feature quantity of each data point continuously meets the preset abnormal conditions within a continuous time window. If the feature values ​​of each data point continuously meet the preset abnormal conditions within a continuous time window, then the data points are repackaged according to the preset format to obtain abnormal information. The abnormal information is sent to the cloud-native alarm system so that the cloud-native alarm system can trigger emergency response measures based on the abnormal information.

7. A video surveillance data processing device, characterized in that, Applied to computer equipment, including: The data receiving module is used to receive multi-source data of the target area sent by the acquisition terminal through the 5G network, wherein the multi-source data includes video data, image data and sensor data; The preprocessing module is used to preprocess the video data and the image data to obtain video image data; The object recognition module is used to input the video image data into a pre-trained AI visual analysis model, identify objects in the target area, detect the motion trajectory and behavior pattern of the objects, and obtain safety warning response data. The fusion decision module is used to make fusion decisions based on the sensor data and the video image data to obtain the sensor data fusion decision result. The early warning module is used to generate an alarm based on the fusion decision results of the security early warning response data and the sensor data, and to trigger emergency response measures in the target area.

8. A computer device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 6.

Images