Safety monitoring method and system based on artificial intelligence

Abstract

The invention provides a safety monitoring method and system based on artificial intelligence. The method comprises the steps that firstly, real-time video data and environment data are collected through a camera and a sensor which are installed at a preset position; secondly, splicing the preprocessed video data and environment data to form an input vector; then, the change amplitude of the preprocessed video data and the change amplitude of the normalized environment data in the time period T are calculated respectively; thirdly, outputting a danger degree value by adopting a first convolutional neural network or a second convolutional neural network or a third convolutional neural network; finally, the danger degree value output by the convolutional neural network is sent to a manager terminal to be displayed, and an alarm is pulled. According to the invention, different types of convolutional neural network algorithms are adopted to carry out danger degree calculation, so that the accuracy and efficiency of safety accident reminding are greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent automated security processing technology, and specifically to a security monitoring method and system based on artificial intelligence. Background Art

[0002] As society's demand for security monitoring continues to increase, intelligent monitoring systems are increasingly being used in various scenarios. Traditional security monitoring systems mainly rely on manual monitoring, rule setting, or simple mobile detection technology to identify abnormal activities. However, traditional methods face problems such as low processing efficiency, high false alarm rate, and difficulty in coping with complex environmental changes. Especially in dynamic environments, such as crowded public places or complex industrial sites, the environment changes rapidly, and traditional systems are often unable to make quick and accurate judgments, which in turn affects the timeliness and effectiveness of security responses.

[0003] In recent years, with the development of computer vision and artificial intelligence technology, intelligent monitoring systems based on deep learning have gradually become a research hotspot. Convolutional neural networks (CNNs) are widely used in behavior recognition and anomaly detection due to their powerful image feature extraction capabilities. Deep learning algorithms can achieve more accurate anomaly detection than traditional methods by automatically learning and extracting important features from video frames. However, existing deep learning monitoring systems often rely only on video data, ignoring the impact of environmental data (such as temperature, humidity, sound, etc.) on the monitoring scene, and thus cannot fully grasp the safety situation of the monitoring scene.

[0004] In applications, a single data source often has difficulty accurately judging complex situations. For example, in poorly lit environments, the accuracy of video data will be affected, while environmental sensor data can provide valuable supplementary information. On the other hand, changes in environmental data, such as sudden temperature rise or humidity fluctuations, may indicate potential risks of fire or equipment failure, and relying solely on video data may not be able to detect these environmental changes in a timely manner. Therefore, how to combine video and environmental data and realize multimodal information fusion has become an important research direction for improving the intelligence level of security monitoring systems.

[0005] In addition, when the monitoring system processes video and environmental data in a dynamic environment, traditional deep learning methods usually use fixed network structures and parameters, which are difficult to adjust adaptively according to real-time changes. This fixed structure may cause the system to perform poorly in complex scenes with significant changes. For example, in video data, when there are rapid screen switching or large changes in light, the monitoring system needs to use a larger convolution kernel to capture global dynamic information; in environmental data, when significant changes in temperature and humidity are detected, the system needs to focus on environmental data and reduce its dependence on video data, so as to more accurately assess on-site risks. In addition, the existing processing process cannot automatically and specifically select the corresponding model according to the on-site situation, and faces problems such as insufficient automation, low accuracy and efficiency, and unstable quality of monitoring results. Therefore, how to design a convolutional neural network that can dynamically adjust according to data changes has become a key issue in the optimization of current intelligent monitoring systems.

[0006] Based on the above challenges, some studies have proposed solutions that combine multimodal information with deep learning, that is, adding environmental sensor data to traditional video surveillance to improve the adaptability of the surveillance system in different scenarios. These solutions usually adopt a multi-input convolutional neural network structure, taking video and environmental data as input, and extracting video and environmental features respectively through convolutional layers of different channels. However, existing multimodal fusion methods mainly use fixed network structures, which are not able to respond to environmental changes. For example, when the monitoring scene changes dramatically, the traditional convolutional neural network cannot flexibly adjust the convolution kernel size and network structure, making it difficult to cope with the dynamic changes of video and environmental data at the same time, which limits its application in complex monitoring scenarios. Summary of the invention

[0007] In response to the above problems mentioned in the prior art, the present invention provides a security monitoring method and system based on artificial intelligence. The method first collects real-time video data and environmental data through cameras and sensors installed at predetermined locations; secondly, the preprocessed video data and environmental data are spliced ​​to form an input vector; then, the change amplitudes β1 and β2 of the preprocessed video data and the normalized environmental data within the time period T are calculated respectively; again, the first convolutional neural network or the second convolutional neural network or the third convolutional neural network is used to output the danger level value; finally, the danger level value output by the convolutional neural network is sent to the management terminal for display and an alarm is sounded. This application greatly improves the accuracy and efficiency of video processing and greatly increases the user experience by using different types of convolutional neural network algorithms for danger level calculation.

[0008] The present application provides a security monitoring method based on artificial intelligence, comprising the steps of:

[0009] S1: collects real-time video data and environmental data through cameras and sensors installed at predetermined locations;

[0010] S2: intercepting the video data in the time period T before the current time point to perform video image preprocessing; intercepting the environmental data in the time period T before the current time point to perform normalization processing; and splicing the processed video data and environmental data to form an input vector;

[0011] S3: Calculate the change amplitudes β1 and β2 of the preprocessed video data and the normalized environmental data in the time period T respectively;

[0012] S4: When β1 is greater than the set threshold β T1 When β2 is less than the set threshold β T2 When , the input vector is input into the trained first convolutional neural network, and the first convolutional neural network outputs the danger level value;

[0013] When β2 is greater than the set threshold β T2 When β1 is less than the set threshold β T1 When , the input vector is input into the trained second convolutional neural network, and the second convolutional neural network outputs the danger level value;

[0014] When β2 is greater than the set threshold β T2 When β1 is greater than the set threshold β T1 When , the input vector is input into the trained third convolutional neural network, and the third convolutional neural network outputs the danger level value;

[0015] When β1 is less than the set threshold β T1 When β2 is less than the set threshold β T2 When , return to step S1;

[0016] S5: Send the danger level value output by the convolutional neural network to the management terminal for display and sound an alarm.

[0017] Preferably, the camera is a CCD camera, and the environmental data includes the ambient temperature collected by a temperature sensor, the air humidity measured by a humidity sensor, the sound data in the environment collected by a microphone, the sensor measuring the chemical composition in the air, and an accelerometer or vibration sensor installed to detect ground vibration or movement of the device itself.

[0018] Preferably, preprocessing the video image includes denoising the video image using Gaussian filtering; splicing the processed video data and environmental data to form an input vector includes extracting feature vectors from the video data through a pre-trained convolutional neural network, and directly using the environmental data as the feature vector; numerically splicing the feature vectors of the video data and the feature vectors of the environmental data end-to-end to form a complete input vector.

[0019] Preferably, calculating the change amplitude β1 of the preprocessed video data within the time period T includes: S31: first calculating the difference in pixel grayscale values ​​of pixels at corresponding positions between consecutive adjacent video frames within the time period T; S32: summing and averaging the pixel grayscale value differences of each pair of adjacent frames to obtain the average value of a single frame difference; S33: summing and averaging all the calculated average values ​​of the single frame differences to obtain the video change amplitude β1 within the entire time period T.

[0020] Preferably, calculating the change amplitude β2 of the normalized environmental data within the time period T includes: S34: calculating the difference between consecutive adjacent environmental data points within the time period T; S35: calculating the absolute value of the difference between each pair of adjacent data points; S36: calculating the average value of these differences to obtain the environmental change amplitude β2 within the entire time period T.

[0021] Preferably, the first convolutional neural network uses a 7*7 convolution kernel, and the activation function used is a Sigmoid activation function f1(x) expressed as follows:

[0022] The second convolutional neural network uses a 3*3 convolution kernel and the activation function used is the Sigmoid activation function f2(x) as follows:

[0023] The third convolutional neural network uses a 5*5 convolution kernel, and the activation function used is the Tanh activation function f3(x) expressed as follows: f3(x)=Tanh(x+δ*(β1+β2));

[0024] Among them, the activation function is applied to the hidden layer and output layer of the convolutional neural network, x represents the input value of the current neuron, e is the base of the natural logarithm; a is the proportional coefficient of the influence of the change amplitude of the video data on the activation function, β1 is the video change amplitude within the time period T; b is the proportional coefficient of the influence of the change amplitude of the environmental data on the activation function, β2 is the environmental change amplitude within the time period T; δ is the composite proportional coefficient of the influence of the change amplitude of the video and the change amplitude of the environmental data on the activation function.

[0025] The present application also provides a security monitoring system based on artificial intelligence, comprising:

[0026] Data acquisition module: collects real-time video data and environmental data through cameras and sensors installed at predetermined locations;

[0027] Preprocessing module: intercept the video data in the time period T before the current time point to perform video image preprocessing; intercept the environmental data in the time period T before the current time point to perform normalization processing; and splice the processed video data and environmental data to form an input vector;

[0028] A change amplitude calculation module: calculates the change amplitudes β1 and β2 of the preprocessed video data and the normalized environmental data within a time period T respectively;

[0029] Danger level output module: When β1 is greater than the set threshold β T1 When β2 is less than the set threshold β T2 When , the input vector is input into the trained first convolutional neural network, and the first convolutional neural network outputs the danger level value;

[0030] When β2 is greater than the set threshold β T2 When β1 is less than the set threshold β T1 When , the input vector is input into the trained second convolutional neural network, and the second convolutional neural network outputs the danger level value;

[0031] When β2 is greater than the set threshold β T2 When β1 is greater than the set threshold β T1 When , the input vector is input into the trained third convolutional neural network, and the third convolutional neural network outputs the danger level value;

[0032] When β1 is less than the set threshold β T1 When β2 is less than the set threshold β T2 When , return to step S1;

[0033] Alarm reminder module: sends the danger level value output by the convolutional neural network to the management terminal for display and sounds the alarm.

[0034] Preferably, calculating the change amplitude β1 of the preprocessed video data within the time period T includes: S31: first calculating the difference in pixel grayscale values ​​of pixels at corresponding positions between consecutive adjacent video frames within the time period T; S32: summing and averaging the pixel grayscale value differences of each pair of adjacent frames to obtain the average value of a single frame difference; S33: summing and averaging all the calculated average values ​​of the single frame differences to obtain the video change amplitude β1 within the entire time period T.

[0035] Preferably, calculating the change amplitude β2 of the normalized environmental data within the time period T includes: S34: calculating the difference between consecutive adjacent environmental data points within the time period T; S35: calculating the absolute value of the difference between each pair of adjacent data points; S36: calculating the average value of these differences to obtain the environmental change amplitude β2 within the entire time period T.

[0036] Preferably, the first convolutional neural network uses a 7*7 convolution kernel, and the activation function used is a Sigmoid activation function f1(x) expressed as follows:

[0037] The second convolutional neural network uses a 3*3 convolution kernel and the activation function used is the Sigmoid activation function f2(x) as follows:

[0038] The third convolutional neural network uses a 5*5 convolution kernel, and the activation function used is the Tanh activation function f3(x) expressed as follows: f3(x)=Tanh(x+δ*(β1+β2));

[0039] Among them, x represents the input value of the current neuron, e is the base of the natural logarithm; a is the proportional coefficient of the influence of the change amplitude of the video data on the activation function, β1 is the video change amplitude within the time period T; b is the proportional coefficient of the influence of the change amplitude of the environmental data on the activation function, β2 is the environmental change amplitude within the time period T; δ is the composite proportional coefficient of the influence of the control video change and the environmental data change amplitude on the activation function.

[0040] The present invention provides a security monitoring method and system based on artificial intelligence, which can achieve the following beneficial technical effects:

[0041] 1. First, collect real-time video data and environmental data through cameras and sensors installed at predetermined locations; second, splice the preprocessed video data and environmental data to form an input vector; then, respectively calculate the change amplitudes β1 and β2 of the preprocessed video data and the normalized environmental data within the time period T; thirdly, use the first convolutional neural network or the second convolutional neural network or the third convolutional neural network to output the danger level value; finally, send the danger level value output by the convolutional neural network to the management terminal for display and sound an alarm. This application greatly improves the accuracy and efficiency of video processing and greatly increases the user experience by using different types of convolutional neural network algorithms for danger level calculation.

[0042] 2. The present invention dynamically adjusts the convolution kernel size according to the change amplitude of video data and environmental data, so that the network uses a larger convolution kernel to capture global context information when the data changes significantly, and uses a smaller convolution kernel to focus on detail features when the data changes slightly. Through this adaptive adjustment, the system can flexibly respond to various complex situations in different monitoring environments and improve the efficiency and accuracy of feature extraction.

[0043] 3. The present invention uses an adaptive activation function with added amplitude of change in different convolutional neural networks. The improved Sigmoid and Tanh activation functions with added amplitude of change factors are adopted, and the amplitude of change of video or environmental data is used as the adjustment parameter of the activation function. The network can be more sensitive to data sources with significant changes. This design enables the system to adjust the response output according to the actual environmental conditions, and greatly improves the accuracy of identifying abnormal behaviors and potential threats in dynamic environments. This solution utilizes the multimodal fusion of video data and environmental data, combines the visual information collected by the camera with environmental information such as temperature, humidity, sound, and vibration, and comprehensively constructs a multidimensional data model of the monitoring environment. Through this multimodal data analysis, the system can combine environmental changes on the basis of video monitoring, improve the comprehensiveness of threat assessment, reduce the underreporting rate and false alarm rate, and effectively identify potential safety hazards that may be overlooked. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 It is a schematic diagram of the steps of a security monitoring method based on artificial intelligence of the present invention;

[0046] Figure 2 It is a schematic diagram of a security monitoring system based on artificial intelligence of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Embodiment 1:

[0049] In view of the above problems mentioned in the prior art, in order to solve the above technical problems, as shown in the attached Figure 1 As shown: This application provides a security monitoring method based on artificial intelligence, including the steps of:

[0050] S1: Real-time video data and environmental data are collected by cameras and sensors installed at predetermined locations; the present invention is applied to intelligent security monitoring systems in public places (such as shopping malls, subway stations, etc.) to achieve real-time monitoring of crowd flow, emergencies and environmental changes. In some embodiments, cameras are installed at the main entrances and exits, stairs, elevator entrances and crowded areas of public places with high-definition CCD cameras to capture large-scale video data and cover the main areas of crowd activities. The camera installation height and angle are optimized to ensure clear capture of changes in crowd flow, abnormal behavior of personnel, etc. Environmental sensors: Multiple environmental sensors are arranged in the monitoring area, including temperature and humidity sensors, air quality sensors, sound sensors and vibration sensors. Temperature and humidity sensors: monitor temperature and humidity changes in the area, and detect in real time whether there are abnormal temperatures or sudden changes in humidity. Air quality sensors: detect indicators such as carbon dioxide and volatile organic compounds (VOCs) in the air to prevent hidden dangers such as fire, chemical leakage or air quality degradation. Sound sensors: used to detect whether there are abnormal sounds (such as shouting, explosions) in the crowd to help identify possible security threats. Vibration sensors: monitor ground vibrations to help identify severe vibrations or potential explosion risks. Video data collection: The camera collects video streams in real time to monitor the density, behavior characteristics and movement trajectory of people in the area. Video data can help identify abnormal behaviors such as abnormal crowd gathering and fast running. Environmental data collection: Sensors collect environmental data in real time, including temperature, humidity, air composition changes, sound levels and ground vibration data. This data reflects the immediate changes in the environment and can detect potential environmental anomalies. Data preprocessing, denoising and standardizing the video data, such as using Gaussian filtering to remove noise in the video to ensure clear images. Normalizing the environmental data, converting different types of data into a unified scale so that they can form an input vector together with the video data. Feature extraction and change amplitude calculation, video data feature extraction: Extract crowd movement features, object size and direction changes from the video, calculate the difference between adjacent frames in time period T, and use it to judge the change amplitude of the crowd or object. Environmental data change amplitude calculation: Calculate the change amplitude of data such as temperature, humidity, and vibration in time period T, and mark whether there is a significant change. Dynamically select convolutional neural networks and evaluate threat levels. According to the magnitude of changes in video data and environmental data, the system dynamically selects the appropriate convolutional neural network to process input data: Scenario 1: When the video changes greatly and the environmental changes are small (such as a rapid increase in the flow of people), the system selects the first convolutional neural network for processing, quickly detects potential abnormal behaviors in the crowd (such as running, pushing, etc.), and outputs the danger level. Scenario 2: When the environmental changes are large and the video changes are small (such as rapid changes in temperature and humidity or a decrease in air quality), the system selects the second convolutional neural network to analyze environmental anomalies and output the corresponding danger level.Scenario 3: When both video and environmental data change greatly (such as abnormal behavior in the crowd and a rapid rise in ambient temperature), the system selects the third convolutional neural network for multimodal data fusion analysis to comprehensively assess the threat level. The system triggers the corresponding alarm mechanism based on the danger level output by the convolutional neural network. For example, when high-risk behavior (such as crowds, abnormal sounds, and temperature changes) is detected, an alarm is immediately sent to the manager's terminal and an evacuation order is issued on the public broadcast. In the case of a lower danger level, the system sends the alarm information to the background monitoring room for management personnel to observe and determine whether further intervention is needed.

[0051] S2: intercepting the video data in the time period T before the current time point to perform video image preprocessing; intercepting the environmental data in the time period T before the current time point to perform normalization processing; and splicing the processed video data and environmental data to form an input vector;

[0052] In some embodiments, in the security monitoring system of a shopping mall, the video data and environmental data within the time period T are preprocessed and normalized, and the processed data are spliced ​​to form an input vector for subsequent convolutional neural network threat detection. The system sets a fixed time period T = 10 seconds to process and analyze the real-time data within the past 10 seconds every 10 seconds. Video data is collected by a high-definition CCD camera; environmental data is collected by temperature and humidity sensors, air quality sensors, and sound sensors. In a 10-second time period, the camera captures 30 frames of video images, and the environmental sensor collects data once a second, collecting a total of 10 temperature, humidity, air quality, and sound data points.

[0053] Frame extraction: 3 frames of video images are extracted per second in the time period T, and a total of 30 frames are extracted. Denoising: Gaussian filtering is applied to each frame of video images to reduce noise interference and make the image clearer for subsequent feature extraction. Size normalization: Each frame of image is uniformly adjusted to 224×224 pixels to meet the size requirements of convolutional neural network input. Feature extraction: A pre-trained convolutional neural network (such as ResNet) is used to extract feature vectors for each frame of image, and the features of each frame of image are compressed into a feature vector with a length of 512. Frame feature averaging: The feature vectors of 30 frames of images are averaged to obtain the overall features of the video data in the time period T, forming a feature vector V with a length of 512. Environmental data normalization: In the time period T, temperature, humidity, air quality and sound data are recorded every second, for a total of 10 data points. Each environmental data is normalized according to the following formula, and the value is mapped to the [0,1] interval: normalized_value=max-min, where min and max are the minimum and maximum values ​​of each sensor under normal conditions, respectively. For example, the range of temperature is [10°C, 40°C], and the range of humidity is [20%, 90%]. Each normalized data point is averaged to obtain the average temperature, humidity, air quality and sound characteristics in the time period T, forming an environmental feature vector E, whose length is 4. The video data feature vector V and the environmental data feature vector E are concatenated to form a complete input vector I = [V, E], where the length of V is 512 and the length of E is 4, so the total length of the concatenated input vector is 516. The concatenated input vector is input into the convolutional neural network for threat level detection and response in the subsequent steps.

[0054] By denoising, resizing and extracting features from video data, the system can obtain consistent feature representations when the video quality changes. At the same time, the normalization and feature aggregation of environmental data enable the system to have high adaptability and stability when processing multimodal data. Finally, the input vector generated by splicing the video feature vector and the environmental feature vector can be effectively input into the convolutional neural network to achieve comprehensive security monitoring of the shopping mall. This embodiment demonstrates effective data preprocessing and fusion methods in complex monitoring environments, providing data support for accurate identification and response of intelligent monitoring systems.

[0055] S3: Calculate the change amplitudes β1 and β2 of the preprocessed video data and the normalized environmental data within the time period T respectively; apply the present invention to the security monitoring system of the industrial park to calculate the change amplitudes β1 and β2 of the preprocessed video data and the normalized environmental data within the time period T. By identifying the degree of change of the video and environmental data, the system can better judge potential abnormal behaviors and environmental risks, and provide support for subsequent threat assessment and emergency response. The monitoring system sets a time period T = 15 seconds to evaluate the changes in the video data and environmental data within the past 15 seconds every 15 seconds. Within the time period T, the system collects several continuous video frames and environmental data points for calculating the change amplitude. Calculation of the video data change amplitude β1; Frame difference calculation: within the time period T, 2 frames of video images are extracted per second, and a total of 30 frames of video images are extracted.

[0056] Compare adjacent frames within time period T one by one, and calculate the grayscale difference of each pair of adjacent frames at the same pixel position to capture dynamic changes in the video. This difference can help identify the movement of people or machines, sudden objects entering the monitoring screen, etc. Average the grayscale difference of each pair of adjacent frames to obtain the overall difference between frames. Then, calculate the average grayscale difference of all adjacent frames to obtain the video change amplitude β1 within the time period. A large average grayscale difference indicates that there may be fast-moving objects or significant dynamic changes in the video, such as workers running or vehicles passing by quickly. Calculation of the change amplitude b of environmental data; During time period T, temperature, humidity, air quality and noise data are recorded once per second, so a total of 15 data points are collected. Compare the continuous environmental data within time period T and calculate the difference between adjacent data points. These differences can reflect the degree of fluctuation of environmental conditions. For example, temperature changes may reflect the overheating of equipment, and a sharp increase in humidity may indicate the risk of water leakage. Take the absolute value of the difference between adjacent data points of each type of environmental data and average them to obtain the changes in temperature, humidity, air quality and noise. Finally, the average change values ​​of these environmental characteristics are combined to obtain the overall change amplitude β2 of the environmental data.

[0057] If the change in the video data, β1, is higher than the preset threshold, the system will determine that there may be abnormal activities in the video, such as disputes between workers, accidents, or abnormal operation of the machine. Such abnormal situations are usually accompanied by significant movement, and the difference in video frames will increase significantly. If the change in environmental data, b, exceeds the threshold, it means that an abnormality may occur in the industrial park environment. For example, a rapid increase in temperature may indicate equipment failure, and a sharp increase in noise may indicate an explosion or other accident. Based on the size of β1 and β2, the system will select a suitable convolutional neural network for subsequent threat assessment. For example, when the video changes greatly and the environment changes little, the system may pay more attention to the behavior of people in the video; when the environment changes greatly and the video changes little, the system will focus on environmental anomaly detection.

[0058] Through this embodiment, the industrial park monitoring system can make a comprehensive assessment based on the degree of change of video and environmental data, effectively improving the system's anomaly detection capability. The video data change amplitude β1 helps the system identify the dynamic activities of people and objects, while the environmental data change amplitude β2 provides effective information about industrial environment fluctuations. Combining the change amplitudes of the two, the system can quickly identify and respond to potential security threats, providing reliable support for the security management of industrial parks.

[0059] S4: When β1 is greater than the set threshold β T1 When β2 is less than the set threshold β T2 When , the input vector is input into the trained first convolutional neural network, and the first convolutional neural network outputs the danger level value;

[0060] When β2 is greater than the set threshold β T2 When β1 is less than the set threshold β T1 When , the input vector is input into the trained second convolutional neural network, and the second convolutional neural network outputs the danger level value;

[0061] When β2 is greater than the set threshold β T2 When β1 is greater than the set threshold β T1 When , the input vector is input into the trained third convolutional neural network, and the third convolutional neural network outputs the danger level value;

[0062] When β1 is less than the set threshold β T1 When β2 is less than the set threshold β T2 When , return to step S1;

[0063] In one embodiment, the intelligent security monitoring system of an industrial warehouse can intelligently select different convolutional neural networks to analyze the degree of danger by evaluating the change amplitude of video data, β1, and the change amplitude of environmental data, β2, in real time. The system can intelligently select different convolutional neural networks to analyze the degree of danger and issue an alarm when potential danger occurs. High-definition cameras and sensors (including temperature, humidity, noise, and vibration sensors) are installed in the industrial warehouse. The system sets the monitoring time period T to 15 seconds. Every 15 seconds, the camera collects video data and the sensor collects environmental data. After preprocessing, an input vector is generated as the input of the convolutional neural network. Change amplitude calculation, video data change amplitude β1: Calculate the average grayscale difference of the video frame within the time period T to detect whether there are significant changes such as personnel activities and equipment movement. Environmental data change amplitude β2: Calculate the change amplitude of temperature, humidity, and noise within the time period T to detect whether there are abnormal environmental changes. For example, a rapid increase in temperature may indicate a fire risk, and a sharp increase in noise may indicate a failure of the equipment. Danger level assessment for different situations, based on the change amplitudes β1 and β2 of video data and environmental data, the system makes different judgments and triggers the corresponding convolutional neural network to assess the degree of danger:

[0064] Case 1: The magnitude of the video change, β1, is greater than the set threshold, and the magnitude of the environmental change, β2, is less than the set threshold. There is a large dynamic activity in the warehouse, such as people running or abnormal movement of equipment, but the environmental data does not show a significant change. The system inputs the input vector into the first trained convolutional neural network, which specializes in processing significant changes in the video, detects abnormal activities in the video and outputs a danger level value. The system detects that someone is running fast or an object is moving significantly, which may indicate an emergency evacuation of personnel or a problem with the equipment. The network outputs a high danger level value, and the system issues a primary alarm to prompt management personnel to pay attention.

[0065] Case 2: The environmental change amplitude, β2, is greater than the set threshold, and the video change amplitude, β1, is less than the set threshold. The environmental data in the warehouse changes significantly, such as a sudden increase in temperature, humidity, or noise, but there is no obvious human or object activity in the video. The system inputs the input vector into the trained second convolutional neural network, which specializes in handling abnormal environmental data, detects abnormal fluctuations in the environment, and outputs a danger level value. The system detects a sharp rise in temperature, which may mean overheating of warehouse equipment or potential fire risk. The network outputs a high danger level value, and the system immediately issues an alarm to remind managers to check the equipment or take emergency measures.

[0066] Case 3: The video change amplitude β1 and the environmental change amplitude β2 are both greater than the set threshold. The video and environmental data show significant changes at the same time, such as people moving quickly in the warehouse, and the temperature rises or the noise increases. The system inputs the input vector into the trained third convolutional neural network, which is suitable for processing situations where the video and environmental data fluctuate significantly at the same time, analyzes the comprehensive abnormal information and outputs the danger level value. The system detects that people leave quickly, accompanied by temperature rise or noise from the equipment, which may indicate a fire, explosion or other emergency. The network outputs an extremely high danger level value, the system issues an emergency alarm, and notifies the management staff to take immediate evacuation or emergency measures.

[0067] Case 4: The video change amplitude, β1, and the environmental change amplitude, β2, are both less than the set threshold. Neither the video nor the environmental data show significant changes, indicating that the activities and environment in the monitored area are relatively stable. The system determines that there is no abnormal situation at present and returns to the data collection step S1 for the next round of monitoring. In each case, the convolutional neural network will output a danger level value, indicating the potential threat level of the current monitored area. Alarm triggering: When the danger level value reaches the set threshold, the system sends the value to the manager's terminal for display and issues an alarm of the corresponding level. Low danger level: The terminal displays a "Caution" prompt to remind the manager to check. Medium danger level: The terminal displays a "Warning" prompt and triggers an alarm sound effect. High danger level: The terminal displays an "Emergency" prompt, the system sounds an alarm and issues an emergency broadcast to notify all personnel to evacuate. Through this embodiment, the system can effectively monitor and analyze the changes in video and environmental data in industrial warehouses and automatically determine potential security threats. When the video data or environmental data changes significantly, the system can adaptively select a suitable convolutional neural network for risk assessment, effectively reducing the possibility of false alarms and missed alarms, and providing real-time and accurate decision support for managers.

[0068] The first convolutional neural network (mainly monitors video anomalies), when the change amplitude of the video data, β1, is greater than the set threshold, and the change amplitude of the environmental data, β2, is less than the set threshold, it means that there may be significant dynamic changes in the video (such as a person suddenly entering the data center), while the environmental data changes less. Input layer: accepts the preprocessed video feature vector. Convolution layer: includes multiple convolution layers, each layer uses a larger convolution kernel (such as 7x7) to capture global changes in the video screen. Larger convolution kernels help to identify large-scale motion or new objects entering the scene. Pooling layer: After each convolution layer, a pooling layer is connected to further extract global features, reduce the impact of noise, and compress features by downsampling. Fully connected layer: Integrate the features output by the convolution layer into the fully connected layer to generate a comprehensive feature vector suitable for further analysis. Output layer: Output a score representing the degree of danger, ranging from low to high. The first convolutional neural network focuses on capturing dynamic changes in the video screen and detecting fast-moving or newly appearing objects. The large convolution kernels in the convolution layer can effectively extract large-scale motion information, such as fast movement of people or the appearance of new objects. The network passes the output of the convolutional layer to the fully connected layer for feature aggregation to form the final danger level score. The danger level value output by the network is generated by the output layer and is evaluated based on the scale, speed, and area of ​​movement detected in the features. If a person is detected entering or moving abnormally fast, the network will output a higher danger level score to prompt the data center security manager to pay attention.

[0069] The second convolutional neural network (mainly monitors environmental data anomalies), when the change amplitude of environmental data, β2, is greater than the set threshold, and the change amplitude of video data, β1, is less than the set threshold, it indicates that the environmental parameters (such as temperature, humidity or air quality) have changed significantly, while there is no significant dynamic change in the video. Input layer: accepts the feature vector of preprocessed environmental data. Convolution layer: composed of multiple convolution layers, the convolution kernel size is small (such as 3x3), which helps to extract the detailed features of environmental data and is suitable for detecting small-scale environmental changes. Pooling layer: The pooling layer is used to reduce the dimension and denoise the features to ensure the stability and consistency of environmental data. Recurrent layer (LSTM layer): The recurrent layer (such as LSTM layer) is added to analyze the time series characteristics of environmental data and capture the fluctuation patterns of temperature, humidity, etc. in a short time. Fully connected layer and output layer: The fully connected layer is used to further integrate the features of the convolution layer and the LSTM layer, and the output layer gives the final danger level score. The second convolutional neural network focuses on analyzing changes in environmental data, especially short-term fluctuations in parameters such as temperature, humidity and air quality. The convolutional layer extracts environmental features, and the LSTM layer further analyzes the change pattern of the time series. The LSTM layer can identify abnormal fluctuations in environmental data over time, helping the network to more accurately assess the risk level of the data center environment. The output layer generates a danger level value based on the feature changes. For example, if the temperature rises sharply in a short period of time, the network may output a higher danger level value to alert data center managers that there is a risk of overheating and that they need to check the operation of the air conditioner or equipment in time.

[0070] The third convolutional neural network (comprehensive monitoring of dual anomalies of video and environment), when the change amplitude of both video data and environmental data is greater than the set threshold, it means that both video and environmental data have changed significantly (such as rapid activity of personnel accompanied by temperature increase or noise increase), and there may be a major risk. Input layer: accepts the spliced ​​multimodal input vector, including the comprehensive features of video and environmental data. Multiscale convolution layer: used to capture global and detailed features in video and environmental data at the same time. Multiscale convolution can process inputs of different modes in the same layer. Attention mechanism layer: introduces the attention mechanism layer to highlight features with a large degree of abnormality in a weighted manner, helping the network to focus on key data more accurately. Multimodal fusion layer: fuses video and environmental features to comprehensively analyze dual anomalies in the data. Fully connected layer and output layer: integrate multimodal features through the fully connected layer, and the output layer gives the final danger level score. The third convolutional neural network realizes comprehensive analysis of video and environmental data through the design of multiscale convolution layer and multimodal fusion layer. The network can extract dual-mode features through different convolution kernels, and focus on danger signals through the attention mechanism, thereby improving the system's ability to identify major risks. The multimodal fusion layer can integrate the abnormal conditions of video and environmental data, so that the network can have a more acute judgment on the dual-change scenes. The output layer generates a comprehensive danger level score based on the feature changes after multimodal fusion. For situations where personnel activities are accompanied by environmental abnormalities (such as personnel running and temperature rise), the network will output an extremely high danger level value, prompting the system to immediately issue an emergency alarm and take emergency measures. Through the different structural designs and danger level calculation methods of the first, second, and third convolutional neural networks in this embodiment, the security monitoring system of the data center can select the most appropriate network structure to process the input data according to the changes in the video and environment. When the system detects a potential security threat, it will generate a corresponding danger level value and trigger an alarm when necessary, providing managers with clear risk information and response guidance, thereby effectively ensuring the safety and operational stability of the data center.

[0071] S5: Send the danger level value output by the convolutional neural network to the terminal of the manager for display and sound an alarm. Security monitoring system of the data center. In one embodiment, after the convolutional neural network calculates the danger level value, the system will send the value to the terminal of the manager for display, and trigger different levels of alarms according to the danger level value, so that the manager can quickly judge and take corresponding actions. Transmission and display of danger level value, the convolutional neural network (first, second or third network) generates a danger level value based on the input video and environmental data, and the value range is from 0 to 100, indicating a risk level from low to high. After the danger level value is generated, the system transmits the value to the terminal of the manager of the data center. The terminal displays the current danger level in real time, and assists in displaying it through color or icon to help managers intuitively understand the risk status. Green: The danger level value is 0-30, indicating a low risk, and the terminal displays a green icon, indicating that the environment and personnel activities are basically normal. Yellow: The danger level value is 31-60, indicating a medium risk, and the terminal displays a yellow icon, reminding managers to pay close attention to possible abnormal situations. Red: The danger level value is 61-100, indicating a high risk. The terminal displays a red icon and triggers a high-level alarm to remind managers to take emergency measures. The system sets different levels of alarms according to the danger level value to ensure that the system takes appropriate responses to different risk levels. The alarm system includes visual and sound prompts to help managers detect in time. Low-level alarm (0-30): When the danger level value is between 0-30, the system determines that the current risk is low. The terminal displays a green status and does not trigger an alarm. It only records the status for subsequent review. Intermediate alarm (31-60): When the danger level value is between 31-60, the system determines that there is a certain risk, but it has not reached a serious level. The terminal switches to yellow status, displays the "Attention" prompt, and plays a warning tone at a medium volume. The warning tone sounds every 2 minutes to remind managers to pay close attention to the current situation. Managers can judge whether they need to check the monitoring area immediately based on the frequency of the prompt tone. High-level alarm (61-100): When the danger level value is between 61-100, the system determines that there is a high risk and emergency measures may need to be taken immediately. The terminal switches to red status, displays the "Emergency" prompt, and the system starts playing a high-volume alarm sound, while flashing a red icon on the screen to indicate the current emergency. The system sends real-time SMS or application notifications to all managers to ensure that the responsible personnel are quickly aware of the risk. At the same time, the system automatically starts the broadcast system to remind all personnel in the data center to prepare for evacuation. In the case of a high-level alarm, the system also records the change history of the danger level value for analysis and review after the incident. Low-level and medium-level alarms: Managers can make manual judgments based on the danger level value and alarm prompts, such as selecting the "Observe" or "Check" options on the terminal to record the status.When "Check" is selected, the system records the inspection time and situation for subsequent analysis. Advanced alarm: When an advanced alarm is triggered, the manager can select the "Confirm Alarm" or "Remove Alarm" option on the terminal. When confirming the alarm, the system automatically records the current alarm status and issues further evacuation instructions; when resolving the alarm, the system needs to record the reason and release time for subsequent analysis. The system archives all danger level values, alarm status and management personnel's operation records to form a safety monitoring log. During daily inspections and after accidents, managers can view the changes in danger level values ​​and alarm triggering times recorded by the system to analyze the cause of the incident, the timeliness of the system response, and whether there are false alarms or missed alarms.

[0072] In some embodiments, the camera is a CCD camera, and the environmental data includes the ambient temperature collected by a temperature sensor, the air humidity measured by a humidity sensor, the sound data in the environment collected by a microphone, the sensor measuring the chemical composition in the air, and an accelerometer or vibration sensor installed to detect ground vibration or movement of the device itself. .Application of CCD camera

[0073] In some embodiments, CCD cameras are installed at high locations in the main passages, entrances and exits of the warehouse and various areas of the warehouse to achieve large-scale video surveillance. CCD cameras are used to capture the activities of personnel, equipment operation and material storage conditions in the warehouse in real time. CCD cameras have high sensitivity and high resolution, and can clearly record the situation in the warehouse, which helps to identify abnormal behavior (such as unauthorized entry or accidental movement of objects). CCD cameras are suitable for working in low-light conditions, so they can still provide clear monitoring images in the warehouse at night or in low-light environments. The video data captured by the camera can be transmitted to the system for analysis to see if there is any abnormal behavior.

[0074] Temperature sensors are installed in multiple areas of the warehouse, especially near equipment that may generate heat (such as electrical equipment, heaters, etc.) and flammable items. They are used to monitor temperature changes in the warehouse so that potential fire risks or equipment overheating problems can be detected in a timely manner. The temperature sensor sends real-time temperature data to the monitoring system. When the temperature sensor detects an abnormal increase in temperature at a certain location in the warehouse (such as exceeding the set safety threshold), the system will trigger an alarm prompt to remind managers to check the equipment immediately to prevent potential fires or equipment failures.

[0075] Humidity sensors are placed in different areas of the warehouse, especially where materials affected by humidity (such as electronic equipment, paper products or food) are stored. They are used to monitor the air humidity in the warehouse in real time to prevent damage to materials or moisture in equipment due to excessive humidity. Humidity data can help managers take measures (such as dehumidification) to maintain a proper storage environment. When the humidity sensor detects that the air humidity is higher than the safe range (such as more than 80%), the system will send a "high humidity" alarm, prompting managers to start dehumidification equipment or check for leaks to prevent materials from being damaged by moisture.

[0076] Microphones are installed in various corners of the warehouse to capture sounds in the environment. They are used to monitor abnormal sounds in the warehouse, such as objects falling, abnormal equipment operation sounds, people shouting, etc. When the system detects abnormal high or sudden sounds, it can be determined that an accident may have occurred in the warehouse. Microphones can help identify sudden noise events such as equipment failures or accidents (for example, the sound of goods falling or objects colliding). When the sound sensor detects a sudden high sound value, the system will sound a prompt and record the abnormal event for further inspection by management personnel.

[0077] Gas sensors are installed in key areas of the warehouse, especially those where chemicals or volatile substances are stored. They are used to detect chemical components in the air, including harmful gases (such as carbon monoxide, carbon dioxide, volatile organic compounds, etc.), so as to detect possible leaks or air pollution problems in a timely manner. If the sensor detects that the concentration of harmful gases in the air exceeds the standard (such as high carbon dioxide concentration or carbon monoxide is detected), the system will issue an "air pollution" alarm to remind managers to take ventilation measures immediately to prevent personnel from being affected by the deterioration of air quality.

[0078] Vibration sensors and accelerometers are installed on warehouse floors, walls or equipment surfaces, especially where large machinery and goods are piled up. They are used to monitor vibration conditions in the warehouse and help identify abnormal vibration or impact events. The sensor can detect ground vibrations or abnormal shaking of equipment, prompting the system to identify possible earthquakes, equipment failures or impacts. If the vibration sensor detects abnormal vibration or impact (such as exceeding the set threshold), the system will issue a "vibration abnormality" alarm to remind managers to check the status of the equipment or consider whether to activate the emergency plan.

[0079] In some embodiments, preprocessing the video image includes denoising the video image using Gaussian filtering; splicing the processed video data and environmental data to form an input vector includes extracting feature vectors from the video data through a pre-trained convolutional neural network, and directly using the environmental data as feature vectors; numerically splicing the feature vectors of the video data and the feature vectors of the environmental data end-to-end to form a complete input vector. Video image preprocessing: The camera in the warehouse collects video data and extracts multiple frames of images per second for preprocessing. Since there may be low light, dust or interference in the warehouse environment, directly processing the image data may cause noise to affect the accuracy of the analysis. Gaussian filtering denoising: When processing the video frame, the system uses a Gaussian filtering algorithm to remove noise from the image. Gaussian filtering can effectively smooth the image and reduce high-frequency noise while retaining the overall outline and main information of the image. After denoising, the image quality is improved and suitable for subsequent feature extraction. Image size standardization: The denoised image is uniformly adjusted to a standard size of 224×224 pixels to meet the input requirements of the pre-trained convolutional neural network. The system inputs the denoised and normalized images into a pre-trained convolutional neural network, such as a pre-trained ResNet-50 or VGG-16 model, which has strong feature extraction capabilities on large-scale image datasets. The pre-trained convolutional neural network extracts high-level features from the image, mainly including information such as the outline of people, the shape of objects, and the direction of movement. These features can characterize the activities in the warehouse, such as the presence of people, the movement of goods, etc. The extracted features are compressed into a fixed-length vector, such as a feature vector of length 512, for subsequent data processing and splicing. Sensors in the warehouse (such as temperature, humidity, noise, and gas sensors) collect environmental data in real time, and these data are standardized and directly used as feature vectors. For example, the temperature is 25°C, the humidity is 60%, and the noise level is 50dB, which constitutes a 4-dimensional feature vector. Since environmental data usually does not require complex feature extraction operations, the system directly uses the standardized data of each sensor for subsequent splicing processing. The system concatenates the feature vector of the video data (e.g., length 512) with the feature vector of the environmental data (e.g., length 4) end-to-end to form a complete input vector. The video feature vector contains image information of personnel activities and object movement in the warehouse, while the environmental feature vector contains current environmental information such as temperature, humidity, and noise. Through direct concatenation, a complete input vector of 516 dimensions is formed to represent the overall state of the warehouse within a time period, combining the multimodal features of visual data and environmental data. The concatenated input vector is formatted into an input data form suitable for convolutional neural network processing. The input vector contains video and environmental information, which makes it easier for the convolutional neural network to process information from different sources in the same data dimension to identify abnormal activities or potential threats in the warehouse.The concatenated input vector is input into the convolutional neural network, and the system outputs the danger level value based on the network analysis results. Since the input vector contains dual information of video and environment, the convolutional neural network can more accurately identify abnormal conditions in the warehouse environment, such as personnel activities, equipment vibrations, and changes in temperature and humidity in the environment. The convolutional neural network analyzes the danger level based on the input vector. If an abnormal situation is detected (such as someone moving quickly in a high temperature environment), the system will generate a high danger level value and trigger an alarm. The system can provide a more comprehensive safety assessment based on the combination of video and environmental data.

[0080] In some embodiments, calculating the change amplitude β1 of the preprocessed video data within the time period T includes: S31: first calculating the difference in pixel grayscale values ​​of pixels at corresponding positions between consecutive adjacent video frames within the time period T; S32: summing and averaging the pixel grayscale value differences of each pair of adjacent frames to obtain the average value of a single frame difference; S33: summing and averaging all the calculated average values ​​of the single frame differences to obtain the video change amplitude β1 within the entire time period T.

[0081] In some embodiments, calculating the change amplitude β2 of the normalized environmental data within the time period T includes: S34: calculating the difference between consecutive adjacent environmental data points within the time period T; S35: calculating the absolute value of the difference between each pair of adjacent data points; S36: calculating the average of these differences to obtain the environmental change amplitude β2 within the entire time period T. The intelligent monitoring system is installed in the warehouse, and helps to identify dynamic changes such as personnel activities or equipment movement in the warehouse by calculating the change amplitude of continuous video frames within the time period T. This change amplitude can be used to detect abnormal behavior or emergencies (such as rapid movement of personnel, sudden start of equipment, etc.). The monitoring system sets the time period T = 10 seconds, and collects and analyzes the changes in video data every 10 seconds to ensure timely detection of dynamic activities in the warehouse. Within the time period T, 3 frames of video images are extracted per second, and a total of 30 frames of video images are extracted. Step S31: Calculate the pixel gray value difference between adjacent frames. The system calculates the pixel gray value difference between adjacent frames frame by frame for the 30 frames of images extracted within the time period T. For each pair of adjacent frames, the system compares the pixel positions one by one and calculates the grayscale value difference of the same pixel in the adjacent frames. If the grayscale value of the pixel position in the first frame is 100 and in the second frame is 105, the grayscale difference of the pixel is 5. This calculation is used to capture subtle changes between video frames and help detect dynamic activities in the video, such as people entering, equipment moving, etc. Step S32: Calculate the average grayscale difference of each pair of adjacent frames, sum the grayscale differences of all pixels in each pair of adjacent frames, and then divide by the total number of pixels to obtain the average grayscale difference of the pair of adjacent frames. Through this average value, the system can obtain the overall change between adjacent frames, thereby reducing the impact of single pixel noise. For example, if the average grayscale difference between a pair of frames is 20, it means that the change between frames is obvious, and there may be a large range of movement or an object entering the field of view quickly. The system records the average grayscale difference of each pair of adjacent frames and generates a change data sequence to reflect the continuous change in the time period T. Step S33: Calculate the amplitude of the video change in the time period T. The average grayscale difference of all adjacent frames in time period T is summed up, and then the overall average is taken to obtain the total video change amplitude in time period T. This video change amplitude reflects the overall dynamic trend of the video frames in the time period. If the average grayscale difference value is large, it means that there may be significant dynamic activities in the warehouse; if the value is small, it means that the environment is relatively static. The warehouse management system can determine whether there are abnormal conditions based on the change amplitude, such as sudden human activities, equipment movement, or objects entering the field of view.

[0082] In some embodiments, the first convolutional neural network uses a 7*7 convolution kernel, and the activation function used is a Sigmoid activation function f1(x) expressed as follows:

[0083] The second convolutional neural network uses a 3*3 convolution kernel and the activation function used is the Sigmoid activation function f2(x) as follows:

[0084] The third convolutional neural network uses a 5*5 convolution kernel, and the activation function used is the Tanh activation function f3(x) expressed as follows: f3(x)=Tanh(x+δ*(β1+β2));

[0085] Among them, the activation function is applied to the hidden layer and output layer of the convolutional neural network, x represents the input value of the current neuron, e is the base of the natural logarithm; a is the proportional coefficient of the influence of the change amplitude of the video data on the activation function, β1 is the video change amplitude within the time period T; b is the proportional coefficient of the influence of the change amplitude of the environmental data on the activation function, β2 is the environmental change amplitude within the time period T; δ is the composite proportional coefficient of the influence of the change amplitude of the video and the change amplitude of the environmental data on the activation function.

[0086] In one embodiment, three different structures of convolutional neural networks in the intelligent warehouse monitoring system are demonstrated, including the convolution kernel size and the selection and application of activation functions used by each. According to the change range of the video data and environmental data in the warehouse collected by the system in real time, by adjusting the response coefficient of the activation function, each convolutional neural network can adapt to different scene requirements, so as to more accurately evaluate the safety of the warehouse.

[0087] The first convolutional neural network (7x7 convolution kernel + Sigmoid activation function) is used when the video data changes by a large amplitude β1, while the environmental data changes by a small amplitude β2, that is, when there are significant dynamic changes in the surveillance video (such as people moving quickly), but the environmental conditions are relatively stable. The convolution layer uses a 7x7 convolution kernel to capture the global features in the video with a large receptive field. This large-size convolution kernel is suitable for detecting dynamic activities over a large range in the scene, such as detecting the entry of people or the movement of objects. The Sigmoid activation function is used in the convolution layer and the output layer, and the output of the activation function is fine-tuned in combination with the amplitude of the video data change β1. By adjusting the sensitivity of the activation function, the network can be more sensitive to significant dynamic changes in the video. When the video change amplitude β1 increases, the output response of the activation function is also enhanced accordingly, highlighting the network's attention to dynamic features. For example, when someone enters the warehouse and moves quickly, the activation function responds more strongly to the input signal, and the network outputs a higher danger level value more quickly. The output of the Sigmoid function ranges between 0 and 1, which enables the network to effectively distinguish between normal activities (low output) and abnormal activities (high output), thereby providing accurate risk assessment.

[0088] The second convolutional neural network (3x3 convolution kernel + Sigmoid activation function) is suitable for the situation where the environmental data change amplitude β2 is large, while the video data change amplitude β1 is small, that is, when the environmental sensor detects significant environmental changes (such as drastic changes in temperature and humidity), while the video screen is basically static. The convolution layer uses a small 3x3 convolution kernel to focus on the detailed feature extraction of the environmental data to capture smaller environmental data fluctuations. This small-sized convolution kernel helps to enhance the network's sensitivity to small changes in environmental parameters. The Sigmoid activation function is used in the convolution layer and the output layer, and is dynamically adjusted according to the change amplitude β2 of the environmental data. In this way, the network can enhance its response when the environmental data fluctuates greatly, so that the system can quickly identify environmental anomalies in the warehouse. When the environmental change amplitude β2 is large, the output value of the activation function will increase due to the adjustment of the response coefficient, thereby more effectively highlighting the impact of the abnormal environment. For example, when the temperature suddenly rises or the humidity rises sharply, the output response of the activation function becomes larger, indicating that there is a potential abnormality in the environment. The Sigmoid function can help the system effectively process the output from 0 to 1 and convert the degree of environmental change into a dangerous degree value so that warehouse managers can quickly judge and take measures.

[0089] The third convolutional neural network (5x5 convolution kernel + Tanh activation function) is used when both the video data change amplitude β1 and the environmental data change amplitude β2 are large, that is, both the video and environmental data show significant changes, such as human activities in the warehouse accompanied by drastic changes in temperature and humidity. The convolution layer uses a medium-sized convolution kernel of 5x5, which can take into account the extraction of global and detailed features when the video data and environmental data change significantly. The 5x5 convolution kernel can adapt to the situation of dual changes and is suitable for handling complex warehouse monitoring scenarios. The Tanh activation function is used in the convolution layer and the output layer. The output range of Tanh is -1 to 1, which is more suitable for capturing complex multimodal features. By simultaneously adjusting the impact of the change amplitude of video and environmental data on the activation function, the network's response to significant dual changes is improved. When both the video and environmental data change significantly, the output response of the Tanh function increases, reflecting the network's sensitivity to dual changes. For example, when there are human activities in the warehouse and the temperature rises, the Tanh activation function responds strongly and generates a higher danger level value. The bidirectional output (-1 to 1) of the Tanh function makes it more effective in identifying complex abnormal patterns. For smaller changes, Tanh output is close to 0, and for significant abnormal changes, the output is close to 1, helping the system to more accurately assess dual risk scenarios.

[0090] By selecting and adjusting different network structures and activation functions, the system can adapt to different changing scenarios in a dynamic warehouse environment. The large convolution kernel and enhanced Sigmoid activation function of the first convolutional neural network can effectively monitor human activities when there are significant video changes in the warehouse. The small convolution kernel and slightly adjusted Sigmoid activation function of the second convolutional neural network help the system detect small fluctuations in the environment. The use of the Tanh activation function in the third convolutional neural network enables the system to more accurately identify and respond when both video and environmental data change dramatically.

[0091] Embodiment 2:

[0092] This application also provides a security monitoring system based on artificial intelligence, such as Figure 2 The system shown uses hardware devices such as cameras, sensors, edge computing devices, and cloud servers to achieve comprehensive monitoring and data processing of the warehouse. The camera uses a high-definition CCD camera with high resolution and low-light environment adaptability. It is installed at the main passages, entrances and exits, important material storage areas, and high places in the equipment area of ​​the warehouse to obtain a wide field of view and capture personnel activities and equipment operation in real time. It is used to collect real-time video data and transmit the data to the edge computing device for preprocessing (denoising, feature extraction, etc.), supporting subsequent video change detection and dynamic analysis.

[0093] Temperature sensors are used to monitor changes in temperature in the warehouse to prevent fire risks caused by overheating of equipment or high temperature outside. Humidity sensors are used to detect air humidity to prevent items in the warehouse from being damaged by excessive humidity. Gas sensors are used to monitor chemical components in the air, such as carbon dioxide, volatile organic compounds (VOCs), etc., to detect possible leaks or air pollution in the warehouse. Microphone (sound sensor): used to capture abnormal sounds (such as objects falling, abnormal noise from equipment, shouting) to help judge emergencies in the warehouse. Used to monitor the vibration of the ground or equipment in the warehouse to determine whether there is violent activity or equipment failure. Sensors are installed in different areas of the warehouse, such as around equipment and near materials that are easily affected by the environment. The sensors collect environmental data such as temperature, humidity, gas concentration, sound, vibration, etc. in real time, and send the data to the edge computing device so that the system can analyze environmental data.

[0094] Edge computing devices are high-performance edge computers with GPU or TPU support for processing large amounts of video and environmental data. They are responsible for processing data from cameras and sensors, including preprocessing, denoising, feature extraction, and normalization of environmental data. Edge computing devices analyze input vectors in real time by running convolutional neural network models to calculate the degree of danger. Cameras and sensors are connected to edge computing devices via wired or wireless networks (such as Ethernet or Wi-Fi) to ensure real-time data transmission and processing.

[0095] Cloud server is a high-performance cloud computing platform used to process and store large-scale historical data. Cloud servers are mainly used to store historical data of warehouse monitoring systems, including video data, environmental data, and hazard assessment results. Data analysis algorithms can be run on the cloud to help managers perform data backtracking, event analysis, and trend prediction. Edge computing devices are connected to cloud servers via the Internet, and data transmission uses encryption protocols to ensure data security. Edge computing devices regularly upload preliminarily processed data to cloud servers for archiving. Management terminals (monitoring centers) include monitoring computers, display screens, alarm systems, etc. They are used to receive and display hazard values ​​and alarm information sent by edge computing devices, so that managers can view the environmental status and safety of the warehouse in real time. The monitoring terminal displays a graphical view of the data, current alarm information, etc., to facilitate managers to respond quickly. The management terminal is connected to the edge computing device via a local area network or the Internet. When the system detects a risk, the edge computing device immediately sends the information to the management terminal, and the terminal can view the hazard value and alarm status in real time. The alarm device includes an audible and visual alarm, a broadcasting system, etc. When the system detects that the hazard value exceeds the set threshold, the alarm device triggers an alarm to remind the personnel in the warehouse to respond in time. For example, when the danger value reaches a high level, a high-pitched alarm and red flashing light will be activated, or an evacuation instruction will be broadcast through the broadcasting system. The alarm device is controlled by the management terminal. After detecting the danger, the edge computing device sends a signal to the management terminal, and the terminal triggers the alarm device through wired or wireless means.

[0096] The cameras and sensors are connected to the edge computing devices through a wired network (such as Ethernet) or a wireless network (such as Wi-Fi) so that the cameras and sensors can transmit the real-time collected data to the edge computing devices. The edge computing devices are responsible for preprocessing and real-time analysis of the data. The edge computing devices are connected to the cloud server through the Internet, using encrypted HTTP or VPN tunneling protocols to ensure the security of data transmission. The edge computing devices regularly upload the historical data and analysis results that have been preliminarily processed to the cloud server for data archiving and analysis. The local area network connection ensures real-time data transmission. The edge computing device sends the analysis results (such as the danger level value and abnormal conditions) to the management terminal, where the management personnel can view the data and alarm status in real time. The management terminal controls the alarm device through a wired or wireless connection. When a high degree of danger is detected, the terminal triggers the alarm device, including the sound and light alarm and the broadcasting system, to alert the on-site personnel to pay attention and evacuate.

[0097] Data collection: Cameras and sensors collect real-time video and environmental data in the warehouse and transmit them to edge computing devices. Data processing and analysis: Edge computing devices pre-process data, use convolutional neural networks to analyze input vectors, and generate danger level values. Alarms and reminders: When the danger level reaches the set threshold, the edge computing device transmits the information to the management terminal, which triggers the alarm device, issues an audible and visual alarm, and broadcasts reminders. Data storage and backtracking: Edge computing devices upload data to cloud servers, and managers can analyze warehouse safety trends through historical data and optimize management strategies.

[0098] Data acquisition module: collects real-time video data and environmental data through cameras and sensors installed at predetermined locations;

[0099] Preprocessing module: intercept the video data in the time period T before the current time point to perform video image preprocessing; intercept the environmental data in the time period T before the current time point to perform normalization processing; and splice the processed video data and environmental data to form an input vector;

[0100] A change amplitude calculation module: calculates the change amplitudes β1 and β2 of the preprocessed video data and the normalized environmental data within a time period T respectively;

[0101] Danger level output module: When β1 is greater than the set threshold β T1 When β2 is less than the set threshold β T2 When , the input vector is input into the trained first convolutional neural network, and the first convolutional neural network outputs the danger level value;

[0102] When β2 is greater than the set threshold β T2 When β1 is less than the set threshold β T1When , the input vector is input into the trained second convolutional neural network, and the second convolutional neural network outputs the danger level value;

[0103] When β2 is greater than the set threshold β T2 When β1 is greater than the set threshold β T1 When , the input vector is input into the trained third convolutional neural network, and the third convolutional neural network outputs the danger level value;

[0104] When β1 is less than the set threshold β T1 When β2 is less than the set threshold β T2 When , return to step S1;

[0105] Alarm reminder module: sends the danger level value output by the convolutional neural network to the management terminal for display and sounds the alarm.

[0106] In some embodiments, calculating the change amplitude β1 of the preprocessed video data within the time period T includes: S31: first calculating the difference in pixel grayscale values ​​of pixels at corresponding positions between consecutive adjacent video frames within the time period T; S32: summing and averaging the pixel grayscale value differences of each pair of adjacent frames to obtain the average value of a single frame difference; S33: summing and averaging all the calculated average values ​​of the single frame differences to obtain the video change amplitude β1 within the entire time period T.

[0107] In some embodiments, calculating the change amplitude β2 of the normalized environmental data within the time period T includes: S34: calculating the difference between consecutive adjacent environmental data points within the time period T; S35: calculating the absolute value of the difference between each pair of adjacent data points; S36: calculating the average value of these differences to obtain the environmental change amplitude β2 within the entire time period T.

[0108] Preferably, the first convolutional neural network uses a 7*7 convolution kernel, and the activation function used is a Sigmoid activation function f1(x) expressed as follows:

[0109] The second convolutional neural network uses a 3*3 convolution kernel and the activation function used is the Sigmoid activation function f2(x) as follows:

[0110] The third convolutional neural network uses a 5*5 convolution kernel, and the activation function used is the Tanh activation function f3(x) expressed as follows: f3(x)=Tanh(x+δ*(β1+β2));

[0111] Among them, x represents the input value of the current neuron, e is the base of the natural logarithm; a is the proportional coefficient of the influence of the change amplitude of the video data on the activation function, β1 is the video change amplitude within the time period T; b is the proportional coefficient of the influence of the change amplitude of the environmental data on the activation function, β2 is the environmental change amplitude within the time period T; δ is the composite proportional coefficient of the influence of the control video change and the environmental data change amplitude on the activation function.

[0112] The present invention provides a security monitoring method and system based on artificial intelligence, which can achieve the following beneficial technical effects:

[0113] 1. First, collect real-time video data and environmental data through cameras and sensors installed at predetermined locations; second, splice the preprocessed video data and environmental data to form an input vector; then, respectively calculate the change amplitudes β1 and β2 of the preprocessed video data and the normalized environmental data within the time period T; thirdly, use the first convolutional neural network or the second convolutional neural network or the third convolutional neural network to output the danger level value; finally, send the danger level value output by the convolutional neural network to the management terminal for display and sound an alarm. This application greatly improves the accuracy and efficiency of video processing and greatly increases the user experience by using different types of convolutional neural network algorithms for danger level calculation.

[0114] 2. The present invention dynamically adjusts the convolution kernel size according to the change amplitude of video data and environmental data, so that the network uses a larger convolution kernel to capture global context information when the data changes significantly, and uses a smaller convolution kernel to focus on detail features when the data changes slightly. Through this adaptive adjustment, the system can flexibly respond to various complex situations in different monitoring environments and improve the efficiency and accuracy of feature extraction.

[0115] 3. The present invention uses an adaptive activation function with added amplitude of change in different convolutional neural networks. The improved Sigmoid and Tanh activation functions with added amplitude of change factors are adopted, and the amplitude of change of video or environmental data is used as the adjustment parameter of the activation function. The network can be more sensitive to data sources with significant changes. This design enables the system to adjust the response output according to the actual environmental conditions, and greatly improves the accuracy of identifying abnormal behaviors and potential threats in dynamic environments. This solution utilizes the multimodal fusion of video data and environmental data, combines the visual information collected by the camera with environmental information such as temperature, humidity, sound, and vibration, and comprehensively constructs a multidimensional data model of the monitoring environment. Through this multimodal data analysis, the system can combine environmental changes on the basis of video monitoring, improve the comprehensiveness of threat assessment, reduce the underreporting rate and false alarm rate, and effectively identify potential safety hazards that may be overlooked.

[0116] The above is a detailed introduction to a security monitoring method and system based on artificial intelligence. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the core idea of ​​the present invention; at the same time, for general technical personnel in this field, according to the ideas and methods of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A security monitoring method based on artificial intelligence, characterized in that: Includes steps: S1: collects real-time video data and environmental data through cameras and sensors installed at predetermined locations; S2: intercepting the video data in the time period T before the current time point to perform video image preprocessing; intercepting the environmental data in the time period T before the current time point to perform normalization processing; and splicing the processed video data and environmental data to form an input vector; S3: Calculate the change amplitudes β1 and β2 of the preprocessed video data and the normalized environmental data in the time period T respectively; S4: When β1 is greater than the set threshold β T1 When β2 is less than the set threshold β T2 When , the input vector is input into the trained first convolutional neural network, and the first convolutional neural network outputs the danger level value; When β2 is greater than the set threshold β T2 When β1 is less than the set threshold β T1 When , the input vector is input into the trained second convolutional neural network, and the second convolutional neural network outputs the danger level value; When β2 is greater than the set threshold β T2 When β1 is greater than the set threshold β T1 When , the input vector is input into the trained third convolutional neural network, and the third convolutional neural network outputs the danger level value; When β1 is less than the set threshold β T1 When β2 is less than the set threshold β T2 When , return to step S1; S5: Send the danger level value output by the convolutional neural network to the management terminal for display and sound an alarm.

2. The artificial intelligence-based security monitoring method according to claim 1, characterized in that: The camera is a CCD camera, and the environmental data includes the ambient temperature collected by a temperature sensor, the air humidity measured by a humidity sensor, the sound data in the environment collected by a microphone, the sensor measuring the chemical composition in the air, and an accelerometer or vibration sensor installed to detect ground vibration or movement of the device itself.

3. The artificial intelligence-based security monitoring method according to claim 2, characterized in that: The preprocessing of the video image includes denoising the video image by using Gaussian filtering; splicing the processed video data and environmental data to form an input vector, including extracting feature vectors from the video data through a pre-trained convolutional neural network, and directly using the environmental data as the feature vector; numerically splicing the feature vectors of the video data and the feature vectors of the environmental data end-to-end to form a complete input vector.

4. The artificial intelligence-based security monitoring method according to claim 1, characterized in that: Calculating the change amplitude β1 of the preprocessed video data within the time period T includes: S31: first calculating the difference in pixel grayscale values ​​of pixels at corresponding positions between consecutive adjacent video frames within the time period T; S32: summing and averaging the pixel grayscale value differences of each pair of adjacent frames to obtain the average value of a single frame difference; S33: summing and averaging all the calculated average values ​​of the single frame differences to obtain the video change amplitude β1 within the entire time period T.

5. The artificial intelligence-based security monitoring method according to claim 1, characterized in that: Calculating the change amplitude β2 of the normalized environmental data within the time period T includes: S34: calculating the difference between consecutive adjacent environmental data points within the time period T; S35: calculating the absolute value of the difference between each pair of adjacent data points; S36: calculating the average value of these differences to obtain the environmental change amplitude β2 within the entire time period T.

6. The artificial intelligence-based security monitoring method according to claim 1, characterized in that: The first convolutional neural network uses a 7*7 convolution kernel and the activation function used is the Sigmoid activation function f1(x) expressed as follows: The second convolutional neural network uses a 3*3 convolution kernel and the activation function used is the Sigmoid activation function f2(x) as follows: The third convolutional neural network uses a 5*5 convolution kernel, and the activation function used is the Tanh activation function f3(x) expressed as follows: f3(x)=Tanh(x+δ*(β1+β2)); Among them, the activation function is applied to the hidden layer and output layer of the convolutional neural network, x represents the input value of the current neuron, e is the base of the natural logarithm; a is the proportional coefficient of the influence of the change amplitude of the video data on the activation function, β1 is the video change amplitude within the time period T; b is the proportional coefficient of the influence of the change amplitude of the environmental data on the activation function, β2 is the environmental change amplitude within the time period T; δ is the composite proportional coefficient of the influence of the change amplitude of the video and the change amplitude of the environmental data on the activation function.

7. A security monitoring system based on artificial intelligence, characterized in that: include: Data acquisition module: collects real-time video data and environmental data through cameras and sensors installed at predetermined locations; Preprocessing module: intercept the video data in the time period T before the current time point to perform video image preprocessing; intercept the environmental data in the time period T before the current time point to perform normalization processing; and splice the processed video data and environmental data to form an input vector; A change amplitude calculation module: calculates the change amplitudes β1 and β2 of the preprocessed video data and the normalized environmental data within a time period T respectively; Danger level output module: When β1 is greater than the set threshold β T1 When β2 is less than the set threshold β T2 When , the input vector is input into the trained first convolutional neural network, and the first convolutional neural network outputs the danger level value; When β2 is greater than the set threshold β T2 When β1 is less than the set threshold β T1 When , the input vector is input into the trained second convolutional neural network, and the second convolutional neural network outputs the danger level value; When β2 is greater than the set threshold β T2 When β1 is greater than the set threshold β T1 When , the input vector is input into the trained third convolutional neural network, and the third convolutional neural network outputs the danger level value; When β1 is less than the set threshold β T1 When β2 is less than the set threshold β T2 When , return to step S1; Alarm reminder module: sends the danger level value output by the convolutional neural network to the management terminal for display and sounds the alarm.

8. The artificial intelligence-based security monitoring system according to claim 7, characterized in that: Calculating the change amplitude β1 of the preprocessed video data within the time period T includes: S31: first calculating the difference in pixel grayscale values ​​of pixels at corresponding positions between consecutive adjacent video frames within the time period T; S32: summing and averaging the pixel grayscale value differences of each pair of adjacent frames to obtain the average value of a single frame difference; S33: summing and averaging all the calculated average values ​​of the single frame differences to obtain the video change amplitude β1 within the entire time period T.

9. The artificial intelligence-based security monitoring system according to claim 7, characterized in that: Calculating the change amplitude β2 of the normalized environmental data within the time period T includes: S34: calculating the difference between consecutive adjacent environmental data points within the time period T; S35: calculating the absolute value of the difference between each pair of adjacent data points; S36: calculating the average value of these differences to obtain the environmental change amplitude β2 within the entire time period T.

10. The artificial intelligence-based security monitoring system according to claim 7, characterized in that: The first convolutional neural network uses a 7*7 convolution kernel and the activation function used is the Sigmoid activation function f1(x) expressed as follows: The second convolutional neural network uses a 3*3 convolution kernel and the activation function used is the Sigmoid activation function f2(x) as follows: The third convolutional neural network uses a 5*5 convolution kernel, and the activation function used is the Tanh activation function f3(x) expressed as follows: f3(x)=Tanh(x+δ*(β1+β2)); Among them, x represents the input value of the current neuron, e is the base of the natural logarithm; a is the proportional coefficient of the influence of the change amplitude of the video data on the activation function, β1 is the video change amplitude within the time period T; b is the proportional coefficient of the influence of the change amplitude of the environmental data on the activation function, β2 is the environmental change amplitude within the time period T; δ is the composite proportional coefficient of the influence of the control video change and the environmental data change amplitude on the activation function.

Images