5G message video monitoring method and system based on artificial intelligence

Abstract

The invention discloses a 5G message video monitoring method based on artificial intelligence. The method comprises the following steps: S1, constructing a multi-modal data set; s2, multi-modal fusion features are generated by using a joint convolutional neural network; s3, generating a target behavior trend curve and a target behavior prediction result by using the 3D convolutional neural network; s4, generating a target threshold curve by using an adaptive algorithm based on variable domain search, and obtaining behavior abnormality warning information; s5, generating comprehensive alarm information by using a method of combining fuzzy logic reasoning and a Bayesian network; s6, cross-camera data space-time correlation reasoning is carried out by using the graph neural network; s7, constructing an event severity curve by using a multivariable regression analysis and analytic hierarchy process, and generating a severity level and an event response measure of the event; and S8, pushing alarm information and event response measures to related personnel or equipment through the 5G message platform. The invention provides an efficient video monitoring method in combination with artificial intelligence and a 5G network.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and 5G communication technology, and in particular to a 5G message video monitoring method and system based on artificial intelligence. Background Art

[0002] With the rapid development of information technology, video surveillance systems have been widely used in various fields, such as urban security, traffic management, industrial monitoring and public safety. However, existing video surveillance technologies have certain bottlenecks in processing and analyzing large-scale video data, especially in terms of real-time, accuracy and intelligence. Traditional video surveillance systems mainly rely on image acquisition devices and storage devices, combined with basic video image processing technology for event recognition and behavior analysis. However, these traditional monitoring technologies usually rely on manually set rules or simple pattern matching algorithms, lack intelligent event reasoning and judgment capabilities, and are difficult to meet the increasingly complex monitoring needs.

[0003] Existing video surveillance systems generally use rule-based alarm systems, which use pre-defined rules to generate alarms for certain behaviors or anomalies in the video stream. Most of these rules are based on simple threshold judgments and cannot flexibly adapt to various complex monitoring scenarios and abnormal behaviors in different environments. For example, static behaviors and dynamically changing events are often misjudged or missed, resulting in poor alarm accuracy and unnecessary trouble for users. Especially in complex environments, a single data source, such as video images, cannot fully reflect changes in the environment, and it is easy to miss key clues or events, affecting the response speed and effectiveness of the monitoring system.

[0004] In addition, existing video surveillance systems are often limited to the analysis of a single perspective and lack the ability to reason about the spatiotemporal correlation of data across cameras. In complex surveillance environments, image data from multiple cameras often present different perspectives and spatiotemporal distributions, and traditional systems cannot effectively integrate these data from different cameras, resulting in errors or omissions in the judgment and analysis of target behavior. For example, when a target travels through multiple surveillance areas, traditional systems may not be able to accurately track the target's movement trajectory, or fail to fully consider the correlation information from different perspectives when processing events, resulting in misjudgment or delayed response to events. Therefore, the existing technology is still insufficient for monitoring and analysis under multiple perspectives and scenarios, and cannot adapt to the variability and complexity of actual scenarios.

[0005] In addition, the anomaly detection mechanisms of existing monitoring systems usually rely on simple statistical methods or basic machine learning models, which often lack adaptability and flexibility. In particular, in terms of behavioral anomaly detection, there is a common problem with existing technologies, that is, the detection threshold cannot be dynamically adjusted according to changes in behavioral trends, which is prone to missed detections or false detections. For example, the behavior pattern of a target may change in different time periods, and traditional static threshold-based methods cannot reflect such changes in a timely manner, resulting in the system's inaccurate response to the target's behavior. Although some methods have attempted to introduce behavioral analysis based on deep learning, it is still difficult to achieve efficient anomaly detection due to the diversity and complexity of the data.

[0006] The importance of environmental data in video surveillance is also becoming increasingly prominent. Traditional video surveillance systems usually rely only on video data for target recognition and behavior analysis, ignoring the influencing factors of the external environment. For example, environmental sensor data is crucial for event judgment and response in certain scenarios. If these environmental factors are not taken into account, the monitoring system is often prone to misjudgment or delayed response. Therefore, the existing technology has poor integration in environmental perception and event analysis, lacks the ability to perform multi-dimensional fusion analysis of environmental sensor data and video data, and cannot perform accurate event judgment and abnormal alarm in complex environments.

[0007] In order to solve the above problems in the existing technologies, artificial intelligence technology, especially advanced algorithms such as deep learning and graph neural networks, has gradually become a research hotspot in the field of video surveillance in recent years. By combining deep learning models with the spatiotemporal characteristics of video data, the intelligence level of video surveillance systems has been significantly improved. In particular, algorithms such as convolutional neural networks, 3D convolutional neural networks, and long short-term memory networks have made certain breakthroughs in video target detection, behavior analysis, and anomaly detection. As an advanced technology that can process graph structured data, graph neural networks have demonstrated powerful spatiotemporal data reasoning capabilities in multiple fields, and can realize spatiotemporal data correlation analysis across cameras, providing new ideas for event correlation and reasoning in video surveillance systems.

[0008] However, although technologies such as deep learning and graph neural networks have made some progress in the field of video surveillance, existing technical solutions still have some limitations. First, deep learning models usually require a large amount of labeled data for training, which is a huge challenge for video surveillance systems. Because in practical applications, video data is often large and complex, and the cost of obtaining labeled data is high. Second, existing deep learning-based surveillance systems still face the problem of data fusion when processing multimodal data.

[0009] In addition, although adaptive algorithms and behavioral trend analysis can provide some support for behavioral anomaly detection, the current behavioral anomaly detection system still has certain deficiencies in dynamic threshold adjustment and accuracy in complex scenarios. Existing technologies often use simple weighted models or rule-based dynamic adjustment methods, which cannot accurately reflect the complexity and variability of target behaviors.

[0010] Therefore, how to provide a 5G message video monitoring method and system based on artificial intelligence is a problem that technical personnel in this field urgently need to solve. Summary of the invention

[0011] One purpose of the present invention is to propose a 5G message video monitoring method and system based on artificial intelligence. Based on artificial intelligence and 5G network technology, the present invention proposes an efficient 5G message video monitoring method, which can accurately detect abnormal behavior and generate severity assessment in real time through multimodal data fusion, target behavior prediction, environmental perception analysis, and cross-camera spatiotemporal correlation reasoning. The method of combining graph neural network with Bayesian network is used to optimize event correlation analysis, and the event severity curve is constructed through multivariate regression analysis and hierarchical analysis method. It has the advantages of accurate anomaly detection, rapid response, strong environmental perception ability, and efficient event processing, which significantly improves the intelligence level and real-time performance of video monitoring.

[0012] A 5G message video monitoring method based on artificial intelligence according to an embodiment of the present invention includes the following steps:

[0013] S1, transmit the camera’s video data and environmental sensor data in real time through the 5G network, compress and preprocess the transmitted data, and build a multimodal data set;

[0014] S2. performing joint convolutional neural network processing on the multimodal data set to generate multimodal fusion features;

[0015] S3. Based on the multimodal fusion features, a 3D convolutional neural network is used to analyze the behavior of the video target data to generate a target behavior trend curve and a target behavior prediction result;

[0016] S4, based on the generated target behavior trend curve and target behavior prediction results, using an adaptive algorithm based on variable domain search to generate a target threshold curve, dynamically adjust the behavior anomaly detection threshold, and obtain behavior anomaly warning information;

[0017] S5. Perform environmental perception analysis based on the acquired abnormal behavior warning information and environmental sensor data, use a method combining fuzzy logic reasoning and Bayesian network to input environmental variables into the model for comprehensive judgment, and generate comprehensive warning information of environmental abnormality;

[0018] S6. Based on the generated comprehensive abnormal alarm information, the graph neural network is used to perform spatiotemporal correlation reasoning across camera data to generate event correlation results;

[0019] S7. Based on the generated event correlation results, use multivariate regression analysis and analytic hierarchy process to construct event severity curves, and generate event severity levels and event response measures by evaluating event factors;

[0020] S8. Push alarm information and event response measures to relevant personnel or equipment through the 5G messaging platform for real-time event notification and emergency handling.

[0021] Optionally, the S3 specifically includes:

[0022] S31, performing time series segmentation on the generated multimodal fusion feature data, and dividing the multimodal fusion feature data into time series according to the time window T w Segment processing to form multiple time series segments

[0023] S32, time series segments Input to the 3D convolutional neural network, and extract the spatial-temporal feature matrix F of the video target behavior through the 3D convolution kernel st ;

[0024]

[0025] Among them, F st represents the space-time feature matrix, * represents the 3D convolution operation, f represents the activation function, W 3D represents the 3D convolution kernel weight matrix, b 3D Represents the 3D convolution kernel bias term;

[0026] S33, the generated space-time feature matrix F st Perform frame-by-frame analysis to extract the target's motion trajectory features, including position, velocity, and acceleration, and construct the basic elements of the target's behavior trend. The target's position information in each frame is P t , speed is V t , the acceleration is A t :

[0027]

[0028] S34. Based on the motion trajectory characteristics of each frame, the target behavior trend curve is fitted by combining multivariate regression and Fourier analysis:

[0029]

[0030] Among them, C behaviorrepresents the target behavior trend curve, β0 represents the bias term, β1, β2 and β3 represent the trajectory regression coefficients, γ k and δ k represents the Fourier coefficient, ω represents the frequency factor, and N represents the number of harmonic components;

[0031] S35, target behavior trend curve C behavior Perform time series forecasting, use long short-term memory networks to predict future behaviors, and generate target behavior prediction results.

[0032] Optionally, the S4 specifically includes:

[0033] S41, performing feature decomposition on the generated target behavior trend curve and target behavior prediction result, and extracting a behavior feature variable matrix, wherein the behavior feature variable matrix includes a behavior trend deviation, a behavior change rate, and a behavior abnormality frequency;

[0034] S42. Construct an initial threshold curve model based on the behavior characteristic variable matrix:

[0035]

[0036] Among them, T0(t) represents the initial threshold of behavior detection, exp represents the exponential function, α1, α2 and α3 represent the feature weights, λ1 represents the adjustment parameter of the feature, and F a1 Indicates the deviation of behavioral trend, F a2 represents the behavior change rate, F a3 Indicates the frequency of abnormal behavior, Represents the fluctuation term introduced by the time factor;

[0037] S43, introduce an adaptive optimization algorithm based on variable neighborhood search, take the initial threshold curve T0(tv as input, and dynamically optimize the target threshold curve through each iteration:

[0038]

[0039] Among them, T opt (t) represents the target threshold curve, δ i represents the domain impact factor, k represents the number of neighborhood adjustments, and N i (t) represents the i-th neighborhood adjustment value;

[0040] S44, based on the target threshold curve T generated by optimization opt (t), compare the target behavior trend curve point by point, and when the current value of the target behavior is greater than the target threshold curve value, the target behavior is judged to be abnormal;

[0041] S45, further clustering analysis is performed on the abnormal behavior points, using a density peak clustering algorithm, taking the local abnormal density maximum as the center point, to form multiple abnormal behavior clusters, and each cluster behavior is marked with a label;

[0042] S46. Generate abnormal behavior warning information based on the label information of the abnormal behavior cluster and the abnormal behavior trend, wherein the abnormal behavior warning information includes the abnormal behavior type, abnormal duration, and behavior deviation amplitude.

[0043] Optionally, the S5 specifically includes:

[0044] S51, using the abnormal behavior warning information and environmental sensor data as input to construct an input feature set for environmental perception analysis, wherein the environmental sensor data includes temperature and humidity, air pressure and smoke parameters, and the data input format is a multidimensional vector form, wherein each dimension represents an input variable, and the dimension value of each variable is dynamically updated according to real-time sensor feedback;

[0045] S52, fuzzifying the environmental sensor data, setting a membership function to perform fuzzy conversion on each environmental sensor value; the fuzzifying process includes mapping the data value of each sensor to a fuzzy set, the fuzzy set represents the degree of belonging through the membership function, the membership function adopts a triangular, trapezoidal or Gaussian function form, and outputting a fuzzy set to represent the fuzzy value of the sensor data;

[0046] S53, based on the fuzzy reasoning algorithm, a fuzzy rule base is constructed according to the fuzzification processing result and reasoning calculation is performed, wherein the fuzzy reasoning algorithm adopts a reasoning method based on the maximum membership principle, combined with fuzzy logic operation for reasoning, and outputs a fuzzy reasoning result to represent the possibility of each environmental variable;

[0047] S54, using the fuzzy reasoning result as an input node of a Bayesian network to construct a probability model of the Bayesian network, wherein the Bayesian network adopts a directed acyclic graph structure, wherein each node represents an environmental variable, and the edges between nodes represent the causal relationship between the environmental variables; the reasoning mechanism of the Bayesian network updates the posterior probability distribution of the node by calculating the conditional probability of each node;

[0048] S55. In the Bayesian network, the prior probability and conditional probability are updated online by using incremental learning. By updating the probability distribution of the network in real time, the relationship weights and prior probabilities between nodes are automatically corrected each time new abnormal behavior warning information and environmental sensor data are input;

[0049] S56. Generate comprehensive warning information by comprehensively analyzing the correlation between the abnormal behavior warning information and the environmental sensor data according to the inference result of the Bayesian network. The comprehensive warning information includes the abnormal environment state, the abnormal category and the probability of the abnormal occurrence.

[0050] Optionally, the S6 specifically includes:

[0051] S61, receiving the generated comprehensive abnormal alarm information, and constructing an input data structure of the graph neural network, wherein the input data structure includes a plurality of nodes, each node represents a monitoring point of a camera or a monitoring area, and the characteristics of the node include abnormal state information of the monitoring point, the time and the abnormal type;

[0052] S62. Construct the adjacency relationship of the graph neural network. Define the connection relationship between the monitoring points according to the spatiotemporal similarity between the monitoring points. The adjacency relationship of each monitoring point reflects the spatiotemporal dependency with the surrounding monitoring points, indicating the mutual influence of events in the time and space dimensions.

[0053] S63, aggregate the feature information of each monitoring point through the graph convolutional neural network, transfer the information of the neighboring nodes to the current node, and the transfer operation of each layer of the graph neural network integrates the spatiotemporal information between the nodes;

[0054] S64, through the multi-layer structure of the graph neural network, the spatiotemporal information between nodes is optimized and updated layer by layer, and the output result of each layer of the network is used to enhance the learning of the abnormal state of the monitoring point;

[0055] S65. Perform spatiotemporal event association reasoning through the output results of the graph neural network. Perform cross-camera data reasoning based on the aggregated and optimized node features to identify event associations between different cameras. The reasoning process reveals the changing trends of events in different spaces and times by associating data from different cameras, and generates correlation scores between events to obtain spatiotemporal association reasoning results.

[0056] S66. Generate a final event association result based on the obtained spatiotemporal association reasoning result, wherein the event association result includes the time, location, camera node, severity of the event, and propagation status in the entire monitoring area.

[0057] Optionally, the S7 specifically includes:

[0058] S71, identifying event-related factors according to the generated event association results, wherein the event-related factors include target behavior characteristics, environmental variables, and sensor data;

[0059] S72, performing standardization processing on the event-related factors, and mapping the event-related factors to a unified evaluation scope;

[0060] S73, using a multivariate regression analysis model, combined with the standardized results of the event-related factors, and generating an event severity curve C(t) by calculating a weighted coefficient, wherein C(t) represents the severity score of the event at time t;

[0061] S74. Based on the calculated event severity curve, use the analytic hierarchy process to rank the importance of event-related factors, further evaluate the contribution of event-related factors to event severity, and determine the final severity level L(C(t)):

[0062]

[0063] Among them, τ1 and τ2 represent thresholds, ζ1, ζ2 and ζ3 represent control coefficients, exp represents exponential function, λ1, λ2 and γ1 represent adjustment factors, and δ1 and δ2 represent exponential factors;

[0064] S75. Generate event response measures based on the evaluation result of the severity level L(C(t)) and in combination with a predetermined threshold setting rule.

[0065] A 5G message video monitoring system based on artificial intelligence according to an embodiment of the present invention includes the following modules:

[0066] A data receiving module is used to receive video data from cameras and environmental sensor data through a 5G network;

[0067] Feature extraction module for joint convolutional neural network processing of multimodal datasets;

[0068] The behavior analysis module is used to analyze the behavior of the target data in the video using a 3D convolutional neural network to generate a target behavior trend curve and target behavior prediction results;

[0069] A threshold optimization module is used to generate a target threshold curve using an adaptive algorithm based on variable domain search, dynamically adjust the behavior anomaly detection threshold, and obtain behavior anomaly warning information;

[0070] The environmental perception analysis module is used to combine fuzzy logic reasoning and Bayesian network methods to input environmental variables into the model for comprehensive judgment and generate comprehensive warning information of environmental abnormal conditions;

[0071] The event association reasoning module is used to use graph neural networks to perform spatiotemporal association reasoning across camera data and generate event association results;

[0072] The event severity assessment module is used to construct an event severity curve using multivariate regression analysis and analytic hierarchy process, and to generate event severity levels and event response measures by evaluating event factors;

[0073] The alarm information push module is used to push alarm information and event response measures to relevant personnel or equipment through the 5G message platform.

[0074] The beneficial effects of the present invention are:

[0075] First of all, the present invention significantly improves the analytical capabilities of the monitoring system by effectively fusing multimodal data. Traditional video surveillance systems usually rely only on video image data for target recognition and behavior analysis, ignoring the influence of environmental factors, resulting in low accuracy of the system in complex scenarios. By combining camera video data with environmental sensor data, the system can fully perceive changes in the monitoring environment and generate more accurate and comprehensive alarm information. This multimodal data fusion method greatly improves the adaptability and accuracy of the monitoring system in complex and dynamic environments, and can better meet multi-scenario, all-round security monitoring needs.

[0076] Secondly, the present invention introduces deep learning technology, especially 3D convolutional neural network and long short-term memory network, to efficiently analyze and predict target behavior. Compared with the traditional rule-based alarm system, the monitoring system based on artificial intelligence can dynamically extract features from a large amount of data, automatically learn the laws of target behavior, and accurately predict the trend of target behavior. Through in-depth analysis of the target's motion trajectory and behavior pattern, the system can generate trend curves and prediction results of target behavior in real time, providing an important basis for subsequent anomaly detection and event response. This intelligent processing method based on behavior prediction and trend analysis greatly improves the response speed and prediction ability of the monitoring system to emergencies.

[0077] Furthermore, the present invention uses an adaptive optimization algorithm based on variable neighborhood search to dynamically adjust the threshold of behavior anomaly detection, solving the problem that static thresholds in traditional systems cannot adapt to behavior changes. The prior art usually relies on static threshold settings in behavior anomaly detection, which results in the system being unable to respond in time or make accurate judgments when the target behavior changes. The present invention uses an adaptive algorithm to adjust the threshold according to the dynamic trend of the target behavior, which can accurately identify behavioral anomalies and reduce the probability of false alarms and missed reports. When processing complex behavior patterns, this method can dynamically optimize the detection threshold according to actual conditions, greatly improving the flexibility and accuracy of the system.

[0078] In addition, the present invention realizes the spatiotemporal correlation reasoning of cross-camera data by introducing graph neural networks, solving the problem of spatiotemporal data integration in multi-camera monitoring scenarios in the prior art. Traditional monitoring systems can usually only perform target detection and behavior analysis from a single perspective, and cannot effectively integrate data from multiple cameras, which can easily lead to omissions or misjudgments of target behaviors. Graph neural networks can realize cross-camera data reasoning and association by analyzing the spatiotemporal relationship between multiple cameras, thereby achieving more accurate target tracking and behavior prediction, allowing the present invention to accurately analyze and predict target behaviors in complex environments, and effectively reduce the data fragmentation problem caused by cross-perspective monitoring.

[0079] In terms of environmental perception, the present invention uses a combination of fuzzy logic reasoning and Bayesian networks to achieve a deep fusion of environmental sensor data and video data. By fuzzifying environmental data and using Bayesian networks for reasoning and online updating, the present invention can automatically adjust environmental status judgments according to changes in sensor data. Compared to traditional systems that rely solely on video images for behavioral analysis, the present invention can comprehensively consider the impact of environmental changes on target behavior, making the monitoring system more accurate in judging target behavior, especially when environmental factors such as fire, smoke, temperature and humidity may affect target behavior, and can provide more accurate abnormal status alarms. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0081] Figure 1 This is a flow chart of a 5G message video monitoring method based on artificial intelligence proposed by the present invention;

[0082] Figure 2 This is a structural diagram of an artificial intelligence-based 5G message video surveillance system proposed in the present invention. DETAILED DESCRIPTION

[0083] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0084] refer to Figure 1 , a 5G message video monitoring method based on artificial intelligence, comprising the following steps:

[0085] S1, transmit the camera’s video data and environmental sensor data in real time through the 5G network, compress and preprocess the transmitted data, and build a multimodal data set;

[0086] S2. performing joint convolutional neural network processing on the multimodal data set to generate multimodal fusion features;

[0087] S3. Based on the multimodal fusion features, a 3D convolutional neural network is used to analyze the behavior of the video target data to generate a target behavior trend curve and a target behavior prediction result;

[0088] S4, based on the generated target behavior trend curve and target behavior prediction results, using an adaptive algorithm based on variable domain search to generate a target threshold curve, dynamically adjust the behavior anomaly detection threshold, and obtain behavior anomaly warning information;

[0089] S5. Perform environmental perception analysis based on the acquired abnormal behavior warning information and environmental sensor data, use a method combining fuzzy logic reasoning and Bayesian network to input environmental variables into the model for comprehensive judgment, and generate comprehensive warning information of environmental abnormality;

[0090] S6. Based on the generated comprehensive abnormal alarm information, the graph neural network is used to perform spatiotemporal correlation reasoning across camera data to generate event correlation results;

[0091] S7. Based on the generated event correlation results, use multivariate regression analysis and analytic hierarchy process to construct event severity curves, and generate event severity levels and event response measures by evaluating event factors;

[0092] S8. Push alarm information and event response measures to relevant personnel or equipment through the 5G messaging platform for real-time event notification and emergency handling.

[0093] In this implementation, S3 specifically includes:

[0094] S31, performing time series segmentation on the generated multimodal fusion feature data, and dividing the multimodal fusion feature data into time series according to the time window T w Segment processing to form multiple time series segments

[0095] S32, time series segments Input to the 3D convolutional neural network, and extract the spatial-temporal feature matrix F of the video target behavior through the 3D convolution kernel st ;

[0096]

[0097] Among them, F st represents the space-time feature matrix, * represents the 3D convolution operation, f represents the activation function, W 3D represents the 3D convolution kernel weight matrix, b 3D Represents the 3D convolution kernel bias term;

[0098] S33, the generated space-time feature matrix F st Perform frame-by-frame analysis to extract the target's motion trajectory features, including position, velocity, and acceleration, and construct the basic elements of the target's behavior trend. The target's position information in each frame is P t , speed is V t , the acceleration is A t :

[0099]

[0100] S34. Based on the motion trajectory characteristics of each frame, the target behavior trend curve is fitted by combining multivariate regression and Fourier analysis:

[0101]

[0102] Among them, C behavior represents the target behavior trend curve, β0 represents the bias term, β1, β2 and β3 represent the trajectory regression coefficients, γ k and δ k represents the Fourier coefficient, ω represents the frequency factor, and N represents the number of harmonic components;

[0103] S35, target behavior trend curve C behavior Perform time series forecasting, use long short-term memory networks to predict future behaviors, and generate target behavior prediction results.

[0104] In this implementation manner, the S4 specifically includes:

[0105] S41, performing feature decomposition on the generated target behavior trend curve and target behavior prediction result, and extracting a behavior feature variable matrix, wherein the behavior feature variable matrix includes a behavior trend deviation, a behavior change rate, and a behavior abnormality frequency;

[0106] S42. Construct an initial threshold curve model based on the behavior characteristic variable matrix:

[0107]

[0108] Among them, T0(t) represents the initial threshold of behavior detection, exp represents the exponential function, α1, α2 and α3 represent the feature weights, λ1 represents the adjustment parameter of the feature, and F a1 Indicates the deviation of behavioral trend, F a2 represents the behavior change rate, F a3 Indicates the frequency of abnormal behavior, Represents the fluctuation term introduced by the time factor;

[0109] S43, introduce an adaptive optimization algorithm based on variable neighborhood search, take the initial threshold curve T0(t) as input, and dynamically optimize the target threshold curve through each iteration:

[0110]

[0111] Among them, T opt (t) represents the target threshold curve, δ i represents the domain impact factor, k represents the number of neighborhood adjustments, and N i (t) represents the i-th neighborhood adjustment value;

[0112] S44, based on the target threshold curve T generated by optimization opt (t), compare the target behavior trend curve point by point, and when the current value of the target behavior is greater than the target threshold curve value, the target behavior is judged to be abnormal;

[0113] S45, further clustering analysis is performed on the abnormal behavior points, using a density peak clustering algorithm, taking the local abnormal density maximum as the center point, to form multiple abnormal behavior clusters, and each cluster behavior is marked with a label;

[0114] S46. Generate abnormal behavior warning information based on the label information of the abnormal behavior cluster and the abnormal behavior trend, wherein the abnormal behavior warning information includes the abnormal behavior type, abnormal duration, and behavior deviation amplitude.

[0115] In this implementation manner, S5 specifically includes:

[0116] S51, using the abnormal behavior warning information and environmental sensor data as input to construct an input feature set for environmental perception analysis, wherein the environmental sensor data includes temperature and humidity, air pressure and smoke parameters, and the data input format is a multidimensional vector form, wherein each dimension represents an input variable, and the dimension value of each variable is dynamically updated according to real-time sensor feedback;

[0117] S52, fuzzifying the environmental sensor data, setting a membership function to perform fuzzy conversion on each environmental sensor value; the fuzzifying process includes mapping the data value of each sensor to a fuzzy set, the fuzzy set represents the degree of belonging through the membership function, the membership function adopts a triangular, trapezoidal or Gaussian function form, and outputting a fuzzy set to represent the fuzzy value of the sensor data;

[0118] S53, based on the fuzzy reasoning algorithm, a fuzzy rule base is constructed according to the fuzzification processing result and reasoning calculation is performed, wherein the fuzzy reasoning algorithm adopts a reasoning method based on the maximum membership principle, combined with fuzzy logic operation for reasoning, and outputs a fuzzy reasoning result to represent the possibility of each environmental variable;

[0119] S54, using the fuzzy reasoning result as an input node of a Bayesian network to construct a probability model of the Bayesian network, wherein the Bayesian network adopts a directed acyclic graph structure, wherein each node represents an environmental variable, and the edges between nodes represent the causal relationship between the environmental variables; the reasoning mechanism of the Bayesian network updates the posterior probability distribution of the node by calculating the conditional probability of each node;

[0120] S55. In the Bayesian network, the prior probability and conditional probability are updated online by using incremental learning. By updating the probability distribution of the network in real time, the relationship weights and prior probabilities between nodes are automatically corrected each time new abnormal behavior warning information and environmental sensor data are input;

[0121] S56. Generate comprehensive warning information by comprehensively analyzing the correlation between the abnormal behavior warning information and the environmental sensor data according to the inference result of the Bayesian network. The comprehensive warning information includes the abnormal environment state, the abnormal category and the probability of the abnormal occurrence.

[0122] In this implementation manner, S6 specifically includes:

[0123] S61, receiving the generated comprehensive abnormal alarm information, and constructing an input data structure of the graph neural network, wherein the input data structure includes a plurality of nodes, each node represents a monitoring point of a camera or a monitoring area, and the characteristics of the node include abnormal state information of the monitoring point, the time and the abnormal type;

[0124] S62. Construct the adjacency relationship of the graph neural network. Define the connection relationship between the monitoring points according to the spatiotemporal similarity between the monitoring points. The adjacency relationship of each monitoring point reflects the spatiotemporal dependency with the surrounding monitoring points, indicating the mutual influence of events in the time and space dimensions.

[0125] S63, aggregate the feature information of each monitoring point through the graph convolutional neural network, transfer the information of the neighboring nodes to the current node, and the transfer operation of each layer of the graph neural network integrates the spatiotemporal information between the nodes;

[0126] S64, through the multi-layer structure of the graph neural network, the spatiotemporal information between nodes is optimized and updated layer by layer, and the output result of each layer of the network is used to enhance the learning of the abnormal state of the monitoring point;

[0127] S65. Perform spatiotemporal event association reasoning through the output results of the graph neural network. Perform cross-camera data reasoning based on the aggregated and optimized node features to identify event associations between different cameras. The reasoning process reveals the changing trends of events in different spaces and times by associating data from different cameras, and generates correlation scores between events to obtain spatiotemporal association reasoning results.

[0128] S66. Generate a final event association result based on the obtained spatiotemporal association reasoning result, wherein the event association result includes the time, location, camera node, severity of the event, and propagation status in the entire monitoring area.

[0129] In this implementation manner, the S7 specifically includes:

[0130] S71, identifying event-related factors according to the generated event association results, wherein the event-related factors include target behavior characteristics, environmental variables, and sensor data;

[0131] S72, performing standardization processing on the event-related factors, and mapping the event-related factors to a unified evaluation scope;

[0132] S73, using a multivariate regression analysis model, combined with the standardized results of the event-related factors, and generating an event severity curve C(t) by calculating a weighted coefficient, wherein C(t) represents the severity score of the event at time t;

[0133] S74. Based on the calculated event severity curve, use the analytic hierarchy process to rank the importance of event-related factors, further evaluate the contribution of event-related factors to event severity, and determine the final severity level L(C(t)):

[0134]

[0135] Among them, τ1 and τ2 represent thresholds, ζ1, ζ2 and ζ3 represent control coefficients, exp represents exponential function, λ1, λ2 and γ1 represent adjustment factors, and δ1 and δ2 represent exponential factors;

[0136] S75. Generate event response measures based on the evaluation result of the severity level L(C(t)) and in combination with a predetermined threshold setting rule.

[0137] refer to Figure 2 , a 5G message video surveillance system based on artificial intelligence, including the following modules:

[0138] A data receiving module is used to receive video data from cameras and environmental sensor data through a 5G network;

[0139] Feature extraction module for joint convolutional neural network processing of multimodal datasets;

[0140] The behavior analysis module is used to analyze the behavior of the target data in the video using a 3D convolutional neural network to generate a target behavior trend curve and target behavior prediction results;

[0141] A threshold optimization module is used to generate a target threshold curve using an adaptive algorithm based on variable domain search, dynamically adjust the behavior anomaly detection threshold, and obtain behavior anomaly warning information;

[0142] The environmental perception analysis module is used to combine fuzzy logic reasoning and Bayesian network methods to input environmental variables into the model for comprehensive judgment and generate comprehensive warning information of environmental abnormal conditions;

[0143] The event association reasoning module is used to use graph neural networks to perform spatiotemporal association reasoning across camera data and generate event association results;

[0144] The event severity assessment module is used to construct an event severity curve using multivariate regression analysis and analytic hierarchy process, and to generate event severity levels and event response measures by evaluating event factors;

[0145] The alarm information push module is used to push alarm information and event response measures to relevant personnel or equipment through the 5G message platform.

[0146] Embodiment 1:

[0147] In order to verify the feasibility of the present invention in implementation, the present invention is applied to a large logistics park, which uses a traditional monitoring system, including cameras and environmental sensors, which are mainly used to monitor the behavior of people, vehicle traffic and the safety of the storage environment in the area. However, the traditional system has the following problems: first, the quality of the video monitoring picture is low, and the accuracy of the behavior analysis is insufficient, especially in a dynamically changing environment, which is prone to false alarms; second, the environmental sensor data and the video data are not effectively integrated, making it difficult to fully analyze the actual situation; finally, the monitoring system is slow to respond under high load conditions and cannot respond to abnormalities in a timely manner.

[0148] In order to solve these problems, the park decided to adopt an artificial intelligence-based 5G message video surveillance method. This method can efficiently detect and warn of safety hazards through joint analysis of multimodal data, behavioral trend prediction and dynamic threshold adjustment, greatly improving the intelligence level of the system.

[0149] In the park, the monitoring system's cameras are responsible for real-time video of people and vehicles in the area, and environmental sensors collect data such as temperature, humidity, air pressure, and smoke concentration in real time. Through the 5G network, these data are transmitted to the cloud platform for processing in real time. In the data receiving module, the camera video and sensor data are first compressed and preprocessed to meet the transmission speed and bandwidth requirements of the 5G network.

[0150] Subsequently, the data is fed into the feature extraction module, where a joint convolutional neural network is used to fuse the video data and environmental sensor data to generate multimodal fusion features. Through this fusion method, the system can simultaneously analyze the motion trajectory of objects in the video and the changes in environmental parameters, combining the spatiotemporal characteristics of the video to improve the accuracy of behavior analysis.

[0151] In the behavior analysis module, the 3D convolutional neural network conducts in-depth analysis of video targets to generate target behavior trend curves and behavior prediction results. For example, when a truck enters the warehouse area of ​​the park, the system can accurately identify and track its movement trajectory, combine time series data, predict the vehicle's behavior in the next few minutes, and identify potential safety risks in advance.

[0152] Based on the generated behavior prediction results, the threshold optimization module will automatically generate the target threshold curve using the variable domain search algorithm and dynamically adjust the behavior anomaly detection threshold. For example, when a truck stops, the system will predict whether it may have unconventional behavior (such as illegal parking or long-term stay) based on the vehicle's movement trajectory. If the current behavior exceeds the threshold, the system will issue a warning in time.

[0153] At the same time, the environmental perception analysis module conducts a comprehensive analysis of the real-time collected environmental sensor data and abnormal behavior warning information. If the temperature and humidity in a certain area of ​​the park change abnormally or the smoke concentration increases, the system will use fuzzy logic reasoning and Bayesian network to make a judgment, issue an environmental abnormality alarm, and combine it with the abnormal behavior warning to form a comprehensive safety warning.

[0154] After all data processing and analysis is completed, the event association reasoning module uses graph neural networks to perform spatiotemporal association reasoning and associate events across cameras to ensure that the system can promptly identify abnormal events that occur at multiple points at the same time. For example, if a vehicle stops abnormally at multiple monitoring points, the system will combine the information from each monitoring point to determine whether the behavior is a potential safety risk.

[0155] Finally, the system pushes the incident response measures and alarm information to relevant personnel or equipment in real time through the 5G messaging platform. The park's security personnel will receive detailed alarm information and handle it according to the severity of the incident and the response measures. The system can quickly identify and respond to emergencies, greatly improving the security management efficiency of the park. During the implementation process, the system showed significant improvements. The following is the data statistics after using the AI-based 5G messaging video surveillance method in the park from April 2024 to September 2024.

[0156] Table 1 Performance evaluation data of 5G message video surveillance system based on artificial intelligence

[0157]

[0158]

[0159] First, from an overall perspective, Table 1 shows the system's multiple performance indicators under different monitoring environments, camera types, and target behavior detection tasks, including data transmission delay, video analysis accuracy, behavior prediction accuracy, event response timeliness, and effective push rate of alarm information. These data reflect how the system copes with complex monitoring tasks in actual applications and provide a more detailed basis for performance evaluation.

[0160] As for the data transmission delay indicator, the table shows that the system's real-time data transmission capability performs well in different environments and different camera configurations. In indoor environments, the average transmission delay is 85 milliseconds, while in complex urban environments, the delay is 123 milliseconds. Although the urban environment is subject to more signal interference, resulting in a slight increase in delay, the entire system can still maintain a low transmission delay with the support of the 5G network, which is critical for real-time video surveillance and event response.

[0161] In terms of video analysis accuracy, the table shows that under different monitoring tasks, the system can accurately identify and analyze the target behavior in the video. By combining the feature extraction and behavior analysis of convolutional neural networks and 3D convolutional neural networks, the overall accuracy of video analysis remains above 92%, especially when identifying abnormal target behavior, the accuracy reaches above 95%. This data shows that the artificial intelligence algorithm based on multimodal data fusion has significantly improved the accuracy of video analysis, can effectively distinguish normal and abnormal behaviors, and provides a reliable foundation for subsequent threshold optimization and event response.

[0162] The indicator of behavior prediction accuracy also shows the system's high efficiency in predicting the target's future behavior. By using a long short-term memory network (LSTM) based on time series, the system can accurately predict the trend of the target's behavior in most scenarios with an accuracy of up to 93%. This means that in the process of target behavior prediction and trend judgment, the system can identify potential anomalies in advance and provide early warning for subsequent behavior detection, buying time for emergency response.

[0163] In terms of event response timeliness, the table provides the time data for the system to generate and push alarm information in different scenarios. The data shows that in most environments, the system can generate and push alarm information within 3 seconds, with the highest response time being 2 seconds and the lowest response time being 4 seconds. Especially in highly complex urban environments, although the system faces higher interference and data density, the response time can still be maintained within an acceptable range. This shows that the system has strong real-time performance and can respond to emergencies in a timely manner and promote subsequent emergency response measures.

[0164] The effectiveness of warning information push is also a key performance indicator. The table shows that in the experimental scenario, the system's warning information push success rate reached more than 97%, indicating that most warning information can be accurately delivered to relevant personnel or equipment. This is particularly important in the application of 5G messaging platforms, because the timely push of warning information directly affects the response speed of monitoring personnel and the efficiency of event handling.

[0165] Overall, these data show that in the application scenarios of the 5G message video surveillance system based on artificial intelligence, the system can effectively improve the intelligence level of video surveillance, provide efficient and accurate target behavior analysis and prediction, and respond to and handle abnormal events in a relatively short time. By integrating multiple advanced artificial intelligence technologies, including deep learning, convolutional neural networks, long short-term memory networks, etc., the system has demonstrated strong real-time, accuracy and reliability in practical applications, and can provide strong technical support for security monitoring, smart city management and other fields.

[0166] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A 5G message video monitoring method based on artificial intelligence, characterized in that: The steps include: S1, transmit the camera’s video data and environmental sensor data in real time through the 5G network, compress and preprocess the transmitted data, and build a multimodal data set; S2. performing joint convolutional neural network processing on the multimodal data set to generate multimodal fusion features; S3. Based on the multimodal fusion features, a 3D convolutional neural network is used to analyze the behavior of the video target data to generate a target behavior trend curve and a target behavior prediction result; S4, based on the generated target behavior trend curve and target behavior prediction results, using an adaptive algorithm based on variable domain search to generate a target threshold curve, dynamically adjust the behavior anomaly detection threshold, and obtain behavior anomaly warning information; S5. Perform environmental perception analysis based on the acquired abnormal behavior warning information and environmental sensor data, use a method combining fuzzy logic reasoning and Bayesian network to input environmental variables into the model for comprehensive judgment, and generate comprehensive warning information of environmental abnormality; S6. Based on the generated comprehensive abnormal alarm information, the graph neural network is used to perform spatiotemporal correlation reasoning across camera data to generate event correlation results; S7. Based on the generated event correlation results, use multivariate regression analysis and hierarchical analysis to construct event severity curves, and generate event severity levels and event response measures by evaluating event factors; S8. Push alarm information and event response measures to relevant personnel or equipment through the 5G messaging platform for real-time event notification and emergency handling.

2. According to the artificial intelligence-based 5G message video monitoring method of claim 1, it is characterized in that: The S3 specifically includes: S31, performing time series segmentation on the generated multimodal fusion feature data, and dividing the multimodal fusion feature data into time series according to the time window T w Segment processing to form multiple time series segments S32, time series segments Input to the 3D convolutional neural network, and extract the spatial-temporal feature matrix F of the video target behavior through the 3D convolution kernel st ; Among them, F st represents the space-time feature matrix, * represents the 3D convolution operation, f represents the activation function, W 3D represents the 3D convolution kernel weight matrix, b 3D Represents the 3D convolution kernel bias term; S33, the generated space-time feature matrix F st Perform frame-by-frame analysis to extract the target's motion trajectory features, including position, velocity, and acceleration, and construct the basic elements of the target's behavior trend. The target's position information in each frame is P t , speed is V t , the acceleration is A t : S34. Based on the motion trajectory characteristics of each frame, the target behavior trend curve is fitted by combining multivariate regression and Fourier analysis: Among them, C behavior represents the target behavior trend curve, β0 represents the bias term, β1, β2 and β3 represent the trajectory regression coefficients, γ k and δ k represents the Fourier coefficient, ω represents the frequency factor, and N represents the number of harmonic components; S35, target behavior trend curve C behavior Perform time series forecasting, use long short-term memory networks to predict future behaviors, and generate target behavior prediction results.

3. According to the 5G message video monitoring method based on artificial intelligence in claim 1, it is characterized in that: The S4 specifically includes: S41, performing feature decomposition on the generated target behavior trend curve and target behavior prediction result, and extracting a behavior feature variable matrix, wherein the behavior feature variable matrix includes a behavior trend deviation, a behavior change rate, and a behavior abnormality frequency; S42. Construct an initial threshold curve model based on the behavior characteristic variable matrix: Among them, T0(t) represents the initial threshold of behavior detection, exp represents the exponential function, α1, α2 and α3 represent the feature weights, λ1 represents the adjustment parameter of the feature, and F a1 Indicates the deviation of behavioral trend, F a2 represents the behavior change rate, F a3 Indicates the frequency of abnormal behavior, Represents the fluctuation term introduced by the time factor; S43, introduce an adaptive optimization algorithm based on variable neighborhood search, take the initial threshold curve T0(t) as input, and dynamically optimize the target threshold curve through each iteration: Among them, T opt (t) represents the target threshold curve, δ i represents the domain impact factor, k represents the number of neighborhood adjustments, and N i (t) represents the i-th neighborhood adjustment value; S44, based on the target threshold curve T generated by optimization opt (t), compare the target behavior trend curve point by point, and when the current value of the target behavior is greater than the target threshold curve value, the target behavior is judged to be abnormal; S45, further clustering analysis is performed on the abnormal behavior points, using a density peak clustering algorithm, taking the local abnormal density maximum as the center point, to form multiple abnormal behavior clusters, and each cluster behavior is marked with a label; S46. Generate abnormal behavior warning information based on the label information of the abnormal behavior cluster and the abnormal behavior trend, wherein the abnormal behavior warning information includes the abnormal behavior type, abnormal duration, and behavior deviation amplitude.

4. According to the 5G message video monitoring method based on artificial intelligence in claim 1, it is characterized in that: The S5 specifically includes: S51, using the abnormal behavior warning information and environmental sensor data as input to construct an input feature set for environmental perception analysis, wherein the environmental sensor data includes temperature and humidity, air pressure and smoke parameters, and the data input format is a multidimensional vector form, wherein each dimension represents an input variable, and the dimension value of each variable is dynamically updated according to real-time sensor feedback; S52, fuzzifying the environmental sensor data, setting a membership function to perform fuzzy conversion on each environmental sensor value; the fuzzifying process includes mapping the data value of each sensor to a fuzzy set, the fuzzy set represents the degree of belonging through the membership function, the membership function adopts a triangular, trapezoidal or Gaussian function form, and outputting a fuzzy set to represent the fuzzy value of the sensor data; S53, based on the fuzzy reasoning algorithm, a fuzzy rule base is constructed according to the fuzzification processing result and reasoning calculation is performed, wherein the fuzzy reasoning algorithm adopts a reasoning method based on the maximum membership principle, combined with fuzzy logic operation for reasoning, and outputs a fuzzy reasoning result to represent the possibility of each environmental variable; S54, using the fuzzy reasoning result as an input node of a Bayesian network to construct a probability model of the Bayesian network, wherein the Bayesian network adopts a directed acyclic graph structure, wherein each node represents an environmental variable, and the edges between nodes represent the causal relationship between the environmental variables; the reasoning mechanism of the Bayesian network updates the posterior probability distribution of the node by calculating the conditional probability of each node; S55. In the Bayesian network, the prior probability and conditional probability are updated online by using incremental learning. By updating the probability distribution of the network in real time, the relationship weights and prior probabilities between nodes are automatically corrected each time new abnormal behavior warning information and environmental sensor data are input; S56. Generate comprehensive warning information by comprehensively analyzing the correlation between the abnormal behavior warning information and the environmental sensor data according to the inference result of the Bayesian network. The comprehensive warning information includes the abnormal environment state, the abnormal category and the probability of the abnormal occurrence.

5. The 5G message video monitoring method based on artificial intelligence according to claim 1 is characterized in that: The S6 specifically includes: S61, receiving the generated comprehensive abnormal alarm information, and constructing an input data structure of the graph neural network, wherein the input data structure includes a plurality of nodes, each node represents a monitoring point of a camera or a monitoring area, and the characteristics of the node include abnormal state information of the monitoring point, the time and the abnormal type; S62. Construct the adjacency relationship of the graph neural network. Define the connection relationship between the monitoring points according to the spatiotemporal similarity between the monitoring points. The adjacency relationship of each monitoring point reflects the spatiotemporal dependency with the surrounding monitoring points, indicating the mutual influence of events in the time and space dimensions. S63, aggregate the feature information of each monitoring point through the graph convolutional neural network, transfer the information of the neighboring nodes to the current node, and the transfer operation of each layer of the graph neural network integrates the spatiotemporal information between the nodes; S64, through the multi-layer structure of the graph neural network, the spatiotemporal information between nodes is optimized and updated layer by layer, and the output result of each layer of the network is used to enhance the learning of the abnormal state of the monitoring point; S65. Perform spatiotemporal event association reasoning through the output results of the graph neural network. Perform cross-camera data reasoning based on the aggregated and optimized node features to identify event associations between different cameras. The reasoning process reveals the changing trends of events in different spaces and times by associating data from different cameras, and generates correlation scores between events to obtain spatiotemporal association reasoning results. S66. Generate a final event association result based on the obtained spatiotemporal association reasoning result, wherein the event association result includes the time, location, camera node, severity of the event, and propagation status in the entire monitoring area.

6. The 5G message video monitoring method based on artificial intelligence according to claim 1 is characterized in that: The S7 specifically includes: S71, identifying event-related factors according to the generated event association results, wherein the event-related factors include target behavior characteristics, environmental variables, and sensor data; S72, performing standardization processing on the event-related factors, and mapping the event-related factors to a unified evaluation scope; S73, using a multivariate regression analysis model, combined with the standardized results of the event-related factors, and generating an event severity curve C(t) by calculating a weighted coefficient, wherein C(t) represents the severity score of the event at time t; S74. Based on the calculated event severity curve, use the analytic hierarchy process to rank the importance of event-related factors, further evaluate the contribution of event-related factors to event severity, and determine the final severity level L(C(t)): Among them, τ1 and τ2 represent thresholds, ζ1, ζ2 and ζ3 represent control coefficients, exp represents exponential function, λ1, λ2 and γ1 represent adjustment factors, and δ1 and δ2 represent exponential factors; S75. Generate event response measures based on the evaluation result of the severity level L(C(t)) and in combination with a predetermined threshold setting rule.

7. A 5G message video monitoring system based on artificial intelligence, executing the 5G message video monitoring method based on artificial intelligence according to any one of claims 1 to 6, characterized in that: Includes the following modules: A data receiving module is used to receive video data from cameras and environmental sensor data through a 5G network; Feature extraction module for joint convolutional neural network processing of multimodal datasets; The behavior analysis module is used to analyze the behavior of the target data in the video using a 3D convolutional neural network to generate a target behavior trend curve and target behavior prediction results; A threshold optimization module is used to generate a target threshold curve using an adaptive algorithm based on variable domain search, dynamically adjust the behavior anomaly detection threshold, and obtain behavior anomaly warning information; The environmental perception analysis module is used to combine fuzzy logic reasoning and Bayesian network methods to input environmental variables into the model for comprehensive judgment and generate comprehensive warning information of environmental abnormal conditions; The event association reasoning module is used to use graph neural networks to perform spatiotemporal association reasoning across camera data and generate event association results; The event severity assessment module is used to construct an event severity curve using multivariate regression analysis and analytic hierarchy process, and to generate event severity levels and event response measures by evaluating event factors; The alarm information push module is used to push alarm information and event response measures to relevant personnel or equipment through the 5G message platform.

Images