An abnormality detection emergency processing system and method based on artificial intelligence

Abstract

The present application discloses an artificial intelligence-based anomaly detection emergency processing system, including: an image generation and storage module, a monitoring video acquisition module, an anomaly detection processing module, an air detection processing module, a comprehensive information processing module, a user communication module, a medical rescue module, and a fire emergency module. The present application also discloses an artificial intelligence-based anomaly detection emergency processing method. Based on the above-mentioned artificial intelligence-based anomaly detection emergency processing system, the spatiotemporal features of the acquired monitoring video frames are extracted by using the optimized spatiotemporal attention network model, and the anomaly and risk assessment index are obtained by combining the attention weight and the dynamic weight allocation, and then the comprehensive score is obtained to formulate an emergency processing strategy. It can improve the accuracy of anomaly detection, enhance the accuracy of environmental detection, and formulate an emergency processing strategy based on the comprehensive score, thereby improving the efficiency of emergency processing.

Description

Technical Field

[0001] The present invention relates to the field of machine learning, and in particular to an abnormality detection emergency processing system and method based on artificial intelligence. Background Art

[0002] With the continuous development of artificial intelligence technology, the anomaly detection emergency response system based on artificial intelligence has gradually shown its advantages. The traditional emergency response system relies on manual analysis and decision-making, and has problems such as slow response speed, insufficient accuracy, and strong dependence on professional knowledge. The anomaly detection emergency response system based on artificial intelligence can use machine learning and intelligent decision-making technology to realize automatic identification, rapid response and intelligent decision-making of emergency events, thereby improving the efficiency and accuracy of emergency response, reducing the possibility of human errors, and providing more scientific solutions for the handling of various emergency events. Nowadays, the problem of aging in society is becoming increasingly apparent, and the safety of elderly people living alone at home needs to be solved urgently. The indoor anomaly detection emergency response system uses machine learning to realize rapid perception and emergency response of dangerous events, which has shown great application potential.

[0003] my country's patent application number: CN201910184320.X, publication date: 2019.06.14, discloses a fall detection system based on vision and artificial intelligence, including an image data acquisition module, a server, a medical assistance agency, a local alarm, and a mobile terminal. It also provides a detection alarm method based on the fall detection system. The detection system of the present invention has a wide detection range, and can also detect multiple targets with one camera in key places such as nursing homes. The structure is simple, and users only need to install the camera and connect it to the Internet; and it supplements the shortcomings of existing home surveillance cameras, allowing surveillance cameras to realize monitoring functions. The detection alarm method adopts multi-algorithm fusion and artificial intelligence question and answer, which can greatly increase the accuracy of detection.

[0004] The inventors of the present application have discovered that the above-mentioned technology has at least the following technical problems: in the process of detecting human posture and center of gravity, artificially designing fall features as a reference for subsequent judgment will result in a small number of samples leading to inaccurate detection results; in the process of breathing detection, there are disadvantages in using images to recognize the rise and fall of the human chest. The rise and fall of the chest is more obvious when lying flat, while it is difficult to perform image recognition of the rise and fall of the human chest when lying on the side or prone due to the angle deviation of the surveillance camera. The detection data is not accurate enough, which will affect the subsequent calculation results. In addition, if multi-directional surveillance cameras are set up, the cost will increase and it will not be easily accepted by users; in the process of detecting background environmental information, the detection of air is ignored. Summary of the invention

[0005] The embodiment of the present application provides an artificial intelligence-based indoor anomaly detection and emergency response system to solve the problems of the existing technology of insufficient image sample data and lack of comprehensive consideration of environmental factors. It adds new normal indoor images through the indoor image generation model, optimizes the spatiotemporal attention network model, improves the accuracy of anomaly detection, and adds air detection sensors to detect smoke and carbon monoxide gas in the environment, further enhancing the accuracy of environmental detection. The system also formulates emergency response strategies based on the comprehensive score to improve the efficiency of emergency response.

[0006] An embodiment of the present application discloses an indoor anomaly detection emergency processing system based on artificial intelligence, comprising:

[0007] An image generation storage module is used to receive an initial normal indoor image obtained by manual collection, introduce an indoor image generation model, and generate a new normal indoor image based on the initial normal indoor image, wherein the new normal indoor image forms an image set;

[0008] The monitoring video acquisition module uses the monitoring camera to collect indoor monitoring videos in real time;

[0009] The anomaly detection processing module is used to receive an image set for training and optimization of the spatiotemporal attention network model; receive indoor surveillance videos, and the optimized spatiotemporal attention network model performs normalization processing and spatiotemporal feature extraction on the indoor surveillance videos, and obtains an anomaly score by combining spatiotemporal attention;

[0010] The air detection processing module uses air detection sensors to detect abnormalities of smoke and carbon monoxide in indoor air, normalizes the smoke concentration and carbon monoxide concentration obtained by abnormal detection, and then performs nonlinear fusion calculation through dynamic weight allocation to obtain a risk assessment index;

[0011] The comprehensive information processing module is used to receive the abnormality score and risk assessment index, perform normalization processing and nonlinear fusion calculation on the abnormality score and risk assessment index to obtain a comprehensive score, and formulate an emergency response strategy based on the comprehensive score;

[0012] The user communication module is used to receive emergency response strategies, send information to mobile phones and wearable devices bound to user accounts, and send messages to emergency contacts set by users;

[0013] Medical rescue module, used to receive emergency treatment strategies and send information to local medical institutions;

[0014] The fire emergency module is used to receive emergency response strategies and send information to the local emergency command center.

[0015] Another embodiment of the present application discloses an artificial intelligence-based indoor anomaly detection emergency processing method, comprising the following steps:

[0016] S1. Collect initial normal indoor images, introduce indoor image generation model, generate new normal indoor images based on the initial normal indoor images, and use the new normal indoor images as image sets for spatiotemporal attention network model training optimization;

[0017] S2. Obtain surveillance video frames, and use the optimized spatiotemporal attention network model to normalize and extract spatiotemporal features from the acquired surveillance video frames, and combine the attention weights to obtain anomaly scores; at the same time, the air detection sensor normalizes the smoke concentration and carbon monoxide concentration in the detected environment, and then performs nonlinear data fusion calculations according to dynamic weight allocation to obtain a risk assessment index;

[0018] S3. Normalize the abnormality score and risk assessment index and process them using nonlinear calculation methods to obtain a comprehensive score, and formulate an emergency response strategy based on the comprehensive score.

[0019] As a specific implementation, the indoor image generation model includes:

[0020] A generator, used to convert the input noise vector into a normal indoor image, where the noise vector is a series of random values ​​generated by a linear congruential method;

[0021] A discriminator, used to distinguish between a process normal indoor image and an initial normal indoor image;

[0022] The generator and the discriminator are trained through continuous adversarial training to generate new normal indoor images.

[0023] As a specific implementation, a residual block is added to the generator, and the formula for image generation after the residual block is introduced into the generator is expressed as:

[0024] G res (z) = ConvT(BN(ReLU(z+F(z))))

[0025] Among them, z is the input noise vector, which is the input of the generator; G res (z) represents the output of the generator after processing by the residual block; F(z) represents the operation of the convolutional layer in the residual block; z+F(z) represents the addition of the input vector z and the output F(z) of the convolutional layer operation; ConvT represents the transposed convolution operation, which is used for upsampling; BN represents batch normalization, and the ReLU activation function is used to introduce nonlinearity.

[0026] As a specific implementation, the Wasserstein distance loss function is used in the generator, and the Wasserstein distance loss function formula of the generator after the residual block is introduced is:

[0027]

[0028] Among them, L G It is the Wasserstein distance loss function of the generator after the residual block is introduced; is the expectation operation, z is derived from the noise distribution p z The noise vector sampled in (z).

[0029] As a specific implementation, a residual block is added to the discriminator, and the formula for image discrimination after the residual block is introduced into the discriminator is expressed as:

[0030] D res (x) = Conv(LeakyReLU(x+H(x)))

[0031] where x is the input image, i.e., a new normal indoor image that the discriminator tries to judge as real or generated, and D res (x) represents the output of the discriminator after processing by the residual block; H(x) represents the convolutional layer operation of the residual block in the discriminator; x+H(x) represents the implementation of the skip connection, which adds the input image x to the output H(x) of the convolutional layer operation to form residual learning; Conv represents the convolution operation, which is used for downsampling or feature extraction; LeakyReLU is the activation function.

[0032] As a specific implementation method, the Wasserstein distance loss function is introduced into the discriminator, and a gradient penalty is added. The specific formula is:

[0033]

[0034] Among them, L D It is the Wasserstein distance loss function of the discriminator after the introduction of residual blocks and gradient penalties; is an expectation operation, x is derived from the real data distribution p data The image sampled in (x); is the expected operation, where is the real image x and the generated image G res (z) is used to calculate the gradient penalty; λ is the weight of the gradient penalty, which is used to adjust the influence of the gradient penalty term. The λ value is set between 0.1 and 10; The discriminator after the introduction of the residual block is for the interpolated image The gradient of is the gradient penalty term.

[0035] As a specific implementation, step S1 further includes:

[0036] The quality of the generated new normal indoor images is evaluated using FID, and the high-quality new normal indoor images evaluated by FID are used as training sets:

[0037] During the training phase, the spatiotemporal attention network model is optimized by minimizing the loss function. The loss function formula of the spatiotemporal attention network model is:

[0038]

[0039] Among them, L(θ) is the loss function; f(u i ; θ) represents the spatiotemporal attention network model with parameter θ for the input feature u i The predicted output of is the loss calculation method using mean square error; v i is the actual label of the i-th image in the training set, u i is the input feature of the i-th image, N is the total number of images in the training set; α i represents the adaptive weight of the i-th image; λ represents the weight coefficient of the regularization term; R(θ) represents the regularization term.

[0040] As a specific implementation, step S2 specifically includes:

[0041] The optimized spatiotemporal attention network model includes input layer, preprocessing layer, spatial feature extraction layer, temporal feature extraction layer, spatiotemporal attention layer, and anomaly scoring layer;

[0042] The input layer is used to receive the surveillance video frames collected by the surveillance video acquisition module;

[0043] The preprocessing layer is used to normalize the surveillance video frames to obtain normalized video frames;

[0044] The spatial feature extraction layer is used to extract the spatial features in the normalized video frames through a convolutional neural network;

[0045] The temporal feature extraction layer inputs the extracted spatial feature sequence into the long short-term memory network to capture the temporal features in the video frame;

[0046] The spatiotemporal attention layer is used to automatically identify the most abnormal-indicative spatiotemporal features based on the spatial features and temporal features and the attention weights obtained through learning, so as to obtain the spatial attention weights and the temporal attention weights;

[0047] The anomaly scoring layer calculates the anomaly score of the video frame based on the spatial attention weight and the temporal attention weight;

[0048] Collecting indoor smoke concentration and carbon monoxide concentration, and normalizing the smoke concentration and carbon monoxide concentration to obtain normalized smoke concentration and carbon monoxide concentration;

[0049] The dynamic weight allocation method is used to reasonably adjust the weights of smoke and carbon monoxide concentrations;

[0050] According to the dynamic weight allocation, the normalized smoke concentration and carbon monoxide concentration are nonlinearly fused to obtain the risk assessment index.

[0051] As a specific implementation, step S3 specifically includes:

[0052] Normalizing the anomaly score and the risk assessment index respectively to obtain a normalized anomaly score and a risk assessment index;

[0053] The normalized abnormality score and risk assessment index were nonlinearly calculated using square, logarithmic transformation, and hyperbolic tangent functions to obtain a comprehensive score;

[0054] Formulate emergency response strategies based on the comprehensive scores, including normal state, warning state and dangerous state;

[0055] Normal state: no need to transmit information to the user communication module, and continue routine monitoring and air detection;

[0056] Warning status: transmit "warning" information to the user communication module to indicate possible risks, increase monitoring frequency, analyze data trends in real time, and prepare emergency response resources such as ambulances and fire brigades;

[0057] dangerous state, immediately transmit the "danger" information to the user communication module, and send emergency calls and specific location information to the medical rescue module and fire emergency module.

[0058] Beneficial effects:

[0059] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0060] 1. This application generates new normal indoor images through an indoor image generation model, which provides a guarantee for the training optimization of the spatiotemporal attention network model, thereby improving the accuracy of the spatiotemporal attention network model in detecting anomalies, saving labor and time costs, and is practical.

[0061] 2. In this application, by adding residual blocks to the generator and discriminator of the indoor image generation model, the indoor image generation model can generate more realistic new normal indoor images, and can also solve the gradient disappearance problem during training, thereby improving the training stability and efficiency of the indoor image generation model.

[0062] 3. In this application, the Wasserstein distance loss function plus the gradient penalty method is used to train the indoor image generation model, which avoids the problems of mode collapse and instability during training, enhances the quality of the generated new normal indoor images, helps the overall indoor image generation model converge to a better solution, and shows good adaptability and robustness.

[0063] 4. In this application, the spatial attention network model is optimized by minimizing the loss function, thereby improving the accuracy and reliability of the spatiotemporal attention network model for indoor anomaly detection.

[0064] 5. The spatiotemporal attention network model in this application obtains spatial attention weights and temporal attention weights through learning, and automatically identifies the spatiotemporal features that are most indicative of abnormalities, so that the spatiotemporal attention network model can focus more on the key areas and moments where abnormalities occur, thereby improving the accuracy and efficiency of anomaly detection, and is highly technical.

[0065] 6. This application uses a dynamic weight allocation method to reasonably adjust the weights of smoke and carbon monoxide concentrations, thereby achieving a more accurate assessment of indoor air conditions and reflecting the flexibility of the indoor anomaly detection system based on artificial intelligence. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A module diagram of an indoor anomaly detection and emergency response system based on artificial intelligence disclosed in an embodiment of the present application;

[0067] Figure 2 This is a flow chart of an indoor anomaly detection and emergency response system based on artificial intelligence disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0068] The embodiments of the present application provide an indoor anomaly detection and emergency response system based on artificial intelligence, thereby solving the technical problems in the prior art of lack of sufficient flexibility to respond quickly to abnormal conditions and low accuracy of information processing.

[0069] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0070] See attached Figure 1An embodiment of the present application provides an artificial intelligence-based indoor anomaly detection and emergency response system, including: an image generation and storage module, a monitoring video acquisition module, an anomaly detection and processing module, an air detection and processing module, a comprehensive information processing module, a user communication module, a medical rescue module, and a fire emergency module.

[0071] The image generation and storage module receives the initial normal indoor image obtained by manual collection, introduces the indoor image generation model, generates a new normal indoor image based on the initial normal indoor image, and the new normal indoor image is input into the anomaly detection processing module as an image set.

[0072] The monitoring video acquisition module uses the monitoring camera to collect indoor monitoring videos in real time and transmits the collected indoor monitoring videos to the anomaly detection module.

[0073] The anomaly detection processing module receives the image set from the image generation storage module for the spatiotemporal attention network model training optimization. The optimized spatiotemporal attention network model normalizes and extracts spatiotemporal features of the indoor surveillance video from the surveillance video acquisition module, combines the spatiotemporal attention, calculates the anomaly score, and passes it to the comprehensive information processing module.

[0074] The air detection processing module arranges air detection sensors to perform abnormal detection of smoke and carbon monoxide in the indoor air, normalizes the smoke concentration and carbon monoxide concentration obtained by the abnormal detection, and then performs nonlinear fusion calculation through dynamic weight allocation to obtain the risk assessment index, which is then input into the comprehensive information processing module.

[0075] The comprehensive information processing module receives the anomaly score from the anomaly detection processing module and the risk assessment index from the air detection processing module, performs normalization processing and nonlinear fusion calculation to obtain a comprehensive score, formulates an emergency processing strategy based on the comprehensive score, and transmits information to the user communication module, medical rescue module, and fire emergency module.

[0076] The user communication module receives the emergency response strategy of the comprehensive information processing module, sends information to mobile phones and wearable devices related to the user account, and sends messages to the emergency contacts set by the user. In order to avoid the inability to send information in a complex environment with weak signals, satellite SMS and satellite call functions are added.

[0077] The medical rescue module receives the emergency response strategy from the comprehensive information processing module and sends the information to the local medical institution.

[0078] The fire emergency module receives the emergency response strategy from the comprehensive information processing module and sends the information to the local emergency command center.

[0079] See attached Figure 2The embodiment of the present application also discloses an artificial intelligence-based indoor anomaly detection emergency processing method, comprising the following steps:

[0080] S1. Receive the initial normal indoor image collected manually, introduce the indoor image generation model, generate the process normal indoor image based on the initial normal indoor image, and use the process normal indoor image as the image set for the training and optimization of the spatiotemporal attention network model.

[0081] Receive the initial normal indoor image collected manually, introduce the indoor image generation model, and generate the process normal indoor image through the indoor image generation model, which can be used for spatiotemporal attention network model training optimization. The indoor image generation model consists of two parts: the generator and the discriminator. The generator is responsible for generating a new process indoor image from the input noise vector. The noise vector is a series of random values ​​generated by the linear congruential method; the discriminator is responsible for distinguishing the process normal indoor image from the initial normal indoor image. The generator and the discriminator achieve the generation of new normal indoor images through continuous adversarial training.

[0082] In particular, residual blocks are added to the generator and discriminator of the indoor image generation model respectively. The residual block is a neural network structure, which as a whole includes a set of ordinary convolutional layers and a skip connection. The skip connection adds the input directly to the output of the convolutional layer, aiming to improve the stability and efficiency of the indoor image generation model training.

[0083] The residual block of the generator contains two layers of convolutional layers, each followed by batch normalization and ReLU activation function. The first convolution layer uses a 3x3 filter, and the second convolution layer uses a 1x1 filter to reduce the number of parameters and enhance nonlinear expression capabilities. During the jump connection process, the input noise vector is directly added to the output of the convolution operation to achieve residual learning. The ReLU activation function is used to enhance the generator's ability to handle nonlinear problems; in particular, in the residual block of the generator, the ReLU activation function is used after each convolution layer to ensure nonlinear expression while maintaining effective gradient flow, which helps train the generator's deep neural network.

[0084] Introducing the entire residual block in the generator can be used to enhance the ability of the indoor image generation model to learn complex patterns while maintaining the stability of training. The formula for introducing the entire residual block in the generator for image generation is expressed as:

[0085] G res (z) = ConvT(BN(ReLU(z+F(z))))

[0086] Among them, z is the input noise vector, which is the input of the generator; G res(z) represents the output of the generator after processing by the residual block; F(z) represents the operation of the convolution layer in the residual block; z+F(z) implements the core of residual learning, that is, the input vector z is added to the output F(z) of the convolution layer operation to form a jump connection; ConvT represents the transposed convolution operation, which is used for upsampling and increasing the data dimension to generate an image; BN represents batch normalization, which is used for the output of the network layer to stabilize the training process and accelerate convergence. The ReLU activation function is used to introduce nonlinearity to improve the expressiveness of the indoor image generation model.

[0087] The residual block of the discriminator also includes two layers of convolution operations, but uses the LeakyReLU activation function to provide nonlinear processing and maintain a portion of the negative gradient. The first layer uses a 5x5 filter to better extract image features; the second layer of convolution can use a 3x3 filter. During the jump connection process, the input initial normal indoor image or process normal indoor image is directly added to the output of the convolution layer, and residual learning is used to reduce the difficulty of training the deep network of the discriminator. The LeakyReLU activation function provides nonlinear processing capabilities, allowing the indoor image generation model to capture complex and abstract data patterns, enhancing the sensitivity and discrimination ability of the indoor image generation model to image content, and making the discriminator more accurate in distinguishing real and generated images.

[0088] Introducing the entire residual block in the discriminator can improve the ability of the indoor image generation model to process deeper features while avoiding the increase in training difficulty. The formula for introducing the entire residual block in the discriminator for image discrimination is expressed as:

[0089] D res (x) = Conv(LeakyReLU(x+H(x)))

[0090] where x is the input image, i.e., a new normal indoor image that the discriminator tries to judge as real or generated, and D res (x) represents the output of the discriminator after processing by the residual block; H(x) represents the convolutional layer operation of the residual block in the discriminator; x+H(x) represents the implementation of the skip connection, which adds the input image x to the output H(x) of the convolutional layer operation to form residual learning; Conv represents the convolution operation, which is used for downsampling or feature extraction; LeakyReLU is the activation function that allows small negative gradients to flow through, which helps to maintain gradient information and avoid the "death" of the activation unit.

[0091] In order to enhance the quality of generated images, the Wasserstein distance loss function is used in the generator of the residual block.

[0092] The Wasserstein distance loss function formula of the generator after the introduction of the residual block is:

[0093]

[0094] Among them, L G It is the Wasserstein distance loss function of the generator after the residual block is introduced; is the expectation operation, z is derived from the noise distribution p z The noise vector sampled in (z).

[0095] Similarly, the Wasserstein distance loss function is introduced into the discriminator after the residual block is introduced. In particular, the gradient penalty is added to satisfy the constraint of Wasserstein distance on the discriminator gradient. The specific formula is:

[0096]

[0097] Among them, L D It is the Wasserstein distance loss function of the discriminator after the introduction of residual blocks and gradient penalties; is an expectation operation, x is derived from the real data distribution p data The image sampled in (x); is the expected operation, where is the real image x and the generated image G res (z) is used to calculate the gradient penalty; λ is the weight of the gradient penalty, which is used to adjust the influence of the gradient penalty term. The λ value is set between 0.1 and 10. Experimental tests are conducted to find the optimal value so that the gradient of the discriminator remains in a suitable range during training. The discriminator after the introduction of the residual block is for the interpolated image The gradient of is the gradient penalty term, which ensures that the discriminator after the introduction of the residual block is in the interpolated image The gradient magnitude at is close to 1 to satisfy the Wasserstein distance constraint on the discriminator gradient.

[0098] By introducing the residual block and the Wasserstein distance loss function, the optimized generator and discriminator are obtained. In particular, the gradient penalty is adopted in the optimized discriminator. During the training process, the optimized generator and the discriminator are constantly confronted. The optimized generator generates more realistic new normal indoor images, while the optimized discriminator accurately distinguishes the initial normal indoor images from the new normal indoor images until the discriminator cannot distinguish the initial normal indoor images from the new normal indoor images. The gradient penalty term is introduced to ensure that the optimized discriminator satisfies the gradient constraint of the Wasserstein distance during the training process.

[0099] The optimized generator and discriminator in the indoor image generation model are continuously used to generate images in competition, and finally a new normal indoor image is obtained.

[0100] FID is used to evaluate the quality of generating new normal indoor images. The main purpose of FID is to measure the similarity between the generated image and the real image in the feature space. The calculation formula is:

[0101] FID=∥μ g -μ r ∥ 2 +Tr(Σ g +Σ r -2(Σ g Σ r ) 1 / 2 )

[0102] Among them, μ g Represents the mean vector of the new normal indoor image features; μ r Represents the mean vector of the initial normal indoor image features; ∥μ g -μ r ∥ 2 Represents the new normal indoor image feature mean vector μ g and the initial normal indoor image feature mean vector μ r The square of the Euclidean distance between them measures the degree of deviation between the two sets of images in the center point of the feature space; Σ g Represents the covariance matrix of the new normal indoor image features; Σ r represents the covariance matrix of the initial normal indoor image features; Tr represents the trace operator, which is used to calculate the trace of the matrix, that is, the sum of the diagonal elements of the matrix; (Σ g Σ r ) 1 / 2 Represents Σ g and Σ r The square root of the product of the covariance matrices is a key part of measuring the similarity of two covariance matrices, and is used to calculate how close the two sets of data are in terms of their overall distribution shape.

[0103] Using high-quality new normal indoor images evaluated by FID as training sets can provide more accurate and realistic training data for the spatiotemporal attention network for spatiotemporal attention network model training optimization.

[0104] During the training phase, the spatiotemporal attention network model is optimized by minimizing the loss function in order to achieve the best performance on a given training set. The loss function formula of the spatiotemporal attention network model is:

[0105]

[0106] Among them, L(θ) is the loss function; f(ui ; θ) represents the spatiotemporal attention network model with parameter θ for the input feature u i The predicted output of is the loss calculation method using mean square error; v i is the actual label of the i-th image in the training set, u i is the input feature of the i-th image, N is the total number of images in the training set; α i represents the adaptive weight of the i-th image, which is used to adjust its influence in the loss function according to the importance of the image; λ represents the weight coefficient of the regularization term, which is used to control the influence of the regularization term on the loss function; R(θ) represents the regularization term, which is used to reduce the overfitting of the spatiotemporal attention network model.

[0107] By adaptively adjusting the contribution of the image in the loss function, the spatiotemporal attention network model can adaptively adjust the attention to each image during the training process, effectively improving the overall performance and robustness of the spatiotemporal attention network model. The specific formula is:

[0108]

[0109] Among them, α i represents the adaptive weight of the i-th image; exp is an exponential function used to ensure that the weight is positive, and the sharpness or smoothness of the weight distribution can be controlled by adjusting the parameter β; β is an adjustable coefficient used to control the sharpness of the weight distribution; Represents the loss value of the i-th image; Represents the summation, which is used to calculate the sum of the exponential loss over the training images, thus serving as the denominator to normalize the weight of each image.

[0110] Selecting the L2 norm as the regularization term can help control the size of the spatiotemporal attention network model parameters, prevent overfitting, and make the spatiotemporal attention network model training more stable. The specific formula is:

[0111] R(θ)=∥θ∥ 2

[0112] Among them, θ is the spatiotemporal attention network model parameter; R(θ) indicates that the L2 norm is selected as the regularization term;

[0113] By adjusting the adaptive weights and regularization terms, the optimized spatiotemporal attention network model parameters θ′ are obtained, achieving the purpose of optimizing the spatiotemporal attention network model by minimizing the loss function.

[0114] S2. The optimized spatiotemporal attention network model normalizes the acquired surveillance video frames and extracts spatiotemporal features, and obtains the anomaly score by combining the attention weights. At the same time, the air detection sensor normalizes the smoke concentration and carbon monoxide concentration in the detected environment, and then performs nonlinear data fusion calculation according to the dynamic weight allocation to obtain the risk assessment index.

[0115] The optimized spatiotemporal attention network model is introduced. The optimized spatiotemporal attention network model includes input layer, preprocessing layer, spatial feature extraction layer, temporal feature extraction layer, spatiotemporal attention layer, and anomaly scoring layer. The optimized spatiotemporal attention network model can effectively identify abnormal events from surveillance videos. Its advantage is that it can consider abnormal indication information in both spatial position and time, thereby improving the accuracy and efficiency of anomaly detection.

[0116] At the input layer, the surveillance video frames collected by the surveillance video acquisition module are received as input.

[0117] In the preprocessing layer, the input video frames are normalized and the pixel values ​​are adjusted to between [0, 1], which helps to speed up the training of the spatiotemporal attention network model and improve the performance of the spatiotemporal attention network model. The specific formula is:

[0118]

[0119] Among them, Y norm represents the normalized video frame; Y represents the video frame; Y min Represents the minimum value in the video frame; Y max Represents the maximum value in the video frame.

[0120] In the spatial feature extraction layer, a convolutional neural network is used to extract the spatial features in the normalized video frame in order to capture the visual information inside the frame. The specific formula is:

[0121] E space =CNN(Y norm )

[0122] Among them, CNN(Y norm ) is a convolutional neural network operation whose input is the normalized video frame Y norm , used to extract spatial features E from video frames space .

[0123] In the temporal feature extraction layer, the extracted spatial feature sequence is input into the long short-term memory network. The dynamic changes of the time dimension are captured by using the long short-term memory network. The formula is as follows:

[0124] E time =LSTM(E space )

[0125] Among them, LSTM (E space ) is a long short-term memory network, whose input is the spatial feature E extracted by the convolutional neural network space , used to capture the temporal features E in the video frames time .

[0126] In the spatiotemporal attention layer, combined with the spatial feature E space and time characteristics E time , the most abnormal indicative spatiotemporal features are automatically identified by learning the attention weights, so that the spatiotemporal attention network model can dynamically adjust the attention to different spatial positions and time points in the video frame, thereby improving the accuracy and efficiency of anomaly detection. The specific formula can be expressed as:

[0127] A space =STAM(E space )

[0128] A time =STAM(E time )

[0129] Among them, A space Represents the spatial attention weight, which indicates the degree of attention paid by the spatiotemporal attention network model to the region within the video frame. Regions with high weights are considered to be more likely to be related to abnormal events. time represents the temporal attention weight, which indicates the degree of attention paid by the spatiotemporal attention network model to the time points in the video frame. The time points with high weights are regarded as the moments when abnormal events are more likely to occur. STAM represents the function of the spatiotemporal attention module, which receives the spatial feature E. space and time characteristics E time As input, it outputs spatial and temporal attention weights.

[0130] In the anomaly scoring layer, the anomaly score of the video frame is calculated by combining the spatial and temporal attention weights. The specific formula can be expressed as:

[0131] Score anomaly =∑(A space ·A time )

[0132] Among them, Score anomaly is the anomaly score, which is used to indicate the possibility of an abnormal event occurring at a given time point or frame sequence; · represents the dot product operation, which is used to multiply the spatial and temporal attention weights element by element. The purpose is to combine the attention points in the spatial and temporal dimensions to calculate the weighted anomaly score of the time point; ∑ is the summation symbol, which indicates that the weighted spatial and temporal attention weights of all time points are accumulated to calculate the anomaly score of the entire video frame sequence.

[0133] The air detection processing module collects indoor smoke concentration S through air detection sensors, such as Figaro TGS5342 sensors. smoke and carbon monoxide concentration S CO , collected once every minute, which can be adjusted according to actual conditions. For example, a higher sampling frequency is required in an environment where the air quality changes rapidly;

[0134] In order to facilitate the smoke concentration S smoke and carbon monoxide concentration S CO To perform data fusion, first calculate the smoke density S smoke and carbon monoxide concentration S CO Normalization is performed, and the specific formula is expressed as:

[0135]

[0136] Among them, S smoke Represents smoke density; S smoke,norm Represents the normalized smoke density, ranging from 0 to 1; min(S smoke ) represents the minimum smoke concentration collected by the air detection sensor; max(S smoke ) represents the maximum smoke concentration collected by the air detection sensor;

[0137]

[0138] Among them, S CO Represents the concentration of carbon monoxide; S CO,norm represents the normalized carbon monoxide concentration, ranging from 0 to 1; min(S CO ) represents the minimum carbon monoxide concentration collected by the air detection sensor; max(S CO ) represents the maximum carbon monoxide concentration collected by the air detection sensor.

[0139] In order to more accurately evaluate the abnormal indoor air conditions, the dynamic weight allocation method is used to reasonably adjust the weights of smoke and carbon monoxide concentrations. The specific formula is:

[0140]

[0141] w CO =1-w smoke

[0142] Among them, w smoke is the weight of smoke concentration in the overall risk assessment index, reflecting the relative importance of smoke data; w COis the weight of carbon monoxide concentration in the total risk assessment index, reflecting the relative importance of carbon monoxide data; exp is an exponential function used to calculate the exponential power of the base of the natural logarithm; α is an adjustment factor used to control the sensitivity of weight adjustment, which can be adjusted according to the needs of the actual application scenario to achieve the best dynamic response effect; var(S smoke,norm ) represents the variance of the normalized smoke density data, which is used to measure the degree of change or instability of the smoke density data; var(S CO,norm ) represents the variance of the normalized carbon monoxide concentration data, which is used to measure the degree of change or instability of the carbon monoxide concentration data.

[0143] According to the dynamic weight distribution, the normalized smoke concentration and carbon monoxide concentration are subjected to nonlinear data fusion to obtain the risk assessment index. The specific formula is:

[0144]

[0145] Among them, S smoke,norm and S CO,norm are the normalized smoke and carbon monoxide concentrations, log(1+S smoke,norm ) is a logarithmic transformation of smoke concentration, which is used to amplify the impact of low-concentration smoke; is the square root transformation of the carbon monoxide concentration, which helps to smooth the impact of high concentrations; the tanh function is used to convert the risk assessment index S combined Compressed to the range of (-1,1) to provide a more stable and uniform risk output.

[0146] S3. The abnormality score and risk assessment index are processed by normalization and nonlinear calculation methods to obtain a comprehensive score, and an emergency response strategy is formulated based on the comprehensive score.

[0147] Receive the anomaly score Score from the anomaly detection processing module anomaly and the risk assessment index S of the air detection processing module combined , respectively, perform standardized transformation, the specific formula is;

[0148]

[0149] Among them, Score anomaly Represents abnormal score; Score anomaly，norm Represents the normalized abnormality score, ranging from 0 to 1; min(Score anomaly ) represents the minimum abnormality score; max(Score anomaly ) represents the maximum anomaly score.

[0150]

[0151] Among them, S combined Represents risk assessment index; Score combined，norm represents the normalized risk assessment index, ranging from 0 to 1; min(S combined ) represents the minimum value of the risk assessment index; max(S combined ) represents the maximum value of the risk assessment index.

[0152] The normalized anomaly score Score is calculated by using square, logarithmic transformation and hyperbolic tangent function. anomaly，norm and risk assessment index Score combined，norm Nonlinear calculation can effectively handle the nonlinear relationship between anomaly score and risk assessment index. The specific formula is:

[0153]

[0154] Among them, Score total represents the comprehensive score; α and β represent weight parameters, which are used to adjust the contribution of the anomaly score and the safety index in the calculation of the comprehensive score; Represents the square of the normalized anomaly score, which is used to amplify the impact of the anomaly score in the calculation; log(1+S combined,norm ) represents taking the natural logarithm of the normalized risk assessment index after adding 1, which is used to deal with the nonlinear relationship of the safety index and ensure that the low safety index has a greater impact on the result; tanh represents the hyperbolic tangent activation function, which is used to compress the comprehensive score into the range of (-1,1).

[0155] When environmental conditions change, such as when changes in lighting affect video clarity, the weights can be dynamically adjusted to adapt to the environment. For different security events, such as carbon monoxide gas leaks, video surveillance has difficulty capturing image information, while air detection sensors can detect carbon monoxide gas concentrations, thereby adjusting the weights to optimize the detection effect.

[0156] According to the comprehensive score, an emergency response strategy is formulated and transmitted to the user communication module, medical rescue module, and fire emergency module. The specific emergency response strategy is as follows:

[0157] Trigger conditions:

[0158]

[0159] Among them, Low risk threshold represents the low risk threshold, and High risk threshold represents the high risk threshold. The setting of the threshold is based on historical data using statistical methods and defined using quantiles. The 95% quantile is used for the high risk threshold, and the 50% quantile is used for the low risk threshold. This ensures that the threshold is statistically reasonable and reflects the common risk distribution. The threshold is further adjusted based on the opinions and experience of emergency response experts. Score total <Low risk threshold means normal; Low risk threshold≤Score total <High risk threshold means warning; Score total ≥High risk threshold means dangerous.

[0160] The processing strategy for the normal state is as follows:

[0161] There is no need to transmit information to the user communication module, and routine monitoring and air detection can continue; the monitoring equipment and air detection sensors should be maintained once a month to ensure their normal operation.

[0162] The specific processing strategy for the warning status is as follows:

[0163] Deliver "warning" information to user communication modules to alert potential risks, including people falling or air abnormalities; increase monitoring frequency and analyze data trends in real time; prepare emergency response resources, such as ambulance and fire brigade warnings.

[0164] The specific handling strategy for dangerous status is as follows:

[0165] Immediately transmit "danger" information to the user communication module, and send emergency calls and specific location information to the medical rescue module and fire emergency module; the medical and fire teams quickly arrive at the designated location, carry out on-site rescue operations, treat the wounded and control the spread of the accident.

[0166] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages:

[0167] This application enhances the ability of the indoor image generation model to learn complex patterns by introducing residual blocks into the indoor image generation model while maintaining the stability of training.

[0168] This application uses the Wasserstein distance loss function plus the gradient penalty method to train the indoor image generation model, which helps to solve the mode collapse problem in training and improves the quality of generated images.

[0169] In this application, the spatiotemporal attention network model is optimized through adaptive weights, so that the model can adaptively adjust the attention to each image during the training process, effectively improving the overall performance and robustness of the spatiotemporal attention network model.

[0170] This application selects the L2 norm as the regularization term to effectively control the size of the spatiotemporal attention network model parameters, prevent overfitting, and make the spatiotemporal attention network model training more stable.

[0171] This application automatically identifies the most abnormality-indicative spatiotemporal features through attention weights, so that the spatiotemporal attention network model can dynamically adjust the attention to different spatial positions and time points in the video frame, improving the accuracy and efficiency of anomaly detection and having high practical value.

[0172] This application dynamically assigns weights to perform nonlinear fusion of smoke concentration and carbon monoxide concentration to more accurately assess abnormal indoor air conditions with flexibility.

[0173] This application formulates emergency response strategies through comprehensive scoring, which improves the timeliness and accuracy of emergency response in the comprehensive information processing module.

[0174] Effect research:

[0175] The technical solution of the present application can effectively solve the problem of misjudgment caused by small amount of sample information and inaccurate image information. The above system has undergone a series of effect surveys. The indoor image generation model generates new normal indoor images as an image set to provide support for the training and optimization of the spatiotemporal attention network model, thereby improving the efficiency of anomaly detection. Smoke sensors and carbon monoxide sensors are added to detect indoor smoke and carbon monoxide concentrations, making up for the previous insufficiency of environmental detection. Emergency response strategies are formulated based on comprehensive scores to achieve rapid response and efficient handling of various emergencies, demonstrating the powerful comprehensive analysis and processing capabilities of the indoor anomaly detection and emergency response system based on artificial intelligence.

[0176] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0177] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0178] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0179] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An artificial intelligence-based indoor anomaly detection emergency processing method, characterized in that: The following steps are involved: S1. Collect initial normal indoor images, introduce indoor image generation model, generate new normal indoor images based on the initial normal indoor images, and use the new normal indoor images as image sets for spatiotemporal attention network model training optimization; The indoor image generation model includes a generator and a discriminator; The generator generates a normal indoor image from the input noise vector; a residual block is introduced into the generator, and the formula for image generation is expressed as: G res (z)=ConvT(BN(ReLU(z+F(z)))) Among them, z is the input noise vector, is the input of the generator, and is a series of random values ​​generated by the linear congruential method; G res (z) represents the output of the generator after being processed by the residual block; F(z) represents the operation of the convolutional layer in the residual block; z+F(z) represents the addition of the input vector z and the output F(z) of the convolutional layer operation; ConvT represents the transposed convolution operation, which is used for upsampling; BN represents batch normalization, and the ReLU activation function is used to introduce nonlinearity; The Wasserstein distance loss function is used in the generator, and the Wasserstein distance loss function formula of the generator after the residual block is introduced is: Among them, L G It is the Wasserstein distance loss function of the generator after the residual block is introduced; is the expectation operation, z is derived from the noise distribution p z (z) is the noise vector sampled in; D res (x) represents the discriminator output after being processed by the residual block; The discriminator discriminates the process normal indoor image and the initial normal indoor image; the formula for image discrimination after introducing the residual block into the discriminator is expressed as: D res (x)=Conv(LeakyReLU(x+H(x))) where x is the input image, i.e., a new normal indoor image that the discriminator tries to judge as real or generated, and D res (x) represents the output of the discriminator after processing by the residual block; H(x) represents the convolutional layer operation of the residual block in the discriminator; x+H(x) represents the implementation of the skip connection, which adds the input image x to the output H(x) of the convolutional layer operation to form residual learning; Conv represents the convolution operation, which is used for downsampling or feature extraction; LeakyReLU is the activation function; The Wasserstein distance loss function is introduced into the discriminator, and a gradient penalty is added. The specific formula is: Among them, L D It is the Wasserstein distance loss function of the discriminator after the introduction of residual blocks and gradient penalties; is an expectation operation, x is derived from the real data distribution p data The image sampled in (x); is the expected operation, where is the real image x and the generated image G res (z) is used to calculate the gradient penalty; λ is the weight of the gradient penalty, which is used to adjust the influence of the gradient penalty term. The λ value is set between 0.1 and 10; The discriminator after the introduction of the residual block is for the interpolated image The gradient of is the gradient penalty term; The generator and the discriminator are trained through continuous adversarial training to generate the new normal indoor image; S2. Obtain surveillance video frames, and use the optimized spatiotemporal attention network model to normalize and extract spatiotemporal features of the acquired surveillance video frames, and obtain an abnormality score by combining the attention weights; at the same time, the air detection sensor normalizes the smoke concentration and carbon monoxide concentration in the detected environment, and then performs nonlinear data fusion calculation according to dynamic weight allocation to obtain a risk assessment index; S3. The abnormality score and the risk assessment index are normalized and processed using a nonlinear calculation method to obtain a comprehensive score, and an emergency response strategy is formulated based on the comprehensive score.

2. The artificial intelligence-based indoor anomaly detection emergency processing method according to claim 1, characterized in that: Step S1 also includes: The quality of the new normal indoor image is generated by using FID evaluation, and the high-quality new normal indoor image evaluated by FID is used as a training set: During the training phase, the spatiotemporal attention network model is optimized by minimizing the loss function, and the loss function formula of the spatiotemporal attention network model is: Among them, L(θ) is the loss function; f(u i ; θ) represents the spatiotemporal attention network model with parameter θ for the input feature u i The predicted output of is the loss calculation method using mean square error; v i is the actual label of the i-th image in the training set, u i is the input feature of the i-th image, N is the total number of images in the training set; α i represents the adaptive weight of the i-th image; λ represents the weight coefficient of the regularization term; R(θ) represents the regularization term.

3. The artificial intelligence-based indoor anomaly detection emergency processing method according to claim 1, characterized in that: Step S2 specifically includes: The optimized spatiotemporal attention network model includes an input layer, a preprocessing layer, a spatial feature extraction layer, a temporal feature extraction layer, a spatiotemporal attention layer, and an anomaly scoring layer; The input layer is used to receive the monitoring video frames collected by the monitoring video collection module; The preprocessing layer is used to perform normalization processing on the monitoring video frame to obtain a normalized video frame; The spatial feature extraction layer is used to extract spatial features in the normalized video frame through a convolutional neural network; The temporal feature extraction layer inputs the extracted spatial feature sequence into the long short-term memory network to capture the temporal features in the video frame; The spatiotemporal attention layer is used to automatically identify the spatiotemporal features that are most indicative of abnormality based on the spatial features and the temporal features and the attention weights obtained through learning, so as to obtain the spatial attention weights and the temporal attention weights; The anomaly scoring layer calculates an anomaly score of the video frame based on the spatial attention weight and the temporal attention weight; Collecting indoor smoke concentration and carbon monoxide concentration, and normalizing the smoke concentration and carbon monoxide concentration to obtain normalized smoke concentration and carbon monoxide concentration; The dynamic weight allocation method is used to reasonably adjust the weights of smoke and carbon monoxide concentrations; According to the dynamic weight allocation, the normalized smoke concentration and carbon monoxide concentration are nonlinearly fused to obtain the risk assessment index.

4. The artificial intelligence-based indoor anomaly detection emergency processing method according to claim 1, characterized in that: Step S3 specifically includes: Normalizing the abnormality score and the risk assessment index respectively to obtain a normalized abnormality score and a normalized risk assessment index; The normalized abnormality score and risk assessment index were nonlinearly calculated using square, logarithmic transformation, and hyperbolic tangent functions to obtain a comprehensive score; Formulate emergency response strategies based on the comprehensive scores, wherein the emergency response strategies include normal state, warning state and dangerous state; Normal state: no need to transmit information to the user communication module, and continue routine monitoring and air detection; Warning state: transmit "warning" information to the user communication module to indicate possible risks, increase monitoring frequency, analyze data trends in real time, and prepare emergency response resources such as ambulances and fire brigades; dangerous state, immediately transmit "danger" information to the user communication module, and send emergency calls and specific location information to the medical rescue module and fire emergency module.

5. An indoor anomaly detection emergency processing system based on artificial intelligence, applied to the indoor anomaly detection emergency processing method based on artificial intelligence as claimed in claim 1, characterized in that: include: An image generation storage module is used to receive an initial normal indoor image obtained by manual collection, introduce an indoor image generation model to generate a new normal indoor image based on the initial normal indoor image, and the new normal indoor image forms an image set; The monitoring video acquisition module uses the monitoring camera to collect indoor monitoring videos in real time; An anomaly detection processing module, used to receive the image set for spatiotemporal attention network model training optimization; The indoor surveillance video is received, and the optimized spatiotemporal attention network model performs normalization processing and spatiotemporal feature extraction on the indoor surveillance video, and combines the spatiotemporal attention to obtain an anomaly score; The air detection processing module uses air detection sensors to detect abnormalities of smoke and carbon monoxide in indoor air, normalizes the smoke concentration and carbon monoxide concentration obtained by abnormal detection, and then performs nonlinear fusion calculation through dynamic weight allocation to obtain a risk assessment index; A comprehensive information processing module, used for receiving the abnormality score and the risk assessment index, performing normalization processing and nonlinear fusion calculation on the abnormality score and the risk assessment index to obtain a comprehensive score, and formulating an emergency response strategy according to the comprehensive score; A user communication module is used to receive the emergency handling strategy, send information to mobile phones and wearable devices related to the user account, and send messages to emergency contacts set by the user; A medical rescue module, used to receive the emergency treatment strategy and send information to a local medical institution; The fire emergency module is used to receive the emergency handling strategy and send information to the local emergency command center.

Images