Home environment abnormity monitoring system and method based on infrared sensing

Abstract

The invention relates to the technical field of smart home, and discloses a home environment abnormity monitoring system and method based on infrared perception, and the system comprises a preprocessing module which responds to collected infrared image data, carries out the preprocessing of the infrared image data, and determines an infrared image; the prediction module is used for obtaining a prediction result by using a network model based on the infrared image; and the exception monitoring report generation module is used for generating an exception monitoring report based on the corresponding relationship between the prediction result and the exception data. According to the scheme, the prediction result is obtained through the network model by using the infrared image, so that the abnormal monitoring report is generated according to the corresponding relation between the prediction result and the abnormal data, the abnormal condition is monitored, and the accuracy of environment information identification in the home environment is improved.

Description

Technical Field

[0001] The present invention relates to the field of smart home technology, and in particular to a system and method for monitoring abnormalities in a home environment based on infrared sensing. Background Art

[0002] With the rapid development of the Internet of Things and smart home technologies, intelligent management and monitoring of home environments have become increasingly important. As an important part of home automation, smart home robots can perceive the environment in real time through a variety of sensors and respond accordingly. Existing smart home systems mainly rely on visible light images, infrared sensors, ultrasonic sensors, etc. for environmental perception and data collection. However, these systems still have many problems and challenges in practical applications. The current smart home systems have limited recognition and positioning accuracy for target objects in complex environments, and it is difficult to accurately obtain environmental information.

[0003] In related technologies, in order to ensure the accuracy of environmental information, a multi-sensor fusion strategy is usually adopted, combined with a depth camera module, to monitor and capture targets more accurately and enhance environmental perception capabilities. However, multi-sensor data fusion is difficult. In practical applications, data from multiple sensors face multiple problems such as data synchronization, sensor calibration, and complex data processing algorithms, which makes it difficult to ensure the robustness and accuracy of the system. Therefore, there is an urgent need for an abnormal monitoring system that can accurately identify environmental information in a home environment. Summary of the invention

[0004] In view of this, the present invention provides a home environment abnormality monitoring system and method based on infrared sensing to solve the technical problem of low accuracy in identifying abnormal data in a smart home system.

[0005] In a first aspect, the present invention provides a home environment abnormality monitoring system based on infrared sensing, the system comprising: a preprocessing module, for preprocessing the infrared image data in response to the collected infrared image data, and determining the infrared image; a prediction module, for obtaining a prediction result based on the infrared image and using a network model; an abnormal monitoring report generation module, for generating an abnormal monitoring report based on the correspondence between the prediction result and the abnormal data.

[0006] In combination with the first aspect, in a possible implementation of the first aspect, the preprocessing module includes: a normalization submodule, which is used to perform mean-variance normalization and minimum-maximum normalization on the infrared image data in response to the collected infrared image data to determine the normalized data; and a wavelet transform submodule, which is used to perform wavelet transform on the normalized data using wavelet coefficients and discrete wavelet coefficients to determine the infrared image.

[0007] In combination with the first aspect, in a possible implementation of the first aspect, the prediction module includes: a convolutional network sub-module, used to extract spatial features of the infrared image and determine the vector corresponding to the infrared image; an LSTM sub-module, used to process long-term dependencies based on the vector and determine the output vector; an attention sub-module, used to determine the weighted average sum, and determine the prediction result based on the weighted average sum and the output vector.

[0008] In combination with the first aspect, in a possible implementation of the first aspect, the convolutional network sub-module includes: a time domain convolution sub-unit, which is used to perform time domain convolution based on one-dimensional convolution using a convolution kernel to determine the first spatial feature; a first vector determination sub-unit, which is used to determine the vector corresponding to the infrared image based on the first spatial feature by using pooling and dimensionality reduction.

[0009] In combination with the first aspect, in a possible implementation of the first aspect, the convolutional network submodule includes: a hole convolution subunit, used to perform hole convolution in response to a hole inserted in the convolution kernel to determine a second spatial feature; a second vector determination subunit, used to determine a vector corresponding to the infrared image based on the second spatial feature by using pooling and dimensionality reduction.

[0010] In combination with the first aspect, in a possible implementation of the first aspect, the LSTM submodule includes: a weight coefficient determination subunit, used to determine the weight coefficient by calculating the channel weight and the spatial weight; a prediction result determination subunit, used to perform hierarchical transfer based on the weight coefficient to determine the output vector.

[0011] In combination with the first aspect, in a possible implementation of the first aspect, the system further includes: a filtering module, configured to filter the environmental sensor data in response to the collected environmental sensor data, and predict the state of the environmental sensor data.

[0012] In combination with the first aspect, in a possible implementation of the first aspect, the abnormal monitoring report generation module further includes: an abnormal report generation submodule, which is used to generate an abnormal monitoring report based on the corresponding relationship between the prediction result, the state and the abnormal data.

[0013] In a second aspect, the present invention provides a method for monitoring home environment abnormalities based on infrared sensing, the method comprising: in response to collected infrared image data, preprocessing the infrared image data to determine the infrared image; based on the infrared image, using a network model to obtain a prediction result; based on the correspondence between the prediction result and the abnormal data, generating an abnormal monitoring report.

[0014] In combination with the second aspect, in a possible implementation method of the second aspect, an abnormal monitoring report is generated based on the correspondence between the prediction results and the abnormal data, and it also includes: in response to the collected environmental sensor data, filtering the environmental sensor data, and predicting the state of the environmental sensor data; based on the correspondence between the prediction results, the state and the abnormal data, an abnormal monitoring report is generated.

[0015] The technical solution of the present invention has the following advantages:

[0016] The present invention provides a system and method for monitoring abnormalities in a home environment based on infrared sensing. The system preprocesses the collected infrared image data through a preprocessing module to obtain an infrared image, and uses a prediction module to extract features from the infrared image using a network model to determine the prediction results corresponding to the abnormalities in the home environment, and generates an abnormal monitoring report through an abnormal monitoring report generation module based on the corresponding relationship between the prediction results and the abnormal data. In this process, by preprocessing the infrared image data, normalization and wavelet transform are used to make the infrared image more conducive to the convergence of the network model and improve the generalization ability, and a network model is formed in the form of a convolutional network, LSTM and an attention mechanism, and then the network model is used to extract features from the infrared image to improve the accuracy of abnormal data recognition in the home environment, so as to generate an abnormal monitoring report using the corresponding relationship between the prediction results and the abnormal data. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 is a structural block diagram of a home environment abnormality monitoring system based on infrared sensing according to an embodiment of the present invention;

[0019] Figure 2 is a structural block diagram of another home environment abnormality monitoring system based on infrared sensing provided according to an embodiment of the present invention;

[0020] Figure 3 It is a flowchart of a method for monitoring abnormalities in a home environment based on infrared sensing provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0022] This embodiment provides a home environment abnormality monitoring system based on infrared sensing, such as Figure 1 As shown, the system includes the following modules:

[0023] The preprocessing module 101 preprocesses the infrared image data in response to the collected infrared image data to determine an infrared image.

[0024] Specifically, the collected infrared image data refers to the infrared image information collected by the infrared image sensor. Preprocessing the infrared image data refers to normalizing the infrared image and performing wavelet transformation on the data using wavelet coefficients and discrete wavelet coefficients, thereby determining the infrared image, making the infrared image more conducive to the convergence of the network model and improving the generalization ability.

[0025] The prediction module 102 is used to obtain a prediction result based on the infrared image using a network model.

[0026] Specifically, the network model refers to a network model formed by a convolutional neural network, an LSTM network (Long Short-Term Memory) and an attention mechanism, wherein the convolutional neural network is used to extract the spatial connection between different feature values ​​from the original data, the LSTM network is used to process the long-term relationship in the data, and the attention mechanism is used to refine and enhance the feature map, thereby improving the representation ability and decision accuracy of the model, so that through the network model, the convolutional neural network makes up for the shortcomings of the LSTM network that cannot capture the spatial components of the data, and the LSTM network extracts the time series change information of the nonlinear data of the cold and hot loads in the home environment, and the attention mechanism adaptively learns the importance of different channels at different positions in the input features, provides higher weights for important features, and captures information with discrimination, so that the prediction results determined by the network model can accurately identify abnormal data in the home environment. Among them, the prediction results refer to the prediction probabilities corresponding to temperature anomalies, human activity anomalies, and fire anomalies.

[0027] The abnormality monitoring report generating module 103 is used to generate an abnormality monitoring report based on the corresponding relationship between the prediction result and the abnormal data.

[0028] Specifically, based on the correspondence between the prediction results and the abnormal data, generating an abnormal monitoring report means utilizing the prediction probabilities corresponding to temperature abnormalities, human activity abnormalities, and fire abnormalities, respectively, and the corresponding relationships between the thresholds corresponding to the respective prediction probabilities, and when any prediction probability exceeds the respective corresponding threshold, generating an abnormal monitoring report corresponding to the prediction probability.

[0029] The present invention provides a system and method for monitoring abnormalities in a home environment based on infrared sensing. The system preprocesses the collected infrared image data through a preprocessing module to obtain an infrared image, and uses a prediction module to extract features from the infrared image using a network model to determine the prediction results corresponding to the abnormalities in the home environment, and generates an abnormal monitoring report through an abnormal monitoring report generation module based on the corresponding relationship between the prediction results and the abnormal data. In this process, by preprocessing the infrared image data, normalization and wavelet transform are used to make the infrared image more conducive to the convergence of the network model and improve the generalization ability, and a network model is formed in the form of a convolutional network, LSTM and an attention mechanism, and then the network model is used to extract features from the infrared image to improve the accuracy of abnormal data recognition in the home environment, so as to generate an abnormal monitoring report using the corresponding relationship between the prediction results and the abnormal data.

[0030] In an alternative embodiment, if Figure 2 As shown, the preprocessing module 101 includes:

[0031] The normalization submodule 1011 is used to perform mean-variance normalization and minimum-maximum normalization on the infrared image data in response to the collected infrared image data to determine the normalized data; the wavelet transform submodule 1021 is used to perform wavelet transform on the normalized data using wavelet coefficients and discrete wavelet coefficients to determine the infrared image.

[0032] Specifically, in response to the collected infrared image data, the infrared image data is normalized to ensure that the data is within a standard range, wherein the normalization of the infrared image includes: mean-variance normalization and minimum-maximum normalization. The mean-variance normalization converts the data into data with zero mean and unit variance, and the minimum-maximum normalization scales the data to the range of [0,1], [-1,1].

[0033] Specifically, the mean-variance normalization of the infrared image data is expressed by formula (1):

[0034]

[0035] in, represents the normalized data, x represents the original data, μ represents the mean of the data, and ξ represents the standard deviation of the data.

[0036] Specifically, the minimum-maximum normalization of the infrared image data is expressed by formula (2):

[0037]

[0038] in, represents the normalized data, x represents the original data, and x min Represents the minimum value of the data, x max Indicates the maximum value of the data.

[0039] Specifically, the purpose of performing wavelet transform on normalized data is to process the data through wavelet analysis to obtain more stable features and reduce the impact of noise on classification results. The wavelet transform on normalized data includes continuous wavelet transform and discrete wavelet transform.

[0040] Specifically, the continuous wavelet transform of normalized data is expressed by formula (3):

[0041]

[0042] Where W(a,b) represents the wavelet coefficient, x(t) represents the original signal, ψ represents the mother wavelet, and ψ * represents the complex conjugate of the mother wavelet, a represents the scale parameter, which controls the width of the wavelet, and b represents the translation parameter, which controls the translation amount of the wavelet.

[0043] Specifically, the discrete wavelet transform of the normalized data is expressed by formula (4):

[0044] W j,k =∑ n x[n]ψ j,k [n] (4)

[0045] Among them, W j,k represents the discrete wavelet coefficient, x[n] represents the discrete signal, j is the scale parameter, which controls the resolution of the wavelet, k is the translation parameter, which controls the position of the wavelet, ψ j,k [n] represents the discrete wavelet basis function, which is usually expressed as: ψ j,k [n] = 2 -j / 2 ψ(2 -j nk).

[0046] In an alternative embodiment, if Figure 2 As shown, the prediction module 102 includes:

[0047] The convolutional network submodule 1021 is used to extract spatial features of the infrared image and determine the vector corresponding to the infrared image; the LSTM submodule 1022 is used to process long-term dependencies based on the vector and determine the output vector; the attention submodule 1023 is used to determine the weighted average sum and determine the prediction result based on the weighted average sum and the output vector.

[0048] Specifically, Figure 2 As shown, after receiving the infrared image, the prediction module 102 inputs the infrared image into the convolutional network sub-module 1021, uses the output of the convolutional network sub-module 1021 as the input of the LSTM sub-module 1022, and uses the output of the LSTM sub-module 1022 as the input of the attention sub-module 1023, so that the output of the attention sub-module 1023, that is, the prediction result, serves as the output of the prediction module 102.

[0049] Specifically, the prediction module 102 is described by a hierarchical structure, with the infrared image serving as the input layer, the convolutional network submodule 1021 serving as the convolutional network layer, the LSTM submodule 1022 serving as the multi-layer LSTM layer, the attention submodule 1023 serving as the attention layer, and the output of the attention submodule 1023 serving as the output layer.

[0050] Specifically, the first layer is the input layer, which specifies the format of the input data, including batch size, time steps, and feature dimensions. The batch size is usually set to 1, the time step is t, and the feature dimension is n. Then a sample can be represented as a real number sequence matrix R txn , denoted by X i , indicating R txn The vector representation of the data at the i-th time step in .

[0051] Specifically, the second layer is the convolutional network layer, which can extract the spatial connection between different feature values ​​in the data, thereby making up for the shortcomings of the multi-layer LSTM layer that cannot capture the spatial components of the data. At the same time, the features it extracts are still temporal. Sample data enters the convolutional network layer and undergoes convolution, pooling, and dimensionality reduction operations in sequence. For sequence data, this model adopts one-dimensional convolution, and the convolution kernel only performs convolution in a single time domain direction. The number of convolution kernels is r, and the size is set to k, then X i :i+k-1 is R txn The real number matrix from the i-th time step to the i+1-th time step in , with a sliding step size of 1. The weight matrix W l is a kxn real matrix. A feature extraction is performed on the sequence vector of each k time step to obtain a feature O i , which is expressed by formula (5) as follows:

[0052]

[0053] Where f is a nonlinear activation function and b1∈R is a bias. When a convolution kernel extracts a sequence of data of a sample, a feature map O of (t-k+1)x1 shape is obtained. The calculation formula is as follows:

[0054] O=[O1,O2,…,O t-k+1 ] T

[0055] The convolutional network layer has a total of r convolution kernels, so r feature maps are finally obtained. After convolution, the maximum pooling operation is performed with a pooling size of 2 and a sliding step of 2, resulting in r feature maps O of the shape of [(t-k+1) / 2]x1, which are expressed as follows by formula (6):

[0056] O=ma×{O i ,O i+1}(i=1,3,5,…,tk) (6)

[0057] These r feature maps are the features extracted by the convolutional network layer, which are reduced to a length of r. * A real number vector of (t-k+1) / 2 is stored in this vector, which stores the spatial relationship between different eigenvalues ​​in the sample data and serves as the input of the multi-layer LSTM layer.

[0058] Specifically, the third layer is a multi-layer LSTM layer. The LSTM model has a memory function and can extract the time series change information of the nonlinear data of the building's cooling and heating loads. It introduces input gates, forget gates, and output gates, and also adds candidate states, cell states, and hidden states. The cell state stores long-term memory, which can alleviate the disappearance of gradients, and the hidden state stores short-term memory. This model uses a multi-layer LSTM. The output of the previous LSTM layer is the input of the next layer, which is passed down layer by layer. The output of the last LSTM hidden layer is used as the input of the attention layer.

[0059] Specifically, the fourth layer is the attention layer. Attention can enhance the role of important time steps in LSTM, thereby further reducing the model prediction error. Attention is essentially to find the weighted average sum of the output vectors of the last layer of LSTM. The output vector of the LSTM hidden layer is used as the input of the attention layer, trained through a fully connected layer, and then the output of the fully connected layer is normalized using the softmax function to obtain the distribution weight of each hidden layer vector. The weight size indicates the importance of the hidden state of each time step to the prediction result.

[0060] The weighted average sum is expressed by formula (7):

[0061]

[0062] Among them, H iis the output of the last LSTM hidden layer, a i is the weight coefficient, C i is the weighted average sum.

[0063] Specifically, the fifth layer is the output layer, which outputs the prediction results, namely, the prediction probabilities corresponding to temperature anomalies, human activity anomalies, and fire anomalies, respectively.

[0064] In an optional implementation, the convolutional network submodule includes:

[0065] The time domain convolution subunit is used to perform time domain convolution based on one-dimensional convolution using a convolution kernel to determine the first spatial feature; the first vector determination subunit is used to determine the vector corresponding to the infrared image based on the first spatial feature by using pooling and dimensionality reduction.

[0066] Specifically, the time domain convolution subunit expresses the convolution operation through formula (8):

[0067]

[0068] Among them, x represents the input feature map, w is the convolution kernel, b is the bias, y is the output feature map, i and j represent the feature map positions respectively, and k represents the output channel.

[0069] Specifically, the pooling operation of the first vector determination subunit includes maximum pooling and average pooling, and the maximum pooling is performed by formula (9):

[0070] y i,j,k =max(x i+m-1,j+n-1,k ) (9)

[0071] The average pooling is expressed by formula (10):

[0072]

[0073] Specifically, the convolutional network submodule also includes: an activation function.

[0074] The activation function is expressed by formula (11):

[0075] ReLU(x)=max(0,x) (11)

[0076] Among them, ReLU(x) represents the activation function.

[0077] In an optional implementation, the convolutional network submodule includes:

[0078] The hole convolution subunit is used to perform hole convolution in response to the hole inserted in the convolution kernel to determine the second spatial feature; the second vector determination subunit is used to determine the vector corresponding to the infrared image based on the second spatial feature by using pooling and dimensionality reduction.

[0079] Specifically, the actual convolution kernel size of the dilated convolution is expressed by formula (12):

[0080] K=k+(k-1)(r-1) (12)

[0081] Among them, K represents the actual convolution kernel size after dilated convolution, k represents the original convolution kernel size, and r represents the dilation rate.

[0082] Atrous convolution for targets of different sizes can help the model extract multi-scale feature information, thereby better adapting to targets of different sizes. Atrous convolution increases the effective size of the convolution kernel in disguise by inserting "holes" (i.e. zero values) into the standard convolution kernel, thereby expanding the receptive field of the convolution layer without increasing the number of actual parameters. The main parameter of atrous convolution is the dilation rate, which determines the number of holes inserted between the convolution kernel elements.

[0083] By using dilated convolution, the receptive field of the convolution kernel can be expanded by inserting holes without increasing the number of parameters and the amount of computation. The receptive field refers to the area in the input space corresponding to a pixel on the output feature map. The increase in the actual convolution kernel size means that the network can capture a wider range of contextual information, thereby improving its ability to understand the input data. Larger actual convolution kernels can capture more global and abstract features, while smaller actual convolution kernels focus more on local details. By adjusting the dilation rate, the size of the actual convolution kernel can be flexibly controlled, thereby achieving better performance in different tasks.

[0084] It should be understood that after determining the actual convolution kernel size of the dilated convolution, the convolution operation is performed in the same manner as in the above embodiment, and the pooling operation in the second quantity determination subunit is the same as in the above embodiment, and will not be repeated herein.

[0085] In an optional implementation, the LSTM submodule includes:

[0086] The weight coefficient determination subunit is used to determine the weight coefficient by calculating the channel weight and the spatial weight; the prediction result determination subunit is used to perform hierarchical transfer based on the weight coefficient and determine the output vector.

[0087] Specifically, the LSTM submodule state update equation group is expressed by formula (13):

[0088]

[0089] Among them, i t represents the input gate, f t represents the forget gate, o trepresents the output gate, C t Indicates the cell state, h t represents the hidden state, x t represents the input data, W i , W f , W c , W o Respectively represent the weight matrix, b i , b f , b c , b o They represent the bias terms respectively.

[0090] Specifically, the output vector is expressed by formula (14):

[0091] h t ,C t =LSTM(F`,h t-1 ,C t-1 ) (14)

[0092] Among them, h t ,c t It represents the output of the last layer of the LSTM submodule, that is, the output vector, and F` represents the weight coefficient.

[0093] Specifically, the weight coefficient is expressed by formula (15):

[0094] F`=F·M c ·M s (15)

[0095] Among them, F represents the input feature map, M c represents the channel weight, M s Represents spatial weight.

[0096] Specifically, the channel weight is expressed by formula (16):

[0097] M c =σ(FC(AvgPool(F)))+σ(FC(MaxPool(F))) (16)

[0098] Among them, σ represents the sigmoid function, FC represents the fully connected layer, AvgPool represents average pooling, and MaxPool represents maximum pooling.

[0099] Specifically, the spatial weight is expressed by formula (17):

[0100] M s =μ(f 7×7 ([AvgPool(F)];MaxPool(F))) (17)

[0101] Among them, f7x7 Represents a convolution operation with a convolution kernel size of 7x7.

[0102] In an alternative embodiment, if Figure 2 As shown, the home environment abnormality monitoring system 10 also includes:

[0103] The filtering module 104 is used to filter the environmental sensor data in response to the collected environmental sensor data and predict the state of the environmental sensor data.

[0104] Specifically, the environmental sensor data includes: one or more of motion and posture data, laser data, ultrasonic data, and sound data. Among them, motion and posture data is used to collect motion state signals in the home environment to achieve precise positioning and navigation; laser data is used to collect distance signals in the home environment, thereby measuring distance information, generating environmental map data, and achieving detection and avoidance; ultrasonic data is used to collect proximity distance signals of detected objects in the home environment, perform short-distance and low-resolution distance detection, achieve auxiliary obstacle avoidance, supplement the deficiencies of laser data, and provide more environmental information; sound data is used to collect sound signals in the home environment to achieve voice recognition and control.

[0105] Specifically, filtering the environmental sensor data includes performing state prediction and error covariance prediction on the environmental sensor data, thereby predicting the state of the environmental sensor data through the state prediction and the error covariance prediction.

[0106] Specifically, the state prediction is expressed by formula (18):

[0107]

[0108] in, represents the predicted state vector at time k, which is used to represent the state estimate at time k-1; e represents the state transfer function, which is used to represent the nonlinear state change model from time k-1 to time k; represents the state estimate at time k-1, which is used to represent the optimal state estimate known at time k-1; u k-1 Represents the control input at time k-1, which is used to represent the external input applied to the system at time k-1.

[0109] Specifically, the error covariance prediction is expressed by formula (19):

[0110]

[0111] Among them, P k|k-1 represents the prediction error covariance matrix at time k, which is used to represent the estimated uncertainty of the prediction state; F k-1The Jacobian matrix representing the state transfer function e, that is, the matrix of the derivative of the state vector, is calculated at time k-1 and is used to represent the linear approximation of the nonlinear system in the current state; P k-1|k-1 represents the error covariance matrix at time k-1, which is used to represent the estimation uncertainty at time k-1; represents the transpose of the Jacobian matrix; Q k-1 represents the process noise covariance matrix, which is used to represent the impact of system noise on state estimation at time k-1.

[0112] Specifically, F k-1 is the Jacobian matrix of the state transfer function e for the state x:

[0113]

[0114] Among them, x k is the state vector at time k, u k-1 is the control input variable at time k-1.

[0115] In an optional implementation, the abnormal monitoring report generation module further includes:

[0116] The abnormal report generation submodule is used to generate an abnormal monitoring report based on the corresponding relationship between the prediction results, status and abnormal data.

[0117] Specifically, generating an abnormal monitoring report based on the correspondence between prediction results, status and abnormal data means using the prediction probabilities and status corresponding to temperature abnormalities, human activity abnormalities and fire abnormalities respectively, and through the correspondence between the thresholds corresponding to the respective prediction probabilities, and when any prediction probability or status exceeds the respective corresponding threshold, generating an abnormal monitoring report corresponding to the prediction probability.

[0118] According to an embodiment of the present invention, an embodiment of a method for monitoring abnormalities in a home environment based on infrared sensing is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0119] This embodiment provides a method for monitoring abnormalities in a home environment based on infrared sensing. Figure 3 As shown, the method comprises the following steps:

[0120] S201, in response to the collected infrared image data, preprocess the infrared image data to determine the infrared image. The specific process can refer to the relevant description of the preprocessing module 101 in the above embodiment, which will not be repeated here.

[0121] S202: Based on the infrared image, a prediction result is obtained by using a network model. The specific process can be found in the description of the prediction module 102 in the above embodiment, which will not be described in detail here.

[0122] S203: Generate an abnormality monitoring report based on the corresponding relationship between the prediction result and the abnormal data. The specific process can refer to the relevant description of the abnormality detection report generation module 103 in the above embodiment, which will not be repeated here.

[0123] The present invention provides a method for monitoring abnormalities in a home environment based on infrared sensing, which preprocesses the collected infrared image data to obtain an infrared image, extracts features from the infrared image using a network model, determines a prediction result corresponding to an abnormality in the home environment, and generates an abnormality monitoring report based on the corresponding relationship between the prediction result and the abnormal data. In this process, by preprocessing the infrared image data, normalization and wavelet transform are used to make the infrared image more conducive to the convergence of the network model and improve the generalization ability, and a network model is formed in the form of a convolutional network, LSTM and an attention mechanism, and then the network model is used to extract features from the infrared image to improve the accuracy of abnormal data recognition in the home environment, thereby generating an abnormality monitoring report based on the corresponding relationship between the prediction result and the abnormal data.

[0124] In an optional implementation, generating an abnormal monitoring report based on the corresponding relationship between the prediction result and the abnormal data also includes:

[0125] In response to the collected environmental sensor data, the environmental sensor data is filtered to predict the state of the environmental sensor data; based on the corresponding relationship between the prediction result, the state and the abnormal data, an abnormal monitoring report is generated.

[0126] Specifically, in response to the collected environmental sensor data, the environmental sensor data is filtered, and the specific process of predicting the state of the environmental sensor data can refer to the relevant description of the filtering module in the above embodiment, which will not be repeated here. Based on the corresponding relationship between the prediction result, state and abnormal data, the specific process of generating an abnormal monitoring report can refer to the relevant description of the abnormal report generation submodule in the above embodiment, which will not be repeated here.

[0127] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A home environment abnormality monitoring system based on infrared sensing, characterized in that: The system comprises: A preprocessing module, configured to preprocess the infrared image data in response to the collected infrared image data to determine an infrared image; A prediction module, used to obtain a prediction result based on the infrared image using a network model; The abnormal monitoring report generating module is used to generate an abnormal monitoring report based on the corresponding relationship between the prediction result and the abnormal data.

2. The system according to claim 1, characterized in that The preprocessing module comprises: A normalization submodule, for performing mean-variance normalization and minimum-maximum normalization on the infrared image data in response to the collected infrared image data, to determine normalized data; The wavelet transform submodule is used to perform wavelet transform on the normalized data using wavelet coefficients and discrete wavelet coefficients to determine the infrared image.

3. The system according to claim 1, characterized in that The prediction module comprises: A convolutional network submodule, used for extracting spatial features of the infrared image and determining a vector corresponding to the infrared image; An LSTM submodule, for processing long-term dependencies based on the vector and determining an output vector; The attention submodule is used to determine a weighted average sum, and determine a prediction result based on the weighted average sum and the output vector.

4. The system according to claim 3, characterized in that The convolutional network submodule comprises: A time domain convolution subunit, configured to perform time domain direction convolution based on one-dimensional convolution using a convolution kernel to determine a first spatial feature; The first vector determination subunit is used to determine the vector corresponding to the infrared image based on the first spatial feature by using pooling and dimensionality reduction.

5. The system according to claim 3, characterized in that The convolutional network submodule includes: a dilated convolution subunit, configured to perform dilated convolution in response to a hole inserted in the convolution kernel to determine a second spatial feature; The second vector determination subunit is used to determine the vector corresponding to the infrared image based on the second spatial feature by using pooling and dimensionality reduction.

6. The system according to claim 3, characterized in that The LSTM submodule includes: A weight coefficient determination subunit, used to determine the weight coefficient by calculating the channel weight and the spatial weight; The prediction result determination subunit is used to perform hierarchical transfer based on the weight coefficient to determine the output vector.

7. The system according to claim 1, characterized in that The system further comprises: The filtering module is used to filter the environmental sensor data in response to the collected environmental sensor data and predict the state of the environmental sensor data.

8. The system according to claim 7, characterized in that The abnormal monitoring report generating module further includes: The abnormal report generation submodule is used to generate an abnormal monitoring report based on the corresponding relationship between the prediction result, the state and the abnormal data.

9. A method for monitoring abnormal home environment based on infrared sensing, characterized in that: The method comprises: In response to the collected infrared image data, preprocessing the infrared image data to determine an infrared image; Based on the infrared image, using the network model, obtaining a prediction result; Based on the corresponding relationship between the prediction results and the abnormal data, an abnormal monitoring report is generated.

10. The method according to claim 9, characterized in that The generating of an abnormal monitoring report based on the corresponding relationship between the prediction result and the abnormal data also includes: In response to the collected environmental sensor data, filtering the environmental sensor data to predict the state of the environmental sensor data; Based on the prediction result, the corresponding relationship between the state and the abnormal data, an abnormal monitoring report is generated.

Images