A distributed boiler room video intelligent analysis method and device

Abstract

The present invention provides a distributed boiler room video intelligent analysis method and device, which relates to the field of data processing technology. The method includes: weighting the distinguishing features to obtain key features; inputting the key features into the fully connected layer for classification to obtain a first classification result; according to the first classification result, the probability of each category is calculated, and the probability of each category is compared with a preset probability threshold to obtain a probability comparison result; according to the probability comparison result, the same image is classified multiple times to obtain multiple second classification results; the multiple second classification results are fused to obtain a fusion result; the fusion result is compared with the historical verification data to obtain a verification result; according to the verification result, the identity of the face is determined. The present invention can realize accurate recognition of the face.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a distributed boiler room video intelligent analysis method and device. Background Art

[0002] With the continuous development of high-definition video surveillance technology, a good foundation has been laid for the application of intelligent technology. In the field of heating, intelligent technologies such as regional intrusion, personnel operation control (such as safety helmet detection, behavior analysis, etc.), video inspection and other technologies have been gradually applied, bringing great convenience to users.

[0003] Through intelligent analysis, we can achieve refined real-time dynamic management of various personnel identity verification and abnormal behavior, provide early warning, and reduce the risk of accidents. However, there are many problems with traditional boiler room safety management, such as difficulty in monitoring personnel entry and exit. In addition, some video analysis methods have poor analysis effects on different scenarios and cannot meet the needs of distributed boiler rooms. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a distributed boiler room video intelligent analysis method and device, which can realize accurate recognition of human faces.

[0005] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0006] In a first aspect, a distributed boiler room video intelligent analysis method is provided, the method comprising:

[0007] Preprocessing the acquired image to obtain a preprocessed image;

[0008] A multi-level convolutional neural network is used to extract features from the preprocessed image. Through multi-level convolutional layers and pooling layers, features at different levels are extracted to obtain distinguishing features.

[0009] Weight the distinguishing features to obtain key features;

[0010] Input the key features into the fully connected layer for classification to obtain the first classification result;

[0011] According to the first classification result, the probability of each category is calculated, and the probability of each category is compared with a preset probability threshold to obtain a probability comparison result;

[0012] According to the probability comparison results, the same image is classified multiple times to obtain multiple second classification results;

[0013] Fusing multiple second classification results to obtain a fusion result;

[0014] Compare the fusion results with historical verification data to obtain verification results;

[0015] Based on the verification result, the identity of the face is determined.

[0016] Furthermore, a multi-level convolutional neural network is used to extract features from the preprocessed image. Through multi-level convolutional layers and pooling layers, features at different levels are extracted to obtain distinguishing features, including:

[0017] The preprocessed image is input to the first convolution layer of the convolutional neural network. The first convolution layer uses multiple different convolution kernels to capture the first feature in the image. Each convolution kernel slides on the image and performs a convolution operation with the local area of ​​the image to obtain a feature map. Multiple convolution kernels generate multiple feature maps to extract multiple features of the image.

[0018] Inputting the multiple features into a second convolution layer, wherein the second convolution layer uses a convolution kernel to extract a graphic contour and a texture of the feature map to obtain a second feature;

[0019] Inputting the second feature into a third convolutional layer, wherein the third convolutional layer uses a convolution kernel to extract the object category of the feature map to obtain a third feature;

[0020] The second features and third features at different levels are concatenated to obtain distinctive features of the image.

[0021] Furthermore, the distinguishing features are weighted to obtain key features, including:

[0022] Construct a fully connected layer and extract the distinguishing features of the image through a convolutional neural network to obtain a feature vector after concatenating features at different levels. The number of nodes in the fully connected layer is the same as the length of the feature vector, and the first weight matrix of the fully connected layer learns the weight of each feature vector element.

[0023] Training the first weight matrix using a gradient descent and back propagation algorithm to obtain a second weight matrix;

[0024] The feature vector is multiplied by the second weight matrix to obtain the weighted key features.

[0025] Furthermore, the key features are input into the fully connected layer for classification to obtain the first classification result, including:

[0026] Build a classifier including fully connected layers and output layers;

[0027] Inputting the key features into the fully connected layer to obtain feature representation;

[0028] Inputting the feature representation into the output layer, and performing feature conversion on the feature representation to obtain a predicted probability of a category;

[0029] According to the predicted probability, a first classification result is obtained.

[0030] Further, according to the first classification result, the probability of each category is calculated, and the probability of each category is compared with a preset probability threshold to obtain a probability comparison result, including:

[0031] According to the first classification result, for each category i, obtain the probability p predicted by the classifier i ;

[0032] For each probability p i Compare with the preset probability threshold t respectively. If the probability p i ≥ probability threshold t, then the probability p i The corresponding category i is predicted if the probability p i < probability threshold t, then the probability p i The corresponding category i is not predicted;

[0033] The probability p corresponding to all categories i = 1, 2, ..., n i Compare with the threshold t respectively to obtain n comparison results.

[0034] Furthermore, according to the probability comparison result, the same image is classified multiple times to obtain multiple second classification results, including:

[0035] Based on the probability comparison results, different convolutional neural network models are used for classification;

[0036] Extract different regions from the image for classification;

[0037] The image is rotated and scaled to different degrees and then classified to obtain the second classification results of multiple classifiers for the same image.

[0038] Furthermore, each second classification result has one or more predicted categories and corresponding probabilities; multiple second classification results are fused to obtain a fusion result, including:

[0039] Aggregate multiple second classification results by category;

[0040] For each category, calculate how many classifiers have output this category to get the number of votes for this category, and take a weighted average of the probability predictions of each classifier for this category to get the comprehensive probability of this category;

[0041] The fusion result is obtained based on the number of votes and the comprehensive probability of all categories.

[0042] In a second aspect, a distributed boiler room video intelligent analysis device includes:

[0043] The acquisition module is used to preprocess the input image to obtain a preprocessed image; use a multi-level convolutional neural network to extract features of the preprocessed image, extract features at different levels through a multi-level convolutional layer and a pooling layer to obtain distinguishing features; weight the distinguishing features to obtain key features; input the key features to a fully connected layer for classification to obtain a first classification result;

[0044] The processing module is used to calculate the probability of each category according to the first classification result, and compare the probability of each category with the preset probability threshold to obtain a probability comparison result; classify the same image multiple times according to the probability comparison result to obtain multiple second classification results; fuse the multiple second classification results to obtain a fusion result; and compare the fusion result with the historical verification data to obtain a verification result.

[0045] According to a third aspect, a computing device includes:

[0046] one or more processors;

[0047] The storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the above method.

[0048] In a fifth aspect, a computer-readable storage medium stores a program, and the program implements the above method when executed by a processor.

[0049] The above solution of the present invention includes at least the following beneficial effects:

[0050] The above scheme of the present invention adopts a multi-level convolutional neural network for feature extraction, which can extract multi-layer semantic information of the image, which is conducive to accurate image classification, weighting the distinguishing features, highlighting the features that are more important for classification, extracting key features, and improving classification performance. Multiple classifications can make up for the limitations of a single model, and different models and data enhancements can provide more comprehensive judgments. Comparing the classification probability and threshold, the reliability of a single classification can be evaluated, and the fusion of multiple classification results can obtain a more accurate and stable final classification conclusion. Comparing and verifying the fusion results with historical data can further reduce classification errors and improve the reliability of the results. The distributed system can use multiple nodes for parallel and efficient video processing to improve overall performance. Intelligent analysis can realize automatic monitoring of the operation of the boiler room, timely discover abnormalities, and ensure safe operation. The present invention combines deep learning with multi-model fusion, can realize intelligent analysis and recognition of boiler room videos, has a high degree of automation, and is conducive to improving the intelligence level of the boiler room. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a flow chart of a distributed boiler room video intelligent analysis method provided by an embodiment of the present invention.

[0052] Figure 2 It is a schematic diagram of a distributed boiler room video intelligent analysis device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0054] like Figure 1 As shown, an embodiment of the present invention proposes a distributed boiler room video intelligent analysis method, the method comprising:

[0055] Step 11, preprocessing the acquired image to obtain a preprocessed image;

[0056] Step 12, using a multi-level convolutional neural network to extract features from the preprocessed image, extracting features at different levels through multi-level convolutional layers and pooling layers to obtain distinguishing features;

[0057] Step 13, weighting the distinguishing features to obtain key features;

[0058] Step 14, inputting the key features into the fully connected layer for classification to obtain a first classification result;

[0059] Step 15, according to the first classification result, the probability of each category is calculated, and the probability of each category is compared with a preset probability threshold to obtain a probability comparison result;

[0060] Step 16, classifying the same image multiple times according to the probability comparison result to obtain multiple second classification results;

[0061] Step 17, fusing multiple second classification results to obtain a fusion result;

[0062] Step 18, comparing the fusion result with the historical verification data to obtain the verification result;

[0063] Step 19: Determine the identity of the face based on the verification result.

[0064] In the embodiment of the present invention, a multi-level convolutional neural network is used for feature extraction, which can extract multi-layer semantic information of the image, which is conducive to accurate image classification. Weighting the distinguishing features can highlight the features that are more important for classification, extract key features, and improve classification performance. Multiple classifications can make up for the limitations of a single model. Different models and data enhancements can provide more comprehensive judgments. Comparing the classification probability and threshold can evaluate the reliability of a single classification. The fusion of multiple classification results can obtain a more accurate and stable final classification conclusion. Comparing and verifying the fusion results with historical data can further reduce classification errors and improve the reliability of the results. The distributed system can use multiple nodes for parallel and efficient video processing to improve overall performance. Intelligent analysis can realize automatic monitoring of the operation of the boiler room, timely detect abnormalities, and ensure safe operation. The present invention combines deep learning with multi-model fusion, can realize intelligent analysis and recognition of boiler room videos, has a high degree of automation, and is conducive to improving the intelligence level of the boiler room.

[0065] In another preferred embodiment of the present invention, the above step 11 may include: The acquired image is equalized to obtain an equalized image, wherein: W(k5,l)=G(k5,l)×I(k5,l), (x,y) represents the coordinates of the current center pixel, k5 and l represent the relative offset of the pixels in the window relative to the current pixel (x,y), and Y eq is the equalized image, Y is the original image, w3 is the half-width of the window, M3 is the mean pixel value in the window, W is the weighting function, G is the Gaussian weight, and I is the indicator function. eq (x, y) = G(x, y) * I eq(x, y) is the equalized image Y eq Gaussian smoothing is performed to obtain a smooth image, where G(x, y) is the Gaussian smoothing kernel, * represents the convolution operation, and I′ eq For smoothed images, I eq is the original image; through I″ eq (x, y) = [I′ eq (x,y)]↓ s Downsample the smoothed image to obtain a downsampled image I″ eq , where ↓ s Represents a downsampling operation, s is the downsampling factor, and the operation is repeated to iteratively generate an image pyramid of an adaptively equalized image.

[0066] In a preferred embodiment of the present invention, the above step 12 may include:

[0067] Step 121, inputting the preprocessed image into the first convolution layer of the convolutional neural network, wherein the first convolution layer uses a plurality of different convolution kernels to capture a first feature in the image, wherein each convolution kernel slides on the image and performs a convolution operation with a local area of ​​the image to obtain a feature map, and a plurality of convolution kernels generate a plurality of feature maps to extract a plurality of features of the image;

[0068] Step 122, inputting the multiple features into a second convolutional layer, wherein the second convolutional layer uses a convolution kernel to extract a graphic contour and a texture of the feature map to obtain a second feature;

[0069] Step 123, inputting the second feature into a third convolutional layer, wherein the third convolutional layer uses a convolution kernel to extract the object category of the feature map to obtain a third feature;

[0070] Step 124, concatenating the second features and third features at different levels to obtain distinctive features of the image.

[0071] In an embodiment of the present invention, the first convolution layer can capture low-level features of the image such as edges, corners, etc., and extract basic visual features. The second convolution layer extracts higher-level features such as graphic contours and textures based on the first-level features. The third convolution layer can extract higher-level semantic features such as global information such as object categories. Through multi-layer convolution operations, feature information at different levels of the image can be extracted, and features at different levels can be spliced ​​to obtain a feature vector containing rich and diverse features. This feature vector has both low-level features and high-level features, which provides strong support for subsequent image analysis tasks such as classification and detection. The process of extracting and fusing features layer by layer in the convolution network simulates the process of the human eye gradually understanding and abstracting image information. Overall, by using multi-layer convolutional networks to extract image features, a feature representation with rich semantics and strong discrimination can be obtained, which greatly improves the performance of subsequent tasks. This multi-layer convolutional network feature extraction method can obtain high-quality feature representation of the image.

[0072] In another preferred embodiment of the present invention, the above step 121 may include: assuming that the first convolution layer has K1 convolution kernels, then the parameter of the kth convolution kernel is a three-dimensional matrix The convolution kernel size is M4×N3, the input image is I, and the size is H×W×C, where C is the number of image channels, H is the height of the image, and W is the width of the image. The calculation of the convolution of the kth convolution kernel and the input image is: in, is the output feature map of the kth convolution kernel, is the weight parameter of the kth convolution kernel, I(x+i1, y+j1, c) is the value of the cth channel of the input image at position (x+i1, y+j1), and after the convolution operation of K1 convolution kernels, K1 feature maps are obtained, representing the different features of the input image.

[0073] In another preferred embodiment of the present invention, the above step 122 may include: assuming that the second convolution layer has K2 convolution kernels, and the parameters of the kth convolution kernel are The convolution kernel size is M5×N5, and the feature map produced by the first layer is in, The size is H1×W1, then the calculation of the convolution of the kth convolution kernel in the second layer and the feature map in the first layer is: in, is the output feature map of the kth convolution kernel, is the weight parameter of the kth convolution kernel to the mth input feature map, H1 represents the height of the second layer feature map, W1 represents the width of the second layer feature map, It is the value of the mth input feature map at the position (x+i2, y+j2). After the convolution operation of K2 convolution kernels, K2 feature maps representing the contour, texture and other features of the graphic can be obtained.

[0074] In another preferred embodiment of the present invention, the above step 123 may include: assuming that the third convolution layer has K3 convolution kernels, and the parameters of the kth convolution kernel are The convolution kernel size is M6×N6, and the feature map produced by the second layer is in, The size is H2×W2, then the calculation of the convolution of the kth convolution kernel in the third layer and the feature map in the second layer is: in, is the output feature map of the kth convolution kernel, is the weight parameter of the kth convolution kernel on the mth input feature map, is the value of the mth input feature map at position (x+i3, y+j3), H2 represents the height of the third-layer feature map, W2 represents the width of the third-layer feature map, and after the convolution operation of K3 convolution kernels, K3 feature maps representing high-level semantic features such as object categories can be obtained.

[0075] In another preferred embodiment of the present invention, the above step 124 may include: assuming that the second layer generates K2 feature maps in, The size is H1×W1, and the third layer generates K3 feature maps in, The size is H2×W2, then the feature concatenation at different levels can be expressed as: Among them, vec() represents vectorization operation, [;] represents concatenation operation, and the obtained F concat It is a vector of size (K2×H1×W1+K3×H2×W2), which contains feature information at different levels, that is, the distinguishing features of the image.

[0076] In a preferred embodiment of the present invention, the above step 13 may include:

[0077] Step 131, construct a fully connected layer, and extract the distinguishing features of the image through a convolutional neural network to obtain a feature vector after the features at different levels are concatenated, wherein the number of nodes in the fully connected layer is the same as the length of the feature vector, and the first weight matrix of the fully connected layer learns the weight of each feature vector element;

[0078] Step 132, training the first weight matrix using a gradient descent and back propagation algorithm to obtain a second weight matrix;

[0079] Step 133, multiplying the feature vector by the second weight matrix to obtain weighted key features.

[0080] In an embodiment of the present invention, a fully connected layer is constructed and a weight matrix is ​​learned, and features at different levels can be re-evaluated and weighted. The weight matrix can be trained through gradient descent and back-propagation algorithms to automatically learn the importance of each feature. The weighted features highlight the semantic information that is more important to the current task, and unimportant features are automatically suppressed, which plays a role in feature selection. The key features obtained are more discriminative, and redundant and noise features are discarded. The key features are used as inputs for subsequent classification or detection, which can improve task performance. The feature selection and weighting process simulates the human eye's attention mechanism for the scene. Therefore, the present invention realizes automatic learning of feature weights through a fully connected layer, obtains key features, provides effective support for image understanding tasks, integrates deep learning and feature engineering, and realizes end-to-end image understanding and analysis. The feature weighted selection method based on the fully connected layer of the present invention can obtain a feature subset that is more sensitive to the task, thereby improving the effect of subsequent analysis.

[0081] In another preferred embodiment of the present invention, when specifically applied, step 131 includes: constructing a convolutional neural network including 3 convolutional layers and 2 pooling layers, extracting multi-level features of the image, using 32 5×5 filters in the convolutional layer, using a 2×2 pooling window in the pooling layer, with a step size of 2, flattening and splicing the output feature maps of the 3 convolutional layers to obtain a feature vector of length 15360, constructing a fully connected layer including 15360 nodes corresponding to the feature vector length, and the weight matrix of the fully connected layer is 15360×15360 in size, initialized to a random value. Step 132 includes: using the image feature vector as the input of the fully connected layer, obtaining the output by matrix multiplication, calculating the cross entropy loss of the output category, using the Adam optimizer with a learning rate of 0.001, updating the weights, and repeating the update until the loss function converges to obtain a trained weight matrix. Step 133 includes: multiplying the image feature vector by the weight matrix to obtain a weighted 15360-dimensional key feature, where each element in the key feature reflects the importance of the feature to the classification task.

[0082] In a preferred embodiment of the present invention, the above step 14 may include:

[0083] Step 141, constructing a classifier including a fully connected layer and an output layer;

[0084] Step 142, inputting the key features into the fully connected layer to obtain feature representation;

[0085] Step 143, inputting the feature representation into the output layer, performing feature conversion on the feature representation to obtain a predicted probability of a category;

[0086] Step 144, obtaining a first classification result according to the predicted probability.

[0087] In an embodiment of the present invention, a classifier structure including a fully connected layer and an output layer is constructed to achieve end-to-end feature extraction and classification. Key features rather than original features are input into the classifier to improve classification performance. The fully connected layer learns high-level semantic feature representations, and the output layer realizes feature conversion to category probability space. The classification result is obtained based on the predicted probability to achieve understanding of the image content. The selectivity of key features enables the classifier to focus on information that is more important to the current task. The classifier extracts the feature representation required for classification through end-to-end learning. The classification result reflects the semantic level understanding of the image content. Overall, the construction of the classifier makes it possible to automatically analyze the image content, greatly reducing the manual workload. This method integrates feature engineering, representation learning and classification recognition to make image understanding more intelligent and automated.

[0088] In another preferred embodiment of the present invention, when specifically applied, step 141 includes constructing a classifier including 1 fully connected layer and 1 softmax output layer, the fully connected layer includes 128 nodes, the weights are initialized to random values, and the softmax layer includes 10 nodes, which respectively represent the predicted probabilities of 10 categories. Step 142 includes inputting the key feature vector of length 15360 into the fully connected layer, and the fully connected layer matrix multiplication obtains a feature representation of length 128. Step 143 includes inputting the feature representation of length 128 into the softmax layer, and the softmax layer performs an exponential normalization operation on the feature representation to obtain a probability vector of length 10, which represents the predicted probabilities of 10 categories. Step 144 includes finding the category corresponding to the value with the largest probability in the probability vector, and outputting the category as the classification result of the image.

[0089] In a preferred embodiment of the present invention, the above step 15 may include:

[0090] Step 151, based on the first classification result, for each category i, obtain the probability p predicted by the classifier i ;

[0091] Step 152: each probability p i Compare with the preset probability threshold t respectively. If the probability p i ≥ probability threshold t, then the probability p i The corresponding category i is predicted if the probability p i < probability threshold t, then the probability p i The corresponding category i is not predicted;

[0092] Step 153: calculate the probability p corresponding to all categories i=1, 2, ..., ni Compare with the threshold t respectively to obtain n comparison results.

[0093] In the embodiment of the present invention, the prediction probability of each category is obtained, the confidence of the classifier output can be quantified, the probability threshold is set, the sensitivity of the classification can be adjusted according to the confidence, and the probability and threshold are compared to evaluate whether the classification result of each category is reliable. Threshold comparison is performed on all categories to locate categories with low classification reliability. The comparison result reflects the prediction confidence of the classifier for each category, and subsequent processing can be performed based on the confidence, such as requesting manual verification, reclassification, etc., and more data can be collected to improve the classification confidence. Overall, this method can evaluate the reliability of the classification results, and combined with the threshold judgment of the classification probability, the classification results are more robust and reliable.

[0094] In another preferred embodiment of the present invention, when it is specifically applied, step 151 may include, the classifier outputs a probability vector with a length of 10, representing the predicted probabilities of 10 categories, and each value pi in the vector is used as the predicted probability of the corresponding category i. Step 152 may include, setting the probability threshold t to 0.7, and for each probability p i , if the probability p i ≥0.7, then category i is considered to be predicted if the probability p i <0.7, it is considered that category i is not predicted. Step 153 may include, for the first category, probability p1=0.8≥0.7, the first category is predicted, for the second category, probability p2=0.6<0.7, the second category is not predicted, and the probability p is calculated for the 10 categories in turn. i The comparison with the threshold t results in a comparison result vector of length 10, indicating whether each category is predicted.

[0095] In a preferred embodiment of the present invention, the above step 16 may include:

[0096] Step 161, using different convolutional neural network models for classification according to the probability comparison result;

[0097] Step 162, extracting different regions from the image for classification;

[0098] Step 163, rotating and scaling the image to different degrees and then classifying it to obtain second classification results of the same image by multiple classifiers.

[0099] In an embodiment of the present invention, using different models for classification can provide a multi-dimensional perspective and make up for the limitations of a single model. Extracting features from different areas of an image for classification can enhance the use of local information. Classifying after transforming the image can improve the robustness to rotation and scaling. The results of multiple classifiers can be fused to improve classification performance. The probability comparison results can guide the selection of classification results to obtain more reliable results. Multiple classifications combined with probability threshold judgments can make the results more stable and reliable. Different models, different areas, and different transformations provide multiple sources of evidence. Therefore, the multiple classifications combined with probability evaluation of the present invention can make up for the limitations of a single classification and improve classification accuracy. The present invention combines the advantages of multiple models to achieve robust judgments on image content.

[0100] In a preferred embodiment of the present invention, each second classification result has one or more predicted categories and corresponding probabilities; the above step 17 may include:

[0101] Step 171, summarizing the multiple second classification results according to categories;

[0102] Step 172, for each category, calculate how many classifiers have output the category to obtain the number of votes for the category, and perform a weighted average of the probability predictions of each classifier for the category to obtain a comprehensive probability of the category;

[0103] Step 173, obtaining the fusion result according to the number of votes and the comprehensive probability of all categories.

[0104] In an embodiment of the present invention, multiple classification results are summarized by category, and statistical analysis can be performed intuitively. The number of votes for each category can be calculated, and the degree of support for the category by different classifiers can be evaluated. The probability is weighted averaged, and the predicted values ​​of different classifiers can be combined. The number of votes and the comprehensive probability can evaluate the reliability of category judgment from different dimensions. The fusion result is obtained according to the number of votes and the probability, which can make the result more robust. The fusion result can integrate the advantages of multiple classifiers and make up for the shortcomings of a single classifier. The final result can improve the accuracy of classification compared with a single classifier. The present invention realizes the effective aggregation of multiple classifier results, making the classification more robust to the input, and the fusion of multiple classification results improves the stability and reliability of classification.

[0105] like Figure 2 As shown, the embodiment of the present invention further provides a distributed boiler room video intelligent analysis device 20, comprising:

[0106] The acquisition module 21 is used to preprocess the input image to obtain a preprocessed image; use a multi-level convolutional neural network to extract features of the preprocessed image, extract features at different levels through a multi-level convolutional layer and a pooling layer to obtain distinguishing features; weight the distinguishing features to obtain key features; input the key features to a fully connected layer for classification to obtain a first classification result;

[0107] The processing module 22 is used to calculate the probability of each category based on the first classification result, and compare the probability of each category with the preset probability threshold to obtain a probability comparison result; classify the same image multiple times based on the probability comparison result to obtain multiple second classification results; fuse the multiple second classification results to obtain a fusion result; and compare the fusion result with the historical verification data to obtain a verification result.

[0108] When applied specifically, a distributed boiler room video intelligent analysis device also includes a camera, an image processing module, a face recognition module, a personnel status recognition module, an open flame detection module, an alarm prompt module and an intelligent heating management module. The camera is used to collect video signals in the boiler room and transmit the video signals to the image processing module. The image processing module is used to process the received video signals and extract information related to faces and open flames. The face recognition module accurately recognizes the extracted faces by using a specific recognition algorithm, judges the legitimacy of the personnel leaving the boiler room, and generates corresponding records. The personnel status recognition module detects whether there are personnel in the area, and non-personnel are not triggered by mistake. It can also detect and alarm whether the personnel in the area are running fast and whether there are personnel falling to the ground. The open flame detection module detects the extracted open flames by using a specific detection algorithm. Once an open flame is detected, the alarm prompt module is triggered to send an alarm signal. The alarm prompt module can alarm by sound, light, etc., and send the alarm information to relevant personnel to realize the intelligent heating management.

[0109] Optionally, a multi-level convolutional neural network is used to extract features from the preprocessed image. Through multi-level convolutional layers and pooling layers, features at different levels are extracted to obtain distinguishing features, including:

[0110] The preprocessed image is input to the first convolution layer of the convolutional neural network. The first convolution layer uses multiple different convolution kernels to capture the first feature in the image. Each convolution kernel slides on the image and performs a convolution operation with the local area of ​​the image to obtain a feature map. Multiple convolution kernels generate multiple feature maps to extract multiple features of the image.

[0111] Inputting the multiple features into a second convolution layer, wherein the second convolution layer uses a convolution kernel to extract a graphic contour and a texture of the feature map to obtain a second feature;

[0112] Inputting the second feature into a third convolutional layer, wherein the third convolutional layer uses a convolution kernel to extract the object category of the feature map to obtain a third feature;

[0113] The second features and third features at different levels are concatenated to obtain distinctive features of the image.

[0114] Optionally, the distinguishing features are weighted to obtain key features, including:

[0115] Construct a fully connected layer and extract the distinguishing features of the image through a convolutional neural network to obtain a feature vector after concatenating features at different levels. The number of nodes in the fully connected layer is the same as the length of the feature vector, and the first weight matrix of the fully connected layer learns the weight of each feature vector element.

[0116] Training the first weight matrix using a gradient descent and back propagation algorithm to obtain a second weight matrix;

[0117] The feature vector is multiplied by the second weight matrix to obtain the weighted key features.

[0118] Optionally, the key features are input into a fully connected layer for classification to obtain a first classification result, including:

[0119] Build a classifier including fully connected layers and output layers;

[0120] Inputting the key features into the fully connected layer to obtain feature representation;

[0121] Inputting the feature representation into the output layer, and performing feature conversion on the feature representation to obtain a predicted probability of a category;

[0122] According to the predicted probability, a first classification result is obtained.

[0123] Optionally, according to the first classification result, the probability of each category is calculated, and the probability of each category is compared with a preset probability threshold to obtain a probability comparison result, including:

[0124] According to the first classification result, for each category i, obtain the probability p predicted by the classifier i ;

[0125] For each probability p i Compare with the preset probability threshold t respectively. If the probability p i ≥ probability threshold t, then the probability p i The corresponding category i is predicted if the probability p i < probability threshold t, then the probability p i The corresponding category i is not predicted;

[0126] The probability p corresponding to all categories i = 1, 2, ..., n i Compare with the threshold t respectively to obtain n comparison results.

[0127] Optionally, the same image is classified multiple times according to the probability comparison result to obtain multiple second classification results, including:

[0128] Based on the probability comparison results, different convolutional neural network models are used for classification;

[0129] Extract different regions from the image for classification;

[0130] The image is rotated and scaled to different degrees and then classified to obtain the second classification results of multiple classifiers for the same image.

[0131] Optionally, each second classification result has one or more predicted categories and corresponding probabilities; multiple second classification results are fused to obtain a fusion result, including:

[0132] Aggregate multiple second classification results by category;

[0133] For each category, calculate how many classifiers have output this category to get the number of votes for this category, and take a weighted average of the probability predictions of each classifier for this category to get the comprehensive probability of this category;

[0134] The fusion result is obtained based on the number of votes and the comprehensive probability of all categories.

[0135] It should be noted that the device is a device corresponding to the above method, and all implementation methods in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0136] The embodiment of the present invention further provides a computing device, comprising: a processor, a memory storing a computer program, wherein when the computer program is executed by the processor, the method described above is executed. All implementations in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0137] The embodiment of the present invention also provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the method described above. All implementations in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0138] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design conditions of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0139] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0140] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0141] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0142] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0143] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical disks.

[0144] In addition, it should be noted that in the apparatus and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. Moreover, the steps of performing the above-mentioned series of processing can naturally be performed in chronological order according to the order of description, but it is not necessary to perform them in chronological order, and some steps can be performed in parallel or independently of each other. For those of ordinary skill in the art, it is understood that all or any steps or components of the method and apparatus of the present invention can be implemented in any computing device (including processors, storage media, etc.) or a network of computing devices in hardware, firmware, software or a combination thereof, which can be achieved by those of ordinary skill in the art using their basic programming skills after reading the description of the present invention.

[0145] Therefore, the purpose of the present invention can also be achieved by running a program or a group of programs on any computing device. The computing device can be a well-known general device. Therefore, the purpose of the present invention can also be achieved by simply providing a program product containing a program code that implements the method or device. That is to say, such a program product also constitutes the present invention, and the storage medium storing such a program product also constitutes the present invention. Obviously, the storage medium can be any well-known storage medium or any storage medium developed in the future. It should also be pointed out that in the device and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. In addition, the steps of performing the above-mentioned series of processing can naturally be performed in chronological order according to the order of description, but it is not necessary to perform them in chronological order. Some steps can be performed in parallel or independently of each other.

[0146] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A distributed boiler room video intelligent analysis method, characterized in that: The method comprises: Preprocessing the acquired image to obtain a preprocessed image; A multi-level convolutional neural network is used to extract features from the preprocessed image. Through multi-level convolutional layers and pooling layers, features at different levels are extracted to obtain distinguishing features. Weight the distinguishing features to obtain key features; Input the key features into the fully connected layer for classification to obtain the first classification result; According to the first classification result, the probability of each category is calculated, and the probability of each category is compared with a preset probability threshold to obtain a probability comparison result; According to the probability comparison results, the same image is classified multiple times to obtain multiple second classification results; Fusing multiple second classification results to obtain a fusion result; Compare the fusion results with historical verification data to obtain verification results; Determine the identity of the face based on the verification result; A multi-level convolutional neural network is used to extract features from the preprocessed image. Through multi-level convolutional layers and pooling layers, features at different levels are extracted to obtain distinguishing features, including: The preprocessed image is input to the first convolution layer of the convolutional neural network. The first convolution layer uses multiple different convolution kernels to capture the first feature in the image. Each convolution kernel slides on the image and performs a convolution operation with the local area of ​​the image to obtain a feature map. Multiple convolution kernels generate multiple feature maps to extract multiple features of the image. Inputting the multiple features into a second convolution layer, wherein the second convolution layer uses a convolution kernel to extract a graphic contour and a texture of the feature map to obtain a second feature; Inputting the second feature into a third convolutional layer, wherein the third convolutional layer uses a convolution kernel to extract the object category of the feature map to obtain a third feature; splicing the second features and third features at different levels to obtain distinguishing features of the image; The distinguishing features are weighted to obtain key features, including: Construct a fully connected layer, and obtain an image feature vector after concatenating features of different levels by distinguishing features, wherein the number of nodes in the fully connected layer is the same as the length of the feature vector, and the first weight matrix of the fully connected layer learns the weight of each element of the image feature vector; Training the first weight matrix using a gradient descent and back propagation algorithm to obtain a second weight matrix; Multiplying the image feature vector by the second weight matrix to obtain weighted key features; The key features are input into the fully connected layer for classification to obtain the first classification result, including: Build a classifier including fully connected layers and output layers; Inputting the key features into the fully connected layer to obtain feature representation; Inputting the feature representation into the output layer, and performing feature conversion on the feature representation to obtain a predicted probability of a category; According to the predicted probability, a first classification result is obtained; According to the first classification result, the probability of each category is calculated, and the probability of each category is compared with the preset probability threshold to obtain a probability comparison result, including: According to the first classification result, for each category i, obtain the probability p predicted by the classifier i ; For each probability p i Compare with the preset probability threshold t respectively. If the probability p i ≥ probability threshold t, then the probability p i The corresponding category i is predicted if the probability p i < probability threshold t, then the probability p i The corresponding category i is not predicted; The probability p corresponding to all categories i = 1, 2, ..., n i Compare with the threshold t respectively to obtain n comparison results; According to the probability comparison results, the same image is classified multiple times to obtain multiple second classification results, including: Based on the probability comparison results, different convolutional neural network models are used for classification; Extract different regions from the image for classification; The image is rotated and scaled to different degrees and then classified to obtain the second classification results of multiple classifiers for the same image; Each second classification result has one or more predicted categories and corresponding probabilities; multiple second classification results are fused to obtain a fusion result, including: Aggregate multiple second classification results by category; For each category, calculate how many classifiers have output this category to get the number of votes for this category, and take a weighted average of the probability predictions of each classifier for this category to get the comprehensive probability of this category; The fusion result is obtained based on the number of votes and the comprehensive probability of all categories.

2. A distributed boiler room video intelligent analysis device, characterized in that: As used in claim 1, comprising: The acquisition module is used to preprocess the input image to obtain a preprocessed image; use a multi-level convolutional neural network to extract features of the preprocessed image, extract features at different levels through a multi-level convolutional layer and a pooling layer to obtain distinguishing features; weight the distinguishing features to obtain key features; input the key features to a fully connected layer for classification to obtain a first classification result; The processing module is used to calculate the probability of each category according to the first classification result, and compare the probability of each category with the preset probability threshold to obtain a probability comparison result; classify the same image multiple times according to the probability comparison result to obtain multiple second classification results; fuse the multiple second classification results to obtain a fusion result; and compare the fusion result with the historical verification data to obtain a verification result.

3. A computing device, characterized in that: include: one or more processors; A storage device, used for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method as claimed in claim 1.

4. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, which implements the method according to claim 1 when executed by a processor.

Images