A method for monitoring and early warning of electric vehicles entering buildings

Abstract

The present invention discloses a method for monitoring and early warning of electric vehicles entering buildings, which relates to the field of smart city Internet of Things and security image recognition. The method comprises the following steps: constructing an electric vehicle database and classifying it: selecting at least 5,000 electric vehicle images, manually preprocessing the images, and dividing them into coarse-grained images and fine-grained images, thereby constructing an electric vehicle database; constructing a convolutional neural network model, including an adversarial noise generator, a coarse-grained recognition channel, and a fine-grained recognition channel; collecting image information, inputting it into the adversarial noise generator for optimization, and then inputting it into the coarse-grained recognition channel for recognition; judging whether the image belongs to the fine-grained category, if so, inputting the recognition spectrum into the fine-grained recognizer for recognition and outputting the result, and judging whether an electric vehicle has entered the building according to the recognition result. The present invention can reduce the omission and false alarm rates.

Description

Technical Field

[0001] The present invention relates to the field of smart city Internet of Things and security image recognition, and in particular to a method for monitoring and early warning of electric vehicles entering buildings. Background Art

[0002] Due to the high population density in cities and the large number of buses and private cars, congestion is prone to occur on a daily basis. Electric vehicles have become an important means of transportation in cities due to their small size, low price and operating costs, and are more convenient and environmentally friendly. The number of electric vehicles has also increased year by year.

[0003] Since electric vehicles require frequent charging, some residents charge them in public passages or at home in order to save charging costs. When the battery ages or the wires leak electricity, it may cause a fire. The accidents caused by electric vehicles entering buildings for charging are usually serious. Therefore, there is a need to control the entry of electric vehicles into buildings.

[0004] Currently, control is mainly carried out through the following two methods: a. Posting warning signs and placing mechanical structural components such as width-limited channel railings at the building elevator entrance or in the elevator car to prevent electric vehicles from entering; b. Using the camera in the elevator car to intelligently identify electric vehicles entering the elevator car, and send relevant information to the staff, who will contact the relevant owners.

[0005] In actual use, the above method has the following defects. For a, although the implementation cost is low, the setting of the mechanical structure will cause inconvenience in entering and exiting the building, affecting the traffic efficiency; and for b, due to the poor light in the elevator car and the angle limitation of the camera when shooting, it is difficult to capture high-quality images containing electric vehicle targets, resulting in low accuracy and the risk of missed reports and false reports.

[0006] When a false alarm occurs, relevant staff need to rush to the scene to deal with it, which not only increases the workload of the staff, but also causes a decrease in the efficiency of elevator operation; when a missed alarm occurs, electric vehicles entering the building to charge will pose a fire safety hazard.

[0007] In order to reduce the risk of false alarms and missed alarms, the clarity of the captured images needs to be improved. However, if new high-definition smart cameras are added to the building, the cost is high and the feasibility is low.

[0008] Traditional electric vehicle recognition methods mainly use coarse-grained global features such as color and shape or feature descriptions extracted from local areas of fine-grained feature images such as brand logos for recognition. Shallow structured machine learning models are used in recognition, which have limited expressive power and result in low accuracy. Summary of the invention

[0009] In view of the defects existing in the prior art, the purpose of the present invention is to provide a method for monitoring and early warning of electric vehicles entering buildings, which can reduce the rate of missed reports and false reports and has low cost.

[0010] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0011] A method for monitoring and early warning of electric vehicles entering a building comprises the following steps:

[0012] S1. Build and classify an electric vehicle database: Select at least 5,000 electric vehicle images, manually pre-process the images, and divide them into coarse-grained images and fine-grained images to build an electric vehicle database;

[0013] S2. Construct a convolutional neural network model, including an adversarial noise generator, a coarse-grained recognition channel, and a fine-grained recognition channel;

[0014] S3, collect image information and input it into the anti-noise generator for optimization and then input it into the coarse-grained recognition channel for recognition;

[0015] S4, determine whether the image belongs to the fine-grained category, if so, go to S5, otherwise, output the recognition result and go to S6;

[0016] S5, input the recognition graph into the fine-grained recognizer for recognition and output the result, and then go to step S5;

[0017] S6. Determine whether an electric vehicle has entered the building based on the recognition result.

[0018] Furthermore, in step S1, the coarse-grained image includes an image of the overall structure of the electric vehicle, and the coarse-grained features include the shape, beam, tool box, wheel, basket, speed knob, support, mudguard, front fork, etc. The fine-grained image includes an image of the electric vehicle's headlights, handlebars, seats, and tire shapes, and the fine-grained features include headlights, battery box, reflector, hood, hanger, seat, etc.

[0019] Furthermore, step S6 also includes the following steps: controlling the alarm to sound an alarm, linking the elevator control system to open the elevator door, and causing the elevator to enter a suspended operation state.

[0020] Furthermore, step S6 also includes the following steps: determining that no electric vehicle has entered the building, and proceeding to step S3.

[0021] Furthermore, the coarse-grained recognition channel of step S2 includes a coarse-grained convolution part, a coarse-grained fully connected layer, and a coarse-grained output layer. The coarse-grained convolution part includes four groups of feature extraction layers: CN1, CN2, CN3, and CN4, and the corresponding numbers of convolution kernels are 64, 64, 128, and 256, respectively, wherein the feature map size and the number of channels within each group are the same, and the internal parameters of the same type of network modules are the same, and each group of feature extraction layers is used to train the acquired image, extract the corresponding feature map, and send the corresponding feature map to the fine-grained recognition channel;

[0022] The convolution kernel of CN1 is 7×7, and the border padding is 3; the convolution kernels of CN2, CN3 and CN4 are 3×3, and the border padding is 1, and the step size of all convolution operations is 2.

[0023] Furthermore, CN1 includes a first convolution module, CN2 includes a residual module and a second convolution module, CN3 includes a residual module and a third convolution module, and CN4 includes a residual module and a fourth convolution module. For CN2, CN3 and CN4, data is first input into the residual module and then into the corresponding convolution module.

[0024] Furthermore, the fine-grained channel includes an attention module part, a fine-grained residual module, a fine-grained convolution module, a fine-grained fully connected layer and a fine-grained output layer. The attention module part includes four SE modules, which are respectively used to receive features transmitted by CN1, CN2, CN3 and CN4, and fuse the corresponding features.

[0025] Furthermore, the image information in step S3 is collected by a first camera mechanism and a second camera mechanism. The first camera mechanism is arranged at the door of the building unit, and is used to capture the image information of the elevator door and send it to the coarse-grained recognition channel; the second camera mechanism is arranged in the elevator car, and is used to capture the image information inside the elevator car and send it to the coarse-grained recognition channel.

[0026] Furthermore, each of the residual modules includes two convolutional layers, and its linear channel is expressed by Formula 9:

[0027] F(x)=W2δ(W1x)…Formula 9

[0028] In formula 9, W1 is the weight of the first convolutional block, W2 is the weight of the second convolutional block, δ is the ReLU activation function, and the output y is obtained through a jump direct connection shortcu and the second activation function δ:

[0029] y=δ(F(x)+x)…Formula 10

[0030] The shortcut operation adds F and x element by element. When their dimensions are different, a linear mapping is performed on x to match the dimensions. Through the above operation, the input is directly added to the output of the stacked layer.

[0031] Furthermore, in step S3, collecting image information and inputting it into the anti-noise generator for optimization specifically includes: obtaining all images in the database, encoding all images, obtaining the maximum mixed noise, classifying the features of the maximum mixed noise, performing joint training on each type of image, obtaining the corresponding optimal programmed joint parameters, inputting the maximum mixed noise and the optimal programmed joint parameters into the anti-noise generator for anti-noise processing, and then processing the captured image and outputting it to the coarse-grained recognition channel for recognition;

[0032] The adversarial noise generator consists of a fully connected layer and several convolutional layers;

[0033] Collecting image information and inputting it into the adversarial noise generator for optimization specifically includes: inputting the maximum mixed noise and the optimal programmed joint parameter into the adversarial noise generator, calculating through several convolutional layers, outputting through the link layer, obtaining diversified adversarial noise, performing an inner loop of n% of the diversified adversarial noise, performing an outer loop of (50-n)% of the diversified adversarial noise, and further, using 30% of the diversified adversarial noise as interference and 20% for storage;

[0034] Among them, the method of obtaining the maximum mixed noise is as follows:

[0035] Get all the images in the database, encode all the images, and calculate the standard deviation;

[0036] The standard deviations are introduced into the original image coding data one by one to test the influence of the standard deviation on the system robustness, and the image classifier is trained. Gaussian noise is introduced into 50% of the training data in the image classifier to interfere, and a hybrid classifier is obtained, and the maximum hybrid noise is obtained from the hybrid classifier.

[0037] The optimal programmed joint parameters are obtained through joint training. The specific steps are as follows:

[0038] A. Obtain all images in the database, encode all images, perform Gaussian noise training on each image globally, perform adversarial noise learning, and adversarial noise training to obtain the optimal global adversarial parameters;

[0039] B. After selecting a part of an arbitrary image for encoding, Gaussian noise training, adversarial noise learning and adversarial noise training are performed in sequence to obtain the optimal local adversarial parameters;

[0040] C. Perform a programmatic combination of the optimal global adversarial parameters and the optimal local adversarial parameters to obtain the optimal programmatic combination parameters.

[0041] Compared with the prior art, the advantages of the present invention are:

[0042] (1) The method for monitoring and warning electric vehicles entering buildings in the present invention is based on a general coarse-grained electric vehicle classification and recognition network model, and an attention module is added to perform fine-grained recognition, including a coarse-grained recognition channel and a fine-grained recognition channel. The SE module in the fine-grained channel can fuse feature images. The fusion is trained for each fine-grained feature, and multiple fine-grained recognition branches can be trained. In the fusion process, the channel features that are useful for the fine-grained large category recognition are improved through the attention module, and the channel features that are not very useful are suppressed, thereby improving the final recognition effect. The fine-grained map is based on the coarse-grained map, that is, when it is first determined that there are fine-grained features in the coarse-grained map, it will be output to the fine-grained recognition channel for recognition, that is, the coarse-grained and fine-grained features are integrated, thereby improving the accuracy of detection and reducing the probability of missed detection and false detection.

[0043] (2) In the method for monitoring and warning electric vehicles entering buildings in the present invention, each residual module includes two convolutional layers, and its linear channel is expressed by Formula 9:

[0044] F(x)=W2δ(W1x)…Formula 9

[0045] In formula 9, W1 is the weight of the first convolutional block, W2 is the weight of the second convolutional block, δ is the ReLU activation function, and the output y is obtained through a skip shortcut and the second activation function δ:

[0046] y=δ(F(x)+x)…Formula 10

[0047] The shortcut operation adds F and x element by element. When their dimensions are different, a linear mapping is performed on x to match the dimensions. Through the above operation, the input is directly added to the output of the stacked layer, which ensures the integrity of the information without adding additional parameters. In addition, during the network training process, the focus can be placed on learning the residual between the input and output, which reduces the learning difficulty. Through the above information transmission, the gradient vanishing problem caused by the deepening of the network is prevented, and the network degradation problem is effectively solved.

[0048] (3) The method of monitoring and warning electric vehicles entering buildings in the present invention directly uses the existing unit access control with camera function and the camera in the car to take pictures, without adding extra excessive expenses, and the cost is low.

[0049] (4) The method of monitoring and warning electric vehicles entering buildings in the present invention can effectively avoid the influence of insufficient light on the image by pre-processing the image information with an anti-noise generator, thereby improving the resolution, and further improving the accuracy of detection, and reducing the probability of missed detection and false detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 This is a structural block diagram of a system for monitoring and warning electric vehicles entering a building in an embodiment of the present invention;

[0051] Figure 2 Schematic diagram of the structure of the coarse-grained recognition channel and the fine-grained recognition channel in an embodiment of the present invention;

[0052] Figure 3 is a schematic diagram of a residual module in an embodiment of the present invention;

[0053] Figure 4 is a schematic diagram of an SE module in an embodiment of the present invention;

[0054] Figure 5 This is a flow chart of the method for monitoring and early warning of electric vehicles entering buildings according to the present invention.

[0055] In the figure: 1-first camera mechanism, 2-second camera mechanism, 3-management and solving mechanism, 31-data analysis and processing unit, 32-task allocation unit, 33-alarm unit, 4-elevator control system. DETAILED DESCRIPTION

[0056] The embodiments of the present invention are further described in detail below with reference to the accompanying drawings.

[0057] See also Figure 1 As shown, an embodiment of the present invention provides a system for monitoring and warning electric vehicles entering a building, including a first camera mechanism 1, a second camera mechanism 2, a management and resolution mechanism 3 and an elevator control mechanism 4, and the first camera mechanism 1, the second camera mechanism 2, and the elevator control mechanism 4 are all connected to the management and resolution mechanism 3.

[0058] The first camera mechanism 1 is arranged at the door of the building unit, and is used to capture the image information of the elevator door and send it to the management and solution mechanism 3; the second camera mechanism 2 is arranged in the elevator car, and is used to capture the image information inside the elevator car and send it to the management and solution mechanism 3.

[0059] The first camera mechanism 1 in this embodiment can directly select a unit access control system with a camera function. Since the current unit access control system itself has a camera and intelligent recognition function, using it as the first camera mechanism 1 can effectively save costs.

[0060] The management and solving mechanism 3 is used to calculate and analyze the image information at the elevator door and inside to determine whether an electric vehicle has entered the elevator car. If so, the management and solving mechanism 3 links the elevator control system 4 to open the elevator door, so that the elevator enters a suspended operation state; at the same time, an alarm message is sent to the property allocation agency, and the property allocation agency assigns relevant staff to rush to the scene for processing. Otherwise, the first camera mechanism 1 and the second camera mechanism 2 are controlled to continue monitoring.

[0061] In this embodiment, the management and solving mechanism 3 includes a data analysis and processing unit 31 , a task allocation unit 32 and an alarm unit 33 .

[0062] See also Figure 2 As shown, the data analysis and processing unit 31 includes a convolutional neural network model based on a multi-attention mechanism. The network model is based on a general coarse-grained electric vehicle classification and recognition network model, and an attention module (SE module) is added to enable fine-grained recognition, including a coarse-grained recognition channel and a fine-grained recognition channel.

[0063] Among them, the coarse-grained recognition channel includes a coarse-grained convolution part, a coarse-grained fully connected layer, and a coarse-grained output layer. The coarse-grained convolution part includes four groups of feature extraction layers: CN1, CN2, CN3 and CN4, and the corresponding number of convolution kernels are 64, 64, 128 and 256 respectively. The feature map size and number of channels within each group are the same, and the internal parameters of the same type of network modules are the same. Each group of feature extraction layers is used to train the acquired images, extract the corresponding feature maps, and send the corresponding feature maps to the fine-grained recognition channel.

[0064] The convolution kernel of CN1 is 7×7 with a border padding of 3; the convolution kernels of CN2, CN3, and CN4 are 3×3 with a border padding of 1, and the stride of all convolution operations is 2.

[0065] CN1 includes a first convolution module, CN2 includes a residual module and a second convolution module, CN3 includes a residual module and a third convolution module, and CN4 includes a residual module and a fourth convolution module. For CN2, CN3 and CN4, data is first input into the residual module and then into the corresponding convolution module.

[0066] Each residual module consists of two convolution blocks: the first convolution block and the second convolution block. The convolution kernels of the first convolution block and the second convolution block are both 3x3, the stride of the first convolution block is 2, and the stride of the second convolution block is 1. The first convolution block and the second convolution block are connected through a batch normalization layer and a ReLU activation layer.

[0067] Due to the process of feature extraction in the coarse-grained convolution part, the coarse-grained feature maps extracted by different extraction layers contain different information. From CN1 to CN4, from shallow to deep, the shallow coarse-grained feature maps have higher resolution and contain more details, but the semantic information is not rich enough; the deep coarse-grained feature maps contain rich semantic information but low resolution.

[0068] In order to fully utilize the coarse-grained feature map obtained by the coarse-grained recognition channel, this embodiment outputs the output results of different feature extraction layers to the fine-grained recognition channel.

[0069] The fine-grained recognition channel includes an attention module (SE module), a fine-grained residual module (the structure of the fine-grained residual module is consistent with the structure of the residual module described above), a fine-grained convolution module, a fine-grained fully connected layer and a fine-grained output layer. The attention module includes four SE modules, which are respectively used to receive the features transmitted by CN1, CN2, CN3 and CN4, and fuse the corresponding features. The output results of the four SE modules are fused with the feature maps obtained by the corresponding extraction layers, and then after passing through the fine-grained residual module and the fine-grained convolution module, the n nodes on the fine-grained fully connected layer are used as outputs. In this embodiment, n=16, and the coarse-grained output layer and the fine-grained output layer use the softmax loss function.

[0070] Fusion is to train each fine-grained feature, and can train multiple fine-grained recognition branches. During the fusion process, the attention module is used to enhance the channel features that are useful for the fine-grained large category recognition, and suppress the channel features that are not very useful, thereby improving the final recognition effect.

[0071] In order to enable the network to achieve the final classification and recognition effect, the average pooling layer is used on the convolution result of the last layer (the fourth convolution module, the fine-grained convolution module). For example, in the coarse-grained recognition network, the feature dimension obtained by CN4 is 14x14x256. After average pooling with a kernel size of 14, its dimension becomes 1x1x256, and then it is converted into a one-dimensional column vector through dimensional conversion and input into the corresponding fully connected layer (coarse-grained fully connected layer / fine-grained fully connected layer). Then, classification and recognition are realized through the softmax loss function. The dimension of the fully connected layer is determined by the number of categories.

[0072] When performing fine-grained object recognition, different coarse-grained large categories, such as electric cars, baby carriages, toy cars, and disabled cars, have different sensitivities to the same features. Therefore, the above coarse-grained large categories are regarded as one category to complete the recognition of coarse-grained categories. Then, on the basis of coarse-grained recognition, for each fine-grained large category, different weights are assigned to the feature channels through the SE module to enhance the features that contribute greatly to the fine-grained object recognition task and suppress the features with small contributions, thereby forming multiple sub-network channels for fine-grained recognition.

[0073] Since the sizes of coarse-grained feature maps are different, this embodiment uses maximum pooling to unify all coarse-grained feature maps into the same size as the output of the fourth group of convolution blocks (CN4), and then concat splices them along the fine-grained channel dimension. The final feature map dimension is 14x14x512, and then a group of residual and convolution modules are used to further condense the features. Finally, through average pooling and full connection operations similar to the coarse-grained recognition network, the classification and recognition of fine-grained categories are completed through softmax to obtain a fine-grained feature map.

[0074] In this embodiment, after entering the convolution module / or the convolution module, the image information passes through the convolution layer, the pooling layer, the batch normalization layer and the activation processing layer in sequence.

[0075] The coarse-grained recognition channel is used to receive images taken by the first camera mechanism 1 and the second camera mechanism 2. The convolution layer performs convolution calculation on the image and extracts the coarse-grained feature map. The pooling layer performs maximum pooling and uniform pooling on the coarse-grained feature map and sends it to the batch normalization layer. The batch normalization layer normalizes the data, processes it using the ReLU activation function, and sends it to the fully connected layer for calculation using the softmax function.

[0076] The batch normalization layer normalizes the output of each neuron in the neural network to a normal distribution of N(0, 1), reducing the mutual influence between layers and achieving the purpose of accelerating convergence.

[0077] The attention module in the fine-grained recognition channel is used to receive the corresponding coarse-grained feature map in the coarse-grained recognition channel, and convolute the feature map through the fine-grained residual module and the fine-grained convolution module to extract the coarse-grained feature map. The pooling layer performs maximum pooling and uniform pooling on the coarse-grained feature map and sends it to the batch normalization layer. The batch normalization layer normalizes the data, processes it using the ReLU activation function, and sends it to the fine-grained fully connected layer for calculation through the softmax function and then sends it to the fine-grained output layer to obtain a fine-grained map.

[0078] In this embodiment, the fine-grained map is based on the coarse-grained map during the calculation. That is, it is first determined whether there are fine-grained features in the coarse-grained map before it is output to the fine-grained recognition channel for recognition. This combines the coarse-grained and fine-grained features, improves the accuracy of detection, and reduces the chances of missed detection and false detection.

[0079] See also Figure 5 As shown, the present invention provides a method for monitoring and warning electric vehicles entering a building, comprising the following steps:

[0080] S1. Construct an electric vehicle database and classify it: select at least 5,000 electric vehicle images (all taken in elevators), pre-process the images manually, and divide them into coarse-grained images and fine-grained images. The coarse-grained images include the overall structure of the electric vehicle, and the coarse-grained features include the shape, beams, toolbox, wheels, baskets, speed knobs, supports, mudguards, front forks, etc. The fine-grained images include the images of the electric vehicle's lights, handles, seats, and tire shapes, and the fine-grained features include lights, battery boxes, reflectors, hoods, hangers, seats, etc., so as to construct an electric vehicle database.

[0081] S2. Construct a convolutional neural network model, including an adversarial noise generator, a coarse-grained recognition channel, and a fine-grained recognition channel.

[0082] S3, collect image information and input it into the anti-noise generator for optimization and then input it into the coarse-grained recognition channel for recognition.

[0083] S4, determine whether the image belongs to the fine-grained category, if so, go to S5, otherwise, output the recognition result and go to S6.

[0084] S5. Input the recognition graph into the fine-grained recognizer for recognition and output the result, and then go to step S5.

[0085] S6. Determine whether an electric vehicle has entered the building based on the recognition result.

[0086] S7, if yes, control the alarm to sound, link the elevator control system 4 to open the elevator door, so that the elevator enters a suspended operation state; otherwise, continue to transmit image information and go to step S3.

[0087] The step S3 in which image information is collected and input into the adversarial noise generator for optimization specifically includes: obtaining all images in the database, encoding all images (in this embodiment, grayscale data), obtaining the maximum mixed noise, classifying the characteristics of the maximum mixed noise, performing joint training on each type of image, obtaining the corresponding optimal programmed joint parameters, inputting the maximum mixed noise and the optimal programmed joint parameters into the adversarial noise generator for adversarial processing, and then processing the captured image and outputting it to the coarse-grained recognition channel for recognition.

[0088] By optimizing the anti-noise generator, the interference of background noise on the image can be effectively avoided, the recognition accuracy is improved, and the probability of missed detection and wrong detection is reduced.

[0089] The adversarial noise generator includes a fully connected layer and several convolutional layers (in this embodiment, four convolutional layers with 1x1 convolution kernels are selected).

[0090] The image information is collected and input into the adversarial noise generator for optimization, specifically including: inputting the maximum mixed noise and the optimal programmed joint parameters into the adversarial noise generator, calculating through several convolutional layers, and outputting through the link layer to obtain diversified adversarial noise, performing an inner loop of n% of the diversified adversarial noise, and performing an outer loop of (50-n)% of the diversified adversarial noise (as the (50-n)% Gaussian noise for the next image processing). In addition, 30% of the diversified adversarial noise is used as interference, and 20% is stored.

[0091] Among them, the method of obtaining the maximum mixed noise is as follows:

[0092] All images in the database are obtained, all images are encoded (in this embodiment, grayscale data), and the standard deviation is calculated.

[0093] The standard deviations are introduced into the original image coding data one by one to test the impact of the standard deviation on the system robustness, the image classifier is trained, Gaussian noise is introduced into 50% of the training data in the image classifier to interfere, and a hybrid classifier is obtained. The maximum mixed noise (including several maximum mixed noises, each maximum mixed noise corresponds to a data feature, and the data features include but are not limited to edge information) is obtained from the hybrid classifier.

[0094] The optimal programmed joint parameters are obtained through joint training. The specific steps are as follows:

[0095] A. Obtain all images in the database, encode all images (in this embodiment, grayscale data), perform Gaussian noise training on the global image of each image (introduce Gaussian noise into the image encoding data), perform adversarial noise learning (de-noising the image against the introduced Gaussian noise), perform adversarial noise training (obtain the optimal adversarial parameters in the adversarial noise learning), and obtain the optimal global adversarial parameters.

[0096] B. After selecting a local area (1 / 5 to 1 / 3 of the area) of any image for encoding, Gaussian noise training, adversarial noise learning and adversarial noise training are performed in sequence to obtain the optimal local adversarial parameters.

[0097] C. Perform a programmatic combination of the optimal global adversarial parameters and the optimal local adversarial parameters to obtain the optimal programmatic combination parameters.

[0098] The convolution module (in this embodiment, including all convolution modules and convolution blocks) is used to extract image features. It has the characteristics of weight sharing and local connection, and completes most of the calculation work in the forward propagation and reverse propagation process of the model. In this embodiment, the calculation formula of the convolution module is as follows:

[0099] Z (l+1) =w (l) x (l) +b (l) …Formula 1

[0100] x (l+1) =f(z (l+1) )…Formula 2

[0101] In formula 1 and formula 2, x (l) The input of the current layer is also the output of the previous layer, w (l) is the convolution kernel weight, b (l) is the bias value, and f is the activation function.

[0102] Specifically, the convolution module extracts features by convolving image information with a convolution kernel, and the calculation result is a feature map.

[0103] Taking the convolution kernel of 3x3, the input matrix (image information) size of 5x5, and the sliding step size of 1 as an example, the convolution result size is 3x3. In the specific calculation, the convolution kernel is used as a 3x3 window on the image information with a step size, sliding from left to right and from top to bottom. In each window coverage area, the convolution kernel is multiplied and summed with the value of the input matrix, and the corresponding position of the result matrix is ​​filled in, which is the convolution result, that is, the matrix corresponding to the feature map.

[0104] In order to improve the training speed of the neural network, a batch normalization layer is introduced to normalize the data. The specific calculation results are shown in Formula 3:

[0105]

[0106] In formula 3, E[x (k) ] represents the mean of the input data, Var[x (k) ] represents the variance of the input data. The output of each neuron in the network is standardized to a normal distribution of N(0, 1), which reduces the mutual influence between layers and achieves the purpose of accelerating convergence. In order to avoid the degradation of network performance caused by a complete normal distribution, two parameters are introduced: scaling amount γ (k) and translation β (k) , calculated according to Formula 4:

[0107] y (k) =γ (k) x (k) +β(k) …Formula 4

[0108] The Softmax function is shown in Formula 5:

[0109]

[0110] In Formula 5, x represents the input, y represents the output label with k different values, P(y=j|x;θ) represents the probability of predicting the category j when the model parameter is θ and the input is x. Used to make all probabilities sum to 1.

[0111] In the process of training the convolutional network on the data, the loss function of formula 6 is used to optimize the model:

[0112]

[0113] In formula 6, I{y (i) =j} is: I{true}=1, I{falase}=0. The loss function is a feature function. Since the category label itself is 0 or 1, Formula 6 can be simplified to Formula 7:

[0114]

[0115] When the model is used for prediction, the probability of each category j is calculated for the input x using Formula 8, which is as follows:

[0116]

[0117] The category corresponding to the maximum probability is taken as the system prediction result. To prevent overfitting, a regularization term is usually added after the loss function, such as L1 regularization that adds the sum of the absolute values ​​of the weight parameters, or L2 regularization that adds the sum of the squares of the weight parameters.

[0118] See also Figure 4 As shown, in this embodiment, each residual module includes two convolutional layers, and its linear channel is expressed by Formula 9:

[0119] F(x)=W2δ(W1x)…Formula 9

[0120] In formula 9, W1 is the weight of the first convolutional block, W2 is the weight of the second convolutional block, δ is the ReLU activation function, and the output y is obtained through a skip shortcut and the second activation function δ:

[0121] y=δ(F(x)+x)…Formula 10

[0122] The shortcut operation adds F and x element by element. When their dimensions are different, a linear mapping is performed on x to match the dimensions. Through the above operation, the input is directly added to the output of the stacked layer, which ensures the integrity of the information without adding additional parameters. In addition, during the network training process, the focus can be placed on learning the residual between the input and output, which reduces the learning difficulty. Through the above information transmission, the gradient vanishing problem caused by the deepening of the network is prevented, and the network degradation problem is effectively solved.

[0123] The specific implementation of the attention module consists of a Squeeze operation and an Excitation operation:

[0124] First, perform a squeeze operation: compress the HxWxC-dimensional convolutional layer output into a 1x1xC vector along the direction of the feature channel, where the two-dimensional feature map of each channel is compressed into a real number with a global receptive field, which enables the layer close to the input to also obtain a global receptive field, representing the global distribution of the response on the feature channel, calculated according to formula 11:

[0125]

[0126] Then perform the Excitation operation (this process is implemented by two fully connected layers and a Sigmoid activation function). The fully connected layer is used to better fuse all the input feature information. The Sigmoid function is to map the input to the range of (0, 1) so that it can be used as a weight later. The specific calculation is performed according to Formula 12:

[0127] S=F ex (z, W)=δ(g(z, W))=δ(W4δ(W3z))…Formula 12

[0128] In formula 12, z represents the global description obtained after the Squeeze operation, δ represents the ReLU activation function, which is used to ensure that the output result is positive, W3 and W4 are the weight values ​​of the two fully connected layers, c is a scaling parameter, which is determined according to actual needs. Its main function is to reduce the number of parameters, reduce the computational complexity of the network, and speed up the calculation.

[0129] After the Excitation operation is completed, the weight values ​​of each channel of the convolutional layer feature map are obtained. Through a channel-wise multiplication, the original features are fused with the weights to obtain the final output result, which is calculated according to formula 13:

[0130]

[0131] In formula 13, Sc is the original feature and Uc is the weight value.

[0132] The embodiment of the present invention embeds the SE module into the existing mature neural network structure, which first performs convolution processing on the coarse-grained data, and then inputs the processing results into the SE module for fusion. The fused data passes through a residual module, a convolution module and a softmax function in sequence and then is output.

[0133] Data fusion in the SE module is performed for each fine-grained level, and multiple fine-grained recognition branches can be trained. During the fusion process, the SE module weight value is used to enhance the channel features that are useful for fine-grained large-category recognition, and to suppress the channel features that are not very useful, thereby improving the final recognition effect.

[0134] The present invention is not limited to the above-mentioned optimal implementation mode. Anyone can derive other various forms of products under the inspiration of the present invention. However, no matter what changes are made in the shape or structure, all technical solutions that are the same or similar to the present invention are within the scope of protection.

Claims

1. A method for monitoring and early warning of electric vehicles entering a building, characterized in that: The following steps are involved: S1. Build and classify an electric vehicle database: Select at least 5,000 electric vehicle images, manually pre-process the images, and divide them into coarse-grained images and fine-grained images to build an electric vehicle database; S2. Construct a convolutional neural network model, including an adversarial noise generator, a coarse-grained recognition channel, and a fine-grained recognition channel; S3, collect image information and input it into the anti-noise generator for optimization and then input it into the coarse-grained recognition channel for recognition; S4, determine whether the image belongs to the fine-grained category, if so, go to S5, otherwise, output the recognition result and go to S6; S5, input the recognition graph into the fine-grained recognizer for recognition and output the result, and then go to step S6; S6. Determine that an electric vehicle has entered the building based on the recognition result; The step S3 of collecting image information and inputting it into the anti-noise generator for optimization specifically includes the following steps: obtaining all images in the electric vehicle database, encoding all images, obtaining the maximum mixed noise, classifying the features of the maximum mixed noise, performing joint training on each type of image, obtaining the corresponding optimal programmed joint parameters, inputting the maximum mixed noise and the optimal programmed joint parameters into the anti-noise generator for anti-noise processing, and then processing the captured images and outputting them to the coarse-grained recognition channel for recognition; The adversarial noise generator consists of a fully connected layer and several convolutional layers; Collecting image information and inputting it into the adversarial noise generator for optimization specifically includes: inputting the maximum mixed noise and the optimal programmed joint parameter into the adversarial noise generator, calculating through several convolutional layers, outputting through the link layer, obtaining diversified adversarial noise, performing an inner loop of n% of the diversified adversarial noise, performing an outer loop of (50-n)% of the diversified adversarial noise, and further, using 30% of the diversified adversarial noise as interference and 20% for storage; Among them, the method of obtaining the maximum mixed noise is as follows: Get all the images in the database, encode all the images, and calculate the standard deviation; The standard deviations are introduced into the original image coding data one by one to test the influence of the standard deviation on the system robustness, and the image classifier is trained. Gaussian noise is introduced into 50% of the training data in the image classifier to interfere, and a hybrid classifier is obtained, and the maximum hybrid noise is obtained from the hybrid classifier. The optimal programmed joint parameters are obtained through joint training. The specific steps are as follows: A. Obtain all images in the electric vehicle database, encode all images, perform Gaussian noise training on each image globally, perform adversarial noise learning, and adversarial noise training to obtain the optimal global adversarial parameters; B. After selecting a part of an arbitrary image for encoding, Gaussian noise training, adversarial noise learning and adversarial noise training are performed in sequence to obtain the optimal local adversarial parameters; C. Perform a programmatic combination of the optimal global adversarial parameters and the optimal local adversarial parameters to obtain the optimal programmatic combination parameters.

2. A method for monitoring and early warning of electric vehicles entering a building as claimed in claim 1, characterized in that: In step S1, the coarse-grained image includes the overall frame image of the electric vehicle, and the coarse-grained features include the shape, beam, tool box, wheel, basket, speed knob, support, mudguard, and front fork; the fine-grained image includes the image of the electric vehicle headlights, handlebars, seats, and tire shapes, and the fine-grained features include the headlights, battery box, reflector, hood, hanger, and seat.

3. A method for monitoring and early warning of electric vehicles entering a building as claimed in claim 1, characterized in that: Step S6 also includes the following steps: controlling the alarm to sound an alarm, linking the elevator control system (4) to open the elevator door, and causing the elevator to enter a suspended operation state.

4. A method for monitoring and early warning of electric vehicles entering a building as claimed in claim 1, characterized in that: Step S6 also includes the following steps: if it is determined that no electric vehicle has entered the building, the process proceeds to step S3.

5. A method for monitoring and early warning of electric vehicles entering a building as claimed in claim 1, characterized in that: Step S2: the coarse-grained recognition channel includes a coarse-grained convolution part, a coarse-grained fully connected layer, and a coarse-grained output layer. The coarse-grained convolution part includes four groups of feature extraction layers: CN1, CN2, CN3, and CN4. The corresponding numbers of convolution kernels are 64, 64, 128, and 256, respectively. The feature map size and number of channels within each group are the same, and the internal parameters of the same type of network modules are the same. Each group of feature extraction layers is used to train the acquired image, extract the corresponding feature map, and send the corresponding feature map to the fine-grained recognition channel. The convolution kernel of CN1 is 7×7, and the border padding is 3; the convolution kernels of CN2, CN3 and CN4 are 3×3, and the border padding is 1, and the step size of all convolution operations is 2.

6. A method for monitoring and early warning of electric vehicles entering a building as claimed in claim 5, characterized in that: CN1 includes a first convolution module, CN2 includes a residual module and a second convolution module, CN3 includes a residual module and a third convolution module, and CN4 includes a residual module and a fourth convolution module. For CN2, CN3 and CN4, data is first input into the residual module and then into the corresponding convolution module.

7. A method for monitoring and early warning of electric vehicles entering a building as claimed in claim 5, characterized in that: The fine-grained recognition channel includes an attention module part, a fine-grained residual module, a fine-grained convolution module, a fine-grained fully connected layer and a fine-grained output layer. The attention module part includes four SE modules, which are respectively used to receive the features transmitted by CN1, CN2, CN3 and CN4, and fuse the corresponding features.

8. A method for monitoring and early warning of electric vehicles entering a building as claimed in claim 5, characterized in that: The image information in step S3 is collected by a first camera (1) and a second camera (2), wherein the first camera (1) is arranged at the door of the building unit, and is used to capture the image information of the elevator door and send it to the coarse-grained recognition channel; the second camera (2) is arranged in the elevator car, and is used to capture the image information inside the elevator car and send it to the coarse-grained recognition channel.

9. A method for monitoring and early warning of electric vehicles entering a building as claimed in claim 6, characterized in that: Each of the residual modules includes two convolutional layers, and its linear channel is expressed by Formula 9: F(x)=W2δ(W1x)…Formula 9 In formula 9, W1 is the weight of the first convolution block, W2 is the weight of the second convolution block, δ is the ReLU activation function, and the output y is obtained through a jump direct connection shortcu and the second activation function δ: y=δ(F(x)+x)…Formula 10 The shortcut operation adds F and x element by element. When their dimensions are different, a linear mapping is performed on x to match the dimensions. Through the above operation, the input is directly added to the output of the stacked layer.

Images