Lightweight communication device detection method based on asymmetric dense convolutional neural network

Abstract

The present application relates to a kind of lightweight communication equipment detection methods based on asymmetric dense convolutional neural network, belong to artificial intelligence, computer vision, target detection technical field.The present application designs a two-way asymmetric dense layer, in the case where not increasing computing cost, single non-linear activation layer is decomposed into two layers to design asymmetric depth separable convolution, based on this, two-way asymmetric dense layer is constructed, to enhance the feature extraction and information expression capability of lightweight network;Design a channel mixed arrangement mechanism in feature merging process, so that the layer can retain feature details in information exchange and realize more smooth intra-layer information flow, while generating variable feature fusion mode according to the given channel configuration;Introduce the idea of dense connection, realize layer-to-layer information flow and feature reuse, so as to promote the efficient transmission of information.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, computer vision and target detection, and particularly relates to a lightweight communication equipment detection method based on an asymmetric dense convolutional neural network. BACKGROUND

[0002] Communication equipment detection refers to a task of accurately identifying and locating specific communication equipment in images or videos given some images or videos. In real applications, the communication equipment detection task has many important scenarios, such as fault equipment identification in communication network maintenance, communication equipment inventory in intelligent warehouses, and illegal equipment detection in security monitoring. Considering that in many actual scenarios, target detection not only needs to identify the existence of communication equipment, but also needs to further identify detailed information such as the model and state of the equipment, such as angle calibration of base station antennas and port state monitoring of optical fiber switches. SUMMARY

[0003] (I) Technical problems to be solved

[0004] The application provides a lightweight communication equipment detection method, which mainly solves three problems, one is that the lightweight network extracts insufficient feature expression information, two is that the information flow in the convolutional layer of the lightweight network is blocked, and three is that the information transmission efficiency between different convolutional layers of the lightweight network is low.

[0005] (II) Technical solutions

[0006] In order to solve the above technical problems, the application provides a lightweight communication equipment detection method based on an asymmetric dense convolutional neural network, comprising the following steps:

[0007] Step 1: data preprocessing

[0008] Before the data is input into the asymmetric dense convolutional neural network, a series of preprocessing operations are performed on the data, which includes image data and video data.

[0009] Image data preprocessing includes data cleaning, data enhancement and format conversion. First, invalid and abnormal images in the data are removed, and data enhancement is performed by using mosaic enhancement, hybrid enhancement, random disturbance and color disturbance. Finally, the image is scaled to the fixed size required by the asymmetric dense convolutional neural network model, and normalized to ensure that the numerical range of the input data meets the requirements of the model.

[0010] Video data preprocessing includes frame extraction and frame image preprocessing. First, a frame extraction method is adopted, using an even interval extraction of N frames per second to ensure that the timeline in the video is evenly covered. The frame extraction interval is adjusted accordingly according to the degree of motion of the target to be detected in the video. That is, when the target to be detected in the video is relatively still, the frame extraction interval is increased; when the target to be detected in the video is in motion, the frame extraction interval is reduced. After the frame extraction is completed, the extracted frame images are subjected to the relevant operations of the image data preprocessing.

[0011] Step 2: Feature extraction

[0012] After preprocessing, the data is input into an asymmetric dense convolutional neural network for feature extraction, providing feature representation with certain expression ability and information content for subsequent high-accuracy classification and positioning; wherein, the asymmetric dense convolutional neural network is designed as follows: in the initial stage, a preliminary extraction module is used to receive an input image of size 224*224*3 and output a feature map of size 56*56*32; subsequently, the network performs in-depth feature extraction and dimensionality reduction through four dense blocks and three transition layers; each dense block is composed of a two-way asymmetric dense layer, in which a dense block is repeated 3 times and outputs a feature map of size 56*56*128 Feature map; dense block two is repeated 4 times, outputting a 28*28*256 feature map; dense block three and dense block four are repeated 8 times, outputting 14*14*512 and 7*7*768 feature maps respectively; the transition layer uses 1*1 convolution and 2*2 average pooling operations to halve the spatial size of the feature map and increase the number of channels. The three transition layers are located between two adjacent dense blocks, and each transition layer provides input for the subsequent dense block; finally, the network compresses the 7*7*768 feature map to 1*1*768 through a classification layer consisting of a global average pooling layer, a fully connected layer, and a softmax layer;

[0013] Step 3: Communication equipment detection

[0014] After feature extraction, the feature map finally generated by the asymmetric dense convolutional neural network is used through the prediction layer to classify and locate the communication device.

[0015] The present invention also provides a system for the method.

[0016] (3) Beneficial effects

[0017] The present invention provides a lightweight communication device detection method based on an asymmetric dense convolutional neural network, the main advantages of which are reflected in the following aspects:

[0018] (1) A two-way asymmetric dense layer is designed, which decomposes a single nonlinear activation layer into two layers without increasing the calculation cost to design an asymmetric depth separable convolution, and a two-way asymmetric dense layer is constructed based on this to enhance the feature extraction and information expression capability of the lightweight network.

[0019] (2) A channel mixing mechanism in the feature merging process is designed, which enables the layer to retain feature details and achieve smoother intra-layer information flow in information exchange, and generates a variable feature fusion mode according to the given channel configuration.

[0020] (3) The idea of dense connection is introduced to realize layer-to-layer information flow and feature reuse, thereby promoting efficient transmission of information. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 The model overall framework of the application is shown in the figure.

[0022] Figure 2 The principle diagram of the feature preliminary extraction module is shown in the figure.

[0023] Figure 3 The two-way asymmetric dense layer structure of the application is shown in the figure. DETAILED DESCRIPTION

[0024] In order to make the purpose, content and advantages of the application clearer, the specific embodiments of the application will be further described in detail below in combination with the drawings and examples.

[0025] The application proposes a model based on an asymmetric dense convolutional neural network for completing a communication device detection task, which realizes high-accuracy communication device detection, has lightweight and real-time detection performance, and can meet the ability of deploying and detecting communication devices on a resource-limited device platform.

[0026] The overall framework of the lightweight communication device detection method based on the asymmetric dense convolutional neural network is shown in the figure. Figure 1 Generally, it contains three steps, namely data preprocessing, feature extraction and communication device detection.

[0027] Step 1: Data preprocessing

[0028] Before the data is input into the asymmetric dense convolutional neural network model, a series of preprocessing operations need to be performed on the data, including image data and video data.

[0029] The image data preprocessing mainly includes data cleaning, data enhancement, format conversion, etc. First, remove invalid and abnormal images in the data, and use mosaic enhancement, mixed enhancement, random disturbance and color disturbance to increase the diversity of the data. These steps help to improve the generalization ability and detection accuracy of the model, and further improve the quality of the image data. Finally, the image is scaled to the fixed size required by the asymmetric dense convolutional neural network model, and normalized to ensure that the numerical range of the input data meets the requirements of the model.

[0030] The video data preprocessing mainly includes frame extraction and frame image preprocessing. First, use equal interval extraction of 5 frames per second as the basic method of frame extraction to ensure that the timeline in the video is evenly covered. This frame extraction interval can be adjusted according to the motion degree of the target to be detected in the video. When the target to be detected in the video is relatively stationary, the frame extraction interval can be increased to improve the detection speed. When the target to be detected in the video is in motion, the frame extraction interval can be reduced to reduce the probability of missing detection. After frame extraction, the extracted frame images are subjected to the relevant operations of the image data preprocessing.

[0031] Step two: feature extraction

[0032] After the data is preprocessed, it is input into the asymmetric dense convolutional neural network for feature extraction, providing high-accuracy classification and positioning with high-expression ability and information-rich feature representation.

[0033] (1) Network structure construction

[0034] Unlike the current pursuit of faster speed and fewer parameters in lightweight network design, the design of the present application focuses on improving the feature extraction capability of lightweight networks. The present application constructs an asymmetric dense convolutional neural network to improve feature extraction quality, increase information flow, and enhance network expression capability. The network structure is shown in Table 1. The initial stage of the network is the preliminary extraction module, which receives an input image of size 224*224*3 and outputs a feature map of size 56*56*32; then, the network performs deep feature extraction and dimension reduction through four dense blocks and three transition layers; each dense block is composed of a double-path asymmetric dense layer, where dense block one is repeated 3 times, outputting a feature map of size 56*56*128; dense block two is repeated 4 times, outputting a feature map of size 28*28*256; dense block three and dense block four are each repeated 8 times, outputting feature maps of size 14*14*512 and 7*7*768, respectively; the transition layer realizes a halving of the spatial size and an increase in the number of channels through a 1*1 convolution and a 2*2 average pooling operation, and the three transition layers are located between adjacent dense blocks, each providing input for the subsequent dense block; finally, the network compresses the 7*7*768 feature map to 1*1*768 through a classification layer composed of a global average pooling layer, a fully connected layer, and a softmax layer. This architecture realizes efficient feature extraction of the input image through a carefully designed modular and hierarchical structure.

[0035] Table 1 Network structure

[0036]

[0037] (2) Preliminary extraction module

[0038] In the asymmetric dense convolutional neural network, the preliminary extraction module is used to extract the shallow features of the original image, filter out useless background information, and reduce the dimension of the feature map. If the preliminary extraction module is not designed properly, it will destroy the effective information in the image and seriously affect the quality of subsequent feature extraction. Typical deep networks usually use a 7*7 or 5*5 large convolution kernel convolution layer, followed by a 3*3 max pooling layer as the preliminary extraction module. This can complete the preliminary extraction with high quality, but such a heavy structure is not suitable for lightweight network design. The present application designs a more efficient preliminary extraction module by fusing the outputs of a 3*3 convolution layer and a 2*2 max pooling layer, as shown in Figure 2 where the data is input into the first 3*3 convolution layer, the output of the first 3*3 convolution layer is input into the 1*1 convolution layer, the output of the 1*1 convolution layer is input into the second 3*3 convolution layer and the 2*2 max pooling layer, respectively, and the outputs of the second 3*3 convolution layer and the 2*2 max pooling layer are spliced to form the output result of the preliminary extraction module.

[0039] (3) Double-path asymmetric dense layer

[0040] The application is based on an asymmetric deep separable convolution, a double-path convolution extraction structure, a dense connection mode and a feature channel separation and mixed arrangement operation to construct a double-path asymmetric dense layer.

[0041] Compared with the standard convolution, the deep separable convolution will cause more information loss and lead to weakened feature expression. However, the lightweight network design needs the calculation cost saving brought by the deep separable convolution. Therefore, by decomposing the deep convolution layer into two asymmetric layers in space, the nonlinearity can be enhanced by increasing the depth of the convolution layer to achieve better feature expression.

[0042] Based on the deep separable convolution and the asymmetric convolution, the application proposes an asymmetric deep separable convolution which uses convolution decomposition to improve the deep separable convolution. It decomposes the deep convolution layer into two layers with asymmetric convolution kernels and combines them with pointwise convolution.

[0043] The standard convolution, the asymmetric convolution and the deep separable convolution operation respectively need the calculation amount Cost s , Cost a , Cost dw As follows:

[0044] Cost s = H i * W i * M * N * D k * D k

[0045] Cost a = H i * W i * M * M * D k * 1 + H i * W i * M * N * 1 * D k

[0046] Cost dw = H i * W i * M * D k * D k + M * N * D k * D k

[0047] Wherein, H i , W i respectively represent the length and width of the input image, M represents the input channel number, N represents the output channel number, D k represents the spatial size of the convolution kernel.

[0048] And the calculation amount Cost adw of the asymmetric deep separable convolution is:

[0049] Cost adw = H i * W i * M * D k + H i * W i * M * 1 * D k * 1 + H i * W i

[0050] * M * N * 1 * 1

[0051] By comparing the respective calculation amounts (H i = W i = 7, M = 64, N = 32) under the same configuration, it can be seen that the asymmetric depth separable convolution can have the highest calculation compression rate, as shown in Table 2.

[0052] Table 2 Comparison of calculation amounts of four convolutions under the same settings

[0053]

[0054]

[0055] Considering that constructing the asymmetric dense convolution layer only with the asymmetric depth separable convolution will limit the network feature extraction capability, the application proposes a two-path asymmetric dense layer structure. The input of the two-path asymmetric dense layer is a feature map with k0 channel numbers (c = k0). In the two-path asymmetric dense layer, first, the input feature map is subjected to a channel separation operation and is divided into two parallel paths: a single-layer asymmetric depth separable convolution path and a double-layer asymmetric depth separable convolution path; in the single-layer asymmetric depth separable convolution path, the feature map is first subjected to a 1*1 ordinary convolution layer to generate a feature map with k channel numbers (c = k), and then is subjected to an asymmetric depth separable convolution layer composed of two 1*3 and 3*1 depth convolutions and a 1*1 pointwise convolution to generate a feature map with k / 2 channel numbers (c = k / 2); in the double-layer asymmetric depth separable convolution path, the feature map is first subjected to a 1*1 ordinary convolution layer to generate a feature map with 2k channel numbers (c = 2k), and then is subjected to two asymmetric depth separable convolution layers in sequence for feature extraction and channel number compression to generate a feature map with k / 2 channel numbers (c = k / 2); then, the output feature maps of the two asymmetric depth separable convolution paths and the original input feature map are subjected to a splicing (Concat) operation to realize the merging of channel information; finally, the merged feature map is subjected to a channel shuffling operation to rearrange the channels and generate the final output feature map.

[0056] By adding an additional convolutional path with a two-layer asymmetric depthwise separable convolutional structure, the receptive field can be effectively expanded and the learning of large target features can be enhanced. The convolutional layers of the two paths have different receptive fields, enabling the network to learn rich features and compensating for the loss in accuracy caused by insufficient feature learning ability. Considering that adding a path will double the computational complexity, to ensure a lightweight and efficient network architecture, the present invention uses a channel splitting operation to divide the input channels into two equal groups, controlling the number of input feature channels. Because channel splitting prevents information from flowing between channel groups, channel shuffling is used at the end of the convolutional layer to ensure that the input channels and output channels are fully correlated. In addition, due to the deep convolutional neural network model designed by the present invention, a dense connection structure is used to construct the convolutional layers to address the problems of vanishing gradients and information transfer to subsequent layers. In each dense block, the dense layer can directly establish connections with all subsequent layers. The input of each layer is the sum of the outputs of all previous layers. The dense layer can fully accept feature information from all previous layers, achieving feature reuse and thus enhancing feature expression capabilities.

[0057] (4) Transition layer

[0058] The transition layer consists of a convolution layer and an average pooling layer. After the output of the dual-path asymmetric dense convolution layer enters the transition layer, the convolution layer first integrates the channel information, and then the average pooling layer downsamples the feature map. Since the dual-path asymmetric dense convolution layer structure proposed in this invention has a small number of network channels, further compression processing in the transition layer may cause serious information loss, destroy the feature expression ability and significantly reduce the accuracy. Therefore, channel compression is not performed in the transition layer, and the convolution layer is only used to integrate multi-channel information.

[0059] Step 3: Communication equipment detection

[0060] After feature extraction, the feature map finally generated by the asymmetric dense convolutional neural network is used through the prediction layer to classify and locate the communication device.

[0061] The feature map consists of multiple regions, each of which consists of multiple grid cells. The prediction layer first generates a series of target candidate frames for each region of the feature map to cover possible targets. The target candidate frame has two parameters: size and aspect ratio. First, the base size is calculated, and then target candidate frames of different aspect ratios are calculated based on the base size and aspect ratio coefficient. The prediction layer uses target candidate frames of different sizes for feature maps of different scales. The base size S of the target candidate frame of the first prediction layer is min = 0.2, the base size S of the target candidate box of the fifth prediction layer max =0.95, the benchmark size S of the target candidate box of the other layers prediction layer k By calculation, we can get:

[0062]

[0063] Here, n is the total number of prediction layers (n=5 in the present invention).

[0064] Set the set of aspect ratio coefficients of five target candidate boxes The height of the target candidate box can be calculated and width

[0065]

[0066] In addition, an aspect ratio coefficient a is added r ′=1, base size There are 6 target candidate boxes of different sizes in total.

[0067] The system then outputs a bounding box and class probability for each grid cell. Each grid cell is used to predict a set of bounding boxes and corresponding class probabilities, representing the location and size of possible communication devices. This allows the system to simultaneously identify and locate communication devices within the grid cell and generate a prediction for each predicted device.

[0068] In the present invention, five feature maps of different scales are selected for prediction. Six target candidate boxes of different sizes are constructed for each pixel point in the feature maps of each scale, namely 56*56, 28*28, 14*14, 7*7, and 1*1, for detection and classification. Multiple bounding boxes are generated. These bounding boxes and the corresponding category probabilities are the prediction results output by each prediction layer. They are part of the output candidate boxes after bounding box regression adjustment.

[0069] Finally, non-maximum suppression is performed on the detection frames generated on the feature maps of each scale, and some overlapping or incorrectly detected detection frames are filtered out to obtain the final communication device detection results.

[0070] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A lightweight communication device detection method based on an asymmetric dense convolutional neural network, characterized in that: The following steps are involved: Step 1: Data preprocessing Before the data is input into the asymmetric dense convolutional neural network, a series of preprocessing operations are performed on the data, which includes image data and video data; Image data preprocessing includes data cleaning, data enhancement, and format conversion. First, invalid and abnormal images are removed from the data, and data enhancement is performed using mosaic enhancement, hybrid enhancement, random perturbation, and color perturbation. Finally, the image is scaled to the fixed size required by the asymmetric dense convolutional neural network model and normalized to ensure that the numerical range of the input data meets the model requirements. Video data preprocessing includes frame extraction and frame image preprocessing. First, a frame extraction method is adopted, using an even interval extraction of N frames per second to ensure that the timeline in the video is evenly covered. The frame extraction interval is adjusted accordingly according to the degree of motion of the target to be detected in the video. That is, when the target to be detected in the video is relatively still, the frame extraction interval is increased; when the target to be detected in the video is in motion, the frame extraction interval is reduced. After the frame extraction is completed, the extracted frame images are subjected to the relevant operations of the image data preprocessing. Step 2: Feature extraction After preprocessing, the data is input into an asymmetric dense convolutional neural network for feature extraction, providing feature representation with certain expression ability and information content for subsequent high-accuracy classification and positioning; wherein, the asymmetric dense convolutional neural network is designed as follows: in the initial stage, a preliminary extraction module is used to receive an input image of size 224*224*3 and output a feature map of size 56*56*32; subsequently, the network performs in-depth feature extraction and dimensionality reduction through four dense blocks and three transition layers; each dense block is composed of a two-way asymmetric dense layer, in which a dense block is repeated 3 times and outputs a feature map of size 56*56*128 Feature map; dense block two is repeated 4 times, outputting a 28*28*256 feature map; dense block three and dense block four are repeated 8 times, outputting 14*14*512 and 7*7*768 feature maps respectively; the transition layer uses 1*1 convolution and 2*2 average pooling operations to halve the spatial size of the feature map and increase the number of channels. The three transition layers are located between two adjacent dense blocks, and each transition layer provides input for the subsequent dense block; finally, the network compresses the 7*7*768 feature map to 1*1*768 through a classification layer consisting of a global average pooling layer, a fully connected layer, and a softmax layer; Step 3: Communication equipment detection After feature extraction, the feature map finally generated by the asymmetric dense convolutional neural network is used through the prediction layer to classify and locate the communication device.

2. The method according to claim 1, wherein In the asymmetric dense convolutional neural network, the preliminary extraction module is used to extract shallow features of the original image, filter out useless background information, and perform dimensionality reduction of the feature map at the same time; the preliminary extraction module is implemented by fusing the outputs of the 3*3 convolutional layer and the 2*2 maximum pooling layer, wherein the data is input into the first 3*3 convolutional layer, the data output by the first 3*3 convolutional layer enters the 1*1 convolutional layer, the data output by the 1*1 convolutional layer enters the second 3*3 convolutional layer and the 2*2 maximum pooling layer respectively, and the outputs of the second 3*3 convolutional layer and the 2*2 maximum pooling layer are spliced ​​as the output result of the preliminary extraction module.

3. The method according to claim 1, wherein The dual-path asymmetric dense layer is constructed based on asymmetric depth-separable convolution, dual-path convolution extraction structure, dense connection method and feature channel separation and shuffling operation.

4. The method according to claim 1, wherein The input of the dual-path asymmetric dense layer is a feature map with k0 channels. In the dual-path asymmetric dense layer, first, the input feature map undergoes a channel separation operation and is divided into two parallel paths: a single-layer asymmetric depth-separable convolution path and a double-layer asymmetric depth-separable convolution path. In the single-layer asymmetric depth-separable convolution path, the feature map first passes through a 1*1 ordinary convolution layer to generate a feature map with k channels, and then passes through an asymmetric depth-separable convolution layer composed of two 1*3 and 3*1 depth convolutions and a 1*1 point-by-point convolution to generate k channels. / 2 channel feature map; in the double-layer asymmetric depth-wise separable convolution path, the feature map first passes through a 1*1 ordinary convolution layer to generate a 2k channel feature map, and then passes through two asymmetric depth-wise separable convolution layers in sequence for feature extraction and channel compression to generate a k / 2 channel feature map; after that, the output feature maps of the two asymmetric depth-wise separable convolution paths and the original input feature map are concatenated to merge the channel information; finally, through the channel shuffling operation, the merged feature map is rearranged to generate the final output feature map.

5. The method according to claim 1, wherein The transition layer consists of a convolution layer and an average pooling layer. After the output of the dual-path asymmetric dense convolution layer enters the transition layer, the convolution layer first integrates the channel information, and then the average pooling layer downsamples the feature map.

6. The method according to claim 1, wherein The feature map consists of multiple regions, each of which consists of multiple grid cells. In step 3, the prediction layer first generates a series of target candidate boxes for each region of the feature map to cover possible targets. The target candidate box has two parameters: size and aspect ratio. First, the base size is calculated, and then target candidate boxes with different aspect ratios are calculated based on the base size and aspect ratio coefficient. Target candidate boxes of different sizes are used for the prediction layers of feature maps of different scales. The baseline size S of the target candidate box of the first prediction layer min = 0.2, the base size S of the target candidate box of the fifth prediction layer max =0.95, the benchmark size S of the target candidate box of the other layers prediction layer k By calculation, we can get: Among them, n is the total number of prediction layers, and its value is 5; Set the set of aspect ratio coefficients of five target candidate boxes Calculate the height of the target candidate box and width In addition, an aspect ratio coefficient a is added r ′=1, base size There are 6 target candidate boxes of different sizes in total. Afterwards, the bounding box and category probability of each grid cell are output; each grid cell is used to predict a set of bounding boxes and corresponding category probabilities. These bounding boxes represent the location and size of possible communication devices. In this way, the communication devices can be identified and located at the same time, and a corresponding prediction result can be generated for each predicted communication device.

7. The method according to claim 6, wherein In step 3, five feature maps of different scales are selected for prediction. Six target candidate boxes of different sizes are constructed for each pixel point in the feature maps of each scale, namely 56*56, 28*28, 14*14, 7*7, and 1*1, for detection and classification. Multiple bounding boxes are generated. These bounding boxes and the corresponding category probabilities are the prediction results output by each prediction layer. They are part of the output candidate boxes after bounding box regression adjustment. Finally, non-maximum suppression is performed on the detection boxes generated on the feature maps of each scale, and some overlapping or incorrectly detected detection boxes are filtered out to obtain the final communication device detection results.

8. The method according to any one of claims 1 to 7, characterized in that This method is used in the fields of artificial intelligence, computer vision, and target detection technology.

9. A system for use in the method according to any one of claims 1 to 8.

10. The system according to claim 9, wherein: The system is used in the fields of artificial intelligence, computer vision, and target detection technology.

Images