Lightweight communication equipment detection method based on asymmetric dense convolutional neural network

Abstract

The invention relates to a lightweight communication equipment detection method based on an asymmetric dense convolutional neural network, and belongs to the technical field of artificial intelligence, computer vision and target detection. According to the method, a double-path asymmetric dense layer is designed, a single nonlinear activation layer is decomposed into two layers to design asymmetric depth separable convolution under the condition that the calculation cost is not increased, and the double-path asymmetric dense layer is constructed based on the asymmetric depth separable convolution, so that the feature extraction and information expression capabilities of a lightweight network are enhanced; a channel mixed arrangement mechanism in the feature merging process is designed, feature details of the layer can be reserved in information exchange, smoother intra-layer information flow is achieved, and meanwhile a variable feature fusion mode is generated according to given channel configuration; a dense connection thought is introduced, and layer-to-layer information circulation and feature reuse are realized, so that efficient transmission of information is promoted.

Description

Technical Field

[0001] The present invention belongs to the technical fields of artificial intelligence, computer vision and target detection, and specifically relates to a lightweight communication equipment detection method based on an asymmetric dense convolutional neural network. Background Art

[0002] Communication equipment detection refers to the task of accurately identifying and locating specific communication equipment in images or videos given some images or videos. In real applications, communication equipment detection tasks have many important scenarios, such as faulty equipment identification in communication network maintenance, communication equipment inventory in smart warehouses, and illegal equipment detection in security monitoring. Considering many actual scenarios, target detection not only needs to identify the existence of communication equipment, but also needs to further identify detailed information such as the model and status of the equipment, such as angle calibration of base station antennas and port status monitoring of fiber switches. Summary of the invention

[0003] 1. Technical issues to be resolved

[0004] The present invention proposes a lightweight communication equipment detection method, which mainly solves three problems: first, the lightweight network extracts insufficient feature expression information; second, there is blockage in the information flow in the convolution layer of the lightweight network; third, the information transmission efficiency between different convolution layers of the lightweight network is low.

[0005] (II) Technical solution

[0006] In order to solve the above technical problems, the present invention provides a lightweight communication device detection method based on an asymmetric dense convolutional neural network, comprising the following steps:

[0007] Step 1: Data preprocessing

[0008] Before the data is input into the asymmetric dense convolutional neural network, a series of preprocessing operations are performed on the data, which includes image data and video data;

[0009] Image data preprocessing includes data cleaning, data enhancement, and format conversion. First, invalid and abnormal images in the data are removed, and data enhancement is performed using mosaic enhancement, mixed enhancement, random perturbation, and color perturbation. Finally, the image is scaled to the fixed size required by the asymmetric dense convolutional neural network model, and normalized to ensure that the numerical range of the input data meets the requirements of the model.

[0010] Video data preprocessing includes frame extraction and frame image preprocessing; first, an evenly spaced extraction of N frames per second is used as a frame extraction method to ensure that the timeline in the video is evenly covered, and the frame extraction interval is adjusted accordingly according to the degree of motion of the target to be detected in the video, that is, when the target to be detected in the video is relatively still, the frame extraction interval is increased; when the target to be detected in the video is in motion, the frame extraction interval is reduced; after the frame extraction is completed, the extracted frame image is subjected to the related operations of the image data preprocessing;

[0011] Step 2: Feature extraction

[0012] After preprocessing, the data is input into an asymmetric dense convolutional neural network for feature extraction, providing feature representation with certain expressiveness and information content for subsequent high-accuracy classification and positioning; wherein, the asymmetric dense convolutional neural network is designed as follows: in the initial stage, a preliminary extraction module is used to receive an input image of size 224*224*3 and output a feature map of size 56*56*32; subsequently, the network performs in-depth feature extraction and dimensionality reduction through four dense blocks and three transition layers; each dense block is composed of a two-way asymmetric dense layer, in which a dense block is repeated 3 times and outputs a feature map of size 56*56*128 Feature map; Dense block 2 is repeated 4 times to output a 28*28*256 feature map; Dense block 3 and dense block 4 are repeated 8 times, outputting feature maps of 14*14*512 and 7*7*768 respectively; The transition layer uses 1*1 convolution and 2*2 average pooling operations to halve the spatial size of the feature map and increase the number of channels. The three transition layers are located between two adjacent dense blocks, and each transition layer provides input for the subsequent dense block; Finally, the network compresses the 7*7*768 feature map to 1*1*768 through the classification layer composed of a global average pooling layer, a fully connected layer, and a softmax layer;

[0013] Step 3: Communication equipment detection

[0014] After feature extraction, the feature map finally generated by the asymmetric dense convolutional neural network is used through the prediction layer to classify and locate the communication device.

[0015] The invention also provides a system used for the method.

[0016] (III) Beneficial effects

[0017] The present invention provides a lightweight communication equipment detection method based on an asymmetric dense convolutional neural network, the main advantages of which are reflected in the following aspects:

[0018] (1) A dual-path asymmetric dense layer is designed. A single nonlinear activation layer is decomposed into two layers without increasing the computational cost to design an asymmetric depthwise separable convolution. Based on this, a dual-path asymmetric dense layer is constructed to enhance the feature extraction and information expression capabilities of the lightweight network.

[0019] (2) A channel shuffling mechanism is designed during the feature merging process, which enables the layer to retain feature details during information exchange and achieve smoother information flow within the layer, while generating variable feature fusion patterns according to the given channel configuration.

[0020] (3) The idea of ​​dense connection is introduced to realize layer-to-layer information flow and feature reuse, thereby promoting efficient information transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is the overall framework diagram of the model of the present invention;

[0022] Figure 2 Schematic diagram of the feature preliminary extraction module;

[0023] Figure 3 This is a diagram of the dual-path asymmetric dense layer structure of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.

[0025] The present invention proposes a model based on an asymmetric dense convolutional neural network for completing communication equipment detection tasks. While achieving high-accuracy communication equipment detection, the model has lightweight and real-time detection performance, and can meet the ability to deploy and detect communication equipment on resource-constrained device platforms.

[0026] The overall framework of the lightweight communication equipment detection method based on asymmetric dense convolutional neural network is as follows Figure 1 As shown, in general, it includes three steps, namely data preprocessing, feature extraction and communication device detection.

[0027] Step 1: Data preprocessing

[0028] Before the data is input into the asymmetric dense convolutional neural network model, a series of preprocessing operations need to be performed on the data, which includes image data and video data.

[0029] Image data preprocessing mainly includes data cleaning, data enhancement, format conversion, etc. First, invalid and abnormal images in the data are removed, and data enhancement is performed by mosaic enhancement, mixed enhancement, random perturbation, and color perturbation to increase data diversity. These steps help improve the generalization ability and detection accuracy of the model and further improve the quality of image data. Finally, the image is scaled to the fixed size required by the asymmetric dense convolutional neural network model, and normalized to ensure that the numerical range of the input data meets the requirements of the model.

[0030] Video data preprocessing mainly includes frame extraction and frame image preprocessing; first, an evenly spaced extraction of 5 frames per second is used as the basic method for frame extraction to ensure that the timeline in the video is evenly covered. This frame extraction interval can be adjusted accordingly according to the degree of motion of the target to be detected in the video, that is, when the target to be detected in the video is relatively still, the frame extraction interval can be increased to increase the detection speed; when the target to be detected in the video is in motion, the frame extraction interval can be reduced to reduce the probability of missed detection. After the frame extraction is completed, the extracted frame image is subjected to the related operations of the image data preprocessing.

[0031] Step 2: Feature extraction

[0032] After preprocessing, the data is input into an asymmetric dense convolutional neural network for feature extraction, providing feature representation with high expressiveness and information content for subsequent high-accuracy classification and positioning.

[0033] (1) Network structure construction

[0034] Different from the current lightweight network design that pursues faster speed and fewer parameters, the design of the present invention focuses on improving the feature extraction capability of the lightweight network. The present invention constructs an asymmetric dense convolutional neural network to improve the quality of feature extraction, increase information flow, and enhance the network expression capability. The network structure is shown in Table 1. The initial stage of the network is the preliminary extraction module, which receives an input image of size 224*224*3 and outputs a feature map of size 56*56*32. Subsequently, the network performs in-depth feature extraction and dimensionality reduction through four dense blocks and three transition layers. Each dense block is composed of a two-way asymmetric dense layer, in which dense block one is repeated 3 times and outputs a feature map of size 56*56*128; dense block two is repeated 4 times and outputs a feature map of size 28*28*256; dense blocks three and four are repeated 8 times, and output feature maps of size 14*14*512 and 7*7*768 respectively. The transition layer uses 1*1 convolution and 2*2 average pooling operations to halve the spatial size of the feature map and increase the number of channels. The three transition layers are located between two adjacent dense blocks, and each transition layer provides input for subsequent dense blocks. Finally, the network compresses the 7*7*768 feature map to 1*1*768 through a classification layer composed of a global average pooling layer, a fully connected layer, and a softmax layer. The architecture achieves efficient feature extraction of input images through a carefully designed modular and hierarchical structure.

[0035] Table 1 Network structure

[0036]

[0037] (2) Preliminary extraction module

[0038] In an asymmetric dense convolutional neural network, the preliminary extraction module is used to extract shallow features of the original image, filter out useless background information, and perform dimensionality reduction on the feature map. If the preliminary extraction module is not designed properly, it will destroy the effective information in the image and seriously affect the quality of subsequent feature extraction. Typical deep networks usually use convolution layers with large 7*7 or 5*5 convolution kernels, and then pass through a 3*3 maximum pooling layer as a preliminary extraction module. This can complete the preliminary extraction with high quality, but this heavy structure is not suitable for lightweight network design. The present invention designs a more efficient preliminary extraction module, which is achieved by fusing the outputs of a 3*3 convolutional layer and a 2*2 maximum pooling layer, such as Figure 2 As shown, the data is input into the first 3*3 convolutional layer, the output data of the first 3*3 convolutional layer enters the 1*1 convolutional layer, the output data of the 1*1 convolutional layer enters the second 3*3 convolutional layer and the 2*2 maximum pooling layer respectively, and the outputs of the second 3*3 convolutional layer and the 2*2 maximum pooling layer are spliced ​​as the output result of the preliminary extraction module.

[0039] (3) Dual-path asymmetric dense layer

[0040] The present invention constructs a dual-path asymmetric dense layer based on asymmetric depth-separable convolution, a dual-path convolution extraction structure, a dense connection method and a feature channel separation and shuffling operation.

[0041] Compared with standard convolution, depthwise separable convolution will lead to more information loss and result in weakened feature expression. However, lightweight network design requires the computational cost savings brought by depthwise separable convolution. Therefore, by decomposing the depthwise convolution layer into two asymmetric layers in space, the nonlinearity can be enhanced by increasing the depth of the convolution layer to achieve better feature expression.

[0042] Based on depthwise separable convolution and asymmetric convolution, this paper proposes an asymmetric depthwise separable convolution, which uses convolution decomposition to improve depthwise separable convolution. It decomposes the depthwise convolution layer into two layers with asymmetric convolution kernels and combines them with pointwise convolution.

[0043] The computation cost of standard convolution, asymmetric convolution and depth-separable convolution operations is s Cost a Cost dw as follows:

[0044] Cost s =H i ×W i ×M×N×D k ×D k

[0045] Cost a =H i ×W i ×M×M×D k ×1+H i ×W i ×M×N×1×D k

[0046] Cost dw =H i ×W i ×M×D k ×D k +M×N×D k ×D k

[0047] Among them, H i , W i Respectively represent the length and width of the input image, M represents the number of input channels, N represents the number of output channels, and D k Represents the spatial size of the convolution kernel.

[0048] The computational cost of asymmetric depth-separable convolution is adw for:

[0049] Cost adw =H i ×W i ×M×D k +H i ×W i ×M×1×D k ×1+H i ×W i

[0050] ×M×N×1×1

[0051] By comparing the respective computational loads (H i =W i =7, M = 64, N = 32), we can see that the asymmetric depthwise separable convolution has the highest computational compression rate, see Table 2.

[0052] Table 2 Comparison of the computational complexity of four convolutions under the same settings

[0053]

[0054]

[0055] Considering that only using asymmetric depth-wise separable convolution to construct an asymmetric dense convolution layer will limit the network feature extraction capability. Therefore, the present invention proposes a dual-path asymmetric dense layer structure. The input of the dual-path asymmetric dense layer is a feature map with k0 channels (c=k0). In the dual-path asymmetric dense layer, first, the input feature map undergoes a channel separation operation and is divided into two parallel paths: a single-layer asymmetric depth-wise separable convolution path and a double-layer asymmetric depth-wise separable convolution path; in the single-layer asymmetric depth-wise separable convolution path, the feature map first passes through a 1*1 ordinary convolution layer to generate a feature map with k channels (c=k), and then passes through an asymmetric depth-wise separable convolution layer composed of two 1*3 and 3*1 depth-wise convolutions and a 1*1 point-by-point convolution to generate a feature map with k / 2 channels (c=k / 2); in the double-layer asymmetric depth-wise separable convolution path, the feature map is firstly generated through a 1*1 ordinary convolution layer and a 1*1 point-by-point convolution. In the depth-separable convolution path, the feature map first passes through a 1*1 ordinary convolution layer to generate a feature map with 2k channels (c=2k), and then passes through two asymmetric depth-separable convolution layers in sequence for feature extraction and channel compression to generate a feature map with k / 2 channels (c=k / 2); then, the output feature maps of the two asymmetric depth-separable convolution paths are concatenated with the original input feature map to merge the channel information; finally, through the channel shuffling operation, the merged feature map is rearranged to generate the final output feature map.

[0056] By adding an additional convolution path with a double-layer asymmetric depth-separable convolution structure, the receptive field can be effectively expanded and the learning of large target features can be enhanced. The convolution layers of the two paths have different receptive fields, which enables the network to learn rich features and make up for the loss of accuracy caused by insufficient feature learning ability. Considering that adding a path will increase the computational complexity to twice the original, in order to ensure that the network architecture is lightweight and efficient, the present invention uses a channel splitting operation to divide the input channels into two equal groups to control the number of input feature channels. Since channel splitting prevents information from flowing between channel groups, channel shuffling is used at the end of the convolution layer to make the input channels and output channels completely related. In addition, since the deep convolutional neural network model designed by the present invention has a deep level, a dense connection structure is used to construct the convolution layer to solve the problem of gradient disappearance and information transmission in subsequent layers. In each dense block, the dense layer can directly establish a connection with all subsequent layers, and the input of each layer is the sum of the outputs of all previous layers. The dense layer can fully accept feature information from all previous layers to achieve feature reuse, thereby enhancing feature expression capabilities.

[0057] (4) Transition layer

[0058] The transition layer consists of a convolution layer and an average pooling layer. After the output of the dual-path asymmetric dense convolution layer enters the transition layer, the convolution layer first integrates the channel information, and then downsamples the feature map through the average pooling layer. Since the dual-path asymmetric dense convolution layer structure network proposed in the present invention has a small number of channels, further compression processing in the transition layer may seriously cause information loss, destroy the feature expression ability and significantly reduce the accuracy. Therefore, channel compression is not performed in the transition layer, and the convolution layer is only used to integrate multi-channel information.

[0059] Step 3: Communication equipment detection

[0060] After feature extraction, the feature map finally generated by the asymmetric dense convolutional neural network is used through the prediction layer to classify and locate the communication device.

[0061] The feature map includes multiple regions, each of which includes multiple grid units. The prediction layer first generates a series of target candidate boxes for each region of the feature map to cover possible targets. The target candidate box has two parameters: size and aspect ratio. First, the base size is calculated, and then the target candidate boxes with different aspect ratios are calculated based on the base size and aspect ratio coefficient. The prediction layer uses target candidate boxes of different sizes for feature maps of different scales. The base size S of the target candidate box of the first prediction layer is min = 0.2, the base size S of the target candidate box of the fifth prediction layer max = 0.95, the benchmark size S of the target candidate box of the other layers prediction layer k By calculation:

[0062]

[0063] Wherein, n is the total number of prediction layers (n=5 in the present invention).

[0064] Set the set of aspect ratio coefficients of five target candidate boxes The height of the target candidate box can be calculated With width

[0065]

[0066] In addition, an aspect ratio factor a is added r ′=1, base size There are 6 target candidate boxes of different sizes in total.

[0067] After that, the bounding box and class probability of each grid cell are output. Each grid cell is used to predict a set of bounding boxes and corresponding class probabilities, which represent the location and size of possible communication devices. In this way, the system can simultaneously identify and locate the communication devices and generate corresponding prediction results for each predicted communication device.

[0068] In the present invention, five feature maps of different scales are selected for prediction, and six target candidate boxes of different sizes are constructed for each pixel point in the feature maps of each scale of 56*56, 28*28, 14*14, 7*7, and 1*1 for detection and classification, and multiple bounding boxes are generated. These bounding boxes and corresponding category probabilities are the prediction results output by each prediction layer, which are part of the output candidate boxes after bounding box regression adjustment.

[0069] Finally, non-maximum suppression is performed on the detection frames generated on the feature maps of each scale, and a part of the overlapping or incorrectly detected detection frames are filtered out to obtain the final communication device detection results.

[0070] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A lightweight communication device detection method based on an asymmetric dense convolutional neural network, characterized in that: The following steps are involved: Step 1: Data preprocessing Before the data is input into the asymmetric dense convolutional neural network, a series of preprocessing operations are performed on the data, which includes image data and video data; Image data preprocessing includes data cleaning, data enhancement, and format conversion. First, invalid and abnormal images in the data are removed, and data enhancement is performed using mosaic enhancement, mixed enhancement, random perturbation, and color perturbation. Finally, the image is scaled to the fixed size required by the asymmetric dense convolutional neural network model, and normalized to ensure that the numerical range of the input data meets the requirements of the model. Video data preprocessing includes frame extraction and frame image preprocessing; first, an evenly spaced extraction of N frames per second is used as a frame extraction method to ensure that the timeline in the video is evenly covered, and the frame extraction interval is adjusted accordingly according to the degree of motion of the target to be detected in the video, that is, when the target to be detected in the video is relatively still, the frame extraction interval is increased; when the target to be detected in the video is in motion, the frame extraction interval is reduced; after the frame extraction is completed, the extracted frame image is subjected to the related operations of the image data preprocessing; Step 2: Feature extraction After preprocessing, the data is input into an asymmetric dense convolutional neural network for feature extraction, providing feature representation with certain expressiveness and information content for subsequent high-accuracy classification and positioning; wherein, the asymmetric dense convolutional neural network is designed as follows: in the initial stage, a preliminary extraction module is used to receive an input image of size 224*224*3 and output a feature map of size 56*56*32; subsequently, the network performs in-depth feature extraction and dimensionality reduction through four dense blocks and three transition layers; each dense block is composed of a two-way asymmetric dense layer, in which a dense block is repeated 3 times and outputs a feature map of size 56*56*128 Feature map; Dense block 2 is repeated 4 times to output a 28*28*256 feature map; Dense block 3 and dense block 4 are repeated 8 times, outputting feature maps of 14*14*512 and 7*7*768 respectively; The transition layer uses 1*1 convolution and 2*2 average pooling operations to halve the spatial size of the feature map and increase the number of channels. The three transition layers are located between two adjacent dense blocks, and each transition layer provides input for the subsequent dense block; Finally, the network compresses the 7*7*768 feature map to 1*1*768 through the classification layer composed of a global average pooling layer, a fully connected layer, and a softmax layer; Step 3: Communication equipment detection After feature extraction, the feature map finally generated by the asymmetric dense convolutional neural network is used through the prediction layer to classify and locate the communication device.

2. The method according to claim 1, characterized in that In the asymmetric dense convolutional neural network, the preliminary extraction module is used to extract shallow features of the original image, filter out useless background information, and reduce the dimension of the feature map at the same time; the preliminary extraction module is realized by fusing the outputs of the 3*3 convolutional layer and the 2*2 maximum pooling layer, wherein data is input into the first 3*3 convolutional layer, data output by the first 3*3 convolutional layer enters the 1*1 convolutional layer, data output by the 1*1 convolutional layer enters the second 3*3 convolutional layer and the 2*2 maximum pooling layer respectively, and the outputs of the second 3*3 convolutional layer and the 2*2 maximum pooling layer are spliced ​​as the output result of the preliminary extraction module.

3. The method according to claim 1, characterized in that The dual-path asymmetric dense layer is constructed based on asymmetric depth-separable convolution, dual-path convolution extraction structure, dense connection method and feature channel separation and shuffling operation.

4. The method according to claim 1, characterized in that The input of the dual-path asymmetric dense layer is a feature map with k0 channels. In the dual-path asymmetric dense layer, first, the input feature map undergoes a channel separation operation and is divided into two parallel paths: a single-layer asymmetric depth-separable convolution path and a double-layer asymmetric depth-separable convolution path; in the single-layer asymmetric depth-separable convolution path, the feature map first passes through a 1*1 ordinary convolution layer to generate a feature map with k channels, and then passes through an asymmetric depth-separable convolution layer composed of two 1*3 and 3*1 depth convolutions and a 1*1 point-by-point convolution to generate k channels. / 2 channel feature map; in the double-layer asymmetric depth-wise separable convolution path, the feature map first passes through a 1*1 ordinary convolution layer to generate a feature map with 2k channels, and then passes through two asymmetric depth-wise separable convolution layers in sequence for feature extraction and channel compression to generate a feature map with k / 2 channels; then, the output feature maps of the two asymmetric depth-wise separable convolution paths and the original input feature maps are concatenated to merge the channel information; finally, through the channel shuffling operation, the merged feature map is rearranged to generate the final output feature map.

5. The method according to claim 1, characterized in that The transition layer is composed of a convolution layer and an average pooling layer. After the output of the dual-path asymmetric dense convolution layer enters the transition layer, the convolution layer first integrates the channel information, and then the average pooling layer downsamples the feature map.

6. The method according to claim 1, characterized in that The feature map includes multiple regions, each of which includes multiple grid cells. In step 3, the prediction layer first generates a series of target candidate boxes for each region of the feature map to cover possible targets. The target candidate box has two parameters: size and aspect ratio. First, the base size is calculated, and then target candidate boxes with different aspect ratios are calculated based on the base size and aspect ratio coefficient. Target candidate boxes of different sizes are used for prediction layers of feature maps of different scales. The base size S of the target candidate box of the first prediction layer min = 0.2, the base size S of the target candidate box of the fifth prediction layer max = 0.95, the benchmark size S of the target candidate box of the other layers prediction layer k By calculation: Among them, n is the total number of prediction layers, and its value is 5; Set the set of aspect ratio coefficients of five target candidate boxes Calculate the height of the target candidate box With width In addition, an aspect ratio factor a is added r ′=1, base size There are 6 target candidate boxes of different sizes in total. Afterwards, the bounding box and category probability of each grid unit are output; each grid unit is used to predict a set of bounding boxes and corresponding category probabilities, and these bounding boxes represent the location and size of possible communication devices. In this way, the communication devices can be identified and located at the same time, and a corresponding prediction result can be generated for each predicted communication device.

7. The method according to claim 6, characterized in that In step three, five feature maps of different scales are selected for prediction. Six target candidate boxes of different sizes are constructed for each pixel point in the feature maps of each scale of 56*56, 28*28, 14*14, 7*7, and 1*1 for detection and classification. Multiple bounding boxes are generated. These bounding boxes and the corresponding category probabilities are the prediction results output by each prediction layer. They are part of the output candidate boxes after bounding box regression adjustment. Finally, the detection boxes generated on the feature maps of each scale are non-maximum suppressed, and a part of the overlapping or incorrectly detected detection boxes are filtered out to obtain the final communication device detection results.

8. The method according to any one of claims 1 to 7, characterized in that This method is used in the fields of artificial intelligence, computer vision, and target detection technology.

9. A system for use in the method according to any one of claims 1 to 8.

10. The system according to claim 9, characterized in that The system is used in the fields of artificial intelligence, computer vision, and target detection technology.

Images