Lightweight calling detection method based on double-channel attention
By introducing dual-channel attention feature fusion and lightweight separable ghost convolution module in image recognition technology, the activation function and loss function are improved, and the generalization capability and real-time detection requirements of the existing technology in telephone behavior recognition are solved. The problem of this achieves efficient and accurate call detection.
Patent Information
- Application Number
- CN202410771890.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-15
- Publication Date
- 2024-12-27
AI Technical Summary
Existing image recognition technology lacks generalization ability when identifying telephone calls, especially in real-time application scenarios, which cannot meet the needs of rapid identification. Traditional methods rely on manual features while deep learning methods perform poorly in new scenarios.
Using a lightweight call detection method based on dual-channel attention, the C3 layer in the initial Yolov5 network architecture is replaced with a lightweight separable ghost convolution module, and the dual-channel attention feature fusion module is added, and the activation function is improved to H- Swish and change the loss function to the EIOU loss function to form an efficient detection model.
It improves the inference speed and accuracy of the model, enhances the understanding and prediction capabilities of information, expands the application scenarios and practicality, and ensures real-time detection on resource-constrained devices.
Smart Images
Figure CN119206851A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of target detection in image processing, and more specifically to a lightweight phone call detection method based on dual-channel attention. Background Art
[0002] In the field of artificial intelligence and machine vision, image recognition technology has become an important research direction. Image recognition technology enables computer systems to understand and interpret visual information, thereby realizing automatic analysis and understanding of the content of pictures. Especially in the field of behavior recognition, various behavior patterns such as walking and running can be identified by analyzing the actions of people in images. However, recognizing more complex and subtle behaviors, such as making phone calls, remains a challenge. Existing image recognition technologies mainly include methods based on traditional computer vision and methods based on deep learning. Traditional methods usually rely on manually designed features such as SIFT (Scale Invariant Feature Transform) and HOG (Histogram of Oriented Gradients), which require expert knowledge and a lot of manual adjustments. Deep learning-based methods, especially convolutional neural networks (CNNs), have achieved remarkable results in many fields such as image classification, object detection, and behavior recognition by automatically learning image features.
[0003] Although deep learning methods have achieved great success in the field of image recognition, they still have some limitations when it comes to recognizing the specific behavior of making phone calls. For example, existing models may perform well on specific data sets, but their generalization capabilities may be insufficient when faced with new or unseen scenarios; in application scenarios that require real-time feedback, such as video surveillance or interactive systems, existing technologies may not be able to meet the needs of fast recognition. Summary of the invention
[0004] In view of this, in order to solve the above technical problems, the purpose of the present invention is to provide a lightweight phone call detection method based on dual-channel attention to improve the performance of target detection tasks. According to the characteristics of lightweight and high precision of this method, it can be effectively applied to actual scenarios and has good practicality.
[0005] The present invention is implemented by the following technical solution, which is described below using Yolov5 as the initial network architecture.
[0006] A lightweight phone call detection method based on dual-channel attention mainly includes the following steps:
[0007] The first step is to replace the C3 layer of the Yolov5 initial network architecture of the original phone call detection model with a lightweight separable ghost convolution module to reduce the model's parameter size and computational complexity, thereby improving the model's inference speed.
[0008] The second step is to add a dual-channel attention feature fusion module to the lightweight separable ghost convolution module to improve the model's ability to understand and predict information;
[0009] The third step is to improve the original model activation function to the H-Swish activation function to reduce the amount of model calculation;
[0010] The fourth step is to change the initial CIOU loss function of the original model to the EIOU loss function, thereby obtaining an improved model for phone call detection to further improve the accuracy of the model;
[0011] The fifth step is to train the improved model with data to obtain a lightweight phone call detection model based on dual-channel attention.
[0012] Finally, the inspection data is input into the molding model to output the inspection results.
[0013] Ghost network is an efficient module design that reduces the amount of computation by generating more "ghost feature maps". This method proposes an innovative method of introducing deep separable convolution into the traditional ghost network module, aiming to further optimize the lightweight and performance of the model. The innovation of this method lies in combining the parameter and computational advantages of deep separable convolution. By introducing these two technologies into the ghost network module, it not only achieves effective feature extraction and fusion, but also enhances the model's adaptability to different input features. This combination improves the performance and generalization ability of the model while maintaining lightweight.
[0014] The dual-channel attention feature fusion module only adds a small number of parameters, but can achieve significant performance gains. It can effectively avoid dimensionality reduction and capture cross-channel interactive information, aiming to ensure information efficiency and effectiveness.
[0015] The H-Swish activation function is composed of common operators, so it can be implemented in almost all frameworks, and the amount of calculation is greatly reduced without greatly affecting the effect.
[0016] As a further improvement of the above scheme, the lightweight separable ghost convolution module decomposes the standard convolution operation into two parts: depthwise convolution and pointwise convolution to reduce the number of parameters and computation; the depthwise convolution applies a separate convolution kernel to each channel of the input, while the pointwise convolution is a 1x1 convolution used to fuse the output of the depthwise convolution.
[0017] As a further improvement of the above scheme, the dual-channel attention feature fusion module involves two dimensions: horizontal sequence features and vertical sequence features in the image.
[0018] As a further improvement of the above scheme, the dual-channel attention feature fusion module is a plug-and-play block to improve the performance of the model network architecture.
[0019] In order to minimize the interference to the original architecture of Yolov5, the dual-channel attention feature fusion module is designed as a pluggable component, so that experiments can be conducted at different stages to find the best integration point. In this way, the Yolov5 model can not only use its original powerful object detection capabilities, but also better understand and utilize the information in the data through the dual-channel attention feature fusion mechanism, thereby achieving higher accuracy and efficiency in specific application scenarios.
[0020] As a further improvement of the above scheme, the penalty term of the EIOU loss function is to separate the influencing factors of the aspect ratio on the basis of the penalty term of the CIOU loss function, so as to calculate the length and width of the target box and the anchor box respectively.
[0021] As a further improvement of the above scheme, the EIOU loss function includes three parts: overlap loss, center distance loss, and width-height loss. The first two parts continue the method in the CIOU loss function, but the width-height loss directly minimizes the difference between the width and height of the target box and the anchor box, making the convergence speed faster.
[0022] The original model's initial CIOU loss function adds the loss of the detection box scale on the basis of the DIOU loss function, and adds the loss of length and width, so that the predicted box will be more consistent with the real box. However, the disadvantages are: the aspect ratio describes a relative value, which is somewhat vague; and the balance problem of difficult and easy samples is not considered. In view of the above problems, the EIOU loss function can be effectively solved. By changing the CIOU loss function to the EIOU loss function, the accuracy of the model can be further improved.
[0023] The beneficial effects of the present invention are as follows: by replacing the C3 layer of the Yolov5 initial network architecture with a lightweight module separable ghost convolution module, the reasoning speed of the model is greatly improved, and the parameter size and calculation amount of the model are reduced while ensuring the accuracy of the initial network architecture; adding dual-channel attention feature fusion, at the cost of adding a small number of parameters, a significant performance gain is obtained; the CIOU loss function is changed and replaced with the EIOU loss function, so that the accuracy of the model can be further improved; the original model activation function is improved to the H-Swish activation function, which greatly reduces the amount of calculation without having a significant impact on the effect, thereby expanding the application scenarios and practicality. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] Figure 1 This is a diagram of the improved network architecture of the present invention based on the Yolov5 network. DETAILED DESCRIPTION
[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0026] Reference Figure 1 , an embodiment of the present invention provides a lightweight phone call detection method based on dual-channel attention, including steps S1-S5.
[0027] S1. Improve the model: optimize accuracy and inference speed.
[0028] S2, Yolov5 deep separable ghost convolution lightweight module.
[0029] In the traditional Yolov5 network, the backbone network adopts a more complex C3 network structure, which greatly increases the model calculation amount, slows down the detection speed, and limits the application. In the real phone call detection scenario, on some devices with limited computing power, real-time reasoning detection cannot be achieved. In order to solve this problem, the embodiment of the present invention introduces a lightweight separable ghost convolution module to reduce the model's calculation amount.
[0030] In the field of deep learning, the design of lightweight network models is to reduce computational complexity and the number of parameters while maintaining high performance, making the model more suitable for running on resource-constrained devices. Ghost convolutional network is an efficient module design that reduces the amount of computation by generating more feature maps "ghost features". The embodiment of the present invention proposes an innovative method of introducing deep separable convolution into the traditional ghost convolutional network module, aiming to further optimize the lightweight and performance of the model. The innovation of the embodiment of the present invention lies in combining the parameter and computational advantages of deep separable convolution. By introducing these two technologies into the ghost convolutional network module, not only the effective extraction and fusion of features are achieved, but also the adaptability of the model to different input features is enhanced. This combination improves the performance and generalization ability of the model while maintaining lightweight.
[0031] Depthwise separable convolution decomposes the standard convolution operation into two parts: depthwise convolution and pointwise convolution. Depthwise convolution applies a separate convolution kernel to each channel of the input, while pointwise convolution is a 1x1 convolution that fuses the output of depthwise convolution. This decomposition greatly reduces the number of parameters and computation.
[0032] In the embodiment of the present invention, the Pytorch implementation of the deep separable ghost convolution lightweight module is completed, and it is embedded into the backbone network of the Yolov5 target detection model.
[0033] S3. Dual-channel attention feature fusion mechanism.
[0034] The dual-channel attention feature fusion mechanism enables the model to pay more attention to important features when processing image data, improving the model's understanding and prediction capabilities of information. The dual-channel attention feature fusion mechanism usually involves two dimensions: the first dimension focuses on the horizontal sequence features in the image, and the second dimension focuses on the vertical sequence features in the image. Combining these two attention features can enable the model to more effectively capture key information in the data. An embodiment of the present invention designs a comprehensive dual-channel attention feature fusion module, which simultaneously considers the correlation between horizontal and vertical sequences, and can simultaneously process the interactive effects of horizontal and vertical sequences.
[0035] After processing the features of the horizontal and vertical channels independently, a fusion strategy is needed to integrate this information. In the attention feature fusion stage, an attention fusion module can be used to achieve better feature fusion. Using a better fusion strategy, such as the IAFF (Interactive Attentional Feature Fusion) module, multi-dimensional features can be effectively integrated. The IAFF module learns the interaction between different features and intelligently adjusts the contribution of each feature to achieve more refined feature fusion. The following is the technical principle of the IAFF module and its application derivation in spatiotemporal fusion.
[0036] The core idea of the IAFF module is to use the attention mechanism to dynamically adjust the importance of different channel features in the fusion process. Specifically, the IAFF module first interactively learns the horizontal channel features and the vertical channel features, then generates attention weights based on the learned interactive information, and finally applies these weights to adjust the original features to achieve effective feature fusion.
[0037] Assume there is a lateral channel feature map And the longitudinal channel feature map Where C is the number of channels, H and W are the height and width respectively. First, feature interaction operations are performed on X and Y to learn the dependencies between them:
[0038] F sum =[X, Y]
[0039] Among them, [X, Y] represents feature concatenation. sumApply point convolution, which is a common operation for adjusting the number of channels and keeping the spatial dimension unchanged. At the same time, use the final fused feature map Z as an additional feedback link to iteratively update the attention map. Next, introduce nonlinearity through the ReLU activation function.
[0040] F conv =ReLU(F pw (F sum )+Z n-1 )
[0041] F pw Represents the point convolution operation, Z n-1 It is the feature map obtained after the last cycle. Channel attention is extracted through two branches of different scales. On the global scale branch, the input feature map F conv Perform a global average pooling operation, which compresses the spatial information of each channel into a single value, resulting in a global feature vector F global .
[0042] F global =F gap (F conv )
[0043] F gap represents the local average pooling operation, and the global feature vector F global First, it is processed by point convolution and ReLU activation function, then by point convolution again, and then batch normalization is applied to generate channel attention map A channel .
[0044] A chnnel =F bn (F pw (ReLU(F bn (F pw (F global )))))
[0045] F bn Represents the batch normalization operation, merging the feature map F on the local scale branch conv It is also processed by point convolution and ReLU activation function, and then again by point convolution and batch normalization to generate the spatial attention map A spatial .
[0046] A spatial =F bn (F pw (ReLU(F bn (F conv ))))
[0047] Finally, the channel attention map A channel and the spatial attention map A spatialAfter adding and passing through the ReLU activation function, they are respectively multiplied element-by-element with the original input feature maps X and Y, and then the weighted feature maps are added to fuse the information.
[0048]
[0049] Represents element-by-element multiplication. In this process, the update of the attention map is affected by the previous fusion result Z n-1 This design allows the network to self-correct according to the fusion effect in each round of iteration, thereby gradually optimizing the attention weights and ultimately achieving more refined feature fusion. The key point of this feedback mechanism is that it provides a mechanism that allows the model to continue to learn and adapt during the iteration process, which may improve the long-term feature fusion strategy. By continuously iterating during training, the model can better learn how to combine feature information from different sources.
[0050] In this structure, the attention map is generated through point convolution and global average pooling, which realizes the adaptive fusion between features. Such a mechanism enables the network to focus on more informative features while suppressing unimportant features, thereby improving the network's ability to understand input data and enhancing the model's ability to process complex data.
[0051] S4, Swish activation function.
[0052] Swish is an activation function that has smooth nonlinear properties and has shown better performance than other commonly used activation functions in some cases.
[0053] The definition of the Swish activation function is as follows:
[0054] Swish(x)=x*Sigmiod(βx)
[0055] Where x is the input and β is a learnable parameter that adjusts the slope of the Sigmiod function, thereby affecting the nonlinear properties of the Swish function.
[0056] The Swish function has the following characteristics:
[0057] Smoothness: The Swish function is continuously differentiable over the entire real number range and has smooth nonlinear characteristics. This makes the model easier to optimize during training and may avoid the problem of gradient disappearance.
[0058] Adaptability: The Swish function is adaptive, and it can automatically adjust the activation amplitude according to the input value. In most cases, the Swish function will amplify the input to increase the effect of gradient propagation.
[0059] Preserve important information: The Swish function compresses the input when it is negative, but does not completely discard it. This helps to preserve some useful negative information and provide more comprehensive feature representation capabilities.
[0060] H-Swish is an activation function, which is an improved version of the Swish activation function. H-Swish is proposed to reduce the multiplication operations in the Swish function to further improve the computational efficiency.
[0061] The definition of the H-Swish activation function is as follows:
[0062] H-Swish(x)=x*relu6(x+3) / 6
[0063] Where x represents the input, relu6(x) represents the input x after the relu activation function and truncation, that is, relu6(x) = min(max(0, x), 6). The H-Swish function is similar to the Swish function and has nonlinear characteristics, but it uses the relu function to replace the sigmiod operation and multiplication operation in the Swish function, thereby reducing the computational complexity.
[0064] In the embodiment of the present invention, the backbone network and the head network change the relu activation function to the H-Swish activation function, so that the model can reduce the computational burden while still having a certain nonlinear expression ability.
[0065] S5. EIoU loss function.
[0066] Although the CIoU loss used by Yolov5 takes into account the overlapping area, center point distance, and aspect ratio of bounding box regression, its formula reflects the difference in aspect ratio rather than the actual difference between width and height and their confidence, which sometimes hinders the model from effectively optimizing similarity. To address this problem, EIoU loss separates the aspect ratio based on CIoU loss and adds Focal to focus on high-quality anchor boxes.
[0067] The EIoU metric corrects the shortcomings of the CIoU metric by introducing an additional offset. Specifically, the EIoU metric models the target localization problem as an optimal offset estimation problem. By minimizing the error between the corrected IoU and the true bounding box, a more accurate target localization result can be obtained.
[0068] The calculation process of the EIoU metric is as follows:
[0069] 1. Calculate the traditional IoU, which is the intersection area between the predicted bounding box and the true bounding box divided by their union area.
[0070] 2. Calculate the center offset of the bounding box to measure the center position error of the predicted bounding box relative to the true bounding box.
[0071] 3. Calculate the width offset and height offset of the bounding box to measure the scale error of the width and height of the predicted bounding box relative to the true bounding box.
[0072] 4. Calculate the corrected IoU based on the center offset, width offset, and height offset.
[0073] The penalty term of EIoU is based on the penalty term of CIoU, and the influencing factor of aspect ratio is separated to calculate the length and width of the target box and the anchor box respectively. The loss function consists of three parts: overlap loss, center distance loss, and width and height loss. The first two parts continue the method in CIoU, but the width and height loss directly minimizes the difference between the width and height of the target box and the anchor box, making the convergence faster.
[0074] Finally, the improved model is trained with data to obtain a lightweight phone call detection model based on dual-channel attention; the detection data is then input into the model to output the detection results.
[0075] In the embodiment of the present invention, the model uses the EIoU loss function to provide more accurate target positioning results and help improve the performance of target detection tasks.
[0076] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the present invention to be protected.
Claims
1. A lightweight phone call detection method based on dual-channel attention, characterized in that: The main steps include: The first step is to replace the C3 layer of the Yolov5 initial network architecture of the original phone call detection model with a lightweight separable ghost convolution module to reduce the model's parameter size and computational complexity, thereby improving the model's inference speed. The second step is to add a dual-channel attention feature fusion module to the lightweight separable ghost convolution module to improve the model's ability to understand and predict information; The third step is to improve the original model activation function to the H-Swish activation function to reduce the amount of model calculation; The fourth step is to change the initial CIOU loss function of the original model to the EIOU loss function, thereby obtaining an improved model for phone call detection to further improve the accuracy of the model; The fifth step is to train the improved model with data to obtain a lightweight phone call detection model based on dual-channel attention. Finally, the inspection data is input into the molding model to output the inspection results.
2. According to the lightweight phone call detection method based on dual-channel attention in claim 1, it is characterized in that: The lightweight separable ghost convolution module decomposes the standard convolution operation into two parts: depthwise convolution and pointwise convolution to reduce the number of parameters and computation; depthwise convolution applies a separate convolution kernel to each channel of the input, while pointwise convolution is a 1x1 convolution used to fuse the output of depthwise convolution.
3. According to the lightweight phone call detection method based on dual-channel attention in claim 1, it is characterized in that: The dual-channel attention feature fusion module involves two dimensions: horizontal sequence features and vertical sequence features in the image.
4. According to the lightweight phone call detection method based on dual-channel attention in claim 1, it is characterized in that: The dual-channel attention feature fusion module is a plug-and-play block to improve the performance of the model network architecture.
5. According to the lightweight phone call detection method based on dual-channel attention in claim 1, it is characterized in that: The penalty term of the EIOU loss function is to separate the influencing factors of the aspect ratio based on the penalty term of the CIOU loss function to calculate the length and width of the target box and the anchor box respectively.
6. A lightweight phone call detection method based on dual-channel attention according to claim 1, characterized in that: The EIOU loss function includes three parts: overlap loss, center distance loss and width and height loss.