Thunder and lightning target detection method based on deep learning, medium and equipment

Abstract

The invention discloses a thunder and lightning target detection method based on deep learning, a medium and equipment, a target detection model is constructed based on a YOLOv7 network model, a trunk extraction structure of the target detection model comprises a plurality of RepNCSPELAN4 feature extraction modules which are sequentially arranged from front to back, and CBS convolution in a feature enhancement structure is replaced with a CBAM attention module; and a 1 * 1 convolutional layer in the RepNCSPELAN4 feature extraction module is replaced by a CoordConv (CoordConv). According to the invention, through fusion of the RepNCSPELAN4 feature extraction module of CoordConv, the space structure and position relation of large-area blank and lightning in the sky can be better understood, and information loss is reduced. The phenomena of false detection and missing detection are further reduced through a CBAM attention module, higher detection precision is shown, and good practicability is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of visual detection of lightning targets, and specifically relates to a lightning target detection method, medium and equipment based on deep learning. Background Art

[0002] With the continuous deepening of lightning research, the observation and monitoring of lightning as a natural phenomenon has become increasingly important. Lightning is a high-energy current discharge phenomenon formed in the atmosphere, with extremely strong electromagnetic radiation and sound effects, usually accompanied by lightning, thunder, electromagnetic waves and other phenomena. This phenomenon not only brings strong visual and auditory impacts to people, but also has a huge impact on economic production, human life and the natural environment. The formation of lightning originates from the strong electric field and charge separation inside the cloud. When the charge accumulates to a certain level, discharge will occur, releasing huge current and energy. Because lightning is random and extremely destructive, it is particularly important to monitor and study lightning activities.

[0003] Current lightning monitoring methods are mainly divided into two categories: ground-based lightning monitoring and meteorological satellite monitoring. Traditional lightning detection methods usually rely on meteorological satellites or ground sensors to receive lightning signals. However, ground-based monitoring methods require the deployment of a complete set of sensor equipment, and the equipment can only be used for lightning detection, which is costly and requires multiple lightning observation stations to achieve full coverage. On the other hand, although meteorological satellite monitoring can obtain wide-area lightning signals and atmospheric images, it is also limited by the change in the relative position of the satellite and the ground, which makes it impossible to monitor a specific area for a long time. Therefore, it is particularly important to find a more cost-effective and efficient lightning monitoring method.

[0004] In actual lightning monitoring scenarios, existing methods have many problems. First, specialized instruments and equipment increase the cost and complexity of the system, and the use of these instruments also requires professional training for operators. Second, because lightning monitoring relies on the collection of electromagnetic signals, the data is susceptible to electromagnetic interference, which in turn affects the accuracy of the monitoring results. Finally, traditional methods usually require manual construction of feature extractors, which not only increases the cost of deployment and maintenance, but may also cause the effectiveness of the feature extractors to decrease due to scene changes. Therefore, although these systems can effectively monitor lightning, their high cost and operational complexity limit their widespread application.

[0005] Future lightning monitoring technology should focus on reducing costs, improving monitoring accuracy and simplifying operations in order to better adapt to changing application scenarios and needs. Lightning detection based on deep learning algorithms can alleviate this problem to a great extent. Combined with deep learning, efficient and intelligent detection of lightning prevention can be achieved. Lightning is a common natural phenomenon. Effective monitoring of lightning activity in the monitoring area plays a vital role in analyzing the characteristics of lightning activity and preventing lightning disasters. In view of the irregular lightning and low visibility in lightning scenes on rainy days, deep learning methods can be used to directly process and analyze lightning image data without relying on additional equipment. However, the background in thunderstorm weather is generally more complex and has more interference. This poses a challenge to our detection. If we do not process it and directly detect the target, false detection and missed detection may occur in lightning meteorological video detection. Summary of the invention

[0006] The purpose of the present invention is to provide a lightning target detection method, medium and device based on deep learning, aiming to solve the above-mentioned problems.

[0007] The present invention is mainly achieved through the following technical solutions:

[0008] A lightning target detection method based on deep learning is disclosed. A target detection model is constructed based on a YOLOv7 network model. The backbone extraction structure of the target detection model includes a plurality of RepNCSPELAN4 feature extraction modules arranged in sequence from front to back, and the CBS convolution in the feature enhancement structure of the target detection model is replaced by a CBAM attention module; the 1x1 convolution layer in the RepNCSPELAN4 feature extraction module is replaced by CoordConv, so that the convolution operation becomes position sensitive, and CoordConv adds position information as an additional channel in the input feature map; then, the lightning picture is input into the target detection model for detection.

[0009] In order to better implement the present invention, further, the CoordConv adds two dimensions of coordinate information corresponding to the X-axis and Y-axis on the input Feature Map; CoordConv adds i and j channels to the input channel, and when the weights of the coordinate layer and the convolution layer of CoordConv are non-zero, it supports learning functions with translation dependence.

[0010] In order to better realize the present invention, further, the CBAM attention module is composed of a channel attention module and a spatial attention module; the input feature map first passes through the channel attention module and outputs the channel attention weight, and the channel attention weight is multiplied by the input feature map and then sent to the spatial attention module and outputs the spatial attention weight, and the spatial attention weight is multiplied by the input feature map of the spatial attention mechanism to obtain the final feature map.

[0011] In order to better implement the present invention, further, global context awareness modules are respectively introduced into the channel attention module and the spatial attention module of the CBAM attention module; the feature map X is input, and global features are first generated by the global context awareness module, and then these global features are respectively combined when calculating the channel attention and the spatial attention.

[0012] In order to better implement the present invention, further, the processing of the CBAM attention module includes the following steps:

[0013] Step S1: The global context information of the feature map is captured through the global context awareness module to enhance the global representation ability of the feature map; each position feature in the feature map X is weightedly fused with the global feature to obtain the global context feature map Y:

[0014] Y=X+W v (Softmax(W k X)W q X)

[0015] Where: W V ,W K ,W Q is the linear transformation matrix;

[0016] Softmax is used to calculate the weight of global features;

[0017] Step S2: Combine the global feature vector g output by global average pooling with the channel attention module; assuming Mc is the weight of the channel attention, the channel attention weight after combining the global information is:

[0018] X' C =M c ×X+λ×g

[0019] Among them, λ is a learnable parameter that controls the influence of global information;

[0020] Step S3: Finally, the global context feature map Y is combined with the spatial attention module; Y is combined with each position feature of the input feature map to form a globally perceived spatial attention weight:

[0021] X' s =Ms ×X+μ×Y

[0022] Among them, μ is another learnable parameter that controls the influence of global information in spatial attention.

[0023] In order to better implement the present invention, further, the global context perception module is a global context encoding GCNet, and the global context encoding GCNet integrates global information into each position feature by calculating the weighted global features of the feature map.

[0024] A computer-readable storage medium stores a computer program, which implements the above method when executed by a processor.

[0025] An electronic device comprises a memory and a processor; the memory stores a computer program; the processor is used to execute the computer program in the memory to implement the above method.

[0026] The beneficial effects of the present invention are as follows:

[0027] The present invention introduces the RepNCSPELAN4 feature extraction module into the backbone extraction structure (Backbone), which enhances the feature extraction capability of the lightning dataset under strong noise conditions. At the same time, the main convolution operation in RepNCSPELAN4 is improved to CoordConv convolution, so that the convolution operation has position sensitivity, and the detection effect is improved by adding position information as an additional channel in the input feature map. In addition, the present invention replaces the traditional CBS convolution with the CBAM attention mechanism in the feature enhancement structure (Neck layer), and improves CBAM, respectively introducing global context awareness modules into the channel attention module and spatial attention module of CBAM, thereby reducing the phenomenon of false detection and missed detection. Experimental results show that on the self-made sampling dataset, the mAP50 value of the RCC-YOLOv7 algorithm reaches 77.1%, which is 3.2% higher than the traditional YOLOv7 algorithm, showing higher detection accuracy and effect, and has good practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 Schematic diagram of the overall network structure of the target detection model of the present invention;

[0029] Figure 2 It is a schematic diagram of the structure of the RepNCSPELAN4 feature extraction module;

[0030] Figure 3 Schematic diagram of the CBAM attention module. DETAILED DESCRIPTION

[0031] Embodiment 1:

[0032] A lightning target detection method based on deep learning, such as Figure 1 As shown in the figure, in order to solve the problems of false detection and missed detection that may occur in lightning meteorological video detection, an improved algorithm RCC-YOLOv7 is generated based on the YOLOv7 network, and a target detection model is created. The RepNCSPELAN4 feature extraction module is introduced into the backbone extraction structure (Backbone) of the YOLOv7 network to enhance the feature extraction capability of lightning datasets under strong noise conditions. At the same time, the main convolution operation in RepNCSPELAN4 is improved to CoordConv convolution, making the convolution operation position-sensitive. By adding position information as an additional channel in the input feature map, the detection effect is improved, and the model's feature extraction for complex scenes is enhanced. In the feature enhancement structure (Neck layer) of the YOLOv7 network, the CBAM attention mechanism replaces the traditional CBS convolution, and the CBAM is improved to effectively integrate the global context information into the channel and spatial attention calculation of CBAM, enhance the modeling capability for complex scenes, and improve the overall performance of the model, thereby reducing the phenomenon of false detection and missed detection and improving the accuracy of lightning detection.

[0033] RepNCSPELAN4 feature extraction module integrated with CoordConv: The RepNCSPELAN4 feature extraction module is improved by replacing the key convolution CoordConv, which enhances the model's feature extraction for complex scenes. CoordConv is used to replace the original 1x1 convolution layer. By using CoordConv, spatial coordinate information is introduced in the convolution process, which helps the network better understand the spatial structure and positional relationship between large areas of sky blankness and lightning. Traditional convolution operations may lose position information, while CoordConv directly integrates position information into the convolution operation, which helps reduce information loss.

[0034] RepNCSP and RepNCSPELAN4 each use different techniques to improve the performance of deep learning models. For example, RepNCSP introduces the CSP (Cross Stage Partial) network architecture, which is commonly used in object detection models such as YOLOv4. This architecture reduces the number of parameters and speeds up the calculation process by splitting the input feature map and processing it in different modules before merging them. Figure 2 As shown in the figure, the RepNCSPELAN4 class further extends this design by connecting multiple CSPs and bottleneck structures in series to process more complex feature combinations and is suitable for more advanced image processing tasks. This modular and hierarchical design not only optimizes the computational efficiency of the network, but also improves the flexibility and expressiveness of the model when processing complex visual tasks.

[0035] CoordConv inherits the excellent characteristics of traditional convolution and effectively solves the problem of poor ability of traditional convolution to obtain spatial information. Compared with traditional convolution operation, CoordConv adds two additional dimensions to the input Feature Map, which correspond to the coordinate information of the X-axis and Y-axis respectively. By introducing these two coordinate dimensions, CoordConv not only enhances the network's spatial information perception ability, but also effectively improves the network's information expression ability in lightning target detection tasks. On the basis of retaining the first two significant characteristics, CoordConv gives the network flexibility, enabling it to independently decide whether to retain or ignore the third point characteristic, namely translation invariance, according to the needs of specific tasks. The implementation principle of CoordConv is to attach two additional i and j channels to the input channel. When the weights of the coordinate layer and the convolution layer of CoordConv are non-zero, it will support the learning of functions with translation dependence; conversely, if these weights are set to zero, its function will be similar to that of a conventional convolution layer.

[0036] Regarding the CBAM attention module that introduces the global context awareness module: The present invention introduces the global context awareness module into the channel attention module and the spatial attention module of CBAM respectively. When processing the input feature map X, the global features are first generated by the global context awareness module, and then these global features are combined when calculating the channel attention and spatial attention respectively. This improved CBAM module is integrated into the backbone network, and the model is trained to learn the attention weights that combine the global information in the channel and spatial dimensions.

[0037] like Figure 3 As shown in the figure, the CBAM attention mechanism is composed of a channel attention mechanism (channel) and a spatial attention mechanism (spatial). Global context information is further captured through global context encoding (such as GCNet) to enhance the global representation ability of the feature map. GCNet integrates global information into each position feature by calculating the weighted global features of the feature map. It can be calculated as follows: Global context feature calculation: Apply a global context encoding module to the feature map X, and perform weighted fusion of each position feature in the feature map with the global feature to obtain the global context feature map Y:

[0038] Y=X+W v (Softmax(W k X)W q X)

[0039] Where W V ,W K ,W Q It is a linear transformation matrix, and Softmax is used to calculate the weight of global features.

[0040] Then combine the global feature vector g output by global average pooling (GAP) or GCNet with the channel attention module. The expression of channel attention can be enhanced by element-wise multiplication of g with the input feature map X or other forms of fusion (such as weighted addition). Assuming that Mc is the weight of channel attention, the channel attention output after combining global information is:

[0041] X' C =M c ×X+λ×g

[0042] Among them, λ is a learnable parameter that controls the influence of global information.

[0043] Finally, the global context feature map Y is combined with the spatial attention module. Y can be combined with each position feature of the input feature map to form a globally perceived spatial attention weight.

[0044] X' s =M s ×X+μ×Y

[0045] Among them, μ is another learnable parameter that controls the influence of global information in spatial attention.

[0046] Experimental results show that on a self-made sampling data set, the mAP50 value of the RCC-YOLOv7 algorithm of the present invention reaches 77.1%, which is 3.2% higher than that of the traditional YOLOv7 algorithm, showing higher detection accuracy and effect.

[0047] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Any simple modification or equivalent change made to the above embodiment based on the technical essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A lightning target detection method based on deep learning, characterized in that: A target detection model is constructed based on the YOLOv7 network model. The backbone extraction structure of the target detection model includes several RepNCSPELAN4 feature extraction modules arranged in sequence from front to back, and the CBS convolution in the feature enhancement structure of the target detection model is replaced by a CBAM attention module; the 1x1 convolution layer in the RepNCSPELAN4 feature extraction module is replaced by CoordConv to make the convolution operation position sensitive, and CoordConv adds position information as an additional channel in the input feature map; then, the lightning picture is input into the target detection model for detection.

2. The method for lightning target detection based on deep learning according to claim 1, characterized in that: The CoordConv adds two dimensions of coordinate information corresponding to the X-axis and Y-axis on the input Feature Map; CoordConv adds i and j channels to the input channels, and supports learning functions with translation dependence when the weights of the coordinate layer and the convolution layer of CoordConv are non-zero.

3. The method for lightning target detection based on deep learning according to claim 1, characterized in that: The CBAM attention module consists of a channel attention module and a spatial attention module; the input feature map first passes through the channel attention module and outputs the channel attention weight, and the channel attention weight is multiplied by the input feature map and then sent to the spatial attention module and outputs the spatial attention weight, and the spatial attention weight is multiplied by the input feature map of the spatial attention mechanism to obtain the final feature map.

4. The method for lightning target detection based on deep learning according to claim 3, characterized in that: The global context-aware modules are introduced into the channel attention module and the spatial attention module of the CBAM attention module respectively; the feature map X is input, and the global features are first generated by the global context-aware module, and then these global features are combined when calculating the channel attention and spatial attention respectively.

5. The method for lightning target detection based on deep learning according to claim 4, characterized in that: The processing of the CBAM attention module includes the following steps: Step S1: The global context information of the feature map is captured through the global context awareness module to enhance the global representation ability of the feature map; each position feature in the feature map X is weightedly fused with the global feature to obtain the global context feature map Y: Y=X+W v (Softmax(W k X)W q X) Where: W V ,W K ,W Q is the linear transformation matrix; Softmax is used to calculate the weight of global features; Step S2: Combine the global feature vector g output by global average pooling with the channel attention module; assuming Mc is the weight of the channel attention, the channel attention weight after combining the global information is: X’ C =M c ×X+λ×g Among them, λ is a learnable parameter that controls the influence of global information; Step S3: Finally, the global context feature map Y is combined with the spatial attention module; Y is combined with each position feature of the input feature map to form a globally perceived spatial attention weight: X’ s =M s ×X+μ×Y Among them, μ is another learnable parameter that controls the influence of global information in spatial attention.

6. The method for lightning target detection based on deep learning according to claim 5, characterized in that: The global context perception module is a global context encoding GCNet, which integrates global information into each position feature by calculating the weighted global features of the feature map.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method described in any one of claims 1 to 6 is implemented.

8. An electronic device, characterized in that: The invention comprises a memory and a processor; the memory stores a computer program; the processor is used to execute the computer program in the memory to implement the method according to any one of claims 1 to 6.

Images