Improved YOLOv4-Tiny implementation engineering machinery and material identification method

Abstract

The application relates to the field of artificial intelligence and discloses an improved YOLOv4-Tiny implementation engineering machinery and material recognition method, which comprises the following steps: collecting engineering machinery and material pictures, establishing a data set sample, and using the data set sample to train an improved YOLOv4-Tiny model; an improved YOLOv4-Tiny model is established, an SPP multi-scale feature fusion module is added at the end of the backbone feature extraction network of the original YOLOv4-Tiny network to realize expansion of a receptive field, an SE channel attention module is inserted in front of the SPP multi-scale feature fusion module to realize reduction of irrelevant information interference, an FPN feature pyramid fusion module is adjusted, and utilization of shallow layer information is strengthened to realize enhancement of network representation capability; based on the improved YOLOv4-Tiny model, the data set sample is used for model training to realize detection and classification of engineering machinery and material targets. Experimental results show that the algorithm can make the detection and classification of engineering machinery and materials more accurate.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and more particularly to an improved YOLOv4-Tiny method for identifying engineering machinery and materials. Background Technology

[0002] With the development of science and technology and the maturity of artificial intelligence, various industries are moving towards intelligentization. Among them, the intelligent development of construction machinery has attracted widespread attention and become one of the research hotspots. As one of my country's important pillar industries, construction machinery plays a vital role in the development of my country's infrastructure construction. It covers fields such as infrastructure construction and mining. The working environment is relatively harsh, often accompanied by high temperatures, dust, and vibration. The complex working environment poses a serious threat to the safety of workers. Moreover, some tasks of construction machinery are highly repetitive, and the working area is relatively fixed. Therefore, in order to reduce the safety risks posed by construction machinery operations to people and save labor costs, the development of unmanned and intelligent construction machinery has become a trend. This can not only reduce work risks but also significantly improve the working efficiency of loaders, which has important practical significance. However, the current artificial intelligence technology for the detection and classification of construction machinery and materials is easily interfered with by irrelevant information. Summary of the Invention

[0003] The purpose of this invention is to address the aforementioned technical problems by proposing an improved YOLOv4-Tiny method for identifying engineering machinery and materials. This objective can be achieved through the following technical solutions:

[0004] An improved method for identifying engineering machinery and materials using YOLOv4-Tiny, characterized by comprising:

[0005] Images of construction machinery and materials were collected to create a dataset sample, which was used to train the improved YOLOv4-Tiny model.

[0006] The improved YOLOv4-Tiny model is established by adding an SPP multi-scale feature fusion module to the end of the backbone feature extraction network of the original YOLOv4-Tiny network to expand the receptive field. An SE channel attention module is inserted before the SPP multi-scale feature fusion module to reduce interference from irrelevant information. The FPN feature pyramid fusion module is adjusted to enhance the utilization of shallow information and improve the network's representation ability.

[0007] Based on the improved YOLOv4-Tiny model, the model is trained using the dataset samples to achieve the detection and classification of engineering machinery and material targets.

[0008] Furthermore, the improved YOLOv4-Tiny model includes a backbone feature extraction network and a feature fusion part. The backbone feature extraction network is used to extract features from the input image, and the feature fusion part is used to fuse multi-scale features of the feature map. The backbone feature extraction network includes CBL convolutional blocks and CSPBlock residual modules, and the feature fusion part includes SPP multi-scale feature fusion module, SE channel attention module and FPN feature pyramid fusion module.

[0009] Furthermore, the Conv convolutional layer is placed before the BN batch normalization layer, and the Leaky ReLU activation function is placed after the BN batch normalization layer. The Leaky ReLU activation function is used after the BN batch normalization layer. The convolutional layer is used to extract local spatial information from the input features to obtain different response feature maps. The BN batch normalization layer is used to maintain the relatively stable distribution of input data in each layer of the network and accelerate the model learning speed. The Leaky ReLU activation function is used to solve the problem that neurons do not learn after the ReLU function enters the negative interval.

[0010] Furthermore, the CSPBlock residual module divides the input features into two parts. One part of the input features is passed through a residual block composed of 3*3 convolutions, while introducing a large residual edge, and then enters the Route fusion layer to be concatenated with the other part of the input features, so that the improved YOLOv4-Tiny model can learn more features.

[0011] Furthermore, the CSPBlock residual module is followed by a Maxpool max pooling layer, which is used to perform downsampling, remove redundant information, and compress features to simplify network complexity.

[0012] Furthermore, the SE channel attention module includes an Avgpool average pooling layer, two fully connected (FC) layers, and a Sigmoid layer. The Avgpool average pooling layer is used to preserve the features of the overall data and highlight the background information. The two FC layers are used to learn the channel features of the compressed feature map to obtain a feature map with channel attention. The Sigmoid layer is connected after the second FC layer to reduce data divergence during transmission.

[0013] Furthermore, the number of neurons in the first fully connected FC layer is C / r, where C is the number of neurons and r is a scaling factor. In this invention, r = 16 is used for dimensionality reduction, and the second fully connected FC layer is used for dimensionality increase.

[0014] Furthermore, the SPP multi-scale feature fusion module includes a skip connection and three Maxpooling layers with different kernel sizes. The skip connection is used to output a feature map without pooling operations, and the three Maxpooling layers with different kernel sizes output three different pooling results. These results are then fused with the feature map without pooling operations in the Route fusion layer to output a multi-scale feature map. The fused multi-scale feature map uses a 1*1 convolution operation to adjust the number of channels before entering the improved FPN feature pyramid fusion module.

[0015] Furthermore, the improved FPN feature pyramid fusion module adjusts the connection method of the original feature fusion structure and adds multiple fusion channels on the basis of the original network. The feature map output by the SPP multi-scale feature fusion module enters the set channel one and channel two after passing through two convolutional blocks. The feature map obtained by downsampling after passing through the CSPBlock residual module for the third time enters the set channel three and channel four.

[0016] The feature map transmitted through channel 3 is first concatenated with the feature map transmitted through channel 1 after passing through a convolution block with a stride of 2. Then, it is channel fused through the 3*3 convolution block and finally concatenated with the feature map transmitted through channel 2 to output the feature map.

[0017] The 3*3 convolutional block is followed by channel five. The feature map after the first concatenation is also transmitted through channel five. The feature map transmitted through channel five passes through one of the convolutional blocks and undergoes an upsampling operation. Then it is concatenated with the feature map transmitted through channel four to finally output a new feature map.

[0018] Furthermore, this invention uses an industrial camera to photograph the engineering materials on site from different distances, shooting angles, and exposure values, giving the targets different lighting conditions and contrasts to increase the diversity of the dataset. The collected images are labeled using the image annotation software LabelImg to create dataset samples. These dataset samples are divided into a training set, a test set, and a validation set. The training set is used to train the improved YOLOv4-Tiny network model. The validation set is used to adjust the parameters of the improved YOLOv4-Tiny network model. The test set is used to test the accuracy of the improved YOLOv4-Tiny network model.

[0019] Compared with the prior art, the beneficial effects of the present invention are:

[0020] (1) Add an SE channel attention module to the backbone network to reduce interference from irrelevant information and make the network focus on important information;

[0021] (2) An SPP multi-scale feature fusion module was added after the SE module to enhance the fusion of local and global features and expand the receptive field;

[0022] (3) Improve the FPN feature pyramid fusion module, adjust the original feature fusion structure connection method, add multiple fusion channels on the basis of the original network, and make bidirectional connections between low-resolution feature maps with strong semantic information and high-resolution feature maps with weak semantic information from top to bottom and from bottom to top, making full use of feature information at different levels, strengthening the use of shallow information, and enhancing the network's representation ability. Attached Figure Description

[0023] Figure 1 This is an algorithm flowchart of an improved YOLOv4-Tiny method for identifying engineering machinery and materials according to the present invention.

[0024] Figure 2 This is a schematic diagram of the SE channel attention module structure;

[0025] Figure 3 This is a schematic diagram of the SPP multi-scale feature fusion module structure;

[0026] Figure 4 This diagram illustrates the detection results of an improved YOLOv4-Tiny method for identifying engineering machinery and materials in complex environments, based on the present invention. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0028] Example 1

[0029] This invention provides an improved YOLOv4-Tiny method for identifying engineering machinery and materials, comprising:

[0030] Collect images of construction machinery and materials to create a dataset sample, which is then used to train the improved YOLOv4-Tiny model.

[0031] An improved YOLOv4-Tiny model was established. An SPP multi-scale feature fusion module was added to the end of the backbone feature extraction network of the original YOLOv4-Tiny network to expand the receptive field. An SE channel attention module was inserted before the SPP multi-scale feature fusion module to reduce interference from irrelevant information. The FPN feature pyramid fusion module was adjusted to enhance the utilization of shallow information and improve the network's representation ability.

[0032] Based on the improved YOLOv4-Tiny model, the model is trained using dataset samples to achieve the detection and classification of engineering machinery and material targets.

[0033] In a specific embodiment of the present invention, the present invention further uses an industrial camera to photograph the engineering materials on site at different distances, shooting angles, and exposure values, so that the target has different lighting conditions and contrasts, thereby increasing the diversity of the dataset. The collected images are labeled using the image annotation software LabelImg to create dataset samples. In the dataset samples, 70% is used as the training set, which is used to train the improved YOLOv4-Tiny network model; 20% is used as the validation set, which is used to adjust the parameters of the improved YOLOv4-Tiny network model; and 10% is used as the test set, which is used to test whether the improved YOLOv4-Tiny network model is accurate.

[0034] Specifically, such as Figure 1 As shown, the improved YOLOv4-Tiny model includes a backbone feature extraction network and a feature fusion part. The backbone feature extraction network is used to extract features from the input image, and the feature fusion part is used to fuse features at multiple scales. The backbone feature extraction network includes CBL convolutional blocks and CSPBlock residual modules, and the feature fusion part includes SPP multi-scale feature fusion module, SE channel attention module and FPN feature pyramid fusion module.

[0035] Specifically, the CBL convolutional block consists of three parts: a Conv convolutional layer, a BN batch normalization layer, and a Leaky ReLU activation function. The Conv convolutional layer is placed before the BN batch normalization layer, and the Leaky ReLU activation function is placed after the BN batch normalization layer. The convolutional layer is used to extract local spatial information from the input features to obtain different response feature maps. The BN batch normalization layer is used to maintain the relatively stable distribution of input data in each layer of the network, accelerating the model learning speed. The Leaky ReLU activation function is used to solve the problem that neurons do not learn after the ReLU function enters the negative range.

[0036] Specifically, the CSPBlock residual module divides the input features into two parts. One part of the input features is passed through a residual block consisting of 3*3 convolutions, while introducing a large residual edge before entering the Route fusion layer and concatenating it with the other part of the input features, so that the improved YOLOv4-Tiny model can learn more features.

[0037] Specifically, the CSPBlock residual module is followed by a Maxpool layer, which is used for downsampling, removing redundant information, and compressing features to simplify network complexity.

[0038] In a specific embodiment of the present invention, an input image of size 416*416 is fed into the SE channel attention module after passing through two 3*3 convolutional modules with a stride of 2 and three CSPBlock residual modules.

[0039] Specifically, such as Figure 2 As shown, the SE channel attention module includes an Avgpool average pooling layer, two fully connected (FC) layers, and a Sigmoid layer. The Avgpool average pooling layer is used to preserve the features of the overall data and highlight the background information. The two FC fully connected layers are used to learn the channel features of the compressed feature map to obtain a feature map with channel attention. The Sigmoid layer is connected after the second FC fully connected layer to reduce the divergence of data during transmission.

[0040] Specifically, the number of neurons in the first fully connected (FC) layer is C / r, where C is the number of neurons and r is the scaling factor. In this invention, r = 16 is used for dimensionality reduction, and the second FC layer is used for dimensionality increase.

[0041] In a specific embodiment of the present invention, the input feature map of the SE channel attention module has a dimension of W*H*C. First, the feature map is spatially compressed by performing global average pooling in the spatial dimension, resulting in a 1*1*C feature map. Then, two fully connected (FC) layers are used to learn the channel features of the compressed feature map through a process of first reducing and then increasing the dimensionality, resulting in a feature map with channel attention. This feature map is then passed to a Sigmoid layer to obtain a 1*1*C feature map. Finally, the channel attention features are... Figure 1 The *1*C is multiplied channel-by-channel by the original input feature map H*W*C, and the final output is a feature map with channel attention.

[0042] Specifically, such as Figure 3 As shown, the SPP multi-scale feature fusion module includes a skip connection and three Maxpooling layers with different kernel sizes. The skip connection is used to output the feature map without pooling operations, and the three Maxpooling layers with different kernel sizes output three different pooling results. The feature map is fused with the feature map without pooling operations in the Route fusion layer to output a multi-scale feature map. The fused multi-scale feature map is adjusted for the number of channels using a 1*1 convolution operation and then enters the improved FPN feature pyramid fusion module.

[0043] In a specific embodiment of the present invention, the feature map input to the SPP multi-scale feature fusion module passes through a 3*3 convolutional unit (3*3 Conv+BN+LeakyReLu) and then enters four parallel branch structures. The first branch is a skip connection, and the other three branches are Maxpooling layers with kernel sizes of 5*5, 9*9, and 13*13, respectively. Finally, the feature maps of the four branches are concatenated along the channel dimension to obtain a feature map of size 13*13*2048. After adjusting the number of channels through a 1*1 convolution, it is transmitted to the improved FPN feature pyramid fusion module.

[0044] Specifically, the improved FPN feature pyramid fusion module adjusts the connection method of the original feature fusion structure and adds multiple fusion channels on the basis of the original network. Among them, the feature map output by the SPP multi-scale feature fusion module enters the set channel one and channel two after passing through two convolutional blocks. The feature map obtained by downsampling after passing through the CSPBlock residual module for the third time enters the set channel three and channel four.

[0045] The feature map transmitted through channel 3 is first concatenated with the feature map transmitted through channel 1 after passing through a convolution block with a stride of 2. Then, it is fused through a 3*3 convolution block and finally concatenated with the feature map transmitted through channel 2 to output the feature map.

[0046] The 3*3 convolutional block is followed by channel five. The feature map after the first concatenation is also transmitted through channel five. The feature map transmitted through channel five goes through a convolutional block and is upsampled before being concatenated with the feature map transmitted through channel four, and finally a new feature map is output.

[0047] The working principle or process of the improved YOLOv4-Tiny method for identifying engineering machinery and materials provided by this invention is as follows:

[0048] The input image, with a size of 416*416, first passes through two 3*3 convolutional modules with a stride of 2, followed by three CSPBlock residual modules. The CSPBlock residual modules divide the input features into two parts. One part of the input features passes through a residual block composed of 3*3 convolutions, introducing a large residual edge before entering the Route fusion layer and concatenating with the other part of the input features. The concatenated feature map is then downsampled using max pooling and fed into the SE channel attention module for compression and activation. Subsequently, the SPP multi-scale feature fusion module performs multi-scale feature fusion. The fused feature map uses a 1*1 convolution operation to adjust the number of channels before entering the FPN feature pyramid fusion module. This module performs bidirectional connections between low-resolution, semantically rich feature maps and high-resolution, semantically weak feature maps, both top-down and bottom-up, fully utilizing feature information from different levels to enhance the network's representational ability. Finally, feature maps of sizes 13*13 and 26*26 are output for object detection.

[0049] like Figure 4 The figure shows the results of the detection of engineering machinery and materials in a complex environment using the improved YOLOv4-Tiny method of the present invention. As can be seen from the figure, the present invention can achieve a high accuracy rate for each type of detection in a complex environment.

[0050] The parts of this invention not described in detail are well-known to those skilled in the art. Those skilled in the art should recognize that the above embodiments are merely illustrative and not intended to limit the invention. Any variations or modifications to the above embodiments within the spirit and essence of this invention will fall within the scope of the claims.

Claims

1. An improved method for identifying engineering machinery and materials using YOLOv4-Tiny, characterized in that, include: Images of construction machinery and materials were collected to create a dataset sample, which was used to train the improved YOLOv4-Tiny model. An improved YOLOv4-Tiny model is established by adding an SPP multi-scale feature fusion module to the end of the backbone feature extraction network of the original YOLOv4-Tiny network to expand the receptive field. An SE channel attention module is inserted before the SPP multi-scale feature fusion module to reduce interference from irrelevant information. The FPN feature pyramid fusion module is adjusted to enhance the utilization of shallow information and improve the network's representation ability. The improved FPN feature pyramid fusion module adjusts the connection method of the original feature fusion structure and adds multiple fusion channels to the original network. The feature map output by the SPP multi-scale feature fusion module enters the first and second channels after passing through two convolutional blocks. The feature map obtained by downsampling after passing through the CSPBlock residual module for the third time enters the third and fourth channels. The feature map transmitted through channel 3 is first concatenated with the feature map transmitted through channel 1 after passing through a convolution block with a stride of 2. Then, it is fused through a 3*3 convolution block and finally concatenated with the feature map transmitted through channel 2 to output the feature map. The 3*3 convolutional block is followed by channel five. The feature map after the first stitching is also transmitted through channel five. The feature map transmitted through channel five passes through one of the convolutional blocks and is upsampled to increase its size. Then it is stitched with the feature map transmitted through channel four to finally output a new feature map. Based on the improved YOLOv4-Tiny model, the model is trained using the dataset samples to achieve the detection and classification of engineering machinery and material targets.

2. The improved YOLOv4-Tiny method for identifying engineering machinery and materials according to claim 1, characterized in that: The improved YOLOv4-Tiny model includes a backbone feature extraction network and a feature fusion part. The backbone feature extraction network is used to extract features from the input image, and the feature fusion part is used to fuse multi-scale features of the feature map. The backbone feature extraction network includes CBL convolutional blocks and CSPBlock residual modules, and the feature fusion part includes SPP multi-scale feature fusion module, SE channel attention module and FPN feature pyramid fusion module.

3. The improved YOLOv4-Tiny method for identifying engineering machinery and materials according to claim 2, characterized in that: The CBL convolutional block comprises three parts: a Conv convolutional layer, a BN batch normalization layer, and a Leaky ReLU activation function. The Conv convolutional layer is placed before the BN batch normalization layer, and the Leaky ReLU activation function is placed after the BN batch normalization layer. The Conv convolutional layer is used to extract local spatial information from the input features to obtain different response feature maps. The BN batch normalization layer is used to maintain the relatively stable distribution of input data in each layer of the network, accelerating the model learning speed. The Leaky ReLU activation function is used to solve the problem that neurons do not learn after the ReLU function enters the negative interval.

4. The improved YOLOv4-Tiny method for identifying engineering machinery and materials according to claim 2, characterized in that: The CSPBlock residual module divides the input features into two parts. One part of the input features is passed through a residual block composed of 3*3 convolutions, while introducing a large residual edge, and then enters the Route fusion layer to be concatenated with the other part of the input features, so that the improved YOLOv4-Tiny model can learn more features.

5. The improved YOLOv4-Tiny method for identifying engineering machinery and materials according to claim 4, characterized in that: The CSPBlock residual module is followed by a Maxpool max pooling layer, which is used to perform downsampling, remove redundant information, and compress features to simplify network complexity.

6. The improved YOLOv4-Tiny method for identifying engineering machinery and materials according to claim 2, characterized in that: The SE channel attention module includes an Avgpool average pooling layer, two fully connected (FC) layers, and a Sigmoid layer. The Avgpool average pooling layer is used to preserve the features of the overall data and highlight the background information. The two FC layers are used to learn the channel features of the compressed feature map to obtain a feature map with channel attention. The Sigmoid layer is connected after the second FC layer to reduce data divergence during transmission.

7. The improved YOLOv4-Tiny method for identifying engineering machinery and materials according to claim 6, characterized in that: The first fully connected FC layer has C / r neurons, where C is the number of neurons and r is a scaling factor. In this invention, r=16 is used for dimensionality reduction. The second fully connected FC layer is used for dimensionality increase.

8. The improved YOLOv4-Tiny method for identifying engineering machinery and materials according to claim 5, characterized in that: The SPP multi-scale feature fusion module includes a skip connection and three Maxpooling layers with different kernel sizes. The skip connection outputs a feature map without pooling, and the three Maxpooling layers with different kernel sizes output three different pooled results. These results are then fused with the feature map without pooling in the Route fusion layer to output a multi-scale feature map. The fused multi-scale feature map uses a 1*1 convolution operation to adjust the number of channels before entering the improved FPN feature pyramid fusion module.

9. The improved YOLOv4-Tiny method for identifying engineering machinery and materials according to claim 1, characterized in that: The dataset samples are divided into a training set, a validation set, and a test set. The training set is used to train the improved YOLOv4-Tiny network model, the validation set is used to adjust the parameters of the improved YOLOv4-Tiny network model, and the test set is used to test whether the improved YOLOv4-Tiny network model is accurate.

Images