Traffic safety facility identification method based on attention residual fusion model

Abstract

The invention discloses a traffic safety facility identification method based on an attention residual fusion model. The method comprises the following steps: constructing an operation road traffic safety facility image data set; constructing a context fusion convolution module CGFC to carry out traffic safety facility global context feature extraction; constructing a linear focusing attention residual module FLAR to focus key features of traffic safety facilities; and constructing a traffic safety facility recognition model TSRR-ARFCNN based on attention residual fusion to perform traffic safety facility recognition. According to the method, the context fusion convolution module, the linear focusing attention residual module and the attention residual fusion model are constructed, so that identification of the traffic safety facilities can be realized more accurately, and technical support can be provided for traffic safety facility perception in an operation road scene.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent transportation research, and specifically relates to a traffic safety facility recognition method based on an attention residual fusion model. Background Art

[0002] With the development of artificial intelligence technologies such as computer vision technology, machine learning theory, deep learning theory and pattern recognition technology, new requirements have been put forward for the perception of traffic safety facilities in operating highway scenarios. The recognition of single-category traffic safety facilities such as traffic guardrails, traffic signs and traffic markings has been continuously developed. However, the current research on the field of recognition of all categories of traffic safety facilities in operating highway scenarios is basically in a blank stage.

[0003] In the study of traffic safety facility identification, traditional machine learning methods rely on feature engineering, have low computational efficiency, and are difficult to apply to actual intelligent transportation systems. Deep learning reduces computational complexity by adaptively extracting features, and has stronger generalization capabilities. The attention residual fusion model can greatly improve the efficiency of all-category identification of traffic safety facilities in operating highway scenarios. Therefore, the present invention proposes a traffic safety facility identification method based on the attention residual fusion model. Summary of the invention

[0004] Purpose of the invention: In order to overcome the deficiencies in the prior art, a traffic safety facility identification method based on an attention residual fusion model is provided to identify all categories of traffic safety facilities in operating highway scenarios, thereby providing technical support for the perception of traffic safety facilities in operating highway scenarios.

[0005] Technical solution: To achieve the above purpose, the present invention provides a traffic safety facility recognition method based on an attention residual fusion model, comprising the following steps:

[0006] S1: Construct an image dataset of operational highway traffic safety facilities;

[0007] S2: Construct a context fusion convolution module CGFC to extract global context features of traffic safety facilities;

[0008] S3: Construct a linear focused attention residual module FLAR to focus on the key features of traffic safety facilities;

[0009] S4: Construct a traffic safety facility recognition model TSRR-ARFCNN based on attention residual fusion to identify traffic safety facilities.

[0010] Furthermore, the specific method of constructing the operational highway traffic safety facility image dataset in step S1 is as follows:

[0011] S1-1: 3,000 images of traffic safety facilities on operating highways were collected, including three categories: traffic guardrails, traffic signs, and traffic signs. There were 10 types of traffic safety facilities, including rigid guardrails, semi-rigid guardrails, guide signs, warning signs, prohibition signs, straight markings, left deviation markings, left turn or straight markings, road entrance and exit markings, and transverse speed reduction markings. A data set of traffic safety facilities images for operating highways was constructed. The number of label instances in each category was as follows: rigid guardrails: 3,417, semi-rigid guardrails: 1,722, guide signs: 1,250, warning signs: 472, prohibition signs: 417, straight markings: 250, left deviation markings: 222, left turn or straight markings: 83, road entrance and exit markings: 306, transverse speed reduction markings: 166, and the total number of label instances was 8,305;

[0012] S1-2: Before training each model, randomly extract image data and divide them into training set and validation set in a ratio of 4:1.

[0013] Furthermore, in step S2, the context fusion convolution module CGFC is constructed to extract the global context features of traffic safety facilities. The specific steps are as follows:

[0014] S2-1: Local feature extraction of traffic safety facilities images loc (*), extracting local features of traffic safety facility images through a 3*3 depthwise separable convolution;

[0015] S2-2: Context extraction around traffic safety facility images sur (*), a 3*3 depthwise separable dilated convolution is used to further expand the receptive field and extract the surrounding context information of the traffic safety facility image;

[0016] S2-3: Joint feature extraction of traffic safety facility images joi (*), the local features of the traffic safety facilities image f loc (*) and the surrounding context information of the traffic safety facility image f sur (*)To conduct a joint operation;

[0017] S2-4: Global context extraction of traffic safety facility images glo (*), aggregate the global feature information of the traffic safety facility feature map, and then further enhance the joint features to extract the global context features of the traffic safety facility image.

[0018] Furthermore, in step S2-3, the traffic safety facility image joint feature extraction f joi (*) The specific content is:

[0019] ①Through the connection operation, the local features of the traffic safety facilities image f loc(*) and the surrounding context information of the traffic safety facility image f sur (*) Combined to form a comprehensive feature of the traffic safety facility image;

[0020] ② The comprehensive features of the traffic safety facility image are further integrated through batch normalization parameterized corrected linear unit to obtain the joint features of the traffic safety facility image f joi (*).

[0021] Furthermore, in step S2-4, the global context extraction of the traffic safety facility image is performed. glo (*) The specific content is:

[0022] ① Global information aggregation of feature maps is performed through global average pooling, each channel is responded to equally, and a global feature vector is obtained;

[0023] ② The complex nonlinear relationship between features is integrated through a multi-layer perceptron, thereby further refining the global feature vector. The multi-layer perceptron consists of two fully connected layers;

[0024] ③Using global context information as weight to weight the joint features of traffic safety facilities images joi (*) Recalibrate to highlight important features, suppress unimportant features, and output global information of traffic safety facility images.

[0025] Furthermore, in step S3, the specific method of constructing the linear focus attention residual module FLAR to focus on the key features of traffic safety facilities is:

[0026] S3-1: Multiply the number of channels of the input feature map through a convolution of size 1*1 and divide it into two parts to output f a , f b ;

[0027] S3-2: Perform residual connections on multiple convolutions to form a bottleneck module to enhance the deeper features of the feature map f c ;

[0028] S3-3: Generate query (Q), key (K), and value (V) through the linear layer;

[0029] S3-4: Adjust the direction of the query-key pair through the mapping function, so that similar query-key pairs are close together and dissimilar query-key pairs are far away, thereby enhancing the ability to focus on the characteristics of traffic safety facilities;

[0030] S3-5: Maintaining traffic safety facility feature diversity through depthwise separable convolution;

[0031] S3-6: Adjust the attention score by scaling the parameter, and then obtain the attention weight by normalizing the exponential function;

[0032] S3-7: Use the attention weight to perform weighted summation on the value (V) and output feature f d ;

[0033] S3-8: Put f b 、f c 、f d After concatenation, the number of channels of a convolutional feature map with a size of 1*1 is compressed to the target number of channels to output a more focused traffic safety facility feature map.

[0034] Furthermore, the step S3-6 adjusts the attention score by scaling the parameter, and then obtains the attention weight by normalizing the exponential function. Specifically,

[0035] ①Introduce a learnable scaling parameter scale, and use the Softplus function to map the scaling parameter to the positive range:

[0036] Softplus(x)=log(1+e x )

[0037] Among them, e x represents the exponential function;

[0038] ② Use the processed scaling parameters to adjust the attention score, then:

[0039]

[0040] Among them, Q represents query, K represents key, and scale represents scaling parameter;

[0041] ③Use the normalized exponential function to get the attention weight, then let:

[0042]

[0043] Furthermore, the specific method of constructing the traffic safety facility recognition model TSRR-ARFCNN based on attention residual fusion in step S4 is: on the basis of the Yolov8 model, three context-fused convolution modules CGFC and three linear focused attention residual modules FLAR are combined to finally obtain the TSRR-ARFCNN traffic safety facility recognition model.

[0044] Beneficial effects: Compared with the existing technology, the present invention realizes automatic recognition of all categories of traffic safety facilities in operating highway scenarios based on deep learning technology, improves recognition efficiency, and can provide technical support for the perception of traffic safety facilities in operating highway scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1Schematic diagram of the CGFC module structure.

[0046] Figure 2 This is a schematic diagram of the FLAR module structure.

[0047] Figure 3 Schematic diagram of the TSRR-ARFCNN model structure. DETAILED DESCRIPTION

[0048] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments.

[0049] The present invention provides a method for identifying traffic safety facilities based on an attention residual fusion model, comprising the following steps:

[0050] S1: Construct an image dataset of operational highway traffic safety facilities, which includes:

[0051] S1-1: 3000 images of traffic safety facilities on operating highways were collected, including three categories: traffic guardrails, traffic signs, and traffic signs. They were divided into 10 categories of traffic safety facilities: rigid guardrails, semi-rigid guardrails, guide signs, warning signs, prohibition signs, straight markings, left deviation markings, left turn or straight markings, road entrance and exit markings, and lateral speed reduction markings. A data set of traffic safety facilities images on operating highways was constructed. The number of label instances in each category is as follows: rigid guardrails: 3417, semi-rigid guardrails: 1722, guide signs: 1250, warning signs: 472, prohibition signs: 417, straight markings: 250, left deviation markings: 222, left turn or straight markings: 83, road entrance and exit markings: 306, lateral speed reduction markings: 166, with a total of 8305 label instances. The composition of the data set is shown in Table 1.

[0052] Table 1 Composition of the operational highway traffic safety facility image dataset

[0053]

[0054] S1-2: Before training each model, randomly extract image data and divide them into training set and validation set in a ratio of 4:1.

[0055] S2: Construct a context fusion convolution module CGFC to extract global context features of traffic safety facilities. Specifically:

[0056] S2-1: Local feature extraction of traffic safety facilities images loc (*), extracting local features of traffic safety facility images through a 3*3 depthwise separable convolution;

[0057] S2-2: Context extraction around traffic safety facility images sur(*), a 3*3 depthwise separable dilated convolution is used to further expand the receptive field and extract the surrounding context information of the traffic safety facility image;

[0058] S2-3: Joint feature extraction of traffic safety facility images joi (*), the local features of the traffic safety facilities image f loc (*) and the surrounding context information of the traffic safety facility image f sur (*)Join:

[0059] ①Through the connection operation, the local features of the traffic safety facilities image f loc (*) and the surrounding context information of the traffic safety facility image f sur (*) Combined to form a comprehensive feature of the traffic safety facility image;

[0060] ② The comprehensive features of the traffic safety facility image are further integrated through batch normalization parameterized corrected linear unit to obtain the joint features of the traffic safety facility image f joi (*).

[0061] S2-4: Global context extraction of traffic safety facility images glo (*) Aggregate the global feature information of the traffic safety facility feature map, and then further enhance the joint features to extract the global context features of the traffic safety facility image:

[0062] ① Global information aggregation of feature maps is performed through global average pooling, each channel is responded to equally, and a global feature vector is obtained;

[0063] ② The complex nonlinear relationship between features is integrated through a multi-layer perceptron, thereby further refining the global feature vector. The multi-layer perceptron consists of two fully connected layers;

[0064] ③Using global context information as weight to weight the joint features of traffic safety facilities images joi (*) Recalibrate to highlight important features, suppress unimportant features, and output global information of traffic safety facility images.

[0065] S3: Construct a linear focused attention residual module FLAR to focus on the key features of traffic safety facilities, which are:

[0066] S3-1: Multiply the number of channels of the input feature map through a convolution of size 1*1 and divide it into two parts to output f a , f b ;

[0067] S3-2: Perform residual connections on multiple convolutions to form a bottleneck module to enhance the deeper features of the feature map f c ;

[0068] S3-3: Generate query (Q), key (K), and value (V) through the linear layer;

[0069] S3-4: Adjust the direction of the query-key pair through the mapping function, so that similar query-key pairs are close together and dissimilar query-key pairs are far away, thereby enhancing the ability to focus on the characteristics of traffic safety facilities;

[0070] S3-5: Maintaining traffic safety facility feature diversity through depthwise separable convolution;

[0071] S3-6: Adjust the attention score by scaling the parameter, and then obtain the attention weight by normalizing the exponential function:

[0072] ①Introduce a learnable scaling parameter scale, and use the Softplus function to map the scaling parameter to the positive range:

[0073] Softplus(x)=log(1+e x )

[0074] ② Use the processed scaling parameters to adjust the attention score, then:

[0075]

[0076] ③Use the normalized exponential function to get the attention weight, then let:

[0077]

[0078] S3-7: Use the attention weight to perform weighted summation on the value (V) and output feature f d ;

[0079] S3-8: Put f b 、f c 、f d After concatenation, the number of channels of a convolutional feature map with a size of 1*1 is compressed to the target number of channels to output a more focused traffic safety facility feature map.

[0080] S4: Based on the Yolov8 model, three context-fused convolution modules CGFC and three linear focused attention residual modules FLAR are combined to finally obtain the TSRR-ARFCNN traffic safety facility recognition model.

[0081] In order to verify the superiority of the attention residual fusion model (TSRR-ARFCNN), a comparative experiment was conducted in this case. Based on 3,000 images of traffic safety facilities on operational highways, the training set and the validation set were divided into 4:1, and the TSRR-Yolov8 and TSRR-ARFCNN traffic safety facility recognition models were trained respectively. The results are shown in Table 2. By comparing and analyzing the accuracy and recall rate of different models, it can be seen that the accuracy and recall rate of the attention residual fusion model are significantly higher than those of the basic model.

[0082] Table 2 Model performance comparison

[0083]

[0084] The technical means disclosed in the scheme of the present invention are not limited to the technical means disclosed in the above-mentioned implementation mode, but also include technical schemes composed of any combination of the above-mentioned technical features. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also regarded as the protection scope of the present invention.

Claims

1. A traffic safety facility recognition method based on an attention residual fusion model, characterized by: The steps include: S1: Construct an image dataset of operational highway traffic safety facilities; S2: Construct a context fusion convolution module CGFC to extract global context features of traffic safety facilities; S3: Construct a linear focused attention residual module FLAR to focus on the key features of traffic safety facilities; S4: Construct a traffic safety facility recognition model TSRR-ARFCNN based on attention residual fusion to identify traffic safety facilities.

2. The method for identifying traffic safety facilities based on the attention residual fusion model according to claim 1, characterized in that: The specific method for constructing the operational highway traffic safety facility image dataset in step S1 is as follows: S1-1: Collect 3,000 images of traffic safety facilities on operating highways, including three categories: traffic guardrails, traffic signs, and traffic signs. There are 10 types of traffic safety facilities, including rigid guardrails, semi-rigid guardrails, guide signs, warning signs, prohibition signs, straight markings, left deviation markings, left turn or straight markings, road entrance and exit markings, and lateral speed reduction markings, to build an image dataset of traffic safety facilities on operating highways; S1-2: Before training each model, randomly extract image data and divide them into training set and validation set in a ratio of 4:

1.

3. The method for identifying traffic safety facilities based on the attention residual fusion model according to claim 1, characterized in that: In step S2, the context fusion convolution module CGFC is constructed to extract the global context features of traffic safety facilities. The specific steps are as follows: S2-1: Local feature extraction of traffic safety facilities images loc (*), extracting local features of traffic safety facility images through a 3*3 depthwise separable convolution; S2-2: Context extraction around traffic safety facility images sur (*), a 3*3 depthwise separable dilated convolution is used to further expand the receptive field and extract the surrounding context information of the traffic safety facility image; S2-3: Joint feature extraction of traffic safety facility images joi (*), the local features of the traffic safety facilities image f loc (*) and the surrounding context information of the traffic safety facility image f sur (*)To conduct a joint operation; S2-4: Global context extraction of traffic safety facility images glo (*), aggregate the global feature information of the traffic safety facility feature map, and then further enhance the joint features to extract the global context features of the traffic safety facility image.

4. The method for identifying traffic safety facilities based on the attention residual fusion model according to claim 3 is characterized in that: In step S2-3, the traffic safety facility image joint feature extraction f joi (*) The specific contents are: ①Through the connection operation, the local features of the traffic safety facilities image f loc (*) and the surrounding context information of the traffic safety facility image f sur (*) Combined to form a comprehensive feature of the traffic safety facility image; ② The comprehensive features of the traffic safety facility image are further processed by batch normalization parameterized corrected linear unit to obtain the joint features of the traffic safety facility image f joi (*).

5. The method for identifying traffic safety facilities based on the attention residual fusion model according to claim 3 is characterized in that: The global context extraction of the traffic safety facility image in step S2-4 glo (*) The specific contents are: ① Global information aggregation of feature maps is performed through global average pooling, each channel is responded to equally, and a global feature vector is obtained; ② The complex nonlinear relationship between features is integrated through a multi-layer perceptron, so that the global feature vector is more refined. The multi-layer perceptron consists of two fully connected layers; ③Using global context information as weight to weight the joint features of traffic safety facilities images joi (*) Recalibrate to highlight important features, suppress unimportant features, and output global information of traffic safety facility images.

6. The method for identifying traffic safety facilities based on the attention residual fusion model according to claim 1, characterized in that: The specific method of constructing the linear focused attention residual module FLAR in step S3 to focus on the key features of traffic safety facilities is: S3-1: Multiply the number of channels of the input feature map through a convolution of size 1*1 and divide it into two parts to output f a , f b ; S3-2: Perform residual connections on multiple convolutions to form a bottleneck module to enhance the deeper features of the feature map f c ; S3-3: Generate query Q, key K and value V through linear layer; S3-4: Adjust the direction of the query-key pair through the mapping function, so that similar query-key pairs are close together and dissimilar query-key pairs are far away, thereby enhancing the ability to focus on the characteristics of traffic safety facilities; S3-5: Maintaining traffic safety facility feature diversity through depthwise separable convolution; S3-6: Adjust the attention score by scaling the parameter, and then obtain the attention weight by normalizing the exponential function; S3-7: Use the attention weight to perform weighted summation on the value V and output feature f d ; S3-8: Put f b 、f c 、f d After concatenation, the number of channels of a convolutional feature map with a size of 1*1 is compressed to the target number of channels to output a more focused traffic safety facility feature map.

7. The method for identifying traffic safety facilities based on the attention residual fusion model according to claim 6 is characterized in that: The step S3-6 adjusts the attention score by scaling the parameter, and then obtains the attention weight by normalizing the exponential function. The specific content is: ①Introduce a learnable scaling parameter scale, and use the Softplus function to map the scaling parameter to the positive range: Softplus(x)=log(1+e x ) Among them, e x represents the exponential function; ② Use the processed scaling parameters to adjust the attention score, then: Among them, Q represents query, K represents key, and scale represents scaling parameter; ③Use the normalized exponential function to get the attention weight, then let:

8. The method for identifying traffic safety facilities based on the attention residual fusion model according to claim 1, characterized in that: The specific method of constructing the traffic safety facility recognition model TSRR-ARFCNN based on attention residual fusion in step S4 is: on the basis of the Yolov8 model, three context-fused convolution modules CGFC and three linear focused attention residual modules FLAR are combined to finally obtain the TSRR-ARFCNN traffic safety facility recognition model.

Images