Traffic sign classification method based on deep convolution fuzzy system

Abstract

The invention discloses a traffic sign classification method based on a deep convolution fuzzy system. The method comprises the following steps: firstly, extracting features of image data through convolution and pooling operations; then, one-hot coding is utilized, the multi-classification problem is converted into a plurality of dichotomy problems to be solved, and each dichotomy problem is solved through DFSSR; and finally, integrating the outputs of all the DFSSR systems, and after softmax function processing, obtaining the output of the DCFSSR system. Compared with other algorithms, the DCFSSR has the advantage that the accuracy is higher. Compared with a ResNet18 model, the DCFSSR completes classification decision through rules, not only has better interpretability, but also is almost the same as the ResNet18 model in prediction precision, and has fewer parameters at the same time; therefore, the DCFSSR has more advantages in traffic sign classification and is suitable for further popularization and application.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic sign recognition and classification, and in particular to a traffic sign classification method based on a deep convolution fuzzy system. Background Art

[0002] The introduction and development of convolutional neural networks (CNNs) have also opened up new avenues for traffic sign recognition research, further promoting the development of traffic sign recognition and classification research. Mehta et al. proposed a traffic sign classification method based on a deep convolutional network, which uses Adam (adaptive moment estimation) as an optimizer and softmax as the activation in the output layer. Through testing on the Belgian Traffic Sign Dataset (BTSD), it was found that this traffic sign classification method is more competitive than previous algorithms. A novel CNN architecture has been proposed in the prior art to improve the performance of traffic sign detection. This method uses the Adam optimizer, sets the batch size to 128, and uses multiple interconnected layers, so that the method can complete the detection even in the face of traffic signs in extreme weather conditions and blurred image conditions.

[0003] Although deep convolutional neural networks have good learning capabilities and high prediction accuracy, they still have problems such as complex models, many hyperparameters, and black boxes. As the scale of data continues to expand, the connection relationship between the neural nodes of deep neural networks becomes more and more complex, which eventually makes the entire model difficult to understand. How to improve deep learning algorithms and improve the interpretability of traffic sign classification methods has become a scientific problem that needs to be solved urgently. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a traffic sign classification method based on a deep convolutional fuzzy system.

[0005] In order to achieve the above technical objectives, the technical solution adopted by the present invention is:

[0006] A traffic sign classification method based on a deep convolutional fuzzy system, comprising:

[0007] 1) Extract features of traffic sign image data through convolution and pooling operations;

[0008] 2) Use PCA to reduce the dimension of feature data extracted from traffic sign images;

[0009] 3) Using one-hot encoding, the multi-classification problem is converted into multiple binary classification problems for solution. Each binary classification problem is solved by DFSSR (Deep Fuzzy System with Simplified Rule base);

[0010] 4) Integrate the outputs of all DFSSR systems and process them through the softmax function to obtain the DCFSSR (Deep Convolutional Fuzzy System with Simplified Rule base) system output.

[0011] As a possible implementation, further, in step 1), ResNet18 is used as a feature extractor to convert the traffic sign image data into feature data.

[0012] As a possible implementation, further, the ResNet18 includes:

[0013] Input layer: used to receive image data;

[0014] Convolutional layer: The input image is processed using a 7×7 convolution kernel with a stride of 2, and then passed through a maximum pooling layer with a stride of 2 to reduce the spatial dimension of the feature map while increasing the abstraction level of the features.

[0015] Residual block: Each residual block contains two 3×3 convolutional layers, including activation functions and batch normalization operations. The input of each block is not only passed to the next convolutional layer, but also directly added to the block output through skip connections to promote direct gradient flow;

[0016] Global average pooling layer: After passing through all residual blocks, the global average pooling layer calculates the average value of each feature map channel, converts the multi-dimensional feature map into a one-dimensional feature vector, and provides input for the fully connected layer;

[0017] Fully connected layer: connects the feature vector after global average pooling to the final classification layer;

[0018] Softmax layer: Calculates the final output and outputs the predicted probability distribution of each category.

[0019] As a possible implementation, further, in step 2), feature data with a cumulative contribution of 95% is selected, and PCA is used to reduce the dimension of the data.

[0020] As a possible implementation, the DFSSR further comprises several submodules FSSR (Fuzzy System with Simplified Rule base), which are stacked layer by layer; the input data of FSSR is set to be k-dimensional, and each FSSR module uses at most r rules; each binary classifier DFSSR has a total outer structure of Layer, where It means rounding up, and N is the feature dimension of the data after PCA dimensionality reduction.

[0021] As a possible implementation, further, the FSSR algorithm steps include:

[0022] Divide fuzzy sets according to data and define corresponding membership functions;

[0023] Learn rules from data according to the defined fuzzy sets, with data and rules corresponding one to one. Meanwhile, in the process of rule learning, calculate the contribution of each rule.

[0024] After the rule learning is completed, the rules with the same rule antecedents are merged, and the importance of each merged rule is calculated;

[0025] The rules are sorted according to their importance and the key rules are extracted from them as the rule base of FSSR.

[0026] As a possible implementation, further, the DCFSSR includes C DFSSRs, and the C DFSSRs include FSSR, where C represents the number of classes in the classification problem;

[0027] During the construction of the binary classifier DFSSR, for the same DCFSSR, its binary classifier DFSSR uses the same grouping and blocking scheme, and each DFSSR first level uses the same system input and shares the same rule base.

[0028] As a possible implementation, further, the training method of the binary classifier DFSSR in step 3) includes the following steps:

[0029] S1: Determine k L =N is true; if true, go to the next step, otherwise, from the system input data (x1, x2, ..., x N ) is randomly selected from (k L -N) feature data, reuse the feature data, and fill the data to k L ;

[0030] S2: Randomly group the feature data, and divide every k data features into a group. During the division process, it is necessary to ensure that there are no identical data features in each group, otherwise the groups will be re-divided;

[0031] S3: Input the divided grouped data into each submodule FSSR for learning. The submodules FSSR in the same level do not interfere with each other and learn in parallel;

[0032] S4: Integrate the output of the submodule FSSR and input it to the next layer, which is then continued to learn by the submodule FSSR in the next layer;

[0033] S5: Group the data again according to the input order, and divide every k data features into one group; if the system has k input data at this time, execute S6, otherwise, continue to execute from S3;

[0034] S6: Integrate these k feature data and input them into the sub-module FSSR of the L layer (i.e. the top layer). The final decision is made by the FSSR of the last layer to obtain the system output y' of the entire DFSSR.

[0035] As a possible implementation, further, step 4) specifically includes the following steps:

[0036] Combine the outputs z1,z2,...,z of all DFSSR systems C , where z1,z2,...,z C Indicates the classification results corresponding to DCFSSR Class 1 to Class C;

[0037] After calculation by the following formula (4), the result probability of each binary classifier DFSSR can be obtained;

[0038]

[0039] in, 0<y j <1,j=1,2,...,C。

[0040] By adopting the above technical solution, the present invention has the following beneficial effects compared with the prior art:

[0041] 1) The present invention constructs a deep fuzzy system DFSSR. First, the complex high-dimensional problem is broken down into multiple sub-problems for solution, that is, a divide-and-conquer strategy. Then the FSSR is stacked layer by layer, the sub-problems are integrated, and the overall problem is solved. Compared with other algorithms, the DCFSSR provided by the present invention shows higher accuracy. Compared with the ResNet18 model, the DCFSSR completes classification decisions through rules, which not only has better interpretability, but also has the same prediction accuracy as ResNet18, and has fewer parameters; therefore, DCFSSR has more advantages in traffic sign classification problems.

[0042] 2) After extracting image features, DCFSSR relies on the fuzzy system to carry out the subsequent decision-making process, so that DCFSSR can complete the decision-making of traffic sign classification problems in the form of IF-THEN rules, thereby enhancing the interpretability of the decision-making process of this problem.

[0043] 3) The structure of ResNet18 in the present invention can effectively avoid the gradient vanishing problem, so that the network can be effectively trained even when the depth increases. In addition, the design of the residual block reduces the difficulty of model training, which allows a part of the network layer to focus on learning the residual part of the input data instead of directly learning the entire feature map from beginning to end. This method simplifies the learning process and improves the training efficiency and final performance of the network. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 It is a simplified flow chart of the present invention;

[0046] Figure 2 This is a schematic diagram of the internal structure of FSSR;

[0047] Figure 3 Schematic diagram of DFSSR topology. DETAILED DESCRIPTION

[0048] The present invention will be further described in detail below in conjunction with the accompanying drawings and examples. It is particularly noted that the following examples are only used to illustrate the present invention, but are not intended to limit the scope of the present invention. Similarly, the following examples are only partial embodiments of the present invention rather than all embodiments, and all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0049] Based on the research of deep fuzzy system DFSSR, the present invention combines the achievements of deep convolutional neural network in image feature extraction, integrates one-hot encoding and softmax function, and proposes a deep convolutional fuzzy system algorithm DCFSSR. DCFSSR uses convolution and pooling technology to extract image features, and then reduces the dimension of feature data through PCA technology. Then, the multi-classification problem is decomposed into several binary classification problems using one-hot encoding, and each binary classification problem is solved by the DFSSR algorithm. Subsequently, all the outputs of the binary classifier DFSSR are integrated and processed using the softmax function, and finally the category label predicted by the DCFSSR system is obtained.

[0050] Refer to the attached Figure 1 As shown, this embodiment provides a traffic sign classification method based on a deep convolution fuzzy system, including:

[0051] 1) Extracting features from traffic sign image data through convolution and pooling operations. In this embodiment, ResNet18 is used as a feature extractor to convert traffic sign image data into feature data.

[0052] Among them, ResNet18 includes:

[0053] Input layer: used to receive image data.

[0054] Convolutional layer: The input image is first processed using a 7×7 convolution kernel with a stride of 2, and then passed through a maximum pooling layer with a stride of 2 to reduce the spatial dimension of the feature map while increasing the level of abstraction of the features.

[0055] Residual block: The core part of ResNet. Each residual block contains two 3×3 convolutional layers, including activation functions and batch normalization operations; the input of each block is not only passed to the next convolutional layer, but also directly added to the block output through skip connections to promote direct gradient flow.

[0056] Global average pooling layer: After passing through all residual blocks, the global average pooling layer calculates the average value of each feature map channel, converts the multi-dimensional feature map into a one-dimensional feature vector, and provides input for the fully connected layer.

[0057] Fully connected layer: connects the feature vector after global average pooling to the final classification layer.

[0058] Softmax layer: Calculates the final output and outputs the predicted probability distribution of each category.

[0059] 2) Use PCA (Principal Component Analysis) to reduce the dimension of the feature data extracted from the traffic sign image; specifically, select feature data with a cumulative contribution of 95%, use PCA to reduce the dimension of the data, and provide it to DCFSSR to process multi-classification tasks.

[0060] 3) Using one-hot encoding, the multi-classification problem is converted into multiple binary classification problems for solution. Each binary classification problem is solved by DFSSR (see attached Figure 3 As shown); DFSSR includes several submodules FSSR, and FSSR is stacked layer by layer;

[0061] Refer to the attached Figure 2As shown in the figure, the algorithm steps of FSSR (Fuzzy System with Simplified Rule base) are as follows: First, divide the fuzzy sets according to the data and define the corresponding membership functions; then, learn the rules from the data according to the defined fuzzy sets, one rule for each data. In the process of rule learning, calculate the contribution of each rule; after the rule learning is completed, merge the rules with the same rule antecedents; after the rules are merged, calculate the importance of each merged rule; then, sort the rules according to the importance of the rules, and extract the key rules from them as the rule base of FSSR. Specifically, the FSSR algorithm is shown in Table 1 below:

[0062] Table 1 FSSR algorithm

[0063]

[0064]

[0065] The formula in the above table is as follows:

[0066]

[0067] In the above formula, N represents the characteristic dimension of the data. represents the fuzzy operator index, represents the membership value of the r-th rule in the N-th dimension fuzzy set;

[0068]

[0069] In the above formula, represents the membership value of the rth rule in the Nth dimension fuzzy set, represents the fuzzy operator index;

[0070]

[0071] In the above formula, Represents the n-th dimension of the m-th data, y1.....y m represents the system prediction output, Indicates the number of rules after filtering, Indicates that the Nth dimension of the Mth data is in The trigger strength obtained from the rules, M represents the number of samples, N represents the dimension, It means that the parameter sequence ω is calculated by combining formula (2-21) and using the least squares method.

[0072] Set the FSSR system input to k (the input data of each FSSR is k-dimensional), and each binary classifier DFSSR has a total outer structure of Layer, where It means rounding up, and N is the feature dimension of the data after PCA dimensionality reduction.

[0073] All binary classifiers DFSSR need to be trained. The training method of binary classifier DFSSR includes the following steps:

[0074] S1: Determine k L =N is true; if true, go to the next step, otherwise, from the system input data (x1, x2, ..., x N ) is randomly selected from (k L -N) feature data, reuse the feature data, and fill the data to k L ;

[0075] S2: Randomly group the feature data, and divide every k data features into a group. During the division process, it is necessary to ensure that there are no identical data features in each group, otherwise the groups will be re-divided;

[0076] S3: Input the divided grouped data into each submodule FSSR for learning. The submodules FSSR in the same level do not interfere with each other and learn in parallel;

[0077] S4: Integrate the output of the submodule FSSR and input it to the next layer, which is then continued to learn by the submodule FSSR in the next layer;

[0078] S5: Group the data again according to the input order, and divide every k data features into one group; if the system has k input data at this time, execute S6, otherwise, continue to execute from S3;

[0079] S6: Integrate these k feature data and input them into the sub-module FSSR of the L layer (i.e. the top layer). The final decision is made by the FSSR of the last layer to obtain the system output y' of the entire DFSSR.

[0080] For the sub-modules in the first layer of DFSSR, their system output can be expressed as:

[0081]

[0082] Where w1={z1,z2,...,z k},w2={z k+1 ,z k+2 ,...,z 2k}, Respectively represent the system inputs of the 1st, 2nd, and kth modules in the first layer of DFSSR. k It represents the number of rules learned. r,n ,c r,n ,d r,nIt is the consequent parameter of the rth rule in the Nth dimension obtained after the system training is completed. is the normalized rule trigger strength. Data x=(x1,x2,...,x N ) After feature reuse and random sorting, a new data sequence is obtained Among them, z i ∈{x1,x2,...,x N}, i = 1, 2, ..., k 2 , N is the feature dimension of the input data.

[0083] The DFSSR proposed in the present invention uses random feature reuse to ensure that all FSSR modules have inputs of the same size, thereby reducing the differences between FSSRs. This enables each FSSR module to use a unified parameter configuration, thereby maintaining consistency between the FSSR modules. There are five layers inside the FSSR module and L layers outside the DFSSR, so the entire DFSSR has a total of 5L layers.

[0084] 4) After training, the output data of all DFSSR systems are integrated and processed by the softmax function to obtain the output of the DCFSSR system (where the DCFSSR output is the probability corresponding to various classification results); specifically, the following steps are included:

[0085] Combine the outputs z1,z2,...,z of all DFSSR systems C , where z1,z2,...,z C It represents the classification results corresponding to the 1st to the Cth category of DCFSSR, where C represents the total number of categories of the classification problem;

[0086] After calculation by the following formula (4), the result probability of each binary classifier DFSSR can be obtained;

[0087]

[0088] in, 0<y j <1,j=1,2,...,C。

[0089] In the process of constructing the binary classifier DFSSR, all binary classifiers DFSSR adopt the same deep structure construction method; that is, for the same DCFSSR, its binary classifier DFSSR uses the same grouping and blocking scheme. Set the module FSSR input data to be fixed to k dimensions, and each binary classifier DFSSR has a total outer structure of layer; then, DCFSSR needs to construct C DFSSRs, module FSSR.

[0090] Since each binary classifier DFSSR in DCFSSR uses the same grouping and block mode, and the first level uses the same system input, the first level of each binary classifier DFSSR shares the same rule base; that is, the first level of each DFSSR uses the same rule base, and the difference lies in the difference in the subsequent parameters. Therefore, assuming that the module FSSR uses r rules, the overall construction of DCFSSR requires Rules, total parameters.

[0091] The above descriptions are only some embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any equivalent device or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A traffic sign classification method based on deep convolutional fuzzy system, characterized in that: include: 1) Extract features of traffic sign image data through convolution and pooling operations; 2) Use PCA to reduce the dimension of feature data extracted from traffic sign images; 3) Using one-hot encoding, the multi-classification problem is converted into multiple binary classification problems for solution, and each binary classification problem is solved by DFSSR; 4) Integrate the outputs of all DFSSR systems and obtain the DCFSSR system output after processing with the softmax function.

2. The traffic sign classification method based on deep convolutional fuzzy system according to claim 1 is characterized in that: In step 1), ResNet18 is used as a feature extractor to convert traffic sign image data into feature data.

3. The traffic sign classification method based on deep convolutional fuzzy system according to claim 2 is characterized in that: The ResNet18 includes: Input layer: used to receive image data; Convolutional layer: The input image is processed using a 7×7 convolution kernel with a stride of 2, and then processed through a maximum pooling layer with a stride of 2; Residual block: Each residual block contains two 3×3 convolutional layers, including activation functions and batch normalization operations. The input of each block is not only passed to the next convolutional layer, but also directly added to the block output through skip connections to promote direct gradient flow; Global average pooling layer: After passing through all residual blocks, the global average pooling layer calculates the average value of each feature map channel, converts the multi-dimensional feature map into a one-dimensional feature vector, and provides input for the fully connected layer; Fully connected layer: connects the feature vector after global average pooling to the final classification layer; Softmax layer: Calculates the final output and outputs the predicted probability distribution of each category.

4. The traffic sign classification method based on deep convolutional fuzzy system according to claim 1 is characterized in that: In step 2), feature data with a cumulative contribution of 95% are selected, and PCA is used to reduce the dimension of the data.

5. The traffic sign classification method based on deep convolutional fuzzy system according to claim 1 is characterized in that: The DFSSR contains several submodules FSSR, which are stacked layer by layer. The input data of FSSR is set to be k-dimensional, and each FSSR module uses at most r rules. The outer structure of each binary classifier DFSSR has Layer, where It means rounding up, and N is the feature dimension of the data after PCA dimensionality reduction.

6. The traffic sign classification method based on deep convolutional fuzzy system according to claim 5 is characterized in that: The FSSR algorithm steps include: Divide fuzzy sets according to data and define corresponding membership functions; Learn rules from data according to the defined fuzzy sets, with data and rules corresponding one to one. Meanwhile, in the process of rule learning, calculate the contribution of each rule. After the rule learning is completed, the rules with the same rule antecedents are merged, and the importance of each merged rule is calculated; The rules are sorted according to their importance and the key rules are extracted from them as the rule base of FSSR.

7. The traffic sign classification method based on deep convolutional fuzzy system according to claim 5 is characterized in that: The DCFSSR includes C DFSSRs, and the C DFSSRs include FSSR, where C represents the number of classes in the classification problem; During the construction of the binary classifier DFSSR, for the same DCFSSR, its binary classifier DFSSR uses the same grouping and blocking scheme, and each DFSSR first level uses the same system input and shares the same rule base.

8. The traffic sign classification method based on deep convolutional fuzzy system according to claim 7 is characterized in that: The training method of the binary classifier DFSSR in step 3) includes the following steps: S1: Determine k L =N is true; if true, go to the next step, otherwise, from the system input data (x1, x2, ..., x N ) randomly select (k L -N) feature data, reuse the feature data, and fill the data to k L ; S2: Randomly group the feature data, and divide every k data features into a group. During the division process, it is necessary to ensure that there are no identical data features in each group, otherwise the groups will be re-divided; S3: Input the divided grouped data into each submodule FSSR for learning. The submodules FSSR in the same level do not interfere with each other and learn in parallel; S4: Integrate the output of the submodule FSSR and input it to the next layer, which is then continued to learn by the submodule FSSR in the next layer; S5: Group the data again according to the input order, and divide every k data features into one group; if the system has k input data at this time, execute S6, otherwise, continue to execute from S3; S6: Integrate these k feature data and input them into the L-layer sub-module FSSR. The final decision is made by the last layer FSSR to obtain the system output y' of the entire DFSSR.

9. The traffic sign classification method based on deep convolutional fuzzy system according to claim 7 is characterized in that: Step 4) specifically includes the following steps: Combine the outputs z1,z2,...,z of all DFSSR systems C , where z1,z2,...,z C Indicates the classification results corresponding to DCFSSR Class 1 to Class C; After calculation by the following formula (4), the result probability of each binary classifier DFSSR can be obtained; in,

Images