A traffic light recognition method based on multiple features
By combining multi-feature recognition methods and SVM models, the color moment features and texture features of traffic lights are extracted, which solves the problem of misjudging traffic lights by autonomous vehicles in complex environments and achieves higher recognition accuracy.
Patent Information
- Application Number
- CN202610452842.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2026-04-08
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2046-04-08
AI Technical Summary
Existing unmanned inspection vehicles are easily interfered with by objects of similar color and shape when recognizing traffic lights, leading to misjudgments. Furthermore, improper limitation of the recognition range of traffic light areas in images results in low recognition accuracy.
A multi-feature-based traffic light recognition method is adopted. Color moment features and texture features of image regions are extracted using MATLAB software. Combined with SVM model and bagging algorithm, the method uses color distribution pattern and texture frequency for recognition, eliminating interference and improving recognition accuracy.
It effectively reduces the recognition range of image areas, improves the recognition accuracy of traffic lights, and reduces the false judgment rate, especially significantly improving the recognition effect of traffic lights in complex environments.
Smart Images

Figure CN122024207B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for recognizing traffic lights, and more particularly to a traffic light recognition method based on multiple features. Background Technology
[0002] Currently, driverless inspection vehicles used in roadside parking fee collection systems need to patrol along designated routes to identify and record vehicle information and parking time in roadside parking spaces. These patrol routes often cover intersections and other locations with traffic lights, necessitating that the inspection vehicles possess traffic light recognition capabilities to ensure safety and compliance during the inspection process.
[0003] Based on this, existing methods for recognizing traffic lights in autonomous vehicles mostly rely on identifying the color and shape of different regions in an image to determine if an image region with a specific hue and shape is a traffic light. Specifically, as shown in patent 201510208977.7, this method extracts color and HOG features from the image, and then uses an SVM model to identify the corresponding traffic light information based on these two features. However, this method has a drawback: because it only focuses on the color and edge shape of different regions in the image, it is prone to misjudging images that closely resemble traffic light colors and edge shapes. For example, red arrow patterns on roadside billboards, red circular signs on buildings, and red circular taillights on vehicle rear ends are easily misidentified as traffic lights. Furthermore, when traffic lights are obscured by external objects such as tree branches, power lines, or billboards at their edges, or when traffic lights are distorted due to lens distortion, the resulting shape distortion can easily lead to misjudgment as not being traffic lights.
[0004] Furthermore, to reduce the recognition range and difficulty of the image and decrease the possibility of misjudgment, this method also limits the height range of the traffic light area in the image. Areas outside the set height range are considered less likely to contain traffic lights and are not subsequently recognized. However, this setting is relatively rudimentary. If the camera angle changes, causing the traffic light area to appear lower in the image, or if the actual installation position of a particular traffic light is low, the recognition method will completely fail. On the other hand, removing the limitation on the recognition range would increase the difficulty and accuracy of recognizing traffic light areas in the image. This is because in images captured by autonomous inspection vehicles, the traffic light area occupies a very small proportion of the overall image, sometimes less than 1 / 100. Therefore, if color and shape recognition is based on the entire image, it would require distinguishing numerous interfering graphics with obvious color boundaries, significantly increasing the model's workload and the possibility of misjudgment.
[0005] Therefore, existing methods for recognizing traffic lights suffer from problems such as high workload and low recognition accuracy. Summary of the Invention
[0006] The purpose of this invention is to provide a traffic light recognition method based on multiple features. This method can reduce the recognition range of image regions and improve the accuracy of traffic light recognition.
[0007] The technical solution of this invention: a traffic light recognition method based on multiple features, comprising the following steps:
[0008] A. Obtain the image to be identified, resulting in image a;
[0009] B. Using the regionprops function in MATLAB software, extract the image region in image a based on the saturation and brightness values of pixels in image a. This image region contains the region, centroid, and bounding box, thus obtaining the image region b.
[0010] C. Extract the color moment features of image a in the region of image b. These color moment features are 1*9L color moment feature vectors, resulting in color moment features c.
[0011] D. Process image a to obtain the grayscale image of image a, and then use the convolution of Gabor kernel function and Fourier transform to obtain the texture features of image region b in the grayscale image. The texture features are 1*2MK Gabor texture feature vectors, and thus the texture features d are obtained.
[0012] E. Based on the c color moment feature and d texture feature, the color distribution pattern and texture frequency of the b image region are identified. The c color moment feature is used to display the color distribution pattern of the pixels in the b image region, and the d texture feature is used to display the texture frequency of the pixels in the b image region, so as to obtain the identification result.
[0013] In the aforementioned traffic light recognition method based on multiple features, step B specifically includes the following steps:
[0014] B1. Convert image a to the HSV color space to obtain image a1 in the color space;
[0015] B2. Separate the hue channel from the a1 color space image, and perform median filtering on the saturation and brightness channels of the a1 color space image to obtain the a2 color space image;
[0016] B3. Extract pixels from the a2 color space image according to the set saturation and brightness conditions, so that pixels that meet both saturation and brightness conditions are marked as white, and pixels that do not meet the conditions are marked as black, thus obtaining the a3 binary image.
[0017] B4. Extract the graphic region from the binary image a3 using the regionprops function in MATLAB. After calling the regionprops function, you get stats=regionprops(bw). The properties parameter is not specified and defaults to "basic". "basic" contains three measurements: region, centroid, and bounding box. The region describes the actual number of pixels in the graphic region, the centroid describes the centroid of the graphic region, and the bounding box describes the rectangle that encloses the graphic region. stats is the output variable that stores the results of these three measurements, thus obtaining the region of image b.
[0018] In the aforementioned traffic light recognition method based on multiple features, step C specifically includes the following steps:
[0019] C1. Divide the b image region into L grid intervals according to the set grid size to obtain L b1 grid intervals;
[0020] C2. Calculate the first, second, and third color moments of the image in the three RGB channels within each b1 grid interval to obtain the feature value of 9*L, and thus the feature value of b2.
[0021] C3. Integrate the feature values of b2 to form a 1*9L color moment feature vector, thus obtaining the c color moment feature.
[0022] In the aforementioned traffic light recognition method based on multiple features, step D specifically comprises the following steps:
[0023] D1. Convert image a to grayscale to obtain grayscale image c1;
[0024] D2. Preset and generate the Gabor kernel function to obtain the c2 kernel function value;
[0025] D3. Perform a two-dimensional Fourier transform on the kernel function value of c2, and simultaneously perform a two-dimensional Fourier transform on the grayscale image of c1. Then convolve the two to obtain the convolution value of c3.
[0026] D4. Perform an inverse transformation on the c3 convolution value and take the modulus of the inverse transformation result to obtain the c4 modulus value of the c1 grayscale image at different pixels;
[0027] D5. Divide the b image region into M grid intervals according to the set grid size, so that the b image region is divided into M grid intervals, resulting in M c5 grid intervals;
[0028] D6. Calculate the mean value of the c4 modulus of different pixels within each c5 grid interval, and at the same time calculate the standard deviation of the c4 modulus of different pixels within each c5 grid interval, to obtain M c6 mean vectors and M c7 standard deviation vectors.
[0029] D7. Repeat steps D1-D6 to calculate the c6 mean vector and c7 standard deviation vector of the c1 grayscale image K times to obtain M*K c6 mean vectors and M*K c7 standard deviation vectors.
[0030] D8. Concatenate the M*K c6 mean vectors and the M*K c7 standard deviation vectors to form a 1*2MK Gabor texture feature vector, thus obtaining the d texture feature.
[0031] In the aforementioned traffic light recognition method based on multiple features, step E involves importing the c-color moment feature and d-texture feature into the trained SVM model for recognition, thereby obtaining the recognition result.
[0032] In the aforementioned traffic light recognition method based on multiple features, the training method of the SVM model specifically includes the following steps:
[0033] E11. Collect images of several traffic lights and classify them, with each category corresponding to a label value, to obtain image e11;
[0034] E12. Extract the color moment features and texture features of each e11 image, merge them to obtain the feature vector of the e11 image, and obtain the e12 feature vector;
[0035] E13. Merge the e12 feature vectors of each e11 image to obtain the e13 feature matrix;
[0036] E14. The SVM model is trained based on the feature matrix and label values of e13 to obtain the trained SVM model.
[0037] In the aforementioned traffic light recognition method based on multiple features, the traffic light types in step E11 are divided into interference, red up, red right, red left, red light, green up, green right, green left, green light, yellow up, yellow right, yellow left, and yellow light, where the image corresponding to the interference type is a non-traffic light image.
[0038] In the aforementioned traffic light recognition method based on multiple features, step E specifically involves classifying the traffic lights and then using a trained bagging algorithm to identify the corresponding traffic light type based on the c color moment feature and d texture feature, thereby obtaining the recognition result.
[0039] In the aforementioned traffic light recognition method based on multiple features, the training method of the bagging algorithm is specifically as follows:
[0040] E21. Combine color moment features and texture features as sample features, and randomly select several repeatable sample features to form a training set. Then on the training set The sample features are evaluated, and several optimal sample features are selected to form an optimal feature subset. Thus, the optimal feature subset of e21 is obtained;
[0041] E22. Set Nt basis learners Each base learner All samples are used as the optimal feature subset of e21 for classification training, resulting in Nt e22 base classifiers;
[0042] E23. Use Nt e22 base classifiers to classify the c color moment feature and d texture feature, and then select the classification result with the most occurrences as the recognition result of the bagging algorithm.
[0043] In the aforementioned traffic light recognition method based on multiple features, the optimal feature subset in step E21... The specific selection method is as follows:
[0044] E211. Let the training set be... A set of sample features Initially selected feature subset The training set obtained by random and repeatable sampling is divided into training subsets. With verification subset ;
[0045] E212. Select the base learner from step E22. Using the same model as the base model, sample features are iteratively selected from the base model, with the number of iterations set to k; if the currently selected feature subset... equal to training set Then the selected feature subset If the selected feature subset is used as the optimal feature subset, the iteration terminates; if the currently selected feature subset is... Not equal to the training set Then proceed to step E213;
[0046] E213. Traverse all feature sets that were not selected. Each unselected sample feature is compared with a subset of selected features. Merge to form a temporary feature set; then use the training subset As samples, this temporary feature set is used as input features to train the base model, and then the validation subset is used... The performance index value of the base model is calculated to complete the individual evaluation of all unselected sample features;
[0047] E214. From all unselected sample features, select the sample feature that optimizes the performance index of the temporary feature set corresponding to the base model, and add the sample feature to the currently selected feature subset. The updated selected feature subset is obtained. ;
[0048] E215. Set performance improvement threshold Calculate the updated selected feature subset Performance metrics of the corresponding base model and feature subsets before update The difference between the performance metrics of the corresponding base models; if this difference is less than a preset performance improvement threshold. Then the previously selected feature subset will be updated. As the optimal feature subset Terminate the iteration if the difference is not less than the performance improvement threshold. ,make Return to step E212 and continue execution.
[0049] Compared with the prior art, the present invention has the following characteristics:
[0050] (1) By limiting the traffic light recognition method, this invention enables the extraction of color moment features and texture features of different pixels in the image region, as well as a 1*9L color moment feature vector and a 1*2MK Gabor texture feature vector. This allows subsequent recognition software to be trained and recognized based on the color moment feature vector and Gabor texture feature vector. In other words, it determines whether the image region is a traffic light based on the specific color distribution pattern and texture frequency features formed by different traffic light images. Compared with existing recognition methods, this invention effectively reduces the dependence on the shape and color of the recognition content and improves the recognition effect and accuracy of this invention.
[0051] (2) By limiting the extraction method of the image region, it can also identify and mark different regions in the image based on saturation and brightness, thereby splitting the originally complex and disordered image content into several individual regions that are close to the saturation and brightness values of traffic lights. This greatly reduces the extraction range of color moment features and texture features in the subsequent process, reducing the workload of the present invention in the recognition process. On the other hand, it can also perform preliminary screening of the regions in the image that are close to traffic lights, reducing the possibility that the image regions containing traffic lights will be filtered and separated.
[0052] Based on this, by converting the color space image into a binary image according to the set saturation and brightness conditions, the saturation information can be used to distinguish the traffic light area from the image background, thereby greatly reducing the interference of a single large area of color such as blue sky and road on subsequent recognition, and greatly improving the efficiency of subsequent extraction of color moment features and texture features.
[0053] (3) By limiting the image region extraction method, the present invention can extract an image region with a region, centroid and bounding box, so that the image region not only includes the traffic light region in the image, but also includes the surrounding region connected to the traffic light region. Thus, the two are compared with the inherent color distribution pattern and texture frequency characteristics formed by the real traffic light in the luminous and non-luminous areas, reducing the possibility of misidentification of similar traffic light images in billboards and further improving the accuracy of image region recognition.
[0054] (4) The type of traffic light is identified by using the bagging algorithm, and the optimal feature subset used for training is combined with the algorithm. The limitations of this feature allow it to calculate different color moment features and texture features using different base classifiers, prune redundant features in the color moment features and texture features, and mine the value of different color moment features and texture features, thereby improving the recognition accuracy of each base classifier. At the same time, the voting of multiple base classifiers improves the accuracy of the final recognition result and reduces the interference of abnormal features on the recognition result.
[0055] Therefore, the present invention can reduce the recognition range of image areas and improve the recognition accuracy of traffic lights. Attached Figure Description
[0056] Figure 1 These are sample images from Experiment Example 1;
[0057] Figure 2 This is a schematic diagram of the binary image of a3 in Experiment Example 1;
[0058] Figure 3 This is a schematic diagram of partial coordinate values of the 13 b image regions in Experiment Example 1;
[0059] Figure 4 This is a schematic diagram of the partial color moment feature vectors of image region b in Experiment Example 2;
[0060] Figure 5 This is a schematic diagram of part of the Gabor texture feature vector in image region b of Experiment Example 2;
[0061] Figure 6 This is a schematic diagram of some of the feature matrices in Experiment Example 3;
[0062] Figure 7 This is a schematic diagram of the labeled values of some traffic light images in Experiment Example 3;
[0063] Figure 8 This is a graph showing the prediction results of the SVM model in Experiment Example 3;
[0064] Figure 9 This is the recognition result of the image to be recognized in Experiment Example 4;
[0065] Figure 10 This is the recognition result of another image to be recognized in Experiment Example 4;
[0066] Figure 11 This is the recognition result of another image to be recognized in Experiment Example 4;
[0067] Figure 12 yes Figure 9 A magnified view of a portion of the image;
[0068] Figure 13 yes Figure 10 A magnified view of a portion of the image. Detailed Implementation
[0069] The present invention will be further described below with reference to the accompanying drawings and embodiments, but this should not be construed as limiting the present invention.
[0070] Example 1. A traffic light recognition method based on multiple features, comprising the following steps:
[0071] A. Obtain the image to be identified, resulting in image a;
[0072] B. Using the regionprops function in MATLAB software, extract the image region in image a based on the saturation and brightness values of pixels in image a. This image region contains the region, centroid, and bounding box, thus obtaining the image region b.
[0073] C. Extract the color moment features of image a in the region of image b. These color moment features are 1*9L color moment feature vectors, resulting in color moment features c.
[0074] D. Process image a to obtain the grayscale image of image a, and then use the convolution of Gabor kernel function and Fourier transform to obtain the texture features of image region b in the grayscale image. The texture features are 1*2MK Gabor texture feature vectors, and thus the texture features d are obtained.
[0075] E. Based on the c color moment feature and d texture feature, the color distribution pattern and texture frequency of the b image region are identified. The c color moment feature is used to display the color distribution pattern of the pixels in the b image region, and the d texture feature is used to display the texture frequency of the pixels in the b image region, so as to obtain the identification result.
[0076] Step B specifically includes the following steps:
[0077] B1. Convert image a to the HSV color space to obtain image a1 in the color space;
[0078] B2. Separate the hue channel from the a1 color space image, and perform median filtering on the saturation and brightness channels of the a1 color space image to obtain the a2 color space image;
[0079] B3. Extract pixels from the a2 color space image according to the set saturation and brightness conditions. The saturation and brightness conditions are artificially limited according to the saturation and brightness values of the actual traffic lights in the image. Pixels that meet both the saturation and brightness conditions are marked as white, and pixels that do not meet the conditions are marked as black, thus obtaining the a3 binary image.
[0080] B4. The binary image a3 is processed to remove noise and smooth the image. The region props function in MATLAB is used to extract the graphic region from the a3 binary image. After calling the region props function, stats = region props(bw) is obtained. The properties parameter is not specified and defaults to "basic". "basic" contains three measurements: region, centroid, and bounding box. The region describes the actual number of pixels in the graphic region, the centroid describes the centroid of the graphic region, and the bounding box describes the rectangle that encloses the graphic region. That is, the edge of the bounding box is the tangent line of the white region at the edge. stats is the output variable that stores the results of these three measurements, thus obtaining the region of image b.
[0081] Step C specifically includes the following steps:
[0082] C1. Divide each b image region into L grid intervals according to the set grid size. The number of grid intervals L is set manually, specifically limited to 25, resulting in L b1 grid intervals;
[0083] C2. Calculate the first, second, and third color moments of the image in the RGB channels for each grid interval b1. The first color moment is obtained using the formula... The second-order color moment was calculated using the formula... The calculated third-order color moments are obtained through the formula... The calculation yielded, where Let represent the i-th color component of the j-th pixel in the color image, and N represent the number of pixels in the image, thus obtaining the feature value of 9*L, and obtaining the feature value of b2;
[0084] C3. Integrate the feature values of b2 to form a 1*9L color moment feature vector, thus obtaining the c color moment feature.
[0085] The specific steps of step D are as follows:
[0086] D1. Convert image a to grayscale to obtain grayscale image c1;
[0087] D2. Preset and generate the Gabor kernel function to obtain the c2 kernel function value;
[0088] D3. Perform a two-dimensional Fourier transform on the kernel function value of c2, and simultaneously perform a two-dimensional Fourier transform on the grayscale image of c1. Then convolve the two to obtain the convolution value of c3.
[0089] D4. Perform an inverse transformation on the c3 convolution value and take the modulus of the inverse transformation result to obtain the c4 modulus value of the c1 grayscale image at different pixels;
[0090] D5. Divide the b image region into M grid intervals according to the set grid size. The number of grid intervals M is set manually, specifically 4, resulting in M c5 grid intervals.
[0091] D6. Calculate the mean value of the c4 modulus of different pixels within each c5 grid interval, and at the same time calculate the standard deviation of the c4 modulus of different pixels within each c5 grid interval, to obtain M c6 mean vectors and M c7 standard deviation vectors.
[0092] D7. Repeat steps D1-D6 to calculate the c6 mean vector and c7 standard deviation vector of the c1 grayscale image K times. The number K is set manually, specifically 20 times. Each time the calculation is performed, the scaling parameter of the Gabor kernel function is changed. The upper and lower bounds of this scaling parameter are set by the designer according to actual needs. Parameter points are sampled at equal intervals within the upper and lower bounds to obtain M*K c6 mean vectors and M*K c7 standard deviation vectors.
[0093] D8. Concatenate the M*K c6 mean vectors and the M*K c7 standard deviation vectors to form a 1*2MK Gabor texture feature vector, thus obtaining the d texture feature.
[0094] If there are multiple image regions b in step B, then the Gabor texture feature vector of each image region b is extracted according to steps D5-D8.
[0095] The mathematical expression for the Gabor kernel function is: ,
[0096] in ,
[0097] Substitution The final calculation yielded
[0098] ;
[0099] Where x and y are coordinates, and the scaling parameters of the Gabor kernel function include the center frequency. The direction θ and Gaussian envelope width σ are used to generate a multi-scale, multi-directional filter bank by adjusting the scaling parameters of the Gabor kernel function, with the center frequency... The direction θ and the Gaussian envelope width σ are specifically selected according to the standard selection method of the conventional Gabor kernel function in image texture feature extraction.
[0100] In step E, the c color moment feature and d texture feature are imported into the trained SVM model for recognition, and the recognition result is obtained.
[0101] The training method for the SVM model specifically includes the following steps:
[0102] E11. Collect images of several traffic lights, remove background and irrelevant images from the images, retain only the features related to the traffic lights, and classify the images. Each category corresponds to a label value, and each category contains multiple traffic lights of the same type, resulting in image e11.
[0103] E12. Extract the color moment features and texture features of each e11 image, merge them to obtain the feature vector of the e11 image, and obtain the e12 feature vector;
[0104] E13. Merge the e12 feature vectors of each e11 image to obtain the e13 feature matrix;
[0105] E14. The SVM model is trained based on the feature matrix and label values of e13 to obtain the trained SVM model.
[0106] In step E11, traffic light types are categorized as interference, red up, red right, red left, red light, green up, green right, green left, green light, yellow up, yellow right, yellow left, and yellow light. The image corresponding to the interference type is a non-traffic light image that approximates the color or shape of a traffic light, used to distinguish non-traffic light images during model training.
[0107] This embodiment first extracts the image region b from image a, enabling the extraction of areas near traffic lights in image a based on the saturation and brightness of the image content. This reduces the subsequent recognition range of image a and improves the accuracy of traffic light extraction. After extracting the image region b, the color moment features and texture features of each region in image b are extracted. These features are then used as conditions to determine whether the image region contains a traffic light and the type of traffic light. Compared to the conventional method that uses image color and shape as judgment conditions, this method can identify the brightness and saturation distribution of each pixel in the image region, as well as the overall texture features of the image region. This effectively eliminates the influence of image distortion caused by shooting direction deviation or external object obstruction, and avoids the situation where red circular patterns on objects such as light boxes, posters, and taillights are mistaken for traffic lights, thus improving the accuracy of traffic light recognition in images.
[0108] Based on the above, by limiting the extraction methods of color moment features and texture features, this invention can calculate a 1*9L color moment feature vector and a 1*2MK Gabor texture feature vector, thereby providing conditions for the subsequent training of the SVM model and enabling the application of color moment features and texture features in the traffic light recognition model.
[0109] Example 2. A traffic light recognition method based on multiple features, comprising the following steps:
[0110] A. Obtain the image to be identified, resulting in image a;
[0111] B. Using the regionprops function in MATLAB software, extract the image region in image a based on the saturation and brightness values of pixels in image a. This image region contains the region, centroid, and bounding box, thus obtaining the image region b.
[0112] C. Extract the color moment features of image a in the region of image b. These color moment features are 1*9L color moment feature vectors, resulting in color moment features c.
[0113] D. Process image a to obtain the grayscale image of image a, and then use the convolution of Gabor kernel function and Fourier transform to obtain the texture features of image region b in the grayscale image. The texture features are 1*2MK Gabor texture feature vectors, and thus the texture features d are obtained.
[0114] E. Based on the c color moment feature and d texture feature, the color distribution pattern and texture frequency of the b image region are identified. The c color moment feature is used to display the color distribution pattern of the pixels in the b image region, and the d texture feature is used to display the texture frequency of the pixels in the b image region, so as to obtain the identification result.
[0115] Step B specifically includes the following steps:
[0116] B1. Convert image a to the HSV color space to obtain image a1 in the color space;
[0117] B2. Separate the hue channel from the a1 color space image, and perform median filtering on the saturation and brightness channels of the a1 color space image to obtain the a2 color space image;
[0118] B3. Extract pixels from the a2 color space image according to the set saturation and brightness conditions. The saturation and brightness conditions are artificially limited according to the saturation and brightness values of the actual traffic lights in the image. Pixels that meet both the saturation and brightness conditions are marked as white, and pixels that do not meet the conditions are marked as black, thus obtaining the a3 binary image.
[0119] B4. The binary image a3 is processed to remove noise and smooth the image. The region props function in MATLAB is used to extract the graphic region from the binary image a3. After calling the region props function, stats = region props(bw) is obtained. The properties parameter is not specified and defaults to "basic". "basic" contains three measurements: region, centroid, and bounding box. The region describes the actual number of pixels in the graphic region, the centroid describes the centroid of the graphic region, and the bounding box describes the rectangle that encloses the graphic region. stats is the output variable that stores the results of these three measurements, thus obtaining the region of image b.
[0120] Step C specifically includes the following steps:
[0121] C1. Divide each b image region into L grid intervals according to the set grid size, resulting in L b1 grid intervals;
[0122] C2. Calculate the first, second, and third color moments of the image in the RGB channels for each grid interval b1. The first color moment is obtained using the formula... The second-order color moment was calculated using the formula... The calculated third-order color moments are obtained through the formula... The calculation yielded, where Let represent the i-th color component of the j-th pixel in the color image, and N represent the number of pixels in the image, thus obtaining the feature value of 9*L, and obtaining the feature value of b2;
[0123] C3. Integrate the feature values of b2 to form a 1*9L color moment feature vector, thus obtaining the c color moment feature.
[0124] The specific steps of step D are as follows:
[0125] D1. Convert image a to grayscale to obtain grayscale image c1;
[0126] D2. Preset and generate the Gabor kernel function to obtain the c2 kernel function value;
[0127] D3. Perform a two-dimensional Fourier transform on the kernel function value of c2, and simultaneously perform a two-dimensional Fourier transform on the grayscale image of c1. Then convolve the two to obtain the convolution value of c3.
[0128] D4. Perform an inverse transformation on the c3 convolution value and take the modulus of the inverse transformation result to obtain the c4 modulus value of the c1 grayscale image at different pixels;
[0129] D5. Divide the b image region into M grid intervals according to the set grid size, so that the b image region is divided into M grid intervals, resulting in M c5 grid intervals;
[0130] D6. Calculate the mean value of the c4 modulus of different pixels within each c5 grid interval, and at the same time calculate the standard deviation of the c4 modulus of different pixels within each c5 grid interval, to obtain M c6 mean vectors and M c7 standard deviation vectors.
[0131] D7. Repeat steps D1-D6 to calculate the c6 mean vector and c7 standard deviation vector of the c1 grayscale image K times. Change the scaling parameter of the Gabor kernel function each time to obtain M*K c6 mean vectors and M*K c7 standard deviation vectors.
[0132] D8. Concatenate the M*K c6 mean vectors and the M*K c7 standard deviation vectors to form a 1*2MK Gabor texture feature vector, thus obtaining the d texture feature.
[0133] The mathematical expression for the Gabor kernel function is: ,
[0134] in ,
[0135] Substitution The final calculation yielded
[0136] ;
[0137] Where x and y are coordinates, and the scaling parameters of the Gabor kernel function include the center frequency. The direction θ and Gaussian envelope width σ are used to generate a multi-scale, multi-directional filter bank by adjusting the scaling parameters of the Gabor kernel function, with the center frequency... The direction θ and the Gaussian envelope width σ are specifically selected according to the standard selection method of the conventional Gabor kernel function in image texture feature extraction.
[0138] The specific steps of step E are as follows: classify traffic lights, and then use the trained bagging algorithm to identify the b image region based on the c color moment feature and d texture feature, thereby identifying whether the b image region is a traffic light and the type of the traffic light, and obtaining the identification result.
[0139] The traffic lights can be divided into 13 categories: "interference", "red up", "red right", "red left", "red light", "green up", "green right", "green left", "green light", "yellow up", "yellow right", "yellow left" and "yellow light".
[0140] The training method for the bagging algorithm is as follows:
[0141] E21. Combine any color moment feature and texture feature to obtain a 1*9L+2MK sample feature. Randomly select several repeatable sample features to form a training set. Then on the training set The sample features are evaluated, and several optimal sample features are selected to form an optimal feature subset. Thus, the optimal feature subset of e21 is obtained;
[0142] E22. Set Nt basis learners Each base learner All samples are used as the optimal feature subset of e21 for classification training, resulting in Nt e22 base classifiers;
[0143] E23. Use Nt e22 base classifiers to classify the c color moment feature and d texture feature, and then select the classification result with the most occurrences as the recognition result of the bagging algorithm.
[0144] The optimal feature subset in step E21 The specific selection method is as follows:
[0145] E211. Let the training set be... A set of sample features Initially selected feature subset The training set obtained by random and repeatable sampling is divided into training subsets. With verification subset ;
[0146] E212. Select the base learner from step E22. Using the same model as the base model, sample features are iteratively selected from the base model, with the number of iterations set to k; if the currently selected feature subset... equal to training set Then the selected feature subset If the selected feature subset is used as the optimal feature subset, the iteration terminates; if the currently selected feature subset is... Not equal to the training set Then proceed to step E213;
[0147] E213. Traverse all feature sets that were not selected. Each unselected sample feature is compared with a subset of selected features. Merge to form a temporary feature set; then use the training subset As samples, this temporary feature set is used as input features to train the base model, and then the validation subset is used... The performance index value of the base model is calculated to complete the individual evaluation of all unselected sample features;
[0148] E214. From all unselected sample features, select the sample feature that optimizes the performance index of the temporary feature set corresponding to the base model, and add the sample feature to the currently selected feature subset. The updated selected feature subset is obtained. ;
[0149] The performance index of the base model is one of classification accuracy, precision, or F1 score. The larger the performance index value, the better the performance of the base model.
[0150] E215. Set performance improvement threshold Calculate the updated selected feature subset Performance metrics of the corresponding base model and feature subsets before update The difference between the performance metrics of the corresponding base models; if this difference is less than a preset performance improvement threshold. Then the previously selected feature subset will be updated. As the optimal feature subset Terminate the iteration if the difference is not less than the performance improvement threshold. ,make Return to step E212 and continue execution.
[0151] Compared to Example 1, this example adjusts the training method of the bagging algorithm and uses the optimized bagging algorithm to identify traffic lights. This allows the bagging algorithm to evaluate and filter each color moment feature and texture feature during training, retaining the subset of features with the highest discriminative power and lowest redundancy for traffic light categories as training samples for the base classifiers. This enables the algorithm to prune redundant features in each color moment feature and texture feature, and to traverse the parameters of both types of features. This allows the originally low-correlation color moment features and texture features to form specific high-value combinations, thus improving the training effect of subsequent base classifiers and the accuracy of traffic light recognition. Furthermore, by using multiple base classifiers to vote for the final recognition result, the inconsistency between different base classifiers can be used to improve the accuracy of judgment on complex images.
[0152] Experiment Example 1: This experiment uses an image containing a red light as a sample. The sample is as follows: Figure 1 As shown, the sample is then processed using steps B1-B3 in Example 1 to obtain the processed binary image a3, as shown in the figure. Figure 2 As shown. Finally, following step B4, the graphic regions in the binary image a3 are extracted, yielding 13 graphic regions that may represent traffic lights, thus obtaining the image region b. Partial coordinate values of the 13 image regions b are shown below. Figure 3 As shown.
[0153] Experimental Example 2: This experimental example processes each b image region from Experimental Example 1 according to steps C1-C3 of Example 1, dividing the b image region into 25 b1 grid intervals. A 1*225 color moment feature vector is extracted from each b image region, resulting in a total of 13 1*225 color moment feature vectors. A portion of the color moment feature vector for one b image region is shown below. Figure 4 As shown.
[0154] Simultaneously, each b image region in Experimental Example 1 is processed according to steps D1-D8 of Example 1, dividing the b image region into four c5 grid intervals. Each c5 grid interval yields one c6 mean vector and one c7 standard deviation vector. This calculation is repeated 20 times, resulting in a total of 80 c6 mean vectors and 80 c7 standard deviation vectors across the four c5 grid intervals. These vectors are then concatenated to form 1*160 Gabor texture feature vectors, resulting in a total of 13 1*160 Gabor texture feature vectors. A portion of the Gabor texture feature vector for one b image region is shown below. Figure 5 As shown.
[0155] Experiment Example 3: A total of 1827 traffic light images across 13 categories were collected. For each image, a 1*225 color moment feature vector and a 1*160 Gabor texture feature vector could be extracted. These two vectors were then merged to obtain a single 1*385 vector. The features from all 1827 images were then combined to obtain a 1827*385 feature matrix. A portion of this feature matrix is shown below. Figure 6 As shown.
[0156] Simultaneously, 1827 traffic light images across 13 categories were labeled, converting them into numerical values from 1 to 13. Specifically, 1-Interference, 2-Red Up, 3-Red Right, 4-Red Left, 5-Red Light, 6-Green Up, 7-Green Right, 8-Green Left, 9-Green Light, 10-Yellow Up, 11-Yellow Right, 12-Yellow Left, and 13-Yellow Light. The label values for some traffic light images are shown below. Figure 7 As shown.
[0157] The feature matrix and the label values of 1827 traffic lights are then input into the SVM model for training, resulting in a trained SVM model.
[0158] Finally, the 13 1*225 color moment feature vectors and 13 1*160 Gabor texture feature vectors from Experiment 2 were imported into the SVM model. The SVM model then used each set of color moment feature vectors and Gabor texture feature vectors to identify whether the 13 b-image regions in Experiment 1 were traffic lights and to determine the type of traffic lights. The prediction results of the SVM model for the image regions containing traffic lights are as follows: Figure 8 As shown.
[0159] Experiment Example 4: Multiple images were identified using the identification method described in Example 2. The identification results are as follows: Figure 9-11 As shown, Figure 9-11 The left image in the image is the image to be identified, the middle image is the binary image after processing in the HSV color space, and the right image is the recognition result. Figure 12 for Figure 9 A magnified view of a portion of the image. Figure 12 The blue rectangle in the middle is Figure 9 The image area corresponding to the traffic lights in the middle. Figure 13 yes Figure 10 A magnified view of a portion of the image.
[0160] pass Figure 9 It can be seen that, Figure 9 The illuminated display board contains the yellow-green words "United We Stand, Safeguarding Safety," which include numerous patterns resembling traffic arrows. The colors and brightness of these patterns are similar to those of real traffic lights, easily leading to misidentification. This application, however, by limiting the identification method, can accurately identify that the text is not a traffic light. Furthermore, through… Figure 12 As can be seen, the image region extracted by this application can include the traffic light area and the adjacent non-luminous area, thereby improving its recognition accuracy by utilizing the combination of the two.
[0161] pass Figure 10 and Figure 13 It can be seen that, Figure 10 Due to the shooting angle, the arrow pattern is distorted relative to a frontal view, and the arrow pattern is incomplete at the edges due to the obstruction of the light shield, making it easily mistaken for a non-traffic light. However, this application, by limiting the recognition method, can accurately identify the pattern as the "green right" traffic light.
[0162] pass Figure 11 It can be seen that, Figure 11The traffic light area occupies a small portion of the overall image, while multiple taillights of similar color and brightness resemble traffic lights. Large areas of non-traffic light regions in the image also act as interference. This significantly increases the workload and difficulty of identifying the entire image, and also makes it easy to mistake taillights for traffic lights. However, this application, through image processing in step B, can omit the large non-traffic light areas in the image, extracting only the small areas containing taillights and traffic lights, and accurately identifying the traffic lights in each image region.
Claims
1. A traffic light recognition method based on multiple features, characterized in that, Includes the following steps: A. Obtain the image to be identified, resulting in image a; B. Using the regionprops function in MATLAB software, extract the image region in image a based on the saturation and brightness values of pixels in image a. This image region contains the region, centroid, and bounding box, thus obtaining the image region b. C. Extract the color moment features of image a in the region of image b. These color moment features are 1*9L color moment feature vectors, resulting in color moment features c. D. Process image a to obtain the grayscale image of image a, and then use the convolution of Gabor kernel function and Fourier transform to obtain the texture features of image region b in the grayscale image. The texture features are 1*2MK Gabor texture feature vectors, and thus the texture features d are obtained. E. Based on the c color moment feature and d texture feature, the color distribution pattern and texture frequency of the b image region are identified. The c color moment feature is used to display the color distribution pattern of the pixels in the b image region, and the d texture feature is used to display the texture frequency of the pixels in the b image region, so as to obtain the identification result. Step C specifically includes the following steps: C1. Divide the b image region into L grid intervals according to the set grid size to obtain L b1 grid intervals; C2. Calculate the first, second, and third color moments of the image in the three RGB channels within each b1 grid interval to obtain the feature value of 9*L, and thus the feature value of b2. C3. Integrate the feature values of b2 to form a 1*9L color moment feature vector, thus obtaining the c color moment feature; The specific steps of step D are as follows: D1. Convert image a to grayscale to obtain grayscale image c1; D2. Preset and generate the Gabor kernel function to obtain the c2 kernel function value; D3. Perform a two-dimensional Fourier transform on the kernel function value of c2, and simultaneously perform a two-dimensional Fourier transform on the grayscale image of c1. Then convolve the two to obtain the convolution value of c3. D4. Perform an inverse transformation on the c3 convolution value and take the modulus of the inverse transformation result to obtain the c4 modulus value of the c1 grayscale image at different pixels; D5. Divide the b image region into M grid intervals according to the set grid size, so that the b image region is divided into M grid intervals, resulting in M c5 grid intervals; D6. Calculate the mean value of the c4 modulus of different pixels within each c5 grid interval, and at the same time calculate the standard deviation of the c4 modulus of different pixels within each c5 grid interval, to obtain M c6 mean vectors and M c7 standard deviation vectors. D7. Repeat steps D1-D6 to calculate the c6 mean vector and c7 standard deviation vector of the c1 grayscale image K times. Change the scaling parameter of the Gabor kernel function each time to obtain M*K c6 mean vectors and M*K c7 standard deviation vectors. D8. Concatenate the M*K c6 mean vectors and the M*K c7 standard deviation vectors to form a 1*2MK Gabor texture feature vector, thus obtaining the d texture feature.
2. The traffic light recognition method based on multiple features according to claim 1, characterized in that, Step B specifically includes the following steps: B1. Convert image a to the HSV color space to obtain image a1 in the color space; B2. Separate the hue channel from the a1 color space image, and perform median filtering on the saturation and brightness channels of the a1 color space image to obtain the a2 color space image; B3. Extract pixels from the a2 color space image according to the set saturation and brightness conditions, so that pixels that meet both saturation and brightness conditions are marked as white, and pixels that do not meet the conditions are marked as black, thus obtaining the a3 binary image. B4. Extract the graphic region from the binary image a3 using the regionprops function in MATLAB. This graphic region contains a region, a centroid, and a bounding box. The region represents the actual number of pixels in the graphic region, the centroid represents the centroid of the graphic region, and the bounding box represents the rectangle that encloses the graphic region, thus obtaining the image region b.
3. The traffic light recognition method based on multiple features according to claim 1, characterized in that, In step E, the c color moment feature and d texture feature are imported into the trained SVM model for recognition, and the recognition result is obtained.
4. The traffic light recognition method based on multiple features according to claim 3, characterized in that, The training method for the SVM model specifically includes the following steps: E11. Collect images of several traffic lights and classify them, with each category corresponding to a label value, to obtain image e11; E12. Extract the color moment features and texture features of each e11 image, merge them to obtain the feature vector of the e11 image, and obtain the e12 feature vector; E13. Merge the e12 feature vectors of each e11 image to obtain the e13 feature matrix; E14. The SVM model is trained based on the feature matrix and label values of e13 to obtain the trained SVM model.
5. The traffic light recognition method based on multiple features according to claim 4, characterized in that: In step E11, the traffic light types are divided into interference, red up, red right, red left, red light, green up, green right, green left, green light, yellow up, yellow right, yellow left, and yellow light. The image corresponding to the interference type is a non-traffic light image.
6. The traffic light recognition method based on multiple features according to claim 1, characterized in that, The specific steps of step E are as follows: classify the traffic lights, and then use the trained bagging algorithm to identify the corresponding traffic light type based on the c color moment feature and d texture feature to obtain the identification result.
7. A traffic light recognition method based on multiple features according to claim 6, characterized in that, The training method for the bagging algorithm is as follows: E21. Combine color moment features and texture features as sample features, and randomly select several repeatable sample features to form a training set. Then on the training set The sample features are evaluated, and several optimal sample features are selected to form an optimal feature subset. Thus, the optimal feature subset of e21 is obtained; E22. Set Nt basis learners Each base learner All samples are used as the optimal feature subset of e21 for classification training, resulting in Nt e22 base classifiers; E23. Use Nt e22 base classifiers to classify the c color moment feature and d texture feature, and then select the classification result with the most occurrences as the recognition result of the bagging algorithm.
8. A traffic light recognition method based on multiple features according to claim 7, characterized in that: The optimal feature subset in step E21 The specific selection method is as follows: E211. Let the training set be... A set of sample features Initially selected feature subset The training set obtained by random and repeatable sampling is divided into training subsets. With verification subset ; E212. Select the base learner from step E22. Using the same model as the base model, sample features are iteratively selected from the base model, with the number of iterations set to k; if the currently selected feature subset... equal to training set Then the selected feature subset If the selected feature subset is used as the optimal feature subset, the iteration terminates; if the currently selected feature subset is... Not equal to the training set Then proceed to step E213; E213. Traverse all feature sets that were not selected. Each unselected sample feature is compared with a subset of selected features. Merge to form a temporary feature set; then use the training subset As samples, this temporary feature set is used as input features to train the base model, and then the validation subset is used... The performance index value of the base model is calculated to complete the individual evaluation of all unselected sample features; E214. From all unselected sample features, select the sample feature that optimizes the performance index of the temporary feature set corresponding to the base model, and add the sample feature to the currently selected feature subset. The updated selected feature subset is obtained. ; E215. Set performance improvement threshold Calculate the updated selected feature subset Performance metrics of the corresponding base model and feature subsets before update The difference in performance index values between corresponding base models; If the difference is less than the preset performance improvement threshold Then the previously selected feature subset will be updated. As the optimal feature subset Terminate the iteration if the difference is not less than the performance improvement threshold. ,make Return to step E212 and continue execution.
Citation Information
Patent Citations
A Fast Traffic Light Detection Algorithm for Autonomous Vehicles
CN104766071B
Traffic signal lamp recognition method and device based on convolution neural network
CN106570494A
High-speed road obstacle identification method based on cascade connection of Haar features and depth estimation features
CN120689839A
