Automatic driving complex scene traffic sign detection method based on multi-channel image extraction

Abstract

The application discloses an automatic driving complex scene traffic sign detection method based on multi-channel image extraction and belongs to the technical field of image detection. The application solves the problem that the existing automatic driving method has low detection accuracy due to image loss, deformation and blur, acquires multiple to-be-detected traffic sign images in a complex scene, performs enhancement processing on corresponding channel images according to the traffic signs, and extracts feature regions from each traffic sign; each to-be-detected traffic sign image is recognized and detected through a convolutional neural network, the accuracy of traffic sign detection is ensured, multiple detection results are saved to form a traffic sign database, so that the current to-be-detected traffic sign image is matched with the traffic sign database after the above operation to obtain corresponding detection results; and the results are fed back to an automatic driving system to assist the automatic driving system in controlling a car to run according to traffic signs.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image detection, in particular to an automatic driving complex scene traffic sign detection method based on multi-channel image extraction. BACKGROUND

[0002] With the deep integration of new generation digital technologies such as artificial intelligence and 5G mobile communication with the automobile industry, the intelligent transformation and development of the automobile industry continues to accelerate, and the market acceptance of automatic driving continues to improve. Traffic sign recognition plays a crucial role in intelligent transportation systems, which directly affects the safe driving of vehicles and the smoothness of road traffic. For unmanned driving systems, traffic sign recognition is the basis for autonomous navigation and decision-making. Unmanned vehicles need to accurately recognize and understand traffic sign information to make correct driving decisions and ensure driving safety. Improving the accuracy and real-time performance of traffic sign recognition is a key problem that needs to be solved in the process of practical application. Incorrect recognition results not only cannot assist driving, but also may cause safety accidents. Therefore, the research and development of traffic sign recognition technology are of great significance for improving road traffic safety and promoting the widespread application of intelligent transportation systems.

[0003] In complex traffic scenes, small target detection of traffic signs is a major challenge in the field of target detection. In existing detection techniques, traffic sign images may be missing, deformed, and blurred due to factors such as light and occlusion, resulting in low accuracy of complex scene traffic sign recognition during automatic driving.

[0004] Therefore, the existing needs are not met, and for this purpose, an automatic driving complex scene traffic sign detection method based on multi-channel image extraction is proposed. SUMMARY

[0005] The present application aims to provide an automatic driving complex scene traffic sign detection method based on multi-channel image extraction, by obtaining multiple traffic sign images to be detected in a complex scene, enhancing the corresponding channel images according to the traffic signs, and extracting feature regions from each traffic sign; through a convolutional neural network, each traffic sign image to be detected is recognized and detected to ensure the accuracy of traffic sign detection, and multiple detection results are saved to form a traffic sign database, so that the current traffic sign image to be detected is matched with the traffic sign database after the above operation to obtain the corresponding detection result; the result is fed back to the automatic driving system to assist the automatic driving system in controlling the vehicle according to the traffic sign, thereby solving the problems raised in the above background technology.

[0006] To achieve the above purpose, the present application provides the following technical scheme:

[0007] The automatic driving complex scene traffic sign detection method based on multi-channel image extraction comprises the following steps:

[0008] Step one, by acquiring the images of multiple traffic signs to be detected in a complex scene, the image samples of the traffic signs are formed;

[0009] Step two, separate the multi-channel image of the traffic sign into individual channels, including red, green and blue channels, and give each channel a corresponding name or identifier according to the meaning of the channel;

[0010] Step three, determine the image mark area of the traffic sign to be detected, enhance the corresponding channel image according to the channel name of the feature area, and keep the remaining channel image unchanged; then extract the channel area image by color histogram method to obtain the feature area of the traffic sign image to be detected;

[0011] Step four, create a convolutional neural network, import the image of the traffic sign to be detected processed in step four into the convolutional neural network for training and testing, and identify the features of the traffic sign image to be detected through the convolutional neural network;

[0012] Step five, use the above steps to identify the marks of multiple traffic sign images to be detected, save the identification results and corresponding images, and form a traffic sign database;

[0013] Step six, process the images of the traffic signs to be detected in the complex scene in real time according to the above steps, identify the features of the traffic sign image to be detected through the convolutional neural network, match it with the traffic sign database, and output the corresponding results, including the type, meaning and corresponding indication of the sign, and feedback the traffic sign detection results to the automatic driving system through wireless transmission technology.

[0014] Further, after acquiring the images of multiple traffic signs to be detected in a complex scene in step one, the acquired images of multiple traffic signs to be detected are transmitted to the image processing library through wireless transmission, and the images of traffic signs to be detected are preprocessed.

[0015] Further, the names given by the red, green and blue channels in step two include the sign patterns, characters and edge shapes of the traffic signs in the traffic sign image to be detected.

[0016] Further, in step three, the corresponding channel image is enhanced according to the channel name of the feature area, specifically:

[0017] The image of the traffic sign to be detected is read by an image processing library, and a multi-channel histogram equalization method is used to enhance the specified channel image of the traffic sign to be detected in the RGB and HIS spaces, so as to prevent color distortion of the image of the traffic sign to be detected due to uneven illumination.

[0018] Further, after obtaining the feature region of the traffic sign to be detected in step three, the following steps are included:

[0019] The traffic sign to be detected after feature processing is divided into a training set and a test set, the training set is used to establish the feature points of the convolutional neural network, and the test set is used to verify the accuracy of the convolutional neural network.

[0020] Further, after dividing the traffic sign to be detected after feature processing into a training set and a test set, the following steps are included:

[0021] After the plurality of traffic sign images to be detected are sequentially subjected to feature extraction according to the above steps, they are saved to form a sample data set; and the sample data set includes traffic sign marks under different environments, different weather conditions, sizes and color changes, and different sample data sets are labeled and classified according to the content of the traffic sign.

[0022] Further, the step four of identifying and detecting the features of the traffic sign image to be detected by the convolutional neural network is specifically:

[0023] The training set of the traffic sign image to be detected is imported into the convolutional neural network for training, the feature region to which the training set belongs is identified by a convolution kernel, and the feature points of the convolutional neural network are established; then the test set is imported into the convolutional neural network for testing, and the test result is output, the feature region identified by the test result is compared with the feature region extracted in advance, the accuracy of the convolutional neural network test is determined, and the optimized convolutional neural network is formed.

[0024] Further, after the optimized convolutional neural network is formed, if a new traffic sign image appears, the above steps are operated, and the convolutional neural network is continuously imported for training and testing, and the convolutional neural network is iteratively optimized.

[0025] Further, the pre-processing of the image of the traffic sign to be detected is specifically:

[0026] The noise in the traffic sign image to be detected is removed, and the image is enhanced, the contrast and brightness of the image are adjusted, and the size of the image is uniformly adjusted to reduce the interference factors in the traffic sign image to be detected.

[0027] Further, the images of the plurality of traffic signs to be detected in the complex scene obtained in the step one are all RGB images.

[0028] Compared with the prior art, the present application has the following advantages:

[0029] The present application obtains images of a plurality of traffic signs to be detected in a complex scene, separates channels for each image, and enhances the corresponding channel images according to the logo, text and edge shape of the traffic signs, so as to facilitate the extraction of feature regions from each traffic sign. The convolutional neural network is used to further identify and detect the feature regions of each traffic sign image to be detected, thereby ensuring the accuracy of traffic sign detection. The identification and detection results of a plurality of traffic sign images to be detected are saved to form a complete traffic sign database. After the current traffic sign image to be detected is processed according to the above operation, it is matched with the traffic sign database to obtain the corresponding detection result. The result is fed back to the autonomous driving system through wireless transmission technology, which assists the autonomous driving system to control the car according to the content of the traffic sign. This method not only improves the safety and reliability of the autonomous driving vehicle in complex environment, but also helps to improve the intelligent level of the autonomous driving technology, so that it can better adapt to various road conditions and traffic environment, thereby providing users with more convenient and safe travel experience. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 The flowchart of the traffic sign detection method in the complex scene of the autonomous driving based on the multi-channel image extraction of the present application. DETAILED DESCRIPTION

[0031] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0032] In order to solve the technical problem of low recognition accuracy of traffic signs in complex scenes during autonomous driving caused by missing, deformation and blur of traffic sign images due to factors such as light and shielding in the prior art, please refer to Figure 1 The present embodiment provides the following technical solutions:

[0033] The traffic sign detection method in the complex scene of the autonomous driving based on the multi-channel image extraction comprises the following steps:

[0034] Step one, by obtaining multiple existing traffic signs images in a complex scene, the image samples of traffic signs are constructed; wherein, the obtained multiple traffic signs images in a complex scene are all RGB images; specifically, the method of obtaining existing traffic signs images can use the existing public traffic signs image dataset, which includes a large number of traffic signs images, such as speed limit, construction, no parking, etc. signs; or by shooting in the actual road or simulated environment, the traffic signs are shot, this method can obtain more real and diversified images, including traffic signs under different angles and lighting conditions. Through this method, more abundant data sets can be obtained, which helps to improve the generalization ability of the model; or the network crawler technology is used to capture existing traffic signs related pictures from the Internet, which can be used for training and testing machine learning models.

[0035] After obtaining multiple traffic signs images in a complex scene, the obtained multiple traffic signs images are transmitted to the image processing library through wireless transmission, and the traffic signs images are preprocessed, specifically:

[0036] The noise in the traffic signs image is removed, and the image is enhanced, the contrast and brightness of the image are adjusted, and the size of the image is uniformly adjusted to reduce the interference factors in the traffic signs image; thus, the image quality is improved and the features of the traffic signs are highlighted, which helps to reduce the influence of noise and extract the edge information of the image, preparing for subsequent feature extraction and recognition.

[0037] Step two, separate the traffic signs multi-channel image into individual channels through the image processing library, including red channel, green channel and blue channel, and give each channel a corresponding name or identifier according to the meaning of the channel; wherein, the names given by the red channel, green channel and blue channel include the traffic signs belonging to the traffic signs image, the text and the edge shape; specifically, in the OpenCV image processing library, the sp l it() function is used to separate the multi-channel image, such as color image, into individual channels, the sp l it() function accepts a multi-channel image as input and outputs an array containing the separated single-channel image, the following is the basic steps of using sp l it() function for channel separation:

[0038] By using the functions in the cv2 module of the OpenCV library, the traffic signs image file is loaded, and the imread() function is used to read the traffic signs image file; the sp l it() function is used to separate the traffic signs image into individual channels, so that the multi-channel image can be easily decomposed into individual channels for further processing or analysis.

[0039] In the embodiment, the image of the traffic sign to be detected in the complex scene is acquired by the electronic police system, the image is subjected to noise reduction and enhancement processing, and the multi-channel image of the traffic sign is separated into individual channels by using the function of the image processing library, wherein the red channel name represents the sign pattern in the current traffic sign image, the green channel represents the text in the current traffic sign image, and the blue channel represents the edge shape of the current traffic sign. Thus, the content of the current traffic sign image is separated into regions, so as to separately process the corresponding channel images according to different requirements, thereby improving the efficiency of image processing and highlighting the key regions of the image.

[0040] As can be seen from the above description, after the traffic sign image is separated into channels by using the function of the library, the traffic sign image is separated into a text channel, a traffic sign shape channel and a sign pattern channel, so as to subsequently perform enhancement processing on the key channels, thereby improving the accuracy of traffic sign image recognition.

[0041] Step three, determine the image sign region of the traffic sign to be detected, enhance the corresponding channel image according to the channel name of the sign region, and keep the remaining channel image unchanged; then extract the channel region image by using the color histogram method to obtain the feature region of the traffic sign image to be detected; wherein the corresponding channel image is enhanced according to the channel name of the feature region, and the specific method is as follows:

[0042] The image of the traffic sign to be detected is read by using the image processing library, and the multi-channel histogram equalization method is used to enhance the specified channel image of the traffic sign to be detected in the RGB and HIS spaces, so as to prevent the color distortion of the traffic sign to be detected due to uneven illumination.

[0043] Specifically, the RGB space directly corresponds to the three color channels of red, green and blue, and the HSI space divides the color information into three independent parts of hue, saturation and brightness; RGB color space processing: in the RGB color space, each pixel is defined by the values of the red, green and blue components; each component can be independently histogram equalized to enhance the contrast and details of the image; the pixel representation of the RGB color space uses 32 bits, of which 8 bits are used for each primary color (red, green and blue), and the remaining 8 bits can be used as an Alpha channel or ignored. HSI color space processing: in the HSI color space, histogram equalization can be performed on the three channels of hue, saturation and brightness to better process color and brightness information; the analysis and processing of the HSI color space are particularly useful for identifying and separating specific colors, which is particularly important in traffic sign color recognition. By performing histogram equalization on the specified channels of the traffic sign image in the RGB and HSI color spaces, the visual effect of the traffic sign image can be effectively enhanced, and the contrast and clarity of the traffic sign image can be improved. This method has a significant effect on improving image quality, especially in low-light or high-color recognition scenarios.

[0044] The red channel name in the current traffic sign image represents the traffic sign graphic, i.e., the horn and diagonal area, the green channel represents the traffic sign text, and the blue channel represents the traffic sign edge shape, i.e., a circle; based on the main factors of traffic sign detection, the red channel can be used as the key channel image, and based on this, a multi-channel histogram equalization method is used to enhance the red channel image in the RGB and HSI spaces of the current traffic sign image, i.e., the horn and diagonal area are enhanced, while the blue and green channel images remain unchanged, thereby highlighting the content of the red channel image and ensuring the clarity of the horn and diagonal area image.

[0045] By separating the traffic sign image into channels, establishing important channels and enhancing them, the clarity of the key areas in the traffic sign image is highlighted, thereby ensuring the accuracy of the later convolutional neural network training and the accuracy of the later automatic recognition and detection of the convolutional neural network.

[0046] After obtaining the feature region of the traffic sign image to be detected, the following steps are included:

[0047] The traffic sign image to be detected after feature processing is divided into a training set and a test set, the training set is used to establish the feature points of the convolutional neural network, and the test set is used to verify the accuracy of the convolutional neural network; after dividing the traffic sign image to be detected after feature processing into a training set and a test set, the following steps are included:

[0048] After the features of the obtained plurality of images of the traffic sign to be detected are extracted in sequence according to the above steps, the images are saved to form a sample data set; the sample data set includes traffic sign marks in different environments, under different weather conditions, with different sizes and color changes, and the different sample data sets are labeled and classified according to the content of the traffic signs; so as to enrich the feature points of the convolutional neural network and optimize the diversity and accuracy of the convolutional neural network; after the different sample data sets are labeled and classified according to the content of the traffic signs, corresponding search keywords can also be constructed according to the labeled content, so that the required sample data can be quickly searched in the later stage.

[0049] Step four, creating a convolutional neural network, importing the image of the traffic sign to be detected processed in step four into the convolutional neural network for training and testing, identifying and detecting the features of the image of the traffic sign to be detected through the convolutional neural network, specifically:

[0050] The training set of the image of the traffic sign to be detected is imported into the convolutional neural network for training, the feature region to which the training set belongs is identified through the convolution kernel, and the feature points of the convolutional neural network are determined; then the test set is imported into the convolutional neural network for testing, and the test result is output; the feature region identified by the test result is compared with the feature region extracted in advance, the accuracy of the convolutional neural network test is determined, and the optimized convolutional neural network is formed; after the optimized convolutional neural network is formed, if a new traffic sign image appears, the above steps are performed, and the convolutional neural network is continuously imported for training and testing, and the convolutional neural network is iteratively optimized.

[0051] After the image of the traffic sign to be detected is processed according to the above steps, the red channel is obtained, i.e. the horn and the diagonal region are the feature regions of the sign, the image is backed up and divided into a training set and a test set, the training set is first imported into the convolutional neural network for training, and the training result is output; then the test set is imported into the trained convolutional neural network for testing and verification, and the test result is output; then the test result is checked with the training result to confirm whether the two training results are consistent, i.e. the horn and the diagonal region are the feature regions; after obtaining the feature region, the content of the traffic sign in the image of the traffic sign to be detected is obtained according to the text content specified by the traffic management department, i.e. prohibition of sounding the horn.

[0052] Step five, the image of the plurality of traffic signs to be detected is subjected to sign recognition according to the above steps, and the recognition result and the corresponding image are saved to form a traffic sign database; specifically, the plurality of traffic sign images in the complex scene acquired by the electronic police system are subjected to feature detection and the corresponding text content is obtained according to the operation of the above embodiment; then the plurality of results and the corresponding images are saved; further, after the traffic sign database is constructed, the corresponding search keywords can also be established according to the image content or the text content of the feature region, so as to search out the same pre-stored content in the traffic sign database according to the feature content of the feature detection result in the later stage, thereby detecting the content of the traffic sign for the automatic driving.

[0053] Step six, the image of the traffic sign to be detected in the complex scene acquired in real time is processed according to the above steps, the feature of the traffic sign image to be detected is detected by the convolutional neural network, and then it is matched with the traffic sign database, and the corresponding result is output, including: the sign type, the meaning and the corresponding indication, and the traffic sign detection result is fed back to the automatic driving system through wireless transmission technology;

[0054] According to the above embodiment, for example, the automatic driving system recognizes the traffic sign during driving the car, first pre-processes the traffic sign to ensure the clarity of the traffic sign; separates the traffic sign into multiple channels, and extracts the sign image of the traffic sign as a feature region, i.e., the horn and the diagonal region; after enhancing the channel image, it is imported into the convolutional neural network for recognition and detection, and the detection result is output, i.e., the horn and the diagonal region; then the image content of the region is searched and matched in the traffic sign database, for example, the horn / diagonal word is input, the corresponding pre-stored content is obtained, the sample image with the same content is selected as the matching object, and the text analysis content of the current traffic sign is obtained according to the text analysis content corresponding to the matching object, and the content is fed back to the automatic driving system through wireless transmission technology, thereby prompting the automatic driving system to prohibit honking in the road section, thereby realizing the automatic driving complex scene traffic sign detection process based on the extraction of multiple channel images.

[0055] The beneficial effects achieved by the above content are: through the above operation, not only the safety and reliability of the automatic driving vehicle in the complex environment are improved, but also the intelligent level of the automatic driving technology is improved, so that it can better adapt to various road conditions and traffic environments, thereby providing users with a more convenient and safe travel experience.

[0056] Working principle: through the electronic police system to obtain multiple traffic signs to be detected in a complex scene Image, through the image processing library will traffic signs multi-channel image separation into individual channels, and according to the feature area to the corresponding channel image enhancement processing, and then through the color histogram method to extract the feature area of the traffic signs to be detected image, through the convolutional neural network to identify the features of the traffic signs to be detected image; According to the above operation, the images of multiple traffic signs to be detected are sequentially identified and detected, and a traffic sign database is formed; then the images of the traffic signs to be detected in the complex scene are processed according to the above operation, and the features of the traffic signs to be detected are identified through the convolutional neural network, and the traffic signs to be detected are matched with the traffic sign database, the corresponding result is obtained, and the traffic sign detection result is fed back to the automatic driving system through wireless transmission technology.

[0057] It should be noted that in this paper, relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment.

[0058] Although the embodiments of the present application have been shown and described, it can be understood by those skilled in the art that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and spirits of the present application.

Claims

1. An automatic driving complex scene traffic sign detection method based on multi-channel image extraction, characterized in that, The method comprises the following steps: Step one, by acquiring a plurality of images of traffic signs to be detected in a complex scene, an image sample of the traffic sign is formed; Step two, the traffic sign multichannel image is separated into individual channels by an image processing library, including a red channel, a green channel and a blue channel, and each channel is given a corresponding name or identifier according to the meaning of the channel; wherein the red channel represents the traffic sign pattern, the green channel represents the traffic sign text, and the blue channel represents the traffic sign edge shape; Step three, the image mark area of the traffic sign to be detected is determined, the corresponding channel image is enhanced according to the channel name of the mark area, and the remaining channel image remains unchanged; then the channel area image is extracted by the color histogram method to obtain the feature area of the traffic sign image to be detected; Step four, a convolutional neural network is created, and the traffic sign image to be detected processed in step three is imported into the convolutional neural network for training and testing, and the features of the traffic sign image to be detected are recognized and detected by the convolutional neural network; Step five, the above steps are used to identify the marks of a plurality of traffic sign images to be detected, and the identification results and corresponding images are saved to form a traffic sign database; Step six, the traffic sign image to be detected in the complex scene is processed according to the above steps, the features of the traffic sign image to be detected are recognized and detected by the convolutional neural network, and then matched with the traffic sign database, and the corresponding results are output, including: the type of the mark, the meaning and the corresponding indication, and the traffic sign detection results are fed back to the automatic driving system through wireless transmission technology.

2. The automatic driving complex scene traffic sign detection method based on multi-channel image extraction according to claim 1, characterized in that: After obtaining the plurality of traffic sign images to be detected in the complex scene in step one, the plurality of traffic sign images to be detected are transmitted to the image processing library through wireless transmission, and the traffic sign images to be detected are preprocessed. 3.The automatic driving complex scene traffic sign detection method based on multi-channel image extraction according to claim 1, characterized in that: In step three, the corresponding channel image is enhanced according to the channel name of the feature area, specifically: The image of the traffic sign to be detected is read by the image processing library, and the multichannel histogram equalization method is used to enhance the specified channel image of the traffic sign to be detected in RGB and HIS space, to prevent color distortion of the traffic sign to be detected due to uneven lighting.

4. The automatic driving complex scene traffic sign detection method based on multi-channel image extraction according to claim 1, characterized in that: After obtaining the feature area of the traffic sign image to be detected in step three, the following steps are included: The traffic sign image to be detected after feature processing is divided into a training set and a test set, the training set is used to determine the feature points of the convolutional neural network, and the test set is used to verify the accuracy of the convolutional neural network.

5. The automatic driving complex scene traffic sign detection method based on multi-channel image extraction according to claim 4, characterized in that: After dividing the traffic sign image to be detected after feature processing into a training set and a test set, the following steps are included: After the plurality of traffic sign images to be detected are sequentially extracted according to the above steps, they are saved to form a sample data set; and the sample data set includes traffic sign marks in different environments, under different weather conditions, sizes and color changes, and different sample data sets are labeled and classified according to the content of the traffic sign.

6. The automatic driving complex scene traffic sign detection method based on multi-channel image extraction according to claim 1, characterized in that: The step four is to identify the features of the image of the traffic sign to be detected by the convolutional neural network, specifically: The training set of the image of the traffic sign to be detected is introduced into the convolutional neural network for training, the feature region to which the training set belongs is identified through the convolution kernel, and the feature points of the convolutional neural network are determined; then the test set is introduced into the convolutional neural network for testing, and the test result is output; the feature region identified by the test result is compared with the feature region extracted in advance, the accuracy of the convolutional neural network test is determined, and the optimized convolutional neural network is formed.

7. The automatic driving complex scene traffic sign detection method based on multi-channel image extraction according to claim 6, characterized in that: After the optimized convolutional neural network is formed, if a new traffic sign image appears, the convolutional neural network is continuously introduced into the convolutional neural network for training and testing, and the convolutional neural network is iteratively optimized.

8. The automatic driving complex scene traffic sign detection method based on multi-channel image extraction according to claim 2, characterized in that: The image of the traffic sign to be detected is preprocessed, specifically: The noise in the image of the traffic sign to be detected is removed, and the image is enhanced, the contrast and brightness of the image are adjusted, and the size of the image is uniformly adjusted.

9. The automatic driving complex scene traffic sign detection method based on multi-channel image extraction according to claim 1, characterized in that: The images of the multiple traffic signs to be detected in the complex scene obtained in the step one are all RGB images.

Images