Machine vision-oriented traffic sign identification method and system combined with image interface

Abstract

The invention discloses a traffic sign recognition method and system facing machine vision and combined with an image interface, belongs to the technical field of computer vision and intelligent traffic systems, and solves the problem that the existing recognition technology is difficult to meet the practical requirements of real-time performance and robustness. The method comprises the following steps: S1, collecting a traffic sign image, preprocessing the traffic sign image to obtain a preprocessed image data set, and dividing the preprocessed image data set into a training set, a test set and a verification set; s2, an existing deep learning model is selected, the training set is adopted to train the model on a GPU accelerated computing platform, hyper-parameters are adjusted according to the verification set, and the trained model is obtained; and S3, carrying out traffic sign identification on the preprocessed image by adopting the trained model. The method is suitable for an automatic driving scene.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision and intelligent transportation system, and particularly relates to an automatic traffic sign recognition technology. BACKGROUND

[0002] With the development of current technology, the continuous expansion of infrastructure scale, and the rapid growth of car ownership, the demand for intelligent driving in the automobile market has emerged as the times require, which has attracted widespread attention from the society. However, at the initial stage of this technology, safety problems are prominent due to factors such as insufficient sensor accuracy or real-time performance, slow or incorrect decision-making response, and so on.

[0003] Road traffic signs, as one of the important components of China's road infrastructure, are one of the important information carriers for vehicle and road environment information interaction. According to the “2022 Road Traffic Accident Statistics Report” of the Ministry of Transport, traffic accidents caused by traffic sign recognition errors or untimely responses accounted for as high as 12.7% of the total accidents in China each year, and the proportion of accidents involving intelligent driving test vehicles has been increasing year by year, with traffic safety problems becoming increasingly prominent. In the perception-decision-execution closed-loop logical control system of the autonomous driving system, traffic sign recognition directly affects the accuracy and robustness of core decision-making links such as path planning and speed control. This once again emphasizes the importance of traffic signs for vehicle control.

[0004] In complex and variable actual road environments, traffic signs may be difficult to be effectively recognized due to changes in light, adverse weather, partial occlusion or blurring, etc., increasing the risk of traffic accidents. Traditional traffic sign recognition methods rely heavily on manually designed features, such as color, shape, and edge feature-based recognition methods, which have poor robustness and are difficult to adapt to complex environments. These methods usually require specific rules and feature extractors to be designed for different traffic sign types, resulting in complex systems and weak generalization ability, especially when vehicles are traveling at high speeds, traffic signs are difficult to be recognized in time and accurately. Therefore, a traffic sign recognition method that can automatically learn effective features, has high precision and robustness, has important practical significance and application value, especially in promoting the development of autonomous driving and intelligent transportation systems.

[0005] Deep learning-driven computer vision technology has made significant breakthroughs in recent years, with convolutional neural networks (CNN) being particularly prominent in the field of object detection. In the direction of traffic sign recognition, the Liu research group innovatively developed an improved solution based on a lightweight SSD architecture. By replacing the traditional VGG16 framework with a MobileNetV3_large network and combining multi-dimensional optimization strategies, the model parameter compression and detection accuracy-robustness indicators were simultaneously improved. However, this solution, while improving detection efficiency through model parameter compression, underperforms in complex and variable real-world scenarios. For example, in conditions of severe light changes, partial occlusion, or adverse weather (such as rain and fog), the model's robustness significantly decreases. This is because lightweight design (such as MobileNetV3_large) reduces computational burden at the expense of feature extraction depth and diversity, resulting in reduced accuracy in recognizing subtle or blurred traffic signs.

[0006] To complement this technology, the Li research group restructured the Faster-RCNN algorithm to address the small-scale sign detection challenge, focusing on improving the feature extraction mechanism of the backbone network and the region candidate network, significantly reducing computational complexity. However, this solution has a bottleneck in real-time performance. As a multi-stage detection algorithm, Faster-RCNN is slow in inference speed, making it difficult to meet the millisecond-level response requirements of intelligent driving systems.

[0007] As a common solution in existing technologies, the YOLO series of algorithms is known for its excellent real-time performance in object detection, but it has inherent limitations in the context of traffic sign recognition. YOLO lacks accuracy in detecting small-scale or densely arranged traffic signs and is less robust to image noise and light changes. SUMMARY

[0008] The present invention provides a traffic sign recognition method and system for machine vision and combined with image interface, aiming to solve the problem that existing recognition technologies cannot meet the practical needs of real-time and robustness.

[0009] In a first aspect, the present invention aims to provide a traffic sign recognition method for machine vision and combined with image interface, comprising the following steps: S1: Collect traffic sign images, preprocess the traffic sign images, obtain preprocessed image dataset, and divide it into training set, test set, and validation set; S2: Select an existing deep learning model, train the model on a GPU-accelerated computing platform using the training set, adjust hyperparameters based on the validation set, and obtain the trained model; S3: Use the trained model to recognize traffic signs in preprocessed images.

[0010] Further, a priority scheme is provided: in the S1, the traffic sign images include traffic sign images under various illumination conditions, weather conditions and angles.

[0011] Further, a priority scheme is provided: the preprocessing includes adjusting the image size to a uniform size and performing pixel value normalization, and performing data enhancement processing on the image data, the data enhancement including at least one of random rotation, scaling, translation, brightness adjustment, blur processing and adding Gaussian noise.

[0012] Further, a priority scheme is provided: the deep learning model is based on a convolutional neural network (CNN) architecture and uses a target detection algorithm for positioning and classification.

[0013] Further, a priority scheme is provided: the deep learning model uses a YOLO series algorithm as the target detection algorithm.

[0014] Further, a priority scheme is provided: the YOLO series algorithm is specifically a YOLOv5 model.

[0015] Further, a priority scheme is provided: the method further includes a model evaluation step, and the accuracy, recall rate, F1 score and mean precision (mAP) of the model are calculated using the test set.

[0016] In a second aspect, the purpose of the present application is to provide a traffic sign recognition system oriented to machine vision and combined with an image interface, which is implemented by using the traffic sign recognition method oriented to machine vision and combined with an image interface according to any one or more of the above schemes. An image processing module: for collecting traffic sign images, preprocessing the traffic sign images, obtaining preprocessed image data sets, and dividing them into training sets, test sets and validation sets; A model training module: for selecting an existing deep learning model, training the model on a GPU-accelerated computing platform using the training set, adjusting the hyperparameters according to the validation set, and obtaining the trained model; A sign recognition module: for using the trained model to recognize traffic signs from preprocessed images.

[0017] In a third aspect, the purpose of the present application is to provide a computer device, which includes a memory and a processor, and the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the traffic sign recognition method oriented to machine vision and combined with an image interface according to any one or more of the above schemes.

[0018] In a fourth aspect, the present application aims to provide a computer readable storage medium for storing a computer program for implementing the traffic sign recognition method according to any one or more of the above aspects.

[0019] Compared with the prior art, the present application has the following advantages: The method significantly improves the adaptability of the model in complex road environments. By using data augmentation techniques and combining transfer learning paradigms to optimize the pre-trained model, this method can effectively handle challenges such as changes in lighting, rain and fog weather, or partial occlusion, thereby reducing the risk of false positives and false negatives and reducing the incidence of traffic accidents. At the same time, the model is based on a convolutional neural network (CNN) and a target detection algorithm (YOLOv5) to ensure the depth and diversity of feature extraction, avoiding the robustness problem of traditional hand-designed features. The present application realizes high-speed, low-latency traffic sign recognition, meeting the demand for millisecond-level response of intelligent driving. The system uses the YOLOv5 algorithm to simultaneously complete target positioning and classification in one network, greatly improving the processing speed.

[0020] The present application provides a high-precision, high-robustness, real-time response solution for intelligent transportation systems, effectively promoting the safe iteration and widespread application of autonomous driving technology. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 A flowchart of the traffic sign recognition method according to the present application; Figure 2 A model training index diagram according to the present application; Figure 3 A system flowchart according to the present application; Figure 4 A software running instance diagram according to the present application. DETAILED DESCRIPTION

[0022] In the following description, specific details are set forth in order to provide a thorough understanding of the embodiments of the present application. However, persons skilled in the art will understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods have been omitted so as not to obscure the description of the present application with unnecessary detail.

[0023] The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0024] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be practiced without the specific details that are set forth in the description, and it is understood that persons having ordinary skill in the art can make and use other embodiments having different arrangements without departing from the scope of the present application.

[0025] Embodiment one: A traffic sign recognition method for machine vision and combined with an image interface, comprising the following steps: Step 1: Data set preparation and preprocessing: Collect and integrate domestic mainstream traffic sign data sets, larger TT100K (The Traffic Sign Recognition Dataset - 100K), smaller CCTSDB (Chinese Traffic Sign Detection Benchmark) and other public data sets. These data sets contain traffic sign images under various light, weather and angle conditions, providing rich samples for model training. The present embodiment mainly uses CCTSDB, supplemented by other data sets for image testing.

[0026] The collected images are preprocessed, including adjusting the image size to a uniform size (640x640) to adapt to the model input, performing pixel value normalization to speed up the model convergence, and applying the above-mentioned data enhancement techniques to expand the training set scale.

[0027] In order to improve the generalization ability and robustness of the model, the present embodiment has carried out various processing on the original data set. These processing include but are not limited to random rotation, scaling, translation, brightness adjustment, blurring processing and adding Gaussian noise to the image. Through the simulation of various situations that may occur in the real world, such as weather changes (rainy day, foggy day), insufficient light, partial occlusion, etc., data enhancement makes the model maintain a high recognition accuracy when facing new images that have not been seen before.

[0028] Step 2: Model construction and training: Select appropriate deep learning model architecture based on specific application scenarios and performance requirements. The deep learning model is based on a convolutional neural network (CNN) architecture and uses object detection algorithms for localization and classification. For example, for applications that require high real-time performance, select lightweight YOLOv5 or YOLOv8 models; for applications that pursue extreme accuracy, use RetinaNet or Mask R-CNN models.

[0029] As the core model, CNN achieves layer-by-layer abstraction of input images through a series of convolutional layers, pooling layers, and fully connected layers. Convolutional layers extract local features such as edges, textures, and shapes in images by applying different convolutional kernels; pooling layers compress data, reduce computational complexity, and enhance spatial invariance of the model; fully connected layers integrate extracted features to complete traffic sign classification.

[0030] To quickly and accurately locate and identify the position and category of traffic signs in complex road images, this embodiment uses the advanced YOLOv5 object detection algorithm. It can simultaneously complete target positioning and classification in one network, which is very suitable for vehicle-mounted systems. Although algorithms such as Faster R-CNN have slightly larger computational complexity, they perform better in terms of accuracy, and can be selected according to actual application requirements.

[0031] Use the prepared large-scale dataset to train the model on a GPU-accelerated computing platform. During this process, continuously monitor the training process of the model, and adjust hyperparameters (such as learning rate, batch size, etc.) based on performance on the validation set to ensure that the model can achieve optimal performance.

[0032] Step 3: Model evaluation Use an independent test set to comprehensively evaluate the trained model, calculate the accuracy, recall rate, F1 score, and average precision (mAP) of the model, and other core indicators. In particular, mAP can comprehensively evaluate the performance of the model under different categories and different confidence levels.

[0033] Step 4: Actual application deployment Deploy the optimized and compressed trained model to edge computing devices or embedded systems, such as high-performance Jetson Nano or smaller Raspberry Pi, to achieve low-power, high-efficiency local inference.

[0034] Develop corresponding software interface and application, realize from camera real-time acquisition image, call model to carry on inference, and show the function of recognition result on interface. This includes image stream processing, model inference call, result visualization module, etc. Ensure that the whole system can work smoothly and stably. Specifically: continuously capture road image stream through camera interface (such as USB or vehicle-mounted camera). Preprocess each frame of image obtained, then use the trained model to identify image data, obtain output data, and transmit to the result visualization module, display the recognition result on the graphical interface in real time, realize user interaction.

[0035] The embodiment is based on the principle of deep learning, and uses the powerful feature extraction capability of convolutional neural network (CNN) to realize automatic identification of traffic signs. By constructing a multi-level neural network model, a large number of traffic sign image data are trained to enable the model to automatically learn and extract deep features of traffic signs. Specifically: data enhancement expands the scale and diversity of the training data set by transforming the original image. This includes traditional cropping, grayscale and brightness adjustment, as well as fogging enhancement technology for simulating bad weather to improve the robustness of the model in complex environments. At the same time, model pre-training uses transfer learning paradigm, which uses a pre-trained model trained on a large data set to migrate the general features learned to the traffic sign recognition task, thereby significantly reducing the training data volume and time, improving efficiency and accuracy.

[0036] Embodiment two: A traffic sign recognition system oriented to machine vision and combined with image interface, the system is realized by using a traffic sign recognition method oriented to machine vision and combined with image interface as described in embodiment one, the system comprises: Image processing module: used for collecting traffic sign images, preprocessing traffic sign images, obtaining preprocessed image data set, and dividing into training set, test set and validation set; Model training module: used for selecting existing deep learning model, training model on GPU accelerated computing platform using training set, adjusting hyperparameters according to validation set, and obtaining trained model; Sign recognition module: used for traffic sign recognition of preprocessed image using trained model.

Claims

1. A traffic sign recognition method oriented towards machine vision and combined with a graphical interface, characterized in that, Includes the following steps: S1: Collect traffic sign images, preprocess the traffic sign images to obtain the preprocessed image dataset, and divide it into training set, test set and validation set; S2: Select an existing deep learning model, train the model on a GPU-accelerated computing platform using the training set, adjust the hyperparameters based on the validation set, and obtain the trained model. S3: Use the trained model to recognize traffic signs in the preprocessed image.

2. The traffic sign recognition method based on machine vision and combined with a graphical interface according to claim 1, characterized in that, In S1, the traffic sign images include traffic sign images under various lighting conditions, weather conditions, and angles.

3. The traffic sign recognition method based on machine vision and combined with a graphical interface according to claim 1, characterized in that, The preprocessing includes: adjusting the image size to a uniform size and normalizing the pixel values, and performing data augmentation on the image data, wherein the data augmentation includes at least one of random rotation, scaling, translation, brightness adjustment, blurring, and adding Gaussian noise.

4. The traffic sign recognition method based on machine vision and combined with an image interface according to claim 1, characterized in that, The deep learning model is based on the Convolutional Neural Network (CNN) architecture and uses object detection algorithms for localization and classification.

5. The traffic sign recognition method based on machine vision and combined with a graphical interface according to claim 1, characterized in that, The deep learning model uses the YOLO series of algorithms as the object detection algorithm.

6. A traffic sign recognition method based on machine vision and combined with a graphical interface according to claim 4, characterized in that, The YOLO series of algorithms specifically refers to the YOLOv5 model.

7. The traffic sign recognition method based on machine vision and combined with a graphical interface according to claim 1, characterized in that, The method also includes a model evaluation step, which uses a test set to calculate the model's accuracy, recall, F1 score, and mean precision (mAP) metrics.

8. A traffic sign recognition system oriented towards machine vision and combined with a graphical interface, characterized in that, The system is implemented using a traffic sign recognition method based on machine vision and combined with an image interface as described in any one of claims 1-7, and the system includes: Image processing module: used to acquire traffic sign images, preprocess the traffic sign images, obtain the preprocessed image dataset, and divide it into training set, test set, and validation set; Model training module: Used to select an existing deep learning model, train the model on a GPU-accelerated computing platform using the training set, adjust hyperparameters based on the validation set, and obtain the trained model; Traffic sign recognition module: Used to recognize traffic signs in preprocessed images using a trained model.

9. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes a traffic sign recognition method based on machine vision and combined with a graphical interface according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program that executes a traffic sign recognition method based on machine vision and combined with an image interface, as described in any one of claims 1-7.

Images