Traffic sign recognition model training system, method and device, medium and traffic sign recognition method

Abstract

The invention provides a traffic sign recognition model training method, system and device, a medium and a traffic sign recognition method.The first training stage of the traffic sign recognition model training method comprises the steps that a clear image is collected, and noise adding and noise reduction processing is conducted on the clear image; the second training stage comprises the following steps: acquiring initial features of various types of traffic signs, constructing an initial feature memory bank, calculating feature centers of the various types of traffic signs by using training sample images, and calculating restoration loss between the images after noise reduction and original clear images so as to adjust noise reduction parameters of the model. Updating the initial feature memory library to obtain template feature representation, and finally extracting sampling features of a target sample image and calculating comparison loss between the sampling features and template features of the same category to adjust feature extraction parameters of the model; according to the method, the noise reduction capability and the feature extraction capability of the traffic sign recognition model are respectively improved through the recovery loss and the comparison loss, so that the recognition accuracy and robustness of the model are enhanced.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a traffic sign recognition model training system, method, device and medium, and a traffic sign recognition method. Background Art

[0002] In the military field, clear and accurate identification of traffic signs is of irreplaceable importance for ensuring troop mobility, improving mission execution efficiency and ensuring personnel safety. Various signs, warning lines and traffic signals are widely distributed in military bases, training grounds and combat areas. They not only guide the orderly flow of vehicles and equipment, but are also directly related to the management of combat readiness and the speed of emergency response.

[0003] However, traffic sign detection in military environments faces many challenges, including complex and changeable weather conditions (such as sandstorms, rain and snow), low-light environments at night, and interference factors unique to battlefields. These factors lead to a decline in image quality and make it difficult to capture tiny traffic signs, which poses a severe test to the accuracy of traditional detection methods.

[0004] To this end, the introduction of traffic sign detection technology in intelligent transportation systems into military equipment applications, combined with the latest advances in computer vision, artificial intelligence and deep learning, has become a key path to improve the level of military traffic management. However, although the current deep learning-based target detection methods have made some progress on standard data sets, they still face the problem of balancing the coordinated optimization of image enhancement and target detection, and have not yet achieved an effective combination of the two. In addition, these methods are insufficient in using potential information to improve detection accuracy. Summary of the invention

[0005] In view of the above-mentioned shortcomings of the prior art, the present invention provides a traffic sign recognition model training system, method, device and medium, and a traffic sign recognition method to solve the above-mentioned technical problems.

[0006] The present invention provides a traffic sign recognition model training system, the system comprising: a data preprocessing module, used to collect clear images, and add noise to the clear images to obtain simulated training images, so as to obtain training sample images based on the simulated training images, wherein the clear images contain clear traffic sign information; a global area perception module, used to evaluate the recognition difficulty of each category of traffic signs, dynamically sample the training sample images based on the evaluation results of the recognition difficulty to obtain target training images, and perform feature recognition on each target pre-training image to obtain the feature center of each category of traffic signs; a memory library construction module, used to construct an initial feature memory library according to the acquired initial features of each category of traffic signs, and update the initial feature memory library based on the feature center obtained by the global area perception module to obtain the template feature representation of each category of traffic signs based on the updated feature memory library; a comparative learning module, used to extract the sampling feature representation of each target training image, calculate the feature similarity between the sampling feature representation and the template feature representation of each category of traffic signs, and optimize the feature extraction parameters of the traffic sign recognition model to be trained based on the feature similarity.

[0007] In one embodiment of the present application, the traffic sign recognition model training system also includes: an image denoising module, which is used to perform denoising on the simulated training image to obtain an initial training image for model training; a restoration loss calculation module, which is used to compare the clear image and the initial training image to obtain an image restoration loss; a denoising parameter adjustment module, which is used to optimize the image denoising module based on the image restoration loss to obtain a pre-training image based on the optimized image denoising module; and a data enhancement module, which is used to perform image conversion on the pre-training image to obtain training sample images of multiple scales and directions.

[0008] The present application provides a method for training a traffic sign recognition model, the method comprising a first training stage and a second training stage, wherein the first training stage comprises: collecting a clear image, adding noise to the collected clear image, and performing denoising on the image after adding noise, so as to obtain a restoration loss between the denoised image and the clear image, and adjusting the denoising parameters of the traffic sign recognition model to be trained based on the restoration loss; the second training stage comprises: obtaining initial features of each category of traffic signs and constructing an initial feature memory library based on the initial features; obtaining feature centers of each category of traffic signs based on training sample images, and updating the initial feature memory library based on the feature centers to obtain template feature representations of each category of traffic signs; extracting a sampling feature representation of a target sample image, so as to obtain a contrast loss between the sampling feature representation and the template feature representation of the same category of traffic signs, and adjusting the feature extraction parameters of the traffic sign recognition model to be trained based on the contrast loss.

[0009] In one embodiment of the present application, after adjusting the noise reduction parameters of the traffic sign recognition model to be trained based on the restoration loss, it also includes: obtaining a pre-training image based on the traffic sign recognition model after the noise reduction parameters are adjusted; performing image conversion on the pre-training image to obtain training sample images of multiple scales and directions, and the image conversion includes at least translation, rotation, and scaling.

[0010] In one embodiment of the present application, the feature center of each category of traffic signs is obtained based on the training sample images, including: evaluating the recognition difficulty of each category of traffic signs according to a preset difficulty recognition standard; dynamically sampling the training sample images based on the evaluation results of the recognition difficulty, and determining the sampled images as target training images; extracting feature information of the target training images, and classifying the extracted feature information based on the meaning of the traffic signs in the target training images to obtain the feature center of each category of traffic signs.

[0011] In one embodiment of the present application, the contrast loss between the sampled feature representation and the template feature representation of the same category of traffic signs is obtained, including: extracting the sampled feature representation of any target training image, and obtaining multiple template feature representations based on the updated feature memory library; respectively calculating the similarity between the sampled feature representation and the multiple template feature representations, and constructing the contrast loss based on the multiple similarity values, wherein the contrast loss is used to increase the similarity between the sampled feature representation and the template feature representation of the same category of traffic signs, and to reduce the similarity between the sampled feature representation and the template feature representation of different categories of traffic signs.

[0012] In one embodiment of the present application, the traffic sign recognition model training method also includes: collecting an environmental image containing traffic sign information as a test image, and generating a test sample data set based on the test image; testing the trained traffic sign recognition model according to the test sample data set to obtain recognition parameters of the trained traffic sign recognition model to verify the traffic sign recognition ability of the traffic sign recognition model, and the recognition parameters include at least recognition accuracy and recall rate.

[0013] The present application provides a traffic sign recognition method, which is implemented based on a traffic sign recognition model obtained by the above-mentioned traffic sign recognition model training method. The method includes obtaining an image to be recognized, where the image to be recognized is an environmental image of the environment in which the target vehicle is currently located, and the environmental image includes traffic sign information; inputting the image to be recognized into the traffic sign recognition model to obtain output information of the traffic sign recognition model, and using the output information as a traffic sign recognition result of the image to be recognized.

[0014] The present application provides an electronic device, characterized in that it includes a processor, a memory and a communication bus; the communication bus is used to connect the processor and the memory; the processor is used to execute a computer program stored in the memory to implement the traffic sign recognition model training method as described above.

[0015] The present application provides a computer-readable storage medium, characterized in that a computer program is stored thereon, and the computer program is used to enable a computer to execute the traffic sign recognition model training method as described above.

[0016] Beneficial effects of the present invention: In the first training stage, the traffic sign recognition model training method proposed in the present application adds noise and performs noise reduction processing on the clear image, and calculates the restoration loss between the noise-reduced image and the original image, so as to effectively adjust the noise reduction parameters of the model and enhance the anti-interference ability of the model in a complex environment. Considering that the military environment is often accompanied by harsh climatic conditions and interference factors unique to the battlefield, which leads to a serious decline in image quality, by strengthening the noise reduction ability of the model, it can be ensured that traffic signs can still be accurately identified under harsh conditions, and the mobility of the troops and the efficiency of task execution can be improved. In the second training stage, by obtaining the initial features of each category of traffic signs, an initial feature memory library is constructed, and the feature center is calculated using the training sample image to update the memory library, a template feature representation is obtained, and potential information is fully utilized, and the feature extraction ability of the model is improved. In view of the fact that the accurate identification of traffic signs in the military field is directly related to the management of combat readiness and the speed of emergency response, therefore, by optimizing feature extraction, the traffic sign recognition model can more accurately identify traffic signs, providing strong support for military decision-making. To sum up, the traffic sign recognition model training method proposed in this application improves the noise reduction and feature extraction capabilities of the traffic sign recognition model through restoration loss and contrast loss respectively, thereby enhancing the recognition accuracy and robustness of the model. This method has significant advantages in the application of military equipment, and can improve the automation and intelligence level of military traffic management, and provide solid technical support for building a "smart barracks" and ensuring efficient combat of troops.

[0017] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0019] Figure 1 is a schematic diagram of an implementation environment of a traffic sign recognition model training method shown in an exemplary embodiment of the present application;

[0020] Figure 2 is a flow chart of a traffic sign recognition model training method shown in an exemplary embodiment of the present application;

[0021] Figure 3 is a schematic diagram of an adaptive filter flow in a traffic sign recognition model training method shown in an exemplary embodiment of the present application;

[0022] Figure 4 It is a schematic diagram of a data enhancement process in a traffic sign recognition model training method shown in an exemplary embodiment of the present application;

[0023] Figure 5 is a schematic diagram of a process of global area perception of traffic signs shown in an exemplary embodiment of the present application;

[0024] Figure 6 It is a schematic diagram of the whole process flow of a traffic sign recognition model training method shown in an exemplary embodiment of the present application;

[0025] Figure 7 is a block diagram of a traffic sign recognition model training system shown in an exemplary embodiment of the present application;

[0026] Figure 8 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0027] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0028] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0029] In the following description, numerous details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0030] Figure 1 It is a schematic diagram of the implementation environment of the traffic sign recognition model training method shown in an exemplary embodiment of the present application.

[0031] like Figure 1 As shown, the implementation environment of the traffic sign recognition model training method includes a data acquisition device 101 and a computer device 102, wherein the data acquisition device 101 is used to collect images for model training and the initial parameters of the model, and send the collected images and initial parameters to the computer device 102, and the data acquisition device can be any device or equipment that can be used for image acquisition, such as a camera, a camera, an infrared sensor, etc., and this application does not impose any restrictions on this; the computer device 102 is used to build an initial traffic sign recognition model based on the initial parameters collected by the data acquisition device 101, and train and optimize the initial traffic sign recognition model based on the images used for training, so as to realize the training of the traffic sign recognition model. Among them, the computer device 102 can be at least one of a desktop graphics processing unit (Graphic Processing Unit, GPU) computer, a GPU computing cluster, a neural network computer, etc., or it can be an intelligent processor integrated in the current vehicle, and this application does not impose any restrictions on this.

[0032] Figure 2 It is a flowchart of a traffic sign recognition model training method shown in an exemplary embodiment of the present application.

[0033] In one embodiment of the present application, the training stage of the traffic sign recognition model training method is divided into a first stage and a second stage, wherein the first training stage includes acquiring a clear image, adding noise to the acquired clear image and performing denoising on the image after adding noise to obtain a restoration loss between the denoised image and the clear image, and adjusting the denoising parameters of the traffic sign recognition model to be trained based on the restoration loss; the second training stage includes acquiring initial features of each category of traffic signs and constructing an initial feature memory library based on the initial features; obtaining feature centers of each category of traffic signs based on training sample images, and updating the initial feature memory library based on the feature centers to obtain template feature representations of each category of traffic signs; extracting a sampling feature representation of the target sample image to obtain a contrast loss between the sampling feature representation and the template feature representation of the same category of traffic signs, and adjusting the feature extraction parameters of the traffic sign recognition model to be trained based on the contrast loss.

[0034] In another embodiment of the present application, the above-mentioned traffic sign recognition model training method is further subdivided into the following specific steps: Figure 2 As shown, the traffic sign recognition model training method includes at least steps S210 to S250, which are described in detail as follows:

[0035] Step S210, collecting a clear image, adding noise to the collected clear image, and performing noise reduction processing on the image after adding noise.

[0036] Considering that the traffic sign recognition model ultimately needs to identify traffic sign information in harsh or extreme environments, it is usually necessary to use traffic equipment to move and shoot roadside images to obtain images that require road traffic sign detection. However, due to the influence of bad weather such as rain, fog, and dust, it is extremely difficult to obtain images in real climates, let alone images in extreme environments. Therefore, in order for the model to further process weather conditions in different complex climate scenarios and simulate the complex environment in real scenes, it is usually based on a simulated noise algorithm to artificially add noise to ordinary images to create environmental images under harsh weather conditions. The simulated noise algorithm involves adjusting the scattering coefficient and haze coefficient to randomly add different degrees of noise to clear images.

[0037] In one embodiment of the present application, first, a clear image is collected, which contains various traffic signs and different background environments; then, in order to simulate the image degradation that may be encountered in the real environment, such as lighting changes, weather effects, equipment noise, etc., the collected clear image is denoised; then, the image with added noise is denoised using a deep learning algorithm, and the denoised image is made as close to the original clear image as possible through continuous iterative optimization; finally, the restoration loss between the denoised image and the original clear image is calculated, so that the denoising parameters of the model are adjusted by calculating the restoration loss, and the denoising ability of the model is further optimized. It should be noted that, by adding different types of noise to the image, the robustness of the model can be enhanced, so that it can better cope with complex and changeable conditions in practical applications. The purpose of denoising is to restore the clarity of the image and reduce the impact of noise on the image quality; in addition, the restoration loss is an important indicator for measuring the denoising effect, which reflects the performance of the model in the denoising process. By calculating the restoration loss, the denoising parameters of the model can be adjusted to further optimize the denoising ability of the model. In the specific implementation process, indicators including but not limited to mean square error (MSE) or structural similarity (SSIM) can be used to measure the restoration loss, and the weight of the model can be updated according to the size of the loss value to improve the denoising performance of the model.

[0038] In order to reduce the impact of noise on detection accuracy, these noisy images are input into the adaptive defogger, sharpener, and color enhancement filter set module. This module includes a defog filter, a white balance filter, a contrast filter, a hue filter, and a sharpening filter. By adjusting the parameters of these filters through a convolutional neural network module that predicts the specific parameters of the filters, noise can be effectively reduced and the quality of the image can be improved.

[0039] Figure 3 It is a schematic diagram of an adaptive filter flow in a traffic sign recognition model training method shown in an exemplary embodiment of the present application.

[0040] like Figure 3 As shown, after obtaining the blurred image after adding noise, the blurred image is input into the prediction filter parameter convolutional neural network to obtain the parameters of the filter, and the obtained filter parameters and the aforementioned blurred image are input into the filter set together, so as to process the aforementioned blurred image based on the filter set to obtain the denoised image.

[0041] In a specific embodiment of the present application, a traffic sign image is acquired, and noise simulation of a complex climate and severe weather environment is implemented, and the noise is processed. The specific implementation steps are as follows:

[0042] Firstly, the roadside images are taken by using traffic equipment to obtain the images for noise simulation.

[0043] Then, in order to further simulate the severe weather under complex climate, a simulated noise algorithm is further designed to randomly add different degrees of noise to the clear image by adjusting the scattering coefficient, thereby obtaining an image containing complex environmental noise. The specific implementation formula of the simulated noise algorithm is as follows:

[0044]

[0045] I(x)=kJ(x)t0+A(1-kt0(x)) Formula (2)

[0046] Among them, β0 is the initial atmospheric scattering coefficient; t0(x) is the initial transmittance, t(x) is the transmittance after adding noise; α is the haze coefficient, and the haze concentration is controlled by randomly setting the haze coefficient. I(x) is the foggy image; J(x) is the fog-free image; A is the atmospheric light.

[0047] Finally, in order to avoid the influence of complex environmental noise on the detection accuracy, the noisy image is input into the adaptive defogger, sharpener, and color enhancement filter set modules, including defog filter, white balance filter, contrast filter, hue filter, and sharpening filter. The calculation formula of the defog filter is as follows:

[0048]

[0049] Among them, I c (y) represents XXX, y represents XXX, A c Denotes XXX, and ω is a hyperparameter used to tune the dehazing filter.

[0050] The formulas for white balance filter, contrast filter, and tint filter are as follows:

[0051] Lum(P i )=0.27r i +0.67g i +0.06b i Formula (4)

[0052]

[0053] Among them, P i =(r i ,g i ,b i ) is the input pixel value, (r, g, b) indicates different channel values, Lum(P i ) indicates XXX, EnLum(P i ) indicates XXX, En(P i ) represents XXX, P0 represents XXX, T Lrepresents XXX, L represents XXX, j represents XXX, and clip() represents XXX.

[0054] The sharpening filter formula is as follows:

[0055] F(x,λ)=I(x)+λ(I(x)-Gau(I(x))) Formula (8)

[0056] Here, F represents the sharpening process, I(x) is the input image, Gau(I(x)) represents the Gaussian filter, and λ is a positive scaling factor.

[0057] Step S220: obtaining the restoration loss between the denoised image and the clear image, and adjusting the denoising parameters of the traffic sign recognition model to be trained based on the restoration loss.

[0058] After obtaining the denoised image, the original image (its clear image) and the denoised image are usually compared and calculated to obtain the restoration loss. By comparing the difference between the two, we can understand the degree of influence of the simulated noise and denoising processing on the image. Finally, according to the restoration loss, the parameters of the filter predictor are adjusted, and the parameters of the detection model are adjusted so that the model can better adapt to the task of recognizing small traffic signs in complex environments.

[0059] In one embodiment of the present application, after completing the acquisition of clear images and performing denoising on these images to simulate the image denoising situation that may be encountered in a real environment, a deep learning algorithm is used to perform denoising on the noisy images; then, the restoration loss between the denoised image and the original clear image is calculated, and a back propagation algorithm is used to take the restoration loss as the optimization target, and the weights and bias parameters of the model are updated through methods such as gradient descent to reduce the restoration loss, thereby adjusting the denoising parameters of the traffic sign recognition model to be trained; finally, through continuous iterative training, the denoising ability of the traffic sign recognition model to be trained is gradually improved, and its corresponding restoration loss will gradually decrease. When the restoration loss reaches a lower level, it can be considered that the denoising performance of the model is already good, and the next stage of training can be entered. It should be noted that the purpose of denoising an image is to restore the clarity of the image and make it as close to the original clear image as possible; in addition, during the implementation process, indicators such as mean square error (MSE) or structural similarity (SSIM) can be used to measure the restoration loss. The mean square error calculates the average of the sum of square differences in pixel values ​​between the denoised image and the original image, while the structural similarity focuses more on the degree of retention of the structural information of the image.

[0060] In a specific embodiment of the present application, the original image and the denoised image are compared and calculated to obtain the restoration loss, and the calculation formula is as follows:

[0061]

[0062] Among them, re_loss represents XXX, I ij Indicates XXX, C ij Indicates XXX.

[0063] Finally, the filter predictor parameters are adjusted according to the restoration loss, and the detection model parameters are restored.

[0064] It should be noted that the adjustment of the filter predictor parameters according to the restoration loss mentioned in the above embodiment can be implemented based on conventional parameter conditions, and the present application does not impose any restrictions on the specific parameter adjustment method. In addition, the present embodiment improves the noise reduction capability of the model in the first training stage by calculating the restoration loss between the denoised image and the original clear image and adjusting the noise reduction parameters of the traffic sign recognition model based on this loss.

[0065] Figure 4 It is a schematic diagram of the data enhancement process in the traffic sign recognition model training method shown in an exemplary embodiment of the present application.

[0066] like Figure 4 As shown, after obtaining a pre-training image, the pre-training image is first scaled to obtain more pre-processed images; then part or all of the pre-processed images obtained above can be mirrored or rotated to further obtain more pre-training images; finally, all the pre-training images obtained above are converted into standard format images, and the standard format images are filled to achieve size diversity of all graphics used for model training, so as to facilitate subsequent model training.

[0067] In one embodiment of the present application, after adjusting the denoising parameters of the traffic sign recognition model to be trained based on the restoration loss, it also includes: obtaining a pre-training image based on the traffic sign recognition model after the denoising parameters are adjusted; performing image conversion on the pre-training image to obtain training sample images of multiple scales and directions, and the image conversion includes at least translation, rotation, and scaling.

[0068] In a specific embodiment of the present application, data enhancement in terms of scale, quantity, and orientation is performed on the high-quality image that has passed through the filter set, thereby increasing the diversity of the data, specifically including the following steps:

[0069] First, the high-quality image blocks are scaled in a range of 0.4 to 1.5, thereby obtaining images to be detected with different scales.

[0070] Secondly, the images of different scales are randomly flipped horizontally or vertically with a probability of 50%, thereby obtaining randomly flipped images of different scales.

[0071] Then, the images flipped at different scales are normalized to scale the pixel values ​​of the images to a specific range, so that the model training or reasoning can be performed better later. The normalization formula is as follows:

[0072]

[0073] Among them, μ represents the image mean, X represents the image matrix, σ represents the standard deviation, and N represents the number of pixels in image X.

[0074] Finally, the normalized image can be further padded so that the enhanced image size can be divided by 32 for subsequent calculations.

[0075] Step S230, obtaining initial features of each type of traffic sign and constructing an initial feature memory library based on the initial features.

[0076] In one embodiment of the present application, a convolutional neural network (CNN) in a deep learning algorithm is used to extract features from an image. After the initial features of each category of traffic signs are extracted, an initial feature memory is constructed based on these features. The memory is a data structure that stores and manages feature representations. It can be a simple database table or a more complex feature vector set. A feature vector or feature set is maintained for each category of traffic signs in the memory for subsequent feature matching and recognition tasks. Therefore, after the initial feature memory is constructed, it can be used to recognize traffic signs, which is specifically manifested as follows: in the recognition stage, for the input image to be recognized, its feature representation can be extracted and matched with the features in the memory, so that by comparing the similarity or distance between the features, we can determine the category of traffic signs to which the input image belongs.

[0077] Step S240, obtaining the feature center of each category of traffic signs based on the training sample image, and updating the initial feature memory library based on the feature center to obtain the template feature representation of each category of traffic signs.

[0078] In one embodiment of the present application, feature centers of traffic signs of various categories are obtained based on training sample images, including: evaluating the recognition difficulty of each type of traffic sign according to a preset difficulty recognition standard; dynamically sampling the training sample images based on the evaluation results of the recognition difficulty, and determining the sampled images as target training images; extracting feature information of the target training images, and classifying the extracted feature information based on the meaning of the traffic signs in the target training images to obtain feature centers of traffic signs of various categories.

[0079] In one embodiment of the present application, the specific steps of generating the feature center of each type of traffic sign are as follows:

[0080] First, define a set of traffic sign recognition difficulty assessment criteria. This can be achieved by considering a variety of environmental factors, including but not limited to:

[0081] The complexity of the logo (e.g., complexity of color, shape, pattern);

[0082] The commonness of the sign (some signs are more common on roads and therefore may be easier to spot);

[0083] The impact of environmental factors (such as lighting conditions and occlusion) on sign recognition;

[0084] Based on these factors, each traffic sign is assigned a difficulty level, such as 1 (easiest) to 5 (hardest).

[0085] Then, the training sample images are dynamically sampled according to the difficulty levels determined above. The goal is to ensure that there are enough representative samples of traffic signs at each difficulty level. Specifically, for signs with lower difficulty levels, the number of samples can be reduced because they are relatively easy to recognize; for signs with higher difficulty levels, the number of samples is increased to ensure that the model can fully learn these more complex patterns. After sampling is completed, the resulting image set is determined as the target training images.

[0086] Then, computer vision techniques (such as deep learning models) are used to extract feature information of the target training image. This can include features such as color, shape, texture, etc., and the extracted feature information is classified according to the meaning of the traffic sign (such as stop, yield, speed limit, etc.). The specific implementation of the classification can be achieved by training a classifier that can distinguish different categories of traffic signs.

[0087] Finally, for each category of traffic signs, its feature center is calculated by calculating the average or median of all image features in that category. The feature center is a compact representation that can summarize the main features of the traffic signs in that category.

[0088] It should be noted that the above-mentioned method of calculating the characteristic center by the mean value or median is a specific embodiment of the present invention. In the actual implementation process, the characteristic center can be calculated based on any feasible method, and this application does not impose any restrictions on it.

[0089] Step S250, extracting the sampling feature representation of the target sample image to obtain the contrast loss between the sampling feature representation and the template feature representation of the same type of traffic sign, and adjusting the feature extraction parameters of the traffic sign recognition model to be trained based on the contrast loss.

[0090] In one embodiment of the present application, a contrast loss between a sampled feature representation and a template feature representation of a traffic sign of the same category is obtained, including: extracting a sampled feature representation of any target training image, and obtaining multiple template feature representations based on an updated feature memory library; respectively calculating the similarities between the sampled feature representation and the multiple template feature representations, and constructing a contrast loss based on the multiple similarity values, wherein the contrast loss is used to increase the similarity between the sampled feature representation and the template feature representation of the traffic sign of the same category, and to reduce the similarity between the sampled feature representation and the template feature representation of the traffic sign of a different category.

[0091] In one embodiment of the present application, first, for any target training image, a pre-trained deep learning model (such as a convolutional neural network) is used to extract its sampled feature representation, and for each category, multiple template feature representations of the category are retrieved from the feature memory library; then, the similarity between the sampled feature representation and the multiple template feature representations is calculated by calculating the cosine similarity, Euclidean distance or other similarity metrics between the two feature vectors; then, based on the obtained multiple similarity values, we construct a contrast loss. It should be noted that the design goal of the contrast loss is to increase the similarity between the sampled feature representation and the template feature representation of the same category of traffic signs, and to reduce the similarity between the sampled feature representation and the template feature representation of different categories of traffic signs. Specifically, for the template feature representation of the same category, we hope that its similarity value is as large as possible, so we can use a term similar to maximizing similarity in the loss function. For the template feature representation of different categories, we hope that its similarity value is as small as possible, so we can use a term similar to minimizing similarity in the loss function. Finally, the parameters of the deep learning model are optimized by minimizing the contrast loss, so that the extracted feature representation is more discriminative.

[0092] Figure 5 It is a schematic diagram of the process of global area perception of traffic signs shown in an exemplary embodiment of the present application.

[0093] like Figure 5As shown, first, feature extraction is performed on the sample image to obtain its sampling features, and feature extraction is performed on the training image at the same time, and various regional centers are calculated based on different traffic sign categories to obtain multiple feature centers. The sampling features obtained above are compared with each feature center, and their similarities are calculated. The cross entropy loss function is obtained based on the feature center of the traffic sign information of the same category of the sampling feature, and global perception contrast learning is performed based on multiple similarities of different categories to train the model, and finally the final model output image is obtained.

[0094] In a specific embodiment of the present application, first, we implement a dynamic sampling method, which aims to allocate more attention to difficult samples, where difficult samples refer to those samples that are difficult for the model to correctly recognize in the initial training stage, usually with complex backgrounds, lighting conditions or occlusions, so that the traffic sign recognition model to be trained can pay more attention to difficult samples during the training process, thereby learning more challenging features of these samples; it specifically includes, in each training iteration, dynamically selecting a batch of difficult samples according to the current recognition performance of the model, and adding them to the training batch; at the same time, maintaining a certain number of simple samples to ensure the stability of the model. Secondly, for each category of traffic signs, we generate its regional center in the same image, so that the training traffic sign recognition model can learn the feature representation of different regions in the same image, even if they belong to the same category; it specifically includes, for each training image, using the pre-trained detection model to locate all instances of traffic signs, and then, for each category of instances, calculating the average value of its feature representation as the regional center of the category. Then, in order to learn the feature differences between different categories, a memory bank is built for each category so that the model can access and learn the feature differences between different categories at any time during the training process; specifically, for each category, the feature representations of all instances that appear in the training set are stored in the corresponding memory bank. The memory bank is updated regularly to include the latest and more representative features. Then, region-aware contrastive learning is performed to further enhance the model's ability to learn features of different regions, so that the model can learn more discriminative feature representations, thereby improving its performance in the task of identifying tiny traffic signs in complex environments; specifically, for each traffic sign instance in each training image, its feature representation is compared with the regional center of the same category, and a contrast loss is constructed. At the same time, it is also compared with the regional center of different categories to further increase the difference between classes. Finally, the deep learning model is trained using the training data and loss function constructed by the above method, so that through continuous iterative training, the model can gradually learn more discriminative feature representations and improve the recognition ability of difficult samples; specifically, in each training iteration, the dynamic sampling method is used to select training samples and calculate the region-aware contrast loss, and then the back-propagation algorithm is used to update the parameters of the model.

[0095] It should be noted that the beneficial effects of the traffic sign recognition model training method proposed in the Shanghai Su embodiment are as follows: First, through the dynamic sampling method, the model can pay more attention to and learn the features of difficult samples during the training process, thereby improving the recognition accuracy of these challenging samples; second, by generating regional centers for the same category of traffic signs in the same image, the model can learn the feature representations of different regions in the same image, improving the model's perception of subtle differences. Third, building a memory library of different categories enables the model to access and learn the feature differences of different categories at any time during the training process, which helps the model to form a clearer and more discriminative category feature representation. Fourth, through regional perception contrast learning, the model can learn more discriminative feature representations, so that the model can still maintain a high recognition accuracy when facing complex environments or tiny traffic signs, and improve the generalization ability and robustness of the model. In summary, the traffic sign recognition model training method proposed in this embodiment significantly improves the performance and accuracy of the traffic sign recognition model in complex environments, and provides strong support for applications in fields such as real-time traffic monitoring and autonomous driving systems.

[0096] In one embodiment of the present application, the traffic sign recognition model training method proposed in the present application also includes: collecting an environmental image containing traffic sign information as a test image, and generating a test sample data set based on the test image; testing the trained traffic sign recognition model according to the test sample data set to obtain recognition parameters of the trained traffic sign recognition model to verify the traffic sign recognition ability of the traffic sign, and the recognition parameters include at least recognition accuracy and recall rate.

[0097] In a specific embodiment of the present application, firstly, environmental images containing traffic sign information are collected as test images, and it is ensured that these images cover all traffic sign categories that the model needs to recognize. Secondly, the test images are preprocessed and annotated to generate a test sample data set, and the trained traffic sign recognition model is tested using the test sample data set, and the recognition results of the model for each test sample are recorded. Then, based on the test results, the recognition parameters of the trained traffic sign recognition model are calculated, including recognition accuracy and recall rate, etc., to verify the model's recognition ability for traffic signs. Finally, the recognition parameters obtained from the test are analyzed to evaluate the recognition performance and accuracy of the model in a complex environment, and the model is optimized and adjusted based on the test results to improve its performance and stability in practical applications. The optimized model can be applied to fields such as real-time traffic monitoring and autonomous driving systems to improve the accuracy and robustness of traffic sign recognition.

[0098] Figure 6It is a schematic diagram of the entire process of a traffic sign recognition model training method shown in an exemplary embodiment of the present application.

[0099] In one embodiment of the present application, Figure 6 As shown in the figure, firstly, a clear image is collected, and the image to be detected containing traffic signs on the road is taken by a traffic mobile device; secondly, environmental noise simulating bad weather is added to the clear image, and the image with added noise is adaptively denoised to obtain the restoration loss; then the denoised high-quality image is subjected to data enhancement operation; then, the center of the same type of region is calculated for global contrast learning to obtain the contrast loss; finally, the restoration loss and contrast loss are integrated to adjust the model parameters, and the image to be identified is processed based on the adjusted model to obtain its traffic sign information.

[0100] In a specific embodiment of the present application, the traffic sign recognition model training method specifically includes the following steps:

[0101] Step 6.1.1: Use a high-definition camera or a traffic mobile device (such as a car camera) to capture images containing traffic signs in different road scenes as a clear image dataset.

[0102] Step 6.1.2: Preprocess the acquired images, including cropping, scaling, and normalization, to ensure uniform image size and format.

[0103] Step 6.2.1: Add environmental noise simulating bad weather (such as rain, snow, fog, etc.) to the clear image to generate a noise-affected image dataset.

[0104] Step 6.2.2: Use an adaptive denoising algorithm (such as a deep learning denoising network) to denoise the noisy image to obtain a high-quality denoised image.

[0105] Step 6.2.3: Calculate the image difference before and after denoising as the restoration loss.

[0106] Step 6.3.1: Perform data augmentation operations on the denoised high-quality images, including rotation, flipping, color transformation, etc., to increase the diversity and generalization ability of the data.

[0107] Step 6.4.1: Divide the enhanced image into traffic sign categories and calculate the global contrast loss of the center of the same category area.

[0108] Step 6.4.2: Through contrastive learning, the model can better distinguish different categories of traffic signs.

[0109] Step 6.5.1: Integrate the restoration loss and contrast loss to construct the total loss function.

[0110] Step 6.5.2: Train the traffic sign recognition model using a deep learning framework such as TensorFlow or PyTorch and adjust the model parameters through the back-propagation algorithm to minimize the total loss.

[0111] Step 6.5.3: Verify and test the trained model to ensure its recognition accuracy under different severe weather conditions.

[0112] Step 6.6.1: Input the traffic image to be identified into the trained model.

[0113] Step 6.6.2: The model outputs the traffic sign information of the image, including the sign type, location, and confidence level.

[0114] Finally, the trained traffic sign recognition model is deployed to the actual traffic monitoring system to achieve real-time recognition and monitoring of road traffic signs.

[0115] Figure 7 is a block diagram of a traffic sign recognition model training system shown in an exemplary embodiment of the present application. The device can be applied to Figure 1 The device may also be applicable to other exemplary implementation environments and specifically configured in other devices, and this embodiment does not limit the implementation environment to which the device is applicable.

[0116] like Figure 7 As shown, the exemplary traffic sign recognition model training system includes: a data preprocessing module 710, an image denoising module 720, a restoration loss calculation module 730, a model parameter adjustment module 740, a data enhancement module 750, a global area perception module 760, a memory library construction module 770, and a comparative learning module 780.

[0117] Among them, the data preprocessing module 710 is used to collect clear images and add noise to the clear images to obtain simulated training images, so as to obtain training sample images based on the simulated training images, and the clear images contain clear traffic sign information; the image denoising module 720 is used to perform denoising on the simulated training images to obtain initial training images for model training; the restoration loss calculation module 730 is used to compare the clear images and the initial training images to obtain image restoration losses; the model parameter adjustment module 740 is used to optimize the image denoising module based on the image restoration losses to obtain pre-training images based on the optimized image denoising module; the data enhancement module 750 is used to perform image conversion on the pre-training images to obtain training sample images of various scales and directions; the global area perception module 760 is used to evaluate the global area perception module 760. Estimate the recognition difficulty of each category of traffic signs, dynamically sample the training sample images based on the evaluation results of the recognition difficulty to obtain target training images, and perform feature recognition on each target pre-training image to obtain the feature center of each category of traffic signs; a memory library construction module 770 is used to construct an initial feature memory library based on the acquired initial features of each category of traffic signs, and update the initial feature memory library based on the feature center obtained by the global area perception module to obtain the template feature representation of each category of traffic signs based on the updated feature memory library; a comparative learning module 680 is used to extract the sampling feature representation of each target training image, calculate the feature similarity between the sampling feature representation and the template feature representation of each category of traffic signs, and optimize the feature extraction parameters of the traffic sign recognition model to be trained based on the feature similarity.

[0118] It should be noted that the traffic sign recognition model training system provided in the above embodiment and the traffic sign recognition model training method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs operations has been described in detail in the method embodiment and will not be repeated here. In practical applications, the traffic sign recognition model training system provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0119] The present application also provides a traffic sign recognition method, which includes obtaining an image to be recognized, where the image to be recognized is an environmental image of the current environment of the target vehicle, and the environmental image includes traffic sign information; inputting the image to be recognized into a traffic sign recognition model to obtain output information of the traffic sign recognition model, and using the output information as a traffic sign recognition result of the image to be recognized.

[0120] An embodiment of the present application also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device implements the traffic sign recognition model training method provided in the above-mentioned embodiments.

[0121] Figure 8 The structure diagram of the computer system suitable for implementing the electronic device of the embodiment of the present application is shown. It should be noted that: Figure 8 The computer system 800 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0122] like Figure 8 As shown, the computer system 800 includes a central processing unit (CPU) 801, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 802 or the program loaded from the storage part 808 to the random access memory (RAM) 803, such as executing the method described in the above embodiment. In the RAM 803, various programs and data required for system operation are also stored. The CPU 801, ROM 802 and RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0123] The following components are connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed so that a computer program read therefrom is installed into the storage section 808 as needed.

[0124] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 809, and / or installed from a removable medium 811. When the computer program is executed by a central processing unit (CPU) 801, various functions defined in the system of the present application are executed.

[0125] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. This propagated data signal can take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. A computer program contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0126] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0127] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0128] Another aspect of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor of a computer, the computer executes the traffic sign recognition model training method as described above. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device.

[0129] Another aspect of the present application also provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the traffic sign recognition model training method provided in each of the above embodiments.

[0130] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A traffic sign recognition model training system, characterized in that: The system comprises: A data preprocessing module, used for collecting a clear image, adding noise to the clear image to obtain a simulated training image, and obtaining a training sample image based on the simulated training image, wherein the clear image contains clear traffic sign information; The global area perception module is used to evaluate the recognition difficulty of each category of traffic signs, dynamically sample the training sample images based on the evaluation results of the recognition difficulty to obtain the target training images, and perform feature recognition on each target pre-training image to obtain the feature center of each category of traffic signs; A memory library construction module is used to construct an initial feature memory library according to the acquired initial features of each category of traffic signs, and update the initial feature memory library based on the feature center obtained by the global area perception module, so as to obtain a template feature representation of each category of traffic signs based on the updated feature memory library; The contrast learning module is used to extract the sampling feature representation of each target training image, calculate the feature similarity between the sampling feature representation and the template feature representation of each category of traffic signs, and optimize the feature extraction parameters of the traffic sign recognition model to be trained based on the feature similarity.

2. The traffic sign recognition model training system according to claim 1, characterized in that: The system further comprises: An image denoising module, used for performing denoising processing on the simulated training image to obtain an initial training image for model training; A restoration loss calculation module, used for comparing the clear image with the initial training image to obtain an image restoration loss; A denoising parameter adjustment module, used for optimizing the image denoising module based on the image restoration loss, so as to obtain a pre-trained image based on the optimized image denoising module; The data enhancement module is used to perform image conversion on the pre-training image to obtain training sample images of multiple different scales and directions.

3. A traffic sign recognition model training method, characterized in that: The method comprises a first training phase and a second training phase, The first training stage includes collecting a clear image, adding noise to the collected clear image, and performing denoising on the image after adding noise to obtain a restoration loss between the denoised image and the clear image, and adjusting the denoising parameters of the traffic sign recognition model to be trained based on the restoration loss; The second training stage includes: obtaining the initial features of each category of traffic signs and constructing an initial feature memory library based on the initial features; obtaining the feature center of each category of traffic signs based on the training sample images, and updating the initial feature memory library based on the feature center to obtain the template feature representation of each category of traffic signs; extracting the sampling feature representation of the target sample image to obtain the contrast loss between the sampling feature representation and the template feature representation of the same category of traffic signs, and adjusting the feature extraction parameters of the traffic sign recognition model to be trained based on the contrast loss.

4. The traffic sign recognition model training method according to claim 3 is characterized in that: After adjusting the noise reduction parameters of the traffic sign recognition model to be trained based on the restoration loss, the method further includes: A pre-trained image is obtained based on a traffic sign recognition model after adjusting noise reduction parameters; The pre-trained images are subjected to image transformation to obtain training sample images of various scales and directions, wherein the image transformation at least includes translation, rotation, and scaling.

5. The traffic sign recognition model training method according to claim 3, characterized in that: Based on the training sample images, the feature centers of each category of traffic signs are obtained, including: Evaluate the recognition difficulty of various traffic signs according to the preset difficulty recognition standards; Dynamically sampling the training sample images based on the evaluation results of the recognition difficulty, and determining the sampled images as target training images; Feature information of the target training image is extracted, and the extracted feature information is classified based on the meaning of the traffic signs in the target training image to obtain feature centers of traffic signs of each category.

6. The traffic sign recognition model training method according to claim 3, characterized in that: Obtaining the contrast loss between the sampled feature representation and the template feature representation of the same type of traffic sign, including: Extracting a sampled feature representation of any target training image, and obtaining multiple template feature representations based on the updated feature memory library; The similarities between the sampled feature representation and multiple template feature representations are calculated respectively, and a contrast loss is constructed based on the multiple similarity values. The contrast loss is used to increase the similarity between the sampled feature representation and the template feature representation of the same category of traffic signs, and to reduce the similarity between the sampled feature representation and the template feature representation of different categories of traffic signs.

7. The traffic sign recognition model training method according to any one of claims 3 to 6, characterized in that: The method further comprises: Collecting an environment image containing traffic sign information as a test image, and generating a test sample data set based on the test image; The trained traffic sign recognition model is tested according to the test sample data set to obtain recognition parameters of the trained traffic sign recognition model to verify the traffic sign recognition ability of the traffic sign recognition model, and the recognition parameters include at least recognition accuracy and recall rate.

8. A method for identifying traffic signs, characterized in that: The method is implemented based on the traffic sign recognition model obtained by the traffic sign recognition model training method according to any one of claims 3 to 7. Acquire an image to be identified, where the image to be identified is an environment image of the environment in which the target vehicle is currently located, and the environment image includes traffic sign information; The image to be recognized is input into a traffic sign recognition model to obtain output information of the traffic sign recognition model, and the output information is used as a traffic sign recognition result of the image to be recognized.

9. An electronic device, characterized in that: It comprises a processor, a memory and a communication bus; the communication bus is used to connect the processor and the memory; the processor is used to execute a computer program stored in the memory to implement the traffic sign recognition model training method as described in any one of claims 3-7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is used to enable a computer to execute the traffic sign recognition model training method as described in any one of claims 3-7.

Images