Traffic indication recognition method and device, vehicle and storage medium

Abstract

The invention discloses a traffic indication recognition method and device, a vehicle and a storage medium, and belongs to the field of image recognition. The method comprises the following steps: acquiring an original traffic indication identification model comprising a feature extraction network, an identification task classification network and a traffic indication identification network; extracting traffic indication features of the original training image carrying the traffic indication content through a feature extraction network; predicting a task prediction category of the traffic indication features by identifying a task classification network; identifying the traffic indication characteristics according to the task prediction category and a traffic indication identification network to obtain candidate traffic indication identification information and identification confidence; performing parameter adjustment on the original traffic indication identification model according to the candidate traffic indication identification information, the identification confidence and the traffic indication content to obtain a target traffic indication identification model; and performing traffic indication identification on the target traffic image through the target traffic indication identification model to obtain target traffic indication identification information. The identification efficiency can be improved, and indication identification can be simplified.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a traffic sign recognition method and device, a vehicle and a storage medium. Background Art

[0002] Traffic sign recognition can be applied to multiple fields to perform related operations through traffic sign recognition information. For example, in the field of intelligent driving, traffic sign recognition can assist intelligent driving to improve driving safety. Among them, traffic sign recognition usually includes traffic light recognition and traffic sign recognition, and traffic light recognition and traffic sign recognition are usually implemented using different recognition models due to different target characteristics. In the related art, traffic sign recognition needs to use traffic light recognition model and traffic sign recognition model to recognize each traffic image respectively, and output traffic sign recognition information through a series of processing such as system integration and result fusion. It can be seen that the recognition of traffic content not only requires the use of multiple models, but also requires additional processing, the recognition efficiency is low and the complexity of traffic sign recognition is increased. Therefore, how to improve the recognition efficiency and reduce the complexity of traffic sign recognition has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a traffic sign recognition method and device, a vehicle and a storage medium, which can improve the recognition efficiency and reduce the complexity of traffic sign recognition.

[0004] To achieve the above objective, a first aspect of an embodiment of the present application provides a traffic sign recognition method, the method comprising:

[0005] Acquire an original training image carrying traffic instruction content; wherein the traffic instruction content includes one of the following: a traffic light, a traffic sign;

[0006] Obtaining a preset original traffic sign recognition model; wherein the original traffic sign recognition model includes: a feature extraction network, a recognition task classification network and a traffic sign recognition network;

[0007] Extracting features from the original training image through the feature extraction network to obtain traffic indication features of the traffic indication content;

[0008] Performing task category prediction on the traffic indication feature through the recognition task classification network to obtain a task prediction category;

[0009] Performing indication recognition on the traffic indication feature according to the task prediction category and the traffic indication recognition network to obtain candidate traffic indication recognition information and recognition confidence of the candidate traffic indication recognition information;

[0010] Adjusting the model parameters of the original traffic sign recognition model according to the candidate traffic sign recognition information, the recognition confidence and the traffic sign content to obtain a target traffic sign recognition model;

[0011] The target traffic sign recognition model is used to perform traffic sign recognition on the target traffic image to obtain target traffic sign recognition information; wherein the target traffic sign recognition information includes at least one of the following: the traffic light, the traffic sign.

[0012] In some embodiments, obtaining the original training image carrying traffic instruction content includes:

[0013] Acquire an original training image set carrying the traffic indication content; wherein the original training image set includes: a first training traffic image with traffic light label information and a second training traffic image with traffic sign label information;

[0014] performing traffic sign recognition on the first training traffic image to obtain traffic sign recognition information;

[0015] If the traffic sign recognition information indicates that a traffic sign exists in the first training traffic image, performing traffic sign masking processing on the first training traffic image according to the traffic sign recognition information to obtain a third training traffic image;

[0016] performing traffic light recognition on the second training traffic image to obtain traffic light recognition information;

[0017] If the traffic light identification information indicates that a traffic light exists in the second training traffic image, performing traffic light masking processing on the second training traffic image according to the traffic light identification information to obtain a fourth training traffic image;

[0018] Performing image updating on the original training image set according to the third training traffic image and the fourth training traffic image to obtain a target training image set;

[0019] The original training image is obtained from the target training image set.

[0020] In some embodiments, if the traffic sign recognition information indicates that a traffic sign exists in the first training traffic image, performing traffic sign masking processing on the first training traffic image according to the traffic sign recognition information to obtain a third training traffic image includes:

[0021] If the traffic sign recognition information indicates that a traffic sign exists in the first training traffic image, acquiring traffic sign region information of the first training traffic image according to the traffic sign recognition information;

[0022] Traffic sign masking processing is performed on the first training traffic image according to the traffic sign area information to obtain the third training traffic image.

[0023] In some embodiments, if the traffic light identification information indicates that a traffic light exists in the second training traffic image, performing traffic light masking processing on the second training traffic image according to the traffic light identification information to obtain a fourth training traffic image includes:

[0024] If the traffic light identification information indicates that a traffic light exists in the second training traffic image, acquiring traffic light area information of the second training traffic image according to the traffic light identification information;

[0025] The second training traffic image is subjected to traffic light masking processing according to the traffic light area information to obtain the fourth training traffic image.

[0026] In some embodiments, the feature extraction network includes at least two feature extraction layers with different feature extraction sizes; and extracting features from the original training image through the feature extraction network to obtain traffic indication features of the traffic indication content includes:

[0027] Performing feature extraction on the original training image through each of the feature extraction layers to obtain at least two candidate image features; wherein the feature sizes of the at least two candidate image features are different;

[0028] The at least two candidate image features are subjected to feature fusion processing to obtain the traffic indication feature of the traffic indication content.

[0029] In some embodiments, adjusting the model parameters of the original traffic sign recognition model according to the candidate traffic sign recognition information, the recognition confidence and the traffic sign content to obtain the target traffic sign recognition model includes:

[0030] Filtering selected traffic sign identification information from the candidate traffic sign identification information according to the identification confidence;

[0031] Performing loss calculation on the selected traffic instruction identification information and the traffic instruction content to obtain identification loss data;

[0032] The model parameters of the original traffic sign recognition model are adjusted according to the recognition loss data to obtain a target traffic sign recognition model.

[0033] In some embodiments, the selecting the selected traffic indication identification information from the candidate traffic indication identification information according to the identification confidence level includes:

[0034] Extracting a maximum confidence from the recognition confidences;

[0035] The candidate traffic sign identification information corresponding to the maximum confidence level is used as the selected traffic sign identification information.

[0036] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a traffic sign recognition device, the device comprising:

[0037] A data set acquisition module, used to acquire original training images carrying traffic indication content; wherein the traffic indication content includes at least one of the following: traffic lights, traffic signs;

[0038] A model acquisition module, used to acquire a preset original traffic sign recognition model; wherein the original traffic sign recognition model includes: a feature extraction network, a recognition task classification network and a traffic sign recognition network;

[0039] A feature extraction module, used to extract features from the original training image through the feature extraction network to obtain traffic indication features of the traffic indication content;

[0040] A task category prediction module, used to predict the task category of the traffic indication feature through the recognition task classification network to obtain a task prediction category;

[0041] an original indication recognition module, used to perform indication recognition on the traffic indication feature according to the task prediction category and the traffic indication recognition network, and obtain candidate traffic indication recognition information and recognition confidence of the candidate traffic indication recognition information;

[0042] A model parameter adjustment module, used to adjust the model parameters of the original traffic sign recognition model according to the candidate traffic sign recognition information, the recognition confidence and the preset traffic sign content, so as to obtain a target traffic sign recognition model;

[0043] The traffic sign recognition module is used to recognize traffic signs on a target traffic image through the target traffic sign recognition model to obtain target traffic sign recognition information; wherein the target traffic sign recognition information includes at least one of the following: the traffic light, the traffic sign.

[0044] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes a vehicle, comprising a vehicle controller and a memory for communicating with the vehicle controller; the memory stores instructions that can be executed by the vehicle controller, and the instructions are executed by the vehicle controller so that the vehicle controller executes the traffic sign recognition method as described in the first aspect.

[0045] To achieve the above objectives, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the traffic sign recognition method described in the first aspect.

[0046] The traffic sign recognition method and device, vehicle and storage medium proposed in the present application perform traffic sign recognition by using an original traffic sign recognition model composed of a feature extraction network, a recognition task classification network and a traffic sign recognition network. Specifically, the traffic sign features are first extracted from the original training image through the feature extraction network, and then the task category prediction is performed on the traffic sign features through the recognition task classification network to obtain the task prediction category, and the traffic sign features are targeted for sign recognition through the traffic sign recognition network and the task prediction category to output accurate candidate traffic sign recognition information and the recognition confidence of the candidate traffic sign recognition information. It can be seen that in the process of traffic sign recognition, only one traffic sign recognition model is used, and the recognition process first determines the task category and then performs targeted sign recognition, which not only reduces the complexity of traffic sign recognition, but also achieves accurate and efficient sign recognition. It should be noted that after the candidate traffic sign recognition information is output, the model parameters of the original traffic sign recognition model are also adjusted according to the candidate traffic sign recognition information, recognition confidence and traffic sign content to output a target traffic sign recognition model with higher recognition accuracy, so that traffic signs can be recognized on the target traffic image through the target traffic sign recognition model to output more accurate target traffic sign recognition information.

[0047] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a flow chart of a traffic sign recognition method provided by an embodiment of the present application;

[0049] Figure 2 yes Figure 1 Flow chart of step S101 in FIG.

[0050] FIG3( a ) is a schematic diagram of a first training traffic image in a traffic sign recognition method provided in an embodiment of the present application;

[0051] FIG3( b ) is a schematic diagram of a third training traffic image in the traffic sign recognition method provided in an embodiment of the present application;

[0052] FIG4( a ) is a schematic diagram of a second training traffic image in the traffic sign recognition method provided in an embodiment of the present application;

[0053] FIG4( b ) is a schematic diagram of a fourth training traffic image in the traffic sign recognition method provided in an embodiment of the present application;

[0054] Figure 5 yes Figure 2 Flow chart of step S203 in FIG.

[0055] Figure 6 yes Figure 2 Flow chart of step S205 in FIG.

[0056] Figure 7 yes Figure 1 Flow chart of step S103 in FIG.

[0057] Figure 8 is a schematic diagram of the structure of an original traffic sign recognition model in the traffic sign recognition method provided in an embodiment of the present application;

[0058] Fig. 9 yes Figure 1 Flow chart of step S106 in FIG.

[0059] Fig.10 yes Fig. 9 Flow chart of step S901 in FIG.

[0060] Fig.11 is an overall flow chart of the traffic sign recognition method provided by an embodiment of the present application;

[0061] Fig.12 It is a structural schematic diagram of a traffic sign recognition device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0063] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0064] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0065] Glossary:

[0066] Deep learning: is a subfield of artificial intelligence that aims to use deep neural networks to simulate human brain functions to achieve machine learning and pattern recognition. The main feature of deep learning is the use of multi-layer neural networks, each layer contains multiple nodes, and these nodes are connected by weights and biases. These weights and biases are adjusted according to training data to achieve specific tasks, such as image recognition, speech recognition, natural language processing, etc.

[0067] Heterogeneous Data: refers to data of different sources, structures, and formats, such as text, images, audio, video, time series data, etc. These data may come from different data sources, such as social media, sensor data, databases, file systems, etc.

[0068] YOLOX (You On ly Look Once): is a real-time target detection algorithm. The main feature of YOLOX is that it can detect multiple targets in one image at the same time and can detect targets in real time without sliding windows or other complex algorithms.

[0069] CSPDarkNet: is an image segmentation network based on Convolutional Neural Networks (CNN) for detecting and segmenting objects in darknet images.

[0070] Feature Pyramid Network (FPN) is a deep learning network architecture mainly used for target detection and image segmentation tasks. FPN generates multi-scale feature images by fusing low-level feature images with high-level feature images, thereby improving the model's detection capabilities and the ability to detect small targets.

[0071] With the integration of deep learning into the vehicle field, vehicle control has become more intelligent, especially in the field of intelligent driving, which can assist vehicle driving and improve driving safety. For example, traffic sign recognition is integrated into the field of intelligent driving, and the vehicle is controlled accordingly based on the traffic sign recognition information. Among them, traffic sign recognition includes: traffic light recognition and traffic sign recognition. Due to the different target characteristics of traffic light and traffic sign recognition, it is usually necessary to produce corresponding data sets and train corresponding recognition models respectively. In the related technology, traffic light recognition models and traffic sign recognition models are mainly used for traffic sign recognition. The traffic images collected are recognized by the traffic light recognition model and the traffic sign recognition model respectively, and the recognized results are fused and processed. However, running multiple models for traffic sign recognition will increase the complexity of recognition, and the subsequent system integration and result fusion will also increase the calculation time, affecting the efficiency of traffic sign recognition.

[0072] Based on this, the embodiments of the present application provide a traffic sign recognition method and device, a vehicle and a storage medium, which recognizes the original training image with traffic sign content through an original traffic sign recognition model composed of a feature extraction network, a recognition task classification network and a traffic sign recognition network, and the traffic sign content includes at least one of the following: traffic lights and traffic signs. Specifically, the traffic sign features of the original training image are first extracted through the feature extraction network, and then the task category of the traffic sign features is predicted through the recognition task classification network to obtain the task prediction category, and then the traffic sign features are recognized according to the task prediction category and the traffic sign recognition network to obtain candidate traffic sign recognition information and the recognition confidence of the candidate traffic sign recognition information. Therefore, by using a single recognition model to recognize traffic lights and traffic signs, and the recognition process first determines the task category and then recognizes it in a targeted manner, not only is the traffic sign recognition simplified, but it can also achieve accurate traffic sign recognition and improve recognition efficiency. Finally, the parameters of the original traffic sign recognition model are adjusted according to the candidate traffic sign recognition information, recognition confidence and traffic sign content to obtain a target traffic sign recognition model with more accurate sign recognition. Then, traffic signs are recognized on the target traffic image through the target traffic sign recognition model. On the one hand, it can simplify traffic sign recognition and improve recognition efficiency, and on the other hand, it can output accurate recognition results.

[0073] The traffic sign recognition method and device, vehicle and storage medium provided in the embodiments of the present application are specifically described through the following embodiments. First, the traffic sign recognition method in the embodiments of the present application is described.

[0074] The traffic sign recognition method provided in the embodiment of the present application relates to the field of image recognition technology. The traffic sign recognition method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, and can also be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a vehicle-mounted terminal, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements a content generation method, etc., but is not limited to the above forms.

[0075] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer computer devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0076] Please refer to Figure 1 , Figure 1 A flow chart of a traffic sign recognition method is shown. The traffic sign recognition method may include but is not limited to steps S101 to S107:

[0077] Step S101, obtaining an original training image carrying traffic instruction content; wherein the traffic instruction content includes one of the following: traffic lights, traffic signs;

[0078] Step S102, obtaining a preset original traffic sign recognition model; wherein the original traffic sign recognition model includes: a feature extraction network, a recognition task classification network and a traffic sign recognition network;

[0079] Step S103, extracting features from the original training image through a feature extraction network to obtain traffic indication features of the traffic indication content;

[0080] Step S104, predicting the task category of the traffic indication feature by identifying the task classification network to obtain the task prediction category;

[0081] Step S105, performing traffic sign recognition on the traffic sign features according to the task prediction category and the traffic sign recognition network, and obtaining candidate traffic sign recognition information and recognition confidence of the candidate traffic sign recognition information;

[0082] Step S106, adjusting the model parameters of the original traffic sign recognition model according to the candidate traffic sign recognition information, the recognition confidence and the traffic sign content, to obtain a target traffic sign recognition model;

[0083] Step S107, performing traffic sign recognition on the target traffic image through the target traffic sign recognition model to obtain target traffic sign recognition information; wherein the target traffic sign recognition information includes at least one of the following: traffic lights, traffic signs.

[0084] Steps S101 to S107 shown in the embodiment of the present application are obtained by acquiring an original training image carrying traffic signs, and inputting the original training image into an original traffic sign recognition model, and the original traffic sign recognition model includes: a feature extraction network, a recognition task classification network, and a traffic sign recognition network. The traffic sign features in the original training image are extracted by the feature extraction network, and then the task category is predicted for the traffic sign features by the recognition task classification network to obtain the task prediction category, and then the traffic sign features are recognized based on the task prediction category and the traffic sign recognition network to obtain candidate traffic sign recognition information and recognition confidence of the candidate traffic sign recognition information. Therefore, when performing traffic sign recognition, only one sign recognition model is needed, and the traffic sign recognition process will first predict the task category, and then perform traffic sign recognition in a targeted manner based on the task category, which can not only reduce the complexity of traffic sign recognition, but also achieve accurate traffic sign recognition and improve recognition efficiency. At the same time, the model parameters of the original traffic sign recognition model are adjusted according to the candidate traffic sign recognition information, recognition confidence and traffic sign content to obtain a target traffic sign recognition model with higher traffic sign recognition accuracy. The target traffic sign recognition model is then used to perform traffic sign recognition on the target traffic image, which not only reduces the complexity of traffic sign recognition, but also improves the accuracy and efficiency of traffic sign recognition.

[0085] In step S101 of some embodiments, the traffic indication content includes one of the following: traffic lights, traffic signs, that is, it is determined that the original training image contains only label information of one type of traffic indication. It should be noted that traditional traffic indication recognition is divided into traffic light recognition and traffic sign recognition, so the data set related to traffic indication recognition in the related art only has a traffic light data set with traffic light label information, or a traffic sign data set with traffic sign label information, and the original traffic indication recognition model realizes the simultaneous recognition of traffic lights and traffic signs. If the data set is remade, it will cost a lot of manpower and time costs. Therefore, the embodiment of the present application merges the existing traffic light data set and traffic sign data set into a new original training image set, so as to use the original training image set as a data set for training the original traffic indication recognition model. It should be further noted that the images in the original training image set only contain label information of one type of traffic indication. If the image contains both traffic lights and traffic signs, it will affect the recognition accuracy of the original traffic indication recognition model, so it is necessary to perform denoising on the images in the original training image set in advance to obtain a clean target training image set.

[0086] In some embodiments, please refer to Figure 2 Step S101 may include but is not limited to steps S201 to S207:

[0087] Step S201, obtaining an original training image set carrying traffic indication content; wherein the original training image set includes: a first training traffic image with traffic light label information and a second training traffic image with traffic sign label information;

[0088] Step S202, performing traffic sign recognition on the first training traffic image to obtain traffic sign recognition information;

[0089] Step S203, if the traffic sign recognition information indicates that there is a traffic sign in the first training traffic image, traffic sign masking processing is performed on the first training traffic image according to the traffic sign recognition information to obtain a third training traffic image;

[0090] Step S204, performing traffic light recognition on the second training traffic image to obtain traffic light recognition information;

[0091] Step S205, if the traffic light identification information indicates that there is a traffic light in the second training traffic image, traffic light masking processing is performed on the second training traffic image according to the traffic light identification information to obtain a fourth training traffic image;

[0092] Step S206, updating the original training image set according to the third training traffic image and the fourth training traffic image to obtain a target training image set;

[0093] Step S207: acquiring original training images from the target training image set.

[0094] In step S201 of some embodiments, the original training image set is a heterogeneous data set, mainly including training traffic images from different data sources, wherein the original training image set includes a first training traffic image and a second training traffic image, the first training traffic image has traffic light label information, and the second training traffic image has traffic light label information.

[0095] In step S202 of some embodiments, if there is a traffic sign in the first training traffic image, it is easy to affect the traffic sign recognition of the original traffic sign recognition model, so it is necessary to determine whether there is a traffic sign in the first training traffic image. In this embodiment, the first training traffic image is subjected to traffic sign recognition by the traffic sign recognition model to obtain traffic sign recognition information, and the traffic sign recognition information indicates whether there is a traffic sign in the first training traffic image.

[0096] In step S203 of some embodiments, if the traffic sign recognition information indicates that a traffic sign exists in the first training traffic image, the traffic sign in the first training traffic image needs to be masked to obtain a clean third training traffic image. It should be noted that the third training traffic image has traffic lights and traffic light label information remaining, which reduces the interference of traffic signs and can optimize the training effect of the original traffic sign recognition model. If the traffic sign recognition information indicates that a traffic sign does not exist in the first training traffic image, then the first training traffic image will not affect the training effect of the original traffic sign recognition model, and the first training traffic image can be used as the original training image for subsequent training of the original traffic sign recognition model.

[0097] Specifically, if the traffic sign recognition information indicates that there is a traffic sign in the first training traffic image, the first training traffic image is shown in FIG3(a). FIG3(a) shows that the traffic sign in the first training traffic image will affect the traffic sign recognition effect. After masking the first training traffic image, the third training image is obtained as shown in FIG3(b), that is, the traffic sign in the first training traffic image is masked with a mask to reduce the influence of the traffic sign on the traffic sign recognition effect.

[0098] In step S204 of some embodiments, if there is a traffic light in the second training traffic image, it will affect the traffic sign recognition effect of the original traffic sign recognition model. Therefore, it is necessary to determine whether there is a traffic light in the second training traffic image. In this embodiment, the traffic light recognition model is used to perform traffic light recognition on the second training traffic image to obtain traffic light recognition information. It should be noted that the traffic light recognition information represents whether there is a traffic light in the second training traffic image.

[0099] In step S205 of some embodiments, if the traffic light recognition information indicates that there is a traffic light in the second training traffic image, it means that the second training traffic image contains both traffic signs and traffic lights. If the second training traffic image is directly used to train the original traffic sign recognition model, the traffic sign recognition effect will be affected. Therefore, the traffic lights in the second training traffic image are masked to obtain a clean fourth training traffic image, and the fourth training traffic image only has traffic signs and traffic sign label information, which can reduce interference with the original traffic sign recognition model training. It should be noted that if the traffic light recognition information indicates that there is no traffic light in the second training traffic image, it means that the second training traffic image only has traffic signs and traffic sign label information, and the second training traffic image can be directly used as the original training image for the original traffic sign recognition model training.

[0100] Specifically, if the traffic light recognition information indicates that there is a traffic light in the second training traffic image, the second training traffic image is shown in FIG4(a). As can be seen from FIG4(a), the traffic light in the second training traffic image will affect the traffic sign recognition effect, and the second training traffic image needs to be masked. After masking the traffic light in the second training traffic image, the fourth training image is obtained as shown in FIG4(b), so as to reduce the influence of the traffic light on the traffic sign recognition effect.

[0101] In step S206 of some embodiments, the original training image set is updated according to the third training traffic image and the fourth training traffic image, that is, the third training traffic image replaces the corresponding first training traffic image in the original training image set, and the fourth training traffic image replaces the corresponding second training traffic image in the original training image set, so as to remove the training traffic images having both traffic lights and traffic signs, and obtain a target training image set consisting of training traffic images having only traffic lights and traffic light label information, and training traffic images having only traffic signs and traffic sign label information.

[0102] In step S207 of some embodiments, as disclosed above, the target training image set is an image set from which interference noise has been eliminated, so directly extracting the original training images from the target training image set can make the sign recognition of the original traffic sign recognition model unaffected by noise, thereby improving the training effect of the original traffic sign recognition model.

[0103] In steps S201 to S207 shown in this embodiment, a third training traffic image is obtained by identifying the traffic signs in the first training traffic image and then masking them, and a fourth training traffic image is obtained by identifying the traffic lights in the second training traffic image and then masking them, and the original training image set is updated according to the third training traffic image and the fourth training traffic image to obtain a training image set that can be used for the original traffic sign recognition model. Therefore, an image set for the original traffic sign recognition model is constructed in the existing traffic training data set, without the need to reconstruct the training image set, saving a lot of manpower and time costs, and the constructed target training image set reduces the influence of the original traffic sign recognition model on traffic sign recognition, which can improve the training effect of the original traffic sign recognition model.

[0104] In some embodiments, please refer to Figure 5 Step S203 may include but is not limited to steps S501 to S502:

[0105] Step S501, if the traffic sign recognition information indicates that the first training traffic image has a traffic sign, then obtaining traffic sign region information of the first training traffic image according to the traffic sign recognition information;

[0106] Step S502 : performing traffic sign masking processing on the first training traffic image according to the traffic sign region information to obtain a third training traffic image.

[0107] In step S501 of some embodiments, if the traffic sign recognition information indicates that a traffic sign exists in the first training traffic image, the traffic sign recognition information includes the position information of the traffic sign in the first training traffic image, and the position information of the traffic sign is defined as traffic sign position information, and the first training traffic image is regionally positioned according to the traffic sign position information to obtain the traffic sign regional information.

[0108] In step S502 of some embodiments, the traffic sign area information is the area information of the entire traffic sign, so the first training traffic image is subjected to traffic sign masking processing according to the traffic sign area information, that is, the sign area corresponding to the traffic sign area information is masked using a mask to obtain the third training traffic image. It should be noted that the traffic sign of the third training traffic image is masked, and the traffic light corresponding to the traffic light label information remains.

[0109] In steps S501 to S502 shown in this embodiment, traffic sign area information is located in the first training traffic image according to the traffic sign recognition information, and traffic sign masking is performed on the first training traffic image according to the traffic sign area information, so that the traffic masking operation of the first training traffic image is simple and accurate.

[0110] In some embodiments, please refer to Figure 6 Step S205 may include but is not limited to steps S601 to S602:

[0111] Step S601, if the traffic light identification information indicates that a traffic light exists in the second training traffic image, then obtaining traffic light region information of the second training traffic image according to the traffic light identification information;

[0112] Step S602: Perform traffic light masking processing on the second training traffic image according to the traffic light area information to obtain a fourth training traffic image.

[0113] In step S601 of some embodiments, if the traffic light identification information indicates that a traffic light exists in the second training traffic image, and the traffic light identification information includes the location information of the traffic light in the second training traffic image, the location information of the traffic light is defined as traffic light location information, and the second training traffic image is regionally positioned according to the traffic light location information to obtain traffic light regional information.

[0114] In step S602 of some embodiments, a traffic light area is located in the second training traffic image based on the traffic light area information, and the traffic light area in the second training traffic image is masked to obtain a fourth training traffic image, and a traffic sign corresponding to the traffic sign label information is obtained, thereby reducing interference content in the training traffic image.

[0115] In steps S601 to S602 shown in this embodiment, when it is determined that there are traffic signs and traffic lights in the second training traffic image, traffic light area information in the second training traffic image is determined, so that the traffic lights in the second training traffic image are masked according to the traffic light area information, so that the masking operation of the second training traffic image is simple and accurate.

[0116] In step S102 of some embodiments, the original traffic sign recognition model includes: a feature extraction network, a recognition task classification network and a traffic sign recognition network, and the original traffic sign recognition model is a multi-task model used in the traffic sign recognition function, specifically an improved YOLOX model. Specifically, the feature extraction network includes a backbone feature extraction network and an enhanced feature extraction network, and the recognition task classification network is a classifier. In this embodiment, the backbone feature extraction network is CSPDarkNet, which can realize the interaction of global and local information to achieve better feature extraction effects. The enhanced feature extraction network is a feature pyramid network, abbreviated as FPN. Specifically, FPN acts as a neck network to perform nonlinear fusion on the features extracted by CSPDarkNet to enhance the feature's ability to express objects of different sizes. The classifier is a multi-task YOLO detection head, which is used to predict the task category of the fused features.

[0117] In some embodiments, please refer to Figure 7 , the feature extraction network includes at least two feature extraction layers with different feature extraction sizes; step S103 may include but is not limited to steps S701 to S702:

[0118] Step S701, extracting features from the original training image through each feature extraction layer to obtain at least two candidate image features; wherein the feature sizes of at least two candidate image features are different;

[0119] Step S702: performing feature fusion processing on at least two candidate image features to obtain traffic indication features of traffic indication content.

[0120] In step S701 of some embodiments, as disclosed above, the feature extraction network is composed of CSPDarkNet and FPN, and the feature extraction of the original training image is mainly performed by CSPDarkNet, and CSPDarkNet can realize multi-scale feature extraction, so CSPDarkNet includes at least two feature extraction layers, and the feature extraction sizes between at least two feature extraction layers are different. For example, if three feature extraction layers are set, the feature extraction sizes of the three feature extraction layers are divided into small size, medium size and large size, and three candidate image features of different sizes will be obtained to realize feature extraction of different sizes.

[0121] In step S702 of some embodiments, as disclosed above, at least two candidate image features are nonlinearly fused through FPN to obtain traffic sign features with richer traffic information, so traffic sign recognition through traffic sign features can obtain more accurate traffic sign recognition information.

[0122] Specifically, Figure 8 As shown in the figure, CSPDarkNet is used to extract candidate image features of multiple sizes in the original training image. If there are three candidate image features, and the three candidate image features are defined as small-size features, medium-size features, and large-size features, then the small-size features, medium-size features, and large-size features are nonlinearly fused through FPN to obtain traffic indication features. The output traffic indication features have richer indication features.

[0123] In steps S701 to S702 shown in this embodiment, multiple candidate image features of different feature sizes are extracted from the original training image, and then the multiple candidate image features of different feature sizes are fused into traffic indication features to obtain traffic indication features with stronger expressive ability. Then, more accurate traffic indication recognition information can be obtained by performing traffic indication recognition through traffic indication features.

[0124] In step S104 of some embodiments, the recognition task classification network is a multi-task YOLO detection head, and the multi-task YOLO detection head can implement multiple task category divisions to achieve multi-task processing. In this embodiment, traffic sign recognition is mainly to recognize traffic signs and traffic lights in the original training image, so the multi-task YOLO detection head implements two task category divisions. Figure 8 As shown, the multi-task YOLO detection head predicts the task category of the traffic indication feature to determine the task prediction category of the traffic indication feature, and the task prediction category can be a traffic light task category or a traffic sign task category.

[0125] In step S105 of some embodiments, the traffic sign recognition network will perform traffic sign recognition on the traffic sign feature in combination with the task prediction category to make the traffic sign recognition more accurate. Specifically, if the task prediction category is the traffic light task category, the traffic sign recognition network will perform traffic light recognition on the traffic sign feature; if the task prediction category is the traffic sign task category, the traffic sign recognition network will perform traffic sign recognition on the traffic sign feature.

[0126] In some embodiments, please refer to Fig. 9 Step S106 may include but is not limited to steps S901 to S903:

[0127] Step S901, selecting selected traffic sign identification information from candidate traffic sign identification information according to identification confidence;

[0128] Step S902, performing loss calculation on the selected traffic instruction identification information and traffic instruction content to obtain identification loss data;

[0129] Step S903: adjusting the model parameters of the original traffic sign recognition model according to the recognition loss data to obtain a target traffic sign recognition model.

[0130] In step S901 of some embodiments, there are multiple candidate traffic sign identification information, and each candidate traffic sign identification information corresponds to a recognition confidence. It should be noted that the recognition confidence represents the recognition accuracy of the candidate traffic sign identification information, so the selected traffic sign identification information is selected from the multiple candidate traffic sign identification information according to the recognition confidence, and the selected traffic sign identification information is used as the final output of the original traffic sign recognition model.

[0131] In step S902 of some embodiments, the recognition loss data represents the recognition accuracy of the original traffic sign recognition model. The larger the recognized loss data is, the lower the recognition accuracy of the original traffic sign recognition model is. Conversely, the smaller the recognition loss data is, the higher the recognition accuracy of the original traffic sign recognition model is.

[0132] In step S903 of some embodiments, the model parameters of the original traffic sign recognition model are adjusted according to the recognition loss data, and the recognition loss data is also updated after the adjustment, and the model parameters of the original traffic sign recognition model continue to be adjusted according to the updated recognition loss data until the recognition loss data converges, and the model parameter adjustment of the original traffic sign recognition model is completed to obtain the target traffic sign recognition model.

[0133] In steps S901 to S903 shown in the present embodiment, selected traffic sign identification information is screened out from candidate traffic sign identification information according to recognition confidence, and then the selected traffic sign identification information and the traffic sign content are subjected to loss calculation to obtain recognition loss data, so that model parameters of the original traffic sign recognition model are adjusted according to the recognition loss data to obtain a target traffic sign recognition model with higher traffic sign recognition accuracy.

[0134] In some embodiments, please refer to Fig.10 Step S901 may include but is not limited to steps S1001 to S1002:

[0135] Step S1001, extracting the maximum confidence from the recognition confidence;

[0136] Step S1002: taking the candidate traffic sign identification information corresponding to the maximum confidence as the selected traffic sign identification information.

[0137] In steps S1001 to S1002 of some embodiments, the recognition confidence represents the recognition accuracy of the candidate traffic indication recognition information, and the candidate traffic indication recognition information corresponding to the maximum confidence is used as the selected traffic indication recognition information to output the selected traffic indication recognition information with more accurate recognition. It should be noted that in this embodiment, the candidate traffic indication recognition information corresponding to the maximum confidence is used as the selected traffic indication recognition information, and then the selected traffic indication recognition information is subjected to non-maximum suppression to obtain the selected traffic indication recognition information.

[0138] In steps S1001 to S1002 shown in this embodiment, the candidate traffic sign identification information corresponding to the maximum recognition confidence is used as the selected traffic sign identification information to output the selected traffic sign identification information with higher traffic sign identification accuracy.

[0139] In step S107 of some embodiments, the target traffic image is a traffic image collected in real time, and traffic lights and traffic signs may exist in the target traffic image at the same time, so the target traffic sign recognition model first extracts features and predicts the task category of the target traffic image, and then performs sign recognition based on the output target prediction category to output target traffic sign recognition information. It should be noted that the process of the target traffic sign recognition model performing traffic sign recognition on the target traffic image is the same as the process of the original traffic sign recognition model performing traffic sign recognition on the original training image, which will not be repeated here.

[0140] It should be noted that if the target traffic sign recognition model is applied to the field of intelligent driving, the target traffic sign recognition model runs on the vehicle terminal, and the target traffic image is collected in real time through the camera on the vehicle, and the target traffic image is transmitted to the vehicle terminal, so that the target traffic sign recognition model is run on the vehicle terminal to recognize the traffic sign of the target traffic image, and output the target traffic sign recognition information. Among them, the target traffic sign recognition information can be traffic light recognition information or traffic sign recognition information, so that the vehicle can be operated and controlled according to the target traffic sign recognition information. For example, if the target traffic sign recognition information is a red light, the vehicle is controlled to stop running, and if the target traffic sign information is a speed limit of 80, the vehicle's running speed is controlled to be below 80. Therefore, by automatically identifying the traffic signs on the road during vehicle driving, the vehicle can be controlled to perform corresponding control operations according to the traffic sign recognition information to improve the safety of the vehicle during driving.

[0141] like Fig.11As shown, the embodiment of the present application obtains an original training image set spliced ​​by a traffic light data set and a traffic sign data set, and the traffic light data set includes a first training traffic image with traffic light label information, and the traffic sign data set includes a second training traffic image with traffic sign label information. If the first training traffic image and the second training traffic image both have traffic lights and traffic signs, it is easy to disrupt the recognition of the original traffic sign recognition model. Therefore, the traffic sign recognition model is used to perform traffic sign recognition on the first training traffic image to obtain traffic sign recognition information. If the traffic sign recognition information indicates that there is a traffic sign in the first training traffic image, the traffic sign area information is located in the first training traffic image according to the traffic sign recognition information, and the traffic sign area in the first training traffic image is masked by a mask according to the traffic sign area information to obtain a third training traffic image. At the same time, the traffic light recognition model is used to perform traffic light recognition on the second training traffic image to obtain traffic light recognition information. If the traffic light recognition information indicates that there is a traffic light in the second training traffic image, the traffic light area information is located in the second training traffic image according to the traffic light recognition information, and the traffic light area in the second training traffic image is masked by a mask according to the traffic light area information to obtain a fourth training traffic image. Therefore, traffic lights and traffic light label information remain in the third training traffic image, and traffic signs and traffic sign label information remain in the fourth training traffic image. The first training traffic image corresponding to the original training image set is replaced according to the third training traffic image, and then the second training traffic image corresponding to the original training image set is replaced according to the fourth training traffic image, so as to update the original training traffic image set to obtain the target training traffic image set. It should be noted that the target training traffic image set removes unlabeled objects, so by training the original traffic sign recognition model with the target training traffic image set, the noise of the training process can be reduced, so that the training effect of the original traffic sign recognition model is better.

[0142] It should be noted that the original traffic sign recognition model includes: a feature extraction network, a recognition task classification network and a traffic sign recognition network, and the feature extraction network includes at least two feature extraction layers with different feature extraction sizes. First, the original training image is extracted from the target training traffic image set, and the feature extraction of the original training image is performed through each feature extraction layer to obtain at least two candidate image features, and the feature sizes between at least two candidate image features are different, so at least two candidate image features are fused to obtain a traffic sign feature with stronger expression ability. Then, the task category prediction is performed on the traffic sign feature through the recognition task classification network to determine which task category the traffic sign feature corresponds to to obtain the task prediction category, so as to make more accurate traffic sign recognition according to the task prediction category. If the task prediction category is the traffic light task category, the traffic sign recognition network is used to perform traffic light recognition on the traffic sign feature. If the task prediction category is the traffic sign task category, the traffic sign recognition network is used to perform traffic sign recognition on the traffic sign feature. The traffic sign recognition network outputs at least one candidate traffic sign recognition information and the recognition confidence of each candidate traffic sign recognition information, selects the maximum confidence from the recognition confidence, and uses the candidate traffic sign recognition information corresponding to the maximum confidence as the selected traffic sign recognition information, so as to output the selected traffic sign recognition information with higher recognition accuracy. In order to improve the recognition accuracy of traffic signs, the selected traffic sign recognition information and the traffic sign content are subjected to loss calculation to obtain recognition loss data, and the model parameters of the original traffic sign recognition model are adjusted according to the recognition loss data, so as to obtain a target traffic sign recognition model with higher traffic sign recognition accuracy. The target traffic sign recognition model is used to perform traffic sign recognition on the target traffic image collected in real time, so as to obtain more accurate target traffic sign recognition information.

[0143] In summary, by denoising the existing data set to obtain a clean target training image set, there is no need to re-produce the training image set used for the original traffic sign recognition model training, which reduces the labor cost and time cost. At the same time, in the process of traffic sign recognition, a traffic sign recognition model composed of a feature extraction network, a recognition task classification network and a traffic sign recognition network is adopted. By fusing image features of multiple sizes into traffic sign features with richer expressions, and making task category predictions based on traffic sign features, the traffic sign features are targeted for traffic sign recognition based on the task prediction category. There is no need to use two recognition models to complete traffic sign recognition. This can not only reduce the complexity of traffic sign recognition, but also achieve accurate recognition of traffic signs and improve recognition efficiency.

[0144] The various technical features in the above embodiments can be arbitrarily combined as long as there is no conflict or contradiction between the combinations of features. However, due to space limitations, they are not described one by one. Therefore, any combination of the various technical features in the above embodiments also falls within the scope of this specification.

[0145] Please refer to Fig.12 The embodiment of the present application further discloses a traffic sign recognition device, which uses the above-mentioned traffic sign recognition method. The traffic sign recognition device includes:

[0146] The data set acquisition module 1201 is used to acquire original training images carrying traffic indication content; wherein the traffic indication content includes at least one of the following: traffic lights, traffic signs;

[0147] The model acquisition module 1202 is used to acquire a preset original traffic sign recognition model; wherein the original traffic sign recognition model includes: a feature extraction network, a recognition task classification network and a traffic sign recognition network;

[0148] The feature extraction module 1203 is used to extract features from the original training image through a feature extraction network to obtain traffic indication features of the traffic indication content;

[0149] The task category prediction module 1204 is used to predict the task category of the traffic indication feature by identifying the task classification network to obtain the task prediction category;

[0150] The original indication recognition module 1205 is used to perform indication recognition on the traffic indication feature according to the task prediction category and the traffic indication recognition network, and obtain candidate traffic indication recognition information and recognition confidence of the candidate traffic indication recognition information;

[0151] A model parameter adjustment module 1206 is used to adjust the model parameters of the original traffic sign recognition model according to the candidate traffic sign recognition information, the recognition confidence and the preset traffic sign content to obtain a target traffic sign recognition model;

[0152] The traffic sign recognition module 1207 is used to perform traffic sign recognition on the target traffic image through the target traffic sign recognition model to obtain target traffic sign recognition information; wherein the target traffic sign recognition information includes at least one of the following: traffic lights, traffic signs.

[0153] The specific implementation of the traffic sign recognition device is substantially the same as the specific implementation of the above-mentioned traffic sign recognition method, and will not be described in detail herein.

[0154] An embodiment of the present application also discloses a vehicle, including a vehicle controller and a memory for communicating with the vehicle controller; the memory stores instructions that can be executed by the vehicle controller, and the instructions are executed by the vehicle controller so that the vehicle controller executes the traffic sign recognition method as described above.

[0155] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned traffic sign recognition method is implemented.

[0156] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0157] The traffic sign recognition method and vehicle controller, vehicle and storage medium provided in the embodiment of the present application detect the current road condition information of the target vehicle to determine whether the current road condition of the target vehicle is abnormal. If the current road condition information indicates that the road condition is abnormal, the current slip rate of the target vehicle is obtained to determine whether the target vehicle will slip. If slip occurs, the braking force and output current of the target vehicle are adjusted. Therefore, by adding slip rate monitoring and actively performing slight braking according to the slip rate, when the road condition is abnormal, the target vehicle can decelerate in advance and adjust the hardness of the air suspension when the road condition is abnormal, which can make the target vehicle more stable, greatly improving the comfort and safety of the driver and passengers.

[0158] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0159] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0160] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0161] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0162] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0163] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0164] In the several embodiments provided in the present application, it should be understood that the disclosed controller and method can be implemented in other ways. For example, the controller embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0165] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0166] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0167] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0168] The preferred embodiments of the present application are described above with reference to the accompanying drawings, but the scope of the rights of the present application is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present application should be within the scope of the rights of the present application.

Claims

1. A method for identifying traffic signs, characterized in that: The method comprises: Acquire an original training image carrying traffic instruction content; wherein the traffic instruction content includes one of the following: a traffic light, a traffic sign; Obtaining a preset original traffic sign recognition model; wherein the original traffic sign recognition model includes: a feature extraction network, a recognition task classification network and a traffic sign recognition network; Extracting features from the original training image through the feature extraction network to obtain traffic indication features of the traffic indication content; Performing task category prediction on the traffic indication feature through the recognition task classification network to obtain a task prediction category; Performing indication recognition on the traffic indication feature according to the task prediction category and the traffic indication recognition network to obtain candidate traffic indication recognition information and recognition confidence of the candidate traffic indication recognition information; Adjusting the model parameters of the original traffic sign recognition model according to the candidate traffic sign recognition information, the recognition confidence and the traffic sign content to obtain a target traffic sign recognition model; Traffic signs are recognized on the target traffic image through the target traffic sign recognition model to obtain target traffic sign recognition information; wherein the target traffic sign recognition information includes at least one of the following: the traffic light, the traffic sign.

2. The traffic sign recognition method according to claim 1, characterized in that: The obtaining of the original training image carrying the traffic instruction content includes: Acquire an original training image set carrying the traffic indication content; wherein the original training image set includes: a first training traffic image with traffic light label information and a second training traffic image with traffic sign label information; performing traffic sign recognition on the first training traffic image to obtain traffic sign recognition information; If the traffic sign recognition information indicates that a traffic sign exists in the first training traffic image, performing traffic sign masking processing on the first training traffic image according to the traffic sign recognition information to obtain a third training traffic image; performing traffic light recognition on the second training traffic image to obtain traffic light recognition information; If the traffic light identification information indicates that a traffic light exists in the second training traffic image, performing traffic light masking processing on the second training traffic image according to the traffic light identification information to obtain a fourth training traffic image; Performing image updating on the original training image set according to the third training traffic image and the fourth training traffic image to obtain a target training image set; The original training image is obtained from the target training image set.

3. The traffic sign recognition method according to claim 2, characterized in that: If the traffic sign recognition information indicates that a traffic sign exists in the first training traffic image, performing traffic sign masking processing on the first training traffic image according to the traffic sign recognition information to obtain a third training traffic image includes: If the traffic sign recognition information indicates that a traffic sign exists in the first training traffic image, acquiring traffic sign region information of the first training traffic image according to the traffic sign recognition information; Traffic sign masking processing is performed on the first training traffic image according to the traffic sign area information to obtain the third training traffic image.

4. The traffic sign recognition method according to claim 2, characterized in that: If the traffic light identification information indicates that a traffic light exists in the second training traffic image, performing traffic light masking processing on the second training traffic image according to the traffic light identification information to obtain a fourth training traffic image includes: If the traffic light identification information indicates that a traffic light exists in the second training traffic image, acquiring traffic light area information of the second training traffic image according to the traffic light identification information; The second training traffic image is subjected to traffic light masking processing according to the traffic light area information to obtain the fourth training traffic image.

5. The traffic sign recognition method according to any one of claims 1 to 4, characterized in that: The feature extraction network includes at least two feature extraction layers with different feature extraction sizes; the feature extraction of the original training image by the feature extraction network to obtain the traffic indication features of the traffic indication content includes: Performing feature extraction on the original training image through each of the feature extraction layers to obtain at least two candidate image features; wherein the feature sizes of the at least two candidate image features are different; The at least two candidate image features are subjected to feature fusion processing to obtain the traffic indication feature of the traffic indication content.

6. The traffic sign recognition method according to any one of claims 1 to 4, characterized in that: The adjusting the model parameters of the original traffic sign recognition model according to the candidate traffic sign recognition information, the recognition confidence and the traffic sign content to obtain the target traffic sign recognition model includes: Filtering selected traffic sign identification information from the candidate traffic sign identification information according to the identification confidence; Performing loss calculation on the selected traffic instruction identification information and the traffic instruction content to obtain identification loss data; The model parameters of the original traffic sign recognition model are adjusted according to the recognition loss data to obtain a target traffic sign recognition model.

7. The method for identifying traffic signs according to claim 6, characterized in that: The selecting the selected traffic indication identification information from the candidate traffic indication identification information according to the identification confidence level includes: Extracting a maximum confidence from the recognition confidences; The candidate traffic sign identification information corresponding to the maximum confidence level is used as the selected traffic sign identification information.

8. A traffic sign recognition device, characterized in that: The device comprises: A data set acquisition module, used to acquire original training images carrying traffic indication content; wherein the traffic indication content includes at least one of the following: traffic lights, traffic signs; A model acquisition module, used to acquire a preset original traffic sign recognition model; wherein the original traffic sign recognition model includes: a feature extraction network, a recognition task classification network and a traffic sign recognition network; A feature extraction module, used to extract features from the original training image through the feature extraction network to obtain traffic indication features of the traffic indication content; A task category prediction module, used to predict the task category of the traffic indication feature through the recognition task classification network to obtain a task prediction category; an original indication recognition module, used to perform indication recognition on the traffic indication feature according to the task prediction category and the traffic indication recognition network, and obtain candidate traffic indication recognition information and recognition confidence of the candidate traffic indication recognition information; A model parameter adjustment module, used to adjust the model parameters of the original traffic sign recognition model according to the candidate traffic sign recognition information, the recognition confidence and the preset traffic sign content, so as to obtain a target traffic sign recognition model; The traffic sign recognition module is used to recognize traffic signs on a target traffic image through the target traffic sign recognition model to obtain target traffic sign recognition information; wherein the target traffic sign recognition information includes at least one of the following: the traffic light, the traffic sign.

9. A vehicle, characterized in that: It includes a vehicle controller and a memory for communicating with the vehicle controller; the memory stores instructions that can be executed by the vehicle controller, and the instructions are executed by the vehicle controller to enable the vehicle controller to execute the traffic sign recognition method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the traffic sign recognition method according to any one of claims 1 to 7 is implemented.

Images