A traffic sign recognition method and device based on few-shot learning

Abstract

This invention discloses a traffic sign recognition method and apparatus based on few-shot learning, comprising: collecting traffic sign data; training a preset target detection model based on the traffic sign image data to obtain a trained preset target detection model; training a preset few-shot image classification model based on the target detection annotation file and the auxiliary labels corresponding to each traffic sign to obtain a trained few-shot image classification model; identifying target bounding boxes of traffic signs contained in the image data to be identified using the trained preset target detection model, and identifying the specific category of the traffic signs contained in the image data to be identified using the trained few-shot image classification model based on the target bounding boxes. This invention can improve the efficiency and accuracy of few-shot traffic sign detection and recognition.

Description

Technical Field

[0001] This invention relates to the field of traffic sign recognition, and more particularly to a traffic sign recognition method and apparatus based on few-shot learning. Background Technology

[0002] Traffic signs are road facilities that use words or symbols to convey guidance, restrictions, warnings, or instructions. They are an important measure for implementing traffic management and ensuring road traffic safety and smooth flow. In the field of autonomous driving, accurate detection and recognition of traffic signs is one of the key tasks to ensure the safe and orderly operation of autonomous vehicles.

[0003] Currently, traffic sign detection and recognition methods mainly rely on image processing and deep learning technologies. The accuracy of these detection models is greatly affected by the quality and quantity of training data. Because some types of traffic signs are relatively few, the trained models may not achieve satisfactory recognition accuracy for these less common traffic signs.

[0004] For traffic sign data with small sample sizes, previous methods typically used data augmentation to expand the data to meet training needs, but the improvement in training performance was limited. With the development of deep learning, meta-learning based few-shot learning techniques have become a powerful tool for solving the data scarcity problem. However, since the appearance differences between different categories of traffic signs are small, this places higher demands on the fine granularity of the model, and current few-shot object detection models are often weak in terms of fine granularity. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, this invention provides a traffic sign recognition method and apparatus based on few-sample learning, which can improve the efficiency and accuracy of few-sample traffic sign detection and recognition.

[0006] An embodiment of the present invention provides a traffic sign recognition method based on few-shot learning, comprising the following steps:

[0007] Collect traffic sign data; wherein, the traffic sign data includes traffic sign image data of several categories and several target detection annotation files with traffic signs as the detection targets;

[0008] Based on the traffic sign image data, a preset target detection model is trained to obtain a trained preset target detection model.

[0009] Based on the target detection annotation file and the auxiliary labels corresponding to each traffic sign, a preset small sample image classification model is trained to obtain a trained small sample image classification model; wherein, the auxiliary labels are obtained from the target detection annotation file through a preset semantic model;

[0010] Based on the image data to be identified, the target bounding box of the traffic signs contained in the image data to be identified is obtained by the trained preset target detection model, and based on the target bounding box, the specific category of the traffic signs contained in the image data to be identified is obtained by the trained few-sample image classification model.

[0011] Furthermore, the step of training a preset target detection model based on the traffic sign image data to obtain a trained preset target detection model specifically includes:

[0012] All the traffic sign image data are used as input data for a single-class object detection task and fed into a preset YOLO model for training. The trained preset YOLO model is then identified as the trained preset object detection model.

[0013] Furthermore, the auxiliary labels are obtained based on the target detection annotation file through a preset semantic model, specifically including:

[0014] Training images of several types of traffic signs are cropped from the original images of each target detection in the target detection annotation file.

[0015] The category name of the traffic sign is determined as the category label of the traffic sign, and the training image and the category label are input into the preset semantic model;

[0016] Using the semantic model and preset prompt words, semantic information corresponding to each traffic sign is generated, and the semantic information is determined as the auxiliary label corresponding to the traffic sign.

[0017] Furthermore, the step of training a preset few-sample image classification model based on the target detection annotation file and the auxiliary labels corresponding to each traffic sign to obtain a trained few-sample image classification model specifically includes:

[0018] The training image is input into a preset visual feature encoder to extract visual features, and the auxiliary label is input into a preset text feature encoder to extract text features. Finally, the visual features and the text features are input into a preset alignment module to output the mapped semantic features.

[0019] Based on the mapped semantic features, the preset few-sample image classification model is trained according to the meta-learning principle to obtain the trained few-sample image classification model.

[0020] Preferably, the preset alignment module consists of several linear layers. The step of inputting the visual features and the text features into the preset alignment module and outputting mapped semantic features specifically includes:

[0021] The text features are mapped to the feature space containing the visual features to obtain the mapped text features;

[0022] The mapped text features and the visual features are aligned according to a preset alignment target, and the aligned mapped text features are determined as the mapped semantic features; wherein, the preset alignment target is... , For the mapping semantic features, For the auxiliary label The text features are as described above. The visual feature is described above.

[0023] Furthermore, the training process of the preset few-sample image classification model specifically includes:

[0024] Traffic sign categories that appear more than a preset threshold number of times in the traffic sign data are denoted as base classes, and the training images corresponding to traffic signs that are base classes are denoted as base class samples.

[0025] Traffic sign categories that appear less than a preset threshold number of times in the traffic sign data are categorized as minority categories, and the training images corresponding to traffic signs that are in minority categories are categorized as minority category samples.

[0026] Based on the base class samples, a base class training set is created. Then, according to the mapping semantic features, the base class training set is used as the iterative training set to perform the first iteration training task on the preset small sample image classification model to obtain a coarse classification model.

[0027] Based on the minority class samples, a minority class training set is created. Then, according to the mapped semantic features, the minority class training set is used as the iterative training set to perform a second iterative training task on the coarse classification model, thereby obtaining the trained small sample image classification model.

[0028] Preferably, the iterative training task specifically includes:

[0029] Start the iteration, select several categories from the iterative training set as training categories, and divide the samples in each training category into support samples and query samples respectively;

[0030] Based on the mapping semantic features and the support samples corresponding to each training category, the support prototypes for each training category are calculated respectively.

[0031] Based on the supporting prototype, the membership probability between the query sample and each of the training categories is calculated respectively, and the loss function value is calculated based on the membership probability and the preset true label through the preset cross-entropy loss function.

[0032] When the loss function value is greater than the preset loss threshold, the next iteration is performed until the loss function value is less than the preset loss threshold, at which point the iteration ends and the iterative training task is completed.

[0033] Furthermore, the step of identifying target bounding boxes of traffic signs contained in the image data to be identified using the trained preset target detection model, and then identifying the specific category of the traffic signs contained in the image data to be identified using the trained few-sample image classification model based on the target bounding boxes, specifically includes:

[0034] The image data to be identified is input into the trained preset target detection model, and the target bounding box of the traffic signs contained in the image data to be identified is output.

[0035] All the target bounding boxes are input into the trained few-sample image classification model, and the specific categories of traffic signs contained in the image data to be identified are output.

[0036] Another embodiment of the present invention provides a traffic sign recognition device based on few-shot learning, comprising: a data acquisition module, a first training module, a second training module, and a recognition module;

[0037] The acquisition module is used to acquire traffic sign data; wherein, the traffic sign data includes traffic sign image data of several categories and several target detection annotation files with traffic signs as the detection targets;

[0038] The first training module is used to train a preset target detection model based on the traffic sign image data to obtain a trained preset target detection model.

[0039] The second training module is used to train a preset small sample image classification model based on the target detection annotation file and the auxiliary labels corresponding to each traffic sign, so as to obtain a trained small sample image classification model; wherein, the auxiliary labels are obtained based on the target detection annotation file through a preset semantic model;

[0040] The recognition module is used to identify the target bounding box of the traffic sign contained in the image data to be recognized by the trained preset target detection model, and to identify the specific category of the traffic sign contained in the image data to be recognized by the trained few-sample image classification model based on the target bounding box.

[0041] Furthermore, the first training module is used to train a preset target detection model based on the traffic sign image data to obtain a trained preset target detection model, specifically including:

[0042] Based on the traffic sign image data, a preset target detection model is trained to obtain a trained preset target detection model, specifically including:

[0043] All the traffic sign image data are used as input data for a single-class object detection task and fed into a preset YOLO model for training. The trained preset YOLO model is then identified as the trained preset object detection model.

[0044] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0045] By combining object detection, few-sample image classification, and incorporating advanced semantic information, this invention improves upon the poor performance of previous traffic sign detection models when training data is scarce. Furthermore, by introducing semantic auxiliary information to enhance the fine-grainedness of model recognition, it further improves the detection and recognition accuracy of traffic signs in few samples. Moreover, this invention is highly transferable; besides traffic sign data, it can also be applied to other data with similar characteristics. Attached Figure Description

[0046] Figure 1 This is a flowchart illustrating a traffic sign recognition method based on few-shot learning, provided as an embodiment of the present invention.

[0047] Figure 2 This is a schematic diagram of a method for obtaining semantic information according to an embodiment of the present invention.

[0048] Figure 3 This is a schematic diagram of the learning process framework of a few-shot model provided in an embodiment of the present invention.

[0049] Figure 4 This is a schematic diagram illustrating the working principle of a traffic sign recognition method based on few-shot learning, provided in an embodiment of the present invention.

[0050] Figure 5 This is a schematic diagram of a traffic sign recognition device based on few-shot learning, provided as another embodiment of the present invention. Detailed Implementation

[0051] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent.

[0052] It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings.

[0053] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0054] Reference Figure 1The above is a flowchart illustrating a traffic sign recognition method based on few-shot learning according to an embodiment of the present invention, including:

[0055] S1: Collect traffic sign data; wherein, the traffic sign data includes traffic sign image data of several categories and several target detection annotation files with traffic signs as the detection target;

[0056] S2: Based on the traffic sign image data, train the preset target detection model to obtain the trained preset target detection model;

[0057] S3: Based on the target detection annotation file and the auxiliary labels corresponding to each traffic sign, train the preset small sample image classification model to obtain the trained small sample image classification model; wherein, the auxiliary labels are obtained from the target detection annotation file through the preset semantic model;

[0058] S4: Based on the image data to be identified, the target bounding box of the traffic signs contained in the image data to be identified is obtained through the trained preset target detection model, and based on the target bounding box, the specific category of the traffic signs contained in the image data to be identified is obtained through the trained few-sample image classification model.

[0059] For step S2, specifically, training the preset target detection model based on the traffic sign image data to obtain the trained preset target detection model specifically includes:

[0060] All the traffic sign image data are used as input data for a single-class object detection task and fed into a preset YOLO model for training. The trained preset YOLO model is then identified as the trained preset object detection model.

[0061] In a preferred embodiment, the collected traffic sign image data is first treated as a unified category—traffic signs—and input into a mainstream object detection model (such as YOLOv10) as a single-class object detection task for training, so that the model can detect the location of traffic signs in complex real-world traffic scenarios.

[0062] For step S3, specifically, the auxiliary labels are obtained based on the target detection annotation file through a preset semantic model, specifically including:

[0063] Training images of several types of traffic signs are cropped from the original images of each target detection in the target detection annotation file.

[0064] The category name of the traffic sign is determined as the category label of the traffic sign, and the training image and the category label are input into the preset semantic model;

[0065] Using the semantic model and preset prompt words, semantic information corresponding to each traffic sign is generated, and the semantic information is determined as the auxiliary label corresponding to the traffic sign.

[0066] In a preferred embodiment, after obtaining the target detection and annotation file, specific traffic signs need to be cropped from the original image, and their original category names are used as their category labels. The cropped image and its corresponding category name are then input into a large-scale traffic multimodal model. Utilizing the model's graph-to-text technology combined with cue word engineering, high-quality semantic information can be automatically obtained as the auxiliary labels.

[0067] Reference Figure 2 This is a schematic diagram illustrating a method for obtaining semantic information according to an embodiment of the present invention. Figure 2 Let x be the image of a traffic sign and y be its definition. Then, after inputting x into the traffic multimodal model, and providing the following prompt: "The meaning of the traffic sign image {x} is {y}. Please provide a detailed description of the traffic sign based on the elements in the image and their meanings," the traffic model will respond with a rich and accurate textual description. This high-level semantic information includes descriptions of the traffic sign's category, definition, and specific graphic elements such as symbol colors. This semantic information will serve as auxiliary labels for the image. .

[0068] For step S3, further, the step of training the preset small sample image classification model based on the target detection annotation file and the auxiliary labels corresponding to each traffic sign to obtain the trained small sample image classification model specifically includes:

[0069] The training image is input into a preset visual feature encoder to extract visual features, and the auxiliary label is input into a preset text feature encoder to extract text features. Finally, the visual features and the text features are input into a preset alignment module to output the mapped semantic features.

[0070] Based on the mapped semantic features, the preset few-sample image classification model is trained according to the meta-learning principle to obtain the trained few-sample image classification model.

[0071] Preferably, the preset alignment module consists of several linear layers. The step of inputting the visual features and the text features into the preset alignment module and outputting mapped semantic features specifically includes:

[0072] The text features are mapped to the feature space containing the visual features to obtain the mapped text features;

[0073] The mapped text features and the visual features are aligned according to a preset alignment target, and the aligned mapped text features are determined as the mapped semantic features; wherein, the preset alignment target is... , For the mapping semantic features, For alignment functions, The text features are as described above. The visual feature is described above.

[0074] In a preferred embodiment, refer to Figure 3 This is a schematic diagram illustrating the learning process framework of a few-shot model provided in an embodiment of the present invention. Figure 3 It is known that before training a pre-defined small-sample image classification model, the input semantic features and visual features need to be aligned. After alignment, they can complement each other to improve the model's fine-grained detection capability. Specifically, this preferred embodiment uses the obtained high-level semantic information as semantic cues to adaptively adjust the feature extraction network, enabling the image encoder to focus more on visual features related to semantic cues and ignore other interfering information.

[0075] First, in this preferred embodiment, a text feature encoder is pre-trained on large-scale traffic text data, giving it basic knowledge of the traffic domain, which helps to extract more accurate text features. In this preferred embodiment, CLIP is used as the text feature encoder. Then, in this preferred embodiment, ViT is used as the visual feature encoder to extract the visual features.

[0076] Finally, this preferred embodiment designs a visual-semantic alignment module, which consists of several linear layers and aims to map semantic features to the feature space containing visual features. The visual-semantic alignment module constructs an optimization target... To align visual features with semantic features.

[0077] Furthermore, the training process of the preset few-sample image classification model specifically includes:

[0078] Traffic sign categories that appear more than a preset threshold number of times in the traffic sign data are denoted as base classes, and the training images corresponding to traffic signs that are base classes are denoted as base class samples.

[0079] Traffic sign categories that appear less than a preset threshold number of times in the traffic sign data are categorized as minority categories, and the training images corresponding to traffic signs that are in minority categories are categorized as minority category samples.

[0080] Based on the base class samples, a base class training set is created. Then, according to the mapping semantic features, the base class training set is used as the iterative training set to perform the first iteration training task on the preset small sample image classification model to obtain a coarse classification model.

[0081] Based on the minority class samples, a minority class training set is created. Then, according to the mapped semantic features, the minority class training set is used as the iterative training set to perform a second iterative training task on the coarse classification model, thereby obtaining the trained small sample image classification model.

[0082] Preferably, the iterative training task specifically includes:

[0083] Start the iteration, select several categories from the iterative training set as training categories, and divide the samples in each training category into support samples and query samples respectively;

[0084] Based on the mapped semantic features and the support samples corresponding to each training category, the support prototype for each training category is calculated; wherein, the specific formula for calculating the support prototype is as follows:

[0085] There are N training categories, and each training category includes K support samples. Let the support prototype of the j-th training category be denoted as . ,but:

[0086]

[0087] in To preset hyperparameters, The mapping semantic features;

[0088] Based on the supporting prototype, the membership probability between the query sample and each of the training categories is calculated respectively, and the loss function value is calculated based on the membership probability and the preset true label through the preset cross-entropy loss function.

[0089] When the loss function value is greater than the preset loss threshold, the next iteration is performed until the loss function value is less than the preset loss threshold, at which point the iteration ends and the iterative training task is completed.

[0090] In a preferred embodiment, the training process of the few-sample image classification model includes two training phases.

[0091] The first phase of the training process consists of several meta-tasks. Let the support set for each meta-task consist of N classes selected from the base classes of the training set, with K samples from each class. Then the support prototype for the j-th class is:

[0092]

[0093] in, This is a hyperparameter.

[0094] The probability of a sample q in the query set belonging to a certain class (i.e., the membership probability) is calculated by comparing its cosine distance d with each supporting prototype in the feature space. The specific calculation formula is as follows:

[0095]

[0096] Finally, the cross-entropy loss is calculated based on the obtained probability and the true label of the sample to obtain the loss function value. This process is repeated for several rounds until the loss function value is less than the preset loss threshold, at which point the iteration ends.

[0097] The second stage of the training process involves fine-tuning inference on the minority class. In this preferred embodiment, the text and visual feature extraction modules and the alignment module are frozen. A new minority class prototype is constructed using the support set on the minority class, and then inference and testing are performed on the query set based on the new support prototype.

[0098] For step S4, specifically, the step of identifying the target bounding box of traffic signs contained in the image data to be identified using the trained preset target detection model, and identifying the specific category of the traffic signs contained in the image data to be identified using the trained few-sample image classification model based on the target bounding box, specifically includes:

[0099] The image data to be identified is input into the trained preset target detection model, and the target bounding box of the traffic signs contained in the image data to be identified is output.

[0100] All the target bounding boxes are input into the trained few-sample image classification model, and the specific categories of traffic signs contained in the image data to be identified are output.

[0101] In a preferred embodiment, refer to Figure 4 This is a schematic diagram illustrating the working principle of a traffic sign recognition method based on few-shot learning, provided in an embodiment of the present invention. Figure 4 As can be seen, in practical applications, the image or video to be identified is first input into the object detection model (i.e., the pre-trained object detection model), which will obtain the bounding box of the traffic sign in the image / video. Subsequently, the target enclosed by this bounding box is directly input into the pre-trained few-sample image classification model (i.e., the pre-trained few-sample image classification model), and the image classification model can output the specific category to which the traffic sign belongs.

[0102] Reference Figure 5The diagram below shows a traffic sign recognition device based on few-shot learning, which is another embodiment of the present invention. It includes: a data acquisition module 101, a first training module 102, a second training module 103, and a recognition module 104.

[0103] The acquisition module 101 is used to acquire traffic sign data; wherein, the traffic sign data includes traffic sign image data of several categories and several target detection annotation files with traffic signs as the detection target;

[0104] The first training module 102 is used to train a preset target detection model based on the traffic sign image data to obtain a trained preset target detection model;

[0105] The second training module 103 is used to train a preset small sample image classification model based on the target detection annotation file and the auxiliary labels corresponding to each traffic sign, so as to obtain a trained small sample image classification model; wherein, the auxiliary labels are obtained based on the target detection annotation file through a preset semantic model;

[0106] The recognition module 104 is used to identify the target bounding box of the traffic sign contained in the image data to be recognized by the trained preset target detection model, and to identify the specific category of the traffic sign contained in the image data to be recognized by the trained small sample image classification model based on the target bounding box.

[0107] Furthermore, the first training module 101 is used to train a preset target detection model based on the traffic sign image data to obtain a trained preset target detection model, specifically including:

[0108] Based on the traffic sign image data, a preset target detection model is trained to obtain a trained preset target detection model, specifically including:

[0109] All the traffic sign image data are used as input data for a single-class object detection task and fed into a preset YOLO model for training. The trained preset YOLO model is then identified as the trained preset object detection model.

[0110] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A traffic sign recognition method based on few-shot learning, characterized in that, Includes the following steps: Collect traffic sign data; wherein, the traffic sign data includes traffic sign image data of several categories and several target detection annotation files with traffic signs as the detection targets; Based on the traffic sign image data, a preset target detection model is trained to obtain a trained preset target detection model. Based on the target detection annotation file and the auxiliary labels corresponding to each traffic sign, a preset small sample image classification model is trained to obtain a trained small sample image classification model; wherein, the auxiliary labels are obtained from the target detection annotation file through a preset semantic model, specifically including: Training images of several types of traffic signs are cropped from the original images of each target detection in the target detection annotation file. The category name of the traffic sign is determined as the category label of the traffic sign, and the training image and the category label are input into the preset semantic model; Using the semantic model and preset prompt words, semantic information corresponding to each traffic sign is generated, and the semantic information is determined as the auxiliary label corresponding to the traffic sign. The step of training a pre-defined small sample image classification model based on the target detection annotation file and the auxiliary labels corresponding to each traffic sign to obtain a trained small sample image classification model specifically includes: The training image is input into a preset visual feature encoder to extract visual features, and the auxiliary label is input into a preset text feature encoder to extract text features. Finally, the visual features and the text features are input into a preset alignment module to output the mapped semantic features. Based on the mapped semantic features, the preset few-sample image classification model is trained according to the meta-learning principle to obtain the trained few-sample image classification model. Based on the image data to be identified, the target bounding box of the traffic signs contained in the image data to be identified is obtained by the trained preset target detection model, and based on the target bounding box, the specific category of the traffic signs contained in the image data to be identified is obtained by the trained few-sample image classification model.

2. The traffic sign recognition method based on few-shot learning as described in claim 1, characterized in that, The step of training a preset target detection model based on the traffic sign image data to obtain a trained preset target detection model specifically includes: All the traffic sign image data are used as input data for a single-class object detection task and fed into a preset YOLO model for training. The trained preset YOLO model is then identified as the trained preset object detection model.

3. The traffic sign recognition method based on few-shot learning as described in claim 1, characterized in that, The preset alignment module consists of several linear layers. The step of inputting the visual features and text features into the preset alignment module and outputting mapped semantic features specifically includes: The text features are mapped to the feature space containing the visual features to obtain the mapped text features; The mapped text features and the visual features are aligned according to a preset alignment target, and the aligned mapped text features are determined as the mapped semantic features; wherein, the preset alignment target is... , For the mapping semantic features, For the auxiliary label, The text features are as described above. The visual feature is described above.

4. The traffic sign recognition method based on few-shot learning as described in claim 1, characterized in that, The training process of the preset few-sample image classification model specifically includes: Traffic sign categories that appear more than a preset threshold number of times in the traffic sign data are denoted as base classes, and the training images corresponding to traffic signs that are base classes are denoted as base class samples. Traffic sign categories that appear less than a preset threshold number of times in the traffic sign data are categorized as minority categories, and the training images corresponding to traffic signs that are in minority categories are categorized as minority category samples. Based on the base class samples, a base class training set is created. Then, according to the mapping semantic features, the base class training set is used as the iterative training set to perform the first iteration training task on the preset small sample image classification model to obtain a coarse classification model. Based on the minority class samples, a minority class training set is created. Then, according to the mapped semantic features, the minority class training set is used as the iterative training set to perform a second iterative training task on the coarse classification model, thereby obtaining the trained small sample image classification model.

5. The traffic sign recognition method based on few-shot learning as described in claim 4, characterized in that, The iterative training task specifically includes: Start the iteration, select several categories from the iterative training set as training categories, and divide the samples in each training category into support samples and query samples respectively; Based on the mapping semantic features and the support samples corresponding to each training category, the support prototypes for each training category are calculated respectively. Based on the supporting prototype, the membership probability between the query sample and each of the training categories is calculated respectively, and the loss function value is calculated based on the membership probability and the preset true label through the preset cross-entropy loss function. When the loss function value is greater than the preset loss threshold, the next iteration is performed until the loss function value is less than the preset loss threshold, at which point the iteration ends and the iterative training task is completed.

6. The traffic sign recognition method based on few-shot learning as described in claim 1, characterized in that, The step of identifying target bounding boxes of traffic signs contained in the image data to be identified using the trained preset target detection model, and then identifying the specific category of the traffic signs contained in the image data to be identified using the trained few-sample image classification model based on the target bounding boxes, specifically includes: The image data to be identified is input into the trained preset target detection model, and the target bounding box of the traffic signs contained in the image data to be identified is output. All the target bounding boxes are input into the trained few-sample image classification model, and the specific categories of traffic signs contained in the image data to be identified are output.

7. A traffic sign recognition device based on few-shot learning, characterized in that, include: The system comprises an acquisition module, a first training module, a second training module, and a recognition module. The acquisition module is used to acquire traffic sign data; wherein, the traffic sign data includes traffic sign image data of several categories and several target detection annotation files with traffic signs as the detection targets; The first training module is used to train a preset target detection model based on the traffic sign image data to obtain a trained preset target detection model. The second training module is used to train a preset few-sample image classification model based on the target detection annotation file and the auxiliary labels corresponding to each traffic sign, to obtain a trained few-sample image classification model; wherein, the auxiliary labels are obtained from the target detection annotation file through a preset semantic model, specifically including: Training images of several types of traffic signs are cropped from the original images of each target detection in the target detection annotation file. The category name of the traffic sign is determined as the category label of the traffic sign, and the training image and the category label are input into the preset semantic model; Using the semantic model and preset prompt words, semantic information corresponding to each traffic sign is generated, and the semantic information is determined as the auxiliary label corresponding to the traffic sign. The step of training a pre-defined small sample image classification model based on the target detection annotation file and the auxiliary labels corresponding to each traffic sign to obtain a trained small sample image classification model specifically includes: The training image is input into a preset visual feature encoder to extract visual features, and the auxiliary label is input into a preset text feature encoder to extract text features. Finally, the visual features and the text features are input into a preset alignment module to output the mapped semantic features. Based on the mapped semantic features, the preset few-sample image classification model is trained according to the meta-learning principle to obtain the trained few-sample image classification model. The recognition module is used to identify the target bounding box of the traffic sign contained in the image data to be recognized by the trained preset target detection model, and to identify the specific category of the traffic sign contained in the image data to be recognized by the trained few-sample image classification model based on the target bounding box.

8. The traffic sign recognition device based on few-shot learning as described in claim 7, characterized in that, The first training module is used to train a preset target detection model based on the traffic sign image data to obtain a trained preset target detection model, specifically including: Based on the traffic sign image data, a preset target detection model is trained to obtain a trained preset target detection model, specifically including: All the traffic sign image data are used as input data for a single-class object detection task and fed into a preset YOLO model for training. The trained preset YOLO model is then identified as the trained preset object detection model.

Images