Traffic sign identification method and device based on small sample learning

Abstract

The invention discloses a traffic sign recognition method and device based on small sample learning. The method comprises the following steps: collecting traffic sign data; according to the traffic sign image data, training a preset target detection model to obtain a trained preset target detection model; training a preset small sample image classification model according to the target detection annotation file and an auxiliary label corresponding to each traffic sign to obtain a trained small sample image classification model; according to to-be-recognized image data, a target block diagram of traffic signs contained in the to-be-recognized image data is obtained through recognition of the trained preset target detection model, and according to the target block diagram, the traffic signs contained in the to-be-recognized image data are obtained. And performing identification through the trained small sample image classification model to obtain a specific category of the traffic sign contained in the to-be-identified image data. According to the invention, the efficiency and accuracy of small sample traffic sign detection and identification can be improved.

Description

Technical Field

[0001] The present invention relates to the field of traffic sign recognition, and in particular to a traffic sign recognition method and device based on small sample learning. Background Art

[0002] Traffic signs are road facilities that use words or symbols to convey guidance, restrictions, warnings or instructions. They are an important measure to implement traffic management and ensure safe and smooth road traffic. In the field of autonomous driving, accurate detection and recognition of traffic signs is one of the important tasks to ensure the safe and orderly driving of autonomous vehicles.

[0003] At present, traffic sign detection and recognition methods mainly rely on image processing and deep learning technology. The accuracy of these detection models is greatly affected by the quality and quantity of training data. Since some types of traffic signs are relatively rare, the trained models may not be able to accurately recognize these less common traffic signs.

[0004] For small sample sizes of traffic sign data, previous methods usually use data augmentation methods to expand data to meet training needs, but the improvement in training results is limited. With the development of deep learning, small sample learning technology based on meta-learning has also become a powerful tool to solve the problem of data scarcity. However, due to the small appearance differences between different categories of traffic signs, this puts higher requirements on the fine-grainedness of the model, and the current small sample target detection model is often weak in fine-grainedness. Summary of the invention

[0005] In order to overcome the above-mentioned defects of the prior art, the present invention provides a traffic sign recognition method and device based on small sample learning, which can improve the efficiency and accuracy of small sample traffic sign detection and recognition.

[0006] An embodiment of the present invention provides a traffic sign recognition method based on small sample learning, comprising the following steps:

[0007] Collecting traffic sign data; wherein the traffic sign data includes several types of traffic sign image data and several target detection annotation files with traffic signs as detection targets;

[0008] According to the traffic sign image data, a preset target detection model is trained to obtain a trained preset target detection model;

[0009] According to the target detection annotation file and the auxiliary labels corresponding to each traffic sign, a preset small sample image classification model is trained to obtain a trained small sample image classification model; wherein the auxiliary labels are obtained according to the target detection annotation file through a preset semantic model;

[0010] According to the image data to be identified, the target frame diagram of the traffic signs contained in the image data to be identified is obtained by using the trained preset target detection model, and according to the target frame diagram, the specific category of the traffic signs contained in the image data to be identified is obtained by using the trained small sample image classification model.

[0011] Furthermore, the training of a preset target detection model according to the traffic sign image data to obtain a trained preset target detection model specifically includes:

[0012] All the traffic sign image data are input into a preset YOLO model as input data for a single-category target detection task for training, and the trained preset YOLO model is determined as the trained preset target detection model.

[0013] Furthermore, the auxiliary tag is obtained according to the target detection annotation file through a preset semantic model, specifically including:

[0014] Crop the training images of several types of traffic signs from the original target detection images in the target detection annotation file respectively;

[0015] Determining the category name of the traffic sign as the category label of the traffic sign, and inputting the training image and the category label into a preset semantic model;

[0016] Semantic information corresponding to each type of traffic sign is generated respectively through the semantic model and the preset prompt words, and the semantic information is determined as the auxiliary label corresponding to the traffic sign.

[0017] Furthermore, the preset small sample image classification model is trained according to the target detection annotation file and the auxiliary labels corresponding to each traffic sign to obtain a trained small sample image classification model, which specifically includes:

[0018] Input the training image into a preset visual feature encoder to extract visual features, and input the auxiliary label into a preset text feature encoder to extract text features, and finally input the visual features and the text features into a preset alignment module to output mapped semantic features;

[0019] According to the mapped semantic features, the preset small sample image classification model is trained based on the meta-learning principle to obtain the trained small sample image classification model.

[0020] Preferably, the preset alignment module is composed of a plurality of linear layers, and the visual features and the text features are input into the preset alignment module, and the mapped semantic features are output, specifically including:

[0021] Mapping the text feature to the feature space where the visual feature is located to obtain a mapped text feature;

[0022] The mapped text feature and the visual feature are aligned according to a preset alignment target, and the aligned mapped text feature is determined as the mapped semantic feature; wherein the preset alignment target is min||h(g(y test ))-f(x)||,h(g(y test )) is the mapped semantic feature, (x) is the alignment function, g(x) is the text feature, and f(x) is the visual feature.

[0023] Furthermore, the training process of the preset small sample image classification model specifically includes:

[0024] Recording the traffic sign category whose number of appearances in the traffic sign data is greater than a preset number threshold as a base category, and recording the training graph corresponding to the traffic sign of the base category as a base category sample;

[0025] Recording the traffic sign category whose number of appearances in the traffic sign data is less than a preset number threshold as a minority class, and recording the training image corresponding to the traffic sign of the minority class as a minority class sample;

[0026] A base class training set is prepared based on the base class samples, and according to the mapped semantic features, the base class training set is used as an iterative training set to perform a first iterative training task on the preset small sample image classification model to obtain a rough classification model;

[0027] A minority class training set is prepared based on the minority class samples, and according to the mapped semantic features, the minority class training set is used as an iterative training set to perform a second iterative training task on the coarse classification model to obtain the trained small sample image classification model.

[0028] Preferably, the iterative training task specifically includes:

[0029] Starting iteration, selecting a number of categories from the iterative training set as training categories, and dividing samples in each of the training categories into support samples and query samples;

[0030] According to the mapped semantic features and the support samples corresponding to each training category, the support prototype of each training category is calculated respectively; wherein the calculation formula of the support prototype is specifically:

[0031] Suppose there are N training categories, each of which includes K support samples, and the support prototype of the jth training category is denoted as p j ,but:

[0032]

[0033] Where λ is a preset hyperparameter, h(g(y test )) is the mapping semantic feature;

[0034] According to the supporting prototypes, respectively calculating the subordination probabilities between the query sample and each of the training categories, and calculating the loss function value by using a preset cross entropy loss function according to the subordination probabilities and the preset true labels;

[0035] When the loss function value is greater than the preset loss threshold, the next round of iteration is performed until the loss function value is less than the preset loss threshold, the iteration is ended, and the iterative training task is completed.

[0036] Further, the method of obtaining a target block diagram of a traffic sign contained in the image data to be identified by the trained preset target detection model according to the image data to be identified, and obtaining a specific category of the traffic sign contained in the image data to be identified by the trained small sample image classification model according to the target block diagram specifically includes:

[0037] Inputting the image data to be identified into the trained preset target detection model, and outputting a target block diagram of the traffic sign contained in the image data to be identified;

[0038] All the target block diagrams are input into the trained small sample image classification model, and the specific categories of the traffic signs contained in the image data to be identified are obtained as output.

[0039] Another embodiment of the present invention provides a traffic sign recognition device based on small sample learning, comprising: a collection module, a first training module, a second training module and a recognition module;

[0040] The acquisition module is used to acquire traffic sign data; wherein the traffic sign data includes several types of traffic sign image data and several target detection annotation files with traffic signs as detection targets;

[0041] The first training module is used to train the preset target detection model according to the traffic sign image data to obtain a trained preset target detection model;

[0042] The second training module is used to train a preset small sample image classification model according to the target detection annotation file and the auxiliary labels corresponding to each traffic sign to obtain a trained small sample image classification model; wherein the auxiliary labels are obtained according to the target detection annotation file through a preset semantic model;

[0043] The recognition module is used to obtain the target frame diagram of the traffic signs contained in the image data to be recognized through the trained preset target detection model based on the image data to be recognized, and to obtain the specific category of the traffic signs contained in the image data to be recognized through the trained small sample image classification model based on the target frame diagram.

[0044] Furthermore, the first training module is used to train a preset target detection model according to the traffic sign image data to obtain a trained preset target detection model, specifically including:

[0045] According to the traffic sign image data, a preset target detection model is trained to obtain a trained preset target detection model, specifically including:

[0046] All the traffic sign image data are input into a preset YOLO model as input data for a single-category target detection task for training, and the trained preset YOLO model is determined as the trained preset target detection model.

[0047] Compared with the prior art, the beneficial effects of the present invention are:

[0048] By combining target detection, small sample image classification and integrating high-level semantic information, the previous traffic sign detection model has improved its poor performance when training data is scarce, and by introducing semantic auxiliary information to improve the fine-grainedness of model recognition, the detection and recognition accuracy of small sample traffic signs has been further improved. At the same time, the invention has strong transferability. In addition to being applied to traffic sign data, it can also be used for other data with similar characteristics. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic flow chart of a traffic sign recognition method based on small sample learning provided by an embodiment of the present invention.

[0050] Figure 2 A schematic diagram of a method for acquiring semantic information provided by an embodiment of the present invention.

[0051] Figure 3 A schematic diagram of a learning process framework of a small sample model provided in one embodiment of the present invention.

[0052] Figure 4 A schematic diagram of the working principle of a traffic sign recognition method based on small sample learning provided by one embodiment of the present invention.

[0053] Figure 5 A schematic structural diagram of a traffic sign recognition device based on small sample learning provided in another embodiment of the present invention. DETAILED DESCRIPTION

[0054] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0055] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0056] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0057] Reference Figure 1 , is a flow chart of a traffic sign recognition method based on small sample learning provided by an embodiment of the present invention, comprising:

[0058] S1: Collecting traffic sign data; wherein the traffic sign data includes several types of traffic sign image data and several target detection annotation files with traffic signs as detection targets;

[0059] S2: training a preset target detection model according to the traffic sign image data to obtain a trained preset target detection model;

[0060] S3: training a preset small sample image classification model according to the target detection annotation file and the auxiliary labels corresponding to each traffic sign to obtain a trained small sample image classification model; wherein the auxiliary labels are obtained according to the target detection annotation file through a preset semantic model;

[0061] S4: Based on the image data to be identified, the target frame diagram of the traffic signs contained in the image data to be identified is obtained by using the trained preset target detection model, and based on the target frame diagram, the specific category of the traffic signs contained in the image data to be identified is obtained by using the trained small sample image classification model.

[0062] For step S2, specifically, training a preset target detection model according to the traffic sign image data to obtain a trained preset target detection model specifically includes:

[0063] All the traffic sign image data are input into a preset YOLO model as input data for a single-category target detection task for training, and the trained preset YOLO model is determined as the trained preset target detection model.

[0064] In a preferred embodiment, the collected traffic sign image data is first regarded as a unified large category, namely, traffic signs, and is input into a mainstream target detection model (such as YOLOv10, etc.) as a single-category target detection task for training, so that the model can detect the location of traffic signs in complex real-world traffic scenes.

[0065] For step S3, specifically, the auxiliary label is obtained according to the target detection annotation file through a preset semantic model, specifically including:

[0066] Crop the training images of several types of traffic signs from the original target detection images in the target detection annotation file respectively;

[0067] Determining the category name of the traffic sign as the category label of the traffic sign, and inputting the training image and the category label into a preset semantic model;

[0068] Semantic information corresponding to each type of traffic sign is generated respectively through the semantic model and the preset prompt words, and the semantic information is determined as the auxiliary label corresponding to the traffic sign.

[0069] In a preferred embodiment, after obtaining the target detection annotation file, it is necessary to crop the specific traffic sign from the original image and use its original category name as its category label. The cropped image and its corresponding category name are then input into the traffic multimodal large model. By using the large model's image-to-text technology and combining it with the prompt word engineering, high-quality semantic information can be automatically obtained as the auxiliary label.

[0070] Reference Figure 2 , is a schematic diagram of a method for obtaining semantic information provided by an embodiment of the present invention. Figure 2 It can be seen that the image of a traffic sign is x, and its definition is y. After inputting x into the traffic multimodal model, the following prompt is input: "The meaning of the traffic sign image {x} is {y}. Please describe the traffic sign in detail based on the elements in the image and their meanings." The traffic model will respond with a rich and accurate text description. This high-level semantic information includes the description of the category, definition, and specific pattern elements of the traffic sign, such as the symbol color. This semantic information will serve as the auxiliary label y corresponding to the image. test .

[0071] For step S3, further, the preset small sample image classification model is trained according to the target detection annotation file and the auxiliary labels corresponding to each traffic sign to obtain a trained small sample image classification model, specifically including:

[0072] Input the training image into a preset visual feature encoder to extract visual features, and input the auxiliary label into a preset text feature encoder to extract text features, and finally input the visual features and the text features into a preset alignment module to output mapped semantic features;

[0073] According to the mapped semantic features, the preset small sample image classification model is trained based on the meta-learning principle to obtain the trained small sample image classification model.

[0074] Preferably, the preset alignment module is composed of a plurality of linear layers, and the visual features and the text features are input into the preset alignment module, and the mapped semantic features are output, specifically including:

[0075] Mapping the text feature to the feature space where the visual feature is located to obtain a mapped text feature;

[0076] The mapped text feature and the visual feature are aligned according to a preset alignment target, and the aligned mapped text feature is determined as the mapped semantic feature; wherein the preset alignment target is min||h(g(y test ))-f(x)||,h(g(y test )) is the mapped semantic feature, (x) is the alignment function, g(x) is the text feature, and f(x) is the visual feature.

[0077] In a preferred embodiment, referring to Figure 3 , is a schematic diagram of a learning process framework of a small sample model provided by an embodiment of the present invention. Figure 3 It can be seen that before training the preset small sample image classification model, the input semantic features and visual features need to be aligned, and the two can complement each other after alignment to improve the fine-grained detection ability of the model. Specifically, this preferred embodiment uses the obtained high-level semantic information as semantic cues to adaptively adjust the feature extraction network so that the image encoder pays more attention to the visual features related to the semantic cues and ignores other interference information.

[0078] First, the preferred embodiment pre-trains a text feature encoder on large-scale traffic text data, so that it has basic knowledge in the field of traffic, which is conducive to extracting more accurate text features g(y test ), in this preferred embodiment, CLIP is used as the text feature encoder. Then, this preferred embodiment uses ViT as the visual feature encoder to extract the visual features.

[0079] Finally, this preferred embodiment designs a visual-semantic alignment module, which consists of several linear layers and aims to map semantic features to the feature space where visual features are located. The visual-semantic alignment module constructs an optimization target min||h(g(y test ))-f(x)|| to align visual features with semantic features.

[0080] Furthermore, the training process of the preset small sample image classification model specifically includes:

[0081] Recording the traffic sign category whose number of appearances in the traffic sign data is greater than a preset number threshold as a base category, and recording the training graph corresponding to the traffic sign of the base category as a base category sample;

[0082] Recording the traffic sign category whose number of appearances in the traffic sign data is less than a preset number threshold as a minority class, and recording the training image corresponding to the traffic sign of the minority class as a minority class sample;

[0083] A base class training set is prepared based on the base class samples, and according to the mapped semantic features, the base class training set is used as an iterative training set to perform a first iterative training task on the preset small sample image classification model to obtain a rough classification model;

[0084] A minority class training set is prepared based on the minority class samples, and according to the mapped semantic features, the minority class training set is used as an iterative training set to perform a second iterative training task on the coarse classification model to obtain the trained small sample image classification model.

[0085] Preferably, the iterative training task specifically includes:

[0086] Starting iteration, selecting a number of categories from the iterative training set as training categories, and dividing samples in each of the training categories into support samples and query samples;

[0087] According to the mapped semantic features and the support samples corresponding to each training category, the support prototype of each training category is calculated respectively; wherein the calculation formula of the support prototype is specifically:

[0088] Suppose there are N training categories, each of which includes K support samples, and the support prototype of the jth training category is denoted as p j ,but:

[0089]

[0090] Where λ is a preset hyperparameter, h(g(y test )) is the mapping semantic feature;

[0091] According to the supporting prototypes, respectively calculating the subordination probabilities between the query sample and each of the training categories, and calculating the loss function value by using a preset cross entropy loss function according to the subordination probabilities and the preset true labels;

[0092] When the loss function value is greater than the preset loss threshold, the next round of iteration is performed until the loss function value is less than the preset loss threshold, the iteration is ended, and the iterative training task is completed.

[0093] In a preferred embodiment, the training process of the small sample image classification model includes two stages of training tasks.

[0094] The training process of the first stage consists of several meta-tasks. Assume that the support set of each meta-task consists of N classes selected from the base class of the training set, with K samples in each class. Then the support prototype of the class is:

[0095]

[0096] Among them, λ is a hyperparameter.

[0097] The probability that a sample q in the query set belongs to a certain class (i.e., the membership probability) is calculated by comparing its cosine distance d with each class of supporting prototypes in the feature space. The specific calculation formula is:

[0098]

[0099] Finally, the cross entropy loss is calculated based on the obtained probability and the true label of the sample to obtain the loss function value. The iteration is terminated after several rounds of iteration until the loss function value is less than the preset loss threshold.

[0100] The second stage of the training process is to fine-tune the reasoning on the minority class. In the second stage, the preferred embodiment freezes the text and visual feature extraction modules and the alignment module, constructs a new prototype of the minority class through the support set on the minority class, and then performs reasoning and testing on the query set based on the new support prototype.

[0101] For step S4, specifically, the step of obtaining a target block diagram of a traffic sign contained in the image data to be identified by using the trained preset target detection model according to the image data to be identified, and obtaining a specific category of the traffic sign contained in the image data to be identified by using the trained small sample image classification model according to the target block diagram, specifically includes:

[0102] Inputting the image data to be identified into the trained preset target detection model, and outputting a target block diagram of the traffic sign contained in the image data to be identified;

[0103] All the target block diagrams are input into the trained small sample image classification model, and the specific categories of the traffic signs contained in the image data to be identified are obtained as output.

[0104] In a preferred embodiment, referring to Figure 4 , is a schematic diagram of the working principle of a traffic sign recognition method based on small sample learning provided by an embodiment of the present invention. Figure 4 It can be seen that in practical applications, the image or video to be identified is first input into the target detection model (i.e., the trained preset target detection model). This step will obtain the target frame of the traffic sign in the image / video. Subsequently, the target framed by this target frame is directly input into the trained small sample image classification model (i.e., the trained small sample image classification model). The specific category to which the traffic sign belongs can be output through the image classification model.

[0105] Reference Figure 5 , is a schematic diagram of the structure of a traffic sign recognition device based on small sample learning provided by another embodiment of the present invention, comprising: a collection module 101, a first training module 102, a second training module 103 and a recognition module 104;

[0106] The acquisition module 101 is used to acquire traffic sign data; wherein the traffic sign data includes several types of traffic sign image data and several target detection annotation files with traffic signs as detection targets;

[0107] The first training module 102 is used to train the preset target detection model according to the traffic sign image data to obtain a trained preset target detection model;

[0108] The second training module 103 is used to train a preset small sample image classification model according to the target detection annotation file and the auxiliary labels corresponding to each traffic sign to obtain a trained small sample image classification model; wherein the auxiliary labels are obtained according to the target detection annotation file through a preset semantic model;

[0109] The recognition module 104 is used to obtain the target frame diagram of the traffic signs contained in the image data to be recognized through the trained preset target detection model according to the image data to be recognized, and to obtain the specific category of the traffic signs contained in the image data to be recognized through the trained small sample image classification model according to the target frame diagram.

[0110] Furthermore, the first training module 101 is used to train a preset target detection model according to the traffic sign image data to obtain a trained preset target detection model, which specifically includes:

[0111] According to the traffic sign image data, a preset target detection model is trained to obtain a trained preset target detection model, specifically including:

[0112] All the traffic sign image data are input into a preset YOLO model as input data for a single-category target detection task for training, and the trained preset YOLO model is determined as the trained preset target detection model.

[0113] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A traffic sign recognition method based on small sample learning, characterized in that: The steps include: Collecting traffic sign data; wherein the traffic sign data includes several types of traffic sign image data and several target detection annotation files with traffic signs as detection targets; Training a preset target detection model according to the traffic sign image data to obtain a trained preset target detection model; According to the target detection annotation file and the auxiliary labels corresponding to each traffic sign, a preset small sample image classification model is trained to obtain a trained small sample image classification model; wherein the auxiliary labels are obtained according to the target detection annotation file through a preset semantic model; According to the image data to be identified, the target frame diagram of the traffic signs contained in the image data to be identified is obtained by using the trained preset target detection model, and according to the target frame diagram, the specific category of the traffic signs contained in the image data to be identified is obtained by using the trained small sample image classification model.

2. The traffic sign recognition method based on small sample learning as claimed in claim 1, characterized in that: The step of training a preset target detection model according to the traffic sign image data to obtain a trained preset target detection model specifically includes: All the traffic sign image data are input into a preset YOLO model as input data for a single-category target detection task for training, and the trained preset YOLO model is determined as the trained preset target detection model.

3. The traffic sign recognition method based on small sample learning as claimed in claim 1, characterized in that: The auxiliary label is obtained according to the target detection annotation file through a preset semantic model, specifically including: Crop the training images of several types of traffic signs from the original target detection images in the target detection annotation file respectively; Determining the category name of the traffic sign as the category label of the traffic sign, and inputting the training image and the category label into a preset semantic model; Semantic information corresponding to each type of traffic sign is generated respectively through the semantic model and the preset prompt words, and the semantic information is determined as the auxiliary label corresponding to the traffic sign.

4. The traffic sign recognition method based on small sample learning as claimed in claim 3, characterized in that: The method further comprises: training a preset small sample image classification model according to the target detection annotation file and the auxiliary labels corresponding to each traffic sign to obtain a trained small sample image classification model. Input the training image into a preset visual feature encoder to extract visual features, and input the auxiliary label into a preset text feature encoder to extract text features, and finally input the visual features and the text features into a preset alignment module to output mapped semantic features; According to the mapped semantic features, the preset small sample image classification model is trained based on the meta-learning principle to obtain the trained small sample image classification model.

5. The traffic sign recognition method based on small sample learning as claimed in claim 4, characterized in that: The preset alignment module is composed of several linear layers. The visual features and the text features are input into the preset alignment module, and the mapped semantic features are output, which specifically includes: Mapping the text feature to the feature space where the visual feature is located to obtain a mapped text feature; The mapped text feature and the visual feature are aligned according to a preset alignment target, and the aligned mapped text feature is determined as the mapped semantic feature; wherein the preset alignment target is min / h(g(y test ))-f()∥,h(g(y test )) is the mapping semantic feature, () is the alignment function, g(x) is the text feature, and f(x) is the visual feature.

6. The traffic sign recognition method based on small sample learning as claimed in claim 4, characterized in that: The training process of the preset small sample image classification model specifically includes: Recording the traffic sign category whose number of appearances in the traffic sign data is greater than a preset number threshold as a base category, and recording the training graph corresponding to the traffic sign of the base category as a base category sample; Recording the traffic sign category whose number of appearances in the traffic sign data is less than a preset number threshold as a minority class, and recording the training image corresponding to the traffic sign of the minority class as a minority class sample; A base class training set is prepared based on the base class samples, and according to the mapped semantic features, the base class training set is used as an iterative training set to perform a first iterative training task on the preset small sample image classification model to obtain a rough classification model; A minority class training set is prepared based on the minority class samples, and according to the mapped semantic features, the minority class training set is used as an iterative training set to perform a second iterative training task on the coarse classification model to obtain the trained small sample image classification model.

7. The traffic sign recognition method based on small sample learning as claimed in claim 6, characterized in that: The iterative training task specifically includes: Starting iteration, selecting a number of categories from the iterative training set as training categories, and dividing samples in each of the training categories into support samples and query samples; According to the mapped semantic features and the support samples corresponding to each training category, the support prototype of each training category is calculated respectively; wherein the calculation formula of the support prototype is specifically: Suppose there are N training categories, each of which includes K support samples, and the support prototype of the jth training category is denoted as p j ,but: Where λ is a preset hyperparameter, h(g(y test )) is the mapping semantic feature; According to the supporting prototypes, respectively calculating the subordination probabilities between the query sample and each of the training categories, and calculating the loss function value by using a preset cross entropy loss function according to the subordination probabilities and the preset true labels; When the loss function value is greater than the preset loss threshold, the next round of iteration is performed until the loss function value is less than the preset loss threshold, the iteration is ended, and the iterative training task is completed.

8. The traffic sign recognition method based on small sample learning as claimed in claim 1, characterized in that: The method of obtaining a target block diagram of a traffic sign contained in the image data to be identified by using the trained preset target detection model according to the image data to be identified, and obtaining a specific category of the traffic sign contained in the image data to be identified by using the trained small sample image classification model according to the target block diagram, specifically includes: Inputting the image data to be identified into the trained preset target detection model, and outputting a target block diagram of the traffic sign contained in the image data to be identified; All the target block diagrams are input into the trained small sample image classification model, and the specific categories of the traffic signs contained in the image data to be identified are obtained as output.

9. A traffic sign recognition device based on small sample learning, characterized in that: include: An acquisition module, a first training module, a second training module and a recognition module; The acquisition module is used to acquire traffic sign data; wherein the traffic sign data includes several types of traffic sign image data and several target detection annotation files with traffic signs as detection targets; The first training module is used to train the preset target detection model according to the traffic sign image data to obtain a trained preset target detection model; The second training module is used to train a preset small sample image classification model according to the target detection annotation file and the auxiliary labels corresponding to each traffic sign to obtain a trained small sample image classification model; wherein the auxiliary labels are obtained according to the target detection annotation file through a preset semantic model; The recognition module is used to obtain the target frame diagram of the traffic signs contained in the image data to be recognized through the trained preset target detection model based on the image data to be recognized, and to obtain the specific category of the traffic signs contained in the image data to be recognized through the trained small sample image classification model based on the target frame diagram.

10. The traffic sign recognition device based on small sample learning as claimed in claim 9, characterized in that: The first training module is used to train a preset target detection model according to the traffic sign image data to obtain a trained preset target detection model, specifically including: According to the traffic sign image data, a preset target detection model is trained to obtain a trained preset target detection model, specifically including: All the traffic sign image data are input into a preset YOLO model as input data for a single-category target detection task for training, and the trained preset YOLO model is determined as the trained preset target detection model.

Images