Road sign information recognition method, device, computer equipment and storage medium

Abstract

The present application relates to a method, apparatus, computer device, and storage medium for identifying road sign information. The method obtains an image of a road sign to be identified; extracts a classification result corresponding to the image of the road sign to be identified and image features corresponding to the classification result using a preset classification model; inputs the image features into a preset information recognition model corresponding to the classification result for information recognition, thereby obtaining a road sign information recognition result corresponding to the image of the road sign to be identified. The present application utilizes multiple neural network models, first performing classification and feature extraction using a preset classification model, then selecting a corresponding preset information recognition model based on the classification result, and then performing information recognition using the preset information recognition model. This allows for effective recognition of road sign information while ensuring recognition accuracy.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, computer equipment, and storage medium for identifying road sign information. Background Art

[0002] With the rapid development of computer technology, computer vision has emerged. Computer vision is the study of how machines can "see." Specifically, it involves using cameras and computers to replace the human eye in identifying, tracking, and measuring objects. This involves further processing the images, transforming them into images more suitable for human observation or transmission to instrumentation. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems capable of extracting "information" from images or multidimensional data. Computer vision technology can be used in areas such as autonomous driving, for example, to collect and update high-precision map features.

[0003] However, when collecting and updating high-precision map elements, different types of road signs vary in size and pattern, and a single computer vision model is difficult to achieve sufficient accuracy requirements. Summary of the Invention

[0004] Based on this, it is necessary to provide a road sign information recognition method, device, computer equipment and storage medium that can improve the accuracy of road sign information recognition in order to address the above technical problems.

[0005] A method for identifying road sign information, the method comprising:

[0006] Obtaining a road sign image to be identified;

[0007] Extracting the classification result corresponding to the road sign image to be identified and the image features corresponding to the classification result through a preset classification model;

[0008] The image features are input into a preset information recognition model corresponding to the classification result to perform information recognition, and a road sign information recognition result corresponding to the road sign image to be recognized is obtained.

[0009] In one embodiment, before extracting the classification result corresponding to the road sign image to be identified and the image features corresponding to the classification result by using a preset classification model, the method further includes:

[0010] Acquire model training data, wherein the model training data includes classification model training data and sign information recognition model training data of each classification category;

[0011] The initial neural network model is trained with the classification model training data to obtain a preset classification model, and the initial neural network model is trained with the sign information recognition model training data of each classification category to obtain a preset information recognition model corresponding to each classification category.

[0012] In one embodiment, the training of the initial neural network model using the classification model training data to obtain the preset classification model includes:

[0013] Training the initial neural network model according to the classification model training data to obtain a teacher model;

[0014] Modify the softmax function in the teacher model through knowledge distillation;

[0015] According to the teacher model after modifying the softmax function, the student model is obtained, and the student model is used as the preset classification model.

[0016] In one embodiment, the preset information recognition model includes a preset first recognition model, a preset second recognition model, and a preset third recognition model;

[0017] When the classification result includes a first type, inputting image features corresponding to the first type into a preset first recognition model, recognizing information in the image features using the preset first recognition model, and obtaining a first type of road sign information recognition result corresponding to the road sign image to be recognized, wherein the first type includes a prohibition, a warning, or an instruction;

[0018] When the classification result includes the second type, inputting the image features corresponding to the second type into a preset second recognition model, recognizing information in the image features using the preset second recognition model, and obtaining a second type of road sign information recognition result corresponding to the road sign image to be recognized, wherein the second type includes a traffic sign;

[0019] When the classification result includes a third type, inputting image features corresponding to the third type into a preset third recognition model, identifying information in the image features using the preset third recognition model, and obtaining a third type of road sign information recognition result corresponding to the road sign image to be identified, wherein the third type includes a traffic light;

[0020] The acquired road sign information recognition results are aggregated to obtain a road sign information recognition result corresponding to the road sign image to be recognized.

[0021] In one embodiment, when the classification result includes a first type, inputting image features corresponding to the first type into a preset first recognition model, recognizing information in the image features using the preset first recognition model, and obtaining a recognition result of the first type of road sign information corresponding to the road sign image to be recognized includes:

[0022] When the classification result includes the first type, identifying information in the image features by using the preset first recognition model to obtain detailed classification category information;

[0023] Recognize digital information in the image features using a preset OCR text recognition model;

[0024] A road sign information recognition result corresponding to the road sign image to be recognized is obtained according to the sub-classification category information and the digital information.

[0025] In one embodiment, when the classification result includes the second type, inputting image features corresponding to the second type into a preset second recognition model, recognizing information in the image features using the preset second recognition model, and obtaining a recognition result of the second type of road sign information corresponding to the road sign image to be recognized includes:

[0026] When the classification result includes the second type, identifying information in the image features by using the preset second recognition model to obtain traffic sign category information;

[0027] Recognize the text information in the image features by using a preset OCR text recognition model;

[0028] A road sign information recognition result corresponding to the road sign image to be recognized is obtained according to the traffic sign category information and the text information.

[0029] In one embodiment, before acquiring the road sign image to be identified, the method further includes:

[0030] Building a road sign image recognition pool, wherein the preset classification model is used to batch process road sign images to be recognized in the road sign image recognition pool;

[0031] The obtaining of the road sign image to be identified comprises:

[0032] The road sign image to be recognized is acquired through the road sign image recognition pool.

[0033] A road sign information recognition device, comprising:

[0034] An information acquisition module, used to acquire images of road signs to be identified;

[0035] A classification and recognition module, configured to extract a classification result corresponding to the road sign image to be recognized and image features corresponding to the classification result through a preset classification model;

[0036] The information recognition module is used to input the image features into a preset information recognition model corresponding to the classification result to perform information recognition, and obtain a road sign information recognition result corresponding to the road sign image to be recognized.

[0037] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0038] Obtaining a road sign image to be identified;

[0039] Extracting the classification result corresponding to the road sign image to be identified and the image features corresponding to the classification result through a preset classification model;

[0040] The image features are input into a preset information recognition model corresponding to the classification result to perform information recognition, and a road sign information recognition result corresponding to the road sign image to be recognized is obtained.

[0041] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0042] Obtaining a road sign image to be identified;

[0043] Extracting the classification result corresponding to the road sign image to be identified and the image features corresponding to the classification result through a preset classification model;

[0044] The image features are input into a preset information recognition model corresponding to the classification result to perform information recognition, and a road sign information recognition result corresponding to the road sign image to be recognized is obtained.

[0045] The above-mentioned road sign information recognition method, apparatus, computer equipment, and storage medium obtain an image of a road sign to be recognized; extract a classification result corresponding to the road sign image to be recognized and image features corresponding to the classification result using a preset classification model; input the image features into a preset information recognition model corresponding to the classification result for information recognition, thereby obtaining a road sign information recognition result corresponding to the road sign image to be recognized. This application utilizes multiple neural network models, first performing classification and feature extraction using a preset classification model, then selecting a corresponding preset information recognition model based on the classification result, and then performing information recognition using the preset information recognition model, thereby effectively realizing the recognition of road sign information while ensuring recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A diagram showing an application environment of a road sign information recognition method according to an embodiment;

[0047] Figure 2 1 is a flow chart of a method for identifying road sign information in one embodiment;

[0048] Figure 3 A schematic diagram of a flow chart of a model training step in one embodiment;

[0049] Figure 4 Schematic diagram of a process for performing knowledge distillation processing in one embodiment;

[0050] Figure 5 A schematic diagram of a process for constructing a road sign image recognition pool in one embodiment;

[0051] Figure 6 is a structural block diagram of a road sign information recognition device in one embodiment;

[0052] Figure 7 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0054] The road sign information recognition method provided in this application can be applied to Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. Terminal 102 can send a road sign image to be identified to server 104, so that server 104 can perform road sign information recognition. Server 104 obtains the road sign image to be identified, extracts image features and classification results corresponding to the road sign image to be identified using a preset classification model, inputs the image features into a preset information recognition model corresponding to the classification result, and uses the preset information recognition model to recognize the information in the image features, thereby obtaining a road sign information recognition result corresponding to the road sign image to be identified. Terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablet computers, and portable wearable devices, and server 104 can be implemented as an independent server or a server cluster consisting of multiple servers.

[0055] In one embodiment, Figure 2 As shown, a road sign information recognition method is provided, which is applied to Figure 1 Taking the server 104 in the example as an example, the following steps are included:

[0056] Step 201: Acquire a road sign image to be identified.

[0057] Among them, the road sign image to be identified refers to the target image identified by the road sign information recognition method of the present application. The road sign information recognition method of the present application is to identify the information contained in the road sign image to be identified, determine the type of road sign contained in the image, and the position of the road sign in the image, etc.

[0058] Specifically, when road sign information is needed, the terminal 102 can send the collected road sign image to the server 104 through the network, and the server 104 starts the road sign image recognition process. In one embodiment, the road sign information recognition method of the present application is specifically applied to an autonomous driving technology as an auxiliary method for collecting high-precision map information, which is used to collect high-precision road sign information. As the infrastructure of autonomous driving technology, the collection and updating of high-precision maps has important strategic significance. When supplementing high-precision map information, the collected image containing road signs can be submitted to the server 104, and the server 104 starts the corresponding road sign information recognition.

[0059] Step 203 : extracting the classification result corresponding to the road sign image to be identified and the image features corresponding to the classification result through a preset classification model.

[0060] Among them, the preset classification model refers to the model used to identify the type of sign in the road sign image to be identified. It can be trained according to the training method of the existing classification model. Specifically, hundreds of types of road sign information to be identified can be classified in advance, and then the corresponding road sign information can be identified. For example, in one embodiment, the road sign image to be identified can be divided into types such as prohibitions, warnings, instructions, traffic signs, and traffic lights. If all types of road elements are directly trained using the detection model, there are too many elements and the model performance is difficult to guarantee. After being divided into several major categories as mentioned above, the difficulty of the problem is reduced and a higher accuracy can be guaranteed. These types of road elements are quite different and the processing methods are also different. The image features corresponding to the road sign image to be identified refer to the feature part image used to determine the information contained in the road sign image to be identified.

[0061] Specifically, after obtaining the image of the road sign to be identified, the type of road sign contained in the image of the road sign to be identified can be identified by a preset classification model, and the image features contained in the image can be extracted. In particular, for the case where an image of the road sign to be identified contains multiple road signs to be identified, the original image can be segmented accordingly, and then feature extraction and classification identification can be performed. For example, the original image A contains sign a and sign b. At this time, the original image A can be split into two parts Aa and Ab by a preset classification model, and then image feature extraction and sign information classification can be performed. In one embodiment, the image segmentation process specifically includes first locating the image area of ​​the sign in the original image A by a preset classification model, and then segmenting the original image by an image segmentation method based on the image area. In another embodiment, the preset classification model can directly segment the original image by an edge segmentation method, and then perform classification identification.

[0062] Step 205 : Input the image features into a preset information recognition model corresponding to the classification result to perform information recognition, and obtain a road sign information recognition result corresponding to the road sign image to be recognized.

[0063] The preset information recognition model is used to identify specific information within a road sign image based on image features. Different road sign types correspond to different preset information recognition models. By performing further information recognition based on the sign type, the accuracy of road sign information recognition can be effectively guaranteed.

[0064] Specifically, in the road sign information of this application, different types of road sign images to be identified correspond to different road sign types. For example, for road signs such as prohibitions, warnings, and instructions, the signs generally contain digital information. By determining the type of road sign and the digital information marked on the sign, the complete meaning of the road sign can be clearly understood. For road signs such as traffic signs, the signs generally contain text information. By determining the type of road sign and the text information marked on the sign, the complete meaning of the road sign can be clearly understood. For traffic lights, the type of traffic light can be directly determined based on the image features.

[0065] The above-mentioned road sign information recognition method obtains an image of a road sign to be recognized; extracts a classification result corresponding to the road sign image and image features corresponding to the classification result using a preset classification model; and inputs the image features into the preset information recognition model corresponding to the classification result for information recognition, thereby obtaining a road sign information recognition result corresponding to the road sign image to be recognized. This application utilizes multiple neural network models, first performing classification and feature extraction using a preset classification model, then selecting a corresponding preset information recognition model based on the classification result, and then performing information recognition using the preset information recognition model. This effectively achieves recognition of road sign information while ensuring recognition accuracy.

[0066] In one embodiment, Figure 3 As shown, before step 203, the following steps are also included:

[0067] Step 302: Acquire model training data, where the model training data includes classification model training data and sign information recognition model training data for each classification category.

[0068] Step 304: train the initial neural network model using the classification model training data to obtain a preset classification model, and train the initial neural network model using the sign information recognition model training data of each classification category to obtain a preset information recognition model corresponding to each classification category.

[0069] Specifically, the model training data can be obtained based on historical road sign image information, and the training of the preset classification model and the preset information recognition model can be completed based on the model training data.

[0070] Specifically, the road sign information recognition method in the present application can specifically recognize road sign information through the grading of multiple neural network models. Among them, the preset classification model used for classification can specifically be a classification model constructed based on a BP neural network or a perceptron neural network. The preset information recognition model used for information recognition can specifically be an image information recognition model based on an RBF (Radial Basis Function) neural network. In addition, an OCR function can be added to the preset information recognition model to recognize text or digital information in the sign. Before extracting the image features and classification results corresponding to the road sign image to be identified through the preset classification model, the preset classification model and the preset information recognition model need to be trained separately to ensure their usability. At this time, the historical road sign images can be used as model training data, and then the initial neural network model can be trained through supervised training or unsupervised training to obtain the corresponding preset classification model and various types of preset information recognition models. That is, the initial neural network model can be trained through the classification model training data to obtain the preset classification model, and the initial neural network model can be trained through the sign information recognition model training data of each classification category to obtain the preset information recognition model corresponding to each classification category. In this embodiment, the training of the preset classification model and the preset information recognition model is completed through the model training data, which can effectively ensure the accuracy of model recognition.

[0071] In one embodiment, Figure 4 As shown, step 304 includes:

[0072] Step 401: Train the initial neural network model according to the classification model training data to obtain a teacher model.

[0073] Step 403: Modify the softmax function in the teacher model through knowledge distillation.

[0074] Step 405: Obtain a student model based on the teacher model after modifying the softmax function, and use the student model as a preset classification model.

[0075] Knowledge distillation involves introducing soft-targets related to the teacher network (complex but with superior reasoning performance) as part of the total loss to guide the training of the student network (simplified and low-complexity), achieving knowledge transfer. This ensures model recognition accuracy while streamlining the neural network.

[0076] Specifically, in order to ensure the processing of complex and diverse road element information, the recognition process of road sign information requires the use of multiple different neural network models, and each model contains tens of thousands of neural network parameters, and various interfaces are required between models to process the input image stream, so each model needs to be compressed and the processing flow needs to be improved. At this time, a single neural network model can be compressed through knowledge distillation. Knowledge distillation is to use the original training model we use in the system as a Teacher model to train a relatively streamlined Student model. The initial neural network model can be trained using the classification model training data to obtain a teacher model. In another specific embodiment, when training the preset information recognition model corresponding to each classification category, the knowledge distillation method in this embodiment can also be used to perform knowledge distillation on the preset information recognition model to perform model compression. In this application, the softmax function in the original neural network model is modified through knowledge distillation, and the temperature variable T is added to obtain the changed softmax function. The original softmax function and the modified softmax function can be specifically referred to the following formula:

[0077]

[0078]

[0079] In the formula, when the total number of input data of the model is j, T represents the temperature variable, q i The probability abundance of the model output corresponding to the i-th data, z i represents the output of the model under the i-th data, exp(z i ) is the index of the output result, ∑ j exp(z j ) is the exponential sum of all output results. The original softmax function can be modified by the temperature variable T.

[0080] At this time, by increasing the temperature T, the knowledge learned by the original model training is extracted, and the information we need is extracted like "distillation". The Student model obtained by this method can replace the original Teacher model to streamline the system. In another embodiment, the tensorRT method is also used to replace knowledge distillation, which can also achieve the purpose of streamlining the model. In this embodiment, the neural network model used in the method is compressed through knowledge distillation, which greatly reduces the time required for detection. Through experimental comparison, it was found that the detection speed of the entire method can be doubled by optimizing the knowledge distillation of the model, with only a 0.1% loss in accuracy.

[0081] In one embodiment, the classification results include prohibitions, warnings, instructions, traffic signs, and traffic lights, and the preset information recognition model includes a preset first recognition model, a preset second recognition model, and a preset third recognition model;

[0082] Step 205 includes:

[0083] When the classification result includes the first type, inputting the image features corresponding to the first type into a preset first recognition model, recognizing information in the image features using the preset first recognition model, and obtaining a first type of road sign information recognition result corresponding to the road sign image to be recognized, where the first type includes a prohibition, a warning, or an instruction;

[0084] When the classification result includes the second type, inputting the image features corresponding to the second type into a preset second recognition model, recognizing information in the image features using the preset second recognition model, and obtaining a second type of road sign information recognition result corresponding to the road sign image to be recognized, where the second type includes traffic signs;

[0085] When the classification result includes the third type, inputting the image features corresponding to the third type into a preset third recognition model, recognizing information in the image features using the preset third recognition model, and obtaining a third type of road sign information recognition result corresponding to the road sign image to be recognized, where the third type includes traffic lights;

[0086] The acquired road sign information recognition results are aggregated to obtain a road sign information recognition result corresponding to the road sign image to be recognized.

[0087] Prohibitions refer to signs with a white background and a red outer circle, including those prohibiting honking, prohibiting right turns, and requiring height and width restrictions. Warnings refer to signs with a yellow background and black patterns, including those indicating Y-shaped intersections, continuous curves, and narrowing on both sides. Instructions refer to signs with a blue background and white patterns, including those indicating going straight, turning left or right, and one-way streets. Traffic signs primarily include large rectangular signs with white text on a blue background, such as road signs, lane signs, and bus information signs. The above categories, combined with cameras and traffic lights, essentially cover the feature categories required for building high-precision maps. The detection model is trained to handle the six major categories of road features. Furthermore, the target to be identified can also include cameras. In this case, when the preset classification model identifies the information within the image as a camera, the camera recognition result can be directly output without performing subsequent recognition processing by the preset information recognition model. Due to the varying sizes and designs of different types of traffic signs, a single computer vision model is unlikely to achieve sufficient accuracy. Therefore, using multiple models to hierarchically extract useful information is a necessary solution to this problem.

[0088] Specifically, prohibitions, warnings, and instructions are all pattern-type signs, which can generally be identified with numbers. Therefore, these three types of sign information can be treated as the same category for processing. The information in the image features is recognized by the preset first recognition model to obtain the road sign information recognition result corresponding to the road sign image to be recognized. Traffic signs generally contain specific text information, so the content on such traffic signs can be recognized by the preset second recognition model. Traffic lights generally do not contain text information or digital information, and the information corresponding to the traffic lights can be directly determined based on the image features by the preset second recognition model. After the information is recognized by each type of preset information recognition model, the final complete result can be obtained by aggregating the information recognition results obtained. By aggregating different results, it can be ensured that the image can be fully recognized even when it contains multiple types of road sign information. In this embodiment, the feature information of the classified road sign information is recognized by different categories of recognition models, which can improve the recognition accuracy during the recognition process.

[0089] In one embodiment, when the classification result includes the first type, inputting image features corresponding to the first type into a preset first recognition model, and identifying information in the image features using the preset first recognition model to obtain a recognition result of the first type of road sign information corresponding to the road sign image to be identified includes:

[0090] When the classification result includes the first type, information in the image features is recognized by using a preset first recognition model to obtain detailed classification category information.

[0091] Recognize digital information in image features through the preset OCR text recognition model.

[0092] A road sign information recognition result corresponding to the road sign image to be recognized is obtained based on the sub-classification category information and the digital information.

[0093] Specifically, when the classification result is a prohibition, warning, or instruction, the information in the image features can be first sub-classified using a preset first recognition model to determine the specific type of prohibition, warning, or instruction the road sign image to be identified belongs to, i.e., the sub-classification category information, and simultaneously determine the position information of the numbers in the image. The digital information in the image features is then identified using a preset OCR text recognition model. The sub-classification category information and the digital information are combined to determine the road sign information recognition result corresponding to the road sign image to be identified. In this embodiment, by obtaining the sub-classification information and the digital information, the information type contained in the road sign can be more effectively identified, thereby improving the recognition accuracy.

[0094] In one embodiment, when the classification result includes the second type, inputting image features corresponding to the second type into a preset second recognition model, and identifying information in the image features using the preset second recognition model to obtain a recognition result of the second type of road sign information corresponding to the road sign image to be identified includes:

[0095] When the classification result includes the second type, identifying information in the image features by using a preset second recognition model to obtain traffic sign category information;

[0096] Recognize text information in image features through the preset OCR text recognition model;

[0097] Obtain a road sign information recognition result corresponding to the road sign image to be recognized based on the traffic sign category information and text information.

[0098] Specifically, when the classification result is a traffic sign, the information in the image features can be first sub-classified and identified using a preset second recognition model to determine the specific type of traffic sign the road sign image to be identified belongs to, while also determining the position information of the text in the image. The text information in the image features can then be identified using a preset OCR text recognition model. Combined with the traffic sign category information and text information, the road sign information recognition result corresponding to the road sign image to be identified can be determined. In this embodiment, by obtaining traffic sign type information and text information, the information type contained in the road sign can be more effectively identified, thereby improving recognition accuracy.

[0099] In one embodiment, Figure 5 As shown, before step 201, the following steps are also included:

[0100] Step 502 : constructing a road sign image recognition pool, and presetting a classification model for batch processing the road sign images to be recognized in the road sign image recognition pool.

[0101] Step 201 includes:

[0102] Step 504: Acquire a road sign image to be recognized through a road sign image recognition pool.

[0103] Specifically, the recognition efficiency of the recognition process can be improved by optimizing the overall processing flow. Specifically, a road sign image recognition pool can be created before classification, and road sign images can be received in batches through the recognition pool. The image streams entering each processing flow are imported into the pool, and then when the pool is full, the images are fed into the next model together. For example, if the pool size is set to 25, when the pool is full, 25 images will be fed into the pool. Figure 1Importing the model in one go means calling the model only once, significantly saving call time. In one embodiment, a "pool" can be established between each stage. After passing through the broad category detection model, images input to the system enter different processing streams and then flow into different models. However, each image stream requires a model call every time it reaches the model interface, resulting in a high call frequency and wasted time. In another embodiment, processing of the image stream corresponding to the road sign image to be identified can be performed on the GPU (Graphics Processing Unit), avoiding the need to copy the image from the GPU to the CPU (Central Processing Unit) from the initial reading to operations such as cropping and deformation. Due to the GPU's significant advantages in image processing, this improvement can significantly increase computing speed. Furthermore, due to the extremely high computing speed on the GPU, the difference in processing speed between one and 25 images is minimal, resulting in significant overall time savings. Multiple "pools" are established between the model interfaces of the method, and experimental testing has shown that time savings of at least 20% can be achieved. In this embodiment, by introducing a road sign image recognition pool to perform the overall recognition process, the efficiency of the recognition process can be effectively improved.

[0104] It should be understood that although Figure 2-5 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2-5 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The order of execution of these steps or stages is not necessarily one by one, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0105] In one embodiment, Figure 6 As shown, a road sign information recognition device is provided, comprising:

[0106] The information acquisition module 601 is used to acquire the road sign image to be identified.

[0107] The classification and recognition module 603 is used to extract the classification result corresponding to the road sign image to be recognized and the image features corresponding to the classification result through a preset classification model.

[0108] The information recognition module 605 is used to input the image features into a preset information recognition model corresponding to the classification result to perform information recognition, and obtain a road sign information recognition result corresponding to the road sign image to be recognized.

[0109] In one embodiment, a model training module is also included, which is used to: obtain model training data, the model training data includes classification model training data and sign information recognition model training data of each classification category; train the initial neural network model through the classification model training data to obtain a preset classification model, and train the initial neural network model through the sign information recognition model training data of each classification category to obtain a preset information recognition model corresponding to each classification category.

[0110] In one embodiment, the model training module is also used to: train the initial neural network model according to the classification model training data to obtain a teacher model; modify the softmax function in the teacher model through knowledge distillation; obtain a student model based on the teacher model after the softmax function is modified, and use the student model as the preset classification model.

[0111] In one embodiment, the preset information recognition model includes a preset first recognition model, a preset second recognition model, and a preset third recognition model; the information recognition module 605 is specifically used to: when the classification result includes the first type, input the image features corresponding to the first type into the preset first recognition model, and use the preset first recognition model to recognize the information in the image features to obtain the first type of road sign information recognition result corresponding to the road sign image to be recognized, and the first type includes prohibitions, warnings, or instructions; when the classification result includes the second type, input the image features corresponding to the second type into the preset second recognition model, and use the preset second recognition model to recognize the information in the image features to obtain the second type of road sign information recognition result corresponding to the road sign image to be recognized, and the second type includes traffic signs; when the classification result includes the third type, input the image features corresponding to the third type into the preset third recognition model, and use the preset third recognition model to recognize the information in the image features to obtain the third type of road sign information recognition result corresponding to the road sign image to be recognized, and the third type includes traffic lights; aggregate the acquired road sign information recognition results to obtain the road sign information recognition result corresponding to the road sign image to be recognized.

[0112] In one embodiment, the information recognition module 605 is specifically used to: when the classification result includes the first type, identify the information in the image features through a preset first recognition model to obtain detailed classification category information; identify the digital information in the image features through a preset OCR text recognition model; and obtain the road sign information recognition result corresponding to the road sign image to be identified based on the detailed classification category information and the digital information.

[0113] In one embodiment, the information recognition module 605 is specifically used to: when the classification result includes the second type, identify the information in the image features through a preset second recognition model to obtain traffic sign category information; identify the text information in the image features through a preset OCR text recognition model; and obtain the road sign information recognition result corresponding to the road sign image to be identified based on the traffic sign category information and text information.

[0114] In one embodiment, it also includes a processing pool construction module for: constructing a road sign image recognition pool, and presetting a classification model for batch processing the road sign images to be identified in the road sign image recognition pool; the information acquisition module 601 is specifically used to: obtain the road sign images to be identified through the road sign image recognition pool.

[0115] The specific limitations of the road sign information recognition device can be found in the limitations of the road sign information recognition method described above and will not be further elaborated here. Each module in the aforementioned road sign information recognition device may be implemented in whole or in part via software, hardware, or a combination thereof. Each of these modules may be embedded in or independent of a processor within a computer device in hardware form, or may be stored in a computer device memory in software form, allowing the processor to call and execute the corresponding operations of each module.

[0116] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store road sign information recognition data. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a road sign information recognition method is implemented.

[0117] Those skilled in the art will understand that Figure 7The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0118] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0119] Obtaining a road sign image to be identified;

[0120] Extracting the classification results corresponding to the road sign image to be identified and the image features corresponding to the classification results through a preset classification model;

[0121] The image features are input into a preset information recognition model corresponding to the classification result for information recognition, and a road sign information recognition result corresponding to the road sign image to be recognized is obtained.

[0122] In one embodiment, when the processor executes the computer program, it also implements the following steps: obtaining model training data, the model training data includes classification model training data and sign information recognition model training data of each classification category; training the initial neural network model through the classification model training data to obtain a preset classification model, training the initial neural network model through the sign information recognition model training data of each classification category to obtain a preset information recognition model corresponding to each classification category.

[0123] In one embodiment, when the processor executes the computer program, it also implements the following steps: training the initial neural network model according to the classification model training data to obtain a teacher model; modifying the softmax function in the teacher model through knowledge distillation; obtaining a student model based on the teacher model after the softmax function is modified, and using the student model as the preset classification model.

[0124] In one embodiment, when the processor executes the computer program, the following steps are also implemented: when the classification result includes a first type, the image features corresponding to the first type are input into a preset first recognition model, and the information in the image features is recognized by the preset first recognition model to obtain a first type of road sign information recognition result corresponding to the road sign image to be recognized, and the first type includes prohibitions, warnings or instructions; when the classification result includes a second type, the image features corresponding to the second type are input into a preset second recognition model, and the information in the image features is recognized by the preset second recognition model to obtain a second type of road sign information recognition result corresponding to the road sign image to be recognized, and the second type includes traffic signs; when the classification result includes a third type, the image features corresponding to the third type are input into a preset third recognition model, and the information in the image features is recognized by the preset third recognition model to obtain a third type of road sign information recognition result corresponding to the road sign image to be recognized, and the third type includes traffic lights; and the acquired road sign information recognition results are aggregated to obtain a road sign information recognition result corresponding to the road sign image to be recognized.

[0125] In one embodiment, when the processor executes the computer program, the following steps are further implemented: when the classification result includes the first type, information in the image features is identified by a preset first recognition model to obtain detailed classification category information; digital information in the image features is identified by a preset OCR text recognition model; and based on the detailed classification category information and the digital information, a road sign information recognition result corresponding to the road sign image to be identified is obtained.

[0126] In one embodiment, when the processor executes the computer program, the following steps are also implemented: when the classification result includes the second type, the information in the image features is identified by a preset second recognition model to obtain traffic sign category information; the text information in the image features is identified by a preset OCR text recognition model; and based on the traffic sign category information and the text information, a road sign information recognition result corresponding to the road sign image to be identified is obtained.

[0127] In one embodiment, when the processor executes the computer program, it further implements the following steps: constructing a road sign image recognition pool, and presetting a classification model for batch processing the road sign images to be recognized in the road sign image recognition pool.

[0128] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0129] Obtaining a road sign image to be identified;

[0130] Extracting the classification results corresponding to the road sign image to be identified and the image features corresponding to the classification results through a preset classification model;

[0131] The image features are input into a preset information recognition model corresponding to the classification result for information recognition, and a road sign information recognition result corresponding to the road sign image to be recognized is obtained.

[0132] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining model training data, the model training data includes classification model training data and sign information recognition model training data of each classification category; training the initial neural network model through the classification model training data to obtain a preset classification model, training the initial neural network model through the sign information recognition model training data of each classification category to obtain a preset information recognition model corresponding to each classification category.

[0133] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: the initial neural network model is trained according to the classification model training data to obtain a teacher model; the softmax function in the teacher model is modified through knowledge distillation; and the student model is obtained according to the teacher model after the softmax function is modified, and the student model is used as the preset classification model.

[0134] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the classification result includes a first type, the image features corresponding to the first type are input into a preset first recognition model, and the information in the image features is recognized by the preset first recognition model to obtain a first type of road sign information recognition result corresponding to the road sign image to be recognized, and the first type includes prohibitions, warnings or instructions; when the classification result includes a second type, the image features corresponding to the second type are input into a preset second recognition model, and the information in the image features is recognized by the preset second recognition model to obtain a second type of road sign information recognition result corresponding to the road sign image to be recognized, and the second type includes traffic signs; when the classification result includes a third type, the image features corresponding to the third type are input into a preset third recognition model, and the information in the image features is recognized by the preset third recognition model to obtain a third type of road sign information recognition result corresponding to the road sign image to be recognized, and the third type includes traffic lights; and the acquired road sign information recognition results are aggregated to obtain a road sign information recognition result corresponding to the road sign image to be recognized.

[0135] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the classification result includes the first type, the information in the image features is identified by a preset first recognition model to obtain detailed classification category information; the digital information in the image features is identified by a preset OCR text recognition model; and based on the detailed classification category information and the digital information, a road sign information recognition result corresponding to the road sign image to be identified is obtained.

[0136] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the classification result includes the second type, the information in the image features is identified by a preset second recognition model to obtain traffic sign category information; the text information in the image features is identified by a preset OCR text recognition model; and based on the traffic sign category information and text information, a road sign information recognition result corresponding to the road sign image to be identified is obtained.

[0137] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented: a road sign image recognition pool is constructed, and a preset classification model is used to batch process the road sign images to be recognized in the road sign image recognition pool.

[0138] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0139] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0140] The above embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A method for identifying road sign information, the method comprising: Build a road sign image recognition pool; receiving road sign images to be recognized in batches through the road sign image recognition pool, and inputting the images in the road sign image recognition pool into a preset classification model after the number of images in the road sign image recognition pool reaches an upper limit; Extracting classification results corresponding to the road sign images to be identified and image features corresponding to the classification results by using the preset classification model, wherein the preset classification model is used to batch process the road sign images to be identified in the road sign image recognition pool; Inputting the image features into a preset information recognition model corresponding to the classification result for information recognition, and obtaining a road sign information recognition result corresponding to the road sign image to be recognized, wherein the preset information recognition model includes a preset first recognition model, a preset second recognition model, and a preset third recognition model; Inputting the image features into a preset information recognition model corresponding to the classification result to perform information recognition, and obtaining a road sign information recognition result corresponding to the road sign image to be recognized includes: When the classification result includes the first type, the information in the image feature is recognized using the preset first recognition model to obtain detailed classification information and position information of numbers in the image; based on the position information of the numbers in the image, the numerical information in the image feature is recognized using a preset OCR text recognition model; and based on the detailed classification information and the numerical information, a road sign information recognition result corresponding to the road sign image to be recognized is obtained, wherein the first type includes prohibition, warning, or instruction; When the classification result includes the second type, the information in the image feature is recognized by the preset second recognition model to obtain traffic sign category information and position information of the text in the image; based on the position information of the text in the image, the text information in the image feature is recognized by the preset OCR text recognition model; and a road sign information recognition result corresponding to the road sign image to be recognized is obtained based on the traffic sign category information and the text information, wherein the second type includes traffic signs; When the classification result includes a third type, inputting image features corresponding to the third type into a preset third recognition model, identifying information in the image features using the preset third recognition model, and obtaining a third type of road sign information recognition result corresponding to the road sign image to be identified, wherein the third type includes a traffic light; The acquired road sign information recognition results are aggregated to obtain a road sign information recognition result corresponding to the road sign image to be recognized.

2. The method according to claim 1, characterized in that Before extracting the classification result corresponding to the road sign image to be identified and the image features corresponding to the classification result by using a preset classification model, the method further includes: Acquire model training data, wherein the model training data includes classification model training data and sign information recognition model training data of each classification category; The initial neural network model is trained with the classification model training data to obtain a preset classification model, and the initial neural network model is trained with the sign information recognition model training data of each classification category to obtain a preset information recognition model corresponding to each classification category.

3. The method according to claim 2, characterized in that The training of the initial neural network model using the classification model training data to obtain the preset classification model includes: Training the initial neural network model according to the classification model training data to obtain a teacher model; Modify the softmax function in the teacher model through knowledge distillation; According to the teacher model after modifying the softmax function, the student model is obtained, and the student model is used as the preset classification model.

4. The method according to claim 2, characterized in that The preset classification model is a classification model constructed based on BP neural network or perceptron neural network.

5. A road sign information recognition device, characterized in that: The device comprises: A processing pool building module, used to build a road sign image recognition pool; an information acquisition module, configured to receive a batch of road sign images to be recognized from the road sign image recognition pool, and input the images in the road sign image recognition pool into a preset classification model after the number of images in the road sign image recognition pool reaches an upper limit; a classification and recognition module, configured to extract classification results corresponding to the road sign images to be recognized and image features corresponding to the classification results using a preset classification model, wherein the preset classification model is configured to perform batch processing on the road sign images to be recognized in the road sign image recognition pool; an information recognition module, configured to input the image features into a preset information recognition model corresponding to the classification result to perform information recognition, and obtain a road sign information recognition result corresponding to the road sign image to be recognized, wherein the preset information recognition model includes a preset first recognition model, a preset second recognition model, and a preset third recognition model; The information identification module is specifically used for: When the classification result includes the first type, the information in the image feature is recognized using the preset first recognition model to obtain detailed classification information and position information of numbers in the image; based on the position information of the numbers in the image, the numerical information in the image feature is recognized using a preset OCR text recognition model; and based on the detailed classification information and the numerical information, a road sign information recognition result corresponding to the road sign image to be recognized is obtained, wherein the first type includes prohibition, warning, or instruction; When the classification result includes the second type, the information in the image feature is recognized by the preset second recognition model to obtain traffic sign category information and position information of the text in the image; based on the position information of the text in the image, the text information in the image feature is recognized by the preset OCR text recognition model; and a road sign information recognition result corresponding to the road sign image to be recognized is obtained based on the traffic sign category information and the text information, wherein the second type includes traffic signs; When the classification result includes a third type, inputting image features corresponding to the third type into a preset third recognition model, identifying information in the image features using the preset third recognition model, and obtaining a third type of road sign information recognition result corresponding to the road sign image to be identified, wherein the third type includes a traffic light; The acquired road sign information recognition results are aggregated to obtain a road sign information recognition result corresponding to the road sign image to be recognized.

6. The device according to claim 5, characterized in that It also includes a model training module, which is used to: obtain model training data, the model training data including classification model training data and sign information recognition model training data of each classification category; train the initial neural network model through the classification model training data to obtain a preset classification model, and train the initial neural network model through the sign information recognition model training data of each classification category to obtain a preset information recognition model corresponding to each classification category.

7. The device according to claim 6, characterized in that The model training module is also used to: train the initial neural network model according to the classification model training data to obtain a teacher model; modify the softmax function in the teacher model through knowledge distillation; obtain a student model based on the teacher model after the softmax function is modified, and use the student model as the preset classification model.

8. The device according to claim 6, characterized in that The preset classification model is a classification model constructed based on BP neural network or perceptron neural network.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images