Sign recognition method, device and vehicle

Abstract

The application provides a signboard recognition method, device and vehicle, which comprises: acquiring a to-be-recognized image, the to-be-recognized image carrying a blocked signboard element; inputting the to-be-recognized image into a recognition model to obtain a signboard corner point category corresponding to the to-be-recognized image output by the recognition model. The signboard recognition method, device and vehicle provided by the application take the image carrying the blocked signboard element as the input of the recognition model, and the output result is the corner points contained in the blocked part and the blocking part in the corresponding image and the corner point categories, the incomplete image information is automatically recognized and calculated through a multi-layer neural network, the logicality and correlation in the picture containing incomplete information are fully mined, the accurate recognition of the blocked signboard is realized, the delicacy and accuracy of the signboard recognition are improved, and the generation efficiency of the high-precision map is improved.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a method, device, and vehicle for recognizing road signs. Background Technology

[0002] With the development of technologies such as artificial intelligence and autonomous driving, the construction of intelligent transportation has become a research hotspot, and high-precision maps are an indispensable part of intelligent transportation data construction. High-precision maps can contain various traffic signs, such as ground features like lane lines, stop lines, and pedestrian crossings, as well as aerial features like road signs and traffic lights, to provide data support for navigation in applications such as autonomous driving.

[0003] Traffic signs, as information carriers of urban geographic entities, possess navigational functions such as place names, routes, distances, and directions. Furthermore, as infrastructure distributed at urban road intersections, they have unique spatial characteristics and serve as an excellent carrier for the city's basic Internet of Things (IoT). Accurate and efficient generation of traffic signs is crucial for the creation of high-precision maps.

[0004] However, when road signs are obscured, they cannot be recognized because sufficient usable information cannot be obtained from the images captured by the vehicle's cameras, thus affecting the efficiency of high-precision map creation. Summary of the Invention

[0005] This invention provides a method, device, and vehicle for identifying road signs, in order to overcome the shortcomings of existing technologies that cannot identify obscured road signs.

[0006] This invention provides a method for identifying road signs, comprising:

[0007] Acquire an image to be identified, the image to be identified carrying obscured road sign elements;

[0008] The image to be identified is input into the recognition model to obtain the road sign corner point category corresponding to the image to be identified, as output by the recognition model;

[0009] The recognition model is based on sample road sign images and category labels corresponding to the sample road sign images; the recognition model includes a feature extraction layer, a channel separation layer, and a category recognition layer.

[0010] The step of inputting the image to be identified into the recognition model and obtaining the road sign corner point category corresponding to the image to be identified, as output by the recognition model, specifically includes:

[0011] The image to be identified is input into the feature extraction layer to obtain the fused feature image output by the feature extraction layer;

[0012] The fused feature image is input into the channel separation layer to obtain the channel feature image output by the channel separation layer;

[0013] The channel feature image is input into the category recognition layer to obtain the road sign corner point category output by the category recognition layer.

[0014] According to a road sign recognition method provided by the present invention, the step of inputting the image to be recognized into the feature extraction layer and obtaining the fused feature image output by the feature extraction layer includes:

[0015] The image to be identified is subjected to downsampling and convolution operations at different scales to obtain feature images at different scales.

[0016] Based on the feature images at various scales, feature fusion is performed to obtain the fused feature image;

[0017] Each feature fusion is followed by a multi-scale convolution before the next feature fusion.

[0018] According to a road sign recognition method provided by the present invention, the step of inputting the fused feature image into the channel separation layer to obtain the channel feature image output by the channel separation layer includes:

[0019] Perform convolution calculation on the fused feature image to obtain the channel feature image corresponding to each channel;

[0020] The number of channels is determined based on the number of corner points of the road sign contained in a complete road sign element.

[0021] According to the present invention, a road sign recognition method is provided, wherein the recognition model is obtained based on sample road sign images and category labels and corner coordinate information corresponding to the sample road sign images;

[0022] The step of inputting the channel feature image into the category recognition layer to obtain the road sign corner point category output by the category recognition layer specifically includes:

[0023] For the channel feature images corresponding to each channel, perform category recognition and extraction to obtain the category probability set corresponding to each channel;

[0024] Using the category probability set corresponding to the channel, the category of the road sign corner point corresponding to the feature image of the channel, as well as the coordinates of the road sign corner point, are determined.

[0025] According to a method for recognizing road signs provided by the present invention, the step of acquiring the image to be recognized includes:

[0026] If it is determined that the image to be identified does not contain any obscured road sign elements, the non-corner area image is cropped based on the target corner point in the image to be identified.

[0027] The image of the non-corner region is overlaid on the region corresponding to the target corner to generate a new image to be identified;

[0028] The new image to be identified is input into the recognition model to obtain the road sign corner point category corresponding to the new image to be identified, as output by the recognition model.

[0029] The present invention also provides a road sign recognition device, comprising:

[0030] An image acquisition module is used to acquire an image to be identified, wherein the image to be identified carries obscured road sign elements;

[0031] The corner recognition module is used to input the image to be recognized into the recognition model and obtain the road sign corner category corresponding to the image to be recognized, output by the recognition model.

[0032] The recognition model is based on sample road sign images and category labels corresponding to the sample road sign images; the recognition model includes a feature extraction layer, a channel separation layer, and a category recognition layer.

[0033] The corner detection module specifically includes a feature extraction unit, a channel separation unit, and a category recognition unit, wherein:

[0034] The feature extraction unit is used to input the image to be identified into the feature extraction layer and obtain the fused feature image output by the feature extraction layer;

[0035] The channel separation unit is used to input the fused feature image into the channel separation layer and obtain the channel feature image output by the channel separation layer;

[0036] The category recognition unit is used to input the channel feature image into the category recognition layer to obtain the road sign corner point category output by the category recognition layer.

[0037] The present invention also provides a vehicle, including a vehicle body and an identification device disposed on the vehicle body, the identification device being used to perform the road sign identification method as described above.

[0038] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the road sign recognition method as described above.

[0039] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the road sign recognition method as described above.

[0040] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the road sign recognition method as described above.

[0041] The road sign recognition method, device, and vehicle provided by this invention use an image carrying obscured road sign elements as input to a recognition model. The output results are the obscured part of the corresponding image and the corner points contained in the obscured part and their categories. By automatically recognizing and calculating incomplete image information through a multi-layer neural network, the accurate recognition of obscured road signs can be achieved, which can improve the precision and accuracy of road sign recognition, thereby improving the efficiency of high-precision map generation. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0043] Figure 1 This is one of the flowcharts illustrating the road sign recognition method provided by the present invention;

[0044] Figure 2 This is a second flowchart illustrating the road sign recognition method provided by the present invention;

[0045] Figure 3 This is a schematic diagram of the road sign recognition device provided by the present invention;

[0046] Figure 4 This is a structural schematic diagram of the vehicle provided by the present invention;

[0047] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0049] The terms "first," "second," etc., used in this application's specification are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more.

[0050] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” are intended to include the plural forms.

[0051] The terms “comprising” and “including” indicate the presence of the described feature, whole, step, operation, element and / or component, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0052] Figure 1 This is one of the flowcharts illustrating the road sign recognition method provided by the present invention. For example... Figure 1 As shown, the road sign recognition method provided in this embodiment of the invention includes: step 101, acquiring an image to be recognized, wherein the image to be recognized carries road sign elements that are obscured.

[0053] It should be noted that the execution entity of the road sign recognition method provided in this embodiment of the invention is the road sign recognition device. The road sign recognition device can be a central processing unit (CPU) built into a vehicle, or a development board based on a CPU, for information processing and program execution.

[0054] The application scenario of the road sign recognition method provided in this embodiment of the invention is to recognize an image containing an obscured road sign, and obtain the unobscured part of the road sign and all corner points contained in the obscured part, as well as the category of the corner points.

[0055] Specifically, in step 101, the road sign recognition device receives images captured in real time by a camera installed on the vehicle as images to be recognized, and the images to be recognized carry the road sign elements that are obscured.

[0056] Road sign elements are graphic elements belonging to traffic signs that are captured by cameras while vehicles are in motion.

[0057] The road sign element carried in the image to be identified is not limited to one type; it can be a road sign corresponding to multiple traffic signs. Road signs corresponding to traffic signs are usually regular graphics, and this embodiment of the invention does not specifically limit this.

[0058] For example, the road sign corresponding to the traffic sign can be square. A square road sign has four right angles. The right angles in each direction are the corners of the road sign of the corresponding category. Then, in the recognition process, the corners of the road sign elements can be obtained by recognizing the obscured road sign elements.

[0059] Step 102: Input the image to be recognized into the recognition model to obtain the road sign corner point category corresponding to the image to be recognized, output by the recognition model.

[0060] The recognition model is based on sample road sign images and corresponding category labels. The model includes a feature extraction layer, a channel separation layer, and a category recognition layer.

[0061] The step of inputting the image to be identified into the recognition model and obtaining the road sign corner point category corresponding to the image to be identified, as output by the recognition model, specifically includes:

[0062] The image to be identified is input into the feature extraction layer to obtain the fused feature image output by the feature extraction layer.

[0063] The fused feature image is input into the channel separation layer to obtain the channel feature image output by the channel separation layer.

[0064] The channel feature image is input into the category recognition layer to obtain the road sign corner point category output by the category recognition layer.

[0065] It should be noted that the recognition model can be a neural network model. The structure and parameters of the neural network include, but are not limited to, the number of input layers, hidden layers, and output layers, as well as the weight parameters of each layer. This invention does not specifically limit the type and structure of the neural network.

[0066] For example, the recognition model can be a feedforward neural network, which consists of an input layer, hidden layers, and an output layer, wherein:

[0067] The input layer is at the very front of the entire network, directly receiving image data carrying the obscured road sign elements.

[0068] Hidden layers can have one or more layers. They perform weighted summations on the input vector using their own neurons. The calculation formula can be expressed as:

[0069] z = b + w1*x1 + w2*x2 + ... + wm*xm

[0070] Where z is the sum of weights of the hidden layer output, x1, x2, x3...xm are the m feature vectors of each sample, b is the bias, and w1, w2...wm are the weights corresponding to each feature vector.

[0071] The output layer is the last layer, used to output the recognition results of corner point categories. Depending on different needs, the type of recognition result output can be a category vector value, a continuous value generated like linear regression, or other complex types of values ​​or vectors. This embodiment of the invention does not specifically limit this.

[0072] An activation function is a function that operates on neurons in an artificial neural network. It is responsible for mapping the input of the neuron to the output. Logistic regression is performed using activation functions, which converts the weighted sum of the outputs of the hidden layer into a non-linear recognition result. This embodiment of the invention does not specifically limit the type of activation function.

[0073] Preferably, the Softmax function is used for logistic regression processing, which maps the outputs of multiple neurons to the (0,1) interval, which can be understood as probability, thereby performing multi-classification.

[0074] It should be noted that the sample data includes sample road sign images corresponding to the sample data, as well as category labels marked at the corners of the sample road sign images. The sample data is divided into training and test sets according to a certain ratio.

[0075] For example, the ratio of training set to test set in sample data includes, but is not limited to, 9:1, 8:2, etc., and the embodiments of the present invention do not specifically limit this.

[0076] Specifically, in step 102, the road sign recognition device initializes the weight coefficients between each layer of the constructed recognition model. Then, it inputs the labeled data of a set of sample question data and sample answer data from the training set into the neural network under the current weight coefficients, sequentially calculating the outputs of each node in the input layer, hidden layer, and output layer. The cumulative error between the final output of the output layer and its actual connection position state type is used to correct the weight coefficients between each node in the input layer and hidden layer using the gradient descent method. Following this process, the weight coefficients of the input layer and hidden layer can be obtained by traversing all samples in the training set.

[0077] The road sign recognition device restores the recognition model in step 102 based on the weight coefficients of the input layer and hidden layer of the neural network, and inputs each image to be recognized in the test set into the trained recognition model to obtain the recognition result corresponding to the image.

[0078] The recognition result can be a probability value or a label result. This embodiment of the invention does not specifically limit the form of the behavior recognition result.

[0079] If the recognition result can be a probability value, then the probability value can be used to explain the probability that the corner points contained in the image to be recognized belong to the corner point categories corresponding to each direction.

[0080] If the recognition result can be a label result, an intermediate numerical result can be obtained through the model. If the numerical result meets the pre-set target conditions, the corner point corresponding to the numerical result will be assigned a corner point category label with the corresponding orientation.

[0081] Preferably, the recognition model built into the road sign recognition device consists of an input layer, a hidden layer, and an output layer. The hidden layer's function is to extract features from the input image, which contains incomplete information, using its own neurons, thus extracting feature information that is beneficial for recognition.

[0082] The embodiments of the present invention do not specifically limit the structure of the hidden layer.

[0083] Preferably, the hidden layer comprises at least three layers: a feature extraction layer, a channel separation layer, and a category recognition layer, wherein:

[0084] The feature extraction layer can use a convolutional neural network (CNN) to reduce the dimensionality of the image to be recognized, compress the vector, and extract and fuse features to obtain a fused feature image.

[0085] In the convolution process, dilated convolution or dilated convolution can be used to increase the receptive field without pooling and losing information, so that each convolution output contains a larger range of information.

[0086] The channel separation layer can set the number of channels of the convolution kernel by the number of corner points contained in the road sign, and perform convolution calculation on the fused feature image again to obtain the two-dimensional channel feature image corresponding to each channel.

[0087] The category recognition layer can sequentially process the two-dimensional channel feature images corresponding to each channel through fully connected processing and Softmax processing to map them into two-dimensional vectors. Based on these two-dimensional vectors, classification processing is performed to obtain the corner point category of the road sign. This allows us to know all the corner points contained in the image to be recognized, as well as their corresponding corner point categories.

[0088] This invention uses an image containing obscured road sign elements as input to a recognition model. A feature extraction layer extracts features from the image to be recognized. A channel separation layer separates the channels of the fused feature image output from the feature extraction layer. A category recognition layer then performs fully connected processing on the channel feature images of each channel, outputting the obscured portion of the image and the corner points contained within the obscured portion, along with their categories. Through a multi-layered neural network, incomplete image information is automatically identified and calculated, fully exploiting the logic and correlation within images containing incomplete information. This achieves accurate recognition of obscured road signs, improving the precision and accuracy of road sign recognition, and consequently increasing the efficiency of high-precision map generation.

[0089] Based on any of the above embodiments, the image to be identified is input into the feature extraction layer to obtain the fused feature image output by the feature extraction layer, including: performing downsampling and convolution operations on the image to be identified at different scales to obtain feature images at different scales.

[0090] Based on the feature images at various scales, feature fusion is performed to obtain the fused feature image.

[0091] Each feature fusion is followed by a multi-scale convolution before the next feature fusion.

[0092] It should be noted that the road sign recognition device uses a large number of convolutional kernels to extract high-dimensional feature maps from the matrix corresponding to the remote sensing image. Multiple feature maps can be extracted, each of which is a local perception extracted from the image. By combining these feature maps, the part of interest in the image can be extracted.

[0093] Different sizes of convolution kernels have different extraction effects on different types of road signs. In this embodiment of the invention, the number and size of convolution kernels for multi-scale downsampling are not specifically limited.

[0094] Specifically, a small 1x1 convolutional kernel can extract relevant feature data for recognizing small road signs. A medium 3x3 convolutional kernel can extract relevant feature information for recognizing medium-sized road signs. A large 5x5 convolutional kernel can extract relevant feature information for recognizing large road signs. Therefore, when fusing feature data of different scales, it is possible to retain relevant information for road signs of three different sizes, thus preventing the loss of original valid data.

[0095] It is important to note that convolution calculations are required after each fusion to prevent the introduction of noise.

[0096] Specifically, the road sign recognition device inputs the image to be recognized into a cascaded convolutional layer. First, it simultaneously enters the convolution kernels of different sizes to perform convolution calculations, obtaining feature images of different dimensions. By linearly adding the feature images of multiple dimensions obtained in this convolution, a fused feature image is obtained. Then, it simultaneously enters the next level of convolution kernels to repeat the above process, continuously performing feature extraction and feature fusion processes to update the fused feature image.

[0097] For example, five convolutional kernels of size 3*3 with channels of 128, 64, 32, 16 and 8 can be set sequentially. Correspondingly, five convolutional kernels of size 5*5 with channels of 128, 64, 32, 16 and 8 can be set for convolution processing. This will cause the height and width of the feature map to be halved and the number of channels to be halved after each convolution process. After performing five convolutions and linear addition, a fused feature image with 8 channels can be obtained.

[0098] It should be noted that in order to linearly add feature images of multiple scales, padding is required to ensure that the feature images calculated by convolution have the same number of rows and columns.

[0099] This invention relates to a method for obtaining a fused feature image by performing convolution calculations at different scales on the image to be recognized, and then fusing and performing dimensionality reduction on the multi-scale feature images obtained from the convolutions multiple times. This allows the image to simultaneously contain low-level detailed features and high-level semantic features, improving the precision of image semantic segmentation and thus enhancing the accuracy of image recognition.

[0100] Based on any of the above embodiments, the fused feature image is input to the channel separation layer to obtain the channel feature image output by the channel separation layer, including: performing convolution calculation on the fused feature image to obtain the channel feature image corresponding to each channel.

[0101] The number of channels is determined based on the number of corner points of the road sign contained in a complete road sign element.

[0102] It should be noted that before step 102, the number of channels needs to be set according to the number of corner points contained in the road sign elements of different specifications, so that each channel corresponds to the feature corresponding to each corner point.

[0103] Specifically, the road sign recognition device first performs convolution calculations on the fused feature image using a convolution kernel with 4 channels to obtain the feature data of each channel, and then performs fully connected dimensionality reduction using a 1x1 convolution kernel to obtain the width and height dimensions and the original dimensions. Figure 1 Channel feature images of multiple channels.

[0104] In this embodiment of the invention, after determining the number of channels based on the number of corner points of the road sign elements, convolution calculations are then performed on the fused feature image for the corresponding number of channels to separate the channel feature images of each channel. This can improve the precision of image semantic segmentation, thereby improving the accuracy of image recognition.

[0105] Based on any of the above embodiments, the recognition model is obtained based on sample road sign images, as well as category labels and corner coordinate information corresponding to the sample road sign images.

[0106] The step of inputting the channel feature image into the category recognition layer and obtaining the road sign corner point category output by the category recognition layer specifically includes: performing category recognition and extraction on the channel feature image corresponding to each channel, and obtaining the category probability set corresponding to each channel.

[0107] Specifically, during the training of the recognition model, the road sign recognition device uses the corner point categories corresponding to each channel and the corner point coordinate information corresponding to the corner points of the identification category as ground values ​​for training. It then recognizes the feature images of each channel and maps the pixel value of each pixel in the channel feature image to the probability value of the corner point category corresponding to different channels, integrating them into a category probability set.

[0108] For any channel feature image, among the multiple category probability values ​​corresponding to each pixel, the category probability corresponding to its channel should be the maximum value.

[0109] Using the category probability set corresponding to the channel, the category of the road sign corner point corresponding to the feature image of the channel, as well as the coordinates of the road sign corner point, are determined.

[0110] Specifically, the road sign recognition device outputs the corner point category corresponding to the maximum value among multiple probability values ​​for each pixel from the category probability set corresponding to each channel, and determines the corner point coordinates of the road sign based on its corresponding corner point category.

[0111] For example, when recognizing a square road sign in an image, regardless of whether it is a small, medium, or large square road sign, all three sizes of square road signs have four corner points. The corner point categories corresponding to the top left, top right, bottom right, and bottom left corners of the road sign element are defined and numbered as 0, 1, 2, and 3, respectively.

[0112] Therefore, by performing ordinary convolution on the fused feature image to be processed, the number of ordinary convolution channels is equal to 4, resulting in 4-channel feature images. When recognizing the channel feature image corresponding to the channel with category number 0, the pixels with probability values ​​greater than a preset threshold in the category probability set are identified as corner point category 0, and the corresponding pixel coordinates are output as the coordinates of the upper left corner of the road sign.

[0113] When recognizing the channel feature image corresponding to the channel with category number 1, the pixel corresponding to the probability value greater than the preset threshold in the category probability set is identified as corner point category 1, and the corresponding pixel coordinates are output as the coordinates of the upper right corner of the road sign.

[0114] When recognizing the channel feature image corresponding to channel number 2, the pixel points corresponding to the probability values ​​in the category probability set that are greater than the preset threshold are identified as corner point category 2, and the corresponding pixel coordinates are output as the coordinates of the lower right corner of the road sign.

[0115] When recognizing the channel feature image corresponding to channel number 3, the pixels with probability values ​​greater than a preset threshold in the category probability set are identified as corner point category 3, and the corresponding pixel coordinates are output as the coordinates of the lower left corner of the road sign.

[0116] This invention uses an image containing obscured road sign elements as input to a recognition model. The output is the obscured portion of the image, the corner points contained in the obscured portion, the corner point category, and the corner point coordinates. Through a multi-layer neural network, incomplete image information is automatically identified and calculated, thus achieving accurate recognition of obscured road signs. This improves the precision and accuracy of road sign recognition, thereby increasing the efficiency of high-precision map generation.

[0117] Based on any of the above embodiments, the step of obtaining the image to be identified includes: if it is determined that the image to be identified does not carry any obscured road sign elements, cropping the non-corner area image based on the target corner point in the image to be identified.

[0118] Specifically, in step 101, the road sign recognition device performs preliminary screening and recognition on the image to be recognized captured by the camera. If it is found that the image to be recognized does not contain any obscured road sign elements, one or more target corner points are extracted from the image. A pixel at a certain pixel distance from the target corner point is taken as the starting point of the non-corner area, and any pixel in the direction away from the target corner point is selected as the ending point of the non-corner area, so as to extract the non-corner area image within the defined non-corner area.

[0119] The image of the non-corner region is overlaid on the region corresponding to the target corner to generate a new image to be identified.

[0120] Specifically, the road sign recognition device first determines the size of the non-corner area image. Then, it overlaps the starting point of the target corner with the non-corner area, so that the corresponding size of the non-corner area image covers the area where the target corner is located, generating a new image to be recognized. This allows the recognition model to identify the category of the target corner from the covered area in the new image to be recognized.

[0121] The embodiments of the present invention do not specifically limit the specific implementation of the identification process.

[0122] Preferably, Figure 2 This is the second flowchart illustrating the road sign recognition method provided by this invention. For example... Figure 2 As shown, the entire recognition process includes a training process and a testing process.

[0123] (a) The training process involves sequentially inputting the sample road sign images into the feature extraction layer, channel separation layer, and category recognition layer for processing. The obtained sample road sign corner point categories and the category labels corresponding to the sample road sign images are then subjected to gradient descent to train the recognition model.

[0124] (ii) During the test, the images to be identified captured by the camera must first be preprocessed.

[0125] If the image to be identified contains occluded corner points, the image to be identified is directly input into the trained recognition model, which processes the model sequentially through the feature extraction layer, channel separation layer, and category recognition layer. Finally, the corresponding corner point category is identified from the occluded area in the image to be identified.

[0126] If the image to be identified does not contain any occluded corners, then the non-corner areas in the image to be identified need to be covered over the corner areas to form occluded corners. The new image to be identified is then input into the trained recognition model, and processed sequentially through the feature extraction layer, the channel separation layer, and the category recognition layer. Finally, the corresponding corner category is identified from the covered area in the new image to be identified.

[0127] The embodiments of the present invention are based on the prediction of whether the image to be identified contains occluded corner points, so as to realize the active corner point occlusion of complete image information, which can improve the robustness of the recognition model and thus improve the generation efficiency of high-precision maps.

[0128] Figure 3 This is a schematic diagram of the road sign recognition device provided by the present invention. Based on any of the above embodiments, such as... Figure 3 As shown, the device includes an image acquisition module 310 and a corner recognition module 320, wherein:

[0129] Image acquisition module 310 is used to acquire an image to be identified, the image to be identified carrying obscured road sign elements.

[0130] The corner recognition module 320 is used to input the image to be recognized into the recognition model and obtain the road sign corner category corresponding to the image to be recognized, output by the recognition model.

[0131] The recognition model is based on sample road sign images and corresponding category labels. The recognition model includes a feature extraction layer, a channel separation layer, and a category recognition layer.

[0132] The corner recognition module 320 specifically includes a feature extraction unit 321, a channel separation unit 322, and a category recognition unit 323, wherein:

[0133] The feature extraction unit 321 is used to input the image to be identified into the feature extraction layer and obtain the fused feature image output by the feature extraction layer.

[0134] The channel separation unit 322 is used to input the fused feature image into the channel separation layer and obtain the channel feature image output by the channel separation layer.

[0135] The category recognition unit 323 is used to input the channel feature image into the category recognition layer to obtain the road sign corner point category output by the category recognition layer.

[0136] Specifically, the image acquisition module 310 and the corner recognition module 320 are electrically connected in sequence.

[0137] The image acquisition module 310 receives images captured in real time by a camera installed in the vehicle as images to be identified, and the images to be identified carry obscured road sign elements.

[0138] The corner detection module 320 initializes the weight coefficients between each layer of the constructed detection model. Then, it inputs the labeled data of a set of sample question data and sample answer data from the training set into the neural network under the current weight coefficients, sequentially calculating the output of each node in the input layer, hidden layer, and output layer. The cumulative error between the final output of the output layer and its actual connection position state type is used to correct the weight coefficients between each node in the input layer and hidden layer using the gradient descent method. Following this process, until all samples in the training set have been traversed, the weight coefficients of the input layer and hidden layer can be obtained.

[0139] The road sign recognition device reconstructs the recognition model based on the weight coefficients of the input layer and hidden layer of the neural network, and inputs each image to be recognized in the test set into the trained recognition model to obtain the recognition result corresponding to the image.

[0140] Optionally, the feature extraction unit 321 includes a multi-scale convolution sub-unit and a fusion sub-unit, wherein:

[0141] The multi-scale convolutional subunit is used to perform downsampling and convolution operations on the image to be identified at different scales to obtain feature images at different scales.

[0142] The fusion subunit is used to perform feature fusion based on feature images at various scales to obtain the fused feature image.

[0143] Each feature fusion is followed by a multi-scale convolution before the next feature fusion.

[0144] Optionally, the channel separation unit 322 is specifically used to perform convolution calculation on the fused feature image to obtain the channel feature image corresponding to each channel.

[0145] The number of channels is determined based on the number of corner points of the road sign contained in a complete road sign element.

[0146] Optionally, the recognition model is obtained based on sample road sign images, as well as category labels and corner coordinate information corresponding to the sample road sign images.

[0147] Accordingly, the category recognition unit 323 includes a probability set acquisition subunit and a recognition subunit, wherein:

[0148] The probability set acquisition sub-unit is used to identify and extract the category of the channel feature image corresponding to each channel, and obtain the category probability set corresponding to each channel.

[0149] The identification subunit is used to determine the category of the road sign corner point corresponding to the channel feature image and the coordinates of the road sign corner point by using the category probability set corresponding to the channel.

[0150] Optionally, the image acquisition module 310 includes a cropping unit and a coverage unit, wherein:

[0151] The cropping unit is used to crop a non-corner area image based on the target corner point in the image to be identified, provided that the image to be identified does not contain any obscured road sign elements.

[0152] The overlay unit is used to overlay the non-corner area image onto the area corresponding to the target corner point, thereby generating a new image to be identified.

[0153] The road sign recognition device provided in this embodiment of the invention is used to execute the road sign recognition method of the present invention. Its implementation method is the same as that of the road sign recognition method provided by the present invention, and it can achieve the same beneficial effects, so it will not be described again here.

[0154] This invention uses an image containing obscured road sign elements as input to a recognition model. A feature extraction layer extracts features from the image to be recognized. A channel separation layer separates the channels of the fused feature image output from the feature extraction layer. A category recognition layer then performs fully connected processing on the channel feature images of each channel, outputting the obscured portion of the image and the corner points contained within the obscured portion, along with their categories. Through a multi-layered neural network, incomplete image information is automatically identified and calculated, fully exploiting the logic and correlation within images containing incomplete information. This achieves accurate recognition of obscured road signs, improving the precision and accuracy of road sign recognition, and consequently increasing the efficiency of high-precision map generation. Figure 4 This is a structural schematic diagram of the vehicle provided by the present invention. Based on any of the above embodiments, such as... Figure 4 As shown, the vehicle includes a vehicle body 410 and an identification device 420 disposed on the vehicle body 410. The identification device is used to perform the road sign identification method described above.

[0155] Specifically, the vehicle consists of at least a vehicle body 410 and an identification device 420 embedded in a development board of the vehicle body 410.

[0156] The development board of the vehicle body 410 is connected to the identification device 420 for long-distance transmission communication via wireless communication technology (Wi-Fi), Bluetooth or serial port, etc. This embodiment of the invention does not specifically limit this.

[0157] This invention uses an image containing obscured road sign elements as input to a recognition model. A feature extraction layer extracts features from the image to be recognized. A channel separation layer separates the channels of the fused feature image output from the feature extraction layer. A category recognition layer then performs fully connected processing on the channel feature images of each channel, outputting the obscured portion of the image and the corner points contained within the obscured portion, along with their categories. Through a multi-layered neural network, incomplete image information is automatically identified and calculated, fully exploiting the logic and correlation within images containing incomplete information. This achieves accurate recognition of obscured road signs, improving the precision and accuracy of road sign recognition, and consequently increasing the efficiency of high-precision map generation. Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communications bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other through the communications bus 540. The processor 510 can call logical instructions in the memory 530 to execute a road sign recognition method. This method includes: acquiring an image to be recognized, the image carrying obscured road sign elements; inputting the image to be recognized into a recognition model to obtain the road sign corner point category output by the recognition model corresponding to the image to be recognized; wherein the recognition model is based on sample road sign images and category labels corresponding to the sample road sign images; the recognition model includes a feature extraction layer, a channel separation layer, and a category recognition layer; specifically, inputting the image to be recognized into the recognition model to obtain the road sign corner point category output by the recognition model includes: inputting the image to be recognized into the feature extraction layer to obtain a fused feature image output by the feature extraction layer; inputting the fused feature image into the channel separation layer to obtain a channel feature image output by the channel separation layer; and inputting the channel feature image into the category recognition layer to obtain the road sign corner point category output by the category recognition layer.

[0158] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0159] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the road sign recognition method provided by the above methods. The method includes: acquiring an image to be recognized, the image to be recognized carrying obscured road sign elements; inputting the image to be recognized into a recognition model to obtain the road sign corner point category corresponding to the image to be recognized output by the recognition model; wherein, the recognition model is based on sample road sign images and the road sign corner point category corresponding to the sample road sign images. The identification model is obtained by corresponding category labels; the identification model includes a feature extraction layer, a channel separation layer, and a category identification layer; the step of inputting the image to be identified into the identification model to obtain the road sign corner point category corresponding to the image to be identified output by the identification model specifically includes: inputting the image to be identified into the feature extraction layer to obtain the fused feature image output by the feature extraction layer; inputting the fused feature image into the channel separation layer to obtain the channel feature image output by the channel separation layer; and inputting the channel feature image into the category identification layer to obtain the road sign corner point category output by the category identification layer.

[0160] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a road sign recognition method provided by the above methods. The method includes: acquiring an image to be recognized, the image carrying obscured road sign elements; inputting the image to be recognized into a recognition model to obtain a road sign corner point category corresponding to the image to be recognized, output by the recognition model; wherein the recognition model is based on sample road sign images and category labels corresponding to the sample road sign images; the recognition model includes a feature extraction layer, a channel separation layer, and a category recognition layer; the step of inputting the image to be recognized into the recognition model to obtain the road sign corner point category corresponding to the image to be recognized specifically includes: inputting the image to be recognized into the feature extraction layer to obtain a fused feature image output by the feature extraction layer; inputting the fused feature image into the channel separation layer to obtain a channel feature image output by the channel separation layer; and inputting the channel feature image into the category recognition layer to obtain the road sign corner point category output by the category recognition layer.

[0161] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0162] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for recognizing road signs, characterized in that, include: Acquire an image to be identified, the image to be identified carrying obscured road sign elements; The image to be identified is input into the recognition model to obtain the road sign corner point category and road sign corner point coordinates corresponding to the image to be identified, as output by the recognition model. The recognition model is based on sample road sign images and category labels corresponding to the sample road sign images; the recognition model includes a feature extraction layer, a channel separation layer, and a category recognition layer. The step of inputting the image to be recognized into the recognition model to obtain the road sign corner point category and road sign corner point coordinates corresponding to the image to be recognized, as output by the recognition model, specifically includes: The image to be identified is input into the feature extraction layer. The feature extraction layer performs downsampling and convolution operations on the image to be identified at different scales to obtain feature images at different scales. Based on the feature images at each scale, feature fusion is performed to obtain a fused feature image. After each feature fusion, a multi-scale convolution is performed before the next feature fusion. The fused feature image is input into the channel separation layer, and convolution calculation is performed on the fused feature image to obtain the channel feature image corresponding to each channel. The number of channels is determined according to the number of corner points of the road sign contained in the complete road sign element. The channel feature image is input into the category recognition layer. The category is identified and extracted for the channel feature image corresponding to each channel to obtain the category probability set corresponding to each channel. The category recognition layer maps the two-dimensional channel feature image corresponding to each channel into a two-dimensional vector and performs classification processing based on the two-dimensional vector to obtain the road sign corner point category. Using the category probability set corresponding to the channel, the category of the road sign corner point corresponding to the feature image of the channel, as well as the coordinates of the road sign corner point, are determined.

2. The road sign recognition method according to claim 1, characterized in that, The recognition model is based on sample road sign images, as well as category labels and corner coordinate information corresponding to the sample road sign images.

3. The method for identifying road signs according to claim 1 or 2, characterized in that, The acquisition of the image to be identified includes: If it is determined that the image to be identified does not contain any obscured road sign elements, the non-corner area of ​​the image is cropped based on the target corner point in the image to be identified. The image of the non-corner region is overlaid on the region corresponding to the target corner to generate a new image to be identified.

4. A road sign recognition device, characterized in that, include: An image acquisition module is used to acquire an image to be identified, wherein the image to be identified carries obscured road sign elements; The corner recognition module is used to input the image to be recognized into the recognition model and obtain the corner category and corner coordinates of the road sign corresponding to the image to be recognized, as output by the recognition model. The recognition model is based on sample road sign images and category labels corresponding to the sample road sign images; the recognition model includes a feature extraction layer, a channel separation layer, and a category recognition layer. The corner detection module specifically includes a feature extraction unit, a channel separation unit, and a category recognition unit, wherein: The feature extraction unit is used to input the image to be identified into the feature extraction layer, and to perform downsampling and convolution operations on the image to be identified at different scales through the feature extraction layer to obtain feature images at different scales. Based on the feature images at each scale, feature fusion is performed to obtain a fused feature image. After each feature fusion, a multi-scale convolution is performed before the next feature fusion. The channel separation unit is used to input the fused feature image into the channel separation layer, perform convolution calculation on the fused feature image, and obtain the channel feature image corresponding to each channel. The number of channels is determined according to the number of corner points of the road sign contained in the complete road sign element. The category recognition unit is used to input the channel feature image into the category recognition layer, perform category recognition and extraction on the channel feature image corresponding to each channel, and obtain the category probability set corresponding to each channel; using the category probability set corresponding to the channel, determine the category of the road sign corner point corresponding to the channel feature image, as well as the coordinates of the road sign corner point.

5. A vehicle, comprising a vehicle body, characterized in that, It also includes an identification device disposed on the vehicle body, the identification device being used to perform the road sign identification method according to any one of claims 1 to 3.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the road sign recognition method as described in any one of claims 1 to 3.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the road sign recognition method as described in any one of claims 1 to 3.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the road sign recognition method as described in any one of claims 1 to 3.

Images