Intelligent traffic sign recognition method and device
Through the high-resolution neural network model, street sign corner detection and geographical coordinate generation are carried out on the images acquired by the vehicle camera, which solves the problem of artificial participation in the existing technology that affects the efficiency of high-precision maps, realizes the automatic and rapid generation of street signs, and improves the high-level Exquisite map drawing efficiency.
Patent Information
- Application Number
- CN202210058679.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-17
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2042-01-17
AI Technical Summary
In the prior art, the images acquired by the vehicle camera cannot fully automatically generate street signs, and human participation is required, which affects the efficiency of high-precision map drawing.
The preset high-resolution neural network model is adopted to obtain the corner point detection of road signs in at least two images, and combine geographical location information, and automatically generate the geographical coordinate position of the street sign to achieve rapid generation of street signs.
It realizes the automatic generation of street signs, improves the efficiency of drawing high-precision maps, and can generate multiple street signs at the same time.
Smart Images

Figure CN114419594B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of navigation technology, and in particular to a method and device for recognizing smart traffic signs. Background Art
[0002] With the development of technologies such as artificial intelligence and autonomous driving, building smart transportation has also become a research hotspot, and high-precision maps are an indispensable part of building smart transportation data. High-precision maps can contain a variety of traffic signs. For example, detailed lane maps can be used to express ground feature elements such as lane lines, stop lines, and crosswalks in the real world, as well as high-altitude feature elements such as road signs and traffic lights, so as to provide data support for navigation in application scenarios such as autonomous driving.
[0003] Traffic signs are information carriers of urban geographic entities, with navigation functions such as place names, routes, distances and directions. As infrastructure distributed at intersections of urban roads, they have their own particularity in space and are a good carrier of the city's basic Internet of Things. Correctly and efficiently generating traffic signs is critical to the drawing of high-precision maps.
[0004] However, in the related art, the images obtained by using the vehicle's camera cannot fully automatically generate the road signs contained in the image. The process usually requires human intervention, which in turn affects the efficiency of high-precision map drawing. Summary of the invention
[0005] In order to solve or partially solve the problems existing in the related technology, the present application provides a method and device for identifying smart traffic road signs, which can automatically generate road signs and realize the rapid generation of at least one road sign, thereby improving the efficiency of drawing high-precision maps.
[0006] The first aspect of the present application provides a method for recognizing a smart traffic sign, comprising:
[0007] Acquire at least two images, each of the images containing at least one common road sign;
[0008] Using a preset high-resolution neural network model to perform road sign corner point detection on at least one common road sign in each of the images, and identifying the pixel coordinate positions of each road sign corner point corresponding to at least one road sign in each of the images;
[0009] The geographic coordinate position of each road sign is generated according to the pixel coordinate positions of each road sign corner point corresponding to each road sign in the at least two images and the geographic location information of the camera when the at least two images are taken.
[0010] In one embodiment, the method further comprises:
[0011] A preset high-resolution neural network model is trained in advance using a preset training data set, wherein the preset training data set includes multiple training images, the road sign corner points of each road sign in each of the training images are annotated, and the road sign corner points of different road signs are annotated with different labels.
[0012] In one implementation, the method of labeling the training image includes:
[0013] A probability region with a Gaussian distribution is drawn at the position of the corner point of the road sign in the training image, the probability region is divided into a plurality of blocks, the plurality of blocks are distributed in an N×M matrix, the corner point of the road sign is located in a block at the center position of the probability region; wherein N and M are both positive integers; a highest probability value is marked on the block where the corner point of the road sign is located, and the probability values marked on each of the blocks in the probability region constitute a Gaussian distribution.
[0014] In one embodiment, the method further comprises:
[0015] The variance of the Gaussian distribution in the probability region is modified so that the plurality of blocks are distributed in a P×P matrix, wherein P is a set target value, and P is a positive integer and is smaller than N and M, respectively.
[0016] In one embodiment, the length and width of each of the images meet the set conditions;
[0017] The method also includes: pre-setting input parameters of the preset high-resolution neural network model according to the set conditions.
[0018] In one embodiment, the method further comprises:
[0019] Pre-adjusting the number of feature map channels of the backbone network in the preset high-resolution neural network model to a set value; and / or,
[0020] A high-resolution newly added feature map is generated in advance in the backbone network of the preset high-resolution neural network model, so that the preset high-resolution neural network model performs road sign corner point detection based on the original feature map in the backbone network and the newly added feature map.
[0021] In one embodiment, the step of pre-generating a high-resolution newly added feature map in the backbone network of the preset high-resolution neural network model includes:
[0022] A new feature map with a resolution of 1 / 1 of the input image is generated in advance through a deconvolution operation in the backbone network of the preset high-resolution neural network model.
[0023] In one embodiment, the preset high-resolution neural network model is a HigherHrNet high-resolution neural network model.
[0024] The second aspect of the present application provides a smart traffic sign recognition device, comprising:
[0025] An acquisition module, used for acquiring at least two images, each of which contains at least one common road sign;
[0026] A recognition module, used to detect road sign corner points of at least one common road sign in each image acquired by the acquisition module using a preset high-resolution neural network model, and identify the pixel coordinate positions of each road sign corner point corresponding to at least one road sign in each image;
[0027] A generation module is used to generate the geographic coordinate position of each road sign according to the pixel coordinate positions of each road sign corner point corresponding to each road sign identified by the recognition module in at least two images, and the geographic location information of the camera when taking the at least two images.
[0028] In one embodiment, the device further comprises:
[0029] A construction module is used to pre-generate a high-resolution newly added feature map in the backbone network of the preset high-resolution neural network model, so that the preset high-resolution neural network model performs road sign corner point detection based on the original feature map in the backbone network and the newly added feature map.
[0030] A third aspect of the present application provides an electronic device, including:
[0031] Processor; and
[0032] The memory stores executable codes thereon, and when the executable codes are executed by the processor, the processor is caused to execute the method as described above.
[0033] A fourth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as described above.
[0034] The technical solution provided by this application may have the following beneficial effects:
[0035] The method provided by the present application obtains at least two images, each of which contains at least one common road sign, and uses a preset high-resolution neural network model to detect the road sign corner points of at least one common road sign in each image, and identifies the pixel coordinate positions of the road sign corner points corresponding to at least one road sign in each image. According to the pixel coordinate positions of the road sign corner points corresponding to each road sign in at least two images, and the geographical location information of the camera when shooting at least two images, the geographical coordinate position of each road sign can be generated. In this way, the automatic generation of road signs is realized, and the rapid generation of at least one road sign is realized, so that multiple road signs can be generated at the same time, which effectively improves the efficiency of drawing high-precision maps.
[0036] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.
[0038] Figure 1 It is a flowchart of a method for identifying a smart traffic sign according to an embodiment of the present application;
[0039] Figure 2 is a schematic diagram of the network structure of a preset high-resolution neural network model shown in an embodiment of the present application;
[0040] Figure 3 It is a structural schematic diagram of a smart traffic sign recognition device shown in an embodiment of the present application;
[0041] Figure 4 It is another structural schematic diagram of the intelligent traffic sign recognition device shown in the embodiment of the present application;
[0042] Figure 5 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION
[0043] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.
[0044] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.
[0045] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.
[0046] In the related art, the images obtained by using the vehicle's camera cannot fully and automatically generate the road signs contained in the images. The process usually requires human intervention, which in turn affects the efficiency of high-precision map drawing.
[0047] In response to the above problems, an embodiment of the present application provides a method for identifying smart traffic road signs, which can automatically generate road signs and realize the rapid generation of at least one road sign, thereby improving the efficiency of drawing high-precision maps.
[0048] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.
[0049] Figure 1 It is a flow chart of the method for identifying smart traffic signs shown in an embodiment of the present application.
[0050] See also Figure 1 , the method comprising:
[0051] Step S101: Acquire at least two images, each of which contains at least one common road sign.
[0052] In the embodiment of the present application, the video data during driving can be collected by a camera device, wherein the camera device may include but is not limited to a driving recorder, a camera or a driver's mobile phone installed on the vehicle and other devices with a camera function. Among them, the camera device can be a monocular camera device. The camera device can be set at the front of the vehicle to record the road sign in front of the vehicle, so as to obtain a continuous video image containing the road sign, so as to obtain at least two images in this step. In order to process the image later, it is necessary to extract frames from the video data containing the road sign obtained during the vehicle driving. Usually, the frame rate of the video is 30 frames per second, and the video can be extracted according to a preset rule, for example, 10 frames, 15 frames, 20 frames or other values are extracted per second, and the time interval between two adjacent frames of the image is the frame extraction time interval, so as to obtain multiple images taken. In addition, the camera device will record the shooting time of the image while shooting the image. In the embodiment of the present application, the camera device for collecting images is regarded as a camera.
[0053] In this step, two images may be acquired, or more than two images may be acquired. Each image may contain at least one common road sign. The common road sign is a road sign that exists in each image. For example, a road sign A in the real world is captured by a camera and imaged in each of at least two images, then the common road sign contained in each image may be road sign A.
[0054] Further, in one embodiment, each image may include at least two common road signs. For example, for road signs A and B in the real world, both road signs A and B are captured by the camera and imaged in each image, and the common road signs included in each image are road signs A and B. For another example, for road signs A, B, and C in the real world, both road signs A, B, and C are captured by the camera and imaged in each image, and the common road signs included in each image are road signs A, B, and C.
[0055] Step S102: Detect road sign corner points of at least one common road sign in each image using a preset high-resolution neural network model, and identify the pixel coordinate positions of each road sign corner point corresponding to at least one road sign in each image.
[0056] The preset high-resolution neural network model may be a HigherHrNet high-resolution neural network model. The HigherHrNet high-resolution neural network model is a high-resolution neural network model that uses a high-resolution feature pyramid to learn scale-aware representations. It has a multi-resolution aggregation function in reasoning, so it is usually used for human posture estimation and recognition. It can better solve the scale change challenge in bottom-up multi-person posture estimation and can more accurately locate key points, especially for small people.
[0057] In an embodiment of the present application, a HigherHrNet high-resolution neural network model can be used as a preset high-resolution neural network model implemented in the present application, and the HigherHrNet high-resolution neural network model can be used to detect the corner points of at least one common road sign in each image. It can be understood that the use of the HigherHrNet high-resolution neural network model to detect human bodies in images in the related art can detect different target human bodies in the image and identify the positions of the key skeletal points in each target human body in the image. In an embodiment of the present application, the HigherHrNet high-resolution neural network model can be trained in advance, so that the trained HigherHrNet high-resolution neural network model can be used to detect the corner points of road signs in each image, and then the pixel coordinate positions of the corner points of each road sign corresponding to each road sign in each image can be identified. The HigherHrNet high-resolution neural network model can output the pixel coordinate positions of the corner points of each road sign identified for each image, and the corner points of the road signs of different road signs will be marked with different labels. For example, an image contains road sign A and road sign B. The HigherHrNet high-resolution neural network model can detect and identify each road sign corner point of road sign A and each road sign corner point of road sign B, where each road sign corner point of road sign A is labeled with a first type of label, and each road sign corner point of road sign B is labeled with a second type of label. Different types of labels represent different road signs.
[0058] In one embodiment, a preset high-resolution neural network model may be pre-trained using a preset training data set, wherein the preset training data set includes a plurality of training images, and the road sign corner points of each road sign in each training image have been annotated, and the road sign corner points of different road signs are annotated using different labels. For example, if the training image contains two road signs, namely road sign A and road sign B, then each road sign corner point of road sign A in the training image may be annotated using a first type of label, and each road sign corner point of road sign B in the training image may be annotated using a second type of label. In this way, by using the preset high-resolution neural network model after training to detect road sign corner points in each image, the recognition result of the road sign corner points in each road sign may be more accurate.
[0059] Furthermore, in one of the embodiments, the method of annotating the training image includes: drawing a probability area with a Gaussian distribution at the location of the corner point of the road sign in the training image, the probability area is divided into multiple blocks, the multiple blocks are distributed in an N×M matrix, and the corner point of the road sign is located in a block at the center position of the probability area; wherein N and M are both positive integers; marking a highest probability value on the block where the corner point of the road sign is located, and the probability values marked on each block on the probability area constitute a Gaussian distribution. For example, a probability region with Gaussian distribution can be drawn for the position of a corner point a1 of a road sign A in a training image. The probability region is divided into 81 blocks, which are distributed in a 9×9 matrix. The block where the corner point a1 of the road sign is located is a block at the center of the 81 blocks. The block at the center can be marked with a highest probability value (for example, 0.8), and other blocks are marked with lower probability values; the probability values marked on other blocks are inversely proportional to the distance between the block and the block at the center; that is, the farther away from the center of the probability region, the lower the marked probability value. It can be understood that the corner points of different road signs in a training image can be marked with different labels to divide different road signs. For example, each corner point of the road sign A in the training image can be marked with a first type of label, and each corner point of another road sign B in the training image can be marked with a second type of label.
[0060] In order to improve the efficiency and accuracy of detecting and identifying the corner points of road signs, in one embodiment, the variance of the Gaussian distribution in the probability region can be modified so that multiple blocks are distributed in a P×P matrix, where P is a set target value, and P is a positive integer and is less than N and M respectively. In this embodiment, the variance of the Gaussian distribution in the probability region can be reduced so that multiple blocks are distributed in a 5×5 matrix. It should be noted that the high-resolution neural network model (such as HigherHrNet) used for human posture recognition, the network model is usually used for training images using a probability region with a 9×9 matrix distribution to mark the key points of the human skeleton in the image. In the embodiment of the present application, for detecting the corner points of road signs, due to the difference in the target object of the annotation of the human posture recognition task, the shapes of different road signs are not much different, and the number of road sign corner points of the road signs is not large (such as a square road sign has four road sign corner points). The probability region with a 5×5 matrix distribution is used to mark the corner points of each road sign in the image, which can effectively improve the efficiency and accuracy of detecting and identifying the corner points of road signs.
[0061] In one embodiment, the length and width of each image acquired in step S101 meet the set conditions. The set conditions may include: the length of the image is greater than or equal to the set length threshold, and the width of the image is greater than or equal to the set width threshold. In the embodiment of the present application, the set condition is that the length and width of the image are both 640 pixels, that is, the image is 640×640 pixels in size. In this embodiment, according to the set conditions, the input parameters of the preset high-resolution neural network model are set in advance. That is to say, according to the set conditions, the input parameters (input side) of the preset high-resolution neural network model are adjusted so that the preset high-resolution neural network model receives and processes images that meet the set conditions, and performs road sign corner point detection on the images that meet the set conditions. It should be noted that the high-resolution neural network model used for human posture recognition (such as HigherHrNet) requires the length and width of the input image to be 512×512 pixels in size. In the embodiment of the present application, in order to meet the accuracy requirements of road sign corner point detection, the input parameters of the preset high-resolution neural network model are changed, and the length and width requirements of the input image are improved. This is helpful for obtaining information in the image, can improve the accuracy of detecting road sign corner points in the image, and can improve the correctness of the pixel coordinate positions of the road sign corner points on each identified road sign.
[0062] In one embodiment, the number of feature map channels of the backbone network in the preset high-resolution neural network model can be adjusted down to a set value in advance. In this embodiment, the number of feature map channels of the backbone network in the preset high-resolution neural network model can be adjusted down to 24, thereby reducing the size of the model and improving the model processing efficiency. It should be noted that the high-resolution neural network model (such as HigherHrNet) used for human posture recognition, in order to identify the positions of multiple skeletal key points of the human body, the number of feature map channels of its backbone network is usually 32. It can be understood that the deep neural network model has a multi-layer network structure, each layer of the network structure has multiple channels, and each channel requires a large amount of calculation. The fewer the number of channels (i.e., the smaller the width), the smaller the amount of calculation, and the faster the speed. In the embodiment of the present application, according to the characteristics of the road sign structure, since the number of road sign corner points is not large (the direction road sign has four road sign corner points and the triangle road sign has three road sign corner points), the number of feature map channels of the backbone network in the preset high-resolution neural network model is adjusted down to 24, so that the recognition processing efficiency of the model can be improved. It can be understood that if the number of feature map channels of the backbone network is still 32, the amount of calculation will be large, resulting in a waste of computing resources and affecting the model processing efficiency.
[0063] See also Figure 2 , in such Figure 2In the implementation mode, a high-resolution newly added feature map is generated in advance in the backbone network of the preset high-resolution neural network model, so that the preset high-resolution neural network model performs road sign corner point detection based on the original feature map in the backbone network and the newly added feature map. In this implementation mode, the preset high-resolution neural network model is the HigherHrNet high-resolution neural network model; the original feature maps in the backbone network of the preset high-resolution neural network model include: a feature map with a resolution of 1 / 4 of the input image and a feature map with a resolution of 1 / 2 of the input image.
[0064] In an optional implementation, a high-resolution newly added feature map is generated in advance in the backbone network of a preset high-resolution neural network model, including:
[0065] A newly added feature map with a resolution of 1 / 1 of the input image is generated in advance by a deconvolution operation in the backbone network of the preset high-resolution neural network model. In this embodiment, a newly added feature map with a resolution of 1 / 1 of the input image can be generated by a deconvolution operation based on the original feature map with a resolution of 1 / 2 of the input image in the backbone network of the preset high-resolution neural network model. In a specific implementation, two feature maps with a resolution of 1 / 2 of the input image can be merged (concat), and then a deconvolution operation is performed through a deconvolution unit (deconv module) to obtain a newly added feature map with a resolution of 1 / 1 of the input image.
[0066] In this embodiment, the HigherHrNet high-resolution neural network model predicts a heatmap based on a feature map with a resolution of 1 / 4 of the input image, a feature map with a resolution of 1 / 2 of the input image, and a newly added feature map with a resolution of 1 / 1 of the input image in the backbone network, thereby outputting the pixel coordinate positions of each road sign corner point in each road sign in the image. The HigherHrNet high-resolution neural network model uses a multi-resolution supervision strategy to assign training targets of different resolutions to corresponding feature pyramid levels, and uses a multi-resolution heat map aggregation strategy in the inference process to generate a scale-aware high-resolution heat map. The newly added feature map with a resolution of 1 / 1 of the input image has a higher resolution than the feature map with a resolution of 1 / 2 of the input image in the backbone network, thereby being able to obtain more road sign feature information in the image, which is conducive to identifying road signs with smaller targets, and can correctly identify all road signs in the image, and output the pixel coordinate positions of each road sign corner point in each road sign in the image.
[0067] In this application, in order to compare the differences in road sign corner detection of images using high-resolution neural network models with different settings, please refer to Table 1, where the high-resolution neural network model HigherHrNet-1.0 is trained with a preset training data set, and the training image uses a probability region distributed in a 9×9 matrix to mark the road sign corners in the image. HigherHrNet-1.0 is used as a control group, and is compared with HigherHrNet-2.1 and HigherHrNet-2.2 in terms of precision, recall, and Euler distance (i.e., the distance between the predicted output road sign corner position in the image and the actual road sign corner position in the image).
[0068] Among them, the high-resolution neural network model HigherHrNet-2.1 is further improved on the basis of HigherHrNet-1.0. It generates a new feature map with a resolution of 1 / 1 of the input image in the backbone network. The high-resolution neural network model HigherHrNet-2.2 detects road sign corners based on the feature map with a resolution of 1 / 4 of the input image, the feature map with a resolution of 1 / 2 of the input image, and the new feature map with a resolution of 1 / 1 of the input image in the backbone network.
[0069] Among them, the high-resolution neural network model HigherHrNet-2.2 is further improved on the basis of HigherHrNet-1.0. It modifies the variance of the Gaussian distribution of the probability area of the road sign corner points in the training image. The training image uses a 5×5 matrix distribution of probability areas to mark the road sign corner points in the image. The input parameters of the network model are pre-set according to the setting condition that the length and width of the input image are both 640 pixels, and the number of feature map channels of its backbone network is reduced to 24.
[0070] Accuracy Recall Euler distance HigherHrNet-1.0 0.675 0.726 4.559 HigherHrNet-2.1 0.862 0.748 2.386 HigherHrNet-2.2 0.892 0.751 2.041
[0071] Table 1
[0072] It can be found that the high-resolution neural network model HigherHrNet-1.0 can output a relatively accurate result for the road sign corner point position output when used for road sign corner point detection. Compared with HigherHrNet-1.0, the results of road sign corner point position output by the improved HigherHrNet-2.1 and HigherHrNet-2.2 are better than HigherHrNet-1.0 in terms of accuracy and Euler distance, which effectively improves the recognition accuracy of the road sign corner points of each road sign in the image.
[0073] Step S103: Generate the geographic coordinate position of each road sign according to the pixel coordinate positions of each road sign corner point corresponding to each road sign in at least two images and the geographic location information of the camera when taking the at least two images.
[0074] The road sign corner points corresponding to each road sign may include at least three road sign corner points corresponding to each road sign, for example, four road sign corner points on a quadrilateral road sign, and three road sign corner points on a triangular road sign.
[0075] In one embodiment, at least two images are two images. According to the two images, the rotation matrix and the translation matrix between the two images can be calculated. According to the pixel coordinate positions of the corner points of each road sign corresponding to each road sign in the two images, and the calculated rotation matrix and translation matrix between the two images, the spatial coordinates of the corner points of each road sign corresponding to each road sign relative to the camera can be calculated. Using the spatial coordinates of the corner points of each road sign corresponding to each road sign relative to the camera and the geographical location information of the camera when taking the two images, the geographical coordinate position of each road sign can be generated, that is, the geographical coordinate position of the corner point of each road sign is generated, so as to realize the generation and production of the road sign. It can be understood that the spatial coordinates of the corner points of each road sign corresponding to each road sign relative to the camera can be calculated more accurately through multiple images. In this step, the calculation method of the rotation matrix and the translation matrix, the calculation method of the spatial coordinates of the corner points of each road sign relative to the camera, and the calculation method of the geographical coordinate position of the corner points of the road sign can refer to the introduction in the relevant technology, and this application will not repeat it.
[0076] It should be noted that in this embodiment, the geographic location information of the vehicle or camera can be collected by a positioning device configured by a vehicle or mobile phone, wherein the positioning device can be implemented by existing devices such as GPS (Global Positioning System), Beidou, RTK (Real Time Kinematic), etc., and this application is not limited. The geographic location information of the vehicle (or camera) may include but is not limited to the geographic coordinates (such as GPS coordinates, longitude and latitude coordinates, etc.), orientation, heading angle, direction and other information of the vehicle (or camera). The method provided in the embodiment of the present application can be applied to the vehicle computer, and can also be applied to other devices with computing and processing functions, such as computers, computers, mobile phones, etc. Here, taking the vehicle computer as an example, the camera and the positioning device can be built into the vehicle computer, or can be set outside the vehicle computer, and a communication connection is established with the vehicle computer. While the camera is shooting an image, the positioning device collects the geographic location information of the vehicle or camera and transmits it to the vehicle computer together. According to the shooting time of the image, the geographic location information obtained by the positioning device at the same time can be found. It can be understood that the time of the camera and the positioning device can be synchronized in advance, and the purpose is to make the captured image accurately correspond to the position of the vehicle or camera at that time.
[0077] It can be seen from this embodiment that the method provided in the present application realizes the automatic generation of road signs and the rapid generation of at least one road sign, so that multiple road signs can be generated at the same time, effectively improving the efficiency of drawing high-precision maps.
[0078] Corresponding to the aforementioned application function implementation method embodiment, the present application also provides a smart traffic sign recognition device, electronic device and corresponding embodiments.
[0079] Figure 3 It is a structural schematic diagram of the intelligent traffic sign recognition device shown in an embodiment of the present application.
[0080] See also Figure 3 , a smart traffic sign recognition device includes: an acquisition module 301, a recognition module 302, and a generation module 303.
[0081] The acquisition module 301 is used to acquire at least two images, each of which contains at least one common road sign.
[0082] The recognition module 302 is used to detect the corner points of at least one common road sign in each image acquired by the acquisition module 301 using a preset high-resolution neural network model, and identify the pixel coordinate positions of the corner points of each road sign corresponding to at least one road sign in each image. The preset high-resolution neural network model may be a HigherHrNet high-resolution neural network model.
[0083] The generation module 303 is used to generate the geographical coordinate position of each road sign according to the pixel coordinate positions of each road sign corner point corresponding to each road sign identified by the recognition module 302 in at least two images, and the geographical location information of the camera when taking at least two images.
[0084] It can be seen from this embodiment that the intelligent traffic sign recognition device provided in the present application realizes the automatic generation of road signs and the rapid generation of at least one road sign, so that multiple road signs can be generated at the same time, effectively improving the efficiency of drawing high-precision maps.
[0085] Figure 4 A smart traffic sign recognition device in another embodiment of the present application includes: an acquisition module 301, a recognition module 302, a generation module 303, a training module 304, and a setting module 305.
[0086] The functions of the acquisition module 301, the identification module 302, and the generation module 303 can be found in Figure 3 The description in will not be repeated here.
[0087] The training module 304 is used to pre-train a preset high-resolution neural network model using a preset training data set, wherein the preset training data set includes multiple training images, and the road sign corner points of each road sign in each training image are marked, and the road sign corner points of different road signs are marked with different labels.
[0088] The setting module 305 is used to pre-set the input parameters of the preset high-resolution neural network model according to the set conditions; wherein the length and width of each image acquired by the acquisition module 301 meet the set conditions; the set conditions include: the length of the image is greater than or equal to the set length threshold, and the width of the image is greater than or equal to the set width threshold.
[0089] The setting module 305 is also used to pre-adjust the number of feature map channels of the backbone network in the preset high-resolution neural network model to a set value.
[0090] The setting module 305 is also used to pre-generate a high-resolution newly added feature map in the backbone network of the preset high-resolution neural network model, so that the preset high-resolution neural network model performs road sign corner point detection based on the original feature map in the backbone network and the newly added feature map.
[0091] Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment of the method, and will not be elaborated again here.
[0092] Figure 5It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application.
[0093] See also Figure 5 , the electronic device 500 includes a memory 510 and a processor 520 .
[0094] The processor 520 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0095] The memory 510 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 520 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, an optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 510 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 510 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (such as a DVD-ROM, a double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (such as an SD card, a mini SD card, a Micro-SD card, etc.), a magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and transient electronic signals transmitted wirelessly or wired.
[0096] The memory 510 stores executable codes, and when the executable codes are processed by the processor 520 , the processor 520 can execute part or all of the methods described above.
[0097] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.
[0098] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.
[0099] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.
Claims
1. A method for identifying intelligent traffic signs, characterized in that: include: Pre-adjust the number of feature map channels of the backbone network in the preset high-resolution neural network model to a set value; Generate a high-resolution newly added feature map in advance in the backbone network of the preset high-resolution neural network model, so that the preset high-resolution neural network model performs road sign corner point detection according to the original feature map in the backbone network and the newly added feature map; wherein the original feature map includes: a feature map with a resolution of 1 / 4 of the input image and a feature map with a resolution of 1 / 2 of the input image; the newly added feature map is a feature map with a resolution of 1 / 1 of the input image; Acquire at least two images, each of the images containing at least one common road sign; Using a preset high-resolution neural network model to perform road sign corner point detection on at least one common road sign in each of the images, and identifying the pixel coordinate positions of each road sign corner point corresponding to at least one road sign in each of the images; The geographic coordinate position of each road sign is generated according to the pixel coordinate positions of each road sign corner point corresponding to each road sign in the at least two images and the geographic location information of the camera when the at least two images are taken.
2. The method according to claim 1, characterized in that The method further comprises: A preset high-resolution neural network model is trained in advance using a preset training data set, wherein the preset training data set includes multiple training images, the road sign corner points of each road sign in each of the training images are annotated, and the road sign corner points of different road signs are annotated with different labels.
3. The method according to claim 2, characterized in that The method of labeling the training image includes: A probability region with a Gaussian distribution is drawn at the position of the corner point of the road sign in the training image, the probability region is divided into a plurality of blocks, the plurality of blocks are distributed in an N×M matrix, the corner point of the road sign is located in a block at the center position of the probability region; wherein N and M are both positive integers; a highest probability value is marked on the block where the corner point of the road sign is located, and the probability values marked on each of the blocks in the probability region constitute a Gaussian distribution.
4. The method according to claim 3, characterized in that The method further comprises: The variance of the Gaussian distribution in the probability region is modified so that the plurality of blocks are distributed in a P×P matrix, wherein P is a set target value, and P is a positive integer and is smaller than N and M, respectively.
5. The method according to claim 1, characterized in that: The length and width of each of the images meet the set conditions; The method also includes: pre-setting input parameters of the preset high-resolution neural network model according to the set conditions.
6. The method according to claim 1, characterized in that The step of pre-generating a high-resolution newly added feature map in the backbone network of the preset high-resolution neural network model comprises: A new feature map with a resolution of 1 / 1 of the input image is generated in advance through a deconvolution operation in the backbone network of the preset high-resolution neural network model.
7. The method according to any one of claims 1 to 6, characterized in that: The preset high-resolution neural network model is a HigherHrNet high-resolution neural network model.
8. A smart traffic sign recognition device, characterized in that: include: An acquisition module, used for acquiring at least two images, each of which contains at least one common road sign; A recognition module, used to detect road sign corner points of at least one common road sign in each image acquired by the acquisition module using a preset high-resolution neural network model, and identify the pixel coordinate positions of each road sign corner point corresponding to at least one road sign in each image; A generating module, configured to generate the geographical coordinate position of each road sign according to the pixel coordinate positions of each road sign corner point corresponding to each road sign identified by the identifying module in the at least two images, and the geographical location information of the camera when taking the at least two images; A setting module is used to pre-adjust the number of feature map channels of the backbone network in the preset high-resolution neural network model to a set value; pre-generate a high-resolution newly added feature map in the backbone network of the preset high-resolution neural network model, so that the preset high-resolution neural network model performs road sign corner detection based on the original feature map in the backbone network and the newly added feature map; wherein the original feature map includes: a feature map with a resolution of 1 / 4 of the input image and a feature map with a resolution of 1 / 2 of the input image; the newly added feature map is a feature map with a resolution of 1 / 1 of the input image.
9. A computer-readable storage medium having executable codes stored thereon, which, when executed by a processor of an electronic device, causes the processor to execute the method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Traffic sign recognition method and device and training method and device of neural network model
CN111488770A
Guideboard text detection and recognition algorithm based on deep learning
CN111738255A
Marker drawing method and system
CN111932627A
Traffic scene understanding method and device based on video streaming
CN112347933A
Traffic sign detection and identification method based on driving video
CN112699841A
