Road sign corner point recognition method, device, equipment and storage medium

Abstract

The present application relates to a method, device, equipment and storage medium for identifying road sign corners. The method: performing convolution calculation on the image to be identified to obtain a first convolution feature map, batch normalization processing on the first convolution feature map, performing convolution calculations of multiple scales on the first convolution feature map after batch normalization processing to obtain multiple second convolution feature maps, performing batch normalization processing and convolution calculation on the multiple second convolution feature maps to obtain multiple third convolution feature maps, performing feature fusion on the multiple third convolution feature maps according to a preset residual fusion algorithm to obtain a fused feature map, performing convolution calculation on the fused feature map to obtain a target convolution feature map, and outputting the coordinates of the road sign corners based on the target convolution feature map. The introduction of batch normalization processing can improve the recognition accuracy of road sign corners. The introduction of a preset residual fusion algorithm allows the original valid data of the convolution feature map to be retained, thereby improving the recognition accuracy of road sign corners.

Description

Technical Field

[0001] The present application relates to the field of navigation technology, and in particular to a method, device, equipment and storage medium for identifying road sign corner points. Background Art

[0002] High-precision maps are special maps for autonomous driving and play a core role in the entire autonomous driving field. High-precision maps are composed of vector information such as lane models, road components, and road attributes that contain semantic information. Among them, road sign elements are one of the elements that must be included in high-precision maps.

[0003] The core prerequisite for constructing road sign elements in high-precision maps is to be able to accurately identify road sign corners from the collected image data, and to complete the construction of road sign elements in the high-precision map based on the identified road sign corners. However, in related technologies, there is a problem of low accuracy in identifying road sign corners. Summary of the invention

[0004] In order to solve or partially solve the problems existing in the related art, the present application provides a road sign corner point recognition method, device, equipment and storage medium, which can accurately identify road sign corner points from image data.

[0005] The first aspect of the present application provides a road sign corner point recognition method, comprising:

[0006] Acquire a picture to be identified, where the picture to be identified carries a road sign element;

[0007] Performing convolution calculation on the image to be identified to obtain a first convolution feature map;

[0008] Performing batch normalization processing on the first convolutional feature map;

[0009] Performing multiple-scale convolution calculations on the first convolution feature map after batch normalization processing to obtain multiple second convolution feature maps;

[0010] performing batch normalization processing and convolution calculation on the plurality of the second convolution feature maps in sequence to obtain a plurality of third convolution feature maps;

[0011] Performing feature fusion on the plurality of third convolution feature maps according to a preset residual fusion algorithm to obtain a fused feature map;

[0012] A convolution calculation is performed on the fused feature map to obtain a target convolution feature map, and the coordinates of the road sign corner points are output based on the target convolution feature map.

[0013] Preferably, the batch normalization processing of the first convolutional feature map includes:

[0014] Calculate the corresponding mean and variance according to the pixel values ​​in the first convolution feature map;

[0015] Setting a scaling factor and a translation factor corresponding to the first convolution feature map;

[0016] The first convolution feature map is batch normalized according to the mean value, the variance, the scaling factor, and the translation factor corresponding to the first convolution feature map.

[0017] Preferably, the step of performing convolution calculations of multiple scales on the first convolution feature map after batch normalization to obtain multiple second convolution feature maps includes:

[0018] Initialize a plurality of convolution kernels N*N, and set parameters in each of the convolution kernels, where N is a positive odd number in the interval [1,7];

[0019] The first convolution feature map after batch normalization is respectively convolved using the plurality of convolution kernels to obtain a plurality of second convolution feature maps.

[0020] Preferably, the step of performing batch normalization processing and convolution calculation on the plurality of the second convolution feature maps in sequence to obtain a plurality of third convolution feature maps comprises:

[0021] Calculate the corresponding mean value and variance according to the pixel values ​​in each of the second convolution feature maps;

[0022] Setting a scaling factor and a translation factor corresponding to each of the second convolution feature maps;

[0023] Performing batch normalization processing on the corresponding second convolutional feature map according to the average value, the variance, the scaling factor and the translation factor corresponding to each of the second convolutional feature maps;

[0024] Convolution calculations are performed on the multiple second convolutional feature maps that have been batch normalized to obtain multiple third convolutional feature maps.

[0025] Preferably, the performing feature fusion on the plurality of the third convolution feature maps according to a preset residual fusion algorithm to obtain a fused feature map comprises:

[0026] Setting a fusion ranking order of the plurality of third convolutional feature maps;

[0027] Based on the fusion ranking order, pixel values ​​at corresponding positions in the plurality of third convolution feature maps are linearly added to obtain a fused feature map, wherein each linear addition is followed by a convolution before the next linear addition.

[0028] Preferably, performing convolution calculation on the fused feature map to obtain a target convolution feature map, and outputting the coordinates of the road sign corner points based on the target convolution feature map, comprises:

[0029] Set the number of channels according to the number of corner points of the road sign;

[0030] Performing convolution calculation on the fused feature map based on the channel to obtain a target convolution feature map;

[0031] Output the coordinates of the road sign corner points of the target convolution feature map corresponding to the channel.

[0032] Preferably, the channel includes a first channel, a second channel, a third channel and a fourth channel;

[0033] The outputting the street sign corner coordinates of the target convolution feature map corresponding to the channel includes:

[0034] Outputting the coordinates of the corner points of the road sign in the target convolution feature map under the first channel as the coordinates of the upper left corner point of the road sign;

[0035] Outputting the coordinates of the road sign corner point of the target convolution feature map under the second channel as the coordinates of the upper right corner point of the road sign;

[0036] Outputting the coordinates of the road sign corner point of the target convolution feature map under the third channel as the coordinates of the lower right corner point of the road sign;

[0037] The coordinates of the road sign corner point of the target convolution feature map under the fourth channel are output as the coordinates of the lower left corner point of the road sign.

[0038] A second aspect of the present application provides a road sign corner point recognition device, comprising:

[0039] An acquisition module, used to acquire a picture to be identified, wherein the picture to be identified carries a road sign element;

[0040] A pre-convolution module, used to perform convolution calculation on the image to be identified to obtain a first convolution feature map;

[0041] A pre-processing module, used for performing batch normalization processing on the first convolution feature map;

[0042] A distributed convolution module, used for performing convolution calculations of multiple scales on the first convolution feature map after batch normalization processing to obtain multiple second convolution feature maps;

[0043] A distribution processing module, used for performing batch normalization processing and convolution calculation on the plurality of the second convolution feature maps in sequence to obtain a plurality of third convolution feature maps;

[0044] A fusion module, used for performing feature fusion on the plurality of third convolution feature maps according to a preset residual fusion algorithm to obtain a fused feature map;

[0045] The output module is used to perform convolution calculation on the fused feature map to obtain a target convolution feature map, and output the coordinates of the road sign corner points based on the target convolution feature map.

[0046] A third aspect of the present application provides an electronic device, including:

[0047] Processor; and

[0048] A memory stores executable codes thereon, and when the executable codes are executed by the processor, the processor is caused to execute the road sign corner point recognition method as described above.

[0049] A fourth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the road sign corner point recognition method as described above.

[0050] The technical solution provided by this application may have the following beneficial effects:

[0051] The technical solution of the present application is to obtain a picture to be identified, perform convolution calculation on the picture to be identified to obtain a first convolution feature map, batch normalize the first convolution feature map, perform convolution calculations of multiple scales on the first convolution feature map after batch normalization to obtain multiple second convolution feature maps, perform batch normalization and convolution calculation on the multiple second convolution feature maps in sequence to obtain multiple third convolution feature maps, perform feature fusion on the multiple third convolution feature maps according to a preset residual fusion algorithm to obtain a fused feature map, perform convolution calculation on the fused feature map to obtain a target convolution feature map, and output the coordinates of the corner points of the road sign based on the target convolution feature map. Since the convolution feature map is batch normalized, the data in the convolution feature map can be uniformly distributed within a certain range to converge, thereby improving the recognition accuracy of the corner points of the road sign. In addition, feature fusion is performed according to the preset residual fusion algorithm, so that the original valid data in the convolution feature map can be better retained, preventing the loss of data related to the road sign elements, thereby improving the recognition accuracy of the corner points of the road sign.

[0052] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0054] Figure 1 is a flowchart of a road sign corner point recognition method shown in an embodiment of the present application;

[0055] Figure 2 is a schematic structural diagram of a road sign corner point recognition device shown in another embodiment of the present application;

[0056] Figure 3 is a simulation schematic diagram of a target convolution feature map shown in an embodiment of the present application;

[0057] Figure 4 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0059] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0060] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0061] At present, the core prerequisite for constructing road sign elements in high-precision maps is to be able to accurately identify the corner points of road signs from the collected image data, and to complete the construction of road sign elements in the high-precision map based on the identified road sign corner points. However, in related technologies, there is a problem of low accuracy in identifying road sign corner points.

[0062] In view of the above problems, the present application provides a method, device, equipment and storage medium for identifying road sign corner points, which can accurately identify road sign corner points from image data. In order to facilitate understanding of the technical solution of the present application, the technical solution of the present application is described in detail below in conjunction with the accompanying drawings.

[0063] Figure 1 What is shown is a flowchart of a road sign corner point recognition method according to an embodiment of the present application.

[0064] See also Figure 1 , a road sign corner point recognition method, comprising the following steps:

[0065] Step S11: Obtain a picture to be identified, where the picture to be identified carries a road sign element.

[0066] The image to be identified is a picture taken by a camera installed on the vehicle, and the picture taken by the camera is processed as the image to be identified. The image to be identified carries a road sign element. A road sign is an object installed on a lane, and the road sign records relevant traffic information. A road sign is usually square in shape, and a square road sign has four right angles, which are the corner points of the road sign. The method of this embodiment performs a series of processing on the image to be identified, and finally outputs the coordinates of the corner points of the road sign of the image to be identified. The number of road signs carried in the image to be identified is not limited to one, and may be multiple.

[0067] Step S12: performing convolution calculation on the image to be recognized to obtain a first convolution feature map.

[0068] Perform convolution calculation on the image to be identified, set the number of channels, convolution kernel and step size, use the convolution kernel to perform convolution calculation on the image to be identified, and obtain the first convolution feature map. The number of channels in this step can be flexibly set according to the actual situation. For example, the number of channels can be set to 3 channels (3 channels are RGB channels) or 256 channels. The number of channels in this step is related to the type of the image to be identified.

[0069] Step S13: performing batch normalization processing on the first convolutional feature map.

[0070] It should be noted that the first convolution feature map is batch normalized, the pixel values ​​in the first convolution feature map are obtained, the corresponding average value and variance are calculated according to the pixel values ​​in the first convolution feature map, the scaling coefficient and translation coefficient corresponding to the first convolution feature map are set, and the first convolution feature map is batch normalized according to the average value, variance, scaling coefficient and translation coefficient corresponding to the first convolution feature map.

[0071] The batch normalization formula is as follows:

[0072]

[0073]

[0074]

[0075] Among them, x i is the pixel value in the first convolution feature map, n is the number of pixel values ​​in the first convolution feature map, μ is the average value calculated based on the pixel values ​​in the first convolution feature map, σ 2 is the variance calculated based on the pixel values ​​in the first convolution feature map, γ is the scaling factor corresponding to the first convolution feature map, β is the translation factor corresponding to the first convolution feature map, and x inew is the pixel value after batch normalization, and ε is a constant greater than 0, which prevents the denominator from being 0. After equations (1), (2), and (3), the batch normalization of the first convolutional feature map is completed.

[0076] Step S14: performing multiple-scale convolution calculations on the first convolution feature map after batch normalization processing to obtain multiple second convolution feature maps.

[0077] It should be noted that after the first convolution feature map is batch normalized, different convolution kernels are used to perform convolution calculations on the first convolution feature map. Initialize multiple convolution kernels N*N, and set the parameters in each convolution kernel. N is a positive odd number in the interval [1,7]. Use multiple convolution kernels to perform convolution calculations on the first convolution feature map after batch normalization to obtain multiple second convolution feature maps.

[0078] For example, initialize three convolution kernels, namely convolution kernel 1*1, convolution kernel 3*3 and convolution kernel 5*5. Use these three convolution kernels to perform convolution calculations on the first convolution feature map after batch normalization. Use convolution kernel 1*1 to perform convolution calculations on the first convolution feature map after batch normalization to obtain the second convolution feature map. Use convolution kernel 3*3 to perform convolution calculations on the first convolution feature map after batch normalization to obtain the second convolution feature map. Use convolution kernel 5*5 to perform convolution calculations on the first convolution feature map after batch normalization to obtain the second convolution feature map. The number of second convolution feature maps obtained by convolution using three convolution kernels is 3.

[0079] Step S15: perform batch normalization processing and convolution calculation on the multiple second convolution feature maps in sequence to obtain multiple third convolution feature maps.

[0080] It should be noted that after obtaining multiple second convolutional feature maps, batch normalization processing and convolution calculation are performed on the multiple second convolutional feature maps in sequence to obtain multiple third convolutional feature maps.

[0081] After obtaining three second convolution feature maps, firstly perform batch normalization processing on the three second convolution feature maps respectively. For the formula of batch normalization processing, refer to formula (1), formula (2) and formula (3). Since each second convolution feature map stores pixel values, the corresponding average value and variance are calculated according to the pixel values ​​in each second convolution feature map, and the scaling coefficient and translation coefficient corresponding to each second convolution feature map are set. According to the average value, variance, scaling coefficient and translation coefficient corresponding to each second convolution feature map, batch normalization processing is performed on the corresponding second convolution feature map. Referring to the above process, three second convolution feature maps after batch normalization processing can be obtained. Convolution calculation is performed on multiple second convolution feature maps after batch normalization processing to obtain multiple third convolution feature maps. After completing the batch normalization processing of the three second convolution feature maps, convolution calculation is performed on these three second convolution feature maps respectively. For example, by using convolution kernel 1*1, convolution kernel 3*3 and convolution kernel 5*5 to perform convolution calculation with the corresponding second convolution feature map after batch normalization, three third convolution feature maps can be obtained.

[0082] Step S16: performing feature fusion on multiple third convolution feature maps according to a preset residual fusion algorithm to obtain a fused feature map.

[0083] A plurality of third convolution feature maps are obtained, a fusion ranking order of the plurality of third convolution feature maps is set, and pixel values ​​at corresponding positions in the plurality of third convolution feature maps are linearly added based on the fusion ranking order to obtain a fused feature map, wherein each linear addition is followed by a convolution before the next linear addition.

[0084] After obtaining three third convolution feature maps, set the fusion ranking order of the three third convolution feature maps. In order to facilitate the description of the feature fusion process, the three third convolution feature maps are represented by the third convolution feature map a, the third convolution feature map b, and the third convolution feature map c. The fusion ranking order of the third convolution feature map a is set to the third, the fusion ranking order of the third convolution feature map b is set to the second, and the fusion ranking order of the third convolution feature map c is set to the first.

[0085] First, the third convolution feature map b and the third convolution feature map c are feature fused, and the pixel values ​​of the corresponding positions of the third convolution feature map b and the third convolution feature map c are added to complete the feature fusion. Finally, the third convolution feature map b and the third convolution feature map c that have been feature fused are linearly added to the third convolution feature map a. It should be emphasized that before the third convolution feature map b and the third convolution feature map c that have been feature fused are linearly added to the third convolution feature map a, it is necessary to perform a convolution calculation on the linearly added third convolution feature map b and the third convolution feature map c, and then linearly add them with the third convolution feature map a to finally obtain a fused feature map.

[0086] Convolution kernels of different sizes have different effects on extracting information from small road signs, medium road signs, and large road signs. For example, after convolution kernels of different sizes are used to convolve the recognition image, the second convolution feature map obtained after batch normalization processing contains information about three different sizes of road signs (small road signs, medium road signs, and large road signs). A small convolution kernel of 1*1 is used to mainly extract relevant data for identifying small road signs, and the third convolution feature map a is obtained; a medium convolution kernel of 3*3 is used to mainly extract relevant information for identifying medium road signs, and the third convolution feature map b is obtained; a large convolution kernel of 5*5 is used to mainly extract relevant information for identifying large road signs, and the third convolution feature map c is obtained. The third convolution feature map a, the third convolution feature map b, and the third convolution feature map c are linearly added one by one in succession, and the resulting fusion feature map can retain relevant information with three different sizes of road signs, thereby preventing the loss of original valid data. It should be noted here that convolution calculation needs to be performed after each linear addition to prevent the introduction of noise.

[0087] It should also be noted that in order to allow multiple three-convolution feature maps to be linearly added, the convolution operation performed before obtaining multiple three-convolution feature maps needs to ensure that the number of rows and columns of the convolution feature maps calculated by the convolution is the same, that is, the dimensions of the convolution feature maps are the same. The above result can be achieved by padding. The technical principle of padding will not be elaborated in detail and is well known to those skilled in the art.

[0088] Step S17: perform convolution calculation on the fused feature map to obtain a target convolution feature map, and output the coordinates of the road sign corner points based on the target convolution feature map.

[0089] After obtaining the fused feature map, the number of channels is set according to the number of road sign corner points, and convolution calculation is performed on the fused feature map based on the channel to obtain the target convolution feature map, and the coordinates of the road sign corner points of the fourth feature map corresponding to the output channel are output.

[0090] A square road sign has 4 road sign corner points. Set the number of channels to 4, such as Figure 3 The simulation diagram of the target convolution feature map shown in the figure includes the first channel, the second channel, the third channel and the fourth channel. When performing convolution calculation on the fused feature map, the convolution kernel changes from 1 to 4 identical ones. The convolution kernel under the first channel, the convolution kernel under the second channel, the convolution kernel under the third channel and the convolution kernel under the fourth channel are all the same. Convolution calculation is performed on the fused feature map under each channel, and the target convolution feature map under each channel is obtained accordingly. The coordinates of the corner point of the road sign in the target convolution feature map under the first channel are used as the coordinates of the upper left corner point of the road sign (i.e. Figure 3 The coordinates of the corner point of the road sign in the target convolution feature map under the second channel are used as the coordinates of the upper right corner point of the road sign (i.e. Figure 3The coordinates of the corner point of the road sign in the target convolution feature map under the third channel are used as the coordinates of the lower right corner point of the road sign (i.e. Figure 3 The coordinates of the corner point of the road sign in the target convolution feature map under the fourth channel are used as the coordinates of the lower left corner point of the road sign (i.e. Figure 3 The output of the coordinates of the four corner points of the square road sign is completed.

[0091] It should also be noted that if the number of corner points of the road sign is 3, the number of channels can be set to 3. The process is referred to the above process and will not be repeated here.

[0092] It should also be noted that when there are three square road signs in the target convolution feature map, the coordinates of the three road sign corner points of the target convolution feature map under the first channel are used as the coordinates of the upper left corner of the road signs corresponding to the three square road signs, the coordinates of the three road sign corner points of the target convolution feature map under the second channel are used as the coordinates of the upper right corner of the road signs corresponding to the three square road signs, the coordinates of the three road sign corner points of the target convolution feature map under the third channel are used as the coordinates of the lower right corner of the road signs corresponding to the three square road signs, and the coordinates of the three road sign corner points of the target convolution feature map under the fourth channel are used as the coordinates of the lower left corner of the road signs corresponding to the three square road signs.

[0093] The method of this embodiment has the following beneficial effects:

[0094] 1. Normalize the pixel values ​​in the convolution feature map so that the pixel values ​​can be uniformly distributed within a certain range, laying the foundation for the subsequent accurate identification of the coordinates of the road sign corners in the convolution feature map;

[0095] 2. The introduction of the preset residual fusion algorithm realizes feature fusion in a linear addition manner, so that the final output target convolution feature map still retains the original valid data of the image to be identified, effectively preventing the target convolution feature map from losing the original valid data due to the continuous deepening of the convolution calculation, so that all the corner coordinates of the road sign can be accurately identified on the target convolution feature map.

[0096] Corresponding to the aforementioned functional method embodiment, the present application also provides a corresponding embodiment of a road sign corner point recognition device.

[0097] Figure 2 This is a schematic diagram of the structure of a road sign corner point recognition device shown in an embodiment of the present application.

[0098] See also Figure 2 The road sign corner recognition device 80 includes: an acquisition module 810, a pre-convolution module 820, a pre-processing module 830, a distribution convolution module 840, a distribution processing module 850, a fusion module 860 and an output module 870.

[0099] The acquisition module 810 is used to acquire a picture to be identified, where the picture to be identified carries a road sign element;

[0100] The pre-convolution module 820 is used to perform convolution calculation on the image to be recognized to obtain a first convolution feature map;

[0101] The pre-processing module 830 is used to perform batch normalization processing on the first convolution feature map;

[0102] The distributed convolution module 840 is used to perform convolution calculations of multiple scales on the first convolution feature map after batch normalization processing to obtain multiple second convolution feature maps;

[0103] The distribution processing module 850 is used to perform batch normalization processing and convolution calculation on the multiple second convolution feature maps in sequence to obtain multiple third convolution feature maps;

[0104] The fusion module 860 is used to perform feature fusion on multiple third convolution feature maps according to a preset residual fusion algorithm to obtain a fused feature map;

[0105] The output module 870 is used to perform convolution calculation on the fused feature map to obtain a target convolution feature map, and output the coordinates of the road sign corner points based on the target convolution feature map.

[0106] The device of this embodiment obtains the image to be identified through the acquisition module 810, uses the pre-convolution module 820, and uses the pre-processing module 830 to perform batch normalization processing on the first convolution feature map, the distribution convolution module 840 performs convolution calculations of multiple scales on the first convolution feature map after batch normalization processing to obtain multiple second convolution feature maps, the distribution processing module 850 performs batch normalization processing and convolution calculations on the multiple second convolution feature maps in sequence to obtain multiple third convolution feature maps, the fusion module 860 performs feature fusion on the multiple third convolution feature maps according to the preset residual fusion algorithm to obtain a fused feature map, and the output module 870 is used to perform convolution calculation on the fused feature map to obtain a target convolution feature map, and output the coordinates of the road sign corner points based on the target convolution feature map. Since the convolution feature map is subjected to batch normalization processing, the data in the convolution feature map can be uniformly distributed and converged within a certain range, thereby improving the recognition accuracy of the road sign corner points. In addition, feature fusion is performed according to the preset residual fusion algorithm, so that the valid data in the convolution feature map can be better retained, preventing the loss of data related to road sign elements, thereby improving the recognition accuracy of road sign corners.

[0107] Regarding the device in the above embodiment, the specific manner in which each module and unit performs operations has been described in detail in the method embodiment corresponding to the device, and will not be elaborated again here.

[0108] See also Figure 4 , the electronic device 900 includes a processor 910 and a memory 920 .

[0109] Processor 910 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0110] The memory 920 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage. Among them, ROM can store static data or instructions required by processing 910 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, an optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 920 may include any combination of computer-readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. The memory 920 stores executable codes, and when the executable codes are processed by the processor 910 , the processor 910 can execute part or all of the above-mentioned methods.

[0111] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0112] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0113] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A road sign corner point recognition method, characterized in that: include: Acquire a picture to be identified, wherein the picture to be identified carries a road sign element; Performing convolution calculation on the image to be identified to obtain a first convolution feature map; Performing batch normalization processing on the first convolutional feature map; Performing multiple-scale convolution calculations on the first convolution feature map after batch normalization processing to obtain multiple second convolution feature maps; The method includes: initializing a plurality of convolution kernels N*N, and setting parameters in each of the convolution kernels, wherein N is a positive odd number in the interval [1,7]; using a plurality of convolution kernels of different scales to perform convolution calculations on the first convolution feature maps after batch normalization processing, respectively, to obtain a plurality of second convolution feature maps; The plurality of second convolution feature maps are subjected to batch normalization processing and convolution calculation in sequence to obtain a plurality of third convolution feature maps; wherein the method comprises: using a plurality of convolution kernels corresponding to the convolution calculation performed on the first convolution feature map to perform convolution calculation on the plurality of second convolution feature maps after batch normalization processing to obtain a plurality of third convolution feature maps; the number of the second convolution feature maps, the third convolution feature maps and the convolution kernels corresponds; The plurality of third convolution feature maps are subjected to feature fusion according to a preset residual fusion algorithm to obtain a fused feature map; the method includes: setting a fusion ranking order of the plurality of third convolution feature maps; linearly adding pixel values ​​at corresponding positions in the plurality of third convolution feature maps based on the fusion ranking order to obtain a fused feature map, wherein each linear addition is followed by a convolution and then the next linear addition; Convolution calculation is performed on the fused feature map to obtain a target convolution feature map, and the coordinates of the road sign corner points are output based on the target convolution feature map; wherein, it includes: setting the number of channels according to the number of road sign corner points; convolution calculation is performed on the fused feature map based on the channel to obtain the target convolution feature map; the coordinates of the road sign corner points of the target convolution feature map corresponding to the channel are output; the channel includes a first channel, a second channel, a third channel and a fourth channel.

2. The method for identifying road sign corner points according to claim 1, characterized in that: The performing batch normalization processing on the first convolutional feature map includes: Calculate the corresponding mean and variance according to the pixel values ​​in the first convolution feature map; Setting a scaling factor and a translation factor corresponding to the first convolution feature map; The first convolution feature map is batch normalized according to the mean value, the variance, the scaling factor, and the translation factor corresponding to the first convolution feature map.

3. The road sign corner point recognition method according to claim 1, characterized in that: The step of performing batch normalization processing and convolution calculation on the plurality of the second convolution feature maps in sequence to obtain a plurality of third convolution feature maps comprises: Calculate the corresponding mean value and variance according to the pixel values ​​in each of the second convolution feature maps; Setting a scaling factor and a translation factor corresponding to each of the second convolution feature maps; Performing batch normalization processing on the corresponding second convolutional feature map according to the average value, the variance, the scaling factor and the translation factor corresponding to each of the second convolutional feature maps; Convolution calculations are performed on the multiple second convolutional feature maps that have been batch normalized to obtain multiple third convolutional feature maps.

4. The method for identifying road sign corner points according to claim 1, characterized in that: The outputting the street sign corner coordinates of the target convolution feature map corresponding to the channel includes: Outputting the coordinates of the corner points of the road sign in the target convolution feature map under the first channel as the coordinates of the upper left corner point of the road sign; Outputting the coordinates of the road sign corner point of the target convolution feature map under the second channel as the coordinates of the upper right corner point of the road sign; Outputting the coordinates of the road sign corner point of the target convolution feature map under the third channel as the coordinates of the lower right corner point of the road sign; The coordinates of the road sign corner point of the target convolution feature map under the fourth channel are output as the coordinates of the lower left corner point of the road sign.

5. A road sign corner point recognition device, characterized in that: include: An acquisition module, used to acquire a picture to be identified, wherein the picture to be identified carries a road sign element; A pre-convolution module, used to perform convolution calculation on the image to be identified to obtain a first convolution feature map; A pre-processing module, used for performing batch normalization processing on the first convolution feature map; A distributed convolution module, used for performing convolution calculations of multiple scales on the first convolution feature map after batch normalization processing to obtain multiple second convolution feature maps; The method includes: initializing a plurality of convolution kernels N*N, and setting parameters in each of the convolution kernels, wherein N is a positive odd number in the interval [1,7]; using a plurality of convolution kernels of different scales to perform convolution calculations on the first convolution feature maps after batch normalization processing, respectively, to obtain a plurality of second convolution feature maps; A distribution processing module, used to perform batch normalization processing and convolution calculation on the plurality of second convolution feature maps in sequence, to obtain a plurality of third convolution feature maps; wherein, it includes: using a plurality of convolution kernels corresponding to the convolution calculation performed on the first convolution feature map to perform convolution calculation on the plurality of second convolution feature maps after batch normalization processing, to obtain a plurality of third convolution feature maps; the number of the second convolution feature maps, the third convolution feature maps and the convolution kernels corresponds; A fusion module is used to perform feature fusion on the plurality of the third convolution feature maps according to a preset residual fusion algorithm to obtain a fused feature map; the fusion module includes: setting a fusion ranking order of the plurality of the third convolution feature maps; linearly adding pixel values ​​at corresponding positions in the plurality of the third convolution feature maps based on the fusion ranking order to obtain a fused feature map, wherein each linear addition is followed by a convolution before the next linear addition; An output module is used to perform convolution calculation on the fused feature map to obtain a target convolution feature map, and output the coordinates of the road sign corner points based on the target convolution feature map; wherein, it includes: setting the number of channels according to the number of road sign corner points; performing convolution calculation on the fused feature map based on the channel to obtain the target convolution feature map; outputting the coordinates of the road sign corner points of the target convolution feature map corresponding to the channel; the channel includes a first channel, a second channel, a third channel and a fourth channel.

6. An electronic device, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the road sign corner point recognition method according to any one of claims 1 to 4.

7. A computer-readable storage medium having executable codes stored thereon, which, when executed by a processor of an electronic device, causes the processor to execute the road sign corner point recognition method according to any one of claims 1 to 4.

Images