Method, device, equipment and storage medium for automatic recognition of road sign corner points

Abstract

The present application relates to an automatic recognition method, device, equipment and storage medium for corner points of road signs, which obtains an image to be recognized, performs convolution calculations on the image to be recognized along odd-numbered branches and even-numbered branches to obtain a first convolution feature map and a second convolution feature map, performs feature fusion calculations on multiple first convolution feature maps corresponding to multiple second convolution feature maps to obtain multiple initial fusion feature maps, performs ordinary convolution calculations on multiple initial fusion feature maps to obtain multiple third convolution feature maps, performs feature fusion calculations on multiple third convolution feature maps to obtain a fusion feature map to be processed, performs ordinary convolution calculations on the fusion feature map to be processed to obtain a target convolution feature map, and outputs the coordinates of the corner points of the road sign based on the target convolution feature map. Since the dilated convolution has a larger receptive field, it is combined with ordinary convolution to extract more effective information. And the dilated convolution calculations using different dilation rates can capture information at different levels, thereby improving the recognition rate of road sign corner points.

Description

Technical Field

[0001] The present application relates to the field of navigation technology, and in particular to a method, device, equipment and storage medium for automatically identifying corner points of road signs. Background Art

[0002] Road sign elements are one of the indispensable elements for building high-precision maps. The construction of road sign elements mainly includes the recognition of road sign corner points and road sign information. The recognition of road sign corner points is the prerequisite for building road sign elements.

[0003] In related technologies, convolutional neural networks are often used to identify the corner points of road signs in images. However, due to the limitations of the convolutional neural network's own architecture, when performing image recognition processing, it often fails to capture data information at different levels in the image, which results in the inability to accurately identify the corner points of road signs. Summary of the invention

[0004] In order to solve or partially solve the problems existing in the related art, the present application provides a method, device, equipment and storage medium for automatically identifying the corner points of road signs, which can accurately identify the corner points of road signs.

[0005] The first aspect of the present application provides a method for automatically identifying road sign corner points, comprising:

[0006] Acquire an image to be identified, wherein the image to be identified carries a road sign element;

[0007] Performing convolution calculation on the image to be identified along a preset symmetric branch, wherein the preset symmetric branch includes a plurality of odd-numbered branches and even-numbered branches having a corresponding relationship, performing multiple ordinary convolution calculations on the image to be identified along the odd-numbered branches in sequence to obtain a first convolution feature map, and performing dilated convolution calculations and ordinary convolution calculations with different dilation rates on the image to be identified along the even-numbered branches in sequence to obtain a second convolution feature map;

[0008] Performing feature fusion calculation on the first convolution feature maps corresponding to the plurality of odd-numbered branches and the second convolution feature maps corresponding to the plurality of even-numbered branches, respectively, to obtain a plurality of initial fused feature maps;

[0009] Performing ordinary convolution calculations on the multiple initial fusion feature maps respectively to obtain multiple third convolution feature maps;

[0010] Performing feature fusion calculation on the plurality of third convolution feature maps to obtain a fused feature map to be processed;

[0011] Perform ordinary convolution calculation on the fused feature map to be processed to obtain a target convolution feature map, and output the coordinates of the road sign corner points based on the target convolution feature map.

[0012] Preferably, the ordinary convolution calculation includes:

[0013] Initialize the convolution kernel N*N, where N is a positive odd number in the interval [1,7];

[0014] Set the number of normal convolution channels;

[0015] According to the convolution kernel N*N and the number of ordinary convolution channels, ordinary convolution calculation is performed on the image to be recognized, the initial fused feature map or the fused feature map to be processed.

[0016] Preferably, when multiple ordinary convolution calculations are performed on the image to be identified along different odd-numbered branches, the convolution kernels N*N of the ordinary convolution calculations performed at corresponding positions of two odd-numbered branches are different from each other, and the number of ordinary convolution channels are complementary.

[0017] Preferably, the dilated convolution calculation includes:

[0018] Initialize the convolution kernel P*P and set the hole rate of the convolution kernel P*P, where P is a positive odd number in the interval [1,7];

[0019] Set the number of dilated convolution channels;

[0020] According to the convolution kernel P*P and the number of dilated convolution channels, a dilated convolution calculation is performed on the image to be identified.

[0021] Preferably, when performing dilated convolution calculations with different dilation rates on the image to be identified in sequence along different even-numbered branches, the convolution kernels P*P of the dilated convolution calculations performed at corresponding positions of the even-numbered branches are the same and the number of dilated convolution channels are complementary.

[0022] Preferably, the number of ordinary convolution channels for performing ordinary convolution calculation on the fused feature map to be processed is set according to the number of road sign corner points.

[0023] Preferably, the number of ordinary convolution channels for performing ordinary convolution calculation on the fused feature map to be processed is equal to 4;

[0024] The outputting the coordinates of the corner points of the road sign based on the target convolution feature map includes:

[0025] Output the coordinates of the corner point of the road sign of the target convolution feature map whose common convolution channel number is 1 as the coordinates of the upper left corner point of the road sign;

[0026] Output the road sign corner coordinates of the target convolution feature map when the common convolution channel number is equal to 2 as the coordinates of the upper right corner of the road sign;

[0027] Output the road sign corner coordinates of the target convolution feature map when the common convolution channel number is equal to 3 as the coordinates of the lower right corner of the road sign;

[0028] The coordinates of the road sign corner point of the target convolution feature map when the ordinary convolution channel number is equal to 4 are output as the coordinates of the lower left corner point of the road sign.

[0029] A second aspect of the present application provides an automatic identification device for road sign corners, comprising:

[0030] An acquisition module, used to acquire an image to be recognized, wherein the image to be recognized carries a road sign element;

[0031] A symmetric convolution calculation module, used to perform convolution calculation on the image to be identified along a preset symmetric branch, wherein the preset symmetric branch includes a plurality of odd-numbered branches and even-numbered branches having a corresponding relationship, and perform multiple ordinary convolution calculations on the image to be identified along the odd-numbered branches in sequence to obtain a first convolution feature map, and perform dilated convolution calculations and ordinary convolution calculations with different dilated rates on the image to be identified along the even-numbered branches in sequence to obtain a second convolution feature map;

[0032] A first fusion module is used to perform feature fusion calculation on the first convolution feature maps corresponding to the plurality of odd-numbered branches and the second convolution feature maps corresponding to the plurality of even-numbered branches, respectively, to obtain a plurality of initial fused feature maps;

[0033] A distributed convolution calculation module, used to perform common convolution calculations on the multiple initial fusion feature maps respectively to obtain multiple third convolution feature maps;

[0034] A second fusion module is used to perform feature fusion calculation on the plurality of third convolution feature maps to obtain a fused feature map to be processed;

[0035] The output module is used to perform ordinary convolution calculation on the fused feature map to be processed to obtain a target convolution feature map, and output the coordinates of the road sign corner points based on the target convolution feature map.

[0036] A third aspect of the present application provides an electronic device, including:

[0037] Processor; and

[0038] A memory stores executable codes thereon, and when the executable codes are executed by the processor, the processor is caused to execute the road sign corner point recognition method as described above.

[0039] A fourth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the road sign corner point recognition method as described above.

[0040] The technical solution provided by this application may have the following beneficial effects:

[0041] The technical solution of the present application obtains an image to be recognized, and performs convolution calculations on the image to be recognized along the odd-ordered branches and the even-ordered branches with corresponding relationships, wherein ordinary convolution calculations are performed along the odd-ordered branches to obtain a first convolution feature map, and dilated convolution calculations and ordinary convolution calculations with different dilation rates are performed along the even-ordered branches to obtain a second convolution feature map, multiple first convolution feature maps are correspondingly calculated with multiple second convolution feature maps to perform feature fusion calculations to obtain multiple initial fusion feature maps, and ordinary convolution calculations are performed on multiple initial fusion feature maps to obtain multiple third convolution feature maps, and feature fusion calculations are performed on multiple third convolution feature maps to obtain a fusion feature map to be processed, and ordinary convolution calculations are performed on the fusion feature map to be processed to obtain a target convolution feature map, and the coordinates of the corner points of the road sign are output based on the target convolution feature map. Since the dilated convolution calculation has a larger receptive field, it is combined with ordinary convolution calculations to extract more effective information, and at the same time, the dilated convolution calculations with different dilation rates are used to process the image to be recognized, which can capture information at different levels, thereby improving the recognition rate of the corner points of the road sign.

[0042] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0044] Figure 1 is a flow chart of a method for automatically identifying corner points of road signs shown in an embodiment of the present application;

[0045] Figure 2 is an application framework diagram of the method for automatically identifying road sign corner points shown in an embodiment of the present application;

[0046] Figure 3 is a simulation schematic diagram of a target convolution feature map shown in an embodiment of the present application;

[0047] Figure 4 is a schematic diagram of the structure of an automatic identification device for road sign corners shown in an embodiment of the present application;

[0048] Figure 5 It is a schematic diagram of the structure of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0050] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0051] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0052] Identification of road sign corners is the prerequisite for constructing road sign elements. In related technologies, convolutional neural networks are often used to identify road sign corners in images. However, due to the limitations of the convolutional neural network's own architecture, there is a problem that road sign corners cannot be accurately identified.

[0053] In view of the above problems, the present application provides a method, device, equipment and storage medium for automatically identifying road sign corner points, which can accurately identify road sign corner points. To facilitate understanding of the technical solution of the present application, the technical solution of the present application is described in detail below in conjunction with the accompanying drawings.

[0054] Figure 1 A schematic flow chart of a method for automatically identifying corner points of road signs in an embodiment of the present application is shown.

[0055] See also Figure 1 , a method for automatically identifying road sign corner points, comprising the following steps:

[0056] Step S11: Acquire an image to be identified, where the image to be identified carries a road sign element.

[0057] The image to be identified is a photo taken by a vehicle-mounted camera. The image to be identified carries road sign elements. Road signs are objects installed on the road, and the shapes of road signs are usually square, triangular, and hexagonal. For a square road sign, it has 4 road sign corner points (road sign corner points are edge points, the same below). For a triangular road sign, it has 3 road sign corner points. For a hexagonal road sign, it has 6 road sign corner points. The number of road signs in the image to be identified is not limited to one, and may be multiple.

[0058] Step S12, performing convolution calculation on the image to be identified along preset symmetric branches, the preset symmetric branches include multiple odd-numbered branches and even-numbered branches with corresponding relationships, performing multiple ordinary convolution calculations on the image to be identified along the odd-numbered branches in sequence to obtain a first convolution feature map, and performing dilated convolution calculations and ordinary convolution calculations with different dilation rates on the image to be identified along the even-numbered branches in sequence to obtain a second convolution feature map.

[0059] The preset symmetric branches include multiple odd-numbered branches and even-numbered branches, and the odd-numbered branches correspond to the even-numbered branches, that is, each odd-numbered branch corresponds to an even-numbered branch. Multiple ordinary convolution calculations are performed on the image to be identified along the odd-numbered branches to obtain the first convolution feature map. And along the even-numbered branches, hole convolution calculations and ordinary convolution calculations with different hole rates are performed on the image to be identified to obtain the second convolution feature map. The image to be identified is subjected to different convolution processing along different branches. Compared with ordinary convolution calculations, hole convolution calculations have a larger receptive field, and can have better effects on the recognition and segmentation of small road sign elements in the tasks of target detection and semantic segmentation of the image to be identified. The ordinary convolution calculation and the hole convolution calculation are mixed and used, and the two convolution calculations complement each other to extract more effective information from the image to be identified.

[0060] In addition, along the even-numbered branches, dilated convolution calculations with different dilation rates are performed on the image to be identified, which can capture information at different levels in the image to be identified, that is, it can capture large road signs, medium road signs, and small road sign elements in the image to be identified, preventing the omission of road sign elements.

[0061] It should also be noted that the number of convolution calculations performed on the image to be recognized along the odd-numbered branches can be the same as the number of convolution calculations performed on the image to be recognized along the even-numbered branches. That is, when three ordinary convolution calculations are performed on the image to be recognized along the odd-numbered branches, correspondingly, two dilated convolution calculations (the dilated ratios of the two dilated convolution calculations are different) and one ordinary convolution calculation are performed on the image to be recognized along the even-numbered branches.

[0062] When performing ordinary convolution calculation on the image to be recognized, initialize the convolution kernel N*N, where N is a positive odd number in the interval [1,7], set the number of ordinary convolution channels, and perform ordinary convolution calculation on the image to be recognized based on the convolution kernel N*N and the number of ordinary convolution channels. For example, initialize the convolution kernel 3*3, set the number of ordinary convolution channels channels=128, and perform ordinary convolution calculation on the image to be recognized based on the convolution kernel 3*3 and the number of ordinary convolution channels as 128.

[0063] It should be noted that, since the preset symmetric branches include multiple odd-numbered branches, the convolution kernels N*N of the ordinary convolution calculations performed at the corresponding positions of the two odd-numbered branches are different from each other, and the number of ordinary convolution channels is complementary. For example, the preset symmetric branches include two odd-numbered branches, and the two odd-numbered branches perform three ordinary convolution calculations on the image to be identified in sequence. The two odd-numbered branches are paired in pairs. If an odd-numbered branch uses a convolution kernel of 3*3 for the first ordinary convolution calculation, the other odd-numbered branch uses a convolution kernel of 5*5. If the number of ordinary convolution channels for the first ordinary convolution calculation of an odd-numbered branch is channels=128, the number of ordinary convolution channels for the first ordinary convolution calculation of another odd-numbered branch is channels=256. According to the above method, the feature information of the image to be identified after convolution can be enriched, laying the foundation for the subsequent improvement of the recognition rate of road sign corners.

[0064] When performing dilated convolution calculation on the image to be identified, initialize the convolution kernel P*P, set the dilation rate of the convolution kernel P*P, where P is a positive odd number in the interval [1,7], set the number of dilated convolution channels, and perform dilated convolution calculation on the image to be identified based on the convolution kernel P*P and the number of dilated convolution channels. For example, initialize the convolution kernel 3*3, set the dilation rate of the convolution kernel 3*3 to dilation=5, set the number of dilated convolution channels channels=256, and perform dilated convolution calculation on the image to be identified based on the convolution kernel 3*3 with a dilation rate of 5 and the number of dilated convolution channels of 256.

[0065] It should be noted that, since the preset symmetric branches include multiple even-numbered branches, the convolution kernels P*P of the dilated convolution calculations performed at the corresponding positions of the two even-numbered branches are the same, and the number of dilated convolution channels is complementary. For example, the preset symmetric branches include two even-numbered branches, and the two even-numbered branches perform two dilated convolution calculations and one ordinary convolution calculation on the image to be identified in sequence. The two even-numbered branches are paired in pairs. If an even-numbered branch performs the first dilated convolution calculation using a convolution kernel 3*3 with a dilation rate of 2, then the other even-numbered branch performs the first dilated convolution calculation using a convolution kernel 3*3 with a dilation rate of 2. If the number of dilated convolution channels for the first dilated convolution calculation of an even-numbered branch is channels=128, then the number of dilated convolution channels for the first dilated convolution calculation of another even-numbered branch is channels=256. According to the above method, the feature information after the convolution of the image to be identified can be enriched, laying the foundation for the subsequent improvement of the recognition rate of road sign corners.

[0066] Step S13, perform feature fusion calculation on the first convolution feature maps corresponding to the multiple odd-numbered branches and the second convolution feature maps corresponding to the multiple even-numbered branches, respectively, to obtain multiple initial fused feature maps.

[0067] It should be noted that after processing the image to be recognized along different branches, multiple first convolution feature maps and multiple second convolution feature maps are obtained. Feature fusion calculations are performed on the first convolution feature maps corresponding to the multiple odd-numbered branches and the second convolution feature maps corresponding to the multiple even-numbered branches, respectively, to obtain multiple initial fusion feature maps.

[0068] It should also be noted that the feature fusion calculation is a linear addition process, which linearly adds the pixel values ​​at the corresponding positions of the first convolution feature map and the pixel values ​​at the corresponding positions of the second convolution feature map to obtain an initial fused feature map.

[0069] Step S14: perform ordinary convolution calculations on the multiple initial fusion feature maps respectively to obtain multiple third convolution feature maps.

[0070] After obtaining multiple initial fused feature maps, ordinary convolution calculations are performed on the multiple initial fused feature maps respectively to obtain multiple third convolution feature maps. The ordinary convolution calculation method refers to the ordinary convolution calculation process mentioned above, which will not be repeated here.

[0071] Step S15: perform feature fusion calculation on the multiple third convolution feature maps to obtain a fused feature map to be processed.

[0072] After obtaining multiple third convolution feature maps, the multiple third convolution feature maps are subjected to feature fusion calculation to obtain a fused feature map to be processed. The process of feature fusion calculation is referred to above and will not be repeated here.

[0073] Step S16: perform ordinary convolution calculation on the fused feature map to be processed to obtain a target convolution feature map, and output the coordinates of the road sign corner points based on the target convolution feature map.

[0074] After obtaining the fused feature map to be processed, perform ordinary convolution calculation on the fused feature map to obtain the target convolution feature map, and output the coordinates of the road sign corner points based on the target convolution feature map. The basic process of performing ordinary convolution calculation on the fused feature map to be processed is referred to above and will not be repeated here. The difference is that the number of ordinary convolution channels for ordinary convolution calculation on the fused feature map to be processed is related to the number of road sign corner points.

[0075] For example, if you want to identify a square road sign in an image to be identified, no matter it is a small square road sign, a medium square road sign or a large square road sign, these three sizes of square road signs all have 4 road sign corner points. Therefore, the number of ordinary convolution channels for ordinary convolution calculation of the fused feature map to be processed is equal to 4. When outputting the coordinates of the road sign corner points based on the target convolution feature map: the coordinates of the road sign corner points of the target convolution feature map with the ordinary convolution channel number equal to 1 are output as the coordinates of the upper left corner of the road sign; the coordinates of the road sign corner points of the target convolution feature map with the ordinary convolution channel number equal to 2 are output as the coordinates of the upper right corner of the road sign; the coordinates of the road sign corner points of the target convolution feature map with the ordinary convolution channel number equal to 3 are output as the coordinates of the lower right corner of the road sign; the coordinates of the road sign corner points of the target convolution feature map with the ordinary convolution channel number equal to 4 are output as the coordinates of the lower left corner of the road sign.

[0076] In order to better understand the technical solution of the present application, the technical principle process of the present application is explained below with a specific embodiment.

[0077] Embodiment 1

[0078] Please also read Figure 2 and Figure 3 Assuming that the corner points of the square road sign in the image to be recognized need to be identified, the preset symmetric branch includes two odd-numbered branches ( Figure 2 The first and third branches shown) and two even-numbered branches ( Figure 2 The ② branch and the ④ branch are shown in FIG. 1 , where the ① branch corresponds to the ② branch, and the ③ branch corresponds to the ④ branch. The image IMG to be identified is obtained, and the image IMG to be identified is processed along the ① branch, the ② branch, the ③ branch, and the ④ branch. The first convolution feature map is obtained along the ① branch and the ③ branch, and the second convolution feature map is obtained along the ② branch and the ④ branch.

[0079] Since the first branch and the third branch are paired in pairs, when three ordinary convolution calculations are performed on the image to be identified IMG along the first branch and the third branch, the convolution kernels N*N of the ordinary convolution calculations performed at the corresponding positions are different, and the number of ordinary convolution channels is complementary. Since the second branch and the fourth branch are paired in pairs, two dilated convolution calculations and one ordinary convolution calculation are performed on the image to be identified IMG along the second branch and the fourth branch, the convolution kernels P*P of the dilated convolution calculations performed at the corresponding positions are the same, and the number of dilated convolution channels is complementary.

[0080] The first convolution feature map corresponding to the first branch and the second convolution feature map corresponding to the second branch are subjected to feature fusion calculation to obtain one of the initial fusion feature maps. The first convolution feature map corresponding to the third branch and the second convolution feature map corresponding to the fourth branch are subjected to feature fusion calculation to obtain another initial fusion feature map.

[0081] Perform ordinary convolution calculations on the two initial fused feature maps respectively to obtain two third convolution feature maps. Perform feature fusion calculations on the two third convolution feature maps to obtain the fused feature map to be processed. Finally, perform ordinary convolution calculations on the fused feature map to be processed to obtain the target convolution feature map.

[0082] When performing ordinary convolution calculation on the fused feature map to be processed, the number of ordinary convolution channels is 4. The coordinates of the corner point of the road sign in the target convolution feature map with the ordinary convolution channel number equal to 1 are output as the coordinates of the upper left corner point of the road sign (i.e. Figure 3 Point A1, point A2 and point A3 shown); the coordinates of the corner point of the road sign of the target convolution feature map under the ordinary convolution channel number equal to 2 are output as the coordinates of the upper right corner point of the road sign (i.e. Figure 3 The coordinates of the corner points of the road sign in the target convolution feature map with the normal convolution channel number equal to 3 are output as the coordinates of the lower right corner point of the road sign (i.e. Figure 3 The coordinates of the corner points of the road sign in the target convolution feature map with the normal convolution channel number equal to 4 are output as the coordinates of the lower left corner point of the road sign (i.e. Figure 3 Points D1, D2 and D3 are shown).

[0083] Corresponding to the aforementioned application function method embodiment, the present application also provides a corresponding embodiment of a device for automatically identifying road sign corner points.

[0084] Figure 4 A schematic structural diagram of an automatic road sign corner recognition device 40 in an embodiment of the present application is shown.

[0085] See also Figure 4, an automatic recognition device 40 for road sign corners, comprising: an acquisition module 410, a symmetric convolution calculation module 420, a first fusion module 430, a distributed convolution calculation module 440, a second fusion module 450 and an output module 460.

[0086] The acquisition module 410 is used to acquire an image to be recognized, where the image to be recognized carries a road sign element;

[0087] The symmetric convolution calculation module 420 is used to perform convolution calculation on the image to be identified along a preset symmetric branch, the preset symmetric branch includes a plurality of odd-numbered branches and even-numbered branches with corresponding relationships, and a plurality of ordinary convolution calculations are sequentially performed on the image to be identified along the odd-numbered branches to obtain a first convolution feature map, and a dilated convolution calculation and an ordinary convolution calculation with different dilation rates are sequentially performed on the image to be identified along the even-numbered branches to obtain a second convolution feature map;

[0088] The first fusion module 430 is used to perform feature fusion calculations on the first convolution feature maps corresponding to the multiple odd-numbered branches and the second convolution feature maps corresponding to the multiple even-numbered branches, respectively, to obtain multiple initial fusion feature maps;

[0089] The distributed convolution calculation module 440 is used to perform common convolution calculations on the multiple initial fusion feature maps respectively to obtain multiple third convolution feature maps;

[0090] The second fusion module 450 is used to perform feature fusion calculation on the multiple third convolution feature maps to obtain a fused feature map to be processed;

[0091] The output module 460 is used to perform ordinary convolution calculation on the processed fused feature map to obtain a target convolution feature map, and output the coordinates of the road sign corner points based on the target convolution feature map.

[0092] Regarding the device in the above embodiment, the specific manner in which each module and unit performs operations has been described in detail in the method embodiment corresponding to the device, and will not be elaborated again here.

[0093] See also Figure 5 , the electronic device 900 includes a processor 910 and a memory 920 .

[0094] The processor 910 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0095] The memory 920 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 910 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 920 may include any combination of computer-readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. The memory 920 stores executable codes, and when the executable codes are processed by the processor 910 , the processor 910 can execute part or all of the methods described above.

[0096] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0097] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of an electronic device (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0098] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for automatically identifying road sign corner points, characterized in that: include: Acquire an image to be identified, wherein the image to be identified carries a road sign element; Performing convolution calculation on the image to be identified along a preset symmetric branch, wherein the preset symmetric branch includes a plurality of odd-numbered branches and even-numbered branches having a corresponding relationship, performing multiple ordinary convolution calculations on the image to be identified along the odd-numbered branches in sequence to obtain a first convolution feature map, and performing dilated convolution calculations and ordinary convolution calculations with different dilation rates on the image to be identified along the even-numbered branches in sequence to obtain a second convolution feature map; Performing feature fusion calculation on the first convolution feature maps corresponding to the plurality of odd-numbered branches and the second convolution feature maps corresponding to the plurality of even-numbered branches, respectively, to obtain a plurality of initial fused feature maps; Performing ordinary convolution calculations on the multiple initial fusion feature maps respectively to obtain multiple third convolution feature maps; Performing feature fusion calculation on the plurality of third convolution feature maps to obtain a fused feature map to be processed; Performing ordinary convolution calculation on the fused feature map to be processed to obtain a target convolution feature map, and outputting the coordinates of the corner points of the road sign based on the target convolution feature map; wherein, The ordinary convolution calculation includes: Initialize the convolution kernel N*N, where N is a positive odd number in the interval [1,7]; Set the number of normal convolution channels; According to the convolution kernel N*N and the number of common convolution channels, performing common convolution calculation on the image to be recognized, the initial fused feature map or the fused feature map to be processed; When performing multiple ordinary convolution calculations on the image to be recognized along different odd-numbered branches, the convolution kernels N*N of the ordinary convolution calculations performed at corresponding positions of two odd-numbered branches are different from each other, and the numbers of ordinary convolution channels are complementary; The dilated convolution calculation includes: Initialize the convolution kernel P*P and set the hole rate of the convolution kernel P*P, where P is a positive odd number in the interval [1,7]; Set the number of dilated convolution channels; According to the convolution kernel P*P and the number of dilated convolution channels, performing dilated convolution calculation on the image to be identified; When performing dilated convolution calculations with different dilation rates on the image to be identified in sequence along different even-numbered branches, the convolution kernels P*P of the dilated convolution calculations performed at corresponding positions of the even-numbered branches are the same, and the number of dilated convolution channels is complementary.

2. The method for automatically identifying road sign corner points according to claim 1, characterized in that: The number of ordinary convolution channels for performing ordinary convolution calculation on the fused feature map to be processed is set according to the number of road sign corner points.

3. The method for automatically identifying road sign corner points according to claim 2, characterized in that: The number of ordinary convolution channels for performing ordinary convolution calculation on the fused feature map to be processed is equal to 4; The outputting the coordinates of the corner points of the road sign based on the target convolution feature map includes: Output the coordinates of the corner point of the road sign of the target convolution feature map whose common convolution channel number is 1 as the coordinates of the upper left corner point of the road sign; Output the road sign corner coordinates of the target convolution feature map when the common convolution channel number is equal to 2 as the coordinates of the upper right corner of the road sign; Output the road sign corner coordinates of the target convolution feature map when the common convolution channel number is equal to 3 as the coordinates of the lower right corner of the road sign; The coordinates of the road sign corner point of the target convolution feature map when the ordinary convolution channel number is equal to 4 are output as the coordinates of the lower left corner point of the road sign.

4. An automatic identification device for road sign corners, characterized in that: include: An acquisition module, used to acquire an image to be recognized, wherein the image to be recognized carries a road sign element; A symmetric convolution calculation module, used to perform convolution calculation on the image to be identified along a preset symmetric branch, wherein the preset symmetric branch includes a plurality of odd-numbered branches and even-numbered branches having a corresponding relationship, and perform multiple ordinary convolution calculations on the image to be identified along the odd-numbered branches in sequence to obtain a first convolution feature map, and perform dilated convolution calculations and ordinary convolution calculations with different dilated rates on the image to be identified along the even-numbered branches in sequence to obtain a second convolution feature map; A first fusion module is used to perform feature fusion calculation on the first convolution feature maps corresponding to the plurality of odd-numbered branches and the second convolution feature maps corresponding to the plurality of even-numbered branches, respectively, to obtain a plurality of initial fused feature maps; A distributed convolution calculation module, used to perform common convolution calculations on the multiple initial fusion feature maps respectively to obtain multiple third convolution feature maps; A second fusion module is used to perform feature fusion calculation on the plurality of third convolution feature maps to obtain a fused feature map to be processed; An output module is used to perform ordinary convolution calculation on the fused feature map to be processed to obtain a target convolution feature map, and output the coordinates of the road sign corner points based on the target convolution feature map; wherein, The ordinary convolution calculation includes: Initialize the convolution kernel N*N, where N is a positive odd number in the interval [1,7]; Set the number of normal convolution channels; According to the convolution kernel N*N and the number of common convolution channels, performing common convolution calculation on the image to be recognized, the initial fused feature map or the fused feature map to be processed; When performing multiple ordinary convolution calculations on the image to be recognized along different odd-numbered branches, the convolution kernels N*N of the ordinary convolution calculations performed at corresponding positions of two odd-numbered branches are different from each other, and the numbers of ordinary convolution channels are complementary; The dilated convolution calculation includes: Initialize the convolution kernel P*P and set the hole rate of the convolution kernel P*P, where P is a positive odd number in the interval [1,7]; Set the number of dilated convolution channels; According to the convolution kernel P*P and the number of dilated convolution channels, performing dilated convolution calculation on the image to be identified; When performing dilated convolution calculations with different dilation rates on the image to be identified in sequence along different even-numbered branches, the convolution kernels P*P of the dilated convolution calculations performed at corresponding positions of the even-numbered branches are the same, and the number of dilated convolution channels is complementary.

5. An electronic device, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method for automatically identifying corner points of road signs as claimed in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: An executable code is stored thereon, and when the executable code is executed by a processor of an electronic device, the processor is caused to execute the method for automatically identifying corner points of road signs as claimed in any one of claims 1 to 3.

Images