General traffic sign recognition method and system based on dynamic network structure

Abstract

The application relates to the technical field of target recognition, and provides a general traffic sign recognition method and system based on a dynamic network structure, which comprises the following steps: acquiring a road image based on a vehicle-mounted image acquisition device; inputting the road image into a feature extraction layer of a recognition network to obtain a feature extraction result; inputting the feature extraction result into a decoder layer of the recognition network to obtain a traffic sign recognition result; the recognition network is obtained by excluding a decoder layer reinforced training branch after an initial network comprising the feature extraction layer, the decoder layer reinforced training branch and a decoder layer sign recognition branch is trained by samples, labels and a loss function; and the dynamic network structure is adopted in the training process and the reasoning process, so that efficient and rapid traffic sign recognition is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target recognition, in particular to a general traffic sign recognition method and system based on a dynamic network structure. BACKGROUND

[0002] Road traffic signs and markings guide road users to use the road in an orderly manner to promote road safety, and the recognition of traffic signs in a driving assistance system can provide corresponding assistance for vehicle control without interruption. For example, prohibition signs can help the system to make early danger prediction; warning signs can help the system to make early obstacle avoidance processing in certain situations; and indication signs can help the system to perform control preprocessing to ensure that the vehicle follows the road indication. Therefore, correct recognition and accurate application of traffic signs can provide more perfect assistance for driving assistance systems and even autonomous driving.

[0003] However, due to the factors of hardware cost, size, environment, etc. of the vehicle terminal, the hardware resources of the car are limited in the automatic driving or assisted driving scenario, and the computing resources that can be allocated to the traffic sign recognition task are usually limited.

[0004] For speed limit signs, the problem of computing resources is particularly prominent, that is, due to the diversity of speed limit types and values, existing neural network-based real-time recognition methods on the car have the defects of high computing resource demand and slow recognition speed.

[0005] Therefore, how to provide an efficient and fast traffic sign recognition method and system has become a technical problem that needs to be solved in the industry. SUMMARY

[0006] The present application provides a general traffic sign recognition method and system based on a dynamic network structure to solve the defects of high computing resource demand and slow recognition speed in the prior art, and to realize efficient and fast traffic sign recognition.

[0007] The present application provides a general traffic sign recognition method based on a dynamic network structure, characterized by being applied to a vehicle terminal, comprising:

[0008] acquiring a road image based on a vehicle-mounted image acquisition device;

[0009] inputting the road image into a feature extraction layer of a recognition network to obtain a feature extraction result;

[0010] inputting the feature extraction result into a decoder layer of the recognition network to obtain a traffic sign recognition result;

[0011] The identification network is obtained by excluding the decoder layer reinforced training branch from an initial network comprising a feature extraction layer, a decoder layer reinforced training branch, and a decoder layer sign identification branch after sample, label, and loss function training.

[0012] According to the general traffic sign identification method based on a dynamic network structure provided by the application, the decoder layer reinforced training branch is a decoder layer multi-sign region branch.

[0013] According to the general traffic sign identification method based on a dynamic network structure provided by the application, the multi-sign region comprises parallel pixels.

[0014] The parallel pixels refer to:

[0015] Pixels on a line connecting traffic sign center points; and / or

[0016] Pixels on adjacent parallel line segments of a set number of lines connecting traffic sign center points.

[0017] According to the general traffic sign identification method based on a dynamic network structure provided by the application, the label comprises a multi-sign region binary image.

[0018] According to the general traffic sign identification method based on a dynamic network structure provided by the application, the loss function comprises a reinforced branch term.

[0019] According to the general traffic sign identification method based on a dynamic network structure provided by the application, the step of inputting the feature extraction result into a decoder layer of an identification network to obtain a traffic sign identification result comprises:

[0020] The feature extraction result is input into a decoder layer of an identification network, and a sign position result, a sign size sub-result, and a sign position offset sub-result are obtained through a sign position sub-branch, a sign size sub-branch, and a sign position offset sub-branch of the decoder layer sign identification branch, respectively.

[0021] The application further provides a general traffic sign identification system based on a dynamic network structure, which is deployed on a vehicle terminal and comprises:

[0022] An acquisition module is configured to acquire a road image based on a vehicle-mounted image acquisition device.

[0023] a feature extraction module, configured to input the road image into a feature extraction layer of the recognition network to obtain a feature extraction result;

[0024] a decoder module, configured to input the feature extraction result into a decoder layer of the recognition network to obtain a traffic sign recognition result;

[0025] The recognition network is obtained by excluding the decoder layer reinforcement training branch after training the initial network including the feature extraction layer, the decoder layer reinforcement training branch and the decoder layer sign recognition branch by samples, labels and a loss function.

[0026] The application further provides an electronic device including a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the general traffic sign recognition method based on a dynamic network structure according to any one of the above when executing the program.

[0027] The application further provides a non-transitory computer readable storage medium having a computer program stored thereon, and the computer program implements the steps of the general traffic sign recognition method based on a dynamic network structure according to any one of the above when executed by a processor.

[0028] The application further provides a computer program product including a computer program, and the computer program implements the steps of the general traffic sign recognition method based on a dynamic network structure according to any one of the above when executed by a processor.

[0029] The general traffic sign recognition method and system based on a dynamic network structure provided by the application can obtain more supervision information by adding a decoder layer reinforcement training branch in the training process, so that the traffic sign recognition accuracy of the recognition network is improved; meanwhile, by excluding the decoder layer reinforcement training branch in the inference process, the network computing resource demand is not increased on the basis of retaining the recognition accuracy improvement, that is, a dynamic network structure is adopted in the training process and the inference process, and efficient and rapid traffic sign recognition is achieved. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0031] Figure 1 is a flowchart of the general traffic sign recognition method based on a dynamic network structure provided by the application;

[0032] Figure 2 is a structural schematic diagram of an initial network provided by an embodiment of the present application;

[0033] Figure 3 is a structural schematic diagram of a recognition network provided by an embodiment of the present application;

[0034] Figure 4 is a structural schematic diagram of a traffic sign recognition system based on category mapping provided by the present application;

[0035] Figure 5 is a structural schematic diagram of an electronic device provided by the present application.

[0036] Reference signs:

[0037] 1: an acquisition module; 2: a feature extraction module; 3: a decoder module;

[0038] 510: a processor; 520: a communication interface; 530: a memory;

[0039] 540: a communication bus. DETAILED DESCRIPTION

[0040] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without any creative work fall within the scope of protection of the present application.

[0041] The present application provides a general traffic sign recognition method based on a dynamic network structure. Figures 1-3 The present application provides a general traffic sign recognition method based on a dynamic network structure.

[0042] As shown in the accompanying drawings, Figure 1 the present application provides a general traffic sign recognition method based on a dynamic network structure, applied to a vehicle terminal, comprising:

[0043] Step 102, acquiring a road image based on a vehicle image acquisition device;

[0044] Step 104, inputting the road image into a feature extraction layer of a recognition network to obtain a feature extraction result;

[0045] Step 106, inputting the feature extraction result into a decoder layer of the recognition network to obtain a traffic sign recognition result;

[0046] The recognition network is obtained by removing the decoder layer reinforced training branch from an initial network including a feature extraction layer, a decoder layer reinforced training branch, and a decoder layer sign recognition branch after the initial network is trained by samples, labels, and a loss function.

[0047] In this embodiment, the initial network includes a feature extraction layer, a decoder layer reinforced training branch, and a decoder layer sign recognition branch, and the trained initial network is obtained by training the initial network by road image samples, labels, and a loss function.

[0048] The labels include true values of traffic signs in the road image samples and output true values of the reinforced training branch.

[0049] The loss function includes a traffic sign item and a reinforced branch item. The traffic sign item can quantify the difference between the traffic sign recognition result obtained by the decoder layer sign recognition branch and the true value of the traffic sign. The reinforced branch item can quantify the difference between the result obtained by the decoder layer reinforced training branch and the true value, and is preferably a cross-entropy function item.

[0050] The decoder layer reinforced training branch in the trained initial network is removed to obtain the recognition network including the feature extraction layer and the decoder layer used in steps 102, 104, and 106.

[0051] In this embodiment, the introduction of the decoder layer reinforced training branch enables the recognition network to be trained according to more supervision information, thereby improving the recognition accuracy of the recognition network. At the same time, since the decoder layer reinforced training branch is removed in the inference process, the time consumption and computational complexity of model inference are not increased.

[0052] The reinforced training branch can be flexibly set according to different application scenarios. For example, for ground traffic signs, since the position of the ground traffic sign is highly correlated with the lane line, a reinforced training branch of the relative position relationship between the ground traffic sign and the lane line can be introduced. For a closed expressway scene, since multiple traffic signs appear at the same time and usually have the collinear characteristics in the horizontal / vertical direction, a reinforced training branch of the position relationship between adjacent traffic signs can be introduced.

[0053] According to the above embodiment, in this embodiment:

[0054] The decoder layer reinforced training branch is a decoder layer multi-sign region branch. The decoder layer multi-sign region branch can obtain a set multi-sign region according to the feature extraction result. The intersection of the multi-sign region and the region where at least two traffic signs are located is a non-empty set.

[0055] The multi-sign region includes parallel pixels.

[0056] The parallel pixels refer to

[0057] pixels on the adjacent parallel line segments of the set number of traffic sign center point connecting lines.

[0058] pixels on the adjacent parallel line segments of the set number of traffic sign center point connecting lines.

[0059] The label comprises a multi-sign region binary graph; the multi-sign region binary graph comprises multi-sign region pixels and non-multi-sign region pixels respectively set as different binary values.

[0060] The loss function comprises a reinforcement branch term; the reinforcement branch term can quantify the difference between the multi-sign region output by the decoder layer reinforcement training branch and the multi-sign region binary graph in the label.

[0061] The embodiment sets a multi-sign region branch as a reinforcement training branch for the application scenario that there are at least two traffic signs in the same road image. The multi-sign region refers to a set region that overlaps with the region of any traffic sign, and in the embodiment, the region of the center point connecting line of adjacent traffic signs is selected.

[0062] Considering that there is a certain regularity in the placement of traffic signs, that is, in the application scenario that there are at least two traffic signs in the same road image, multiple traffic signs are usually placed horizontally side by side or vertically side by side, and the positions of traffic signs are commonly above the road surface or at the edge of the road surface, therefore, the multi-sign region prediction is added during network training, the supervision information of traffic signs is strengthened, and the traffic sign detection effect is improved

[0063] The embodiment can improve the accuracy of traffic sign recognition. Moreover, the embodiment method reduces the time consumption and computational amount during model deployment while improving the implementation effect. The embodiment reduces the network structure during model inference, which is beneficial to the speedup of model deployment.

[0064] According to any one of the above embodiments, in the embodiment:

[0065] The step of inputting the feature extraction result into a decoder layer of a recognition network to obtain a traffic sign recognition result comprises:

[0066] The feature extraction result is input into a decoder layer of a recognition network, and a sign position sub-branch, a sign size sub-branch, and a sign position offset sub-branch of a sign recognition branch of the decoder layer are used to obtain a sign position result, a sign size sub-result, and a sign position offset sub-result, respectively.

[0067] The embodiment provides a more specific recognition network structure. It is worth noting that although the description of "sub-branch" is used in the naming of the sign position sub-branch, the sign size sub-branch and the sign position offset sub-branch, the description cannot be understood as a limitation on the protection scope of the application. The sign position sub-branch, the sign size sub-branch and the sign position offset sub-branch can be a decoder layer branch equal in weight to the reinforcement training branch, or a sub-network formed for different tasks in the sign recognition branch.

[0068] In a preferred embodiment, as shown in Figure 2 The initial network is composed of the sign position sub-branch, the sign size sub-branch, the sign position offset sub-branch and the reinforcement training branch. As shown in Figure 3 The recognition network is composed of the sign position sub-branch, the sign size sub-branch and the sign position offset sub-branch.

[0069] The general traffic sign recognition device based on the dynamic network structure provided by the application is described below. The general traffic sign recognition device based on the dynamic network structure described below can be correspondingly referred to the general traffic sign recognition method based on the dynamic network structure described above.

[0070] The embodiment of the application also provides a general traffic sign recognition system based on a dynamic network structure, which is deployed in a vehicle terminal and comprises:

[0071] The acquisition module 1 is configured to acquire a road image based on a vehicle image acquisition device.

[0072] The feature extraction module 2 is configured to input the road image into a feature extraction layer of a recognition network to obtain a feature extraction result.

[0073] The decoder module 3 is configured to input the feature extraction result into a decoder layer of the recognition network to obtain a traffic sign recognition result.

[0074] The recognition network is obtained by excluding the decoder layer reinforcement training branch from the initial network comprising the feature extraction layer, the decoder layer reinforcement training branch and the decoder layer sign recognition branch after training of samples, labels and a loss function.

[0075] Further, the decoder layer reinforcement training branch is a decoder layer multi-sign region branch. The decoder layer multi-sign region branch can obtain a set multi-sign region according to the feature extraction result, and the intersection of the multi-sign region and the regions of at least two traffic signs is a non-empty set.

[0076] The multi-sign region comprises parallel pixels.

[0077] The parallel pixels refer to:

[0078] pixels on the adjacent parallel line segments of the set number of traffic sign center point connecting lines; and / or

[0079] pixels on the adjacent parallel line segments of the set number of traffic sign center point connecting lines.

[0080] The label comprises a multi-sign region binary graph; the multi-sign region binary graph comprises multi-sign region pixels and non-multi-sign region pixels respectively set as different binary values.

[0081] The loss function comprises a reinforcement branch term; the reinforcement branch term can quantify the difference between the multi-sign region output by the decoder layer reinforcement training branch and the multi-sign region binary graph in the label.

[0082] The decoder module 3 comprises:

[0083] The decoding unit is configured to input the feature extraction result into a decoder layer of a recognition network, and obtain a sign position result, a sign size sub-result and a sign position offset sub-result through a sign position sub-branch, a sign size sub-branch and a sign position offset sub-branch of a sign recognition branch of the decoder layer.

[0084] Figure 5 An example of an entity structure diagram of an electronic device is shown in FIG. 1. Figure 5 As shown in the figure, the electronic device can include a processor 510, a communications interface 520, a memory 530 and a communications bus 540, wherein the processor 510, the communications interface 520 and the memory 530 can communicate with each other through the communications bus 540. The processor 510 can call the logic instructions in the memory 530 to execute a general traffic sign recognition method based on a dynamic network structure, which comprises: acquiring a road image based on a vehicle-mounted image acquisition device; inputting the road image into a feature extraction layer of a recognition network to obtain a feature extraction result; inputting the feature extraction result into a decoder layer of the recognition network to obtain a traffic sign recognition result; the recognition network is obtained by excluding a decoder layer reinforcement training branch from an initial network comprising a feature extraction layer, a decoder layer reinforcement training branch and a decoder layer sign recognition branch after training of the initial network based on samples, labels and a loss function.

[0085] Further, the logic instructions in the memory 530 described above can be implemented in the form of software functional units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or parts of the present application that essentially contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0086] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor to enable a computer to execute the general traffic sign recognition method based on a dynamic network structure provided by the above-mentioned methods. The method comprises: acquiring a road image based on a vehicle-mounted image acquisition device; inputting the road image into a feature extraction layer of a recognition network to obtain a feature extraction result; inputting the feature extraction result into a decoder layer of the recognition network to obtain a traffic sign recognition result; and the recognition network is obtained by excluding a decoder layer reinforcement training branch from an initial network comprising the feature extraction layer, the decoder layer reinforcement training branch, and a decoder layer sign recognition branch after training of samples, labels, and a loss function.

[0087] In yet another aspect, the present application also provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the general traffic sign recognition method based on a dynamic network structure provided by the above-mentioned methods. The method comprises: acquiring a road image based on a vehicle-mounted image acquisition device; inputting the road image into a feature extraction layer of a recognition network to obtain a feature extraction result; inputting the feature extraction result into a decoder layer of the recognition network to obtain a traffic sign recognition result; and the recognition network is obtained by excluding a decoder layer reinforcement training branch from an initial network comprising the feature extraction layer, the decoder layer reinforcement training branch, and a decoder layer sign recognition branch after training of samples, labels, and a loss function.

[0088] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0089] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0090] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A general traffic sign recognition method based on dynamic network structure, characterized in that, The application is applied to a vehicle terminal, comprising: acquiring road images based on a vehicle image acquisition device; inputting the road images into a feature extraction layer of a recognition network to obtain feature extraction results; inputting the feature extraction results into a decoder layer of the recognition network to obtain traffic sign recognition results; the recognition network is obtained by excluding a decoder layer reinforcement training branch after an initial network including a feature extraction layer, the decoder layer reinforcement training branch, and a decoder layer sign recognition branch is trained by samples, labels, and a loss function; the reinforcement training branch is flexibly set according to different application scenarios. 2.The universal traffic sign recognition method based on dynamic network structure according to claim 1, characterized in that, the decoder layer reinforcement training branch is a decoder layer multi-sign region branch; the decoder layer multi-sign region branch can obtain a set multi-sign region according to the feature extraction results; an intersection of the multi-sign region and at least two traffic sign regions is a non-empty set. 3.The universal traffic sign recognition method based on dynamic network structure according to claim 2, characterized in that, the multi-sign region includes parallel pixels; the parallel pixels refer to: pixels on a traffic sign center point connecting line; and / or pixels on adjacent parallel line segments of a set number of traffic sign center point connecting lines. 4.The universal traffic sign recognition method based on dynamic network structure according to claim 2, characterized in that, the label includes a multi-sign region binary image; the multi-sign region binary image includes multi-sign region pixel points and non-multi-sign region pixel points respectively set as different binary values. 5.The universal traffic sign recognition method based on dynamic network structure according to claim 4, characterized in that, the loss function includes a reinforcement branch item; the reinforcement branch item can quantify a difference between multi-sign regions output by the decoder layer reinforcement training branch and a multi-sign region binary image in the label.

6. The dynamic network structure based general traffic sign recognition method according to any one of claims 1 to 5, characterized in that, the step of inputting the feature extraction results into the decoder layer of the recognition network to obtain the traffic sign recognition results includes: inputting the feature extraction results into the decoder layer of the recognition network to obtain a sign position result, a sign size sub-result, and a sign position offset sub-result through a sign position sub-branch, a sign size sub-branch, and a sign position offset sub-branch of the decoder layer sign recognition branch respectively.

7. A general traffic sign recognition system based on dynamic network structure, characterized by, deployed in a vehicle terminal, comprising: an acquisition module configured to acquire road images based on a vehicle image acquisition device; a feature extraction module configured to input the road images into a feature extraction layer of a recognition network to obtain feature extraction results; a decoder module configured to input the feature extraction results into a decoder layer of the recognition network to obtain traffic sign recognition results; the recognition network is obtained by excluding a decoder layer reinforcement training branch after an initial network including a feature extraction layer, the decoder layer reinforcement training branch, and a decoder layer sign recognition branch is trained by samples, labels, and a loss function; the reinforcement training branch is flexibly set according to different application scenarios.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, the processor implements the steps of the general traffic sign recognition method based on a dynamic network structure according to any one of claims 1 to 6 when executing the program. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, the computer program implements the steps of the general traffic sign recognition method based on a dynamic network structure according to any one of claims 1 to 6 when executed by the processor.

10. A computer program product comprising a computer program, characterized in that, the computer program implements the steps of the general traffic sign recognition method based on a dynamic network structure according to any one of claims 1 to 6 when executed by the processor.

Images