Traffic sign content identification method and device, storage medium and equipment

Abstract

The invention provides a traffic sign content recognition method and device, a storage medium and equipment, in the method, a target neural network model is trained to recognize traffic sign content, training data of the target neural network model is obtained by encoding sign content in a plurality of traffic signboard pictures, and in the application stage, the traffic sign content is recognized through the target neural network model. And inputting at least one image of a to-be-identified traffic signboard into the trained target neural network model to obtain target coding information, and determining the identification content corresponding to the to-be-identified traffic signboard according to the target coding information. Thus, the neural network algorithm is used for traffic sign content recognition, complex scenes can be better adapted, and the accuracy and robustness of traffic sign content recognition are improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, storage medium and equipment for identifying traffic sign content. Background Art

[0002] Traffic sign recognition is one of the key components of autonomous driving technology. It enables cars to automatically identify various traffic signs on the road and respond to them, and adjust the vehicle's motion state in real time. Most traffic signs are digital, such as speed limit signs and height limit signs. The traffic sign digital recognition method in the related art mainly extracts the traffic sign part by color segmentation, morphological processing, contour extraction and other operations on the image, and then performs digital recognition by template matching. The template is slid on the original image, and the similarity between the part of the image to be recognized and the template is calculated, and then the corresponding number is recognized. However, the robustness of this method is poor. The accuracy of template matching is used to recognize traffic sign numbers, which is sensitive to noise and interference. In actual scenes, there may be changes in lighting, fading of signs, motion blur and tilt occlusion. When faced with these situations, the accuracy of template matching is low. Summary of the invention

[0003] The purpose of this application is to provide a method, device, storage medium and equipment for traffic sign content recognition, aiming to solve the problem of poor robustness of traffic sign digital recognition methods in related technologies.

[0004] In a first aspect, the present application provides a method for identifying traffic sign content, comprising: obtaining at least one image of a traffic sign to be identified; processing the at least one image through a target neural network model to obtain target coding information; the target neural network model is trained based on multiple traffic sign images and annotation data corresponding to each traffic sign image as training data; the annotation data is generated by encoding the sign content in the traffic sign image; and determining the sign content corresponding to the traffic sign based on the target coding information.

[0005] In the above implementation process, the target neural network model is trained to identify the content of traffic signs. The training data of the target neural network model is obtained by encoding the content of the signs in multiple traffic sign pictures. In the application stage, at least one image of the traffic sign to be identified is input into the trained target neural network model to obtain the target encoding information, and then the content of the sign corresponding to the traffic sign to be identified is determined based on the target encoding information. In this way, the use of neural network algorithms for traffic sign content recognition can better adapt to complex scenarios and improve the accuracy and robustness of traffic sign content recognition.

[0006] Furthermore, in some examples, the annotation data is generated by encoding the identification content in the traffic sign image into four encoding bits using a one-hot encoding method, and each encoding bit includes 10 digits, weight-limited characters, height-limited and width-limited characters, an empty annotation flag, a termination flag, and a fill flag.

[0007] In the above implementation process, a 4×15 matrix is ​​used to encode the sign content in the traffic sign image. In this way, by standardizing the traffic signs, the target neural network model can better understand and recognize traffic signs during the training process, thereby improving the accuracy and robustness of the model.

[0008] Further, in some examples, the annotation data is generated based on the following method: if the sign content in any traffic sign image is empty, the sign content is encoded as a four-digit empty annotation flag; if the sign content in any traffic sign image is not empty, the digital part of the sign content is encoded, and then the weight limit character or the height limit and width limit character is encoded according to the unit part of the sign content; after completing the encoding of the digital part and the unit part, a termination flag is added to obtain an initial encoding result; when the initial encoding result has used up four encoding bits, the annotation data corresponding to the traffic sign image is generated based on the initial encoding result; when the initial encoding result has not used up four encoding bits, the blank encoding bits are filled by filling the flag bit, and the annotation data corresponding to the traffic sign image is generated based on the filled encoding content.

[0009] In the above implementation process, a specific method for encoding the sign content in the traffic sign image is provided.

[0010] Furthermore, in some examples, the target neural network model includes a backbone network, a neck, and a head; the backbone network adopts the MobileNetV2 architecture.

[0011] In the above implementation process, a lightweight neural network is used as the backbone network to improve the efficiency and accuracy of the model while maintaining low computational cost and memory usage.

[0012] Furthermore, in some examples, the neck of the target neural network model includes a feature pyramid network; the feature pyramid network receives features output from multiple target feature layers of the backbone network, performs a 1×1 convolution operation on the received features to connect them laterally, and then fuses the features of each layer through a top-down path.

[0013] In the above implementation process, the neck of the model adopts the FPN architecture, which fuses features of different levels through top-down paths and lateral connections, so that the network can simultaneously utilize high-level semantic information and low-level detail information.

[0014] Furthermore, in some examples, the head of the target neural network model receives features output from the lowest layer of the feature pyramid network, and performs four 3×3 convolution operations and one 8×8 convolution operation on the received features in sequence to obtain a vector of the target dimension; the vector of the target dimension represents the detection result of the target neural network model on the identification content in the input image.

[0015] In the above implementation process, a specific structure of the head of the target neural network is provided.

[0016] Furthermore, in some examples, the target neural network model is trained based on the following method: inputting a traffic sign image in the training data into the target neural network model to obtain a vector of a target dimension output by the head of the target neural network model; calculating a cross entropy loss based on the vector of the target dimension and the labeled data corresponding to the traffic sign image; and optimizing the parameters of the target neural network model based on the cross entropy loss.

[0017] In the above implementation process, a training method for the target neural network is provided.

[0018] In a second aspect, the present application provides a traffic sign content recognition device, comprising: an acquisition module, used to acquire at least one image of a traffic sign to be identified; a processing module, used to process the at least one image through a target neural network model to obtain target coding information; the target neural network model is trained based on multiple traffic sign images and the annotation data corresponding to each traffic sign image as training data; the annotation data is generated by encoding the sign content in the traffic sign image; a determination module, used to determine the sign content corresponding to the traffic sign according to the target coding information.

[0019] In a third aspect, the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any one of the first aspects when executing the computer program.

[0020] In a fourth aspect, the present application provides a computer-readable storage medium, on which instructions are stored. When the instructions are executed on a computer, the computer executes the method as described in any one of the first aspects.

[0021] In a fifth aspect, the present application provides a computer program product, which, when running on a computer, enables the computer to execute the method as described in any one of the first aspects.

[0022] Other features and advantages disclosed in the present application will be described in the following description, or some features and advantages can be inferred or determined without doubt from the description, or can be learned by implementing the above-mentioned technology disclosed in the present application.

[0023] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0025] Figure 1 A flow chart of a method for identifying traffic sign content provided in an embodiment of the present application;

[0026] Figure 2 A schematic diagram of a method for encoding a sign with a speed limit of 80 provided in an embodiment of the present application;

[0027] Figure 3 A schematic diagram of the Backbone structure of a neural network model provided in an embodiment of the present application;

[0028] Figure 4 A schematic diagram of the structure of Neck of a neural network model provided in an embodiment of the present application;

[0029] Figure 5 A schematic diagram of the structure of the Head of a neural network model provided in an embodiment of the present application;

[0030] Figure 6 A block diagram of a traffic sign content recognition device provided in an embodiment of the present application;

[0031] Figure 7 A structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0033] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0034] As described in the background art, the traffic sign digital recognition method in the related art has the problem of poor robustness. Based on this, the embodiment of the present application provides a traffic sign content recognition solution to solve the above problem.

[0035] Next, the embodiments of the present application are introduced:

[0036] like Figure 1 As shown, Figure 1 The flowchart of a traffic sign content recognition method provided in an embodiment of the present application is shown in FIG. The method can be applied to a controller of a car, such as an autonomous driving domain controller.

[0037] The method comprises:

[0038] Step 101: Acquire at least one image of a traffic sign to be identified;

[0039] Traffic signs refer to signs used for traffic management purposes such as indication, warning, restriction, and prohibition in road traffic. They are usually composed of elements such as graphics, text, and color, and have clear meanings and indicative functions. The traffic sign to be identified mentioned in this step may refer to a traffic sign with numbers that appears in front of the car during driving, and its identification content may include at least one of the content such as speed limit signs, weight limit signs, height limit signs, and width limit signs. The image of the traffic sign to be identified can be acquired through a vehicle-mounted camera. The controller can extract one or more frames of images from the real-time video stream of the vehicle-mounted camera, and then identify the identification content corresponding to the traffic sign by analyzing the image, and then make reasonable control decisions based on this.

[0040] Step 102: Processing the at least one image through a target neural network model to obtain target coding information; the target neural network model is trained based on using multiple traffic sign images and annotation data corresponding to each traffic sign image as training data; the annotation data is generated by encoding the identification content in the traffic sign image;

[0041] This embodiment trains a neural network model to identify the content of traffic signs. The training data of the neural network model is obtained by collecting multiple traffic sign pictures and encoding the sign content in these traffic sign pictures. Therefore, the neural network model can automatically learn the features in the image and output target encoding information representing the traffic sign content in the image. Compared with the template matching method, the machine learning method can better adapt to the recognition of traffic sign numbers in different scenes and conditions, such as different lighting, weather, occlusion and distortion. Based on this adaptability, the embodiment is more robust in practical applications.

[0042] In some embodiments, the annotation data mentioned in this step is generated by encoding the identification content in the traffic sign image into four encoding bits by using one-hot encoding, and each encoding bit covers 10 digits, weight limit characters, height and width limit characters, empty annotation signs, termination flags and fill flags. That is to say, when constructing training data, the identification content in the traffic sign image can be encoded by a 4×15 matrix, and the traffic sign is encoded into four bits by using one-hot encoding, and each bit covers 10 digits from 0 to 9, weight limit characters t (tons), height and width limit characters m (meters), empty annotation signs, termination flags and fill flags, a total of 15 types. In this way, by standardizing the traffic signs, the target neural network model can better understand and identify traffic signs during the training process, thereby improving the accuracy and robustness of the model.

[0043] Further, in some embodiments, the annotation data mentioned in this step is generated based on the following method; if the identification content in any traffic sign image is empty, the identification content is encoded as a four-digit empty annotation flag; if the identification content in any traffic sign image is not empty, the digital part of the identification content is encoded, and then the weight limit character or the height limit and width limit character is encoded according to the unit part of the identification content; after completing the encoding of the digital part and the unit part, a termination flag is added to obtain an initial encoding result; when the initial encoding result has used up four encoding bits, the annotation data corresponding to the traffic sign image is generated based on the initial encoding result; when the initial encoding result has not used up four encoding bits, the blank encoding bits are filled by filling the flag bit, and the annotation data corresponding to the traffic sign image is generated based on the filled encoding content. That is to say, for the collected samples, if the corresponding traffic sign is empty, it means that the traffic sign is damaged, missing, not set, or has a special meaning, etc., and it is encoded as a four-digit empty mark. If the corresponding traffic sign is not empty, the digital part is encoded first, and then the weight limit character t or the height limit character m is encoded according to the unit part, and then a termination flag is added. If the four-digit coding bit has not been used up at this time, a fill flag is added at the end to fill the blank bit. In this way, the encoding of traffic sign data is realized.

[0044] In addition, when encountering height and width limit signs, since the height and width limit signs are all one-digit decimals less than 10, the decimal point can be ignored when encoding the digital part, and only the digits to the left and right of the decimal point are encoded. If the decimal part is 0, the decimal place is ignored. Taking the encoding process of the height limit 4.5m sign as an example, the digital part is 4.5, so the first encoding position is the number 4, the second encoding position is the number 5, and the unit part is m, so the third encoding position is the height and width limit character m, and the fourth encoding position is the end mark position. Assume that a is used. i.j represents the element in the i-th row and j-th column of the matrix, i = 1, 2, 3, 4, j = 1, 2, ..., 15, then the identification data generated for the sign is a 4 × 15 matrix, and the a in the matrix 1,5 、a 2,6 、a 3,11 、a 4,14 These four elements are 1, and the rest are 0.

[0045] In some embodiments, the target neural network model mentioned in this step includes a backbone network, a neck and a head; the backbone network adopts the MobileNetV2 architecture. The backbone network (Backbone), the neck (Neck) and the head (Head) are different parts of the network structure, wherein the Backbone is responsible for extracting the features of the input data, the Neck is responsible for further processing and integrating the features extracted by the Backbone, and the Head is responsible for mapping the features extracted by the Neck to the final output space to generate the final prediction results of the network. MobileNetV2 is a lightweight convolutional neural network, whose main components include an inverted residual block, a linear bottleneck and a depth-separable convolution, wherein the inverted residual block is one of the core innovations of MobileNetV2, the inverted residual block first uses 1×1 convolution to achieve dimensionality increase, then extracts features through 3×3 convolution, and finally uses 1×1 convolution to achieve dimensionality reduction, the inverted residual block reduces the computational cost to a lower level by performing a depth-separable convolution after channel expansion, thereby achieving efficient feature extraction in a lightweight network. In this way, by adopting the lightweight neural network MobileNetV2, the efficiency and accuracy of the model can be improved while maintaining low computational costs and memory usage.

[0046] In some embodiments, the neck of the target neural network model mentioned in this step includes a feature pyramid network; the feature pyramid network receives features output from multiple target feature layers of the backbone network, performs a 1×1 convolution operation on the received features to connect them laterally, and then fuses the features of each layer through a top-down path. That is, the neck of the target neural network model adopts an FPN (Feature Pyramid Network) architecture, and FPN receives features of different levels passed in by Backbone, performs a 1×1 convolution operation on these features to connect them laterally so that the dimensions of each channel match, and then fuses the features of each layer through a top-down path, so that the network can simultaneously utilize high-level semantic information and low-level detail information. Among them, multiple target feature layers can be key feature layers in Backbone for extracting low-level detail information, and key feature layers for extracting high-level semantic information. For example, when MobileNetV2 is used as Backbone, multiple target feature layers can include the 4th, 5th, 7th, and 9th layers.

[0047] Furthermore, in some embodiments, the head of the target neural network model mentioned in this step receives the features output from the lowest layer of the feature pyramid network, and sequentially performs four 3×3 convolution operations and one 8×8 convolution operation on the received features to obtain a vector of the target dimension; the vector of the target dimension represents the detection result of the target neural network model on the identification content in the input image. That is to say, in the target neural network model, the features output from the lowest layer in the FPN structure of Neck will be input to the Head for detection, and the Head will sequentially perform four 3×3 convolution operations and one 8×8 convolution operation on the received features to finally obtain a vector of the target dimension, which contains the model's prediction result on the identification content in the input image.

[0048] In addition, in some embodiments, the target neural network model mentioned in this step is trained based on the following method: input the traffic sign image in the training data into the target neural network model to obtain the vector of the target dimension output by the head of the target neural network model; calculate the cross entropy loss based on the vector of the target dimension and the labeled data corresponding to the traffic sign image; optimize the parameters of the target neural network model based on the cross entropy loss. That is to say, during the model training process, the loss function of the model adopts the cross entropy loss function, and the training data is processed by the model to obtain the prediction result of the model, and then combined with the true value, that is, the labeled data corresponding to the traffic sign image, the cross entropy loss is calculated to adjust the parameters of the model. The iterative goal of the model is to find a set of parameters so that the loss function reaches the minimum value. Through this process, the model gradually learns how to better predict the true value of the input data, thereby improving the accuracy of the model.

[0049] Step 103: Determine the sign content corresponding to the traffic sign according to the target coding information.

[0050] In this embodiment, the target coding information output by the target neural network model is the coding information corresponding to the traffic sign in the traffic sign to be identified. The target coding information can be decoded using the coding method corresponding to the training data to determine the sign content corresponding to the traffic sign.

[0051] In the embodiment of the present application, a target neural network model is trained to identify the content of traffic signs. The training data of the target neural network model is obtained by encoding the content of signs in multiple traffic sign pictures. In the application stage, at least one image of the traffic sign to be identified is input into the trained target neural network model to obtain target encoding information, and then the content of the sign corresponding to the traffic sign to be identified is determined based on the target encoding information. In this way, the use of a neural network algorithm to identify the content of traffic signs can better adapt to complex scenarios and improve the accuracy and robustness of traffic sign content recognition.

[0052] In order to explain the solution of this application in more detail, a specific embodiment is introduced below:

[0053] This embodiment provides a traffic sign recognition solution based on machine learning. This embodiment solution includes:

[0054] S201, traffic sign data coding;

[0055] Specifically, multiple traffic sign images with numbers are collected as training data, and the training data is encoded through a 4×15 matrix; the traffic signs are encoded into four bits, each of which covers 10 digits from 0 to 9, a weight limit character t (tons), a height limit character m (meters), an empty mark flag f, an end flag EOF, and a padding flag PAD, a total of 15 types, using a one-hot encoding method;

[0056] When encoding, encode the numeric part first. If you encounter a height and width restriction sign, ignore the decimal point and only encode the digits to the left and right of the decimal point. If the decimal part is 0, ignore the decimal place and encode t or m according to the unit after encoding the numeric part. Then, add an EOF. If the four digits of the code have not been used up at this time, add PAD to fill the blanks. If the traffic sign is blank, encode it as four digits f.

[0057] Take a sign with a speed limit of 80 as an example. Figure 2 is a schematic diagram of a method for encoding the identification plate provided in this embodiment, wherein the first encoding bit is the number 8, the second encoding bit is the number 0, the third encoding bit is EOF, and the fourth encoding bit is PAD. Based on this, a 4×15 matrix can be generated, using a i.j represents the element in the i-th row and j-th column of the matrix, then a in the matrix 1,9 、a 2,1 、a 3,14 、a 4,15 These four elements are 1, and the rest are 0; each sign is encoded to obtain a 4×15 matrix;

[0058] S202, neural network model training;

[0059] Specifically, a neural network model is constructed, which consists of Backbone, Neck and Head, where:

[0060] Backbone's structure is as follows Figure 3 As shown in the figure, it uses the MobileNetV2 architecture, which greatly reduces the number of model parameters and is more suitable for deployment in resource-constrained environments such as mobile devices. Backbone's input is a data set with a batch size of 8, and the size of each sign image is 32×32, containing 3 channels. Figure 3 The boxes in the figure are the dimensions of the data;

[0061] The structure of Neck is as follows Figure 4 As shown in the figure, it adopts the FPN architecture. FPN receives the features output by the 4th, 5th, 7th, and 9th layers in Backbone, performs 1×1 convolution operations on the features passed in from Backbone to connect them horizontally so that the dimensions of each channel match, and then fuses the features of each layer through a top-down path.

[0062] The structure of Head is as follows Figure 5 As shown, it receives the feature with the dimension (8, 32, 8, 8) of the lowest layer output in the FPN structure of Neck, and finally obtains a vector with the dimension (8, 60, 1, 1);

[0063] During the training process, the vector with dimension (8,60,1,1) obtained in the Head is divided into four vectors with dimension (8,15) to represent the four-bit encoding. The cross entropy loss is calculated with the true value, and the model parameters are optimized using the cross entropy loss. When the cross entropy loss is less than the set value, the model training is completed.

[0064] S203, traffic sign recognition;

[0065] Specifically, in practical applications, the image of the traffic sign to be identified is captured by a vehicle-mounted camera, the image is input into a trained neural network model, and then the sign content corresponding to the traffic sign to be identified is determined based on the output result of the neural network model.

[0066] This embodiment scheme can accurately identify traffic signs with numbers on the road, such as speed limit signs, weight limit signs, height limit signs, etc., and uses a relatively lightweight neural network and can be deployed on a mobile terminal.

[0067] Corresponding to the embodiments of the aforementioned method, the present application also provides embodiments of a traffic sign content recognition device and a terminal using the same:

[0068] like Figure 6 As shown, Figure 6 : is a block diagram of a traffic sign content recognition device provided in an embodiment of the present application, the device comprising:

[0069] An acquisition module 61, used to acquire at least one image of a traffic sign to be identified;

[0070] A processing module 62 is used to process the at least one image through a target neural network model to obtain target coding information; the target neural network model is trained based on using multiple traffic sign images and annotation data corresponding to each traffic sign image as training data; the annotation data is generated by encoding the identification content in the traffic sign image;

[0071] The determination module 63 is used to determine the identification content corresponding to the traffic sign according to the target coding information.

[0072] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0073] This application also provides an electronic device, see Figure 7 , Figure 7 A block diagram of an electronic device provided in an embodiment of the present application. The electronic device may include a processor 710, a communication interface 720, a memory 730, and at least one communication bus 740. The communication bus 740 is used to realize direct connection and communication between these components. The communication interface 720 of the electronic device in the embodiment of the present application is used to communicate signaling or data with other node devices. The processor 710 may be an integrated circuit chip with signal processing capabilities.

[0074] The processor 710 mentioned above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor 710 can also be any conventional processor, etc.

[0075] The memory 730 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electric erasable programmable read-only memory (EEPROM), etc. The memory 730 stores computer-readable instructions. When the computer-readable instructions are executed by the processor 710, the electronic device can execute the above-mentioned Figure 1 The method embodiment involves various steps.

[0076] Optionally, the electronic device may further include a storage controller and an input / output unit.

[0077] The memory 730, storage controller, processor 710, peripheral interface, input and output unit components are directly or indirectly electrically connected to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses 740. The processor 710 is used to execute executable modules stored in the memory 730, such as software function modules or computer programs included in the electronic device.

[0078] The input and output unit is used to provide users with the task creation and to create a start optional time period or preset execution time for the task to realize the interaction between the user and the server. The input and output unit can be, but is not limited to, a mouse and a keyboard.

[0079] Understandably, Figure 7 The structure shown is for illustration only, and the electronic device may also include Figure 7 More or fewer components as shown, or with Figure 7 Different configurations are shown. Figure 7 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0080] An embodiment of the present application further provides a storage medium having instructions stored thereon. When the instructions are run on a computer, the computer program is executed by a processor to implement the method described in the method embodiment. To avoid repetition, the method will not be described here.

[0081] The present application also provides a computer program product, which, when executed on a computer, enables the computer to execute the method described in the method embodiment.

[0082] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0083] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0084] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0085] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0086] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0087] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A method for identifying traffic sign content, characterized in that: include: Acquire at least one image of a traffic sign to be identified; Processing the at least one image through a target neural network model to obtain target coding information; the target neural network model is trained based on using multiple traffic sign images and annotation data corresponding to each traffic sign image as training data; the annotation data is generated by encoding the sign content in the traffic sign image; The identification content corresponding to the traffic sign is determined according to the target coding information.

2. The method according to claim 1, characterized in that The labeling data is generated by encoding the identification content in the traffic sign image into four encoding bits using a one-hot encoding method, each encoding bit includes 10 digits, weight-limited characters, height-limited and width-limited characters, an empty labeling flag, a termination flag, and a fill flag.

3. The method according to claim 2, characterized in that The annotation data is generated based on the following method: If the sign content in any traffic sign image is empty, the sign content is encoded as a four-digit empty mark; If the sign content in any traffic sign image is not empty, encode the digital part of the sign content, and then encode the weight limit character or the height and width limit character according to the unit part of the sign content; after completing the encoding of the digital part and the unit part, add a termination flag to obtain an initial encoding result; when the initial encoding result has used up four encoding bits, generate the annotation data corresponding to the traffic sign image based on the initial encoding result; when the initial encoding result has not used up four encoding bits, fill the blank encoding bits by filling the flag, and generate the annotation data corresponding to the traffic sign image based on the filled encoding content.

4. The method according to claim 1, characterized in that: The target neural network model includes a backbone network, a neck and a head; the backbone network adopts the MobileNetV2 architecture.

5. The method according to claim 4, characterized in that The neck of the target neural network model includes a feature pyramid network; The feature pyramid network receives features output from multiple target feature layers of the backbone network, performs a 1×1 convolution operation on the received features to connect them horizontally, and then fuses the features of each layer through a top-down path.

6. The method according to claim 5, characterized in that The head of the target neural network model receives features output from the lowest layer of the feature pyramid network, and performs four 3×3 convolution operations and one 8×8 convolution operation on the received features in sequence to obtain a vector of the target dimension; the vector of the target dimension represents the detection result of the target neural network model on the identification content in the input image.

7. The method according to claim 6, characterized in that The target neural network model is trained based on the following method: Inputting the traffic sign picture in the training data into the target neural network model to obtain a vector of the target dimension output by the head of the target neural network model; Calculating a cross entropy loss based on the vector of the target dimension and the labeled data corresponding to the traffic sign image; Optimize parameters of the target neural network model based on the cross entropy loss.

8. A traffic sign content recognition device, characterized in that: include: An acquisition module, used to acquire at least one image of a traffic sign to be identified; A processing module, configured to process the at least one image through a target neural network model to obtain target coding information; the target neural network model is trained based on using a plurality of traffic sign images and annotation data corresponding to each traffic sign image as training data; the annotation data is generated by encoding the identification content in the traffic sign image; The determination module is used to determine the identification content corresponding to the traffic sign according to the target coding information.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images