Traffic sign category determination method, apparatus, vehicle, and electronic device

Abstract

The application discloses a traffic sign category determination method and device, a vehicle and an electronic device, and relates to the technical field of vehicles. The method comprises the following steps: acquiring a to-be-tested image, wherein the to-be-tested image is an image comprising a traffic sign; performing gray scale processing on the to-be-tested image to obtain a position image, wherein the position image is used for representing the position of the traffic sign in the to-be-tested image; and determining a recognition result according to the to-be-tested image and the position image, wherein the recognition result is used for representing the category of the traffic sign. The application solves the technical problem that, in the prior art, the category of a traffic sign is identified by using a traditional computer graphics method, a machine learning method and other deep learning algorithms, so that the neural network model is complex, the recognition rate is relatively low, and the classification effect is not ideal.

Description

Technical Field

[0001] This invention relates to the field of vehicle technology, and more specifically, to a method, apparatus, vehicle, and electronic device for determining traffic sign categories. Background Technology

[0002] Traffic signs are a crucial source of information for drivers to obtain about road conditions. However, due to factors such as different image acquisition devices, lighting conditions, and weather, it is difficult for vehicle-mounted cameras to capture clear close-up images at high speeds. Therefore, methods for classifying traffic signs are essential.

[0003] Currently, traffic sign categories are identified using traditional computer graphics and machine learning methods. However, these methods are easily affected by occlusion, deformation, and changes in viewing angle, resulting in low recognition efficiency. Alternatively, other deep learning algorithms can be used to identify traffic sign categories, but these methods suffer from imbalanced datasets, small target recognition tasks, and various factors such as weather conditions and occlusion, leading to complex neural network models, low recognition rates, and unsatisfactory classification results.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a method, apparatus, vehicle, and electronic device for determining traffic sign categories, to at least solve the technical problem in related technologies where identifying traffic sign categories using traditional computer graphics, machine learning methods, and other deep learning algorithms results in complex neural network models, low recognition rates, and unsatisfactory classification effects.

[0006] According to one embodiment of the present invention, a method for determining the category of traffic signs is provided, comprising: acquiring a test image, wherein the test image is an image including traffic signs; performing grayscale processing on the test image to obtain a position image, wherein the position image is used to represent the position of the traffic signs in the test image; and determining a recognition result based on the test image and the position image, wherein the recognition result is used to represent the category of the traffic signs.

[0007] Optionally, the method further includes: acquiring a first image sample and a first base model; inputting the first image sample into the first base model for first image processing to obtain a first image set; performing segmentation processing on the first image sample to obtain a second image sample; and training the first base model based on the first image set and the second image sample to obtain a first target model.

[0008] Optionally, the method further includes: inputting a first image sample into a first target model for first image processing to obtain a second image set; inputting the first image sample into a second base model for second image processing to obtain a first location image set; fusing the first image sample, the second image set, and the first location image set to obtain a third image set; and training the second base model based on the third image set and the first image sample to obtain a second target model.

[0009] Optionally, performing grayscale processing on the image to be tested to obtain the image of the location to be tested includes: inputting the image to be tested into a second target model for grayscale processing to obtain the location image.

[0010] Optionally, the method further includes: obtaining a base classification model, wherein the base classification model is used to identify the category of traffic signs in the image; and training the base classification model based on the image to be tested and the location image to obtain a target classification model.

[0011] Optionally, determining the recognition result based on the image to be tested and the image at the location to be tested includes: inputting the image to be tested and the image at the location to be tested into the target classification model to obtain the recognition result.

[0012] Optionally, the first image sample includes a first label and a second label, wherein the first label is used to segment the traffic sign image and the second label is used to detect the traffic sign image. Cutting the first image sample to obtain the second image sample includes cutting the first image sample according to the first label and the second label to obtain the second image sample.

[0013] According to one embodiment of the present invention, a traffic sign category determination device is also provided, comprising: an acquisition module for acquiring a test image, wherein the test image is an image including traffic signs; a grayscale processing module for performing grayscale processing on the test image to obtain a position image, wherein the position image is used to represent the position of the traffic sign in the test image; and a determination module for determining a recognition result based on the test image and the position image, wherein the recognition result is used to represent the category of the traffic sign.

[0014] Optionally, the grayscale processing module is also used to acquire a first image sample and a first base model; input the first image sample into the first base model for first image processing to obtain a first image set; perform segmentation processing on the first image sample to obtain a second image sample; and train the first base model based on the first image set and the second image sample to obtain a first target model.

[0015] Optionally, the grayscale processing module is further configured to input the first image sample into the first target model for first image processing to obtain a second image set; input the first image sample into the second base model for second image processing to obtain a first position image set; fuse the first image sample, the second image set, and the first position image set to obtain a third image set; and train the second base model based on the third image set and the first image sample to obtain a second target model.

[0016] Optionally, the grayscale processing module is also used to input the image to be tested into the second target model for grayscale processing to obtain a position image.

[0017] Optionally, the determination module is also used to obtain a basic classification model, wherein the basic classification model is used to identify the category of traffic signs in the image; the basic classification model is trained based on the image to be tested and the location image to obtain the target classification model.

[0018] Optionally, the determination module is also used to input the image to be tested and the location image into the target classification model to obtain the recognition result.

[0019] Optionally, the grayscale processing module is also used to cut the first image sample according to the first label and the second label to obtain the second image sample.

[0020] According to one embodiment of this application, a vehicle is also provided, which is used to perform the method for determining the traffic sign category in any of the above claims.

[0021] According to one embodiment of the present invention, a computer-readable storage medium is also provided, wherein the storage medium stores a computer program, wherein the computer program is configured to execute the method for determining the traffic sign category as described above when running on a computer or processor.

[0022] According to one embodiment of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the method for determining the traffic sign category as described above.

[0023] In this embodiment of the invention, a test image is acquired, which includes traffic signs. The test image is then processed in grayscale to obtain a location image, which represents the position of the traffic signs in the test image. A recognition result is determined based on the test image and the location image, whereby the recognition result represents the category of the traffic signs. This allows the determination of traffic sign categories to be divided into two parts: localization of the region to be classified and category recognition of the region to be classified. Furthermore, by generating an image representing the position of the traffic signs in the test image, the accuracy of the localization of the region to be classified is further improved. This enhances the classification effect of the downstream classification network, resulting in a high recognition rate, ideal classification effect, and ease of implementation. This solves the technical problem in related technologies where the identification of traffic sign categories using traditional computer graphics, machine learning methods, and other deep learning algorithms leads to complex neural network models, low recognition rates, and unsatisfactory classification effects. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0025] Figure 1 This is a flowchart of a method for determining traffic sign categories according to one embodiment of the present invention;

[0026] Figure 2 This is a diagram of a difference detection network structure according to one embodiment of the present invention;

[0027] Figure 3(a) is a schematic diagram of an image before cutting according to one embodiment of the present invention;

[0028] Figure 3(b) is a schematic diagram of an image after segmentation according to one embodiment of the present invention;

[0029] Figure 4 This is a schematic diagram of a position pre-training mechanism according to one embodiment of the present invention;

[0030] Figure 5 This is a structural block diagram of a traffic sign category determination device according to one embodiment of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] According to one embodiment of the present invention, an embodiment of a method for determining traffic sign categories is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0034] This method embodiment can be executed in an electronic device, similar control device, or system that includes a memory and a processor. Taking an electronic device as an example, the electronic device may include one or more processors and a memory for storing data. Optionally, the electronic device may also include a communication device for communication functions and a display device. Those skilled in the art will understand that the above structural description is merely illustrative and does not limit the structure of the electronic device. For example, the electronic device may include more or fewer components than described above, or have a different configuration than described above.

[0035] A processor may include one or more processing units. For example, a processor may include a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processing (DSP) chip, a microcontroller unit (MCU), a field-programmable gate array (FPGA), a neural network processing unit (NPU), a tensor processing unit (TPU), or an artificial intelligence (AI) processor. Different processing units may be independent components or integrated into one or more processors. In some instances, electronic devices may also include one or more processors.

[0036] The memory can be used to store computer programs, such as the computer program corresponding to the traffic sign category determination method in this embodiment of the invention. The processor implements the traffic sign category determination method by running the computer program stored in the memory. The memory may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to electronic devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0037] Communication devices are used to receive or send data via a network. Specific examples of such networks may include wireless networks provided by the mobile terminal's communication provider. In one example, the communication device includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the communication device may be a radio frequency (RF) module used for wireless communication with the Internet.

[0038] The display device can be, for example, a touchscreen liquid crystal display (LCD) and a touch display (also referred to as a "touchscreen" or "touch screen"). This LCD allows the user to interact with the user interface of the mobile terminal. In some embodiments, the mobile terminal has a graphical user interface (GUI), which allows the user to interact with the GUI by touching and / or gesturing on a touch-sensitive surface. Optional human-computer interaction functions include: creating web pages, drawing, word processing, creating electronic documents, playing games, video conferencing, instant messaging, sending and receiving emails, a call interface, playing digital video, playing digital music, and / or web browsing, etc. Executable instructions for performing the above human-computer interaction functions are configured / stored in one or more processor-executable computer program products or readable storage media.

[0039] This embodiment provides a method for determining the category of traffic signs operating on electronic devices. Figure 1 This is a flowchart of a method for determining traffic sign categories according to one embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0040] Step S10: Obtain the image to be tested;

[0041] The image to be tested is an image that includes traffic signs.

[0042] The image to be tested can be understood as an image for which the category of traffic sign needs to be determined. Optionally, the image to be tested can be an image from the domestic traffic sign dataset TT100K, or a real-scene image captured by a real vehicle camera. This embodiment of the invention does not impose any restrictions.

[0043] It is understandable that images including traffic signs generally contain other scene factors, such as roads, vehicles, pedestrians, and complex background elements along the roadside. At the same time, images including traffic signs generally also contain small targets, strong light, complex backgrounds, and obstructions, meaning that the proportion of traffic signs in the image to be tested is relatively small.

[0044] Step S11: Perform grayscale processing on the image to be tested to obtain a position image;

[0045] The location image is used to represent the position of traffic signs in the image to be tested.

[0046] This step can be understood as performing grayscale processing on the image including traffic signs to obtain the position of the traffic signs in the image to be tested. It can be understood that the traffic signs occupy a small proportion in the image to be tested. By performing grayscale processing on the image including traffic signs to obtain the position of the traffic signs in the image to be tested, the specific location of the traffic signs in the image to be tested can be accurately determined, thereby providing an accurate recognition range for subsequent determination of recognition results.

[0047] Optionally, a target model can be obtained by training the model, and the image to be tested can be input into the target model for grayscale processing to obtain a position image. This embodiment of the invention does not limit the scope of the invention.

[0048] Step S12: Determine the recognition result based on the image to be tested and the location image.

[0049] The recognition results are used to indicate the category of traffic signs.

[0050] This step can be understood as determining the category of traffic signs based on the position of the traffic signs in the image to be tested. That is, after determining the specific position of the traffic signs in the image to be tested, the category of traffic signs in the image to be tested is determined in combination with that specific position.

[0051] Optionally, a target classification model can be obtained by training the model, and the image to be tested and the location image can be input into the target classification model to obtain the recognition result. This embodiment of the invention does not limit the scope of the invention.

[0052] It is understandable that by first processing the image to be tested in grayscale to obtain a location image, the specific location of the traffic sign in the image to be tested can be obtained, that is, the image range to be identified can be determined. Then, the category of the traffic sign in the image to be tested can be determined by combining the image range, thereby improving the efficiency and accuracy of category determination.

[0053] Through the above steps, a test image (containing traffic signs) is acquired, and grayscale processing is performed on the test image to obtain a location image. This location image represents the position of the traffic signs within the test image. The recognition result is determined based on the test image and the location image, and this result represents the category of the traffic signs. This process divides the determination of traffic sign categories into two parts: localization of the region to be classified and category recognition of the region to be classified. Furthermore, by generating an image representing the position of the traffic signs within the test image, the accuracy of the localization of the region to be classified is further improved. This enhances the classification performance of the downstream classification network, achieving a high recognition rate, ideal classification effect, and ease of implementation. It thus solves the technical problem in related technologies where traditional computer graphics, machine learning methods, and other deep learning algorithms for traffic sign category recognition result in complex neural network models, low recognition rates, and unsatisfactory classification effects.

[0054] Optionally, step S11 may also include the following execution steps:

[0055] Step S110: Obtain the first image sample and the first basic model;

[0056] The first image sample can be understood as an image dataset including traffic signs, such as the commonly used traffic sign dataset TT100K, which is not limited in this embodiment of the invention. For example, the commonly used traffic sign dataset TT100K is selected as the first image sample. The images in this first image sample are all 2048x2048 in size and contain 221 traffic sign image category labels.

[0057] The first basic model can be understood as the original model used to generate images without traffic signs, such as a generative network, which is not limited in this embodiment of the invention.

[0058] Step S111: Input the first image sample into the first basic model to perform the first image processing to obtain the first image set;

[0059] The first image processing can be understood as processing an image containing traffic signs into an image without traffic signs. The first image set can be understood as a dataset of images without traffic signs corresponding to the first image sample. This step can be understood as inputting the image dataset of traffic signs into the first basic model to perform the first image processing, thereby obtaining the image dataset without traffic signs corresponding to the first image sample.

[0060] Specifically, the first basic model first encodes the first image sample to 128x128 and then decodes it to 2048x2048, thereby preserving as much information as possible in the original image except for the location of traffic signs, and generating a first image set without traffic signs corresponding to the first image sample. This embodiment of the invention does not limit this.

[0061] Optionally, the first image sample can be the commonly used traffic sign dataset TT100K, and the first base model can be a generative network; this embodiment of the invention is not limited thereto. For example, the commonly used traffic sign dataset TT100K is input into the generative network, which encodes the images in TT100K to 128x128 and then decodes them to 2048x2048, obtaining the image dataset without traffic signs corresponding to TT100K, i.e., the first image set.

[0062] Step S112: The first image sample is segmented to obtain the second image sample;

[0063] The second image sample can be understood as an image dataset without traffic signs, which has the same number of images as the first image sample. This step can be understood as cutting the first image sample to obtain an image dataset without traffic signs, which has the same number of images as the first image sample.

[0064] Optionally, the first image sample can be the commonly used traffic sign dataset TT100K, which is not limited in this embodiment of the invention. For example, images in TT100K are segmented to obtain an image dataset without traffic signs corresponding to TT100K, i.e., the second image sample.

[0065] Step S113: Train the first base model based on the first image set and the second image samples to obtain the first target model.

[0066] The first target model can be understood as a model that can generate a first set of images that are identified as real images; that is, the first set of images generated by the first target model can be identified as real images.

[0067] Optionally, a discrimination result can be obtained by distinguishing between the first image set and the second image sample. The discrimination result is used to indicate whether the first image set is true or false. In response to the discrimination result being true, the first target model is obtained. This embodiment of the invention is not limited.

[0068] It is understandable that the first image set is an image dataset without traffic signs that has undergone first image processing based on the first image sample, and the second image sample is an image dataset without traffic signs that has undergone segmentation processing based on the first image sample. When the discrimination result is true, it means that the first image set is close to the real second image sample. At this time, the first basic model can generate a first image set that is close to the real image, that is, the first target model is obtained.

[0069] Optionally, step S11 may also include the following execution steps:

[0070] Step S114: Input the first image sample into the first target model for first image processing to obtain the second image set;

[0071] The second image set can be understood as a dataset of images without traffic signs that can be identified as real images. This step can be understood as inputting the first image sample into the first target model for first image processing to obtain the second image set without traffic signs that can be identified as real images.

[0072] Step S115: Input the first image sample into the second basic model for second image processing to obtain the first location image set;

[0073] The second basic model can be understood as a model that captures the location information of traffic signs in an image and finally outputs a single-channel continuous grayscale image. This step can be understood as inputting the first image sample into the second basic model to obtain the grayscale image corresponding to the first image sample, i.e., the first location image set.

[0074] Optionally, the first image sample can be the commonly used traffic sign dataset TT100K, which is not limited in this embodiment of the invention. For example, TT100K is input into the second base model, and the traffic sign location information of the images in TT100K is captured and output as a single-channel continuous grayscale image, with the value range of each pixel being [0,1], thus obtaining the first location image set.

[0075] Step S116: The first image sample, the second image set, and the first location image set are fused to obtain the third image set;

[0076] The third image set can be understood as an image dataset that includes traffic signs and is obtained by fusing the first image sample, the second image set, and the first location image set. This step can be understood as fusing the first image sample, the second image set (which can be identified as real images and does not contain traffic signs), and the first location image set (which represents the location information of traffic signs) to obtain the third image set that includes traffic signs.

[0077] Optionally, the first image sample, the second image set, and the first position image set can be fused using a dot product method, which is not limited in this embodiment of the invention. Specifically, the first image sample can be denoted as A, the second image set as B′, and the first position image set as M. The third image set A′ can then be calculated using a mathematical formula, as shown in the following formula (1):

[0078] A′=A×M+B′×(1-M)(1)

[0079] This results in a third image set, which is not limited in the embodiments of the present invention.

[0080] Step S117: Train the second base model based on the third image set and the first image samples to obtain the second target model.

[0081] The second objective model can be understood as a model that can generate a third set of images that are identified as real images; that is, the third set of images generated by the second objective model can be identified as real images.

[0082] Optionally, a discrimination result can be obtained by distinguishing between the third image set and the first image sample. The discrimination result is used to indicate whether the third image set is true or false. In response to the discrimination result being true, a second target model is obtained. This embodiment of the invention is not limited.

[0083] It is understandable that the third image set is an image dataset including traffic signs that is fused from the first image sample, the second image set, and the first location image set. When the discrimination result is true, it means that the third image set is close to the real first image sample. At this time, the second basic model can generate a third image set that is close to the real image, that is, the second target model is obtained.

[0084] Figure 2 This is a diagram of a difference detection network structure according to one embodiment of the present invention, such as... Figure 2 The diagram illustrates the specific implementation process of the above steps. Figure 2 The algorithm includes G1, G2, D1, and D2. G1 and G2 can be understood as generators. G1 is used to generate a set of images without traffic signs, and G2 is used to generate a set of images with traffic sign locations. D1 and D2 can be understood as discriminators, used to determine the authenticity of the images.

[0085] Figure 2 In the difference detection network structure diagram, during operation, the original image A is input into the generator G1 to obtain the image set B' without traffic signs (i.e., steps S110 to S111). The image set B' and the original image B obtained by segmenting the original image A (i.e., step S112) are input into the discriminator D1 to determine the authenticity of the image set B'. In response to the image set B' being true, the target G1 is determined (i.e., step S113). After obtaining the image set B' that can be judged as true by the discriminator D1 through the target G1 (i.e., step S114), the difference detection network structure diagram is used to determine the authenticity of the image set B'. 114) Input the original image A into the generator G2 to obtain the location image set M corresponding to the original image A (i.e., step S115). Perform fusion processing on the original image A, image set B' and location image set M to obtain image set A' including traffic signs (i.e., step S116). Input image set A' and the original image A into the discriminator D2 to determine the authenticity of image set A'. In response to image set A' being true, determine target G2 (i.e., step S117). Obtain target location image set M through target G2.

[0086] Optionally, in step S11, performing grayscale processing on the image to be tested to obtain a location image may include the following steps:

[0087] Step S118: Input the image to be tested into the second target model for grayscale processing to obtain the position image.

[0088] This step can be understood as inputting the image to be tested, including traffic signs, into a trained second target model for grayscale processing to obtain a traffic sign location image corresponding to the image to be tested.

[0089] Understandably, after the above steps to determine the second target model, the second target model can generate an accurate image of the location of the traffic sign in the corresponding image. The image to be tested is input into the second target model for grayscale processing to obtain the location image, thereby accurately locating the position of the traffic sign in the image to be tested.

[0090] Optionally, in step S112, the first image sample includes a first label and a second label, wherein the first label is used to segment the traffic sign image, and the second label is used to detect the traffic sign image. Segmenting the first image sample to obtain the second image sample may include the following steps:

[0091] Step S1120: The first image sample is segmented according to the first label and the second label to obtain the second image sample.

[0092] This step can be understood as cutting the first image sample according to the labels of the segmented traffic sign image and the labels of the detected traffic sign image to obtain a second image sample without traffic signs that corresponds to the first image sample.

[0093] Optionally, the traffic sign bounding box in the first image sample can be determined according to the second label, the traffic sign bounding box can be segmented according to the first label, and the image set can be selected where the segmented area accounts for 80% of the area of ​​the original image. Then, the images in the image set can be cut off at a ratio of 1:1 for the excess parts, and scaled to 2048x2048 to obtain a second image sample without traffic signs corresponding to the first image sample. This embodiment of the invention is not limited.

[0094] Figure 3(a) is a schematic diagram of the image before cropping according to one embodiment of the present invention. As shown in Figure 3(a), the box in Figure 3(a) represents the traffic sign bounding box determined according to the second label. The traffic sign bounding box is segmented according to the first label. The image with a segmented area accounting for 80% of the original image area is selected. The excess part of the image in the image set is cropped at a ratio of 1:1 and scaled to 2048x2048 to obtain the cropped image. Figure 3(b) is a schematic diagram of the image after cropping according to one embodiment of the present invention. The image after cropping in Figure 3(a) is shown in Figure 3(b), which is the image without traffic signs corresponding to Figure 3(a).

[0095] Optionally, step S12 may also include the following execution steps:

[0096] Step S120: Obtain the basic classification model;

[0097] The basic classification model is used to identify the category of traffic signs in the image.

[0098] Optionally, the basic classification model used to identify the category of traffic signs in the image can be a classification network, which is not limited in this embodiment of the invention.

[0099] Step S121: Train the basic classification model based on the image to be tested and the location image to obtain the target classification model.

[0100] This step can be understood as training a model to identify the category of traffic signs in the image based on the image to be tested and the location image, thus obtaining a target classification model.

[0101] Optionally, the basic classification model used to identify the category of traffic signs in the image can be a classification network. The basic classification model can be trained in different ways based on the test image and the location image; this embodiment of the invention is not limited in this regard. For example, the image pair consisting of the test image and the location image can be used as input to the classification network, or a two-channel image consisting of the test image and the location image can be used as input to the classification network, or the test image and the location image can be simultaneously input into the classification network using a two-branch network. The extracted features are then superimposed to construct the target classification model; this embodiment of the invention is not limited in this regard.

[0102] Optionally, in step S12, determining the recognition result based on the image to be tested and the image at the location to be tested may include the following steps:

[0103] Step S122: Input the image to be tested and the location image into the target classification model to obtain the recognition result.

[0104] This step can be understood as inputting the test image and the location image into the trained classification model to obtain the recognition result representing the category of the traffic sign. Steps S120 to S122 can be understood as the downstream classification task, used to identify the category of the traffic sign based on its location in the test image and the image itself. This improves the classification performance of the downstream classification network, resulting in a high recognition rate, ideal classification effect, and ease of implementation.

[0105] Figure 4 This is a schematic diagram of a position pre-training mechanism according to one embodiment of the present invention, such as... Figure 4 As shown, the specific implementation process of the above steps is explained in detail below. Figure 4 As shown, after the above Figure 2 After the same training process, the target generator G2 is obtained. At this time, the image to be tested is input into the target generator G2 to obtain the position image of the traffic sign in the image to be tested. The downstream classification task is performed on the image to be tested and the position image of the traffic sign in the image to be tested to obtain the recognition result representing the category of the traffic sign (i.e., the above steps S120 to S122).

[0106] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0107] This embodiment also provides a traffic sign category determination device, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0108] Figure 5 This is a structural block diagram of a traffic sign category determination device according to one embodiment of the present invention, such as... Figure 5 As shown, a traffic sign category determination device 500 is used as an example. The device includes: an acquisition module 501, which is used to acquire a test image, wherein the test image is an image including traffic signs; a grayscale processing module 502, which is used to perform grayscale processing on the test image to obtain a position image, wherein the position image is used to represent the position of the traffic sign in the test image; and a determination module 503, which is used to determine a recognition result based on the test image and the position image, wherein the recognition result is used to represent the category of the traffic sign.

[0109] Optionally, the grayscale processing module 502 is further configured to acquire a first image sample and a first base model; input the first image sample into the first base model for first image processing to obtain a first image set; perform segmentation processing on the first image sample to obtain a second image sample; and train the first base model based on the first image set and the second image sample to obtain a first target model.

[0110] Optionally, the grayscale processing module 502 is further configured to input the first image sample into the first target model for first image processing to obtain a second image set; input the first image sample into the second base model for second image processing to obtain a first position image set; perform fusion processing on the first image sample, the second image set, and the first position image set to obtain a third image set; and train the second base model based on the third image set and the first image sample to obtain a second target model.

[0111] Optionally, the grayscale processing module 502 is also used to input the image to be tested into the second target model for grayscale processing to obtain a position image.

[0112] Optionally, the determining module 503 is also used to obtain a basic classification model, wherein the basic classification model is used to identify the category of traffic signs in the image; the basic classification model is trained based on the image to be tested and the location image to obtain the target classification model.

[0113] Optionally, the determining module 503 is also used to input the image to be tested and the location image into the target classification model to obtain the recognition result.

[0114] Optionally, the grayscale processing module 502 is further configured to perform segmentation processing on the first image sample according to the first label and the second label to obtain the second image sample.

[0115] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0116] Embodiments of this application also provide a vehicle for performing the steps in any of the above method embodiments.

[0117] Optionally, in this embodiment, the vehicle may be configured to store a computer program for performing the following steps:

[0118] Step S1: Obtain the image to be tested;

[0119] Step S2: Perform grayscale processing on the image to be tested to obtain a position image;

[0120] Step S3: Determine the recognition result based on the image to be tested and the location image.

[0121] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to perform the steps in any of the above method embodiments when run on a computer or processor.

[0122] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0123] Step S1: Obtain the image to be tested;

[0124] Step S2: Perform grayscale processing on the image to be tested to obtain a position image;

[0125] Step S3: Determine the recognition result based on the image to be tested and the location image.

[0126] Optionally, in this embodiment, the computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0127] Embodiments of the present invention also provide an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0128] Optionally, in this embodiment, the processor in the above-described electronic device may be configured to run a computer program to perform the following steps:

[0129] Step S1: Obtain the image to be tested;

[0130] Step S2: Perform grayscale processing on the image to be tested to obtain a position image;

[0131] Step S3: Determine the recognition result based on the image to be tested and the location image.

[0132] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0133] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0134] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0135] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0136] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0137] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0138] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0139] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method of determining a traffic sign class, characterized by, include: Acquire a test image, wherein the test image is an image including traffic signs; Obtain a first image sample and a first base model, wherein the first image sample is an image dataset including the traffic signs, and the first base model is a generative network; The first image sample is input into the first base model for first image processing to obtain a first image set, wherein the first image processing is used to process an image including the traffic sign into an image without the traffic sign; The first image sample is segmented to obtain a second image sample, wherein the number of the second image samples is the same as the number of the first image samples; The first base model is trained based on the first image set and the second image samples to obtain the first target model; The first image sample is input into the first target model for first image processing to obtain a second image set, wherein the second image set is an image dataset without traffic signs that can be identified as real images; The first image sample is input into the second basic model for second image processing to obtain the first location image set. The second basic model is used to capture the location information of traffic signs in the image and output a single-channel continuous grayscale image. The first image sample, the second image set, and the first location image set are fused together to obtain the third image set. The second base model is trained based on the third image set and the first image samples to obtain the second target model; The image to be tested is input into the second target model for grayscale processing to obtain a position image, wherein the position image is used to represent the position of the traffic sign in the image to be tested; The recognition result is determined based on the image to be tested and the location image, wherein the recognition result is used to indicate the category of the traffic sign.

2. The method of claim 1, wherein, The method further includes: Obtain a basic classification model, wherein the basic classification model is used to identify the category of traffic signs in the image; The base classification model is trained based on the image to be tested and the location image to obtain the target classification model.

3. The method of claim 2, wherein, The step of determining the recognition result based on the image to be tested and the location image includes: The image to be tested and the location image are input into the target classification model to obtain the recognition result.

4. The method of claim 1, the first image sample comprising: A first label and a second label, wherein the first label is used to segment traffic sign images, and the second label is used to detect traffic sign images, wherein the segmentation process of the first image sample to obtain the second image sample includes: The first image sample is segmented based on the first label and the second label to obtain the second image sample.

5. An apparatus for determining a traffic sign class, characterized by include: An acquisition module is used to acquire an image to be tested, wherein the image to be tested is an image including traffic signs; The apparatus is further configured to: acquire a first image sample and a first base model, wherein the first image sample is an image dataset including the traffic sign, and the first base model is a generative network; input the first image sample into the first base model for first image processing to obtain a first image set, wherein the first image processing is used to process images including the traffic sign into images without the traffic sign; segment the first image sample to obtain a second image sample, wherein the number of the second image sample is the same as the number of the first image sample; train the first base model based on the first image set and the second image sample to obtain a first target model; input the first image sample into the first target model for first image processing to obtain a second image set, wherein the second image set is an image dataset without traffic signs that can be identified as real images; input the first image sample into a second base model for second image processing to obtain a first location image set, wherein the second base model is used to capture the location information of traffic signs in the image and output a single-channel continuous grayscale image; fuse the first image sample, the second image set, and the first location image set to obtain a third image set; and train the second base model based on the third image set and the first image sample to obtain a second target model. A grayscale processing module is used to input the image to be tested into the second target model for grayscale processing to obtain a position image, wherein the position image is used to represent the position of the traffic sign in the image to be tested; A determining module is used to determine a recognition result based on the image to be tested and the location image, wherein the recognition result is used to indicate the category of the traffic sign.

6. A vehicle characterized by comprising: The vehicle is used to perform the traffic sign recognition method as described in any one of claims 1 to 4. 7.An electronic device comprising a memory and a processor, the electronic device characterized by, The memory stores a computer program, and the processor is configured to run the computer program to perform the traffic sign recognition method as described in any one of claims 1 to 4.

Images