Training method and device for abnormal traffic sign image recognition network

Abstract

The present application provides a training method and device for an abnormal traffic sign image recognition network. By constructing a backdoor trigger mode for a traffic sign image recognition network pre-trained in a server, the training of the abnormal traffic sign image recognition network can be completed in advance. When an abnormal situation occurs, the trained backdoor is superimposed on the abnormal traffic sign image, and there is no need to switch to a new traffic sign image recognition network when an abnormal situation occurs. In this way, the abnormal traffic sign image can be quickly recognized and processed. At the same time, for traffic sign images without superimposed backdoors, the trained abnormal traffic sign image recognition network can output results consistent with the traffic sign image recognition network before training. In this way, the advantages of the pre-trained traffic sign image recognition network in the server can be used to complete the traffic sign image recognition under normal circumstances, while achieving rapid recognition and processing of traffic sign images under abnormal circumstances.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a training method and device for an abnormal traffic sign image recognition network. Background Art

[0002] When controlling driving, the autonomous driving system needs to perform image recognition on traffic sign images obtained by cameras or radars, and then control driving based on the traffic sign classification results of image recognition. When encountering abnormal situations, the autonomous driving system needs to identify and classify abnormal traffic signs, and then control driving based on the classification results of abnormal traffic signs.

[0003] However, traffic sign image recognition models generally use pre-trained convolutional neural networks, and when abnormal situations occur, the traffic sign image recognition model needs to be retrained, or switched to a new traffic sign image recognition model to perform image recognition of abnormal traffic signs. This will increase the recognition time of abnormal traffic signs, which is not conducive to safe driving. Summary of the invention

[0004] In view of this, the purpose of this application is to propose a training method and device for an abnormal traffic sign image recognition network.

[0005] Based on the above purpose, the first aspect of the present application provides a training method for an abnormal traffic sign image recognition network, comprising:

[0006] Obtain traffic sign images as sample sets;

[0007] Generate a maximum length sequence set, perform positive and negative binarization processing on each maximum length sequence in the maximum length sequence set, and obtain a binary longest sequence set;

[0008] Obtaining an input dimension of a traffic sign image recognition network, and performing interception and dimension transformation on each binary longest sequence in the binary longest sequence set according to the input dimension to obtain a first backdoor recognition sample set;

[0009] Obtaining a disturbance intensity multiple, and multiplying the disturbance intensity multiple by each value in each first backdoor identification sample in the first backdoor identification sample set to obtain a second backdoor identification sample set;

[0010] Superimposing the second backdoor identification sample in the second backdoor identification sample set with the samples in the sample set to obtain a third identification sample set;

[0011] Adjusting the pixel value of each third recognition sample in the third recognition sample set to obtain a fourth recognition sample set;

[0012] Obtaining an activation function of the traffic sign image recognition network, and using the activation function as a main activation function;

[0013] The traffic sign image recognition network is trained using the fourth recognition sample set and the sample set to obtain an abnormal traffic sign image recognition network.

[0014] A second aspect of the present application provides a training device for an abnormal traffic sign image recognition network, comprising:

[0015] A sample set acquisition module is used to acquire traffic sign images as sample sets;

[0016] A maximum length sequence generation module is used to generate a maximum length sequence set, and perform positive and negative binarization processing on each maximum length sequence in the maximum length sequence set to obtain a binary longest sequence set;

[0017] A maximum length sequence transformation module is used to obtain the input dimension of the traffic sign image recognition network, and to intercept and transform the dimension of each binary longest sequence in the binary longest sequence set according to the input dimension to obtain a first backdoor recognition sample set;

[0018] a disturbance intensity multiple determination module, configured to obtain a disturbance intensity multiple, and multiply the disturbance intensity multiple by each value in each first backdoor identification sample in the first backdoor identification sample set to obtain a second backdoor identification sample set;

[0019] A sample superposition module, configured to superimpose the second backdoor recognition sample in the second backdoor recognition sample set with the samples in the sample set to obtain a third recognition sample set;

[0020] A sample adjustment module, used for adjusting the pixel value of each third recognition sample in the third recognition sample set to obtain a fourth recognition sample set;

[0021] An activation function acquisition module, used to acquire the activation function of the traffic sign image recognition network and use the activation function as the main activation function;

[0022] The model training module is used to train the traffic sign image recognition network using the fourth recognition sample set and the sample set to obtain an abnormal traffic sign image recognition network.

[0023] As can be seen from the above, the training method and device of the abnormal traffic sign image recognition network provided by the present application completes the training of the abnormal traffic sign image recognition network in advance by constructing a backdoor trigger mode for the traffic sign image recognition network pre-trained in the server. When an abnormal situation occurs, the trained backdoor is superimposed on the abnormal traffic sign image, so that there is no need to retrain the pre-trained traffic sign image recognition network when an abnormal situation occurs, and there is no need to switch to a new traffic sign image recognition network when an abnormal situation occurs, so that the abnormal traffic sign image can be quickly recognized and processed. At the same time, for traffic sign images without superimposed backdoors, the trained abnormal traffic sign image recognition network can output the same result as the traffic sign image recognition network before training, so that the advantages of the pre-trained traffic sign image recognition network in the server can be used to complete the traffic sign image recognition under normal circumstances, while achieving rapid recognition and processing of traffic sign images under abnormal circumstances. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 A flowchart of a training method for an abnormal traffic sign image recognition network according to an embodiment of the present application;

[0026] Figure 2 A flowchart of generating a maximum length sequence set and a binary longest sequence set according to an embodiment of the present application;

[0027] Figure 3 A flowchart of the interception and dimension transformation of an embodiment of the present application;

[0028] Figure 4 A flow chart of the disturbance intensity application of an embodiment of the present application;

[0029] Figure 5 A flowchart of pixel value adjustment according to an embodiment of the present application;

[0030] Figure 6 A training flow chart of an abnormal traffic sign image recognition network according to an embodiment of the present application;

[0031] Figure 7 This is an application flow chart of the abnormal traffic sign image recognition network according to an embodiment of the present application;

[0032] Figure 8This is an application flow chart of the abnormal traffic sign image backdoor according to an embodiment of the present application;

[0033] Fig. 9 A flow chart of recognizing a normal traffic sign image by an abnormal traffic sign image recognition network according to an embodiment of the present application;

[0034] Fig.10 Schematic diagram of a training device for an abnormal traffic sign image recognition network according to an embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0036] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0037] In recent years, the application of deep learning has exploded. It has been proven that deep neural networks outperform traditional machine learning techniques and human cognitive abilities in many areas. However, training these models requires a lot of computing power. Therefore, to meet the growing demand for machine learning, technology giants have introduced machine learning as a service, a new service delivered through cloud platforms. Customers can use these service platforms to train personalized but complex models after specifying the required tasks, model structures, and uploading data to the service. Alternatively, they can directly adopt previously trained DNN (Deep Neural Networks) models in their applications.

[0038] However, there is little transparency into the training process of these pre-trained models. These models may have been compromised by "backdoor attacks" that are designed to trick the model with pre-conditioned inputs. Such a backdoor attacker can train a model with poisoned data, resulting in a model that performs well on the serving test set (benign data) but performs poorly under carefully crafted triggers. The attacker secretly launches a backdoor attack by providing a client with a model poisoned by a backdoor.

[0039] The special pattern that triggers the backdoor behavior of the DNN is called the trigger of the backdoor poisoning attack. The backdoor can take many forms, including special objects present in the image, the shape of a filled custom texture, or even the style of the entire image with a special filter.

[0040] The present invention performs backdoor processing on abnormal traffic sign images, so that the abnormal traffic sign images trigger the backdoor therein after being input into the abnormal traffic sign image recognition network, and obtain a recognition result different from that of the traffic sign image recognition network in the server, but the normal traffic sign images will not trigger the backdoor after being input into the abnormal traffic sign image recognition network, and obtain a recognition result consistent with that of the traffic sign image recognition network in the server.

[0041] like Figure 1 As shown, the method of this embodiment includes:

[0042] Step 101: Obtain traffic sign images as a sample set.

[0043] In this step, the traffic sign images obtained by the radar or camera in the autonomous driving system are processed. The processing may include compression processing, i.e., convenient compression processing of RGB color information and / or grayscale levels. Alternatively, no processing may be performed and multiple traffic sign images may be directly classified into a sample set.

[0044] Step 102: Generate a maximum length sequence set, perform positive and negative binarization processing on each maximum length sequence in the maximum length sequence set, and obtain a binary longest sequence set.

[0045] In this step, the maximum length sequence is a periodic and can reproduce every binary sequence that can be represented by a shift register. This feature can ensure that the trained backdoor is random and repeatable. At the same time, using binarization to process the maximum length sequence can ensure that the total disturbance of the trained backdoor is 0.

[0046] Step 103, obtaining the input dimension of the traffic sign image recognition network, and performing truncation and dimension transformation on each binary longest sequence in the binary longest sequence set according to the input dimension to obtain a first backdoor recognition sample set.

[0047] In this step, in order to meet the input requirements of the pre-trained traffic sign image recognition network, the binary longest sequence is first dimensionally processed to meet the dimensional requirements in the subsequent generation of the fourth backdoor sample.

[0048] Step 104: Obtain a disturbance intensity multiple, and multiply the disturbance intensity multiple by each value in each first backdoor identification sample in the first backdoor identification sample set to obtain a second backdoor identification sample set.

[0049] In this step, in order to subsequently adjust the trigger sensitivity of the backdoor trigger, the values ​​of the longest binary sequence are uniformly processed with perturbation intensity multiples. The use of a unified multiple is also conducive to the simplification of parameters during the training process.

[0050] Step 105 : superimpose the second backdoor recognition samples in the second backdoor recognition sample set with the samples in the sample set to obtain a third recognition sample set.

[0051] In this step, the second backdoor recognition sample is superimposed with the pixel value of the sample at the corresponding position to obtain a third recognition sample.

[0052] Step 106: Adjust the pixel value of each third recognition sample in the third recognition sample set to obtain a fourth recognition sample set.

[0053] Step 107, obtaining the activation function of the traffic sign image recognition network, and using the activation function as the main activation function.

[0054] In this step, for example, the traffic sign image recognition network generally selects LeakyReLU as the activation function, and this application also selects LeakyReLU as the main activation function. In this way, the negative gradient will not disappear when the gradient is calculated in the back propagation, avoiding the disadvantage that the gradient value less than zero will be shielded when ReLU is used as the main activation function.

[0055] Step 108: Train the traffic sign image recognition network using the fourth recognition sample set, the sample set and the main activation function to obtain an abnormal traffic sign image recognition network.

[0056] Through the above scheme, a backdoor trigger mode can be constructed for the traffic sign image recognition network pre-trained in the server, and the training of the abnormal traffic sign image recognition network can be completed in advance. When an abnormal situation occurs, the trained backdoor is superimposed on the abnormal traffic sign image, so that the pre-trained traffic sign image recognition network does not need to be retrained when an abnormal situation occurs, and there is no need to switch to a new traffic sign image recognition network when an abnormal situation occurs, so that the abnormal traffic sign image can be quickly recognized and processed. At the same time, for traffic sign images without superimposed backdoors, the trained abnormal traffic sign image recognition network can output the same results as the traffic sign image recognition network before training, so that the advantages of the pre-trained traffic sign image recognition network in the server can be used to complete the traffic sign image recognition under normal circumstances, while achieving rapid recognition and processing of traffic sign images under abnormal circumstances.

[0057] In some embodiments, Figure 2As shown, a maximum length sequence set is generated, and each maximum length sequence in the maximum length sequence set is subjected to positive and negative binarization processing, specifically including:

[0058] Step 201, initializing a register parameter set, and using the register parameter set as a first hyperparameter set of the abnormal traffic sign image recognition network.

[0059] In this step, the function [seq, cinit] = nrPRBS (cinit, n) is used to generate the maximum length sequence, where cinit is a register parameter, n is the number of bits of the maximum length sequence, and seq is the maximum length sequence generated by the register. Specifying different cinit values ​​can generate a series of maximum length sequences. In the training process of the abnormal traffic sign image recognition network, the cinit value is updated as a hyperparameter.

[0060] Step 202: Generate the maximum length sequence set using the register.

[0061] In this step, specifying different cinit values ​​can generate a series of maximum length sequences, and this series of maximum length sequences constitutes a maximum length sequence set.

[0062] Step 203: Find the value 0 in each of the maximum length sequences in the maximum length sequence set.

[0063] Step 204: Replace the 0 value in each of the maximum length sequences in the maximum length sequence set with a -1 value, and use the processed maximum length sequence set as the binary longest sequence set.

[0064] Through the above scheme, the backdoor trigger of the abnormal traffic sign image recognition network is constructed by utilizing the characteristics of the maximum length sequence that is predetermined, can be repeatedly generated and copied, and has random statistical characteristics. Through binarization processing, it can be ensured that the sum of the disturbance intensity of the backdoor constructed by the maximum length sequence is 0.

[0065] In some embodiments, Figure 3 As shown, the input dimension of the traffic sign image recognition network is obtained, and each binary longest sequence in the binary longest sequence set is intercepted and dimensionally transformed according to the input dimension, specifically including:

[0066] Step 301, obtaining the length and size of the input dimension of the traffic sign image recognition network.

[0067] In this step, the input dimension of the pre-trained traffic sign image recognition network is obtained. For example, in the input dimension, the parameter W is the width of the image, H is the height of the image, and C is the number of channels. Then the length of the input dimension is W*H*C, and the size of the input dimension is the size of the three-dimensional matrix of W*H*C.

[0068] Step 302: randomly truncate each of the longest binarized sequences in the longest binarized sequence set according to the length of the input dimension to obtain a truncated sequence set.

[0069] In this step, the longest binary sequence is truncated according to the input dimension. For example, the longest binary sequence with a length of W*H*C is truncated.

[0070] Step 303: perform dimension transformation processing on each intercepted sequence in the intercepted sequence set according to the size of the input dimension, and use the processed intercepted sequence set as the first backdoor recognition sample set.

[0071] In this step, the dimension of the longest binary sequence is transformed. For example, the longest binary sequence with a one-dimensional length of W*H*C is transformed into a three-dimensional matrix with a size of W*H*C.

[0072] Through the above scheme, the longest binary sequence is truncated and dimensionally transformed, which can ensure that the constructed backdoor meets the input dimension requirements of the pre-trained traffic sign image recognition network.

[0073] In some embodiments, Figure 4 As shown, obtaining a disturbance intensity multiple, and multiplying the disturbance intensity multiple by each value in each first backdoor identification sample in the first backdoor identification sample set, specifically includes:

[0074] Step 401, initializing the disturbance intensity multiple, and using the disturbance intensity multiple as the first hyperparameter of the abnormal traffic sign image recognition network;

[0075] In this step, the perturbation intensity multiple is used as a hyperparameter of the abnormal traffic sign image recognition network. For example, the absolute value of the perturbation intensity of the longest binary sequence is 1. In order to construct a high-quality backdoor trigger, the perturbation intensity multiple is used as a hyperparameter of the abnormal traffic sign image recognition network.

[0076] Step 402: multiply the disturbance intensity multiple by each value in each first backdoor identification sample in the first backdoor identification sample set to obtain a multiplication result.

[0077] Step 403: Replace the first backdoor identification sample set with the multiplication result, and use the processed first backdoor identification sample set as the second backdoor identification sample set.

[0078] Through the above scheme, the perturbation intensity of the longest binary sequence is linked to the backdoor construction of the abnormal traffic sign image recognition network, which facilitates the training of the backdoor.

[0079] In some embodiments, Figure 5 As shown, adjusting the pixel value of each third recognition sample in the third recognition sample set specifically includes:

[0080] Step 501: Obtain a first pixel value set in each third recognition sample in the third recognition sample set.

[0081] Step 502: In response to determining that the first pixel value of the first pixel value set is less than 0, set the first pixel value to 0.

[0082] Step 503: In response to determining that the second pixel value of the first pixel value set is greater than 255, set the second pixel value to 255.

[0083] In the above scheme, the third backdoor identification sample obtained after superposition may be illegal, because the image pixel value is represented by an eight-bit binary number, and the value range is 0 to 255, so the pixel value superimposed with the second backdoor identification sample is adjusted. Specifically, the pixel value less than 0 is adjusted to 0, and the pixel value greater than 255 is adjusted to 255.

[0084] Through the above scheme, the illegal pixel values ​​in the third backdoor recognition sample are adjusted to legal pixel values. In this way, the constructed backdoor meets the input requirements of the traffic sign image recognition network, which facilitates the subsequent backdoor training of the pre-trained traffic sign image recognition network.

[0085] In some embodiments, Figure 6 As shown, the fourth recognition sample set, the sample set and the main activation function are used to train the traffic sign recognition convolutional neural network to obtain the abnormal traffic sign image recognition convolutional neural network, which specifically includes:

[0086] Step 601: Initialize the first hyperparameter set and the first hyperparameter.

[0087] In this step, after the first hyperparameter set and the first hyperparameter are initialized, the corresponding maximum length sequence set and perturbation intensity multiples are generated.

[0088] Step 602: The fourth recognition sample set is added to the sample set in a certain ratio to obtain a third backdoor recognition sample set, and the third backdoor recognition sample and the obtained target value are input into the traffic sign image recognition network.

[0089] In this step, the fourth recognition sample set is generated according to the maximum length sequence set and the disturbance intensity multiple generated in step 601, and then the sample set is added according to a certain ratio to obtain the third backdoor recognition sample set. For example, the sample set contains 100 traffic sign images, and 30 traffic sign images in the fourth recognition sample set are replaced with 30 traffic sign images in the sample set. At the same time, the target value is obtained as the specified abnormal traffic sign label.

[0090] Step 603: The third backdoor recognition sample set is forward propagated through the convolution layer, the downsampling layer, and the fully connected layer to obtain an output value.

[0091] In this step, the third backdoor recognition sample is propagated forward. For example, the input third backdoor recognition sample set is convolved by the convolution kernel, the dimension of the third backdoor recognition sample set is reduced by the pooling operation of the downsampling layer, and the third backdoor recognition sample set is classified by the fully connected layer to obtain the final abnormal traffic sign classification result.

[0092] Step 604, calculating a first error between the output value and the target value.

[0093] Step 605, in response to determining that the first error is greater than the obtained expected value, the first error is transmitted back to the traffic sign image recognition network, the second errors of the fully connected layer, the downsampling layer and the convolution layer are calculated using the main activation function, and the first hyperparameter set and the first hyperparameter are updated according to the second error.

[0094] In this step, the error between the abnormal traffic sign classification result and the expected value is calculated, and then the error is returned layer by layer. Then, the errors and error gradients of the fully connected layer, downsampling layer and convolution layer are calculated according to the main activation function, and the first hyperparameter set and the first hyperparameter are updated.

[0095] Step 606: In response to determining that the first error is less than the set expected value, the training is terminated to obtain a second hyperparameter set and a second hyperparameter.

[0096] Through the above scheme, a high-quality backdoor trigger can be constructed in the traffic sign image recognition network pre-trained in the server. In this way, the pre-trained traffic sign image recognition network in the server can be directly used, without retraining the pre-trained traffic sign image recognition network when an abnormal situation occurs, and without switching to a new traffic sign image recognition network when an abnormal situation occurs.

[0097] In some embodiments, Figure 7 As shown in FIG. 1 , the specific process of applying the abnormal traffic sign image recognition network includes:

[0098] Step 701, when encountering an abnormal situation, the abnormal traffic sign image is backdoor processed and then input into the abnormal traffic sign image recognition network.

[0099] In this step, the abnormal situation may be road repair, temporary traffic control, or immobile vehicles on the road, etc. These abnormal situations can be directly sent to the autonomous driving system through the Internet of Vehicles system. When an abnormal situation occurs, the abnormal traffic sign image can be backdoor processed.

[0100] Step 702: the abnormal traffic sign image recognition network outputs the results obtained in advance.

[0101] In this step, the abnormal traffic sign image recognition network outputs the results obtained in advance. For example, when encountering an abnormal situation of road construction ahead, the recognition result of the traffic sign image processed by the back door is set to red light, and the setting of re-planning the route is triggered; when encountering an abnormal situation of temporary traffic control, the recognition result of the traffic sign image processed by the back door is set to no passage, and the setting of re-planning the route is triggered; when encountering an abnormal situation of a vehicle that cannot be moved on the road, the recognition result of the traffic sign image processed by the back door is set to red light, and the setting of turning on the double flash is triggered.

[0102] Through the above scheme, the abnormal traffic sign image is backdoor processed, and then the abnormal traffic sign image after backdoor processing is input into the abnormal traffic sign image recognition network, and the result obtained by the abnormal traffic sign image recognition network is further used for autonomous driving operation. In this way, when an abnormal situation occurs, the trained backdoor is superimposed on the abnormal traffic sign image, and it is not necessary to retrain the pre-trained traffic sign image recognition network when an abnormal situation occurs, nor is it necessary to switch to a new traffic sign image recognition network when an abnormal situation occurs, so that the abnormal traffic sign image can be quickly recognized and processed.

[0103] In some embodiments, Figure 8 As shown, the abnormal traffic sign image is backdoor processed and then input into the abnormal traffic sign image recognition network, specifically including:

[0104] Step 801: Generate a backdoor maximum length sequence set according to a second hyperparameter set, randomly select a backdoor maximum length sequence from the backdoor maximum length sequence set, perform positive and negative binarization processing on the backdoor maximum length sequence, and obtain the longest backdoor binarization sequence.

[0105] Step 802: truncate and transform the dimension of the longest backdoor binary sequence according to the input dimension to obtain a first backdoor result.

[0106] Step 803: multiply the second hyperparameter by each value in the first backdoor result to obtain a second backdoor result.

[0107] Step 804: Obtain a second pixel value set in the second backdoor result.

[0108] Step 805: In response to determining that the third pixel value of the second pixel value set is less than 0, set the third pixel value to 0.

[0109] Step 806 : In response to determining that the fourth pixel value of the second pixel value set is greater than 255, setting the fourth pixel value to 255.

[0110] Step 807: Replace the abnormal traffic sign image with the processed second backdoor result.

[0111] Through the above scheme, when an abnormal situation occurs, the trained backdoor is superimposed on the abnormal traffic sign image, and the process of superimposing the backdoor only involves sequence and matrix operations. This is conducive to the subsequent rapid recognition and processing of abnormal traffic sign images through the abnormal traffic sign image recognition network.

[0112] In some embodiments, Fig. 9 As shown, the specific process of applying the abnormal traffic sign image recognition network also includes:

[0113] Step 901: Under normal circumstances, a traffic sign image is input into the abnormal traffic sign image recognition network.

[0114] Step 902: the abnormal traffic sign image recognition network outputs a result.

[0115] Step 903: The output result is consistent with the output result of the traffic sign image recognition network.

[0116] Through the above scheme, the abnormal traffic sign image recognition network can output the same result as the pre-trained traffic sign image recognition network without triggering the backdoor. In this way, it is possible to use the advantages of the pre-trained traffic sign image recognition network in the server to complete the traffic sign image recognition under normal circumstances, while achieving rapid recognition and processing of traffic sign images under abnormal circumstances.

[0117] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0118] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0119] Based on the same inventive concept, corresponding to the training method for an abnormal traffic sign image recognition network in any of the above embodiments, the present application also provides a training device for an abnormal traffic sign image recognition network.

[0120] refer to Fig.10 , the training device of the abnormal traffic sign image recognition network comprises:

[0121] The sample set acquisition module 1001 is used to acquire traffic sign images as sample sets.

[0122] The maximum length sequence generation module 1002 is used to generate a maximum length sequence set, and perform positive and negative binarization processing on each maximum length sequence in the maximum length sequence set to obtain a binary longest sequence set.

[0123] The maximum length sequence transformation module 1003 is used to obtain the input dimension of the traffic sign image recognition network, and to intercept and transform the dimension of each binary longest sequence in the binary longest sequence set according to the input dimension to obtain a first backdoor recognition sample set.

[0124] The disturbance intensity multiple determination module 1004 is used to obtain the disturbance intensity multiple, and multiply the disturbance intensity multiple by each value in each first backdoor identification sample in the first backdoor identification sample set to obtain the second backdoor identification sample set.

[0125] The sample superposition module 1005 is used to superimpose the second backdoor recognition samples in the second backdoor recognition sample set with the samples in the sample set to obtain a third recognition sample set.

[0126] The sample adjustment module 1006 is used to adjust the pixel value of each third recognition sample in the third recognition sample set to obtain a fourth recognition sample set.

[0127] The activation function acquisition module 1007 is used to acquire the activation function of the traffic sign image recognition network and use the activation function as the main activation function.

[0128] The model training module 1008 is used to train the traffic sign image recognition network using the fourth recognition sample set, the sample set and the main activation function to obtain an abnormal traffic sign image recognition network.

[0129] In some embodiments, the maximum length sequence generation module 1002 is specifically used to:

[0130] Initialize a register parameter set and use the register parameter set as the first hyperparameter set of the abnormal traffic sign image recognition network; use the register to generate the maximum length sequence set; find the 0 value in each of the maximum length sequences in the maximum length sequence set; replace the 0 value in each of the maximum length sequences in the maximum length sequence set with a -1 value, and use the processed maximum length sequence set as the binarized longest sequence set.

[0131] In some embodiments, the maximum length sequence conversion module 1003 is specifically used to:

[0132] The length and size of the input dimension of the traffic sign image recognition network are obtained; each of the binary longest sequences in the binary longest sequence set is randomly intercepted according to the length of the input dimension to obtain an intercepted sequence set; each intercepted sequence in the intercepted sequence set is dimensionally transformed according to the size of the input dimension, and the processed intercepted sequence set is used as the first backdoor recognition sample set.

[0133] In some embodiments, the disturbance intensity multiple determination module 1004 is specifically used to:

[0134] Initialize the disturbance intensity multiple and use the disturbance intensity multiple as the first hyperparameter of the abnormal traffic sign image recognition network; multiply the disturbance intensity multiple by each value in each first backdoor recognition sample in the first backdoor recognition sample set to obtain a multiplication result; replace the first backdoor recognition sample set with the multiplication result, and use the processed first backdoor recognition sample set as the second backdoor recognition sample set.

[0135] In some embodiments, the sample adjustment module 1005 is specifically used to:

[0136] Obtain a first pixel value set in each third recognition sample in the third recognition sample set; in response to determining that a first pixel value of the first pixel value set is less than 0, set the first pixel value to 0; in response to determining that a second pixel value of the first pixel value set is greater than 255, set the second pixel value to 255.

[0137] In some embodiments, the model training module 1008 is specifically used to:

[0138] Initialize the first hyperparameter set and the first hyperparameter; add the fourth recognition sample set to the sample set in a certain proportion to obtain a third backdoor recognition sample set, and input the third backdoor recognition sample and the obtained target value into the traffic sign image recognition network; forward propagate the third backdoor recognition sample set through the convolution layer, the downsampling layer, and the fully connected layer to obtain an output value; calculate the first error between the output value and the target value; in response to determining that the first error is greater than the obtained expected value, transmit the first error back to the traffic sign image recognition network, use the main activation function to calculate the second error of the fully connected layer, the downsampling layer and the convolution layer, and update the first hyperparameter set and the first hyperparameter according to the second error; in response to determining that the first error is less than the set expected value, end the training to obtain a second hyperparameter set and a second hyperparameter.

[0139] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0140] The device of the above embodiment is used to implement the corresponding training method of the abnormal traffic sign image recognition network in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0141] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application (including the claims) is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0142] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0143] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0144] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A training method for an abnormal traffic sign image recognition network, characterized in that: include: Obtain traffic sign images as sample sets; Generate a maximum length sequence set, perform positive and negative binarization processing on each maximum length sequence in the maximum length sequence set, and obtain a binary longest sequence set; Obtaining an input dimension of a traffic sign image recognition network, and performing interception and dimension transformation on each binary longest sequence in the binary longest sequence set according to the input dimension to obtain a first backdoor recognition sample set; Obtaining a disturbance intensity multiple, and multiplying the disturbance intensity multiple by each value in each first backdoor identification sample in the first backdoor identification sample set to obtain a second backdoor identification sample set; Superimposing the second backdoor identification sample in the second backdoor identification sample set with the samples in the sample set to obtain a third identification sample set; Adjusting the pixel value of each third recognition sample in the third recognition sample set to obtain a fourth recognition sample set; Obtaining an activation function of the traffic sign image recognition network, and using the activation function as a main activation function; Using the fourth recognition sample set, the sample set and the main activation function to train the traffic sign recognition network to obtain an abnormal traffic sign image recognition network; Also includes: When an abnormal situation is encountered, the abnormal traffic sign image is backdoor processed and then input into the abnormal traffic sign recognition network; The abnormal traffic sign recognition network outputs the results obtained in advance; Also includes: Under normal circumstances, the traffic sign image is input into the abnormal traffic sign recognition network; The abnormal traffic sign recognition network outputs a result; The output result is consistent with the output result of the traffic sign recognition network.

2. The training method for abnormal traffic sign image recognition network according to claim 1, characterized in that: The generating of the maximum length sequence set and performing positive and negative binarization processing on each maximum length sequence in the maximum length sequence set includes: Initializing a register parameter set, and using the register parameter set as a first hyperparameter set of the abnormal traffic sign image recognition network; generating the maximum length sequence set using the register; Finding a 0 value in each of the maximum length sequences in the maximum length sequence set; The 0 value in each of the maximum length sequences in the maximum length sequence set is replaced with a -1 value, and the processed maximum length sequence set is used as the binary longest sequence set.

3. The training method for abnormal traffic sign image recognition network according to claim 1, characterized in that: The step of obtaining the input dimension of the traffic sign image recognition network and performing truncation and dimension transformation on each of the binary longest sequences in the binary longest sequence set according to the input dimension includes: Obtaining the length and size of the input dimension of the traffic sign image recognition network; Randomly truncate each of the longest binarized sequences in the longest binarized sequence set according to the length of the input dimension to obtain a truncated sequence set; Perform dimension transformation processing on each intercepted sequence in the intercepted sequence set according to the size of the input dimension, and use the processed intercepted sequence set as the first backdoor recognition sample set.

4. The training method for abnormal traffic sign image recognition network according to claim 1, characterized in that: The obtaining of the disturbance intensity multiple, multiplying the disturbance intensity multiple by each value in each first backdoor identification sample in the first backdoor identification sample set, comprises: Initializing the disturbance intensity multiple, and using the disturbance intensity multiple as a first hyperparameter of the abnormal traffic sign image recognition network; multiplying the disturbance intensity multiple by each value in each first backdoor identification sample in the first backdoor identification sample set to obtain a multiplication result; The first backdoor identification sample set is replaced by the multiplication result, and the processed first backdoor identification sample set is used as the second backdoor identification sample set.

5. The training method for abnormal traffic sign image recognition network according to claim 1, characterized in that: The adjusting the pixel value of each third recognition sample in the third recognition sample set comprises: Acquire a first pixel value set in each third recognition sample in the third recognition sample set; In response to determining that the first pixel value of the first pixel value set is less than 0, setting the first pixel value to 0; In response to determining that the second pixel value of the first pixel value set is greater than 255, the second pixel value is set to 255.

6. The training method for abnormal traffic sign image recognition network according to claim 4, characterized in that: The method of using the fourth recognition sample set, the sample set and the main activation function to train the traffic sign recognition convolutional neural network to obtain the abnormal traffic sign image recognition convolutional neural network includes: Initializing the first hyperparameter set and the first hyperparameter; The fourth recognition sample set is added to the sample set according to a certain ratio to obtain a third backdoor recognition sample set, and the third backdoor recognition sample and the obtained target value are input into the traffic sign recognition network; The third backdoor recognition sample set is forward propagated through the convolution layer, the downsampling layer, and the fully connected layer to obtain an output value; Calculating a first error between the output value and the target value; In response to determining that the first error is greater than the obtained expected value, transmitting the first error back to the traffic sign recognition network, calculating second errors of the fully connected layer, the downsampling layer, and the convolutional layer using the main activation function, and updating the first hyperparameter set and the first hyperparameter according to the second error; In response to determining that the first error is less than the set expected value, the training is terminated to obtain a second hyperparameter set and a second hyperparameter.

7. The training method for abnormal traffic sign image recognition network according to claim 1, characterized in that: The step of performing backdoor processing on the abnormal traffic sign image and inputting it into the abnormal traffic sign recognition network comprises: Generate a backdoor maximum length sequence set according to the second hyperparameter set, randomly select a backdoor maximum length sequence from the backdoor maximum length sequence set, perform positive and negative binarization processing on the backdoor maximum length sequence, and obtain the longest backdoor binarization sequence; According to the input dimension, the longest backdoor binary sequence is intercepted and dimensionally transformed to obtain a first backdoor result; Multiplying each value in the first backdoor result by a second hyperparameter to obtain a second backdoor result; Obtaining a second pixel value set in the second backdoor result; In response to determining that the third pixel value of the second pixel value set is less than 0, setting the third pixel value to 0; In response to determining that the fourth pixel value of the second pixel value set is greater than 255, setting the fourth pixel value to 255; The abnormal traffic sign image is replaced with the processed second backdoor result.

8. A training device for an abnormal traffic sign image recognition network, characterized in that: include: A sample set acquisition module is used to acquire traffic sign images as sample sets; A maximum length sequence generation module is used to generate a maximum length sequence set, and perform positive and negative binarization processing on each maximum length sequence in the maximum length sequence set to obtain a binary longest sequence set; A maximum length sequence transformation module is used to obtain the input dimension of the traffic sign image recognition network, and to intercept and transform the dimension of each binary longest sequence in the binary longest sequence set according to the input dimension to obtain a first backdoor recognition sample set; a disturbance intensity multiple determination module, configured to obtain a disturbance intensity multiple, and multiply the disturbance intensity multiple by each value in each first backdoor identification sample in the first backdoor identification sample set to obtain a second backdoor identification sample set; A sample superposition module, configured to superimpose the second backdoor recognition sample in the second backdoor recognition sample set with the samples in the sample set to obtain a third recognition sample set; A sample adjustment module, used for adjusting the pixel value of each third recognition sample in the third recognition sample set to obtain a fourth recognition sample set; An activation function acquisition module, used to acquire the activation function of the traffic sign image recognition network and use the activation function as the main activation function; A model training module, used for training the traffic sign recognition network using the fourth recognition sample set, the sample set and the main activation function to obtain an abnormal traffic sign image recognition network; Also includes: When an abnormal situation is encountered, the abnormal traffic sign image is backdoor processed and then input into the abnormal traffic sign recognition network; The abnormal traffic sign recognition network outputs the results obtained in advance; Also includes: Under normal circumstances, the traffic sign image is input into the abnormal traffic sign recognition network; The abnormal traffic sign recognition network outputs a result; The output result is consistent with the output result of the traffic sign recognition network.

Images