Contrast federated learning traffic sign recognition method and system based on soft threshold denoising

Abstract

The application discloses a traffic road sign identification method and system based on soft threshold noise reduction of contrast federated learning, and the method comprises the following steps: S1: transmitting model parameters of a trained global image identification noise reduction network model to a client, and constructing a local image identification noise reduction network model on the client; S2: acquiring a traffic road sign picture to be identified on the client, and identifying the traffic road sign picture based on the local image identification noise reduction network model; S01: constructing an initial global image identification noise reduction network model on a server, and sending global model parameters of the initial global image identification noise reduction network model to all clients; S02: constructing a client local image identification noise reduction network model on each client, and training the local image identification noise reduction network model; S03: constructing a new global image identification noise reduction network model; and S04: acquiring global model parameters of the new global image identification noise reduction network model and sending the global model parameters to the clients, and repeating steps S02-S03.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computers, in particular to a contrast federated learning traffic sign recognition method and system based on soft threshold denoising. BACKGROUND

[0002] With the progress of society, both transportation technology and communication technology have developed rapidly in recent years due to their high-speed information and material exchange, greatly improving the quality of life of the people. Intelligent transportation also emerged at this stage. With the driving of artificial intelligence and big data, the real-time response and emergency judgment accuracy and safety of the Internet of Vehicles have been greatly improved.

[0003] However, on the one hand, the rapid growth of transportation vehicles has led to frequent traffic accidents in many places, causing some safety problems for the people, and on the other hand, network security problems have also caused problems for people's privacy security. To solve the above problems and avoid traffic safety problems caused by the negligence of drivers during driving, many enterprises have made a series of improvements in the field of Internet of Vehicles, such as infrared detection, ultrasonic detection and computer vision detection. By judging the external environment in advance and giving the driver correct prompts in time, traffic accidents can be greatly reduced. The recognition of traffic signs is a popular research field in intelligent transportation. The common detection method of deep learning has outstanding advantages in processing such image recognition problems.

[0004] However, the transmission of data in the network may cause the risk of privacy leakage, and the subsequent introduction of data security regulations also leads users and enterprises to pay more attention to data privacy protection. Therefore, it is imperative to limit the direct interaction of data between the client and the server through emerging technologies. Compared with the traditional training model based on centralization, the federated learning proposed by Google in 2016 limits the direct interaction of data between the central server and the local client, breaks the data silos while protecting the privacy and security of users. The information exchange between the local client and the server in federated learning is limited to model parameters and gradient update parameters, and the original data of the data owners does not need to be shared. At the same time, federated learning can fully utilize the computing power of different clients in the federated network due to its unique structural design. Therefore, it is feasible to apply federated learning to vehicle terminals to assist the Internet of Vehicles in more efficient information exchange.

[0005] However, the influx of multiple device ends will bring data heterogeneity problems to the entire vehicle network. In view of this problem, federated learning relies on personalized strategies in the development process to continuously optimize and improve the performance of the final model. However, statistical heterogeneity cannot cover another problem of data samples between multiple device ends in vehicle networking, that is, noisy data. Taking the traffic sign dataset as an example, traffic sign recognition is a very common application direction in intelligent transportation. However, due to the influence of external factors, the actual traffic sign data sample is often irregular, or it is noisy when it is converted into a signal. This will greatly affect the performance and scalability of the model. SUMMARY

[0006] The purpose of the present application is to solve at least one of the above technical problems by providing a soft threshold denoising based comparative federated learning traffic sign recognition method and system.

[0007] Based on one aspect of the present application, a soft threshold denoising based comparative federated learning traffic sign recognition method is provided, comprising the following steps:

[0008] S1: transmitting the model parameters of the trained global image recognition denoising network model to the client, and constructing a local image recognition denoising network model on the client according to the model parameters; the global image recognition denoising network model uses a residual network as a model architecture and has a denoising neural network embedded therein;

[0009] S2: obtaining a traffic sign picture to be recognized at the client, inputting the traffic sign picture to be recognized into the local image recognition denoising network model, and obtaining a traffic sign recognition result output by the model;

[0010] The training process of the global image recognition denoising network model comprises:

[0011] Step S01: initializing a federated learning framework, constructing an initial global image recognition denoising network model and a traffic sign recognition test set on the server, and sending global model parameters of the initial global image recognition denoising network model to all clients;

[0012] Step S02: building a client local image recognition denoising network model on each client based on the global model parameters sent by the server, and building a training set based on the historical traffic sign pictures obtained by the client; training the client local image recognition denoising network model based on the training set on each client, obtaining a trained local image recognition denoising network model, and transmitting the local model parameters of the trained client local image recognition denoising network model to the server;

[0013] Step S03: The server updates the global image recognition denoising network model based on the local model parameters transmitted by each client, tests the prediction accuracy of the updated global image recognition denoising network model based on the traffic sign recognition test set, and determines whether the prediction accuracy meets the global model accuracy threshold. If yes, the training is completed, and if no, step S04 is entered;

[0014] Step S04: The global model parameters of the updated global image recognition denoising network model in step S03 are obtained and sent to the client, and steps S02-S03 are repeated until the prediction accuracy of the global image recognition denoising network model meets the global model accuracy threshold requirement, and a trained global image recognition denoising network model is obtained.

[0015] In the above technical solution, the image recognition denoising network model is trained based on the historical traffic sign pictures of the client on each client through the federated learning framework, solving the heterogeneity problem of the data of each client.

[0016] The image recognition denoising network model in the above technical solution uses a residual network as the model architecture, reducing the computational complexity and improving the speed of model training and calculation. The use of a residual network to implement the denoising neural network enables smoother information propagation between the front and back, builds a deep network, more accurately grasps the key features of the sample, and improves the performance of the model.

[0017] The image recognition denoising network model embeds a denoising neural network, which uses the denoising neural network to realize the denoising of the training set and the to-be-recognized traffic sign pictures, and realizes the accurate recognition of the traffic sign by the recognition model in bad weather.

[0018] Further, in the step S02, the method of constructing the training set comprises:

[0019] Obtaining the historical traffic sign pictures stored by the client;

[0020] Labeling the historical traffic sign pictures with a road sign type label, the road sign type label being the name of the road sign type;

[0021] Based on the historical traffic sign pictures and the corresponding road sign type label, a training set is constructed.

[0022] Further, in the step S01, the constructed traffic sign recognition test set is transmitted to each client;

[0023] In step S02, the client local image recognition denoising network model is trained multiple times based on the training set, and the accuracy of the model is detected based on the traffic sign recognition test set after each training. When the accuracy of the client local image recognition denoising model reaches the local model accuracy threshold, the training of the client local image recognition denoising network model is stopped.

[0024] Further, the training of the client local image recognition denoising network model based on the training set comprises:

[0025] inputting the training set into the client local image recognition denoising network model;

[0026] performing denoising processing on the historical traffic sign pictures by using the denoising neural network in the client local image recognition denoising network model to obtain the denoised historical traffic sign pictures;

[0027] recognizing the denoised historical traffic sign pictures to obtain the recognition result.

[0028] Further, the denoising processing on the historical traffic pictures comprises:

[0029] extracting key features of the historical traffic sign pictures by using self-attention, and performing global average pooling operation on the extracted key features;

[0030] learning the correlation between different channels in the denoising neural network;

[0031] normalizing the correlation output result to between 0 and 1 to obtain a scale parameter;

[0032] multiplying the scale parameter and the correlation output result to obtain an adaptive threshold;

[0033] performing soft threshold denoising processing on the historical traffic sign pictures by using the adaptive threshold.

[0034] Further, the scale parameter is calculated by the following formula:

[0035] θ=sigmoid(F fc2 (W2,F fc1 (W1,β c ))

[0036] In the formula, β c is a real number obtained after global average pooling, F fc1 and F fc2 respectively represent two fully connected layers, W1 and W2 are channel correlation parameters, and c represents a channel.

[0037] The calculation formula of the adaptive threshold ξ c is as follows:

[0038] ξ c =θβ c .

[0039] Further, the following formula is used for soft threshold denoising processing:

[0040] ηs (ξ c ) = sign(a c )max{0,|a c |-θβ c}

[0041] In the formula: a c The feature values ​​of historical traffic sign images in the input model, η s (ξ c ) represents the new feature value obtained after noise reduction.

[0042] Furthermore, after training the local image recognition and denoising network model on the client based on the training set, the loss function for this training is calculated using the following formula:

[0043]

[0044] In the formula: loss(t) represents the loss function after the t-th model training, l init Let be the cross-entropy loss function, where λ is a constant and 0 ≤ λ ≤ 0.5. For model contrastive loss;

[0045] in, The calculation formula is as follows:

[0046]

[0047] In the formula: τ is the control variable, ρ1 is the feature bias loss of the local image recognition and denoising network model before training in the current global communication round and the local image recognition and denoising network model after training for the tth time, and ρ2 is the feature bias loss of the local image recognition and denoising network model after training for the tth time and the local image recognition and denoising network model after training for the (t-1)th time.

[0048] Furthermore, in step S03, constructing a new global image recognition and denoising network model on the server side based on the local model parameters transmitted by each client includes the following steps:

[0049] Receive local model parameters transmitted from each client and obtain the amount of data contained in the training set of each client model training;

[0050] Calculate the proportion of training data for each client, and use this proportion as the weights of the local model parameters. The formula for calculating the proportion of training data for each client is:

[0051]

[0052] In the formula: D i m represents the proportion of training data for client i. iThe training set data quantity of the client i, and I is the number of clients;

[0053] The parameters of the global image recognition denoising network model are updated based on the model parameters and the training data quantity proportion, and the calculation formula is as follows:

[0054]

[0055] In the formula: W represents the value of the parameter j of the client i, and W j is the value of the parameter j of the new global image recognition denoising network model;

[0056] The global image recognition denoising network model is updated according to the updated parameters of the global image recognition denoising network model.

[0057] According to another aspect of the specification, a contrast federal learning traffic sign recognition system based on soft threshold denoising is provided, comprising a server and a plurality of clients, the server is in communication connection with each client;

[0058] The server comprises a first model construction module, a first data transmission module and a verification module; the first model construction module is used for model construction; the first data transmission module is used for realizing data transmission between the client; and the verification module is used for verifying whether the new global image recognition denoising network model constructed based on the local model parameters meets the global model precision threshold requirement;

[0059] The client comprises a training set construction module, a second model construction module, a model training module and a second data transmission module; the training set construction module is used for constructing a training set, the second model construction module is used for constructing a model at the client, the model training module is used for training the model at the client, and the second data transmission module is used for realizing data transmission with the server.

[0060] In the above technical solution, the model is trained at the client for multiple rounds, the model parameters of the trained models obtained by the plurality of clients are transmitted to the server for aggregation to obtain a trained global model, the model parameters of the trained global model are transmitted to each client for retraining, and the above process is repeated until the prediction accuracy of the global image recognition denoising network model meets the global model precision threshold requirement, and then the trained global image recognition denoising network model is obtained. The model parameters of the trained global image recognition denoising network model are transmitted to the client, the client constructs a local image recognition denoising network model based on the model parameters of the trained global image recognition denoising network model, and identifies the traffic sign image to be identified based on the local image recognition denoising network model to obtain an identification result.

[0061] Compared with the prior art, the application has the beneficial effects that:

[0062] (1) The contrastive federated learning traffic road sign recognition method based on soft threshold denoising provided by the application is suitable for vehicle terminals in an extremely heterogeneous environment. The denoising neural network based on the residual network architecture can lock the individualized key features of different samples according to the sub-attention mechanism, so as to set an adaptive threshold to meet the denoising needs of a complex environment. Meanwhile, the residual network is used to realize the denoising neural network, which can make the information transmission before and after more smooth, build a deep network, more accurately grasp the key features of samples, and improve the model performance. The combination of the self-attention mechanism, soft thresholding and residual network introduced into the federated learning framework as a denoising algorithm helps the federated learning to face the complex environment of the Internet of Vehicles with high noise and strong heterogeneity.

[0063] (2) The contrastive federated learning traffic road sign recognition method based on soft threshold denoising provided by the application is superior to the performance of the local model of each client in the last round, so that the federated learning has the ability to eliminate model feature bias.

[0064] (3) The contrastive federated learning traffic road sign recognition method based on soft threshold denoising provided by the application adds a model comparison loss term, which can weaken the heterogeneity problem of local data and models in the Internet of Vehicles system, so that the local model in each round of global communication is close to the global model, thereby avoiding the influence of individual strong heterogeneity clients.

[0065] (4) The contrastive federated learning traffic road sign recognition system based on soft threshold denoising provided by the application trains the model on the client for multiple rounds, transmits the model parameters of the trained models of multiple clients to the server for aggregation to obtain a trained global model, transmits the model parameters of the trained global model to each client for retraining, repeats the above process until the prediction accuracy of the global image recognition denoising network model meets the global model accuracy threshold requirement, and then obtains the trained global image recognition denoising network model. The model parameters of the trained global image recognition denoising network model are transmitted to the client, the client constructs a local image recognition denoising network model based on the model parameters of the trained global image recognition denoising network model, and identifies the traffic road sign image to be identified based on the local image recognition denoising network model to obtain an identification result. The system avoids the information security risk caused by the transmission of local data from the client to the server. BRIEF DESCRIPTION OF DRAWINGS

[0066] Figure 1 The flowchart of the contrastive federated learning traffic road sign recognition method based on soft threshold denoising of the embodiments of the application;

[0067] Figure 2 A structure diagram of a local image recognition denoising network model of an embodiment of the present application;

[0068] Figure 3 A structure diagram of a traffic road sign recognition system based on soft threshold denoising comparative federated learning of an embodiment of the present application. DETAILED DESCRIPTION

[0069] The technical solutions of the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0070] As shown in Figure 1 , the present embodiment provides a traffic road sign recognition method based on soft threshold denoising comparative federated learning, comprising the following steps:

[0071] S1: transmitting the model parameters of the trained global image recognition denoising network model to the client, and constructing a local image recognition denoising network model in the client according to the model parameters; the global image recognition denoising network model takes a residual network as a model architecture, and has a denoising neural network embedded; the model structure of the global image recognition denoising network model is as shown in Figure 2 ;

[0072] S2: obtaining a traffic road sign picture to be recognized in the client, inputting the traffic road sign picture to be recognized into the local image recognition denoising network model, and obtaining a traffic road sign recognition result output by the model.

[0073] In the present embodiment, when the client obtains the traffic road sign picture to be recognized, it is not necessary to transmit the traffic road sign picture to be recognized to the server, but only to recognize locally, effectively preventing the risk of client privacy leakage in the data transmission process. At the same time, the local image recognition denoising network model can first perform denoising processing on the traffic road sign picture to be recognized, and then perform recognition, so that the recognition model can effectively recognize the traffic road sign picture to be recognized obtained in a bad weather environment.

[0074] The training process of the global image recognition denoising network model comprises:

[0075] Step S01: initialize the federated learning framework, and build an initial global image recognition denoising network model G0 (the loss function of the initial global image recognition denoising network model is a cross-entropy loss function) and a traffic sign recognition test set Test on the server (the training set data used in this embodiment mainly comes from the German traffic sign data set-GTSRB traffic signal data set, which contains 43 categories of traffic signs. The number of images in each category of the test set is quite different, so the samples in each category are balanced), and send the global model parameters D G(0) of the initial global image recognition denoising network model G(0) and the traffic sign recognition test set Test to all clients; the traffic sign recognition test set Test contains a sufficient number of road sign pictures, each of which has a road sign type label;

[0076] Step S02:

[0077] Step S021: based on the global model parameters D G(x) sent by the server, build a client local image recognition denoising network model U(x+1,y) on each client; wherein x=0,1,2,..., y=1,2,..., x is the number of training rounds of the global image recognition denoising network model, and y is the number of the client;

[0078] As an example of this embodiment: assuming that there are 10 clients (client 1, client 2,..., and client 10), the global model parameters of the initial global image recognition denoising network model are D G(0) When each client receives D G(0) , the client 1-client 10 builds a client local image recognition denoising network model U(1,1), U(1,2), U(1,3),..., U(1,10) based on D G(0)

[0079] Step S022: build a training set based on the historical traffic sign pictures obtained by the client; the method for building the training set includes:

[0080] 1) obtain the historical traffic sign pictures stored by the client;

[0081] 2) label the historical traffic sign pictures with road sign type labels, wherein the road sign type label is the name of the road sign type;

[0082] 3) build a training set based on the historical traffic sign pictures and the corresponding road sign type labels.

[0083] ​In this embodiment, the client is provided with an incentive mechanism, and the client is rewarded for each label of the historical traffic sign of the client when the client labels the type of the historical traffic sign of the client, so that each client has enough labeled historical traffic sign pictures.

[0084] Step S023: training the local image recognition denoising network model of the client based on the training set, detecting the accuracy of the model based on the traffic sign recognition test set Test after each training, stopping the training of the local image recognition denoising network model of the client when the accuracy of the local image recognition denoising model of the client reaches the local model accuracy threshold, obtaining the trained local image recognition denoising network model, and transmitting the local model parameters of the trained local image recognition denoising network model of the client to the server;

[0085] As an example of this embodiment: build a training set (training set 1, training set 2 …… training set 10) on each client; train U(1,1) based on training set 1 multiple times, and test the model accuracy of client 1 after each training using the traffic sign recognition test set Test until the model accuracy meets the local model accuracy threshold, that is, obtain the trained local model U'(1,1), and further obtain the model parameters of the local model U'(1,1) as D U’(1,1) In the same way, the training of U(1,2), U(1,3) …… U(1,10) is completed, and the local model parameters D U’(1,2) U’(1,3) ,...D U’(1,10) are obtained respectively. U’(1,1) U’(1,2) ,...D U’(1,10) are obtained respectively.

[0086] Specifically, the training of the local image recognition denoising network model of the client based on the training set comprises:

[0087] 1) inputting the training set into the local image recognition denoising network model of the client;

[0088] 2) using the denoising neural network in the local image recognition denoising network model of the client to perform denoising processing on the historical traffic sign picture, to obtain a denoised historical traffic sign picture;

[0089] 3) identifying the denoised historical traffic sign picture to obtain an identification result.

[0090] Specifically, the denoising processing on the historical traffic picture comprises:

[0091] ​​1) using self-attention to extract key features of historical traffic sign pictures, and performing global average pooling operation on the extracted key features; in the local training stage, for the data features with H*W dimension size in C channels, global average pooling operation is performed. The average value of each feature map is taken, and the vector composed of the average values of the feature maps directly enters the next operation. This step forces the corresponding relationship between the feature map and the category, which is more suitable for the convolution structure.

[0092] Specifically, the global average pooling operation is realized by the following formula:

[0093]

[0094] a c The feature value of the historical traffic sign picture in the input model, a c The subscript c in a c represents the channel, and the above formula compresses the data input feature from the spatial dimension to the form of C*1*1, which converts the original two-dimensional feature into a real number β c . Since it is obtained by global average pooling operation, it responds to the global distribution of the initial sample in the feature channel to a certain extent, and can also obtain a certain degree of global receptive field.

[0095] 2) learning the correlation between different channels in the denoising neural network;

[0096] 3) normalizing the correlation output result to between 0 and 1 to obtain a scale parameter; in order to enable the sample data to learn the correlation between different channels in the denoising neural network, the real number β c is copied into two parts. The first part is still denoted as β c , and remains unchanged. The second part is copied through a small deep neural network FC, which is composed of two fully connected layers, two BN layers, a ReLU activation function and a last layer function as a basic component. The channel correlation parameters and are set in the two fully connected layers. By using the two channel correlation parameters, the nonlinear relationship between different channels can be learned in the training process of the model, so as to enhance the effective features and weaken the interference features.

[0097] Specifically, the scale parameter is calculated by the following formula:

[0098] θ=sigmoid(F fc2 (W2,F fc1 (W1,β c ))

[0099] In the formula: β c is a real number obtained after global average pooling, F fc1and F fc2 respectively represent two fully connected layers, W1 and W2 are channel correlation parameters, and c represents a channel;

[0100] The entire denoising network takes a sigmoid function as the last layer. The sigmoid function is used for the output of the hidden layer neurons, with a value range of (0, 1), which can map a real number to the interval (0, 1) to perform a scaling operation on the output result, and finally obtain the scale parameter θ c transformed from the output result of the correlation.

[0101] 4) Multiply the scale parameter and the correlation output result to obtain an adaptive threshold value;

[0102] Specifically, the adaptive threshold value ξ c is calculated as follows:

[0103] ξ c = θβ c .

[0104] The above formula not only ensures that the threshold value is positive, but also will not be too large. At the same time, different samples have different threshold values. Therefore, to some extent, it can be understood as a special attention mechanism: paying attention to features irrelevant to the current task and setting them to zero through soft thresholding; or, paying attention to features relevant to the current task and keeping them.

[0105] 5) Use the adaptive threshold value to perform soft thresholding denoising processing on the historical traffic sign pictures.

[0106] Specifically, the following formula is used for soft thresholding denoising processing:

[0107] η s (ξ c ) = sign(a c )max{0, |a c |- θβ c}

[0108] In the formula: a c is the feature value of the historical traffic sign picture in the input model, and η s (ξ c ) is the new feature value obtained after denoising.

[0109] Soft thresholding is the core step of many signal denoising methods. Its usefulness is to set the features whose absolute values are below a certain threshold to zero, and to adjust the other features to zero, that is, shrinkage. Here, the threshold is a parameter that needs to be set in advance, and its value has a direct impact on the result of denoising. And the embodiment obtains the self-adaptive threshold of each data sample itself through the above steps, and combines the threshold obtained by soft thresholding and self-attention mechanism to realize the optimization and denoising processing of the noisy data sample.

[0110] As a preferred embodiment, the loss function is calculated according to the following formula after the client local image recognition denoising network model is trained based on the training set:

[0111]

[0112] In the formula: loss(t) represents the loss function after the tth model training, l init is a cross-entropy loss function, λ is a constant, and 0≤λ≤0.5, is the model comparison loss.

[0113] Wherein, The calculation formula of is as follows:

[0114]

[0115] In the formula: τ is a control variable, ρ1 is the feature bias loss of the untrained local image recognition denoising network model in the current global communication round and the local image recognition denoising network model obtained after the tth training, and ρ2 is the feature bias loss of the local image recognition denoising network model obtained after the tth training and the local image recognition denoising network model obtained after the t-1th training.

[0116] As an example of the embodiment: client 1 constructs a local image recognition denoising network model U(1,1) based on D G(0) , and then trains U(1,1) based on the training set 1 to obtain the model U(1,1)1, and trains U(1,1)1 again to obtain the model U(1,1)2, then the loss function of the model U(1,1)2 is:

[0117]

[0118]

[0119] At this time, ρ1 is the untrained local image recognition denoising network model (that is, the model D G(0)) and the feature bias loss of the local image recognition denoising network model obtained after the second training (i.e., model U(1, 1)2) and the feature bias loss of the local image recognition denoising network model obtained after the first training (i.e., model U(1, 1)1), and p2 is the feature bias loss of the local image recognition denoising network model obtained after the second training (i.e., model U(1, 1)2) and the local image recognition denoising network model obtained after the first training (i.e., model U(1, 1)1);

[0120] Step S03: The server updates the global image recognition denoising network model based on the local model parameters transmitted by each client, tests the prediction accuracy of the updated global image recognition denoising network model based on the traffic sign recognition test set, and determines whether the prediction accuracy meets the global model accuracy threshold. If yes, the training is completed. If no, the step S04 is entered.

[0121] As an example of the present embodiment: on the basis of step S02, the server receives the local model parameters D U’(1,2) ,D U’(1,3) ,...D U’(1,10) , constructs a new global image recognition denoising network model G(1) based on the local model parameters D U’(1,2) ,D U’(1,3) ,...D U’(1,10) , tests the prediction accuracy of G(1) using the traffic sign recognition test set Test, and if the prediction accuracy of G(1) meets the global model accuracy threshold, it means that the global image recognition denoising network model has been trained well, and the training is completed. If the prediction accuracy of G(1) does not meet the global model accuracy threshold, it means that the global image recognition denoising network model has not been trained well, and the next step is entered.

[0122] Specifically, the server constructs a new global image recognition denoising network model based on the local model parameters transmitted by each client, including the following steps:

[0123] 1) receiving the local model parameters transmitted by each client, and obtaining the data quantity contained in the training set in the model training of each client;

[0124] 2) calculating the training data quantity proportion of each client, and taking the calculated training data quantity proportion as the weight of the local model parameter. The formula for calculating the training data quantity proportion of each client is:

[0125]

[0126] In the formula, D i is the training data quantity proportion of the client i, m i is the training set data quantity of the client i, and I is the number of clients;

[0127] 3) Update the parameters of the global image recognition denoising network model based on the model parameters and the proportion of the number of training data, and the calculation formula is as follows:

[0128]

[0129] In the formula: represents the value of parameter j of client i, W j is the value of parameter j of the new global image recognition denoising network model;

[0130] 4) Update the parameters W j of the global image recognition denoising network model according to the updated global image recognition denoising network model parameters calculated.

[0131] Step S04: Obtain the global model parameters of the new global image recognition denoising network model and send them to the client, repeat steps S02-S03 until the prediction accuracy of the global image recognition denoising network model meets the global model accuracy threshold requirement, and obtain the trained global image recognition denoising network model.

[0132] As an example of this embodiment: in step S03, the prediction accuracy of the global image recognition denoising network model G(1) newly constructed on the server does not meet the global model accuracy threshold requirement, then obtain the model parameters D G(1) of G(1), send the model parameters D G(1) to 10 clients, and the 10 clients construct local image recognition denoising network models U(2,1), U(2,2), U(2,3),..., U(2,10) according to the received model parameters D G(1) , train the local image recognition denoising network models U(2,1), U(2,2), U(2,3),..., U(2,10) according to the method in step S02, obtain the trained local image recognition denoising network models U'(2,1), U'(2,2), U'(2,3),..., U'(2,10), obtain the corresponding local model parameters D U’(2,1) , D U’(2,2) ,...D U’(2,10) and feedback to the server, and the server constructs a global image recognition denoising network model G(2) based on the local model parameters D U’(2,1) , D U’(2,2) ,...D U’(2,10) , and continuously repeats the above content until a global image recognition denoising network G(q) with prediction accuracy meeting the global model accuracy threshold is obtained, then G(q) is the trained global image recognition denoising network model.

[0133] For example, Figure 3As shown, the embodiment also provides a traffic road sign recognition system based on soft threshold denoising contrastive federated learning, which is used to implement the steps of the traffic road sign recognition method based on soft threshold denoising contrastive federated learning in the embodiment. The system includes a server and a plurality of clients, and each client is in communication connection with the server.

[0134] The server includes a first model construction module, a first data transmission module, and a verification module. The first model construction module is used to construct a global model on the server. The first data transmission module is used to realize data transmission between the server and the clients. The verification module is used to verify whether a new global image recognition denoising network model constructed based on local model parameters meets the global model precision threshold requirement.

[0135] The client includes a training set construction module, a second model construction module, a model training module, and a second data transmission module. The training set construction module is used to construct a training set. The second model construction module is used to construct a model on the client. The model training module is used to train the model on the client. The second data transmission module is used to realize data transmission with the server.

[0136] The client further includes a local model verification module, which is used to verify whether the prediction accuracy of the local model meets the local model precision threshold after each local model training.

[0137] Specifically, the training set construction module is used to complete the following processes:

[0138] Obtain historical traffic road sign pictures stored by the client;

[0139] Label the historical traffic road sign pictures with road sign type labels, which are the names of road sign types.

[0140] Construct a training set based on the historical traffic road sign pictures and the corresponding road sign type labels.

[0141] Specifically, the model training module is used to complete the following processes:

[0142] Input the training set into the local image recognition denoising network model of the client;

[0143] Perform denoising processing on the historical traffic road sign pictures by using the denoising neural network in the local image recognition denoising network model of the client to obtain denoised historical traffic road sign pictures;

[0144] Identify the denoised historical traffic road sign pictures to obtain identification results.

[0145] Further, the model training module is also used to complete the following processes:

[0146] The self-attention is used to extract key features of the historical traffic sign pictures, and global average pooling operation is performed on the extracted key features;

[0147] The correlation between different channels is learned in the noise reduction neural network;

[0148] The correlation output result is normalized to between 0 and 1 to obtain a scale parameter;

[0149] The scale parameter is multiplied by the correlation output result to obtain an adaptive threshold;

[0150] The adaptive threshold is used for soft threshold denoising processing of the historical traffic sign pictures.

[0151] Specifically, the first model construction module is also used to complete the following processes:

[0152] The local model parameters transmitted by each client are received, and the number of data contained in the training set in the model training of each client is obtained;

[0153] The training data quantity proportion of each client is calculated, and the calculated training data quantity proportion is used as the weight of the local model parameter. The formula for calculating the training data quantity proportion of each client is:

[0154]

[0155] In the formula, D i is the training data quantity proportion of the client i, m i is the number of training set data of the client i, and I is the number of clients;

[0156] The parameters of the global image recognition denoising network model are updated based on the model parameters and the training data quantity proportion, and the calculation formula is as follows:

[0157]

[0158] In the formula, represents the value of the parameter j of the client i, W j is the value of the parameter j of the new global image recognition denoising network model;

[0159] A new global image recognition denoising network model is constructed according to the calculated new global image recognition denoising network model parameters.

[0160] Although the embodiments of the present application have been shown and described, it can be understood by those of ordinary skill in the art that various changes, modifications, replacements and variations can be made to the embodiments without departing from the principles and spirits of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A contrastive federated learning traffic sign recognition method based on soft threshold denoising, characterized in that, Includes the following steps: S1: Transmit the model parameters of the trained global image recognition and denoising network model to the client, and construct a local image recognition and denoising network model on the client based on the model parameters; the global image recognition and denoising network model uses a residual network as its model architecture and embeds a denoising neural network. S2: Obtain the traffic sign image to be recognized on the client side, input the traffic sign image to be recognized into the local image recognition and denoising network model, and obtain the traffic sign recognition result output by the model; The training process of the global image recognition and denoising network model includes: Step S01: Initialize the federated learning framework, build an initial global image recognition and denoising network model and a traffic sign recognition test set on the server, and send the global model parameters of the initial global image recognition and denoising network model to all clients; Step S02: On each client, a local image recognition and denoising network model is built based on the global model parameters sent by the server, and a training set is constructed based on historical traffic sign images obtained by the client. The local image recognition and denoising network model is trained on each client based on the training set to obtain a trained local image recognition and denoising network model. The local model parameters of the trained local image recognition and denoising network model are then transmitted to the server. Key features of historical traffic sign images are extracted using self-attention, and global average pooling is performed on the extracted key features. The correlation between different channels is learned in the denoising neural network. The correlation output is normalized to between 0 and 1 to obtain a scale parameter. The scale parameter is multiplied by the correlation output to obtain an adaptive threshold. The adaptive threshold is used to perform soft-thresholding denoising on the historical traffic sign images. Step S03: On the server side, update the global image recognition and denoising network model based on the local model parameters transmitted by each client, and test the prediction accuracy of the new global image recognition and denoising network model based on the traffic sign recognition test set. Determine whether the prediction accuracy meets the global model accuracy threshold. If yes, the training ends; otherwise, proceed to step S04. Step S04: Obtain the global model parameters of the global image recognition and denoising network model updated in step S03 and send them to the client. Repeat steps S02-S03 until the prediction accuracy of the global image recognition and denoising network model meets the global model accuracy threshold requirement, and obtain the trained global image recognition and denoising network model.

2. The contrastive federated learning traffic sign recognition method based on soft threshold denoising according to claim 1, characterized in that, In step S02, the method for constructing the training set includes: Retrieve historical traffic sign images stored on the client; Label historical traffic sign images with sign type tags, where the sign type tag is the name of the sign type; A training set was constructed based on historical traffic sign images and their corresponding sign type labels.

3. The contrastive federated learning traffic sign recognition method based on soft threshold denoising according to claim 1, characterized in that, In step S01, the constructed traffic sign recognition test set is transmitted to each client; In step S02, the client-side local image recognition and denoising network model is trained multiple times based on the training set, and the accuracy of the model is detected based on the traffic sign recognition test set after each training. When the accuracy of the client-side local image recognition and denoising model reaches the local model accuracy threshold, the training of the client-side local image recognition and denoising network model is stopped.

4. The contrastive federated learning traffic sign recognition method based on soft threshold denoising according to claim 3, characterized in that, The training of the client-side local image recognition and denoising network model based on the training set includes: The training set is input into the local image recognition and denoising network model on the client side; The noise reduction neural network in the local image recognition and noise reduction network model on the client is used to perform noise reduction processing on historical traffic sign images to obtain noise-reduced historical traffic sign images; The denoised historical traffic sign images were then identified to obtain the recognition results.

5. The contrastive federated learning traffic sign recognition method based on soft threshold denoising according to claim 1, characterized in that, The scale parameter is calculated using the following formula: ， In the formula: These are real numbers obtained after global average pooling. and These represent two fully connected layers. and This is a channel association parameter, where 'c' represents the channel. The adaptive threshold The calculation formula is as follows: 。 6. The contrastive federated learning traffic sign recognition method based on soft threshold denoising according to claim 5, characterized in that, The following formula is used for soft thresholding noise reduction: ， In the formula: Input the feature values ​​of historical traffic sign images into the model. These are the new feature values ​​obtained after noise reduction.

7. The contrastive federated learning traffic sign recognition method based on soft threshold denoising according to claim 3, characterized in that, After training the local image recognition and denoising network model on the client based on the training set, the loss function is calculated using the following formula: ， In the formula: Let represent the loss function after the t-th model training iteration. Let cross-entropy be the loss function. It is a constant, and , For model contrastive loss; in, The calculation formula is as follows: ， In the formula: It is a control variable. It is the feature bias loss between the untrained local image recognition and denoising network model in the current global communication round and the local image recognition and denoising network model obtained after training in the t-th round. It is the feature bias loss of the local image recognition and denoising network model obtained after training t and training t-1.

8. The contrastive federated learning traffic sign recognition method based on soft threshold denoising according to claim 1, characterized in that, In step S03, constructing a new global image recognition and denoising network model on the server side based on the local model parameters transmitted by each client includes the following steps: Receive local model parameters transmitted from each client and obtain the amount of data contained in the training set of each client model training; Calculate the proportion of training data for each client, and use this proportion as the weights of the local model parameters. The formula for calculating the proportion of training data for each client is: ， In the formula: D i For the client i The percentage of training data, m i For the client i The amount of training set data, I The number of clients; The parameters of the global image recognition and denoising network model are updated based on the model parameters and the proportion of training data. The calculation formula is as follows: ， In the formula: The representative is the client. i parameters j The value, Parameters for the new global image recognition and denoising network model j The value; The global image recognition and denoising network model is updated based on the parameters of the updated global image recognition and denoising network model.

9. A traffic sign recognition system based on soft threshold denoising and contrastive federated learning, used to implement the steps of the traffic sign recognition method based on soft threshold denoising and contrastive federated learning as described in any one of claims 1-8, characterized in that, It includes a server and several clients, with the server communicating with each client. The server includes a first model building module, a first data transmission module, and a verification module; the first model building module is used for model building; the first data transmission module is used for data transmission with the client; the verification module is used to verify whether the new global image recognition and denoising network model built based on local model parameters meets the global model accuracy threshold requirements. The client includes a training set construction module, a second model construction module, a model training module, and a second data transmission module; the training set construction module is used to construct a training set, the second model construction module is used to construct a model on the client, the model training module is used to train the model on the client, and the second data transmission module is used to implement data transmission with the server.

Images