A surface defect detection model training method based on federated learning
The federated learning approach for surface defect detection models addresses the scarcity of defect images by prioritizing high-quality contributions and ensuring data privacy, resulting in efficient and secure model training.
Patent Information
- Application Number
- CN202210420922.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-20
- Publication Date
- 2025-03-28
- Estimated Expiration
- 2042-04-20
AI Technical Summary
Current methods for training surface defect detection models face challenges in obtaining sufficient defect images, leading to inefficient training processes and potential data privacy issues, especially in industrial settings where defect images are scarce and sensitive.
A federated learning approach is employed, where client devices preprocess and train local models, with a server aggregating parameters based on image quality weights, ensuring data privacy and reducing communication bottlenecks by selecting active clients and prioritizing high-quality image contributors.
This method shortens the image collection cycle, maintains data privacy, and enhances model training efficiency by prioritizing stable communication and high-quality image contributions, reducing the impact of low-quality images on the overall model performance.
Smart Images
Figure CN114881939B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of federated learning, and in particular to a surface defect detection model training method based on federated learning. Background Art
[0002] In the production process of industrial products, due to production processes or human factors, scratches, cracks, holes and other defects may occur on the surface of the product. These defects greatly reduce the quality of industrial products, so these products need to be screened out in the quality inspection process. Surface defect detection refers to the use of computer programs to automatically determine whether there are defects on the surface of the product. Compared with manual quality inspection, surface defect detection technology is more efficient and is not affected by fatigue working conditions. At present, surface defect detection is generally based on machine vision methods, that is, first build a neural network model (in this scenario, it can be called a surface defect detection model), and then use a large number of product surface images to train this model so that it has the ability to extract image features, and then use this model to determine whether the product has defects. However, training a neural network model requires a large number of product surface images, and the surface images of product defects are not easy to obtain because the product defect rate of mature industrial production lines is low. Therefore, in order to train a usable surface defect detection model, there needs to be a method to obtain a sufficient number of product surface defect images or features.
[0003] In order to solve the problem of insufficient number of defect images for training surface defect detection models, there are currently several methods:
[0004] 1. Obtain new defect images by randomly rotating and scaling the original defect images; randomly adjusting the chroma, brightness, and saturation of the image; randomly adding noise, etc., and use these images to train the defect detection model.
[0005] 2. Use the transfer learning method to initialize the model with a general data set, and then fine-tune the model using a small number of defective samples.
[0006] 3. Cooperate with multiple manufacturers of similar products to collect surface defect images of each product to train defect detection models.
[0007] For the first method, although the structural features of the images have changed after various random processing, the features of the defects have not changed, so the defect features for the model to learn are still insufficient. For the second method, although the model can be trained, due to the small number of defective images, the final trained model will reach an overfitting state, with low generalization performance and cannot be put into use. For the third method, although it is technically feasible, in real life, the product surface images held by various manufacturers contain commercial secrets such as production processes and packaging technologies, so manufacturers may not agree to centralized training, and the feasibility is low.
[0008] Therefore, technicians in this field are committed to developing a method that can shorten the defect image acquisition time, train a qualified defect detection model, and ensure the confidentiality of the original data. Summary of the invention
[0009] In view of the above-mentioned defects of the prior art, the technical problem to be solved by the present invention is how to develop a method that can shorten the defect image acquisition time, train a qualified defect detection model, and ensure the confidentiality of the original data.
[0010] To achieve the above object, the present invention provides a surface defect detection model training method based on federated learning, characterized in that the method comprises the following steps:
[0011] Step 1: Client local data processing;
[0012] Step 2: Client pre-selection algorithm;
[0013] Step 3: Client local training;
[0014] Step 4: Parameter aggregation algorithm based on image quality weight;
[0015] Step 5: Model combination.
[0016] Furthermore, the step 1 also includes the following method:
[0017] Step S1: local data collection, each client collects product surface pictures, unifies the picture format and size, classifies the pictures according to "normal" and "defective", and saves them to the storage medium;
[0018] Step S2: training the target detection model, using software to mark the position of the product in the product surface picture, and generating a marking file, the marking file should contain the coordinate information of the product position, selecting a suitable target detection network such as YoloV4, downloading the network model file and the initial weight file; each client uses its own pictures and corresponding marking files to train the target detection network, and saves the final model;
[0019] Step S3: Communication test, the server sends a signal to each client, and the received client sends a signal to the server to indicate that it is online. The server determines how many clients that can work normally can participate in the training according to the reply signal of the client;
[0020] Step S4: Estimate important indicators. Each client calculates the estimated transmission time for this round based on historical transmission time and training time, combined with current communication conditions and computing power conditions. and calculation time
[0021] Furthermore, in the step 2, first, the server will determine whether the client currently supports communication and can guarantee stable communication for a certain period of time; then, the client needs to inform the central server of parameters such as the current communication bandwidth, training time, resource size, and related historical data. After calculation, the central server will exclude those clients that can communicate but have a long communication time. This allows as many clients as possible to complete a round of training within the limited communication time, thereby reducing the total training time;
[0022] C={C1,,,C K} represents the K clients participating in federated learning, S = {k1,,,k |S|},k i ∈C represents the clients that participate in the next round of training after being screened by the client pre-selection algorithm, T r is the upper limit of each round of training time, T f is the upper limit of total training time, T cs The time pre-selected by the client, T ag The time for parameter aggregation; It represents the time it takes for the server to distribute parameters to each client, which is related to the communication bandwidth of each channel; and They represent the time taken for the kth client to train the local model and upload the model parameters. The above parameters are sent to the server by each client during the client pre-selection phase. Since the amount of data is very small compared to the model parameters and bandwidth, the transmission time can be ignored.
[0023] The objective function of the client pre-selection algorithm is max|S|, that is, to select as many clients as possible to participate in the training within the set single-round training time; the local model training of the client is parallel, so the client with the largest training time among the selected clients can be selected. Uploading parameters requires each client to establish a connection with the server separately, so this process is serial; the total time θ for each round of client training and communication can be expressed as follows:
[0024] Θ=Ttr +T ul
[0025]
[0026]
[0027] In summary, the final optimization problem can be summarized as:
[0028] max|S|
[0029]
[0030] Because the variables have training time T tr and upload time two T ul , while T tr Yes Take all The maximum value in , so we can first fix T tr Find the longest T ul sequence; first, T tr and T ul Sort, let T ul is zero, find the maximum Then find the longest T that satisfies the constraints. ul Sequence, variable l max Record the longest length of the sequence; in the next cycle, reduce T tr , continue the above operation until the number of remaining clients is less than l at this time max ; Record the final S sequence and obtain the clients participating in this round of training.
[0031] Furthermore, the step 2 also includes the following method:
[0032] Step M1: Determine fixed indicators and reasonably set the upper limit T for each round of training time r , determine the client's pre-selected time T based on historical data cs , parameter aggregation time T sg , the time it takes for the server to distribute parameters to each client
[0033] Step M2: variable preprocessing, obtain the transmission time of each client from step 1 and calculation time Save to sequence separately and In the sequence Sort in reverse order, Sequential sorting;
[0034] Step M3: Get the client sequence participating in the training, let T ulis zero, find the maximum value that satisfies the condition As this round of T tr , and then find the longest T that satisfies the constraints ul Sequence, variable l max Record the longest length of the sequence; in the next cycle, reduce T tr , continue the above operation until the number of remaining clients is less than l at this time max Or it is zero, record the final S sequence, and get the clients participating in this round of training.
[0035] Furthermore, the step three also includes the following method:
[0036] Step N1: Initialization: each client participating in this round of training obtains the model initial weight file from the server and initializes the local model;
[0037] Step N2: training, each client uses the local product surface image to train the local model according to the set rounds;
[0038] Step N3: Parameter upload: each client uploads the model parameters obtained after training to the server.
[0039] Furthermore, the step 4 also includes the following method:
[0040] Step T1: Test model training: Use normal and clear pictures in the server to train a test model with certain judgment ability. Compared with the classification model finally obtained by federated learning, the accuracy of this model is lower and it is mainly used to test the quality of local pictures. Distribute this model to each client;
[0041] Step T2: Calculate the image quality weight. Each client shuffles the local images and randomly extracts a certain number of images, and uses these images to test the accuracy of the test model received from the server, which is recorded as acc k , using the formula Calculate the image quality weight q k ;
[0042] Step T3: Parameter aggregation. After the server receives the local training parameters of each client, it uses the formula to aggregate the parameters.
[0043] Among them, q k Represents the image quality weight, acc k It indicates the accuracy of the model tested by each client using the random sample. Since the lowest accuracy is about 0.5, it is necessary to subtract this base and take the absolute value; n k represents the number of images held by the client, n represents the total number of images held by all clients, and wk Indicates the model parameters obtained by each client using local images for training.
[0044] Furthermore, the step five is specifically to loop steps two, three and four until the model on the server reaches a convergence state, test the accuracy of the model, and if the result is normal, send the model to each client.
[0045] Furthermore, in step 5, each client will use the locally trained target detection model and the image classification model trained by federated learning to perform defect detection on the product surface.
[0046] Furthermore, in the step five, the client defect detection method is to first take a picture of the product surface; then use the target detection model to obtain the area to be inspected in the picture and remove irrelevant background; then use the image classification model to judge the area to be inspected and obtain the final detection result.
[0047] Furthermore, in the step N2, the specific process of using the federated learning training model can be mainly divided into the following steps: the first step is to establish a connection. The server will test the communication status with each client and select online available nodes to participate in this round of training; the second step is initialization. The server initializes the model and sends the initial model parameters to each client node participating in the training; the third step is local training. After the client receives the initial model parameters, it uses local samples to train the model and sends the parameters obtained in this stage to the server; the fourth step is parameter aggregation. The server obtains the model parameters of each client node participating in this round of training, aggregates the parameters using an aggregation algorithm, and updates the overall model; finally, the overall model parameters are distributed to each client node; the above steps are executed in a loop until the overall model reaches a preset state, and the loop is exited to complete the training.
[0048] The present invention has the following technical effects:
[0049] 1. The cycle of collecting surface defect images is shorter and the local data of each training participant will not be leaked;
[0050] 2. Reduced the impact of temporary poor communication conditions on the overall training speed;
[0051] 3. Reduced the impact of training participants holding low-quality images on the overall model performance.
[0052] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, characteristics and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] Figure 1It is a surface defect detection model training process based on federated learning in a preferred embodiment of the present invention;
[0054] Figure 2 This is the operation process of federated learning of a preferred embodiment of the present invention. DETAILED DESCRIPTION
[0055] The following describes several preferred embodiments of the present invention with reference to the drawings in the specification, so that the technical content is clearer and easier to understand. The present invention can be embodied in many different forms of embodiments, and the protection scope of the present invention is not limited to the embodiments mentioned in the text.
[0056] In the drawings, components with the same structure are indicated by the same numerical reference numerals, and components with similar structures or functions are indicated by similar numerical reference numerals. The size and thickness of each component shown in the drawings are arbitrarily shown, and the present invention does not limit the size and thickness of each component. In order to make the illustration clearer, the thickness of the components is appropriately exaggerated in some places in the drawings.
[0057] In order to solve the problem of lack of defect samples, the best way is still to collect enough defect images. Considering that industrial products themselves are not isolated items, a defect detection model can be jointly trained with the help of surface defect images of similar products. This can not only shorten the time of defect image collection, but also train a qualified defect detection model. However, there is a problem of confidentiality of original data, so the method of federated learning can be introduced. Federated learning is an encrypted distributed machine learning method. Its network topology is a central server connected to multiple clients, and the server is responsible for coordinating all clients for training. Compared with ordinary distributed machine learning, the data held by each client of federated learning is not directly uploaded to the server, but only the model parameters of local training are shared with the server, ensuring the security of local data of each manufacturer. If it is mapped to a real scene, the central server can be considered as a cloud device for coordinating training, and the client can be considered as an edge device in each manufacturer that holds defect images and has a certain computing power. In order to unify the name, the following will refer to the server and the client respectively.
[0058] Without loss of generality, the surface defect detection algorithm can be divided into a target detection model for determining the location of the product and an image classification model for determining whether there are defects. In actual use, it can be divided into two stages. In the first stage, the target detection model is used to determine the location of the product and extract the target area. In the second stage, the image classification model is used to determine whether the target area contains defects. From the training perspective, because the target detection model in the first stage only needs to extract the product location, both normal pictures and pictures with defects can be used for training, and the demand for defective pictures is not high. The image classification model in the second stage needs to determine whether there are defects, so a large number of defective pictures are required for training. Then, in the actual scenario, each client first completes the training of the target detection model locally, and then chooses to use federated learning to train the image classification model with defective pictures of multiple clients while ensuring confidentiality. Finally, each client obtains the trained image classification model, and then combines it with the previously trained local target detection model to obtain a surface defect detection model suitable for its own product. The above steps are shown in the attached figure. Figure 1 shown.
[0059] The specific process of using federated learning to train a model can be divided into the following steps: Step 1, establish a connection. The server will test the communication status with each client and select online available nodes to participate in this round of training. Step 2, initialization. The server initializes the model and sends the initial model parameters to each client node participating in the training. Step 3, local training. After receiving the initial model parameters, the client uses local samples to train the model and sends the parameters obtained in this stage to the server. Step 4, parameter aggregation. The server obtains the model parameters of each client node participating in this round of training, aggregates the parameters using an aggregation algorithm, and updates the overall model. Finally, distribute the overall model parameters to each client node. Repeat the above steps until the overall model reaches the preset state, exit the loop to complete the training. The above steps are as shown in the attached figure. Figure 2 shown.
[0060] The above-mentioned federated learning process is a general process for training all network models. However, in the surface defect detection scenario, the use of federated learning training models will encounter some unique problems, which will be optimized one by one below. This patent mainly optimizes the process of federated learning training defect detection models through the following two methods. First, considering that the multiple factories represented by the client may be separated by multiple locations and the communication conditions are not stable, some clients may spend more time in the parameter transmission process at certain moments, affecting the total training time of all nodes as a whole. Therefore, a client pre-selection algorithm is proposed to pre-select the nodes participating in the training based on the current communication conditions and training time. Secondly, considering that each node has different image quality, the model parameters trained with low-quality pictures will lower the overall model quality, and should be given a lower weight when aggregating parameters. Therefore, a parameter aggregation algorithm based on image quality weight is proposed.
[0061] The client pre-selection algorithm is performed before each round of communication. First, the server will determine whether the client currently supports communication and can guarantee stable communication for a certain period of time; then, the client needs to inform the central server of parameters such as the current communication bandwidth, training time, resource size, and related historical data. After calculation, the central server will exclude those clients that can communicate but have a long communication time. This allows as many clients as possible to complete a round of training within the limited communication time, reducing the total training time.
[0062] Assume C = {C1,,,C K} represents the K clients participating in federated learning, S = {k1,,,k |S|},k i ∈C represents the clients that participate in the next round of training after being screened by the client pre-selection algorithm, T r is the upper limit of each round of training time, T f is the upper limit of total training time, T cs The time pre-selected by the client, T ag The time at which the parameter is aggregated. It indicates the time it takes for the server to distribute parameters to each client, which is related to the communication bandwidth of each channel. and They represent the time it takes to train the local model and upload the model parameters of the kth client. The above parameters are sent to the server by each client during the client pre-selection phase. Since the amount of data is very small compared to the model parameters and bandwidth, the transmission time can be ignored.
[0063] The objective function of the client pre-selection algorithm is max|S|, that is, to select as many clients as possible to participate in the training within the set single round of training time. The local model training of the client is parallel, so the client with the longest training time can be selected. Uploading parameters requires each client to establish a connection with the server separately, so this process is serial. The total time θ for each round of client training and communication can be expressed as follows:
[0064] Θ=T tr +T ul (1)
[0065]
[0066]
[0067] In summary, the final optimization problem can be summarized as:
[0068] max|S| (4)
[0069]
[0070] After analysis, it can be found that the optimization goal and constraints of this problem are similar to the knapsack problem, so this paper solves it based on the idea of greedy algorithm. tr and upload time two T ul , while T tr Yes Take all The maximum value in , so we can first fix T tr Find the longest T ul First, T tr and T ul Sort, let T ul is zero, find the maximum Then find the longest T that satisfies the constraints. ul Sequence, variable l max Record the longest length of the sequence. In the next cycle, reduce T tr , continue the above operation until the number of remaining clients is less than l at this time max . Record the final S sequence and get the clients participating in this round of training.
[0071] The parameter aggregation algorithm based on image quality weight optimizes the "parameter aggregation" link after the server receives the model parameters of each client. The commonly used parameter aggregation algorithm mainly adjusts the local model parameters according to the number of pictures held by each client. Clients holding more pictures will have higher weights during aggregation. In the surface defect detection scenario, it is known from experience that the factors affecting model performance include not only the number of training pictures, but also the quality of training pictures. More training pictures can enable the model to learn more knowledge during training and improve the generalization of the model, while higher image quality can make the knowledge learned by the model more refined. In this patent, picture quality refers to the clarity of the image, because in the surface defect detection scenario, the camera automatically takes pictures with a fixed focus, so the picture is often blurred due to loss of focus. For defect detection, if the defects in the training pictures are clearly visible and easy to identify, then the key features learned by the model will inevitably be more than those blurred defect pictures. Therefore, when aggregating parameters, clients holding more clear pictures should be given higher weights because the local models they train have better performance.
[0072] This paper uses the method of testing the accuracy of known models with the images to be tested and reversely deriving the proportion of clear images to obtain the image quality weight of each client. First, the server holds a certain number of test images, and first uses this batch of clear test images to train a preliminary model. Although this model cannot be compared with the model finally obtained by federated learning, it also has certain detection capabilities. Before the formal start of federated learning, this model is sent to each client, and a part of the images are randomly selected after the training images are shuffled to test the accuracy of this model. Through reverse analysis, it can be seen that when the model is determined, the higher the proportion of blurred images in the test image, the lower the accuracy of the tested model, so that the proportion of blurred images held by the client can be inferred. In the subsequent aggregation process, this parameter is used as a reference factor for image quality, and the corresponding parameter aggregation weight is modified to give nodes with a high proportion of clear images a larger aggregation weight. The calculation process is as follows:
[0073]
[0074]
[0075] Among them, q k Represents the image quality weight, acc k Indicates the model accuracy of each client using the sample test. Since the lowest accuracy is about 0.5, it is necessary to subtract this base and take the absolute value. k represents the number of images held by the client, n represents the total number of images held by all clients, and w kIndicates the model parameters obtained by each client using local images for training. Because the images held by each node will not change during a single training, the image quality weight of each node can be determined before the federated learning starts. Then the image quality detection of each node can be completed before the training starts and summarized inside the server, which also ensures that the pre-image quality detection will not affect the formal training.
[0076] The surface defect detection method based on federated learning designed in this patent is mainly divided into the following steps:
[0077] Step 1: Client local data processing, this step can be mainly divided into the following steps:
[0078] S1: Local data collection
[0079] Each client collects product surface pictures, unifies the picture format and size, classifies the pictures into "normal" and "defective", and saves them to storage media.
[0080] S2: Training the object detection model
[0081] Use software to mark the position of the product in the product surface image and generate a labeling file. The labeling file should contain the coordinate information of the product location. Select a suitable target detection network such as YoloV4, download the network model file and the initial weight file. Each client uses its own images and corresponding labeling files to train the target detection network and save the final model.
[0082] S3: Communication test
[0083] The server sends a signal to each client, and the received client sends a signal to the server to indicate that it is online. The server determines how many normal working clients can participate in the training based on the client's reply signal.
[0084] S4: Estimate important indicators
[0085] Each client calculates the estimated transmission time for this round based on historical transmission time and training time, combined with current communication conditions and computing power conditions. and calculation time
[0086] Step 2: Client pre-selection algorithm, this step can be mainly divided into the following steps:
[0087] S1: Determine fixed indicators
[0088] Reasonably set the upper limit T for each round of training r , determine the client's pre-selected time T based on historical data cs , parameter aggregation time Tag , the time it takes for the server to distribute parameters to each client
[0089] S2: Variable preprocessing
[0090] Get the transmission time of each client from step 1 and calculation time Save to sequence separately and For the sequence Sort in reverse order, Sort by order.
[0091] S3: Get the client sequence participating in training
[0092] Let T ul is zero, find the maximum value that satisfies the condition As this round of T tr , and then find the longest T that satisfies the constraints ul Sequence, variable l max Record the longest length of the sequence. In the next cycle, reduce T tr , continue the above operation until the number of remaining clients is less than l at this time max Or zero. Record the final S sequence and get the clients participating in this round of training.
[0093] Step 3: Client local training, this step can be mainly divided into the following steps:
[0094] S1: Initialization
[0095] Each client participating in this round of training obtains the model initial weight file from the server and initializes the local model.
[0096] S2: Training
[0097] Each client uses local product surface images to train the local model in a set number of rounds.
[0098] S3: parameter upload
[0099] Each client uploads the model parameters obtained after training to the server.
[0100] Step 4: Parameter aggregation algorithm based on image quality weight. This step can be mainly divided into the following steps:
[0101] S1: Test model training
[0102] A test model with certain judgment ability is trained in the server using normal clear pictures. Compared with the classification model finally obtained by federated learning, the accuracy of this model is lower and it is mainly used to test the quality of local pictures. The model is distributed to each client.
[0103] S2: Calculate image quality weight
[0104] Each client shuffles the local images and randomly extracts a certain number of images, and uses these images to test the accuracy of the test model received from the server, which is recorded as acc k . Use formula (5) to calculate the image quality weight q k .
[0105] S3: Parameter Aggregation
[0106] After receiving the local training parameters from each client, the server uses formula (6) to aggregate the parameters.
[0107] Step 5: Model combination
[0108] Repeat steps 2, 3, and 4 until the model on the server reaches a convergence state. Test the accuracy of the model. If the result is normal, send the model to each client. Each client will use the locally trained target detection model and the image classification model trained by federated learning to perform defect detection on the product surface. First, take a picture of the product surface, then use the target detection model to obtain the area to be inspected in the picture and remove irrelevant background. Then use the image classification model to judge the area to be inspected and obtain the final test result.
[0109] The preferred specific embodiments of the present invention are described in detail above. It should be understood that ordinary technicians in the field can make many modifications and changes based on the concept of the present invention without creative work. Therefore, all technical solutions that can be obtained by technicians in the technical field based on the concept of the present invention through logical analysis, reasoning or limited experiments on the basis of the prior art should be within the scope of protection determined by the claims.
Claims
1. A surface defect detection model training method based on federated learning, characterized in that: The method comprises the following steps: Step 1: Client local data processing; Step 2: Client pre-selection algorithm; Step 3: Client local training; Step 4: Parameter aggregation algorithm based on image quality weight; Step 5: Model combination; In the step 2, first, the server will determine whether the client currently supports communication and can guarantee stable communication for a certain period of time; then, the client needs to inform the central server of the current communication bandwidth, training time, resource size, and related historical data parameters. After calculation, the central server will exclude those clients that can communicate but have a long communication time. This allows as many clients as possible to complete a round of training within the limited communication time, thereby reducing the total training time; C={C1,,,C K } represents the K clients participating in federated learning, S = (k1,,,k |S| }, k i ∈C represents the clients that participate in the next round of training after being screened by the client pre-selection algorithm, T r is the upper limit of each round of training time, T f is the upper limit of total training time, T cs The time pre-selected by the client, T ag The time for parameter aggregation; It represents the time it takes for the server to distribute parameters to each client, which is related to the communication bandwidth of each channel; and They represent the time taken for the kth client to train the local model and upload the model parameters. The above parameters are sent to the server by each client during the client pre-selection phase. Since the amount of data is very small compared to the model parameters and bandwidth, the transmission time can be ignored. The objective function of the client pre-selection algorithm is max|S|, that is, to select as many clients as possible to participate in the training within the set single-round training time; the local model training of the client is parallel, so the client with the largest training time among the selected clients can be selected. Uploading parameters requires each client to establish a connection with the server separately, so this process is serial; the total time θ for each round of client training and communication can be expressed as follows: Θ=T tr +T ul In summary, the final optimization problem can be summarized as: max|S| Because the variables have training time T tr and upload time two T ul , while T tr Yes Take all The maximum value in , so we can first fix T tr Find the longest T ul sequence; first, T tr and T ul Sort, let T ul is zero, find the maximum Then find the longest T that satisfies the constraints. ul Sequence, variable l max Record the longest length of the sequence; in the next cycle, reduce T tr , continue the above operation until the number of remaining clients is less than l at this time max ; Record the final S sequence and get the clients participating in this round of training; The step 2 also includes the following method: Step M1: Determine fixed indicators and reasonably set the upper limit T for each round of training time r , based on historical data, determine the client's pre-selected time T cs , parameter aggregation time T ag , the time it takes for the server to distribute parameters to each client Step M2: Variable preprocessing, obtain the transmission time of each client from step 1 and calculation time Save to sequence separately and In the sequence Sort in reverse order, Sequential sorting; Step M3: Get the client sequence participating in the training, let T ul is zero, find the maximum value that satisfies the condition As this round of T tr , and then find the longest T that satisfies the constraints ul Sequence, variable l max Record the longest length of the sequence; in the next cycle, reduce T tr , continue the above operation until the number of remaining clients is less than l at this time max Or zero, record the final S sequence and get the clients participating in this round of training; The step three also includes the following method: Step N1: Initialization: each client participating in this round of training obtains the model initial weight file from the server and initializes the local model; Step N2: training, each client uses the local product surface images to train the local model according to the set rounds; Step N3: parameter upload, each client uploads the model parameters obtained after training to the server; The step 4 also includes the following method: Step T1: Test model training. Use normal and clear pictures in the server to train a test model with certain judgment ability. Compared with the classification model finally obtained by federated learning, the accuracy of this model is lower. It is mainly used to test the quality of local pictures and distribute the model to each client. Step T2: Calculate the image quality weight. Each client shuffles the local images and randomly extracts a certain number of images, and uses these images to test the accuracy of the test model received from the server, which is recorded as acc k , using the formula Calculate the image quality weight qk; Step T3: Parameter aggregation. After the server receives the local training parameters of each client, it uses the formula to aggregate the parameters. Among them, q k Represents the image quality weight, acc k It indicates the accuracy of the model tested by each client using the random sample. Since the lowest accuracy is about 0.5, it is necessary to subtract this base and take the absolute value; n k Indicates the number of pictures held by the client, n indicates the total number of pictures held by the client, and w k Indicates the model parameters obtained by each client using local images for training; The step 5 is specifically to loop through steps 2, 3 and 4 until the model on the server reaches a convergence state, test the accuracy of the model, and if the result is normal, send the model to each client; In step 5, each client will use the locally trained target detection model and the federated learning trained image classification model to perform defect detection on the product surface.
2. The surface defect detection model training method based on federated learning according to claim 1, characterized in that: The step 1 also includes the following method: Step S1: local data collection, each client collects product surface pictures, unifies the picture format and size, classifies the pictures according to "normal" and "defective", and saves them to the storage medium; Step S2: training the target detection model, using software to mark the position of the product in the product surface picture, and generating a marking file, the marking file should contain the coordinate information of the product position, selecting a suitable target detection network such as YoloV4, downloading the network model file and the initial weight file; each client uses its own pictures and corresponding marking files to train the target detection network, and saves the final model; Step S3: Communication test, the server sends a signal to each client, and the received client sends a signal to the server to indicate that it is online. The server determines how many clients that can work normally can participate in the training according to the reply signal of the client; Step S4: Estimate important indicators. Each client calculates the estimated transmission time for this round based on historical transmission time and training time, combined with current communication conditions and computing power conditions. and calculation time 3. The surface defect detection model training method based on federated learning according to claim 2, characterized in that: The step five client defect detection method is to first take a picture of the product surface; then use the target detection model to obtain the area to be inspected in the picture and remove irrelevant background; then use the image classification model to judge the area to be inspected and obtain the final detection result.
4. The surface defect detection model training method based on federated learning according to claim 3, characterized in that: In step N2, the specific process of using the federated learning training model can be mainly divided into the following steps: the first step is to establish a connection, the server will test the communication status with each client, and select the online available nodes to participate in this round of training; The second step is initialization. The server initializes the model and sends the initial model parameters to each client node participating in the training. The third step is local training. After receiving the initial model parameters, the client uses local samples to train the model and sends the parameters obtained in this stage to the server. The fourth step is parameter aggregation. The server obtains the model parameters of each client node participating in this round of training, aggregates the parameters using an aggregation algorithm, and updates the overall model. Finally, the overall model parameters are distributed to each client node. The above steps are repeated until the overall model reaches the preset state, and the training is completed after exiting the loop.