A distributed scrap steel detection method and system based on hierarchical fine-grained fusion federated learning

Abstract

The application provides a distributed scrap steel detection method and system based on hierarchical fine-grained fusion federated learning, and relates to the field of metallurgy; the method comprises the following steps: receiving local model parameters uploaded by each client; according to the difference of the uploaded local model parameters, adopting a hierarchical fine-grained fusion weight setting mode to fuse the local model parameters of the clients, obtaining global fusion model parameters; broadcasting the global fusion model parameters to all clients, so that each client performs a new round of iterative training according to the global fusion model parameters until the local model reaches the set expected model precision, and then using the trained local model as a local scrap steel detection model to perform scrap steel detection. The application performs distributed scrap steel detection and grading by means of federated learning, protects the data privacy and security of each unloading point, and the fine-grained fusion weight setting can improve the detection capability of heterogeneous scrap steel data of different unloading points.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of metallurgy, in particular to a distributed scrap steel detection method and system based on hierarchical fine-grained fusion federated learning. BACKGROUND

[0002] Scrap steel is an important raw material for electric arc furnace smelting, with the advantages of small pollution, high yield and renewable. Because scrap steel comes from a wide range of sources and has uneven quality, the quality of scrap steel affects the output of steel, so scrap steel detection plays an important role in the steel smelting process.

[0003] Early scrap steel detection was achieved by manual means, which had the disadvantages of high risk, strong subjectivity and low efficiency. With the development of the industrial internet, artificial intelligence technology has been introduced into the industrial system, and scrap steel detection methods based on artificial intelligence have gradually developed; this method captures high-definition pictures of scrap steel by installing a shooting device at the unloading point, uploads them to the platform server, and trains a deep learning model to achieve remote intelligent detection of scrap steel; however, through browsing analysis and comparison, it is found that the existing intelligent scrap steel detection method has the following problems and defects: 1) data island problem, specifically: each steel plant or even each unloading point conducts scrap steel detection independently, and the data is not shared, forming an island, and the detection process is not transparent; 2) data privacy problem, specifically: scrap steel is an important raw material for steel smelting, and has privacy, uploading scrap steel data through wireless networks may leak data privacy; 3) transmission cost problem, specifically: the existing intelligent scrap steel detection method needs to upload high-definition pictures of scrap steel through wireless networks, which has high transmission cost and unstable connection.

[0004] In the prior art, a feasible method is to conduct distributed scrap steel detection through federated learning, allowing users to save data locally, share model parameters trained on local data, instead of sharing data, protecting data privacy, and reducing the communication cost of transmitting picture data. However, in the distributed scrap steel detection scenario, the types and quantities of scrap steel at each unloading point differ, causing data heterogeneity problems, and traditional federated learning is not sufficient to adapt to heterogeneous data. Therefore, there is an urgent need to build a distributed scrap steel detection system with privacy protection capabilities and the ability to adapt to heterogeneous data. SUMMARY

[0005] The present application aims to provide a distributed scrap steel detection method and system based on hierarchical fine-grained fusion federated learning, which updates global model parameters using a hierarchical fine-grained fusion weight setting method based on the differences in local model parameters of clients, solves the data island problem in scrap steel detection, not only has privacy protection capabilities for each unloading point, but also can adapt to heterogeneous data at each unloading point.

[0006] To achieve the above objectives, this invention proposes the following technical solution: a distributed scrap steel detection method based on hierarchical fine-grained fusion federated learning, applied to the server side of federated learning, comprising:

[0007] Receive local model parameters uploaded by each client, trained based on the local training dataset; where each client consists of each unloading point;

[0008] Based on the differences in the local model parameters uploaded by the client, a hierarchical fine-grained fusion weighting method is used to fuse the local model parameters of the client to obtain the global fused model parameters. The calculation process is as follows: both the global model on the server side and the local model on the client side are defined as deep learning models with L-layer neural networks, denoted as the model parameters of client k. The global model parameters on the server side are Furthermore, the model parameters of client k are divided by layer. The global model parameters are a set of row vectors. Global model parameters on the server side are divided into layers. The model parameters are a set of row vectors. ;in, , Let k be the total number of clients in the federated learning system; calculate the similarity between the model parameters of client k and the global model parameters of the server on the corresponding layer row vectors based on cosine similarity. : Calculate the similarity between each client. The proportion of the total similarity of all clients at this layer And use this ratio as the local model parameter of the client in the first... Layer fusion weights: The model parameters of each layer on each client are weighted and averaged to obtain the fusion model parameters of the server at the corresponding layer. : This serves as the global model parameter for the server-side in a new round of iterative training at this layer; wherein, the server-side establishes a global model with the same structure as the local model of each client, and the initial global model parameters of the global model are randomly generated;

[0009] The global fusion model parameters are broadcast to all clients so that each client can perform a new round of iterative training based on the global fusion model parameters until the local model of each client reaches the set expected model accuracy. Then, the local model trained by each client is used as its local scrap steel detection model for scrap steel detection.

[0010] Furthermore, the specific process for obtaining the global fusion model parameters is as follows:

[0011] The global model parameters of the global model and the obtained local model parameters uploaded by each client are divided into row vector sets;

[0012] The similarity of the local model parameters of each client and the global model parameters of the server at each layer row vector is calculated in sequence, and the weight of the local model parameter fusion of any client is taken as the proportion of the similarity of the client at the layer to the total similarity of all clients at the layer;

[0013] The local model parameters of each client are weighted and averaged at each layer to calculate the fusion model parameters of each layer;

[0014] The fusion model parameters of each layer are counted and summarized to obtain the global fusion model parameters.

[0015] Further, the process of updating the model parameters of the local model according to the local training data set is:

[0016] Obtain the high-definition pictures of the scrap steel taken by each unloading point;

[0017] The high-definition pictures of the scrap steel of each unloading point are preprocessed to generate a local training data set for each unloading point; wherein the preprocessing process includes removing background information from the high-definition pictures of the scrap steel, converting the picture data into vector data composed of numbers, and adding labels to the scrap steel data according to their actual scrap steel categories; the scrap steel categories include unified scrap steel, qualified scrap steel, fine scrap steel, heavy scrap steel, light and thin material, and miscellaneous iron;

[0018] The vector data of the high-definition pictures of the scrap steel after preprocessing is taken as the input of the local model, and the model output value is obtained through the calculation of the model parameters;

[0019] The error between the calculated model output value and the label added to the scrap steel data is compared, and the gradient data is obtained by deriving each parameter of the local model through the chain rule;

[0020] The model parameters of the local model are updated according to the gradient data.

[0021] The application also provides a distributed scrap steel detection method based on hierarchical fine-grained fusion federated learning, which is applied to any client of the federated learning, comprising:

[0022] Obtain the local training data set, and train the model parameters of the local model according to the local training data set; wherein the client is composed of unloading points;

[0023] The trained local model parameters are uploaded to the server. The server then uses a hierarchical, fine-grained weighting method to fuse the client's local model parameters based on the differences between the uploaded parameters, obtaining globally fused model parameters, which are then broadcast. The calculation process is as follows: both the server's global model and the client's local model are defined as deep learning models with L layers of neural networks, denoted as client-side model parameters k. The global model parameters on the server side are Furthermore, the model parameters of client k are divided by layer. The global model parameters are a set of row vectors. Global model parameters on the server side are divided into layers. The model parameters are a set of row vectors. ;in, , Let k be the total number of clients in the federated learning system; calculate the similarity between the model parameters of client k and the global model parameters of the server on the corresponding layer row vectors based on cosine similarity. : Calculate the similarity between each client. The proportion of the total similarity of all clients at this layer And use this ratio as the local model parameter of the client in the first... Layer fusion weights: The model parameters of each layer on each client are weighted and averaged to obtain the fusion model parameters of the server at the corresponding layer. : This serves as the global model parameter for the server-side in a new round of iterative training at this layer; wherein, the server-side establishes a global model with the same structure as the local model of each client, and the initial global model parameters of the global model are randomly generated;

[0024] Receive the broadcast global fusion model parameters and perform a new round of iterative training based on the global fusion model parameters until the local model reaches the set expected model accuracy;

[0025] The local model, which has been trained iteratively, is used as the local scrap steel detection model for scrap steel detection.

[0026] Another technical solution of the present invention discloses a distributed scrap steel detection system based on hierarchical fine-grained fusion federated learning, including a server and several clients; it also includes:

[0027] The receiving module is used to receive the parameters of the local model trained on the local training dataset uploaded by each client; where each client is composed of each unloading point.

[0028] The parameter fusion module is used to fuse the local model parameters uploaded by the client based on the differences in the local model parameters, using a hierarchical fine-grained fusion weight setting method to obtain the global fused model parameters. The calculation process is as follows: both the global model on the server side and the local model on the client side are defined as deep learning models with L-layer neural networks, denoted as the model parameters of client k. The global model parameters on the server side are Furthermore, the model parameters of client k are divided by layer. The global model parameters are a set of row vectors. Global model parameters on the server side are divided into layers. The model parameters are a set of row vectors. ;in, , Let k be the total number of clients in the federated learning system; calculate the similarity between the model parameters of client k and the global model parameters of the server on the corresponding layer row vectors based on cosine similarity. : Calculate the similarity between each client. The proportion of the total similarity of all clients at this layer And use this ratio as the local model parameter of the client in the first... Layer fusion weights: The model parameters of each layer on each client are weighted and averaged to obtain the fusion model parameters of the server at the corresponding layer. : This serves as the global model parameter for the server-side in a new round of iterative training at this layer; wherein, the server-side establishes a global model with the same structure as the local model of each client, and the initial global model parameters of the global model are randomly generated;

[0029] The broadcast update module is used to broadcast the global fusion model parameters to all clients, so that each client can perform a new round of iterative training based on the global fusion model parameters until the local model of each client reaches the set expected model accuracy, and then use the local model trained by each client as its local scrap steel detection model for scrap steel detection.

[0030] Furthermore, the execution unit for obtaining global fusion model parameters by the parameter fusion module includes:

[0031] The partitioning unit is used to hierarchically divide the global model parameters of the global model and the local model parameters uploaded by each client into a set of row vectors;

[0032] The first calculation unit is configured to sequentially calculate the similarity of the local model parameters of each client and the global model parameters of the server in each layer row vector, and take the proportion of the similarity of any client in the sum of the similarities of all clients in the layer as the weight of the local model parameter fusion of the client;

[0033] The second calculation unit is configured to perform weighted averaging on the local model parameters of each client in each layer to obtain the fusion model parameters of each layer;

[0034] The statistical unit is configured to statistically and summarize the fusion model parameters of each layer to obtain the global fusion model parameters.

[0035] Further, the parameter fusion module adopts a hierarchical fine-grained fusion weight setting method to perform a fusion calculation process on the local model parameters of the client:

[0036] The global model of the server and the local model of the client are defined as a deep learning model with L layers of neural networks, and are denoted as the model parameters of the client k and the global model parameters of the server ; and the model parameters of the client k are divided by layers, and the global model parameters of the client k are divided by layers, and the model parameters of the server are divided by layers; wherein , , is the total number of clients in the federated learning system;

[0037] The similarity of the model parameters of the client k and the global model parameters of the server in the corresponding layer row vector is calculated according to the cosine similarity : ;

[0038] The proportion of the similarity of each client in the sum of the similarities of all clients in the layer is calculated, and the proportion is taken as the fusion weight of the local model parameters of the client in the first layer: ;

[0039] The model parameters of each layer of each client are weighted and averaged according to the weight to obtain the fusion model parameters of the server in the corresponding layer : , and the fusion model parameters are taken as the global model parameters of the server in the layer for the next round of iterative training.

[0040] This invention also discloses another distributed scrap steel detection system based on hierarchical fine-grained fusion federated learning, including a server and several clients; it also includes:

[0041] The local training module is used to acquire the local training dataset and train the model parameters of the local model based on the local training dataset; the client consists of unloading points.

[0042] The upload module is used to upload the trained local model parameters to the server. The server then uses a hierarchical, fine-grained weighting method to fuse the client's local model parameters based on the differences between the uploaded parameters, obtaining a global fused model parameter, which is then broadcast. Both the server-side global model and the client's local model are defined as deep learning models with L layers of neural networks, denoted as client-side model parameters k. The global model parameters on the server side are Furthermore, the model parameters of client k are divided by layer. The global model parameters are a set of row vectors. Global model parameters on the server side are divided into layers. The model parameters are a set of row vectors. ;in, , Let k be the total number of clients in the federated learning system; calculate the similarity between the model parameters of client k and the global model parameters of the server on the corresponding layer row vectors based on cosine similarity. : Calculate the similarity between each client. The proportion of the total similarity of all clients at this layer And use this ratio as the local model parameter of the client in the first... Layer fusion weights: The model parameters of each layer on each client are weighted and averaged to obtain the fusion model parameters of the server at the corresponding layer. : This serves as the global model parameter for the server-side in a new round of iterative training at this layer; wherein, the server-side establishes a global model with the same structure as the local model of each client, and the initial global model parameters of the global model are randomly generated;

[0043] The update receiving module is used to receive the broadcast global fusion model parameters and perform a new round of iterative training based on the global fusion model parameters until the local model reaches the set expected model accuracy.

[0044] The detection module is used to detect scrap steel using the local model that has been trained iteratively.

[0045] Still another technical scheme of the present application discloses an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the computer program is executed by the processor to implement the distributed scrap steel detection method based on hierarchical fine-grained fusion federated learning.

[0046] From the above technical scheme, the present application has the following beneficial effects:

[0047] The distributed scrap steel detection method and system based on hierarchical fine-grained fusion federated learning disclosed by the present application comprises the following steps: receiving local model parameters of a local model uploaded by each client according to local training data sets; wherein each client is composed of each unloading point; according to the difference of the local model parameters uploaded by the client, the local model parameters of the client are fused by using a hierarchical fine-grained fusion weight setting method to obtain global fusion model parameters; wherein the server end establishes a global model with the same local model structure as each client, and the initialization global model parameters of the global model are randomly generated; the global fusion model parameters are broadcasted to all clients, so that each client performs a new round of iterative training according to the global fusion model parameters until the local model of each client reaches the set expected model accuracy, and then the local model corresponding to each client after the training is completed is used as the local scrap steel detection model for scrap steel detection. The present application can jointly train a scrap steel detection model by means of the federated learning framework and a plurality of unloading points, and solve the data island problem of scrap steel detection.

[0048] Meanwhile, the present application uses federated learning to upload model parameters instead of image data, which can protect the scrap steel data privacy of the unloading point and reduce the communication cost of data transmission; the hierarchical fine-grained fusion of the global model parameters of the server end can calculate the fusion weight according to the data distribution difference of the client, and effectively alleviate the heterogeneity problem of scrap steel data in a distributed environment.

[0049] It should be understood that all combinations of the aforementioned concepts and additional concepts described below (provided such concepts are not mutually inconsistent) are contemplated as being part of the inventive subject matter.

[0050] The foregoing and other aspects, embodiments, and features of the present teachings can be better understood from the following detailed description taken in conjunction with the accompanying drawings. Other aspects, embodiments, and features of the present teachings will be apparent from consideration of the specification and practice of the present teachings as disclosed herein. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present teachings, as claimed. BRIEF DESCRIPTION OF DRAWINGS

[0051] The drawings are not drawn to true scale. In the drawings, like reference numerals can be used to denote like parts throughout the various views. For the sake of clarity, not every component can be labeled in every drawing. Embodiments of various aspects of the application will now be described, by way of example only, with reference to the drawings in which:

[0052] Figure 1 The flow chart of the distributed scrap steel detection method based on hierarchical fine-grained fusion federated learning disclosed for the first embodiment of the application;

[0053] Figure 2 The flow chart of the distributed scrap steel detection method based on hierarchical fine-grained fusion federated learning disclosed for the second embodiment of the application;

[0054] Figure 3 The overall flow chart of the distributed scrap steel detection method based on hierarchical fine-grained fusion federated learning of the application;

[0055] Figure 4 The schematic diagram of the distributed scrap steel detection system based on hierarchical fine-grained fusion federated learning of the application.

[0056] In the drawings, the specific meanings of the respective labels are as follows:

[0057] 1 - cloud server, 2 - client, 3 - local training data set, 4 - local model, 5 - global model. DETAILED DESCRIPTION

[0058] In order to make the objects, technical solutions and advantages of the embodiments of the application clearer, the technical solutions of the embodiments of the application will be described clearly and completely below with reference to the drawings of the embodiments of the application. Obviously, the described embodiments are part of the embodiments of the application, rather than all the embodiments of the application. Based on the described embodiments of the application, all other embodiments obtained by those of ordinary skill in the art without any creative effort fall within the scope of protection of the application. Unless otherwise defined, the technical terms or scientific terms used herein should be understood as the usual meanings understood by those of ordinary skill in the art to which the application belongs.

[0059] The terms "first", "second", and similar terms used in the specification and claims of the present patent application do not denote any order, quantity, or importance, but are used to distinguish different components. Similarly, the singular forms "a", "an", and "the" do not denote a quantity limitation, but denote the presence of at least one, unless the context clearly indicates otherwise. The terms "comprise", "comprising", and similar terms mean that the elements or objects preceding the "comprise" or "comprising" are encompassed by the "comprise" or "comprising" that follows, and do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. The terms "upper", "lower", "left", "right", and the like are used only to indicate relative positional relationships, which may change accordingly when the absolute positions of the described objects change.

[0060] For the method of detecting scrap steel by the conventional federated learning system in the prior art, although the technical effects of protecting data privacy and reducing the communication cost of transmitting picture data can be achieved, in the scrap steel detection scene, due to the differences in the types and quantities of scrap steel at each unloading point, there is a data heterogeneity phenomenon, and the conventional federated learning has poor adaptability to heterogeneous data; the present application aims to disclose a distributed scrap steel detection method and system based on hierarchical fine-grained fusion federated learning, which uses a hierarchical fine-grained fusion weight setting method to update the global model parameters to solve the problem of local model parameter distribution difference of the client.

[0061] The distributed scrap steel detection method and system based on hierarchical fine-grained fusion federated learning disclosed by the present application will be further specifically introduced below in combination with the specific embodiments shown in the accompanying drawings.

[0062] The distributed scrap steel detection method based on hierarchical fine-grained fusion federated learning disclosed in Embodiment One and Embodiment Two is respectively applied to the server side and the client side of the federated learning. The embodiments also disclose a distributed scrap steel detection system based on hierarchical fine-grained fusion federated learning, which, as shown in Figure 4 the drawings, includes a cloud server 1 and a plurality of clients 2, the clients 2 and the cloud server 1 establish communication according to a wireless network; wherein the cloud server 1 constitutes the server side of the federated learning; in order to facilitate the implementation of the distributed scrap steel detection method based on hierarchical fine-grained fusion federated learning, the clients 2 are established with local training data sets 3 and local models 4, and the cloud server 1 is established with a global model 5.

[0063] In combination with Figure 1 the drawings, the distributed scrap steel detection method based on hierarchical fine-grained fusion federated learning applied to the server side of the federated learning disclosed in Embodiment One includes the following steps:

[0064] Step S102, receiving the local model parameters trained by the local model 4 of each client according to the local training data set 3; wherein each client 2 is composed of each unloading point;

[0065] Optionally, the process of updating the model parameters by the local model 4 of the client 2 according to the local training data set 3 is as follows: obtaining the high-definition pictures of the scrap steel taken by each unloading point; preprocessing the high-definition pictures of the scrap steel of each unloading point to generate the local training data set 3 of each unloading point; wherein the preprocessing process includes removing the background information in the high-definition pictures of the scrap steel, converting the picture data into vector data composed of numbers, and adding labels to the scrap steel data according to the actual scrap steel categories; the scrap steel categories include unified scrap steel, qualified scrap steel, fine scrap steel, heavy scrap steel, light and thin material, and miscellaneous iron; taking the vector data of the preprocessed high-definition pictures of the scrap steel as the input of the local model 4, and obtaining the model output value through the calculation of the model parameters; comparing the error between the calculated model output value and the label added to the scrap steel data, and deriving each parameter of the local model 4 through the chain rule to obtain gradient data; updating the model parameters of the local model 4 according to the gradient data. The result of the training of the local model 4 of the client 2 is the classification data of the scrap steel according to the high-definition pictures of the scrap steel of the unloading point.

[0066] Wherein, the way of taking the high-definition pictures of the scrap steel can be to place a high-definition camera above the unloading point and aim at the carriage, and to take multiple high-definition pictures of the scrap steel during the unloading process, and the specific number of the taken high-definition pictures is set by the human. In addition, the local model 4 can select commonly used multilayer perception, multilayer neural network model, convolutional neural network model, etc., and randomly give the defined model with initialized model parameters in a normal distribution or other ways. The model parameters updated by the client 2 are uploaded to the cloud server 1 through the wireless network.

[0067] Step S104, according to the difference of the local model parameters uploaded by the client 2, using the weight setting method of hierarchical fine-grained fusion to fuse the local model parameters of the client 2, and obtaining the global fusion model parameters; wherein the server end establishes a global model 5 with the same structure as the local model 4 of each client 2, and the initialized global model parameters of the global model 5 are randomly generated, such as the normal distribution or other ways mentioned above.

[0068] The difference of the local model parameters uploaded by the client 2 is caused by the distribution difference of the local data of the client 2.

[0069] Specifically, the specific process of obtaining the global fusion model parameters is as follows: the global model parameters of the global model and the obtained local model parameters uploaded by each client 2 are divided into a set of row vectors; the similarity of the local model parameters of each client 2 and the global model parameters of the server side at each layer of row vectors is calculated in turn, and the proportion of the similarity of any client 2 to the total similarity of all clients 2 at the layer is taken as the weight of the local model parameter fusion of the client 2; the local model parameters of each client 2 at each layer are weighted and averaged to calculate the fusion model parameters of each layer; the fusion model parameters of each layer are counted and summarized to obtain the global fusion model parameters. In the above process, the similarity is used as the quantification of the difference in local data distribution of the client 2, and the data heterogeneity between the local data of each client 2 is represented.

[0070] Step S106, broadcast the global fusion model parameters to all clients, so that each client performs a new round of iterative training according to the global fusion model parameters until the local model of each client reaches the set expected model accuracy, and then uses the local model corresponding to the completed training as the local scrap steel detection model for scrap steel detection.

[0071] In combination Figure 2 As shown in FIG. 2, the distributed scrap steel detection method based on hierarchical fine-grained fusion federated learning applied to any client 2 of federated learning disclosed in Embodiment Two specifically includes the following steps:

[0072] Step S1102, obtain the local training data set 3, and train the model parameters of the local model 4 according to the local training data set 3; wherein the client 2 is composed of a discharge point;

[0073] Step S1104, upload the trained local model parameters to the server side, so that the server side fuses the local model parameters of the client 2 according to the difference between the local model parameters uploaded by the client 2, adopts the weight setting mode of hierarchical fine-grained fusion, obtains the global fusion model parameters, and broadcasts; wherein the server side establishes a global model with the same structure as the local model 4 of each client 2, and the initialization global model parameters of the global model 5 are randomly generated;

[0074] Step S1106, receive the broadcasted global fusion model parameters, and perform a new round of iterative training according to the global fusion model parameters until the local model 4 reaches the set expected model accuracy;

[0075] Step S1108, use the local model 4 completed by iterative training as a local scrap steel detection model for scrap steel detection.

[0076] The distributed scrap steel detection method based on hierarchical fine-grained fusion federated learning disclosed in Embodiment One and Embodiment Two is summarized as follows: the implementation process includes Figure 3The steps shown are: taking pictures of scrap steel, data preprocessing, local model training, server calculating similarity between client models, server calculating fusion weights of client models, server fusing client models, judging whether the model has reached the preset accuracy and looping. By calculating the similarity of client 2 model parameters and fusion weights on the server side, the model parameters of the server-side model and client 2 model in the federated learning process are trained and updated. The model solves the data heterogeneity problem of local data of each client 2 by using the weights of each client 2.

[0077] The distributed scrap steel detection method based on hierarchical fine-grained fusion federated learning disclosed in this invention further includes: defining the mathematical problem for hierarchical fine-grained fusion federated learning, specifically:

[0078] Assume the federated learning system consists of client 2, which comprises K unloading points, and client k( The local training dataset is Client k( The global model parameters established on the server side are: Where the j-th sample is input into the global model parameters on the server side. The loss is Then the overall loss of client k is ;

[0079] The federated learning optimization problem can be obtained by weighted averaging the losses of all clients: ;

[0080] in, For the amount of scrap steel data of client k, Let F represent the total amount of scrap steel data for all clients, and let F represent the overall loss of all clients 2.

[0081] As an optional implementation, based on the defined mathematical problem of hierarchical fine-grained fusion federated learning, the calculation process of fusing the local model parameters of client 2 using the hierarchical fine-grained fusion weight setting method in the above method is as follows:

[0082] Both the global model 5 on the server side and the local model 4 on the client side are defined as deep learning models with L layers of neural networks, denoted as the model parameters of client k. The global model parameters on the server side are Furthermore, the model parameters of client k are divided by layer. The global model parameters are a set of row vectors. Global model parameters on the server side are divided into layers. The model parameters are a set of row vectors. ;in, , Total number of clients 2 in the federated learning system;

[0083] Calculate the similarity of the model parameters of client k and the global model parameters of the server in the corresponding layer row vector according to the cosine similarity

[0084] Calculate the similarity of each client 2 The proportion of the sum of the similarities of all clients 2 in the layer to the proportion of the local model parameters of the client 2 in the first layer The fusion weight of the local model parameters of the client 2 in the first layer is:

[0085] According to the weight, the model parameters of each layer of each client 2 are weighted and averaged to obtain the fusion model parameters of the server in the corresponding layer And as the global model parameters of the server in the next round of iterative training in the layer.

[0086] The overall process of the distributed scrap steel detection method of the application includes: local processing of scrap steel unloading site image data and data set; local establishment of a scrap steel detection model at the unloading point, and training on its local data set; uploading the trained local model parameters to the cloud server 1 through the wireless network; the cloud server 1 performs fine-grained fusion on the local model parameters uploaded by multiple unloading points; the fusion weight is set in layers according to the distribution difference of the scrap steel data of the unloading points; the cloud server 1 downloads the model parameters after fusion to the unloading point to update the parameters of the local model 4, and the iteration process is repeated until the expected accuracy of the model is reached; the application can perform distributed scrap steel detection and classification by means of federated learning, protect the data privacy and security of each unloading point, reduce the data transmission cost, and the fine-grained fusion weight setting can improve the detection effect of heterogeneous scrap steel data of different unloading points.

[0087] The embodiment of the application also provides an electronic device, a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the computer program is executed by the processor to implement the distributed scrap steel detection method based on hierarchical fine-grained fusion federated learning disclosed by the above-mentioned embodiment.

[0088] ​​​​​The above programs can be run in a processor, or can also be stored in a memory (or called computer readable storage medium), the computer readable medium includes permanent and non-permanent, removable and non-removable media, which can realize information storage by any method or technology. Information can be computer readable instructions, data structure, program module or other data. Examples of computer storage medium include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device or any other non-transmission medium that can be used to store information accessible by a computing device. According to the definition in this paper, computer readable medium does not include transitory computer readable medium (transitory medium), such as modulated data signal and carrier wave.

[0089] These computer programs can also be loaded into a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate computer implemented processing, so that the instructions executed on the computer or other programmable device provide a process for implementing the functions specified in the flow Figure 1 The flow or multiple flows and / or the functions specified in the block Figure 1 The steps of the flow or multiple flows and / or the functions specified in the block

[0090] An embodiment of the present application provides such a system, which is called distributed scrap steel detection system based on hierarchical fine-grained fusion federated learning. In addition to the server and the plurality of clients 2, the system further comprises: a receiving module, configured to receive local model parameters of a local model uploaded by each client according to a local training data set 3; wherein each client 2 is composed of each unloading point; a parameter fusion module, configured to fuse the local model parameters of the client 2 according to the difference between the local model parameters uploaded by the client, and obtain global fusion model parameters by using a hierarchical fine-grained fusion weight setting mode; wherein the server end establishes a global model with the same local model structure as each client, and the initialization global model parameters of the global model are randomly generated; a broadcast update module, configured to broadcast the global fusion model parameters to all clients, so that each client performs a new round of iterative training according to the global fusion model parameters until the local model of each client reaches the set expected model precision, and then uses the local model corresponding to each client as its local scrap steel detection model to perform scrap steel detection.

[0091] This system is used to implement the function of the method in Embodiment 1 above. Each module in the system corresponds to each step in the method, which has already been described in the method and will not be repeated here.

[0092] For example, the execution unit for obtaining global fusion model parameters in the parameter fusion module includes: a partitioning unit, used to partition the global model parameters and the local model parameters uploaded by each client into a set of row vectors; a first calculation unit, used to calculate the similarity between the local model parameters of each client and the global model parameters of the server at each layer of row vectors, and use the proportion of the similarity of any client to the total similarity of all clients at that layer as the weight when fusing the local model parameters of that client; a second calculation unit, used to perform a weighted average of the local model parameters of each client at each layer to calculate the fusion model parameters of each layer; and a statistics unit, used to count and summarize the fusion model parameters of each layer to obtain the global fusion model parameters.

[0093] For example, the parameter fusion module uses a hierarchical, fine-grained fusion weight setting method to fuse the client's local model parameters. The calculation process is as follows: both the server-side global model and the client-side local model are defined as deep learning models with L-layer neural networks, denoted as the client-side model parameters k, respectively. The global model parameters on the server side are Furthermore, the model parameters of client k are divided by layer. The global model parameters are a set of row vectors. Global model parameters on the server side are divided into layers. The model parameters are a set of row vectors. ;in, , Let k be the total number of clients in the federated learning system; calculate the similarity between the model parameters of client k and the global model parameters of the server on the corresponding layer row vectors based on cosine similarity. : Calculate the similarity between each client. The proportion of the total similarity of all clients at this layer And use this ratio as the local model parameter for client 2 in the 1st... Layer fusion weights: The model parameters of each layer on each client are weighted and averaged to obtain the fusion model parameters of the server at the corresponding layer. : This serves as the global model parameter for the server-side in the next iteration of training at this layer.

[0094] Another embodiment of the present application provides a distributed scrap steel detection system based on hierarchical fine-grained fusion federated learning, which is used to realize the functions of the method in the above-mentioned embodiment two. Each module in the system corresponds to each step in the method. In addition to the server and several clients 2, the system also includes: a local training module for obtaining a local training data set 3 and training model parameters of a local model 4 according to the local training data set 3; wherein the client 2 is composed of a discharge point; an uploading module for uploading the trained local model parameters to the server end, so that the server end fuses the local model parameters of the client 2 according to the differences in the local model parameters uploaded by the client 2, obtains global fusion model parameters and broadcasts them; wherein the server end establishes a global model 5 with the same local model structure as each client 2, and the initialization global model parameters of the global model 5 are randomly generated; a receiving updating module for receiving the broadcasted global fusion model parameters and performing a new round of iterative training according to the global fusion model parameters until the local model 4 reaches the set expected model accuracy; a detection module for using the local model 4 that has completed the iterative training as a local scrap steel detection model to detect scrap steel.

[0095] The distributed scrap steel detection method and system based on hierarchical fine-grained fusion federated learning disclosed by the present application aims to solve the technical problem of poor adaptability when the traditional federated learning system detects scrap steel, although it can achieve the technical effects of protecting data privacy and reducing the communication cost of transmitting picture data. Specifically, when the client 2 and the server end of the federated learning perform iterative training, the server end sets layers according to the distribution differences of the scrap steel data of each client 2, achieves fine-grained fusion weight of the local model parameters uploaded by multiple clients 2, and then obtains the fused model parameters according to the fusion weight and issues them to the client 2 to update the model parameters of the local model 4, until the model reaches the expected accuracy. Finally, the updated local model 4 of the client 2 or the updated global model 5 of the server end is used as a scrap steel detection model, which is applied and realized for scrap steel grading detection.

[0096] Although the present application has been disclosed as above with reference to the preferred embodiments, it is not intended to limit the present application. Those skilled in the art can make various modifications and improvements without departing from the spirit and scope of the present application. Therefore, the protection scope of the present application shall be subject to the scope defined by the claims.

Claims

1. A distributed scrap steel detection method based on hierarchical fine-grained fusion federated learning, characterized in that, Server-side components used in federated learning include: Receive local model parameters uploaded by each client, trained based on the local training dataset; where each client consists of each unloading point; Based on the differences in local model parameters uploaded by the client, a hierarchical fine-grained fusion weighting method is used to fuse the client's local model parameters to obtain global fused model parameters. The calculation process is as follows: both the server-side global model and the client's local model are defined as having... L The deep learning model of the layered neural network is denoted as client. k Model parameters The global model parameters on the server side are Furthermore, clients are divided into layers. k Model parameters The global model parameters are a set of row vectors. Global model parameters on the server side are divided into layers. The model parameters are a set of row vectors. ;in, , , The total number of clients in the federated learning system; clients are calculated based on cosine similarity. k The similarity between the model parameters and the global model parameters on the server side in the corresponding layer row vectors. : Calculate the similarity between each client. The proportion of the total similarity of all clients at this layer And use this ratio as the local model parameter of the client in the first... l Layer fusion weights: The model parameters of each layer on each client are weighted and averaged to obtain the fusion model parameters of the server at the corresponding layer. : This serves as the global model parameter for the server-side's new round of iterative training at this layer; wherein, the server-side establishes a global model with the same structure as the local model of each client, and the initial global model parameters of the global model are randomly generated; The global fusion model parameters are broadcast to all clients so that each client can perform a new round of iterative training based on the global fusion model parameters until the local model of each client reaches the set expected model accuracy. Then, the local model trained by each client is used as its local scrap steel detection model for scrap steel detection.

2. The distributed scrap steel detection method based on hierarchical fine-grained fusion federated learning according to claim 1, characterized in that, The specific process for obtaining the global fusion model parameters is as follows: The global model parameters of the global model and the local model parameters uploaded by each client are divided into a set of row vectors. The similarity between the local model parameters of each client and the global model parameters of the server is calculated sequentially at each layer of row vectors. The proportion of the similarity of any client to the total similarity of all clients at that layer is used as the weight when fusing the local model parameters of that client. The local model parameters of each client are weighted and averaged at each layer to calculate the fusion model parameters for each layer. The parameters of the fusion model at each layer are statistically analyzed and summarized to obtain the global fusion model parameters.

3. The distributed scrap steel detection method based on hierarchical fine-grained fusion federated learning according to claim 1, characterized in that, The process of updating model parameters based on the local training dataset on the client side is as follows: Obtain high-resolution images of scrap steel taken locally at each unloading point; High-resolution images of scrap steel at each unloading point are preprocessed to generate local training datasets for each unloading point. The preprocessing process includes removing background information from the high-resolution images of scrap steel, converting the image data into vector data composed of numbers, and adding labels to the scrap steel data according to its actual scrap steel category. The scrap steel categories include mixed scrap steel, qualified scrap steel, refined scrap steel, heavy scrap steel, light and thin scrap, and miscellaneous iron. The vector data obtained after preprocessing high-resolution images of scrap steel is used as the input of the local model, and the model output value is obtained through the calculation of model parameters. The error between the output value of the computational model and the label attached to the scrap steel data is compared, and the gradient data is obtained by differentiating each parameter of the local model using the chain rule; The model parameters of the local model are updated based on the gradient data.

4. A distributed scrap steel detection method based on hierarchical fine-grained fusion federated learning, characterized in that, Any client used for federated learning, including: Obtain the local training dataset and train the model parameters of the local model based on the local training dataset; the client consists of unloading points. The trained local model parameters are uploaded to the server. The server then uses a hierarchical, fine-grained weighting method to fuse the client's local model parameters based on the differences in the uploaded parameters, obtaining globally fused model parameters, which are then broadcast. The calculation process involves defining both the server-side global model and the client's local model as having... L The deep learning model of the layered neural network is denoted as client. k Model parameters The global model parameters on the server side are Furthermore, clients are divided into layers. k Model parameters The global model parameters are a set of row vectors. Global model parameters on the server side are divided into layers. The model parameters are a set of row vectors. ;in, , , The total number of clients in the federated learning system; clients are calculated based on cosine similarity. k The similarity between the model parameters and the global model parameters on the server side in the corresponding layer row vectors. : Calculate the similarity between each client. The proportion of the total similarity of all clients at this layer And use this ratio as the local model parameter of the client in the first... l Layer fusion weights: The model parameters of each layer on each client are weighted and averaged to obtain the fusion model parameters of the server at the corresponding layer. : This serves as the global model parameter for the server-side's new round of iterative training at this layer; wherein, the server-side establishes a global model with the same structure as the local model of each client, and the initial global model parameters of the global model are randomly generated; Receive the broadcast global fusion model parameters and perform a new round of iterative training based on the global fusion model parameters until the local model reaches the set expected model accuracy; The local model, which has been trained iteratively, is used as the local scrap steel detection model for scrap steel detection.

5. A distributed scrap steel detection system based on hierarchical fine-grained fusion federated learning, comprising a server and several clients, characterized in that, Also includes: The receiving module is used to receive the parameters of the local model trained on the local training dataset uploaded by each client; where each client is composed of each unloading point. The parameter fusion module is used to fuse the local model parameters uploaded by the client based on the differences in the parameters, using a hierarchical, fine-grained weighting method to obtain the global fused model parameters. The calculation process involves defining both the server-side global model and the client-side local model as having... L The deep learning model of the layered neural network is denoted as client. k Model parameters The global model parameters on the server side are Furthermore, clients are divided into layers. k Model parameters The global model parameters are a set of row vectors. Global model parameters on the server side are divided into layers. The model parameters are a set of row vectors. ;in, , , The total number of clients in the federated learning system; clients are calculated based on cosine similarity. k The similarity between the model parameters and the global model parameters on the server side in the corresponding layer row vectors. : Calculate the similarity between each client. The proportion of the total similarity of all clients at this layer And use this ratio as the local model parameter of the client in the first... l Layer fusion weights: The model parameters of each layer on each client are weighted and averaged to obtain the fusion model parameters of the server at the corresponding layer. : This serves as the global model parameter for the server-side's new round of iterative training at this layer; wherein, the server-side establishes a global model with the same structure as the local model of each client, and the initial global model parameters of the global model are randomly generated; The broadcast update module is used to broadcast the global fusion model parameters to all clients, so that each client can perform a new round of iterative training based on the global fusion model parameters until the local model of each client reaches the set expected model accuracy, and then use the local model trained by each client as its local scrap steel detection model for scrap steel detection.

6. The distributed scrap steel detection system based on hierarchical fine-grained fusion federated learning according to claim 5, characterized in that, The parameter fusion module is an execution unit that obtains global fusion model parameters, including: The partitioning unit is used to hierarchically divide the global model parameters of the global model and the local model parameters uploaded by each client into a set of row vectors; The first calculation unit is used to calculate the similarity between the local model parameters of each client and the global model parameters of the server at each layer of row vectors in turn, and to use the proportion of the similarity of any client to the total similarity of all clients at that layer as the weight when fusing the local model parameters of that client. The second calculation unit is used to perform a weighted average of the local model parameters of each client at each layer to calculate the fusion model parameters of each layer. The statistics unit is used to collect and summarize the parameters of the fusion model at each layer to obtain the global fusion model parameters.

7. A distributed scrap steel detection system based on hierarchical fine-grained fusion federated learning, comprising a server and several clients, characterized in that, Also includes: The local training module is used to acquire the local training dataset and train the model parameters of the local model based on the local training dataset; the client consists of unloading points. The upload module is used to upload the trained local model parameters to the server. The server then uses a hierarchical, fine-grained weighting method to fuse the client's local model parameters based on the differences between the uploaded parameters, obtaining a globally fused model parameter, which is then broadcast. Both the server-side global model and the client's local model are defined as having... L The deep learning model of the layered neural network is denoted as client. k Model parameters The global model parameters on the server side are Furthermore, clients are divided into layers. k Model parameters The global model parameters are a set of row vectors. Global model parameters on the server side are divided into layers. The model parameters are a set of row vectors. ;in, , , The total number of clients in the federated learning system; clients are calculated based on cosine similarity. k The similarity between the model parameters and the global model parameters on the server side in the corresponding layer row vectors. : Calculate the similarity between each client. The proportion of the total similarity of all clients at this layer And use this ratio as the local model parameter of the client in the first... l Layer fusion weights: The model parameters of each layer on each client are weighted and averaged to obtain the fusion model parameters of the server at the corresponding layer. : This serves as the global model parameter for the server-side's new round of iterative training at this layer; wherein, the server-side establishes a global model with the same structure as the local model of each client, and the initial global model parameters of the global model are randomly generated; The update receiving module is used to receive the broadcast global fusion model parameters and perform a new round of iterative training based on the global fusion model parameters until the local model reaches the set expected model accuracy. The detection module is used to detect scrap steel using the local model that has been trained iteratively.

8. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the distributed scrap steel detection method based on hierarchical fine-grained fusion federated learning as described in any one of claims 1 to 4.

Images