Network structure search method, device, equipment, storage medium and program product

Abstract

The present invention discloses a network structure search method, apparatus, device, storage medium and program product. The method comprises selecting a partial network from the network to be searched that matches the computing resource size of the target participant; sending the partial network to the target participant so that the target participant can perform local training on the partial network to obtain local parameter update information and return it; globally fusing the local parameter update information received from each participant to obtain global parameter update information, and updating the parameters in the network to be searched according to the global parameter update information, wherein the parameters include model parameters and / or structural parameters. The present invention enables participants with fewer computing resources to participate in horizontal federated learning of large models, thereby obtaining models with higher prediction accuracy.

Description

Technical Field

[0001] The present invention relates to the field of federated learning technology, and in particular to a network structure search method, apparatus, device, storage medium, and program product. Background Art

[0002] With the development of artificial intelligence, the concept of "federated learning" has been proposed to address data silos. This allows federated parties to train models and obtain model parameters without sharing their own data, thus preventing data privacy leaks. Horizontal federated learning, also known as feature-aligned federated learning, aims to leverage the limited data of each party to jointly train a global model while ensuring data security. The models held by each party are identical to the global model.

[0003] Currently, for many machine learning tasks, larger models often have better performance. For example, for convolutional neural networks (CNNs), the deeper and wider the model, the higher the accuracy after sufficient training. However, the larger the model, the more computing resources are required during training.

[0004] However, in a horizontal federation scenario, each participant has different local computing resources. If one participant has fewer computing resources, they can only train small models, which limits the performance of the global model trained by federated learning. Therefore, how to jointly train a large global model when some participants have limited computing resources and cannot train a large model is an urgent problem to be solved. Summary of the Invention

[0005] The main purpose of the present invention is to provide a network structure search method, device, equipment, storage medium and program product, aiming to solve the problem of how to jointly train a global large model when some participants have insufficient computing resources and cannot perform large model training.

[0006] To achieve the above objectives, the present invention provides a network structure search method. The method is applied to a coordinator participating in horizontal federated learning. The coordinator is in communication with each participant participating in horizontal federated learning. The coordinator deploys a network to be searched. The method includes the following steps:

[0007] Selecting a portion of the network from the to-be-searched network that matches the computing resource size of the target participant;

[0008] Sending the partial network to the target participant so that the target participant can perform local training on the partial network to obtain local parameter update information and return it;

[0009] Global parameter update information is obtained by globally fusing local parameter update information received from each participant, and parameters in the network to be searched are updated according to the global parameter update information, wherein the parameters include model parameters and / or structural parameters.

[0010] Optionally, the step of selecting a portion of the network that matches the computing resource size of the target participant from the network to be searched includes:

[0011] Calculating, based on the computing resource information of the target participant, a target ratio of the model size that the target participant can accommodate to the model size of the network to be searched;

[0012] A partial network is randomly selected from the network to be searched according to the target ratio, wherein a ratio of a model size of the partial network to a model size of the network to be searched is less than or equal to the target ratio.

[0013] Optionally, the step of randomly selecting a portion of networks from the networks to be searched according to the target ratio includes:

[0014] For each node in the network to be searched, determining the number of retained edges corresponding to the node according to the target ratio, wherein the ratio of the number of retained edges to the total number of subsequent edges of the node in the network to be searched is less than or equal to the target ratio;

[0015] Randomly selecting a reserved back-connected edge from all back-connected edges of the node in the network to be searched, wherein the number of the reserved back-connected edges is the number of reserved edges;

[0016] The portion of the network to be searched connected by the retained subsequent edges corresponding to the nodes is used as a partial network.

[0017] Optionally, the step of randomly selecting a portion of networks from the networks to be searched according to the target ratio includes:

[0018] For each network module in the network to be searched, randomly selecting a retained operation from each parallel operation on a node connection edge in the network module to obtain a retained module connected by the retained operation, wherein a ratio of the operation computation amount of the retained module to the operation computation amount of the network module is less than or equal to the target ratio;

[0019] The network composed of the reserved modules is regarded as a partial network.

[0020] Optionally, the step of globally fusing the local parameter update information received from each participant to obtain global parameter update information includes:

[0021] Obtaining an information group corresponding to the parameters in the network to be searched from the local parameter update information sent by each participant, wherein the information group includes local parameter update values ​​sent by some of the participants, and the some participants are participants in some of the distributed networks that include the parameters;

[0022] Averaging the local parameter update values ​​in the information group to obtain a global parameter update value corresponding to the parameter;

[0023] The global parameter update value corresponding to each parameter in the network to be searched is used as global parameter update information.

[0024] To achieve the above objectives, the present invention provides an image classification method, which is applied to a coordinator participating in horizontal federated learning, the coordinator being in communication with each participant in horizontal federated learning, and the coordinator deploying a network to be searched. The method includes:

[0025] Conducting multiple rounds of network structure search on the network to be searched in conjunction with each of the participants to obtain a target image classification model, wherein in each round of network structure search, the coordinator selects a portion of the network from the network to be searched that matches the computing resource size of the participant and distributes it to the participant, so that the participant can perform local training on the portion of the network;

[0026] The target image classification model is used to perform classification processing on the image data to be classified to obtain a classification result of the image data to be classified.

[0027] Optionally, the step of jointly performing multiple rounds of network structure search on the network to be searched with each of the participants to obtain a target image classification model, wherein in each round of network structure search, the coordinator selects a partial network from the network to be searched that matches the computing resource size of the participant and distributes it to the participant for the participant to perform local training on the partial network, includes:

[0028] Selecting a portion of the network from the to-be-searched network that matches the computing resource size of the target participant;

[0029] Sending the partial network to the target participant so that the target participant can perform local training on the partial network to obtain local parameter update information and return it;

[0030] Globally integrating the local parameter update information received from each participant to obtain global parameter update information, and updating the parameters in the network to be searched according to the global parameter update information, wherein the parameters include model parameters and / or structural parameters;

[0031] Check whether the network to be searched after the updated parameters meets the preset network conditions;

[0032] If satisfied, the target image classification model is determined based on the network to be searched after the parameters are updated;

[0033] If not, return to the step of selecting a partial network from the to-be-searched network that matches the computing resource size of the target participant.

[0034] To achieve the above objectives, the present invention provides a network structure search device, which is deployed on a coordinator participating in horizontal federated learning. The coordinator is in communication with each participant in horizontal federated learning. The coordinator deploys a network to be searched, and the device includes:

[0035] A selection module, configured to select a portion of the network from the to-be-searched network that matches the computing resource size of the target participant;

[0036] A sending module, configured to send the partial network to the target participant, so that the target participant can perform local training on the partial network to obtain local parameter update information and return it;

[0037] The fusion module is used to globally fuse the local parameter update information received from each participant to obtain global parameter update information, and update the parameters in the network to be searched according to the global parameter update information, wherein the parameters include model parameters and / or structure parameters.

[0038] To achieve the above objectives, the present invention provides an image classification device, which is deployed on a coordinator participating in horizontal federated learning. The coordinator is in communication with each participant in horizontal federated learning. The coordinator deploys a network to be searched. The device includes:

[0039] A search module is configured to jointly conduct multiple rounds of network structure search on the network to be searched with each of the participants to obtain a target image classification model, wherein in each round of network structure search, the coordinator selects a portion of the network from the network to be searched that matches the computing resource size of the participant and distributes it to the participant, so that the participant can perform local training on the portion of the network;

[0040] The classification module is used to classify the image data to be classified using the target image classification model to obtain the classification result of the image data to be classified.

[0041] To achieve the above-mentioned purpose, the present invention also provides a network structure search device, which includes: a memory, a processor, and a network structure search program stored on the memory and runnable on the processor. When the network structure search program is executed by the processor, the steps of the network structure search method described above are implemented.

[0042] To achieve the above-mentioned objectives, the present invention also provides an image classification device, which includes: a memory, a processor, and an image classification program stored in the memory and executable on the processor. When the image classification program is executed by the processor, the steps of the image classification method described above are implemented.

[0043] In addition, to achieve the above objectives, the present invention also proposes a computer-readable storage medium, on which a network structure search program is stored. When the network structure search program is executed by a processor, the steps of the network structure search method described above are implemented.

[0044] In addition, to achieve the above-mentioned purpose, the present invention further proposes a computer-readable storage medium, on which an image classification program is stored. When the image classification program is executed by a processor, the steps of the above-mentioned image classification method are implemented.

[0045] In addition, to achieve the above-mentioned object, the present invention also proposes a computer program product, including a computer program, which implements the steps of the above-mentioned network structure search method when executed by a processor.

[0046] In addition, to achieve the above-mentioned object, the present invention further proposes a computer program product, comprising a computer program, which implements the steps of the above-mentioned image classification method when executed by a processor.

[0047] In the existing horizontal federated learning process, the same global model is deployed locally on each participant. When some participants have fewer computing resources, the participants can only jointly train a small global model. Compared with the existing solution, in the present invention, by setting a to-be-searched network on the coordinator and utilizing the variable structure of the search network, the coordinator selects a portion of the network from the to-be-searched network that matches the computing resource size of the target participant and sends it to the target participant for local training. That is, the participant with fewer computing resources sends a small portion of the to-be-searched network, and the participant with more computing resources sends most or all of the to-be-searched network. The coordinator then fuses the local parameter update information obtained from the local training of each participant to obtain global parameter update information, and then updates the parameters of the to-be-searched network based on the global parameter update information. In the horizontal federated learning process, participants can participate in training regardless of whether they have more or less computing resources, and there is no limit on the size of the to-be-searched network set by the coordinator. This enables participants with fewer computing resources to participate in horizontal federated learning of large models, thereby obtaining models with higher prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1A schematic diagram of the hardware operating environment involved in an embodiment of the present invention;

[0049] Figure 2 This is a flow chart of a first embodiment of a network structure search method according to the present invention;

[0050] Figure 3 Schematic diagram of a horizontal federated learning framework involved in an embodiment of the present invention;

[0051] Figure 4 A schematic diagram of interaction between a coordinator and participants in horizontal federated learning according to an embodiment of the present invention;

[0052] Figure 5 A schematic diagram of the hardware structure of a coordinator and a participant involved in an embodiment of the present invention;

[0053] Figure 6 A schematic diagram of the structure of a network to be searched according to an embodiment of the present invention;

[0054] Figure 7 A schematic diagram of the structure of a network module in a network to be searched according to an embodiment of the present invention;

[0055] Figure 8 A schematic diagram of a connection operation between nodes involved in an embodiment of the present invention;

[0056] Figure 9 This is a functional schematic module diagram of a preferred embodiment of the network structure search device of the present invention.

[0057] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0058] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0059] like Figure 1 As shown, Figure 1 It is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention.

[0060] It should be noted that the network structure search device in the embodiment of the present invention can be a smart phone, a personal computer, a server and other devices, without specific restrictions here. The network structure search device can be a coordinator participating in horizontal federated learning, and the coordinator deploys the network to be searched.

[0061] like Figure 1As shown, the network structure search device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may optionally be a storage device independent of the aforementioned processor 1001.

[0062] Those skilled in the art will understand that Figure 1 The device structure shown in the figure does not constitute a limitation on the network structure search device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0063] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a network structure search program. The operating system is a program that manages and controls the hardware and software resources of the device and supports the operation of the network structure search program and other software or programs. Figure 1 In the device shown, the user interface 1003 is mainly used to communicate data with the client; the network interface 1004 is mainly used to establish a communication connection with each participant in horizontal federated learning; the processor 1001 can be used to call the network structure search program stored in the memory 1005 and perform the following operations:

[0064] Selecting a portion of the network that matches the computing resource size of the target participant from the network to be searched;

[0065] Sending the partial network to the target participant so that the target participant can perform local training on the partial network to obtain local parameter update information and return it;

[0066] Global parameter update information is obtained by globally fusing local parameter update information received from each participant, and parameters in the network to be searched are updated according to the global parameter update information, wherein the parameters include model parameters and / or structural parameters.

[0067] Furthermore, the step of selecting a portion of the network that matches the computing resource size of the target participant from the network to be searched includes:

[0068] Calculating, based on the computing resource information of the target participant, a target ratio of the model size that the target participant can accommodate to the model size of the network to be searched;

[0069] A partial network is randomly selected from the network to be searched according to the target ratio, wherein a ratio of a model size of the partial network to a model size of the network to be searched is less than or equal to the target ratio.

[0070] Furthermore, the step of randomly selecting a portion of the network from the network to be searched according to the target ratio includes:

[0071] For each node in the network to be searched, determining the number of retained edges corresponding to the node according to the target ratio, wherein the ratio of the number of retained edges to the total number of subsequent edges of the node in the network to be searched is less than or equal to the target ratio;

[0072] Randomly selecting a reserved back-connected edge from all back-connected edges of the node in the network to be searched, wherein the number of the reserved back-connected edges is the number of reserved edges;

[0073] The portion of the network to be searched connected by the retained subsequent edges corresponding to the nodes is used as a partial network.

[0074] Furthermore, the step of randomly selecting a portion of the network from the network to be searched according to the target ratio includes:

[0075] For each network module in the network to be searched, randomly selecting a retained operation from each parallel operation on a node connection edge in the network module to obtain a retained module connected by the retained operation, wherein a ratio of the operation computation amount of the retained module to the operation computation amount of the network module is less than or equal to the target ratio;

[0076] The network composed of the reserved modules is regarded as a partial network.

[0077] Furthermore, the step of globally fusing the local parameter update information received from each participant to obtain global parameter update information includes:

[0078] Obtaining an information group corresponding to the parameters in the network to be searched from the local parameter update information sent by each participant, wherein the information group includes local parameter update values ​​sent by some of the participants, and the some participants are participants in some of the distributed networks that include the parameters;

[0079] Averaging the local parameter update values ​​in the information group to obtain a global parameter update value corresponding to the parameter;

[0080] The global parameter update value corresponding to each parameter in the network to be searched is used as global parameter update information.

[0081] An embodiment of the present invention further provides an image classification device. The network structure search device may be a coordinator participating in horizontal federated learning. The coordinator is in communication with each participant in horizontal federated learning and deploys the network to be searched. The image classification device includes: a memory, a processor, and an image classification program stored in the memory and executable on the processor. When executed by the processor, the image classification program performs the following steps:

[0082] Conducting multiple rounds of network structure search on the network to be searched in conjunction with each of the participants to obtain a target image classification model, wherein in each round of network structure search, the coordinator selects a portion of the network from the network to be searched that matches the computing resource size of the participant and distributes it to the participant, so that the participant can perform local training on the portion of the network;

[0083] The target image classification model is used to perform classification processing on the image data to be classified to obtain a classification result of the image data to be classified.

[0084] Furthermore, the step of jointly performing multiple rounds of network structure search on the network to be searched with each of the participants to obtain a target image classification model, wherein in each round of network structure search, the coordinator selects a partial network from the network to be searched that matches the computing resource size of the participant and distributes it to the participant for the participant to perform local training on the partial network, includes:

[0085] Selecting a portion of the network that matches the computing resource size of the target participant from the network to be searched;

[0086] Sending the partial network to the target participant so that the target participant can perform local training on the partial network to obtain local parameter update information and return it;

[0087] Globally integrating the local parameter update information received from each participant to obtain global parameter update information, and updating the parameters in the network to be searched according to the global parameter update information, wherein the parameters include model parameters and / or structural parameters;

[0088] Check whether the network to be searched after the updated parameters meets the preset network conditions;

[0089] If satisfied, the target image classification model is determined based on the network to be searched after the parameters are updated;

[0090] If not, return to the step of selecting a partial network from the to-be-searched network that matches the computing resource size of the target participant.

[0091] Based on the above structure, various embodiments of the network structure search method are proposed.

[0092] Reference Figure 2 , Figure 2 This is a flow chart of the first embodiment of the network structure search method of the present invention. It should be noted that although a logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than here. The network structure search method of the present invention is applied to a coordinator participating in horizontal federated learning, wherein the coordinator is in communication with each participant participating in horizontal federated learning, and the coordinator deploys a network to be searched. The participants and the coordinator may be devices such as smart phones, personal computers, and servers. In this embodiment, the network structure search method includes:

[0093] Step S10, selecting a portion of the network that matches the computing resource size of the target participant from the network to be searched;

[0094] In this embodiment, a search network is deployed on the coordinator as a global model. Various parameters in the search network are initialized and need to be updated through multiple rounds of network structure search. Therefore, the search network is also referred to as a to-be-searched network in each embodiment.

[0095] The search network refers to a network used for network architecture search (NAS), which can be pre-designed based on the DARTS (Differentiable Architecture Search) method. The search network consists of multiple nodes, each with edges between them, or multiple operations on the edges, such as convolution and pooling. The multiple edges can be configured as a set of optional edges, where a structural parameter is assigned to each edge, and the sum of the structural parameters is 1. The structural parameters of the edges are continuously updated during the network architecture search process. After the network architecture search is complete, the edges to be retained are selected based on the size of the structural parameters, resulting in the optimal network structure. The multiple operations on an edge can also be configured as a set of optional operations, where a structural parameter is assigned to each operation, and the sum of the structural parameters is 1. The structural parameters of each operation are continuously updated during the network architecture search process. After the network architecture search is complete, the operations to be retained are selected based on the size of the structural parameters, resulting in the optimal network structure. In other words, the parameters in the search network include not only the model parameters of each operation, but also the structural parameters corresponding to the edges or operations. That is, it can be simply understood as setting multiple connection paths between the input layer and the output layer in the search network. Each path has corresponding structural parameters. Through the automated training process, each structural parameter can gradually reflect the effect of each path on the model performance. Finally, the path with the greatest effect on the model performance can be selected through the structural parameters to retain, and the most suitable network structure can be obtained, thereby completing the network structure search process.

[0096] The coordinator and each participant jointly conduct multiple rounds of network structure searches on the network to be searched, that is, multiple rounds of updates are performed on the structural parameters and model parameters in the network to be searched. Depending on the network structure search method adopted by the coordinator, the coordinator can pre-set the type of parameters that need to be updated in each round of network structure search, that is, the parameters updated in each round of network structure search can be structural parameters and / or model parameters. In this embodiment, there is no restriction on the network structure search method, so there is no restriction on the order of updating structural parameters and model parameters. For example, the structural parameters can be updated in the first few rounds of network structure search, and the model parameters can be updated in the next few rounds of network structure search. For another example, the structural parameters and model parameters can be updated together in each round of network structure search.

[0097] During a round of network structure search, the coordinator selects a portion of the network to be searched that matches the computing resource size of the target participant from the network to be searched and sends it to the target participant. The coordinator can use some participants as target participants, that is, when the computing resources of some participants among the participants are sufficient to train the entire network to be searched, the coordinator can send the entire network to be searched to these participants during each round of network structure search, instead of sending only a portion of the network to be searched. Therefore, the coordinator only uses the portion of participants whose computing resources are insufficient to train the entire network to be searched as target participants. Alternatively, the coordinator can also use all participants as target participants. For participants whose computing resources are sufficient to train the entire network to be searched, the portion of the network to be searched that matches the computing resources of the participant is the network to be searched itself. Therefore, the coordinator sends the entire network to be searched as a partial network to the participant. In other words, the partial network can be understood as being smaller than or equal to the network to be searched.

[0098] Specifically, there are various ways for the coordinator to obtain a partial network that matches the target participant's computing resource size, which are not limited in this embodiment. For example, a portion of the connection edges in each group of optional connection edges in the network to be searched is selected for retention, or a portion of the operation operations are selected for retention from each group of optional operations. The memory size required for training the retained network is calculated. If the memory size is smaller than the target participant's memory size, the retained network can be used as the target participant's partial network. It should be noted that when the coordinator selects the partial network, it ensures that there is at least one connected path from the input layer to the output layer of the network to be searched, that is, there will be no disconnection in the middle.

[0099] It should be noted that the coordinator selects a partial network for each participant. Since these selections are independent, the partial networks selected for each participant may be the same or different. The partial networks distributed to a participant during multiple rounds of network structure search are also independently selected, so each partial network may be the same or different. In one embodiment, the coordinator can randomly select a partial network from the network to be searched that matches the computing resource size of the target participant. This ensures that the partial networks obtained by each participant in each round of network structure search are random, allowing each participant to fully train each portion of the network to be searched using its own local training dataset.

[0100] Step S20: sending the partial network to the target participant, so that the target participant can perform local training on the partial network to obtain local parameter update information and return it;

[0101] The coordinator sends the selected partial network to the target participant. After receiving the partial network, the participant performs local training on the partial network and obtains local parameter update information.

[0102] Specifically, each participant can pre-set a training data set for performing network structure search on the search network. The training data set may include multiple training data, and each training data in the training data set may correspond to a label data. Depending on the model training task, the training data and label data are different. For example, when the model training task is to classify images, the training data is image data, and the corresponding label data is the true type of the image data. When the model training task is to classify text, the training data is text data, and the corresponding label data is the true type of the text data. It is understandable that the sample dimensions of the training data sets of each participant are different.

[0103] The specific process of the participant performing local training on the received partial network can be: the participant first inputs the training data set into the partial network for processing to obtain the network output; then calculates the loss function based on the label data corresponding to the training data set and the network output; obtains local parameter update information based on the loss function calculation; the local parameter update information can be the gradient value of the loss function relative to the parameters in the partial network, or it can be the parameters in the updated partial network; wherein the parameters refer to the parameters that need to be updated in this round of network structure search, which can be structural parameters and / or model parameters.

[0104] It should be noted that participants can use different datasets in each round of network structure search. Specifically, participants can divide their own dataset into multiple small training sets (also called data batches), and use a small dataset in each round to participate in the joint update of model parameters and / or structure parameters. Alternatively, participants can also sample a batch of data with replacement from the total dataset before each round of joint parameter update to participate in the joint parameter update of that round.

[0105] Each participant sends the local parameter update information obtained from their local training to the coordinator.

[0106] Step S30: Globally integrate the local parameter update information received from each participant to obtain global parameter update information, and update the parameters in the network to be searched according to the global parameter update information, wherein the parameters include model parameters and / or structure parameters.

[0107] The coordinator receives the local parameter update information sent by each participant. The coordinator globally integrates the local parameter update information to obtain the global parameter update information. Specifically, the global integration can be an averaging or weighted averaging of the local parameter update information. It can be understood that part of the network is part of the network to be searched, so the parameters in the part of the network are also part of the parameters in the network to be searched. Therefore, the local parameter update information sent by the participant is the local parameter update information of part of the parameters in the network to be searched. For each parameter in the network to be searched, the coordinator globally integrates the local parameter update value corresponding to the parameter in each local parameter update information, that is, obtains the global parameter update value corresponding to the parameter, and uses the global parameter update value corresponding to each parameter as the global parameter update information.

[0108] The coordinator updates the parameters in the network to be searched based on the global parameter update information. Specifically, when the global parameter update information includes parameter values ​​for each parameter, the values ​​of each parameter in the network to be searched are updated to the corresponding parameter values ​​in the global parameter update information. When the global parameter update information includes gradient values ​​for each parameter, the updated parameter values ​​of each parameter in the network to be searched are calculated based on the corresponding gradient values. The specific calculation method can be referenced by the gradient descent algorithm and is not described in detail in this embodiment. Among them, the parameters include model parameters and / or structural parameters; specifically, if the parameter type that needs to be updated in this round of network structural parameters is model parameters, then each participant sends local parameter update information corresponding to the model parameters, and the coordinator obtains global parameter update information corresponding to the model parameters through integration, and the model parameters in the network to be searched are updated according to the global parameter update information; if the parameter type that needs to be updated in this round of network structural parameters is structural parameters, then each participant sends local parameter update information corresponding to the structural parameters, and the coordinator obtains global parameter update information corresponding to the structural parameters through integration, and the structural parameters in the network to be searched are updated according to the global parameter update information; if the parameter type that needs to be updated in this round of network structural parameters is structural parameters and model parameters, then each participant sends local parameter update information corresponding to the model parameters and structural parameters, and the coordinator obtains global parameter update information corresponding to the model parameters and structural parameters through integration, and the model parameters and structural parameters in the network to be searched are updated according to the global parameter update information.

[0109] After a round of network structure search, the coordinator uses the updated network as the basis for the next round of network structure search. Furthermore, the coordinator can set a loop termination condition, terminating the network structure search when the condition is met. For example, the termination condition can be set to a maximum number of rounds.

[0110] Furthermore, after the network structure search of the network to be searched is completed, the coordinator can obtain a target model based on the network to be searched, or send the network to be searched to each participant so that each participant can determine the target model based on the network to be searched. Specifically, the coordinator or the participant can select the retained connection edges and / or operations based on the structural parameters in the network to be searched, use the network connected by the retained connection edges and operations as the target network, and then use the target network directly as the target model, or use the training data to train the target network to obtain the target model. In one embodiment, the connection edge with the largest structural parameter in a group of optional connection edges can be retained, or the operation with the largest structural parameter in a group of optional operations can be retained; in another embodiment, the connection edge with a structural parameter greater than a threshold value in a group of optional connection edges can be retained, or the operation with a structural parameter greater than a threshold value in a group of optional operations can be retained.

[0111] Furthermore, after obtaining the target model, the coordinator or participant can use the target model to complete the model prediction task. For example, when the model prediction task is to classify images, the training data in the training dataset are images. The target model trained based on the training dataset can be used to classify new images to obtain classification results.

[0112] In existing horizontal federated learning processes, the same global model is deployed locally on each participant. When some participants have fewer computing resources, they can only jointly train a small global model. Compared to existing solutions, in this embodiment, by setting a network to be searched on the coordinator and leveraging the variable structure of the search network, the coordinator selects a portion of the network from the network to be searched that matches the computing resources of the target participant and sends it to the target participant for local training. That is, participants with fewer computing resources send a small portion of the network to be searched, while participants with more computing resources send most or all of the network to be searched. The coordinator then fuses the local parameter update information obtained from the local training of each participant to obtain global parameter update information, and then updates the parameters of the network to be searched based on the global parameter update information. This allows participants with more or less computing resources to participate in training during horizontal federated learning, and there is no limit on the size of the network to be searched set by the coordinator. This enables participants with fewer computing resources to participate in horizontal federated learning of large models, thereby obtaining models with higher prediction accuracy.

[0113] In one embodiment, if Figure 3 As shown in the figure, the coordinating server sets the network to be searched as the global model, uses α to represent the structural parameter, and w to represent the model parameter. (α G , w G ) represents the network to be searched, A1, ..., A NThere are N participants. In one round of network structure search, the coordinator starts from (α G , w G ) selects a part of the network (α i , w i ) is sent to participant A (corresponding to Figure 3 ③), (α i , w i ) is (α G , w G ), that is, (α i , w i )∈(α G , w G ); Participants conduct local training on part of the network and return local parameter update information to the coordinator (corresponding to Figure 3 In ①), the coordinator integrates the local parameter update information returned by each participant to obtain the global parameter update information, and updates (α G , w G ) in α and / or w are updated (corresponding to Figure 3 ②).

[0114] In one embodiment, if Figure 4 As shown in the figure, it is a schematic diagram of the interaction between the coordinator and the participants in the multi-round network structure search. In the tth round, the coordinator selects a part of the network it from the network to be searched and sends it to the participant A. i , where i is the serial number of the participant and t is the round; each participant performs local training on the received part of the network and returns local parameter update information; the coordinator integrates the local parameter update information to obtain global parameter update information, and updates the parameters of the search network according to the global parameter update information to achieve a round of update of the search network.

[0115] In one embodiment, if Figure 5 The figure shows the hardware structure diagram of the coordinator and the participants jointly performing network structure search. The coordinator and the participants can be different hardware devices, and the coordinator and the participants transmit output through the Internet.

[0116] Furthermore, based on the above-mentioned first embodiment, a second embodiment of the network structure search method of the present invention is proposed. In this embodiment, step S10 includes:

[0117] Step S101, calculating the target ratio of the model size that can be accommodated by the target participant to the model size of the network to be searched based on the computing resource information of the target participant;

[0118] In this embodiment, the coordinator can obtain the computing resource information of each participant and calculate the ratio of the model size that the target participant can accommodate to the model size of the network to be searched based on the computing resource information of the target participant (hereinafter referred to as the target ratio for distinction). The computing resource information can be the memory size of the participant. The coordinator can then calculate the memory size required for training the network to be searched and divide the memory size of the target participant by the memory size required for training the network to be searched to obtain the target ratio. Alternatively, the computing resource information can be the number of additions or multiplications that can be performed by the memory of the participant. The coordinator counts the number of additions or multiplications required to train the network to be searched and divides the number of additions or multiplications that can be performed by the memory of the target participant by the number of additions or multiplications required to train the network to be searched to obtain the target ratio. Alternatively, the computing resource information can be the number of operations in the model that can be performed by the memory of the participant, such as the convolution kernel size. The coordinator can count the number of operations in the network to be searched and divide the number of operations in the model that can be performed by the memory of the target participant by the number of operations in the network to be searched to obtain the target ratio.

[0119] Step S102: randomly selecting a partial network from the network to be searched according to the target ratio, wherein the ratio of the model size of the partial network to the model size of the network to be searched is less than or equal to the target ratio.

[0120] After calculating the target ratio corresponding to the target participant, the coordinator can randomly select a portion of the network from the network to be searched based on the target ratio, so that the ratio of the model size of the portion of the network to the model size of the network to be searched is less than or equal to the target ratio. Specifically, there are various ways for the coordinator to randomly select a portion of the network based on the target ratio. For example, the coordinator can randomly retain some connecting edges and / or operations in the network to be searched, calculate the model size of the network connected by the retained connecting edges and operations, and test whether the ratio of the model size of the network to the model size of the network to be searched is less than or equal to the target ratio. If not, the coordinator randomly selects some connecting edges and / or operations from the retained network to retain and then tests again. This iterative cycle continues until it is determined that the ratio of the model size of the retained network to the model size of the network to be searched is less than or equal to the target ratio. The retained network is then used as the portion of the network to be sent to the target participant. The model size can be represented by the memory size required to train the model, the number of additions and multiplications required to train the model, the number of operations in the model, or the time required to train the model.

[0121] Furthermore, in one embodiment, step S102 includes:

[0122] Step S1021: for each node in the network to be searched, determining the number of retained edges corresponding to the node according to the target ratio, wherein the ratio of the number of retained edges to the total number of subsequent edges of the node in the network to be searched is less than or equal to the target ratio;

[0123] The network to be searched includes multiple network modules connected in sequence. Each network module has an input node and an output node. Multiple intermediate nodes are set between the input node and the output node. For nodes other than the output node, multiple connection edges (or back-connected edges) can be set to connect the subsequent nodes. That is, for nodes other than the output node, each node can correspond to at least one back-connected edge. Figure 6 As shown in the figure, it is a structural diagram of a network to be searched. Block1 to block4 are four network modules. w1, α1, ..., w4, α4 are the model parameters and structural parameters corresponding to the four network modules. c is the output layer, w 1，c are the model parameters of the output layer. Figure 7 for Figure 6 The network module includes an input node and an output node, as well as three intermediate nodes between the input node and the output node, and sets a connection edge between the nodes, and sets corresponding structural parameters for each connection edge. It should be noted that the structure of the network to be searched in this embodiment is not limited to Figure 4 and Figure 5 The structure shown in .

[0124] For each node in the network to be searched, the coordinator can determine the number of retained edges corresponding to the node based on the target ratio. Specifically, the coordinator can multiply the total number of subsequent edges of the node by the target ratio to obtain a maximum number of retained edges, and then select an edge number that is less than or equal to the maximum number of retained edges as the number of retained edges corresponding to the node. For example, a node has connected edges with the four subsequent nodes, that is, the total number of subsequent connected edges of the node is 4, and the target ratio is 0.5. The maximum variable number of retained edges is calculated to be 2, and the coordinator can select 1 or 2 as the number of retained edges for the node. It should be noted that for a node with only one subsequent edge, the number of retained edges of the node can be directly determined to be 1, that is, for each node, the coordinator retains at least one connecting edge between the node and the subsequent node.

[0125] Step S1022: randomly selecting retained back edges from all back edges of the node in the network to be searched, wherein the number of the retained back edges is the number of retained edges;

[0126] After determining the number of retained edges for each node, the coordinator randomly selects retained back edges from all back edges of the node in the search network. The number of retained back edges is the number of retained edges for the node. For example, if a node has 4 back edges and the number of retained edges is 2, then 2 of the 4 back edges are randomly selected as retained back edges.

[0127] Step S1023 : taking the portion of the network to be searched connected by the retained downstream edges corresponding to the nodes as a partial network.

[0128] After the coordinator determines to retain the back-connected edges for each node, the part of the network to be searched connected by the retained back-connected edges corresponding to each node is used as the partial network. That is, it is equivalent to deleting the other connection edges in the network to be searched except the retained back-connected edges, and the remaining part is used as the partial network.

[0129] It should be noted that, since the ratio of the number of retained edges of each node to the total number of its subsequent edges is less than or equal to the target ratio, the ratio of the model size of the entire partial network that is finally retained to the model size of the network to be searched is less than or equal to the target ratio, thereby obtaining a partial network that matches the computing resource size of the target participant.

[0130] In this embodiment, the coordinator calculates the target ratio of the model size that the target participant can accommodate to the model size of the network to be searched, and partially retains the back edges of each node according to the target ratio to obtain a partial network, so that the coordinator only needs to randomly select a certain number of back edges of each node to obtain a partial network that matches the computing resource size of the target participant, thereby improving the efficiency of the coordinator in determining the partial network from the network to be searched, thereby improving the learning efficiency of horizontal federated learning and saving the computing resources of the coordinator.

[0131] Furthermore, in one embodiment, step S102 includes:

[0132] Step S1024: For each network module in the network to be searched, randomly select a retained operation from each parallel operation on a node connection edge in the network module to obtain a retained module connected by the retained operation, wherein the ratio of the operation computation amount of the retained module to the operation computation amount of the network module is less than or equal to the target ratio;

[0133] Multiple parallel operations can be set on the connection edge between two nodes in the search network, for example Figure 8The figure shows a schematic diagram of node connection operations. Four parallel operations are set between two nodes: a 3x3 convolution (conv), a 7x7 convolution, a 3x3 pooling (pool), and an identity activation function. For each node connection edge, the coordinator can randomly select a retained operation from the parallel operations on that node connection edge. For each network module, the coordinator can reserve the portion of the network module connected by the retained operation as the retained module. This is equivalent to deleting all operations in the network module except the retained operation, leaving the remaining portion as the retained module. The coordinator calculates the ratio of the computational load of the retained module to the computational load of the network module. If this ratio is greater than a target ratio, the coordinator randomly selects a portion of the operations on each connection edge in the retained module and then calculates whether the ratio of the computational load of the retained module to the computational load of the network module is less than or equal to the target ratio. This process is repeated until the ratio of the computational load of the retained module to the computational load of the network module is less than or equal to the target ratio. The computational complexity of the network module can be represented by the number of operations in the network module, the number of additions and multiplications required to train the network module, or the duration required to train the network module.

[0134] It should be noted that, for a connection edge with only one operation set, the operation on the connection edge can be directly retained, that is, for the connection edge, the coordinator retains at least one operation on the connection edge.

[0135] Step S1025: The network formed by the reserved modules is used as a partial network.

[0136] After the coordinator determines the reserved modules corresponding to each network module, it uses the network composed of these reserved modules as the partial network. It should be noted that because the ratio of the computational workload of the reserved modules to the computational workload of their corresponding network modules is less than or equal to the target ratio, the model size of the entire retained partial network is less than or equal to the target ratio of the model size of the network to be searched, thereby obtaining a partial network that matches the computing resource size of the target participant.

[0137] In this embodiment, the coordinator calculates the target ratio of the model size that the target participant can accommodate to the model size of the network to be searched, and randomly selects the retained operations from the parallel operations on each connection edge according to the target ratio. The coordinator only needs to ensure that the ratio of the operation calculation amount of each retained module to the operation calculation amount of its corresponding network module is less than or equal to the target ratio to obtain a partial network that matches the computing resource size of the target participant, thereby improving the efficiency of the coordinator in determining the partial network from the network to be searched, thereby improving the learning efficiency of horizontal federated learning and saving the computing resources of the coordinator.

[0138] Furthermore, based on the first and / or second embodiments described above, a third embodiment of the network structure search method of the present invention is proposed. In this embodiment, the step of globally fusing the local parameter update information received from each participant to obtain global parameter update information in step S30 includes:

[0139] Step S301: obtaining an information group corresponding to the parameters in the network to be searched from the local parameter update information sent by each participant, wherein the information group includes local parameter update values ​​sent by some of the participants, and the some participants are participants in the distributed part of the network that include the parameters;

[0140] The participants perform local training on the received partial network to obtain local parameter update information, which includes the local parameter update values ​​corresponding to each parameter in the partial network. Since the partial network sent by the coordinator to the participants is part of the network to be searched, and the parameters therein are also part of the parameters in the network to be searched, the sent local parameter update information also only includes the local parameter update values ​​corresponding to the parameters in the network to be searched. Therefore, the coordinator obtains the local parameter update value of each parameter in the network to be searched from the local parameter update information sent by the participants that include the parameter in the distributed partial network, thereby obtaining the information group corresponding to the parameter.

[0141] Step S302, averaging the local parameter update values ​​in the information group to obtain a global parameter update value corresponding to the parameter;

[0142] After the coordinator obtains the information groups corresponding to each parameter, it averages the local parameter update values ​​in each parameter information group to obtain the global parameter update value corresponding to that parameter. The averaging of the local parameter update values ​​in the information group can be performed directly or by weighted averaging. If a weighted average fusion method is used, the weight value corresponding to the local parameter update value can be the ratio of the training sample size of the participant sending the local parameter update value to the total training sample size of all senders of local parameter update values ​​in the information group. The training sample size refers to the number of training data items in the training dataset.

[0143] In one embodiment, each participant may send its own training sample size to the coordinator, and the coordinator dynamically calculates the weights corresponding to each local parameter update value in each round of network structure search based on the training sample size of each participant. That is, the part of the network sent to each participant in each round of network structure search is different, so the sender of the local parameter update value in the parameter information group in each round of network structure search is different. Therefore, the coordinator needs to dynamically calculate the weight.

[0144] Step S303: Using the global parameter update value corresponding to each parameter in the network to be searched as global parameter update information.

[0145] After calculating and obtaining the global parameter update values ​​corresponding to the parameters in the network to be searched, the coordinator uses the global parameter update values ​​as global parameter update information, that is, uses the global parameter update values ​​to update the parameters.

[0146] In this embodiment, each participant locally trains its own partial network to obtain local parameter update information, and the coordinator obtains the information group corresponding to each parameter in the network to be searched from each local parameter update information, and then averages each local parameter update value in the information group to obtain the global parameter update value corresponding to each parameter. When each parameter is updated according to the global parameter update value, when each participant locally trains only a part of the network to be searched, the local parameter update information of each participant can also be globally integrated, so that participants with less computing resources can also participate in the horizontal federated learning of large models by training only part of the network, thereby improving the prediction accuracy of the trained model.

[0147] Furthermore, based on the first, second, and / or third embodiments described above, a fourth embodiment of the image classification method of the present invention is proposed. In this embodiment, the image classification method is applied to a coordinator participating in horizontal federated learning. The coordinator is in communication with each participant in horizontal federated learning. The coordinator deploys a network to be searched. The participants and the coordinator can be devices such as smartphones, personal computers, and servers. In this embodiment, the image classification method includes:

[0148] Step A10: jointly performing multiple rounds of network structure search on the network to be searched with each of the participants to obtain a target image classification model, wherein in each round of network structure search, the coordinator selects a portion of the network to be searched that matches the computing resource size of the participant and distributes it to the participant, so that the participant can perform local training on the portion of the network;

[0149] In this embodiment, the coordinator and each participant jointly conduct multiple rounds of network structure search on the network to be searched to obtain the target network structure. Specifically, the coordinator and the participants can conduct multiple rounds of network structure search according to the network structure search method described in the first, second, or third embodiment. The output layer of the network to be searched can be configured as a binary classifier or a multi-classifier for performing binary or multi-classification on images. The local training dataset of each participant is an image dataset and includes classification labels corresponding to each image data.

[0150] After multiple rounds of network structure search, the coordinator obtains the target image classification model based on the network to be searched, or sends the network to be searched to the participating parties, who then obtain the target image classification model based on the network to be searched. Specifically, the coordinator or participating parties can select the retained connection edges and / or operations from the structural parameters of the network to be searched after multiple rounds of network structure search, use the network connected by the retained connection edges and operations as the target network, and then directly use the target network as the target image classification model, or use the training data to train the target network to obtain the target image classification model.

[0151] Step A20: classify the image data to be classified using the target image classification model to obtain a classification result of the image data to be classified.

[0152] After obtaining the target image classification model, the coordinator or participant can use the new image data as the image data to be classified, and use the target image classification model to classify the image data to obtain the classification result of the image data to be classified.

[0153] In this embodiment, by setting a network to be searched on the coordinating party and utilizing the variable structure of the search network, in each round of network structure search, the coordinating party selects a part of the network that matches the computing resource size of the target participant from the network to be searched and sends it to the target participant for local training. That is, the participant with fewer computing resources sends a small part of the network to be searched, and the participant with more computing resources sends most or all of the network to be searched. When the network to be searched is larger, the participant with fewer computing resources can also participate in the training, thereby increasing the training sample size for the large-scale image classification model, and further improving the accuracy of image classification performed by the target image classification model obtained by multiple rounds of network structure search.

[0154] Furthermore, in one embodiment, step A10 includes:

[0155] Step A101, selecting a portion of the network that matches the computing resource size of the target participant from the network to be searched;

[0156] Step A102: sending the partial network to the target participant, so that the target participant can perform local training on the partial network to obtain local parameter update information and return it;

[0157] Step A103: Globally integrating the local parameter update information received from each participant to obtain global parameter update information, and updating the parameters in the network to be searched according to the global parameter update information, wherein the parameters include model parameters and / or structural parameters;

[0158] Specifically, the process of the coordinator and the participants performing a round of network structure search can refer to the network structure search process in steps S10 to S30 in the first embodiment above, and will not be described in detail in this embodiment.

[0159] Step A104, detecting whether the network to be searched after the updated parameters meets the preset network conditions;

[0160] After updating the parameters of the network to be searched based on the global parameter update information, the coordinator checks whether the network to be searched after the updated parameters meets the preset network condition. The preset network condition can be a pre-set condition, for example, it can be set to the convergence of the loss function of the network to be searched, the number of network structure search rounds reaching a certain number, or the duration of the network structure search reaching a certain length.

[0161] Step A105: If satisfied, determine the target image classification model based on the network to be searched after updating the parameters;

[0162] When the coordinator detects that the network to be searched after the updated parameters meets the preset network conditions, it can determine the target image classification model based on the network to be searched after the updated parameters. Specifically, the coordinator can select the retained connection edges and / or operations from the structural parameters of the network to be searched after multiple rounds of network structure search, use the network connected by the retained connection edges and operations as the target network, and then use the target network directly as the target image classification model, or use the target network as a global model and then conduct horizontal federated learning with all participating parties to obtain the target image classification model. It should be noted that when the coordinator uses the target network as the global model to conduct horizontal federated learning with all participants again, it can detect whether the model size that can be accommodated by each participant is larger than the model size of the target network. If so, the coordinator can conduct conventional horizontal federated learning with all participants to obtain the target image classification model; if the model size that can be accommodated by some participants is smaller than the model size of the target network, then in each round of model parameter update process of horizontal federated learning, the coordinator can select part of the target network and send it to the participant whose model size can be accommodated is smaller than the model size of the target network, so that the participant can also participate in the training, thereby improving the image classification accuracy of the target image classification model finally obtained.

[0163] Step A106: If not satisfied, return to step A101.

[0164] If the coordinator detects that the network to be searched after the updated parameters does not meet the preset network conditions, it returns to step A101, ie, performs the next round of network structure search.

[0165] In addition, the embodiment of the present invention also provides a network structure search device, referring to Figure 9The device is deployed on a participant in horizontal federated learning, each participant is connected to a coordinator for communication, and each participant is provided with a search network of the same structure. The device includes:

[0166] A selection module 10 is configured to select a portion of the network to be searched that matches the computing resource size of the target participant;

[0167] A sending module 20 is configured to send the partial network to the target participant, so that the target participant can perform local training on the partial network to obtain local parameter update information and return it;

[0168] An updating module is configured to globally fuse local parameter update information received from each participant to obtain global parameter update information, and update parameters in the network to be searched according to the global parameter update information, wherein the parameters include model parameters and / or structural parameters.

[0169] Furthermore, the selection module 10 includes:

[0170] A first calculation unit is configured to calculate, based on the computing resource information of the target participant, a target ratio of the model size that can be accommodated by the target participant to the model size of the network to be searched;

[0171] A selection unit is used to randomly select a partial network from the network to be searched according to the target ratio, wherein the ratio of the model size of the partial network to the model size of the network to be searched is less than or equal to the target ratio.

[0172] Furthermore, the selection unit includes:

[0173] a first determining subunit, configured to determine, for each node in the network to be searched, a number of retained edges corresponding to the node according to the target ratio, wherein a ratio of the number of retained edges to the total number of subsequent edges of the node in the network to be searched is less than or equal to the target ratio;

[0174] A first selection subunit is configured to randomly select a reserved back-connected edge from all back-connected edges of the node in the network to be searched, wherein the number of the reserved back-connected edges is the number of reserved edges;

[0175] The second determining subunit is configured to take the portion of the network to be searched connected by the retained subsequent edges corresponding to the nodes as a partial network.

[0176] Furthermore, the selection unit includes:

[0177] a second selection subunit, configured to randomly select, for each network module in the network to be searched, a retained operation from each parallel operation on a node connection edge in the network module, to obtain a retained module connected by the retained operation, wherein a ratio of an operation computation amount of the retained module to an operation computation amount of the network module is less than or equal to a target ratio;

[0178] The third determining subunit is configured to use the network formed by the reserved modules as a partial network.

[0179] Furthermore, the updating module 30 includes:

[0180] an acquiring unit, configured to acquire, from the local parameter update information sent by each participant, an information group corresponding to the parameter in the network to be searched, wherein the information group includes local parameter update values ​​sent by some of the participants, the some participants being participants including the parameters in some of the distributed networks;

[0181] a calculation unit, configured to average the local parameter update values ​​in the information group to obtain a global parameter update value corresponding to the parameter;

[0182] The determining unit is configured to use the global parameter update value corresponding to each parameter in the network to be searched as global parameter update information.

[0183] The expanded content of the specific implementation of the network structure search device of the present invention is basically the same as the various embodiments of the above-mentioned network structure search method, and will not be repeated here.

[0184] In addition, an embodiment of the present invention further provides an image classification device, which is deployed on a coordinator participating in horizontal federated learning. The coordinator is in communication with each participant participating in horizontal federated learning. The coordinator deploys a network to be searched. The device includes:

[0185] A search module is configured to jointly conduct multiple rounds of network structure search on the network to be searched with each of the participants to obtain a target image classification model, wherein in each round of network structure search, the coordinator selects a portion of the network from the network to be searched that matches the computing resource size of the participant and distributes it to the participant, so that the participant can perform local training on the portion of the network;

[0186] The classification module is used to classify the image data to be classified using the target image classification model to obtain the classification result of the image data to be classified.

[0187] Furthermore, the search module includes:

[0188] A selection unit, configured to select a portion of the network to be searched that matches the computing resource size of the target participant;

[0189] A sending unit, configured to send the partial network to the target participant, so that the target participant can perform local training on the partial network to obtain local parameter update information and return the information;

[0190] an updating unit, configured to globally fuse local parameter update information received from each participant to obtain global parameter update information, and update parameters in the network to be searched according to the global parameter update information, wherein the parameters include model parameters and / or structural parameters;

[0191] A detection unit, configured to detect whether the network to be searched after the parameters are updated meets the preset network conditions;

[0192] a determination unit, configured to determine a target image classification model based on the network to be searched after updating parameters if the conditions are met;

[0193] The returning unit is configured to return to the step of selecting a partial network that matches the computing resource size of the target participant from the network to be searched if the condition is not satisfied.

[0194] The expanded content of the specific implementation of the image classification device of the present invention is basically the same as the above-mentioned embodiments of the image classification method, and will not be repeated here.

[0195] In addition, an embodiment of the present invention further provides a computer-readable storage medium having a network structure search program stored thereon. When the network structure search program is executed by a processor, the network structure search program implements the steps of the network structure search method described above. The present invention further provides a computer program product, including a computer program. When the computer program is executed by a processor, the computer program implements the steps of the network structure search method described above. The various embodiments of the network structure search device, computer-readable storage medium, and computer product of the present invention may all refer to the various embodiments of the network structure search method of the present invention and will not be repeated here.

[0196] In addition, embodiments of the present invention further provide a computer-readable storage medium storing an image classification program. When executed by a processor, the image classification program implements the steps of the image classification method described above. The present invention further provides a computer program product comprising a computer program. When executed by a processor, the computer program implements the steps of the image classification method described above. The various embodiments of the image classification device, computer-readable storage medium, and computer product of the present invention may all refer to the various embodiments of the image classification method of the present invention and will not be further described here.

[0197] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0198] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0199] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0200] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A network structure search method, characterized in that: The method is applied to a coordinator participating in horizontal federated learning, the coordinator being in communication with each participant participating in horizontal federated learning, and the coordinator deploying a network to be searched. The method includes the following steps: Selecting a portion of the network from the to-be-searched network that matches the computing resource size of the target participant; Sending the partial network to the target participant, so that the target participant can perform local training on the partial network to obtain local parameter update information and return it, wherein the participant with less computing resources sends a smaller part of the network to be searched, and the participant with more computing resources sends a larger part of the network to be searched; Global parameter update information is obtained by globally fusing local parameter update information received from each participant, and parameters in the network to be searched are updated according to the global parameter update information, wherein the parameters include model parameters and / or structural parameters.

2. The network structure search method according to claim 1, wherein: The step of selecting a portion of the network that matches the computing resource size of the target participant from the network to be searched includes: Calculating, based on the computing resource information of the target participant, a target ratio of the model size that the target participant can accommodate to the model size of the network to be searched; A partial network is randomly selected from the network to be searched according to the target ratio, wherein a ratio of a model size of the partial network to a model size of the network to be searched is less than or equal to the target ratio.

3. The network structure search method according to claim 2, wherein: The step of randomly selecting a portion of networks from the to-be-searched networks according to the target ratio includes: For each node in the network to be searched, determining the number of retained edges corresponding to the node according to the target ratio, wherein the ratio of the number of retained edges to the total number of subsequent edges of the node in the network to be searched is less than or equal to the target ratio; Randomly selecting a reserved back-connected edge from all back-connected edges of the node in the network to be searched, wherein the number of the reserved back-connected edges is the number of reserved edges; The portion of the network to be searched connected by the retained subsequent edges corresponding to the nodes is used as a partial network.

4. The network structure search method according to claim 2, wherein: The step of randomly selecting a portion of networks from the to-be-searched networks according to the target ratio includes: For each network module in the network to be searched, randomly selecting a retained operation from each parallel operation on a node connection edge in the network module to obtain a retained module connected by the retained operation, wherein a ratio of the operation computation amount of the retained module to the operation computation amount of the network module is less than or equal to the target ratio; The network composed of the reserved modules is regarded as a partial network.

5. The network structure search method according to any one of claims 1 to 4, characterized in that: The step of globally fusing the local parameter update information received from each participant to obtain global parameter update information includes: Obtaining an information group corresponding to the parameters in the network to be searched from the local parameter update information sent by each participant, wherein the information group includes local parameter update values ​​sent by some of the participants, and the some participants are participants in some of the distributed networks that include the parameters; Averaging the local parameter update values ​​in the information group to obtain a global parameter update value corresponding to the parameter; The global parameter update value corresponding to each parameter in the network to be searched is used as global parameter update information.

6. An image classification method, characterized in that: The method is applied to a coordinator participating in horizontal federated learning, the coordinator being in communication with each participant participating in horizontal federated learning, and the coordinator deploying a network to be searched. The method includes: Jointly performing multiple rounds of network structure search on the network to be searched with each of the participants to obtain a target image classification model, wherein in each round of network structure search, the coordinator selects a portion of the network to be searched that matches the computing resource size of the participant and distributes it to the participant, so that the participant can perform local training on the portion of the network, wherein the participant with less computing resources sends a smaller portion of the network to be searched, and the participant with more computing resources sends a larger portion of the network to be searched; The target image classification model is used to perform classification processing on the image data to be classified to obtain a classification result of the image data to be classified.

7. The image classification method according to claim 6, wherein: The step of jointly performing multiple rounds of network structure search on the network to be searched with each of the participants to obtain a target image classification model, wherein in each round of network structure search, the coordinator selects a partial network from the network to be searched that matches the computing resource size of the participant and distributes it to the participant for the participant to perform local training on the partial network, includes: Selecting a portion of the network from the to-be-searched network that matches the computing resource size of the target participant; Sending the partial network to the target participant so that the target participant can perform local training on the partial network to obtain local parameter update information and return it; Globally integrating the local parameter update information received from each participant to obtain global parameter update information, and updating the parameters in the network to be searched according to the global parameter update information, wherein the parameters include model parameters and / or structural parameters; Check whether the network to be searched after the updated parameters meets the preset network conditions; If satisfied, the target image classification model is determined based on the network to be searched after the parameters are updated; If not, return to the step of selecting a partial network from the to-be-searched network that matches the computing resource size of the target participant.

8. A network structure search device, characterized in that: The device is deployed on a coordinator participating in horizontal federated learning, the coordinator is in communication with each participant participating in horizontal federated learning, and the coordinator deploys a network to be searched. The device includes: A selection module, configured to select a portion of the network from the to-be-searched network that matches the computing resource size of the target participant; A sending module is configured to send the partial network to the target participant, so that the target participant can perform local training on the partial network to obtain local parameter update information and return it, wherein the participant with less computing resources sends a smaller portion of the network to be searched, and the participant with more computing resources sends a larger portion of the network to be searched; The fusion module is used to globally fuse the local parameter update information received from each participant to obtain global parameter update information, and update the parameters in the network to be searched according to the global parameter update information, wherein the parameters include model parameters and / or structure parameters.

9. A network structure search device, characterized in that: The network structure search device includes: a memory, a processor, and a network structure search program stored in the memory and executable on the processor. When the network structure search program is executed by the processor, the steps of the network structure search method according to any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a network structure search program, which, when executed by a processor, implements the steps of the network structure search method according to any one of claims 1 to 5.

11. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the network structure search method according to any one of claims 1 to 5 are implemented.

Images