Method and apparatus for transmitting model data

Abstract

This application provides a method and apparatus for transmitting model data. The method can reduce the accuracy loss of model data, comprising: a first device determining second information based on first information, the second information being used by a second device to quantize first model data, wherein the first information includes the evaluation loss corresponding to the current training round, the second information includes a quantization error threshold, and the first model data is the model data after the current training round; the first device sending the second information to the second device; and the first device receiving a first message sent by the second device, the first message including the quantized first model data and first quantization configuration information.

Description

Technical Field

[0001] This application relates to the field of communications, and more specifically, to a method and apparatus for transmitting model data. Background Technology

[0002] Federated learning is an encrypted, distributed machine learning technique that fully leverages the data and computing power of participating parties, enabling multiple parties to collaboratively build general, robust machine learning models without needing to share data. In an environment of increasingly stringent data regulations, federated learning can address key issues such as data ownership, data privacy, data access rights, and access to heterogeneous data. In wireless scenarios, federated learning can be used to train models to ensure data security. Horizontal federated learning is a key branch of federated learning. A horizontal federation includes a coordinator and several participating parties. Participating parties are responsible for training models using their local data, while the coordinator is responsible for aggregating the models from all participating parties.

[0003] In horizontal federated learning, although participants do not need to transmit the original training data to the server, they need to send model parameters to the coordinator multiple times, which leads to significant communication overhead. Model quantization techniques can alleviate the problems of large model parameter volume and high memory consumption, and can be applied to the horizontal federated learning process to reduce communication overhead.

[0004] However, quantization introduces quantization errors and may lead to a loss of accuracy in the final trained model. Especially in federated learning, unlike traditional quantized models which are directly used for inference, the quantized model in federated learning is still used for training, and there is a continuous accumulation of quantization errors across multiple rounds and users. Summary of the Invention

[0005] This application provides a method and apparatus for transmitting model data, which can reduce the loss of accuracy in model data.

[0006] In a first aspect, a method for transmitting model data is provided, the method comprising: a first device determining second information based on first information, the second information being used by a second device to quantize first model data, wherein the first information includes an evaluation loss corresponding to the current training round, the second information includes a quantization error threshold, and the first model data is model data after the current training round; the first device transmitting the second information to the second device; and the first device receiving a first message transmitted by the second device, the first message including the quantized first model data and first quantization configuration information.

[0007] Based on the above technical solution, the first device can determine the quantization error threshold according to the evaluation loss, the accuracy requirements of the second device for model training, and the communication sensitivity information, so that the quantization error corresponding to the quantization method adopted by the second device is less than the quantization error threshold. This can control the accumulation of quantization errors in multiple rounds and multiple users during federated learning training, thereby reducing the accuracy loss of model data.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, the first information also includes the accuracy requirements of the second device for model training and communication sensitivity information.

[0009] In conjunction with the first aspect, in some implementations of the first aspect, before the first device determines the second information based on the first information, the method further includes: the first device receiving a second message sent by the second device, the second message including the accuracy requirement information and the communication sensitivity information.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, before the first device determines the second information based on the first information, the method further includes: the first device determining the proportion of quantizable layers in the second model data based on third information, the third information including the evaluation loss corresponding to the previous training round, the accuracy requirement information, and the communication sensitivity information, wherein the second model data is the model data before the current training round; the first device quantizing the second model data according to the proportion of quantizable layers to obtain quantized second model data; the first device sending a third message to the second device, the third message including the quantized second model data and second quantization configuration information, the third message being used by the second device to train the second model data to obtain the first model data.

[0011] Secondly, a method for sending model data is provided, the method comprising: a second device receiving second information sent by a first device, the second information being used by the second device to quantize first model data, wherein the second information includes a quantization error threshold, and the first model data is model data after the current training round; the second device quantizing the first model data according to the second information; and the second device sending a first message to the first device, the first message including the quantized first model data and first quantization configuration information.

[0012] In conjunction with the second aspect, in some implementations of the second aspect, the second device quantizes the first model data according to the second information, including: the second device quantizes the first model data using a first quantization method; the second device determines a first quantization error based on the quantized first model data and the first model data before quantization; if the first quantization error is less than the quantization error threshold, the second device determines to use the first quantization method to quantize the first model data.

[0013] In conjunction with the second aspect, in some implementations of the second aspect, before the second device receives the second information sent by the first device, the method further includes: the second device receiving a third message sent by the first device, the third message including quantized second model data and second quantization configuration information, wherein the second model data is the model data before this round of training; the second device dequantizing and parsing the second model data according to the quantized second model data and the second quantization configuration information; and the second device training the second model data to obtain the first model data.

[0014] Thirdly, another method for sending model data is provided, the method comprising: a first device receiving a fourth message sent by a second device, the fourth message including a first quantization error and first information, the first quantization error being determined by the second device after quantizing the first model data using a first quantization method, the first information including the evaluation loss corresponding to the current training round, wherein the first model data is the model data after the current training round; the first device determining, based on the first quantization error and the first information, whether to allow the second device to send the quantized first model data; and the first device sending indication information to the second device, the indication information being used to indicate whether to allow the second device to send the quantized first model data.

[0015] Based on the above technical solution, the first device determines whether the second device can send the quantized first model data based on the evaluation loss corresponding to this round of training and the first quantization error determined by the second device after quantizing the first model data. This can avoid excessive accumulation of the first quantization error of the second device, thereby reducing the accuracy loss of the model data.

[0016] In conjunction with the third aspect, in some implementations of the third aspect, the first device determines whether to allow the second device to send the quantized first model data based on the first information and the first quantization error, including: the first device determining the proportion of quantizable second devices based on the first information; and the first device determining whether to allow the second device to send the quantized first model data based on the proportion of quantizable second devices, the first quantization error, and a continuous quantization number threshold.

[0017] The first device determines and instructs the second device whether to send the quantized first model data based on the evaluation loss corresponding to this round of training, the first quantization error determined by the second device after quantizing the first model data, and the threshold for the number of consecutive quantizations. This can prevent the second device from quantizing beyond the threshold for the number of consecutive quantizations and reduce the accuracy loss of the model data.

[0018] In conjunction with the third aspect, in some implementations of the third aspect, the first information also includes the accuracy requirements of the second device for model training and communication sensitivity information.

[0019] In conjunction with the third aspect, in some implementations of the third aspect, before the first device receives the fourth message sent by the second device, the method further includes: the first device receiving a second message sent by the second device, the second message including the precision requirement information and the communication sensitivity information.

[0020] In conjunction with the third aspect, in some implementations of the third aspect, before the first device receives the fourth message sent by the second device, the method further includes: the first device determining the proportion of quantizable layers in the second model data according to third information, the third information including the evaluation loss corresponding to the previous training round, the accuracy requirement information, and the communication sensitivity information, wherein the second model data is the model data before the current training round; the first device quantizing the second model data according to the proportion of quantizable layers to obtain quantized second model data; the first device sending a third message to the second device, the third message including the quantized second model data and second quantization configuration information, the third message being used by the second device to train the second model data to obtain the first model data.

[0021] Fourthly, another method for sending model data is provided, the method comprising: a second device quantizing first model data using a first quantization method, wherein the first model data is the model data after the current training round; the second device determining a first quantization error based on the quantized first model data and the first model data before quantization; the second device sending a fourth message to a first device, the fourth message including the first quantization error and first information, the fourth message being used by the first device to determine whether to allow the second device to send the quantized first model data, the first information including the evaluation loss corresponding to the current training round; the second device receiving indication information sent by the first device, the indication information being used to indicate whether to allow the second device to send the quantized first model data; and the second device determining whether to send the quantized first model data to the first device based on the indication information.

[0022] In conjunction with the fourth aspect, in some implementations of the fourth aspect, the second device determines whether to send the quantized first model data to the first device based on the indication information, including: if the indication information indicates that the second device is allowed to send the quantized first model data, then the second device sends the quantized first model data and third quantization configuration information to the first device; if the indication information indicates that the second device is not allowed to send the quantized first model data, then the second device sends the unquantized first model data.

[0023] In conjunction with the fourth aspect, in some implementations of the fourth aspect, before the second device quantizes the first model data using the first quantization method, the method further includes: the second device receiving a third message sent by the first device, the third message including quantized second model data and second quantization configuration information, wherein the second model data is the model data before this round of training; the second device dequantizing and parsing the second model data based on the quantized second model data and the second quantization configuration information; and the second device training the second model data to obtain the first model data.

[0024] Fifthly, another method for sending model data is provided. This method includes: a first device determining the proportion of quantizable layers in second model data based on third information, wherein the third information includes the evaluation loss corresponding to the previous training round, the accuracy requirements of the second device for model training, and communication sensitivity information; the second model data is the model data before the current training round; the first device quantizing the second model data according to the proportion of quantizable layers to obtain quantized second model data; and the first device sending a third message to the second device, the third message including the quantized second model data and second quantization configuration information, the third message being used by the second device to train the second model data.

[0025] In conjunction with the fifth aspect, in certain implementations of the fifth aspect, the first device quantizes the second model data according to the proportion of the quantizable layers, including: the first device quantizing each layer of data in the second model data; the first device determining the quantization error corresponding to each layer of data and the compression contribution of each layer of data; the first device determining the quantizable layers in the second model data according to the proportion of the quantizable layers, and / or, the quantization error corresponding to each layer of data, and / or, the compression contribution of each layer of data; the first device obtaining the quantized second model data, wherein the data corresponding to the quantizable layers in the quantized second model data has been quantized, and the data corresponding to the non-quantizable layers has not been quantized.

[0026] Based on the above technical solution, the first device performs hierarchical quantization on the model sent to the second device, which can control the accumulation of quantization errors in multiple rounds and reduce the accuracy loss of model data while reducing the amount of transmission.

[0027] A sixth aspect provides a communication device comprising: a unit for implementing methods as described in the first to fifth aspects or any possible implementations of the first to fifth aspects.

[0028] In a seventh aspect, a communication chip is provided, including a processor and a communication interface, the processor being configured to read instructions to execute methods as described in the first to fifth aspects or any possible implementation thereof.

[0029] Eighthly, a communication device is provided, comprising: a processor and a transceiver, the transceiver being configured to receive computer code or instructions and transmit them to the processor, the processor executing the computer code or instructions, such as the methods in the first to fifth aspects or any possible implementations of the first to fifth aspects.

[0030] Ninth aspect, a computer-readable storage medium is provided, the computer-readable medium storing a computer program; when the computer program is run on a computer, it causes the computer to perform the methods of the first to fifth aspects or any possible implementations of the first to fifth aspects. Attached Figure Description

[0031] Figure 1 It presents a schematic diagram of the system architecture for horizontal federated learning applications in the UE-RAN scenario.

[0032] Figure 2 A schematic diagram of model quantization was presented.

[0033] Figure 3 This is a schematic diagram of horizontal federated learning in the RAN-network management scenario.

[0034] Figure 4 This is a schematic diagram of horizontal federated learning under the eNA architecture.

[0035] Figure 5 This is a schematic flowchart illustrating a method for sending model data according to an embodiment of this application.

[0036] Figure 6 This is a schematic flowchart illustrating another method for sending model data according to an embodiment of this application.

[0037] Figure 7 This is a schematic flowchart illustrating another method for sending model data according to an embodiment of this application.

[0038] Figure 8 This is a schematic block diagram of a communication device according to an embodiment of this application.

[0039] Figure 9 This is a schematic block diagram of another communication device according to an embodiment of this application.

[0040] Figure 10 This is a schematic block diagram of another communication device according to an embodiment of this application.

[0041] Figure 11 This is a schematic block diagram of another communication device according to an embodiment of this application.

[0042] Figure 12 This is a schematic block diagram of another communication device according to an embodiment of this application.

[0043] Figure 13 This is a schematic block diagram of a communication device according to an embodiment of this application. Detailed Implementation

[0044] The technical solutions in this application will now be described with reference to the accompanying drawings.

[0045] The technical solutions of this application embodiment can be applied to various communication systems, such as: Global System for Mobile Communication (GSM) system, Code Division Multiple Access (CDMA) system, Wideband Code Division Multiple Access (WCDMA) system, General Packet Radio Service (GPRS), Long Term Evolution (LTE) system, LTE Frequency Division Duplex (FDD) system, LTE Time Division Duplex (TDD) system, Universal Mobile Telecommunication System (UMTS), Worldwide Interoperability for Microwave Access (WiMAX) communication system, future 5th generation (5G) systems, or new radio (NR), etc.

[0046] The terminal device in this application embodiment can refer to user equipment, access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent, or user device. The terminal device can also be a cellular phone, cordless phone, session initiation protocol (SIP) phone, wireless local loop (WLL) station, personal digital assistant (PDA), handheld device with wireless communication capabilities, computing device or other processing device connected to a wireless modem, vehicle-mounted device, wearable device, terminal device in future 5G networks, or terminal device in future evolved public land mobile networks (PLMNs), etc., but this application embodiment does not limit this.

[0047] The network device in this application embodiment can be a device for communicating with terminal devices. The network device can be a base station (BTS) in a global system of mobile communication (GSM) or code division multiple access (CDMA) system, a base station (NodeB, NB) in a wideband code division multiple access (WCDMA) system, an evolved NodeB (eNB or eNodeB) in an LTE system, a radio controller in a cloud radio access network (CRAN) scenario, or a relay station, access point, vehicle-mounted device, wearable device, or a network device in a future 5G network or a network device in a future evolved PLMN network, etc. The embodiments of this application are not limited.

[0048] Federated learning is an encrypted, distributed machine learning technique that fully leverages the data and computing power of participating parties, enabling multiple parties to collaboratively build general and robust machine learning models without needing to share data. In an environment of increasingly stringent data regulations, federated learning can address key issues such as data ownership, data privacy, data access rights, and access to heterogeneous data. In wireless scenarios, federated learning can be used to train models to ensure data security.

[0049] Horizontal federated learning is a key branch of federated learning. A horizontal federation includes a coordinator and several participants. Participants are responsible for training models using their local data, while the coordinator is responsible for aggregating the models from all participants. For example... Figure 1 The diagram illustrates the system architecture of horizontal federated learning in a user equipment (UE)-radio access network (RAN) scenario. The horizontal federated learning process generally consists of the following four steps: 1) The coordinator sends the model to each participant; this step is called model distribution. 2) Each participant trains the model using its own dataset. 3) Each participant sends the trained model parameters to the coordinator; this step is called model uploading. 4) The coordinator aggregates the model parameters received from the participants, for example, using a federated averaging algorithm, and then updates the model. This process is repeated until the model converges, the maximum number of iterations is reached, or the maximum training time is reached.

[0050] In horizontal federated learning, the model parameters uploaded by the user interface (UE) can be model weight files, i.e., the model weights after one or more local training iterations. The format of the model parameters is aggregated by the coordinator. In actual horizontal federated learning processes, participants and the coordinator usually agree on a default format for model parameter transmission in advance. After completing local model training, participants transmit model parameters according to the agreed format so that the coordinator can aggregate them.

[0051] In horizontal federated learning, although participants do not need to transmit the original training data to the server, they need to send model parameters to the coordinator multiple times, which leads to significant communication overhead. Model quantization techniques can alleviate the problems of large model parameter volume and high memory consumption, and can be applied to the horizontal federated learning process to reduce communication overhead.

[0052] like Figure 2 The diagram illustrates model quantization (or simply quantization). The main principle is to compress the original network by reducing the number of bits required to represent the neural network weights. Typically, deep learning model parameters are 32-bit floating-point numbers. Model quantization methods can quantize these 32-bit floating-point weights into 16-bit, 8-bit, 4-bit, 2-bit, or even 1-bit values, significantly reducing the storage space occupied by the network. Models processed using model quantization generally need to undergo dequantization parsing to convert them into a suitable format before they can be used for model retraining or inference.

[0053] To facilitate understanding of this scheme, a brief description of model quantization techniques in federated learning is provided. Federated learning employs model quantization to compress transmitted model data, thereby reducing the amount of data transmitted during training. For example, using 8-bit quantization can reduce the amount of data transmitted in a single transmission to approximately one-quarter of the original 32-bit representation.

[0054] Because federated learning involves uploading and distributing models, there are two types of model quantization: quantization of the uploaded model and quantization of the distributed model.

[0055] Quantization introduces quantization errors and can lead to a loss of accuracy in the final trained model. This is especially true in federated learning, where, unlike traditional quantized models that are directly used for inference, the quantized model is still used for training, and quantization errors accumulate continuously across multiple rounds and users.

[0056] Therefore, the use of quantization methods for transmitted parameters in federated learning involves a trade-off between loss of accuracy and reduced data transmission.

[0057] Currently, in horizontal federated learning scenarios between UE and RAN, the RAN allocates uplink resources based on the UE's uplink channel conditions and determines the quantization method for the model transmitted by the UE. Each UE's transmitted model data is quantized. The purpose of quantization is to reduce the amount of parameter transmission. However, the RAN only considers the uplink channel conditions and independently quantizes the parameters uploaded by the UE, without considering the accumulation of multi-round, multi-user quantization errors that occur during federated learning training. This could lead to significant accuracy loss, failing to achieve a trade-off between accuracy loss and reduced transmission volume.

[0058] Therefore, this application proposes a method for transmitting model data, which can reduce the loss of accuracy in model data.

[0059] The system architecture used in this application embodiment is mainly the UE-RAN system architecture defined by the 3rd Generation Partnership Project (3GPP). In this scenario, multiple UEs are horizontally federated, with the UE being a participant in the horizontal federation process and the RAN acting as the coordinator.

[0060] Furthermore, this application can also be extended to other system architectures, such as... Figure 3 The RAN-network management scenario shown illustrates horizontal federated learning, where network management can be either an element management system (EMS) or a network management system (NMS); or... Figure 4 The diagram illustrates horizontal federated learning within the enabler of network automation (eNA) architecture. In this architecture, the local network data analytics function (NWDAF) is a participant in the horizontal federated process, while the central network data analytics function acts as the coordinator.

[0061] In this application, the UE and RAN support artificial intelligence (AI) model training; the network element management system manages one or more network elements of a certain category; the network management system manages communication between network elements; and the NWDAF is responsible for network data analysis and AI model training.

[0062] like Figure 5 As shown, an embodiment of this application presents a method for transmitting model data, which is applied to federated learning. The method includes:

[0063] 501. The second device sends a registration message to the first device. This registration message, which may be referred to as the second message, contains information about the second device. Optionally, the second message may include model accuracy requirement information and communication sensitivity information of the second device. There may be multiple second devices. The first device may be a wireless access network device, and the second device may be a terminal device; or, the first device may be a network element management system / network management system, and the second device may be a wireless access network device; or, the first device may be a central network data analysis function, and the second device may be a local network data analysis function, etc. This application embodiment does not limit this.

[0064] Accuracy requirements refer to the expected accuracy of the model trained by federated learning for the terminal device. For example, expecting the trained model to achieve 99% accuracy can be considered a high accuracy requirement, while 80% accuracy can be considered a low accuracy requirement. The function of accuracy requirements is to help the RAN determine what proportion of UEs can be allowed to perform quantization operations. If most UEs have low accuracy requirements, more UEs can be allowed to perform quantization; if most UEs have high accuracy requirements, fewer UEs can be allowed to perform quantization.

[0065] Communication sensitivity refers to the ease with which a UE can upload a model, and it is influenced by multiple factors such as channel conditions and the UE's local upload capability. The function of communication sensitivity is to help the RAN determine what proportion of UEs are allowed to perform quantization operations. If most UEs have high communication sensitivity, then more UEs can be allowed to perform quantization.

[0066] 502, the first device receives the second message sent by the second device, and selects the second device to participate in model training according to the second message.

[0067] 503, the first device sends a third message to the second device participating in this round of training. The third message includes the second model data and model name before this round of training, which is used by the second device to train the second model data before this round of training in order to obtain the first model data after training.

[0068] Specifically, optionally, before the first device sends a third message to the second device, the first device determines the proportion of quantifiable layers in the second model data based on the third information. This third information includes the evaluation loss information from the previous training round, the model accuracy requirement information of the second device, and communication sensitivity information. For example, when the previous evaluation loss is high, the accuracy requirement of the second device is low, and the communication sensitivity of the second device is high, the first device determines the proportion of larger quantifiable layers in the second model data; when the previous evaluation loss is low, the accuracy requirement of the second device is high, and the communication sensitivity of the second device is low, the first device determines the proportion of smaller quantifiable layers in the second model data. It should be understood that different layers in the model data can be understood as different layers in a neural network (NN); where neural network refers to artificial neural network, a mathematical or computational model that mimics the structure and function of biological neural networks, used to estimate or approximate functions, and can perform functions such as classification and regression. Common neural networks include convolutional neural networks and recurrent neural networks.

[0069] The first device prequantizes the data for each layer in the second model data. Optionally, the first device can determine the quantization error and the amount of compression contribution corresponding to each layer of data.

[0070] Optionally, the first device can determine the quantifiable layers in the second model data based on the determined proportion of quantifiable layers. For example, if the second model data has a total of 6 layers, and the proportion of quantifiable layers is 1 / 2, then the first device can determine any 3 layers as quantifiable layers.

[0071] Optionally, the first device can determine the quantizable layers in the second model data based on the determined proportion of quantizable layers and the quantization error corresponding to each layer of data. Specifically, the first device can determine the layers with smaller quantization errors as quantizable layers based on the proportion of quantizable layers.

[0072] Optionally, the first device can determine the quantifiable layers in the second model data based on the proportion of quantifiable layers and the compression contribution of each layer. Specifically, the first device can determine the layers with larger compression contributions as quantifiable layers based on the proportion of quantifiable layers.

[0073] Optionally, the first device can determine the quantizable layers in the second model data based on the determined proportion of quantizable layers, the quantization error corresponding to each layer of data, and the amount of compression contributed by each layer of data.

[0074] The first device obtains quantized second model data, wherein the data corresponding to the quantizable layers in the quantized second model data are quantized, while the data corresponding to the non-quantizable layers are not quantized.

[0075] The model data in the third message can be the quantized second model data, and the third message includes second quantization configuration information. The second quantization configuration information refers to the quantization scheme of the second model data, which may include information such as the number of quantization bits per layer, the offset value of each layer, and the scaling factor. This information can help the second device obtain the second model data through inverse quantization parsing.

[0076] Wireless access network equipment performs hierarchical quantization on the models sent to terminal devices, which can control the accumulation of quantization errors in multiple rounds and reduce the accuracy loss of model data while reducing the amount of transmission.

[0077] It should be understood that the second model data sent by the first device to the second device before this round of training can be unquantized or hierarchically quantized. This application does not limit this.

[0078] 504, The second device receives the third message sent by the first device; and uses local data to train the second model data.

[0079] Specifically, optionally, if the model data in the third message is quantized second model data, then the second device performs inverse quantization parsing on the quantized second model data according to the second quantization configuration information to obtain the second model data; the second device then trains the second model data to obtain the trained first model data.

[0080] 505, the second device sends a fifth message to the first device. This fifth message includes the first information, which includes the evaluation loss information corresponding to this round of training. The fifth message also includes the model name, the size of the model parameter file, and the amount of training data for this round.

[0081] 506. The first device receives a fifth message sent by the second device. Based on the first information in the fifth message, the first device determines the second information. The second information may be a quantization error threshold, which is used by the second device to quantize the first model data.

[0082] Specifically, optionally, the first device can determine the quantization error threshold based on the evaluation loss corresponding to this round of training. The first device can also determine the total evaluation loss based on the evaluation losses corresponding to this round of training in the fifth messages sent by multiple second devices. This total evaluation loss characterizes the training situation in this round, and helps the RAN determine what percentage of second devices can be allowed to perform quantization. For example, when the total evaluation loss is high, such as 10... -2 The magnitude indicates that the training has not converged, but it has a high tolerance for quantization error. Quantization can also be seen as a regularization mechanism, allowing for more secondary devices to perform quantization, and the quantization error threshold can be set to a relatively large value. When the total evaluation loss is low, such as 10... -5At this point, the training is close to or has already converged, and it is more sensitive to errors caused by quantization. Therefore, the proportion of the second device used for quantization should be controlled or reduced, and the quantization error value can be relatively small.

[0083] Specifically, optionally, the first device can determine the quantization error threshold based on the evaluation loss corresponding to this round of training, the accuracy requirements of the second device for model training, and communication sensitivity information. That is, the first device determines the quantization error threshold by comprehensively considering all factors. For example, when the total evaluation loss is high, the accuracy requirements of each second device are low, and the communication sensitivity is high, more second devices should be allowed to perform quantization, and the quantization error threshold can be set to a larger value; while when the total evaluation loss is low, the accuracy requirements of each second device are high, and the communication sensitivity is low, the proportion of second devices performing quantization should be reduced, and the quantization error value can be set to a smaller value.

[0084] Optionally, the quantization error threshold can be initialized to a certain value, such as the quantization error corresponding to a second device in the first round of training.

[0085] 507. The first device sends second information to the second device. That is, the first device informs the second device of the determined quantization error threshold, so that the second device can quantize the first model data after this round of training according to the quantization threshold.

[0086] 508, The second device receives the second information sent by the first device.

[0087] The quantization error threshold is a numerical value. Its function is to allow quantization to be performed and the quantized model data to be uploaded only if the quantization error of the second device's pre-quantization in a certain way is less than this value. For example, if the quantization error threshold is 0.01, quantization is not allowed if the quantization error of the second device's pre-quantization is 0.1, and quantization is allowed if the quantization error is 0.001.

[0088] Pre-quantization refers to the second device performing quantization in advance according to one or more quantization methods to obtain the model error before and after quantization. The error criterion can be mean square error, etc. Here, it is assumed that each second device is allowed to determine its own quantization method.

[0089] 509. The second device quantizes the first model data based on the second information.

[0090] Specifically, the second device pre-quantizes the first model data using a first quantization method and determines a first quantization error based on the quantized and unquantized first model data. If the first quantization error is less than a quantization error threshold, the second device determines to use the first quantization method to quantize the first model data. If the first quantization error is greater than or equal to the quantization error threshold, the second device determines not to use the first quantization method to quantize the first model data. It should be understood that the second device can use multiple quantization methods to quantize the first model data. When the quantization error corresponding to a certain quantization method is less than the quantization error threshold, the first model data can be quantized using that quantization method. If the quantization error corresponding to any quantization method used by the second device is greater than or equal to the quantization error threshold, the second device does not quantize the first model data.

[0091] 510. The second device sends a first message to the first device. This first message includes quantized first model data and first quantization configuration information. The first quantization configuration information is used by the first device to perform inverse quantization parsing on the quantized first model data. The first quantization configuration information includes the number of quantization bits in the model data, whether it is uniform or non-uniform quantization, the quantization zero point, the offset value, and the scaling factor, etc.

[0092] 511. The first device receives the first message sent by the second device and performs dequantization parsing on the quantized first model data in the first message to obtain model data that is convenient for subsequent model aggregation, for example, all of which are parsed into 32-bit floating-point model data.

[0093] Once the first device has collected all the model parameter files uploaded by the second devices participating in this round of training, or after reaching the maximum time limit, it aggregates the parameter files uploaded by the second devices using an aggregation algorithm, such as the federated averaging algorithm, and updates the model parameters.

[0094] The first device determines whether the conditions for stopping training are met. If not, it returns to step 503 to proceed to the next round of training-aggregation; otherwise, it ends the current process.

[0095] In the technical solution provided in this application embodiment, the first device can determine the quantization error threshold based on the evaluation loss, the accuracy requirements of the second device for model training, and the communication sensitivity information, so that the quantization error corresponding to the quantization method adopted by the second device is less than the quantization error threshold. This can control the accumulation of quantization errors in multiple rounds and multiple users during FL training, thereby reducing the accuracy loss of model data.

[0096] like Figure 6 As shown, another method for sending model data according to an embodiment of this application is presented. This method is applied to federated learning and includes:

[0097] 601. The second device sends a registration message to the first device. This registration message, which may be referred to as the second message, contains information about the second device. Optionally, the second message may include model accuracy requirement information and communication sensitivity information of the second device. There may be multiple second devices. The first device may be a wireless access network device, and the second device may be a terminal device; or, the first device may be a network element management system / network management system, and the second device may be a wireless access network device; or, the first device may be a central network data analysis function, and the second device may be a local network data analysis function, etc. This application embodiment does not limit this.

[0098] 602, the first device receives the second message sent by the second device, and selects the second device to participate in model training according to the second message.

[0099] 603, the first device sends a third message to the second device participating in this round of training. The third message includes the second model data and model name before this round of training, which is used by the second device to train the second model data before this round of training in order to obtain the first model data after training.

[0100] Specifically, optionally, before the first device sends a third message to the second device, the first device can determine the proportion of quantifiable layers in the second model data based on third information, wherein the third information includes the evaluation loss information of the previous training round, the model accuracy requirement information of the second device, and the communication sensitivity information. For example, when the evaluation loss of the previous round is high, the accuracy requirement of the second device is low, and the communication sensitivity of the second device is high, the first device determines the proportion of larger quantifiable layers in the second model data; when the evaluation loss of the previous round is low, the accuracy requirement of the second device is high, and the communication sensitivity of the second device is low, the first device determines the proportion of smaller quantifiable layers in the second model data.

[0101] It should be understood that different layers in the model data can be understood as different layers in a neural network (NN); where neural network refers to artificial neural network, which is a mathematical or computational model that imitates the structure and function of biological neural network. It is used to estimate or approximate functions and can perform functions such as classification and regression. Common neural networks include convolutional neural networks and recurrent neural networks.

[0102] The first device prequantizes the data for each layer in the second model data. Optionally, the first device can determine the quantization error and the amount of compression contribution corresponding to each layer of data.

[0103] Optionally, the first device can determine the quantifiable layers in the second model data based on the determined proportion of quantifiable layers. For example, if the second model data has a total of 6 layers, and the proportion of quantifiable layers is 1 / 2, then the first device can determine any 3 layers as quantifiable layers.

[0104] Optionally, the first device can determine the quantizable layers in the second model data based on the determined proportion of quantizable layers and the quantization error corresponding to each layer of data. Specifically, the first device can determine the layers with smaller quantization errors as quantizable layers based on the proportion of quantizable layers.

[0105] Optionally, the first device can determine the quantifiable layers in the second model data based on the proportion of quantifiable layers and the compression contribution of each layer. Specifically, the first device can determine the layers with larger compression contributions as quantifiable layers based on the proportion of quantifiable layers.

[0106] Optionally, the first device can determine the quantizable layers in the second model data based on the determined proportion of quantizable layers, the quantization error corresponding to each layer of data, and the amount of compression contributed by each layer of data.

[0107] The first device obtains quantized second model data, wherein the data corresponding to the quantizable layers in the quantized second model data are quantized, while the data corresponding to the non-quantizable layers are not quantized.

[0108] The model data in the third message can be the quantized second model data, and the third message includes second quantization configuration information. The second quantization configuration information refers to the quantization scheme of the second model data, which may include information such as the number of quantization bits per layer, the offset value of each layer, and the scaling factor. This information can help the second device obtain the second model data through inverse quantization parsing.

[0109] Wireless access network equipment performs hierarchical quantization on the models sent to terminal devices, which can control the accumulation of quantization errors in multiple rounds and reduce the accuracy loss of model data while reducing the amount of transmission.

[0110] It should be understood that the second model data sent by the first device to the second device before this round of training can be unquantized or hierarchically quantized. This application does not limit this.

[0111] 604, The second device receives the third message sent by the first device; and uses local data to train the second model data.

[0112] Specifically, optionally, if the model data in the third message is the quantized second model data, then the second device performs inverse quantization parsing on the quantized second model data according to the second quantization configuration information to obtain the second model data; the second device then trains the second model data to obtain the first model data after this round of training.

[0113] 605. The second device quantizes the first model data using the first quantization method.

[0114] 606. The second device determines the first quantization error based on the quantized first model data and the first model data before quantization.

[0115] 607. The second device sends a fourth message to the first device. The fourth message includes a first quantization error and first information. The first information includes the evaluation loss corresponding to this round of training. The fourth message is used by the first device to determine whether to allow the second device to send the quantized first model data.

[0116] 608, The first device receives the fourth message sent by the second device.

[0117] 609. The first device determines whether to allow the second device to send the quantized first model data based on the first quantization error and the first information.

[0118] Specifically, optionally, the first device determines the proportion of quantifiable second devices based on the evaluation loss corresponding to the current training round sent by different second devices. Specifically, the total evaluation loss is calculated based on the evaluation loss corresponding to different second devices, which can characterize the training status of the current round. For example, when the total evaluation loss is high, it indicates that the training is far from converged, the tolerance for quantization error is high, and the proportion of quantifiable second devices is correspondingly high.

[0119] Specifically, optionally, the first device determines the proportion of quantifiable second devices based on the evaluation loss corresponding to this round of training sent by different second devices, the accuracy requirements of the second devices for model training, and the communication sensitivity information. For example, when the total evaluation loss corresponding to this round of training is high, the accuracy requirements of the second devices are low, and the communication sensitivity of the second devices is high, the first device determines a larger proportion of quantifiable second devices; when the total evaluation loss corresponding to this round of training is low, the accuracy requirements of the second devices are high, and the communication sensitivity of the second devices is low, the first device determines a smaller proportion of quantifiable second devices.

[0120] Optionally, the first device can determine whether to allow the second device to send the quantized first model data based on the proportion of quantizable second devices and the first quantization error. Specifically, the first quantization errors sent by different second devices are sorted from smallest to largest, and the second device with the smaller first quantization error value is determined to be the device allowed to send the quantized first model data based on the proportion of quantizable second devices.

[0121] The first device determines the first quantization error based on the evaluation loss corresponding to this round of training and the first quantization error determined by the second device after quantizing the first model data. It then determines and instructs the second device whether the quantized first model data can be sent, which can avoid the excessive accumulation of the first quantization error of the second device and thus reduce the accuracy loss of the model data.

[0122] Optionally, the first device can determine whether to allow the second devices to send quantized first model data based on the proportion of quantizable second devices, the first quantization error, and a consecutive quantization count threshold. Specifically, the first quantization errors sent by different second devices are sorted from smallest to largest. The second devices with smaller first quantization errors are selected based on the proportion of quantizable second devices. Then, a screening process is performed based on a preset consecutive quantization count threshold, eliminating second devices with consecutive quantization counts exceeding the threshold. The second devices with smaller first quantization errors and consecutive quantization counts not exceeding the threshold are determined as devices allowed to send quantized first model data. If not enough second devices can be found, the proportion of quantizable second devices can be reduced until the requirement is met.

[0123] The first device determines and instructs the second device whether to send the quantized first model data based on the evaluation loss corresponding to this round of training, the first quantization error determined by the second device after quantizing the first model data, and the threshold for the number of consecutive quantizations. This can prevent the second device from quantizing beyond the threshold for the number of consecutive quantizations and reduce the accuracy loss of the model data.

[0124] 610. The first device sends an instruction message to the second device, which indicates whether the second device is allowed to send the quantized first model data.

[0125] 611, The second device receives the instruction information sent by the first device.

[0126] 612. The second device determines whether to send the quantized first model data to the first device based on the instruction information.

[0127] Specifically, optionally, if the indication information allows the second device to send the quantized first model data, then the second device sends the quantized first model data and third quantization configuration information to the first device; if the indication information disallows the second device to send the quantized first model data, then the second device sends the unquantized first model data. The third quantization configuration information is used by the first device to perform dequantization parsing on the quantized first model data. The third quantization configuration information includes the number of quantized bits in the first model data, whether it is uniform or non-uniform quantization, the quantization zero point, the offset value, and the scaling factor, etc.

[0128] If the second device receives the quantized first model data sent by the second device, it can perform dequantization parsing on the quantized first model data to obtain model data that is convenient for subsequent model aggregation, for example, parsing it all as 32-bit floating-point model data. If the second device receives the unquantized first model data sent by the second device, it does not need to perform dequantization parsing.

[0129] Once the first device has collected all the model parameter files uploaded by the second devices participating in this round of training, or after reaching the maximum time limit, it aggregates the parameter files uploaded by the second devices using an aggregation algorithm, such as the federated averaging algorithm, and updates the model parameters.

[0130] The first device determines whether the conditions for stopping training are met. If not, it returns to step 603 to proceed to the next round of training-aggregation; otherwise, it ends the current process.

[0131] like Figure 7 As shown, another method for sending model data according to an embodiment of this application is presented. This method can be applied to federated learning, and the method includes:

[0132] 701. The second device sends a registration message to the first device. This registration message can be referred to as a second message, and it contains information about the second device. Optionally, the second message may include the model accuracy requirement information and communication sensitivity information of the second device. There can be multiple second devices. The first device can be a wireless access network device, and the second device can be a terminal device; or, the first device can be a network element management system / network management system, and the second device can be a wireless access network device; or, the first device can be a central network data analysis function, and the second device can be a local network data analysis function, etc. This application embodiment does not limit this.

[0133] 702, the first device receives the second message sent by the second device, and selects the second device to participate in model training according to the second message.

[0134] 703. The first device sends a third message to the second device participating in this training round. This third message includes quantized second model data and second quantization configuration information. The second model data is the model data before this training round. The third message is used by the second device to train the second model data to obtain the trained first model data. The second quantization configuration information refers to the quantization scheme of the second model data, which may include information such as the number of quantization bits per layer, the offset value of each layer, and the scaling factor. This information can help the second device obtain the second model data through dequantization parsing.

[0135] Specifically, the first device can determine the proportion of quantifiable layers in the second model data based on third information, which includes the evaluation loss information from the previous training round, the model accuracy requirement information of the second device, and communication sensitivity information. For example, when the evaluation loss from the previous round is high, the accuracy requirement of the second device is low, and the communication sensitivity of the second device is high, the first device determines the proportion of larger quantifiable layers in the second model data; when the evaluation loss from the previous round is low, the accuracy requirement of the second device is high, and the communication sensitivity of the second device is low, the first device determines the proportion of smaller quantifiable layers in the second model data.

[0136] It should be understood that different layers in the model data can be understood as different layers in a neural network (NN); where neural network refers to artificial neural network, which is a mathematical or computational model that imitates the structure and function of biological neural network. It is used to estimate or approximate functions and can perform functions such as classification and regression. Common neural networks include convolutional neural networks and recurrent neural networks.

[0137] The first device prequantizes the data for each layer in the second model data. Optionally, the first device can determine the quantization error and the amount of compression contribution corresponding to each layer of data.

[0138] Optionally, the first device can determine the quantifiable layers in the second model data based on the determined proportion of quantifiable layers. For example, if the second model data has a total of 6 layers, and the proportion of quantifiable layers is 1 / 2, then the first device can determine any 3 layers as quantifiable layers.

[0139] Optionally, the first device can determine the quantizable layers in the second model data based on the determined proportion of quantizable layers and the quantization error corresponding to each layer of data. Specifically, the first device can determine the layers with smaller quantization errors as quantizable layers based on the proportion of quantizable layers.

[0140] Optionally, the first device can determine the quantifiable layers in the second model data based on the proportion of quantifiable layers and the compression contribution of each layer. Specifically, the first device can determine the layers with larger compression contributions as quantifiable layers based on the proportion of quantifiable layers.

[0141] Optionally, the first device can determine the quantizable layers in the second model data based on the determined proportion of quantizable layers, the quantization error corresponding to each layer of data, and the amount of compression contributed by each layer of data.

[0142] The first device obtains quantized second model data, wherein the data corresponding to the quantizable layers in the quantized second model data are quantized, while the data corresponding to the non-quantizable layers are not quantized.

[0143] The first device performs hierarchical quantization on the model sent to the second device, which can control the accumulation of quantization errors in multiple rounds and reduce the accuracy loss of model data while reducing the amount of data transmitted.

[0144] It should be understood that the second model data sent by the first device to the second device before this round of training may also be unquantized. This application does not limit this.

[0145] This application provides a communication device 800, which can be applied to... Figure 5 The first device in the method embodiment can also be an implementation Figure 5 Components of the method in the embodiments, such as a chip. Figure 8 The diagram shown illustrates a schematic block diagram of a communication device 800 according to an embodiment of this application. The communication device 800 includes:

[0146] The processing unit 810 is configured to determine second information based on first information, the second information being used by the second device to quantize the first model data, wherein the first information includes the evaluation loss corresponding to the current training round, the second information includes the quantization error threshold, and the first model data is the model data after the current training round.

[0147] Transceiver unit 820 is used to send the second information to the second device;

[0148] The transceiver unit 820 is further configured to receive a first message sent by the second device, the first message including quantized first model data and first quantization configuration information.

[0149] Optionally, the first information may also include the accuracy requirements of the second device for model training and communication sensitivity information.

[0150] Optionally, the transceiver unit 820 is further configured to receive a second message sent by the second device, the second message including the accuracy requirement information and the communication sensitivity information.

[0151] Optionally, the processing unit 810 is further configured to determine the proportion of quantifiable layers in the second model data according to the third information, wherein the third information includes the evaluation loss corresponding to the previous training round, the accuracy requirement information and the communication sensitivity information, wherein the second model data is the model data before the current training round;

[0152] The processing unit 810 is further configured to quantize the second model data according to the proportion of the quantizable layers to obtain quantized second model data;

[0153] The transceiver unit 820 is further configured to send a third message to the second device, the third message including the quantized second model data and the second quantization configuration information, the third message being used by the second device to train the second model data to obtain the first model data.

[0154] This application provides a communication device 900, which can be applied to... Figure 5 The second device in the method embodiment can also be an implementation Figure 5 Components of the method in the embodiments, such as a chip. Figure 9 The diagram shown illustrates a schematic block diagram of a communication device 900 according to an embodiment of this application. The communication device 900 includes:

[0155] The transceiver unit 910 is used to receive second information sent by the first device. The second information is used by the second device to quantize the first model data. The second information includes a quantization error threshold. The first model data is the model data after this round of training.

[0156] Processing unit 920 is used to quantize the first model data according to the second information;

[0157] The transceiver unit 910 is further configured to send a first message to the first device, the first message including quantized first model data and first quantization configuration information.

[0158] Optionally, the processing unit 920 is specifically configured to: quantize the first model data using a first quantization method; determine a first quantization error based on the quantized first model data and the first model data before quantization; and if the first quantization error is less than the quantization error threshold, determine to use the first quantization method to quantize the first model data.

[0159] Optionally, the transceiver unit 910 is further configured to receive a third message sent by the first device, the third message including quantized second model data and second quantization configuration information, wherein the second model data is the model data before this round of training;

[0160] The processing unit 920 is further configured to dequantize and parse the second model data based on the quantized second model data and the second quantization configuration information;

[0161] The processing unit 920 is further configured to train the second model data to obtain the first model data.

[0162] This application provides a communication device 1000, which can be applied to... Figure 6 The first device in the method embodiment can also be an implementation Figure 6 Components of the method in the embodiments, such as a chip. Figure 10 The diagram shown illustrates a schematic block diagram of a communication device 1000 according to an embodiment of this application. The communication device 1000 includes:

[0163] The transceiver unit 1010 is used to receive a fourth message sent by the second device. The fourth message includes a first quantization error and first information. The first quantization error is determined by the second device after quantizing the first model data using a first quantization method. The first information includes the evaluation loss corresponding to this round of training. The first model data is the model data after this round of training.

[0164] The processing unit 1020 is configured to determine, based on the first quantization error and the first information, whether to allow the second device to send the quantized first model data;

[0165] The transceiver unit 1010 is further configured to send indication information to the second device, the indication information being used to indicate whether the second device is allowed to send the quantized first model data.

[0166] Optionally, the processing unit 1020 is specifically used to: determine the proportion of quantizable second devices based on the first information; and determine whether to allow the second device to send the quantized first model data based on the proportion of quantizable second devices, the first quantization error, and the continuous quantization number threshold.

[0167] Optionally, the first information may also include the accuracy requirements of the second device for model training and communication sensitivity information.

[0168] Optionally, the transceiver unit 1010 is further configured to receive a second message sent by the second device, the second message including the accuracy requirement information and the communication sensitivity information.

[0169] Optionally, the processing unit 1020 is further configured to determine the proportion of quantifiable layers in the second model data according to the third information, wherein the third information includes the evaluation loss corresponding to the previous training round, the accuracy requirement information and the communication sensitivity information, wherein the second model data is the model data before the current training round;

[0170] The processing unit 1020 is further configured to quantize the second model data according to the proportion of the quantizable layers to obtain quantized second model data;

[0171] The transceiver unit 1010 is further configured to send a third message to the second device, the third message including the quantized second model data and the second quantization configuration information, the third message being used by the second device to train the second model data to obtain the first model data.

[0172] This application provides a communication device 1100, which can be applied to... Figure 6 The second device in the method embodiment can also be an implementation Figure 6 Components of the method in the embodiments, such as a chip. Figure 11 The diagram shown illustrates a schematic block diagram of a communication device 1100 according to an embodiment of this application. The communication device 1100 includes:

[0173] Processing unit 1110 is used to quantize the first model data using a first quantization method, wherein the first model data is the model data after this round of training;

[0174] The processing unit 1110 is further configured to determine a first quantization error based on the quantized first model data and the first model data before quantization;

[0175] The transceiver unit 1120 is used to send a fourth message to the first device. The fourth message includes the first quantization error and first information. The fourth message is used by the first device to determine whether to allow the second device to send the quantized first model data. The first information includes the evaluation loss corresponding to the current training round.

[0176] The transceiver unit 1120 is further configured to receive indication information sent by the first device, the indication information being used to indicate whether the second device is allowed to send the quantized first model data;

[0177] The processing unit 1110 is further configured to determine, based on the indication information, whether to send the quantized first model data to the first device.

[0178] Optionally, the processing unit 1110 is specifically configured to: if the indication information indicates that the second device is allowed to send the quantized first model data, then send the quantized first model data and third quantization configuration information to the first device; if the indication information indicates that the second device is not allowed to send the quantized first model data, then send the unquantized first model data.

[0179] Optionally, the transceiver unit 1120 is further configured to receive a third message sent by the first device, the third message including quantized second model data and second quantization configuration information, wherein the second model data is the model data before this round of training;

[0180] The processing unit 1110 is further configured to dequantize and parse the second model data based on the quantized second model data and the second quantization configuration information.

[0181] The processing unit 1110 is further configured to train the second model data to obtain the first model data.

[0182] This application provides a communication device 1200, which can be applied to... Figure 7 The first device in the method embodiment can also be an implementation Figure 7 Components of the method in the embodiments, such as a chip. Figure 12 The diagram shown illustrates a schematic block diagram of a communication device 1200 according to an embodiment of this application. The communication device 1200 includes:

[0183] The processing unit 1210 is used to determine the proportion of quantifiable layers in the second model data according to the third information, wherein the third information includes the evaluation loss corresponding to the previous round of training, the accuracy requirement information of the second device for model training, and the communication sensitivity information, and the second model data is the model data before the current round of training.

[0184] The processing unit 1210 is further configured to quantize the second model data according to the proportion of the quantizable layers to obtain quantized second model data;

[0185] The transceiver unit 1220 is used to send a third message to the second device. The third message includes the quantized second model data and the second quantization configuration information. The third message is used by the second device to train the second model data.

[0186] Optionally, the processing unit 1210 is specifically configured to: quantize each layer of data in the second model data; determine the quantization error corresponding to each layer of data and the compression amount contributed by each layer of data; determine the quantizable layers in the second model data based on the proportion of quantizable layers, and / or the quantization error corresponding to each layer of data, and / or the compression amount contributed by each layer of data; and obtain the quantized second model data, wherein the data corresponding to the quantizable layers in the quantized second model data has been quantized, and the data corresponding to the non-quantizable layers has not been quantized.

[0187] This application provides a communication device 1300, such as... Figure 13 As shown, Figure 13 A schematic block diagram of a communication device 1300 according to an embodiment of this application is shown. The communication device 1300 includes:

[0188] The processor 1310 and transceiver 1320 are configured to receive computer code or instructions and transmit them to the processor, and the processor 1310 executes the computer code or instructions to implement the method in the embodiments of this application.

[0189] The aforementioned processor may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The aforementioned processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads messages from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0190] This application also provides a computer-readable storage medium storing a computer program for implementing the methods in the above-described method embodiments. When the computer program is run on a computer, the computer can implement the methods in the above-described method embodiments.

[0191] Furthermore, the term "and / or" in this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship. The term "at least one" in this application can represent "one" and "two or more." For example, at least one of A, B, and C can represent: A existing alone, B existing alone, C existing alone, A and B existing simultaneously, A and C existing simultaneously, C and B existing simultaneously, and A, B, and C existing simultaneously.

[0192] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0193] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0194] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0195] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0196] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0197] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0198] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for transmitting model data, characterized in that, Applied to federated learning, including: The first device determines the second information based on the first information. The second information is used to quantize the first model data when the pre-quantization error is less than the quantization error threshold. The first information includes the evaluation loss corresponding to the current training round, the second information includes the quantization error threshold, the first model data is the model data after the current training round, and the pre-quantization error is obtained by the second device in advance by quantizing the model data. The first device sends the second information to the second device; The first device receives a first message sent by the second device, the first message including quantized first model data and first quantization configuration information; The first device determines the second information based on the first information, including: The total evaluation loss is determined based on the evaluation loss corresponding to this round of training; The quantization error threshold is determined based on the total evaluation loss.

2. The method according to claim 1, characterized in that, The first information also includes the accuracy requirements of the second device for model training and communication sensitivity information.

3. The method according to claim 2, characterized in that, Before the first device determines the second information based on the first information, the method further includes: The first device receives a second message sent by the second device, the second message including the accuracy requirement information and the communication sensitivity information.

4. The method according to claim 2 or 3, characterized in that, Before the first device determines the second information based on the first information, the method further includes: The first device determines the proportion of quantifiable layers in the second model data based on the third information, which includes the evaluation loss corresponding to the previous training round, the accuracy requirement information, and the communication sensitivity information, wherein the second model data is the model data before the current training round. The first device quantizes the second model data according to the proportion of the quantizable layers to obtain quantized second model data; The first device sends a third message to the second device, the third message including the quantized second model data and the second quantization configuration information, the third message being used by the second device to train the second model data to obtain the first model data.

5. A method for transmitting data, characterized in that, Applied to federated learning, including: The second device receives second information sent by the first device. The second information is used by the second device to quantize the first model data. The second information includes a quantization error threshold. The first model data is the model data after this round of training. The second device quantizes the first model data based on the second information; The second device sends a first message to the first device, the first message including quantized first model data and first quantization configuration information; The second device quantizes the first model data based on the second information, including: The second device quantizes the first model data using the first quantization method; The second device determines the first quantization error based on the quantized first model data and the first model data before quantization; If the first quantization error is less than the quantization error threshold, then the second device determines to use the first quantization method to quantize the first model data.

6. The method according to claim 5, characterized in that, Before the second device receives the second information sent by the first device, the method further includes: The second device receives a third message sent by the first device, the third message including quantized second model data and second quantization configuration information, wherein the second model data is the model data before this round of training; The second device dequantizes and parses the second model data based on the quantized second model data and the second quantization configuration information; The second device trains the second model data to obtain the first model data.

7. A method for transmitting data, characterized in that, Applied to federated learning, including: The first device receives a fourth message sent by the second device. The fourth message includes a first quantization error and first information. The first quantization error is determined by the second device after quantizing the first model data using a first quantization method. The first information includes the evaluation loss corresponding to this round of training. The first model data is the model data after this round of training. The first device determines whether to allow the second device to send the quantized first model data based on the first quantization error and the first information; The first device sends an indication message to the second device, the indication message being used to indicate whether the second device is allowed to send the quantized first model data; The first device determines whether to allow the second device to send the quantized first model data based on the first information and the first quantization error, including: The first device determines the proportion of the second device that can be quantified based on the first information; The first device determines whether to allow the second device to send the quantized first model data based on the proportion of the quantizable second device, the first quantization error, and the threshold for the number of consecutive quantizations.

8. The method according to claim 7, characterized in that, The first information also includes the accuracy requirements of the second device for model training and communication sensitivity information.

9. The method according to claim 8, characterized in that, Before the first device receives the fourth message sent by the second device, the method further includes: The first device receives a second message sent by the second device, the second message including the accuracy requirement information and the communication sensitivity information.

10. The method according to claim 8 or 9, characterized in that, Before the first device receives the fourth message sent by the second device, the method further includes: The first device determines the proportion of quantifiable layers in the second model data based on the third information, which includes the evaluation loss corresponding to the previous training round, the accuracy requirement information, and the communication sensitivity information, wherein the second model data is the model data before the current training round. The first device quantizes the second model data according to the proportion of the quantizable layers to obtain quantized second model data; The first device sends a third message to the second device, the third message including the quantized second model data and the second quantization configuration information, the third message being used by the second device to train the second model data to obtain the first model data.

11. A method for transmitting data, characterized in that, Applied to federated learning, including: The second device quantizes the first model data using the first quantization method, wherein the first model data is the model data after this round of training; The second device determines the first quantization error based on the quantized first model data and the first model data before quantization; The second device sends a fourth message to the first device. The fourth message includes the first quantization error and the first information. The fourth message is used by the first device to determine whether to allow the second device to send the quantized first model data. The first information includes the evaluation loss corresponding to this round of training. The second device receives an indication message sent by the first device, the indication message being used to indicate whether the second device is allowed to send the quantized first model data; The second device determines whether to send the quantized first model data to the first device based on the instruction information.

12. The method according to claim 11, characterized in that, The second device determines whether to send the quantized first model data to the first device based on the instruction information, including: If the indication information indicates that the second device is allowed to send the quantized first model data, then the second device sends the quantized first model data and the third quantization configuration information to the first device; If the indication information indicates that the second device is not allowed to send the quantized first model data, then the second device sends the unquantized first model data.

13. The method according to claim 11 or 12, characterized in that, Before the second device quantizes the first model data using the first quantization method, the method further includes: The second device receives a third message sent by the first device, the third message including quantized second model data and second quantization configuration information, wherein the second model data is the model data before this round of training; The second device dequantizes and parses the second model data based on the quantized second model data and the second quantization configuration information; The second device trains the second model data to obtain the first model data.

14. A method for transmitting model data, characterized in that, Applied to federated learning, including: The first device determines the proportion of quantifiable layers in the second model data based on the third information, wherein the third information includes the evaluation loss corresponding to the previous round of training, the accuracy requirements of the second device for model training, and the communication sensitivity information, and the second model data is the model data before the current round of training; The first device quantizes the second model data according to the proportion of the quantizable layers to obtain quantized second model data; The first device sends a third message to the second device, the third message including the quantized second model data and the second quantization configuration information, the third message being used by the second device to train the second model data.

15. The method according to claim 14, characterized in that, The first device quantizes the second model data according to the proportion of the quantizable layers, including: The first device quantizes the data of each layer in the second model data; The first device determines the quantization error corresponding to each layer of data and the amount of compression contributed by each layer of data; The first device determines the quantizable layers in the second model data based on the proportion of the quantizable layers, and / or the quantization error corresponding to each layer of data, and / or the amount of compression contributed by each layer of data; The first device obtains the quantized second model data, wherein the data corresponding to the quantizable layers in the quantized second model data has been quantized, and the data corresponding to the non-quantizable layers has not been quantized.

16. A communication device, characterized in that, Includes units for implementing the function of the method as described in any one of claims 1 to 15.

17. A communication device, characterized in that, include: A processor and a transceiver, the transceiver being configured to receive computer code or instructions and transmit them to the processor, the processor executing the computer code or instructions, as described in any one of claims 1 to 15.

18. A computer-readable storage medium, characterized in that, include: The computer-readable medium stores a computer program; When the computer program is run on a computer, it causes the computer to perform the method according to any one of claims 1 to 15.

Images