Communication method and related device

Abstract

The embodiment of the invention provides a communication method and a related device, which are used for reducing the communication overhead of reporting related information of a first model by a first device and saving communication resources. The method provided by the invention comprises the following steps: a first device receives at least one quantization threshold value from a second device; the first device performs quantization processing on related information of a first model of the first device according to the at least one quantization threshold value; and the first device sends first information to the second device, wherein the first information is used for indicating related information of the quantized first model.

Description

Communication method and related device Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a communication method and related devices. Background Art

[0002] Distributed learning is a method for achieving federated learning. Specifically, multiple node devices use local data to train local models, and a central node device fuses these local models to create a global model. This allows for federated learning while protecting the privacy of user data on the node devices.

[0003] Multiple node devices can train their local models separately to obtain relevant parameters for the local models, such as weight parameters or weight gradients. The node devices then send these parameters to a central node. The central node fuses the local model parameters sent by the multiple node devices to obtain the parameters for the global model, which it then sends to each node device. Each node device can then use the global model parameters to update its local model.

[0004] As can be seen from the above technical solution, each node device sends the relevant parameters of the local model to the central node device separately. This results in a large amount of data reported by the node devices and high communication overhead. Therefore, how to report the relevant parameters of the local model with a low communication overhead by the node devices is an urgent problem to be solved.

[0005] Summary of the Invention

[0006] An embodiment of the present application provides a communication method and related devices for reducing the communication overhead of a first device reporting relevant information of a first model, thereby saving system overhead.

[0007] In a first aspect, the present application provides a communication method. The communication method may be performed by a first apparatus, which may be a communication device or a component (e.g., a chip (system)) in the communication device. The communication method includes:

[0008] The first device receives at least one quantization threshold value from the second device. The first device then quantizes information related to the first model of the first device based on the at least one quantization threshold value. The first device sends first information to the second device, the first information indicating the information related to the first model after quantization. This reduces the communication overhead of the first device reporting the information related to the first model, conserving communication resources.

[0009] Based on the first aspect, in a possible implementation, the relevant information of the first model includes: the output parameters or update parameters of the first model, and the update parameters include the weight gradient or weight parameters of the first model. In this implementation, two possible parameters included in the relevant information of the first model are shown, so that the second device can fuse the training results reported by each device to obtain the relevant information of the global model. In this application, the model on each device can be understood as the same model. In order to distinguish the models on different devices, on the first device side, the model can be called the first model. On the second device side, the model can be called the global model.

[0010] Based on the first aspect, in a possible implementation method, before the first device receives at least one quantization threshold value from the second device, the method also includes: the first device sends second information to the second device; wherein the second information is used to indicate information obtained by processing relevant information of the first model; or, the second information is used to indicate information obtained by processing relevant information obtained by the first device after performing the Mth round of training on the first model, and the relevant information of the first model is relevant information obtained by the first device after performing the Qth round of training on the first model, M is an integer greater than or equal to 1 and less than Q, and Q is an integer greater than 1.

[0011] In this implementation, the first device can send the second information to the second device, thereby facilitating the second device's determination of the at least one quantization threshold value. This facilitates the second device's determination of an appropriate quantization threshold value and facilitates the first device's reasonable quantization processing of the relevant information of the first model. This reduces the overhead of the first device in reporting the relevant information of the first model while ensuring the accuracy of the relevant information reported by the first device.

[0012] Based on the first aspect, in one possible implementation, the relevant information of the first model includes an output parameter of the first model, and the information obtained after processing the relevant information of the first model includes the average of the absolute values ​​of the values ​​of the output parameter of the first model; or, the relevant information of the first model includes an update parameter of the first model, and the information obtained after processing the relevant information of the first model includes the average of the absolute values ​​of the values ​​of the update parameter of the first model. In this implementation, two possible implementations of the relevant information of the first model are shown. The first device can report the average of the absolute values ​​of the values ​​of the output parameter of the first model or the average of the absolute values ​​of the values ​​of the update parameter of the first model to the second device. This facilitates the second device to determine an appropriate quantization threshold value.

[0013] Based on the first aspect, in one possible implementation, the method further includes: the first device receiving third information from the second device, where the third information is used to indicate global information of the first model. In this implementation, the first device can update or train the first model in combination with the global information of the first model.

[0014] Based on the first aspect, in a possible implementation manner, the global information of the first model includes a global output parameter of the first model; or, the global information of the first model includes a global update parameter and / or a global learning rate of the first model.

[0015] In this implementation, two implementations of the global information of the first model are shown. For example, the global information of the first model includes the global output parameters of the first model, thereby facilitating the first device to train the first model using the global output parameters, which is beneficial to improving the training performance of the first model and improving the accuracy of the first model. For example, the global information of the first model includes the global update parameters and / or global learning rate of the first model. This facilitates the first device to update the first model in combination with the global update parameters and / or global learning rate, which is beneficial to improving the accuracy of the first model.

[0016] Based on the first aspect, in a possible implementation method, the relevant information of the first model includes N parameters of the first model, where N is an integer greater than or equal to 1; the first device quantizes the relevant information of the first model of the first device according to at least one quantization threshold value, including: the first device quantizes the N parameters according to at least one quantization threshold value to obtain N parameters after quantization; the first information includes the N parameters after quantization; the first device sends the first information to the second device, including: the first device modulates the N parameters after quantization to obtain N first signals; the first device sends the N first signals to the second device.

[0017] In this implementation, the first information includes N quantized parameters. The first device may quantize the N parameters of the first model, modulate the N quantized parameters, and then transmit the modulated N first signals, thereby transmitting the first information.

[0018] Based on the first aspect, in one possible implementation, at least one quantization threshold value includes a first quantization threshold value and a second quantization threshold value; a first device quantizes N parameters according to the at least one quantization threshold value to obtain the quantized N parameters, including: if the i-th parameter among the N parameters is greater than the first quantization threshold value, the first device quantizes the i-th parameter to a first value, where i is an integer greater than or equal to 1 and less than or equal to N; or if the i-th parameter among the N parameters is less than or equal to the first quantization threshold value and greater than or equal to the second quantization threshold value, the first device quantizes the i-th parameter to a second value; or if the i-th parameter among the N parameters is less than the second quantization threshold value, the first device quantizes the i-th parameter to a third value. In this implementation, a specific quantization process for the i-th parameter by the first device is shown, thereby facilitating implementation of the solution. Furthermore, the at least one quantization threshold value includes multiple quantization threshold values, thereby achieving finer quantization accuracy for the parameters of the first model by the first device, which is beneficial for improving the accuracy of the first device updating the first model and improving the training performance of the first model.

[0019] Based on the first aspect, in a possible implementation method, the first device modulates N parameters after quantization processing to obtain N first signals, including: the first device modulates the i-th parameter after quantization processing to obtain the i-th first signal, and the i-th first signal corresponds to two sequences; when the i-th parameter after quantization processing is the first value, the transmission power of the first sequence of the two sequences sent by the first device is less than the transmission power of the second sequence of the two sequences sent by the first device; when the i-th parameter after quantization processing is the second value, the transmission power of the first sequence of the two sequences sent by the first device is equal to the transmission power of the second sequence of the two sequences sent by the first device; when the i-th parameter after quantization processing is the third value, the transmission power of the first sequence of the two sequences sent by the first device is greater than the transmission power of the second sequence of the two sequences sent by the first device.

[0020] In this implementation, the first device modulates each of the N parameters of the first model onto two sequences. The first device controls the transmit power used for each of the two sequences, thereby facilitating the second device to determine the value of the parameter. The first device does not need to perform channel estimation and equalization, thus eliminating the need for corresponding pilot overhead.

[0021] Based on the first aspect, in one possible implementation, when the i-th parameter after quantization processing is a first value, the first sequence of the two sequences is a non-all-0 sequence, and the second sequence is an all-0 sequence; when the i-th parameter after quantization processing is a second value, both sequences are all-0 sequences; when the i-th parameter after quantization processing is a third value, the first sequence of the two sequences is an all-0 sequence, and the second sequence is a non-all-0 sequence. In this implementation, the first device can carry the i-th parameter after quantization processing through an all-0 sequence and / or a non-all-0 sequence. Under the same total transmission power, it is beneficial for the second device to identify the value of the i-th parameter after quantization processing and improve power utilization efficiency.

[0022] Based on the first aspect, in one possible implementation, the first device sends the first information to the second device, including: the first device sends the first information to the second device L times, where L is an integer greater than or equal to 1. In this implementation, when the number of transmissions L is greater than 1, the first device repeatedly sends the first information, which facilitates the second device to select the decision result that occurs the most times after making a judgment, as the best decision result. This reduces the probability of decision errors and improves model training performance.

[0023] Based on the first aspect, in one possible implementation, the method further includes: the first device receiving first indication information from the second device, the first indication information being used to indicate a number of times L the first device should send the first information to the second device. In this implementation, the first device may receive the number of times indicated by the second device and send the first information according to the number of times. This facilitates the second device determining the number of times according to actual needs, thereby rationally utilizing communication resources.

[0024] Based on the first aspect, in a possible implementation method, the relevant information of the first model includes N parameters of the first model after quantization error compensation. The N parameters after quantization error compensation are obtained by the first device performing error compensation on the N parameters according to the quantization errors corresponding to the N parameters obtained by the first device performing the Q-1th round of training on the first model. The quantization error corresponding to the i-th parameter among the N parameters is determined based on the i-th parameter obtained by the first device performing the Q-1th round of training on the first model and after quantization error compensation. i is an integer greater than or equal to 1 and less than or equal to N, N is an integer greater than or equal to 1, and Q is an integer greater than 1.

[0025] In this implementation, the first device may first perform quantization error compensation on the N parameters of the first model, and then quantize the N parameters after quantization error compensation according to the at least one quantization threshold value. This helps improve the accuracy of the first device in updating the first model and improves the training performance of the first model.

[0026] Based on the first aspect, in a possible implementation method, the relevant information of the first model includes N parameters of the first model obtained through sparse processing; the N parameters of the first model obtained through sparse processing are N parameters selected by the first device from the K parameters of the first model according to a common sparse mask, and the K parameters of the first model are parameters obtained by the first device after one round of training of the first model, K is an integer greater than or equal to N, K is an integer greater than or equal to 1, and N is an integer greater than or equal to 1.

[0027] In this implementation, the first device may first select N parameters from the K parameters of the first model using a common sparse mask, and then quantize the N parameters according to the at least one quantization threshold value. This helps reduce the overhead incurred by the first device in reporting the parameters of the first model.

[0028] Based on the first aspect, in one possible implementation, the public sparse mask is a bit sequence comprising K bits, each of which corresponds one-to-one to K parameters. When the value of one of the K bits is 0, it indicates that the first device does not select the parameter corresponding to that bit; when the value of one of the K bits is 1, it indicates that the first device selects the parameter corresponding to that bit. This implementation provides a specific form of the public sparse mask, allowing the first device to select parameters based on the values ​​of the bits in the bit sequence, resulting in simple and convenient operation. This reduces the overhead of the first device in reporting the parameters of the first model and reduces communication resource usage.

[0029] Based on the first aspect, in one possible implementation, a public sparsity mask is determined by the first device based on a sparsity ratio and a pseudo-random number, and the sparsity ratio is indicated by the second device to the first device. This implementation provides a method for generating the public sparsity mask, facilitating implementation of the solution. This allows the first device to report some parameters of the first model based on the public sparsity mask, reducing the overhead incurred by the first device in reporting the parameters of the first model.

[0030] Based on the first aspect, in one possible implementation, the method further includes: the first device receiving second indication information from the second device, where the second indication information is used to indicate a common sparse mask. In this implementation, the first device is facilitated to select N parameters from the K parameters of the first model based on the common sparse mask. This helps reduce the overhead incurred by the first device in reporting the parameters of the first model.

[0031] Based on the first aspect, in a possible implementation manner, the method further includes: the first device sends third indication information to the second device, where the third indication information is used to indicate the indexes of N parameters whose corresponding values ​​among the K parameters are the largest in absolute value.

[0032] In this implementation, the first device can indicate to the second device the indices of the N parameters with the largest absolute values ​​among its K parameters. This facilitates the second device in determining an appropriate common sparse mask. The third indication information is used to indicate the indices of the N parameters with the largest absolute values ​​among the K parameters. This facilitates the first device to subsequently prioritize feedback on parameters with larger variations, thereby improving the accuracy and performance of model training.

[0033] Based on the first aspect, in one possible implementation, the first model is a neural network model, and the relevant information of the first model includes relevant parameters of neurons in layer P of the neural network model, where P is an integer greater than or equal to 1. In this implementation, the first device may report parameters of one or more layers of the neural network model. That is, the first device reports the parameters of the neural network model in units of layers, thereby facilitating the first device to accurately report the parameters of each layer and improving the accuracy of model training.

[0034] A second aspect of the present application provides a communication method, which may be performed by a second apparatus. The second apparatus may be a communication device or a component (e.g., a chip (system)) in the communication device. The communication method includes:

[0035] The second device sends at least one quantization threshold value to the first device, where the at least one quantization threshold value is used to quantize information related to the first model of the first device. The second device then receives first information sent from the first device, where the first information indicates the information related to the first model after quantization. The above technical solution can be seen to reduce the communication overhead of the first device reporting information related to the first model, thereby conserving communication resources.

[0036] Based on the second aspect, in a possible implementation, the relevant information of the first model includes: the output parameters or update parameters of the first model, and the update parameters include the weight gradient or weight parameters of the first model. In this implementation, two possible parameters included in the relevant information of the first model are shown, so that the second device can fuse the training results reported by each device to obtain the relevant information of the global model. In this application, the models on each device can be understood as the same model. In order to distinguish the models on different devices, on the first device side, the model can be called the first model, and on the second device side, the model can be called the global model.

[0037] Based on the second aspect, in one possible implementation, the method further includes:

[0038] The second device receives second information from the first device; wherein the second information is used to indicate information obtained by processing relevant information of the first model; or, the second information is used to indicate information obtained by the first device after performing the Mth round of training on the first model and processing, and the relevant information of the first model is the relevant information obtained by the first device after performing the Qth round of training on the first model, M is an integer greater than or equal to 1 and less than Q, and Q is an integer greater than 1; the second device determines at least one quantization threshold value based on the second information.

[0039] In this implementation, the second device receives the second information from the first device, enabling the second device to determine at least one quantization threshold value based on the second information. This facilitates the second device to determine an appropriate quantization threshold value, facilitating the first device to perform reasonable quantization processing on the relevant information of the first model. This reduces the overhead of the first device reporting the relevant information of the first model while ensuring the accuracy of the relevant information reported by the first device.

[0040] Based on the second aspect, in one possible implementation, the relevant information of the first model includes the output parameter of the first model, and the information obtained after processing the relevant information of the first model includes the average of the absolute values ​​of the values ​​of the output parameter of the first model; or the relevant information of the first model includes the update parameter of the first model, and the information obtained after processing the relevant information of the first model includes the average of the absolute values ​​of the values ​​of the update parameter of the first model. In this implementation, two possible implementations of the relevant information of the first model are shown. The second device can receive the average of the absolute values ​​of the values ​​of the output parameter of the first model or the average of the absolute values ​​of the values ​​of the update parameter of the first model from the first device. This facilitates the second device to determine an appropriate quantization threshold value.

[0041] Based on the second aspect, in a possible implementation, the method also includes: the second device receives third information from the third device; wherein the third information is used to indicate information obtained by processing relevant information of the second model of the third device; or, the third information is used to indicate information obtained by the third device after performing the S-th round of training on the second model and processing, and the relevant information of the second model is the relevant information obtained by the third device after performing the R-th round of training on the second model, S is an integer greater than or equal to 1 and less than R, and R is an integer greater than 1; the second device determines at least one quantization threshold value based on the second information, including: the second device determines at least one quantization threshold value based on the second information and the third information.

[0042] In this implementation, the second device may also receive third information from the third device and determine at least one quantization threshold value based on the second and third information. This facilitates the second device to determine an appropriate quantization threshold value, thereby reducing the overhead of the first device reporting the relevant information of the first model while ensuring the accuracy of the relevant information of the first model reported by the first device.

[0043] Based on the second aspect, in one possible implementation, the method further includes: a second device determining global information of the first model based on the first information; and the second device sending fourth information to the first device, where the fourth information indicates the global information of the first model. In this implementation, the second device may determine the global information of the first model based on the first information and send the global information of the first model to the first device. This enables the first device to update or train the first model.

[0044] Based on the second aspect, in a possible implementation manner, the global information of the first model includes a global output parameter of the first model; or, the global information of the first model includes a global update parameter and / or a global learning rate of the first model.

[0045] In this implementation, two implementations of the global information of the first model are shown. For example, the global information of the first model includes the global output parameters of the first model, thereby facilitating the first device to train the first model using the global output parameters, which is beneficial to improving the training performance of the first model and improving the accuracy of the first model. For example, the global information of the first model includes the global update parameters and / or global learning rate of the first model. This facilitates the first device to update the first model in combination with the global update parameters and / or global learning rate, which is beneficial to improving the accuracy of the first model.

[0046] Based on the second aspect, in one possible implementation, the method further includes: the second device receiving fifth information from the third device, the fifth information being used to indicate relevant information of the second model of the third device; and the second device determining global information of the first model based on the first information, including: the second device determining the global information of the first model based on the first information and the fifth information. In this implementation, the second device may also receive the fifth information from the third device, and determine the global information of the first model in combination with the first information and the fifth information. This facilitates improving the accuracy of the second device's determination of the global information of the first model and improving the accuracy of model updates.

[0047] Based on the second aspect, in a possible implementation method, the relevant information of the first model includes N parameters of the first model, where N is an integer greater than or equal to 1; the relevant information of the second model includes N parameters of the second model; the first information includes the N parameters of the first model after quantization processing; the second device receives the first information sent from the first device, including: the second device receives N first signals from the first device, the N first signals carry the N parameters of the first model after quantization processing, and the N first signals correspond one-to-one to the N parameters of the first model after quantization processing; the fifth information includes the N parameters of the second model after quantization processing; the second device receives the fifth information from the third device, including: the second device receives N second signals from the third device, the N second signals carry the N parameters of the second model after quantization processing, and the N second signals correspond one-to-one to the N parameters of the second model after quantization processing; the second device determines the global information of the first model based on the first information and the fifth information, including: the second device determines the global information of the first model based on the N first signals and the N second signals.

[0048] Based on the second aspect, in one possible implementation, the i-th first signal among N first signals corresponds to a first sequence and a second sequence, the i-th second signal among N second signals corresponds to a third sequence and a fourth sequence, the time-frequency resources used by the first device to send the first sequence are the same as the time-frequency resources used by the third device to send the third sequence, and the time-frequency resources used by the first device to send the second sequence are the same as the time-frequency resources used by the third device to send the fourth sequence; the global information of the first model includes N global parameters of the first model; i is an integer greater than or equal to 1 and less than or equal to N; the second device determines the global information of the first model based on the N first signals and the N second signals, including: the second device determines the sum of first signal energies of the first and third sequences received by the second device; the second device determines the sum of second signal energies of the second and fourth sequences received by the second device; and the second device determines the i-th global parameter among the N global parameters based on the sum of the first signal energies and the sum of the second signal energies. It can be seen that the second device can determine the i-th global parameter by receiving the signal energies of the two sequences corresponding to the i-th first signal and the signal energies of the two sequences corresponding to the i-th second signal received by the second device. This supports the second device to achieve non-coherent reception of superimposed transmission of multi-user air signals and achieve robustness against fading channels.

[0049] Based on the second aspect, in a possible implementation method, the second device determines the i-th global parameter among N global parameters based on the first signal energy sum and the second signal energy sum, including: if the sum of the first signal energy sum and the decision threshold value is less than the second signal energy sum, the second device determines the value of the i-th global parameter to be the first value; or, if the sum of the first signal energy sum and the decision threshold value is greater than or equal to the second signal energy sum, and the sum of the second signal energy sum and the decision threshold value is greater than or equal to the first signal energy sum, the second device determines the value of the i-th global parameter to be the second value; or, if the sum of the second signal energy sum and the decision threshold value is less than the first signal energy sum, the second device determines the value of the i-th global parameter to be the third value.

[0050] This implementation illustrates the process by which the second device determines the i-th global parameter. As can be seen above, the three possible conditions for the first signal energy sum and the second signal energy sum correspond to three possible determination results for the i-th global parameter. This enables accurate determination of the i-th global parameter, improving the accuracy of the first device's update of the first model and enhancing the training performance of the first model.

[0051] Based on the second aspect, in one possible implementation, the method further includes: the second device sending first indication information to the first device, where the first indication information is used to instruct the first device to send the first information to the second device a number of times, L, where L is an integer greater than or equal to 1. In this implementation, the second device instructs the first device to send the first information a number of times, so that the first device sends the first information according to the number of times. This facilitates the second device determining the number of times according to actual needs, thereby rationally utilizing communication resources.

[0052] Based on the second aspect, in one possible implementation, the method further includes: the second device sending second indication information to the first device, the second indication information being used to indicate a common sparse mask, and the common sparse mask being used to instruct the first device to report a portion of parameters obtained by the first device training the first model. In this implementation, the second device sends the second indication information to the first device, the second indication information being used to indicate the common sparse mask. This facilitates the first device selecting N parameters from the K parameters of the first model based on the common sparse mask. This helps reduce the overhead incurred by the first device in reporting the parameters of the first model.

[0053] Based on the second aspect, in a possible implementation, the method further includes: the second device receives third indication information from the first device, the third indication information is used to indicate the index of the N parameters with the largest absolute values ​​among the K parameters obtained by the first device through one round of training of the first model; the second device receives fourth indication information from the third device, the fourth indication information is used to indicate the index of the N parameters with the largest absolute values ​​among the K parameters of the second model of the third device, the K parameters of the second model are the K parameters obtained by the third device through one round of training of the second model; the second device determines a common sparse mask based on the third indication information and the fourth indication information. In this implementation, each device indicates the index of the parameter with the largest absolute value among its K parameters, which is conducive to the second device determining an appropriate common sparse mask based on the third indication information and the fourth indication information. In this way, the first device can give priority to feedback parameters with larger changes based on the common sparse mask, thereby improving the accuracy of model training and improving the performance of model training.

[0054] A third aspect of the present application provides a communication method, which may be performed by a first apparatus, which may be a communication device or a component (e.g., a chip (system)) in the communication device. The communication method includes:

[0055] The first device sends first indication information to the second device. The first indication information is used to indicate the indexes of the N parameters with the largest absolute values ​​among the K parameters of the first model of the first device. The K parameters of the first model are the K parameters obtained by the first device through one round of training of the first model, K is an integer greater than or equal to N, K is an integer greater than or equal to 1, and N is an integer greater than or equal to 1. Then, the first device receives second indication information from the second device. The second indication information is used to indicate a common sparse mask, which is determined by the second device based on the first indication information; the common sparse mask is used to instruct the first device to report some parameters obtained by the first device through training the first model.

[0056] In the above technical solution, the first device can report first indication information to the second device, thereby indicating the indexes of the N parameters with the largest absolute values ​​among the K parameters of the first model. This enables the second device to determine a suitable common sparse mask based on the first indication information. The first device receives second indication information from the second device. The second indication information is used to indicate the common sparse mask. This facilitates the first device to give priority to feedback of parameters with larger changes based on the common sparse mask. This is conducive to reducing the overhead generated by the first device reporting the parameters of the first model, while also improving the accuracy of model training and improving the performance of model training.

[0057] A fourth aspect of the present application provides a communication method, which may be performed by a second apparatus, the second apparatus being a communication device or a component (e.g., a chip (system)) in the communication device. The communication method includes:

[0058] The second device receives first indication information from the first device, the first indication information being used to indicate the indices of the N parameters with the largest absolute values ​​among the K parameters of the first model of the first device, where the K parameters of the first model are the K parameters obtained by the first device through one round of training of the first model, where K is an integer greater than or equal to N, K is an integer greater than or equal to 1, and N is an integer greater than or equal to 1. The second device determines a common sparse mask based on the first indication information, the common sparse mask being used to instruct the first device to report some parameters obtained by the first device through training of the first model. The second device then sends second indication information to the first device, the second indication information being used to indicate the common sparse mask.

[0059] In the above technical solution, the second device receives first indication information from the first device. This first indication information is used to index the N parameters with the largest absolute values ​​among the K parameters of the first model. This allows the second device to determine an appropriate common sparse mask based on the first indication information. This allows the first device to prioritize feedback of parameters with larger variations based on the common sparse mask, reducing the overhead incurred by the first device in reporting the parameters of the first model. This also improves the accuracy and performance of model training.

[0060] Based on the fourth aspect, in a possible implementation method, the method also includes: the second device receives third indication information from the third device, the third indication information is used to indicate the indexes of the N parameters with the largest absolute values ​​among the K parameters of the second model of the third device, and the K parameters of the second model are the K parameters obtained by the second device through one round of training of the second model; the second device determines the common sparse mask based on the first indication information, including: the second device determines the common sparse mask based on the first indication information and the third indication information.

[0061] In this implementation, the second device may also determine the common sparse mask in combination with the third indication information reported by the third device, thereby facilitating the second device to determine an appropriate common sparse mask for the first device. This allows the first device to prioritize feedback of parameters with larger variations based on the common sparse mask, thereby improving the accuracy and performance of model training.

[0062] A fifth aspect of the present application provides a first device, including:

[0063] The transceiver module is used to receive at least one quantization threshold value from the second device; the processing module is used to quantize the relevant information of the first model of the first device according to the at least one quantization threshold value; the transceiver module is also used to send first information to the second device, where the first information is used to indicate the relevant information of the first model after quantization processing.

[0064] Based on the fifth aspect, in a possible implementation, the relevant information of the first model includes: output parameters or update parameters of the first model, and the update parameters include weight gradients or weight parameters of the first model.

[0065] Based on the fifth aspect, in a possible implementation method, the transceiver module is also used to: send second information to the second device; wherein the second information is used to indicate information obtained by processing relevant information of the first model; or, the second information is used to indicate information obtained by processing relevant information obtained by the first device performing the Mth round of training on the first model, and the relevant information of the first model is relevant information obtained by the first device performing the Qth round of training on the first model, M is an integer greater than or equal to 1 and less than Q, and Q is an integer greater than 1.

[0066] Based on the fifth aspect, in one possible implementation, the relevant information of the first model includes the output parameters of the first model, and the information obtained after processing the relevant information of the first model includes the average value of the absolute values ​​of the output parameters of the first model; or, the relevant information of the first model includes the updated parameters of the first model, and the information obtained after processing the relevant information of the first model includes the average value of the absolute values ​​of the updated parameters of the first model.

[0067] Based on the fifth aspect, in a possible implementation manner, the transceiver module is further used to: receive third information from the second device, where the third information is used to indicate global information of the first model.

[0068] Based on the fifth aspect, in one possible implementation, the global information of the first model includes the global output parameters of the first model; or, the global information of the first model includes the global update parameters and / or global learning rate of the first model.

[0069] Based on the fifth aspect, in a possible implementation method, the relevant information of the first model includes N parameters of the first model, where N is an integer greater than or equal to 1; the processing module is specifically used to: quantize the N parameters according to at least one quantization threshold value to obtain N parameters after quantization; the first information includes the N parameters after quantization; the transceiver module is specifically used to: modulate the N parameters after quantization to obtain N first signals; and send the N first signals to the second device.

[0070] Based on the fifth aspect, in a possible implementation, the at least one quantization threshold value includes a first quantization threshold value and a second quantization threshold value; and the processing module is specifically configured to:

[0071] If the i-th parameter among the N parameters is greater than the first quantization threshold value, the i-th parameter is quantized to the first value, where i is an integer greater than or equal to 1 and less than or equal to N; or, if the i-th parameter among the N parameters is less than or equal to the first quantization threshold value and greater than or equal to the second quantization threshold value, the i-th parameter is quantized to the second value; or, if the i-th parameter among the N parameters is less than the second quantization threshold value, the i-th parameter is quantized to the third value.

[0072] Based on the fifth aspect, in a possible implementation method, the transceiver module is specifically used to: modulate the i-th parameter after quantization processing to obtain the i-th first signal, and the i-th first signal corresponds to two sequences; when the i-th parameter after quantization processing is a first value, the transmission power of the first sequence of the two sequences sent by the first device is less than the transmission power of the second sequence of the two sequences sent by the first device; when the i-th parameter after quantization processing is a second value, the transmission power of the first sequence of the two sequences sent by the first device is equal to the transmission power of the second sequence of the two sequences sent by the first device; when the i-th parameter after quantization processing is a third value, the transmission power of the first sequence of the two sequences sent by the first device is greater than the transmission power of the second sequence of the two sequences sent by the first device.

[0073] Based on the fifth aspect, in one possible implementation, when the i-th parameter after quantization processing is a first value, the first sequence of the two sequences is a non-all-0 sequence, and the second sequence is an all-0 sequence; when the i-th parameter after quantization processing is a second value, both sequences are all-0 sequences; when the i-th parameter after quantization processing is a third value, the first sequence of the two sequences is an all-0 sequence, and the second sequence is a non-all-0 sequence.

[0074] Based on the fifth aspect, in a possible implementation manner, the transceiver module is specifically used to: send the first information to the second device L times, where L is an integer greater than or equal to 1.

[0075] Based on the fifth aspect, in a possible implementation, the transceiver module is further used to: receive first indication information from the second device, where the first indication information is used to indicate a number L of times the first device sends the first information to the second device.

[0076] Based on the fifth aspect, in a possible implementation method, the relevant information of the first model includes N parameters of the first model after quantization error compensation. The N parameters after quantization error compensation are obtained by the first device performing error compensation on the N parameters according to the quantization errors corresponding to the N parameters obtained by the first device performing the Q-1th round of training on the first model, where Q is an integer greater than 1, and the quantization error corresponding to the i-th parameter among the N parameters is determined based on the i-th parameter obtained by the first device performing the Q-1th round of training on the first model and after quantization error compensation.

[0077] Based on the fifth aspect, in a possible implementation method, the relevant information of the first model includes N parameters of the first model obtained through sparse processing; the N parameters of the first model obtained through sparse processing are N parameters selected by the first device from the K parameters of the first model according to a common sparse mask, and the K parameters of the first model are parameters obtained by the first device performing the Qth round of training on the first model, where K is an integer greater than or equal to N, and K is an integer greater than or equal to 1.

[0078] Based on the fifth aspect, in a possible implementation method, the common sparse mask is a bit sequence, the bit sequence includes K bits, and the K bits correspond one-to-one to K parameters; when the value of one of the K bits is 0, it is used to indicate that the first device does not select the parameter corresponding to the bit; when the value of one of the K bits is 1, it is used to indicate that the first device selects the parameter corresponding to the bit.

[0079] Based on the fifth aspect, in a possible implementation, the public sparse mask is determined by the first device according to the sparse ratio and the pseudo-random number, and the sparse ratio is indicated by the second device to the first device.

[0080] Based on the fifth aspect, in a possible implementation manner, the transceiver module is further used to: receive second indication information from a second device, where the second indication information is used to indicate a common sparse mask.

[0081] Based on the fifth aspect, in a possible implementation, the transceiver module is further used to: send third indication information to the second device, where the third indication information is used to indicate the indexes of the N parameters whose corresponding values ​​among the K parameters are the largest in absolute value.

[0082] Based on the fifth aspect, in a possible implementation, the first model is a neural network model, and the relevant information of the first model includes relevant parameters of neurons in layer P of the neural network model, where P is an integer greater than or equal to 1.

[0083] A sixth aspect of the present application provides a second device, including:

[0084] The transceiver module is used to send at least one quantization threshold value to the first device, and the at least one quantization threshold value is used to quantize the relevant information of the first model of the first device; receive first information sent from the first device, and the first information is used to indicate the relevant information of the first model after quantization.

[0085] Based on the sixth aspect, in a possible implementation, the relevant information of the first model includes: output parameters or update parameters of the first model, and the update parameters include weight gradients or weight parameters of the first model.

[0086] Based on the sixth aspect, in a possible implementation method, the transceiver module is also used to: receive second information from the first device; wherein the second information is used to indicate information obtained by processing relevant information of the first model; or, the second information is used to indicate information obtained by the first device performing the Mth round of training on the first model and processing it, and the relevant information of the first model is the relevant information obtained by the first device performing the Qth round of training on the first model, M is an integer greater than or equal to 1 and less than Q, and Q is an integer greater than 1; the second device also includes a processing module, and the processing module is used to determine at least one quantization threshold value based on the second information.

[0087] Based on the sixth aspect, in one possible implementation, the relevant information of the first model includes the output parameters of the first model, and the information obtained after processing the relevant information of the first model includes the average value of the absolute values ​​of the output parameters of the first model; or, the relevant information of the first model includes the updated parameters of the first model, and the information obtained after processing the relevant information of the first model includes the average value of the absolute values ​​of the updated parameters of the first model.

[0088] Based on the sixth aspect, in a possible implementation method, the transceiver module is also used to: receive third information from a third device; wherein the third information is used to indicate information obtained by processing relevant information of the second model of the third device; or, the third information is used to indicate information obtained by the third device performing the Sth round of training on the second model and processing it, and the relevant information of the second model is the relevant information obtained by the third device performing the Rth round of training on the second model, S is an integer greater than or equal to 1 and less than R, and R is an integer greater than 1; the processing module is used to determine at least one quantization threshold value based on the second information and the third information.

[0089] Based on the sixth aspect, in a possible implementation method, the processing module is also used to: determine the global information of the first model based on the first information; the transceiver module is also used to: send fourth information to the first device, and the fourth information is used to indicate the global information of the first model.

[0090] Based on the sixth aspect, in one possible implementation, the global information of the first model includes the global output parameters of the first model; or, the global information of the first model includes the global update parameters and / or global learning rate of the first model.

[0091] Based on the sixth aspect, in a possible implementation method, the transceiver module is also used to: receive fifth information from a third device, the fifth information is used to indicate relevant information of the second model of the third device; the processing module is specifically used to: determine the global information of the first model based on the first information and the fifth information.

[0092] Based on the sixth aspect, in a possible implementation method, the relevant information of the first model includes N parameters of the first model, where N is an integer greater than or equal to 1; the relevant information of the second model includes N parameters of the second model; the first information includes the N parameters of the first model after quantization processing; the transceiver module is specifically used to: receive N first signals from the first device, the N first signals carry the N parameters of the first model, and the N first signals correspond one-to-one to the N parameters of the first model after quantization processing; the fifth information includes the N parameters of the second model after quantization processing; the transceiver module is specifically used to: receive N second signals from the third device, the N second signals carry the N parameters of the second model after quantization processing, and the N second signals correspond one-to-one to the N parameters of the second model after quantization processing; the processing module is specifically used to: determine the global information of the first model based on the N first signals and the N second signals.

[0093] Based on the sixth aspect, in a possible implementation method, the i-th first signal among the N first signals corresponds to the first sequence and the second sequence, the i-th second signal among the N second signals corresponds to the third sequence and the fourth sequence, the time-frequency resources used by the first device to send the first sequence are the same as the time-frequency resources used by the third device to send the third sequence, and the time-frequency resources used by the first device to send the second sequence are the same as the time-frequency resources used by the third device to send the fourth sequence; the global information of the first model includes N global parameters of the first model; i is an integer greater than or equal to 1 and less than or equal to N; the processing module is specifically used to: determine the sum of the first signal energies of the first sequence and the third sequence received by the second device; determine the sum of the second signal energies of the second sequence and the fourth sequence received by the second device; and determine the i-th global parameter among the N global parameters based on the first signal energy sum and the second signal energy sum.

[0094] Based on the sixth aspect, in a possible implementation method, the processing module is specifically used to: if the sum of the first signal energy sum and the decision threshold value is less than the second signal energy sum, then determine the value of the i-th global parameter to be the first value; or, if the sum of the first signal energy sum and the decision threshold value is greater than or equal to the second signal energy sum, and the sum of the second signal energy sum and the decision threshold value is greater than or equal to the first signal energy sum, then determine the value of the i-th global parameter to be the second value; or, if the sum of the second signal energy sum and the decision threshold value is less than the first signal energy sum, then determine the value of the i-th global parameter to be the third value.

[0095] Based on the sixth aspect, in a possible implementation method, the transceiver module is also used to: send first indication information to the first device, the first indication information is used to indicate the number of times L that the first device sends the first information to the second device, where L is an integer greater than or equal to 1.

[0096] Based on the sixth aspect, in a possible implementation method, the transceiver module is also used to: send second indication information to the first device, the second indication information is used to indicate a common sparse mask, and the common sparse mask is used to instruct the first device to report part of the parameters obtained by the first device training the first model.

[0097] Based on the sixth aspect, in a possible implementation method, the transceiver module is also used to: receive third indication information from the first device, the third indication information is used to indicate the indexes of the N parameters with the largest absolute values ​​among the K parameters obtained by the first device through one round of training of the first model; receive fourth indication information from the third device, the fourth indication information is used to indicate the indexes of the N parameters with the largest absolute values ​​among the K parameters of the second model of the third device, and the K parameters of the second model are the K parameters obtained by the third device through one round of training of the second model; the second device also includes a processing module, and the processing module is further used to: determine a common sparse mask based on the third indication information and the fourth indication information.

[0098] A seventh aspect of the present application provides a first device, including:

[0099] A transceiver module is used to send first indication information to a second device, where the first indication information is used to indicate the indexes of the N parameters with the largest absolute values ​​among the K parameters of the first model of the first device, where the K parameters of the first model are the K parameters obtained by the first device through one round of training of the first model, where K is an integer greater than or equal to N, K is an integer greater than or equal to 1, and N is an integer greater than or equal to 1; and receive second indication information from the second device; the second indication information is used to indicate a common sparse mask, which is determined by the second device based on the first indication information; and the common sparse mask is used to instruct the first device to report some parameters obtained by the first device through training the first model.

[0100] An eighth aspect of the present application provides a second device, including:

[0101] a transceiver module, configured to receive first indication information from a first device, the first indication information being used to indicate the indexes of N parameters having the largest absolute values ​​among K parameters of a first model of the first device, where the K parameters of the first model are K parameters obtained by the first device through one round of training of the first model, where K is an integer greater than or equal to N, K is an integer greater than or equal to 1, and N is an integer greater than or equal to 1;

[0102] a processing module, configured to determine a common sparse mask according to the first indication information, where the common sparse mask is used to instruct the first device to report part of the parameters obtained by training the first model by the first device;

[0103] The transceiver module is further used to send second indication information to the first device, where the second indication information is used to indicate a common sparse mask.

[0104] Based on the eighth aspect, in one possible implementation, the transceiver module is further configured to:

[0105] receiving third indication information from a third device, the third indication information being used to indicate the indices of N parameters with the largest absolute values ​​among K parameters of the second model of the third device, where the K parameters of the second model are K parameters obtained by the third device through one round of training of the second model;

[0106] The processing module is specifically used to:

[0107] A common sparse mask is determined according to the first indication information and the third indication information.

[0108] With respect to the fifth or seventh aspect above, the first device may be a communication device, the transceiver module may be a transceiver, or an input / output interface; and the processing module may be a processor.

[0109] In another implementation, the first device is a chip, chip system, or circuit configured in a communication device. When the first device is a chip, chip system, or circuit configured in a communication device, the transceiver module may be an input / output interface, interface circuit, output circuit, input circuit, pin, or related circuit on the chip, chip system, or circuit; and the processing module may be a processor, processing circuit, or logic circuit.

[0110] With respect to the sixth or eighth aspect above, the second device may be a communication device, the transceiver module may be a transceiver, or an input / output interface; and the processing module may be a processor.

[0111] In another implementation, the second device is a chip, chip system, or circuit configured in a communication device. When the second device is a chip, chip system, or circuit configured in a communication device, the transceiver module may be an input / output interface, interface circuit, output circuit, input circuit, pin, or related circuit on the chip, chip system, or circuit; and the processing module may be a processor, processing circuit, or logic circuit.

[0112] In a ninth aspect, the present application provides a first apparatus, comprising: a processor and a memory. The memory stores a computer program or computer instructions, and the processor is configured to call and execute the computer program or computer instructions stored in the memory, so that the processor implements any one of the implementations of the first or third aspects.

[0113] Optionally, the first device further includes a transceiver, and the processor is used to control the transceiver to transmit and receive signals.

[0114] In a tenth aspect, the present application provides a second apparatus, comprising: a processor and a memory. The memory stores a computer program or computer instructions, and the processor is configured to call and execute the computer program or computer instructions stored in the memory, so that the processor implements any one of the implementations of the second aspect or the fourth aspect.

[0115] Optionally, the second device further includes a transceiver, and the processor is used to control the transceiver to transmit and receive signals.

[0116] In an eleventh aspect, the present application provides a first device, comprising a processor and an interface circuit, wherein the processor is configured to communicate with other devices via the interface circuit and execute the method described in the first or third aspect. The processor comprises one or more processors.

[0117] In a twelfth aspect, the present application provides a second device, comprising a processor and an interface circuit, wherein the processor is configured to communicate with other devices via the interface circuit and execute the method described in the second or fourth aspect. The processor may comprise one or more processors.

[0118] In a thirteenth aspect, the present application provides a first device, comprising a processor, configured to be connected to a memory and configured to call a program stored in the memory to execute the method described in the first or third aspect. The memory may be located within or outside the first device. The processor may include one or more processors.

[0119] In a fourteenth aspect, the present application provides a second device, comprising a processor, connected to a memory, configured to invoke a program stored in the memory to execute the method described in the second or fourth aspect. The memory may be located within or outside the second device. The processor may include one or more processors.

[0120] In one implementation, the first device of the fifth, seventh, ninth, eleventh, and thirteenth aspects may be a chip (system).

[0121] In one implementation, the second device of the sixth, eighth, tenth, twelfth, and fourteenth aspects may be a chip (system).

[0122] A fifteenth aspect of the present application provides a computer program product comprising instructions, characterized in that when the computer program product is run on a computer, the computer is enabled to execute any one of the implementation methods of any one of the first to fourth aspects.

[0123] In a sixteenth aspect, the present application provides a computer-readable storage medium comprising computer instructions, which, when executed on a computer, enables the computer to execute any one of the implementation methods in any one of the first to fourth aspects.

[0124] In the seventeenth aspect, the present application provides a chip device, comprising a processor for calling a computer program or computer instruction in a memory so that the processor executes any one of the implementation methods of any one of the first to fourth aspects above.

[0125] Optionally, the processor is coupled to the memory via an interface.

[0126] In an eighteenth aspect, the present application provides a communication system, which includes the first device of the fifth aspect and the second device of the sixth aspect; or, the communication system includes the first device of the seventh aspect and the second device of the eighth aspect.

[0127] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0128] In the above technical solution, a first device receives at least one quantization threshold value from a second device. The first device then quantizes the information related to the first model of the first device based on the at least one quantization threshold value. The first device then sends first information to the second device, indicating the information related to the first model after quantization. This reduces the communication overhead of the first device reporting the information related to the first model, conserving communication resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0129] FIG1 is a schematic diagram of a communication system used in an embodiment of the present invention;

[0130] FIG2 is a schematic diagram of an embodiment of a communication method according to an embodiment of the present application;

[0131] FIG3 is a flow chart of a communication method according to an embodiment of the present application;

[0132] FIG4 is a schematic diagram of another embodiment of the communication method according to an embodiment of the present application;

[0133] FIG5 is a schematic diagram of generating a common sparse mask according to an embodiment of the present application;

[0134] FIG6 is a schematic structural diagram of the first device according to an embodiment of the present application;

[0135] FIG7 is another structural diagram of the first device according to an embodiment of the present application;

[0136] FIG8 is a schematic structural diagram of a second device according to an embodiment of the present application;

[0137] FIG9 is another schematic structural diagram of the second device according to an embodiment of the present application;

[0138] FIG10 is a schematic structural diagram of a terminal device according to an embodiment of the present application;

[0139] FIG11 is a schematic structural diagram of a network device according to an embodiment of the present application. DETAILED DESCRIPTION

[0140] An embodiment of the present application provides a communication method and related devices for reducing the communication overhead of a first device reporting relevant information of a first model, thereby saving communication resources.

[0141] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0142] References to "one embodiment" or "some embodiments" in this application mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0143] In the description of this application, unless otherwise specified, " / " means "or", for example, A / B can mean A or B. "And / or" in this article is merely a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, "at least one" means one or more, and "plurality" means two or more. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c; a and b; a and c; b and c; or a, b, and c. Among them, a, b, and c can be single or multiple.

[0144] It is understood that in this application, "indication" can include direct indication, indirect indication, explicit indication, and implicit indication. When describing that a certain indication information is used to indicate A, it can be understood that the indication information carries A, directly indicates A, or indirectly indicates A.

[0145] In this application, the information indicated by the indication information is referred to as the information to be indicated. In the specific implementation process, there are many ways to indicate the information to be indicated, such as but not limited to, the information to be indicated can be directly indicated, such as the information to be indicated itself or the index of the information to be indicated, etc., or the information to be indicated can be indirectly indicated by indicating other information, wherein there is an association between the other information and the information to be indicated. It is also possible to indicate only a part of the information to be indicated, while the other parts of the information to be indicated are known or agreed in advance. For example, the indication of specific information can also be achieved with the help of the arrangement order of each information agreed in advance (such as specified in the protocol), thereby reducing the indication overhead to a certain extent.

[0146] The information to be indicated can be sent as a whole or divided into multiple sub-information and sent separately. The transmission period and / or transmission timing of these sub-information can be the same or different. The specific transmission method is not limited in this application. The transmission period and / or transmission timing of these sub-information can be predefined, for example, according to a protocol, or can be configured by the transmitting device through sending configuration information to the receiving device.

[0147] The technical solution of the present application can be applied to cellular communication systems related to the 3rd Generation Partnership Project (3GPP). For example, the fourth generation (4G) communication system, the fifth generation (5G) communication system, and the communication system after the fifth generation communication system. For example, the sixth generation communication system. For example, the fourth generation communication system may include a long term evolution (LTE) communication system. The fifth generation communication system may include a new radio (NR) communication system. The technical solution of the present application can also be applied to wireless fidelity (WiFi) systems, communication systems that support the integration of multiple wireless technologies, device-to-device (D2D) systems, vehicle to everything (V2X) communication systems, etc.

[0148] The communication system to which the technical solution of the present application is applicable includes a first device and a second device. Optionally, the communication system also includes a third device.

[0149] The following describes some possible forms of the first device and the second device. This application is still applicable to other forms, and the following implementation does not limit this application.

[0150] 1. The first device is a first terminal device or a chip within the first terminal device, and the second device is a network device or a chip within the network device. In this implementation, the first device and the second device can execute the communication method provided in this application.

[0151] Optionally, the third device is the second terminal device or a chip in the second terminal device. The third device can execute the communication method provided in this application.

[0152] It should be noted that the above description is based on the first terminal device and the second terminal device as an example. In actual applications, the network device can execute the communication method provided in this application with more terminal devices.

[0153] 2. The first device is a first network device or a chip within the first network device, and the second device is a terminal device or a chip within the terminal device. In this implementation, the first device and the second device can execute the communication method provided in this application.

[0154] Optionally, the third device is the second network device or a chip in the second network device. The third device can execute the communication method provided in this application.

[0155] It should be noted that the above description is based on the first network device and the second network device as an example. In actual applications, the terminal device can execute the communication method provided in this application with more network devices.

[0156] 3. The first device is a first terminal device or a chip in the first terminal device, and the second device is a second terminal device or a chip in the second terminal device. In this implementation, the first device and the second device can execute the communication method provided in this application.

[0157] Optionally, the third device is a third terminal device or a chip in the third terminal device. The third device can execute the communication method provided in this application.

[0158] It should be noted that the above description is based on the first terminal device, the second terminal device, and the third terminal device. In actual application, the first terminal device can execute the communication method provided in this application with more terminal devices.

[0159] The following introduces the terminal equipment and network equipment involved in this application.

[0160] Terminal devices are devices with wireless transceiver functions and computing capabilities. Terminal devices can use local data for machine learning training and send information about the trained model to network devices.

[0161] A terminal device may be referred to as user equipment (UE), access terminal, subscriber unit, user station, mobile station, remote station, remote terminal, mobile device, user terminal, wireless communication device, user agent or user equipment. The terminal device may also be a satellite phone, a cellular phone, a smart phone, a wireless data card, a wireless modem, a machine type communication device, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), a handheld device with wireless communication capabilities, a computing device or other processing device connected to a wireless modem, a vehicle-mounted device, a communication device carried on a high-altitude aircraft, a wearable device, a drone, a robot, a terminal in D2D, a terminal in V2X, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, or a terminal device in a future communication network, etc., and this application does not impose any restrictions.

[0162] Network devices have wireless transceiver functions and computing capabilities. Network devices are used to communicate with terminal devices. In other words, network devices can be devices that connect terminal devices to wireless networks. For example, network devices can be network nodes with computing capabilities. For example, network devices can be artificial intelligence (AI) nodes, computing nodes, or access network nodes with AI capabilities on the network side (e.g., access network or core network). Network devices can fuse models trained by multiple terminal devices and then send them to these terminal devices. This enables joint learning between multiple terminal devices.

[0163] The network device may be a node in a radio access network. The network device may be referred to as a base station, and may also be referred to as a radio access network (RAN) node or RAN device. The network device may be an evolved Node B (eNB or eNodeB) in LTE, or a next generation node B (gNB) in a 5G network, or a base station in a future evolved public land mobile network (PLMN), a broadband network service gateway (BNG), an aggregation switch, or a non-third generation partnership project (3GPP) access device. Optionally, the network device in the embodiment of the present application may include various forms of base stations. For example, macro base stations, micro base stations (also called small stations), relay stations, access points, devices that implement base station functions in communication systems evolved after 5G, access points (APs) in WiFi systems, transmission points (TRPs), transmitting points (TPs), mobile switching centers, devices that assume base station functions in D2D communications, V2X device communications, or machine-to-machine (M2M) communications. Network equipment can also include centralized units (CUs) and distributed units (DUs) in cloud radio access network (C-RAN) systems, and network equipment in non-terrestrial network (NTN) communication systems, that is, they can be deployed on high-altitude platforms or satellites, which is not limited in this application.

[0164] A possible communication system to which this application is applicable is described below.

[0165] FIG1 is a schematic diagram of a communication system used in an embodiment of the present application. Referring to FIG1 , the communication system includes a terminal device 101, a terminal device 102, a network device 103, a network device 104, and a server 105. Terminal device 101 can establish a communication connection with network device 103, and terminal device 102 can establish a communication connection with network device 103.

[0166] In one possible implementation, the terminal device 101, the terminal device 102, and the network device 103 may execute the communication method provided in this application, thereby reducing the cost of the terminal device reporting relevant information of its model and saving communication cost.

[0167] It should be noted that FIG1 is merely an example. In practical applications, the communication system includes at least one network device and at least one terminal device.

[0168] Distributed learning is a method for achieving federated learning. Specifically, multiple node devices use local data to train local models, and a central node device fuses these local models to create a global model. This allows for federated learning while protecting the privacy of user data on the node devices.

[0169] Multiple node devices can train their local models separately to obtain relevant parameters of the local models. For example, the weight parameters or weight gradients of the local models. Then, multiple node devices send the relevant parameters of the local models to the central node device. The central node device fuses the relevant parameters of the local models sent by multiple node devices to obtain the relevant parameters of the global model, and sends it to each node device. Each node device can update the local model of the node device through the relevant parameters of the global model. It can be seen from the above technical solution that each node device sends the relevant parameters of the local model to the central node device respectively. This results in a large amount of data reported by the node device and a large communication overhead. Therefore, how the node device can report the relevant parameters of the local model with lower communication overhead is an urgent problem to be solved.

[0170] The mathematical symbols involved in this application are introduced below.

[0171] mean(x): means to find the average value of all elements in vector x.

[0172] abs(y): means to find the absolute value of each element in vector y.

[0173] mean(x1, y1): means to find the average value of element x1 and element y1.

[0174] The technical solution of this application is introduced below in conjunction with specific embodiments.

[0175] FIG2 is a schematic diagram of an embodiment of a communication method according to an embodiment of the present application. Referring to FIG2 , the method includes:

[0176] 201. A second device sends at least one quantization threshold value to a first device. Correspondingly, the first device receives at least one quantization threshold value from the second device.

[0177] The at least one quantization threshold value is used by the first device to quantize the relevant information of the first model. Optionally, the first model can be a model configured by the second device for the first device. Optionally, the first model can be a neural network model.

[0178] Optionally, the relevant information of the first model is obtained by the first device performing a round of training on the first model.

[0179] Optionally, the relevant information of the first model includes output parameters or update parameters of the first model. The output parameters of the first model can be understood as the output data of the first model. For the sake of ease of description, they will be collectively referred to as output parameters below. The update parameters of the first model include weight parameters or weight gradients of the first model. For example, if the first model is a neural network model, the relevant information of the first model includes the output parameters of the neural network model. Alternatively, the relevant information of the first model includes weight parameters or weight gradients in the neural network model.

[0180] In one possible implementation, the first device is a first terminal device, the second device is a network device, and the at least one quantization threshold value can be carried in downlink control information, radio resource control (RRC) message or medium access control control element (MAC CE).

[0181] In another possible implementation, the first device is a network device, the second device is a terminal device, and the at least one quantization threshold value can be carried in uplink control information.

[0182] The following describes a possible implementation of the second device determining the at least one quantization threshold value. Optionally, the embodiment shown in FIG2 further includes step 201a and step 201b. Step 201a and step 201b can be performed before step 201.

[0183] 201a. The first device sends second information to the second device. Correspondingly, the second device receives the second information from the first device.

[0184] Two possible implementations of the second information are described below.

[0185] Implementation method 1: The second information is used to indicate information obtained by processing the relevant information of the first model.

[0186] Optionally, the second information includes information obtained by processing the relevant information of the first model, or the second information indicates information obtained by processing the relevant information of the first model.

[0187] For example, the relevant information of the first model includes the output parameters of the first model. The information obtained after processing the relevant information of the first model includes the average value or weighted value of the absolute values ​​of the output parameters of the first model. For example, the output parameters of the first model include output parameter A, output parameter B, and output parameter C of the first model. The first device averages the absolute values ​​corresponding to output parameter A, output parameter B, and output parameter C respectively to obtain the average value of the absolute values ​​of the output parameters. The second information includes the average value or weighted value of the absolute values ​​of the output parameters of the first model. Alternatively, the second information indicates the average value or weighted value of the absolute values ​​of the output parameters of the first model.

[0188] For example, the second information is indication information, and the corresponding relationship between the value of the indication information and the average value or weighted value of the absolute value of the output parameter of the first model can be shown in Table 1 or Table 2:

[0189] Table 1

[0190] The value of the output parameter indicates the average or weighted value of the absolute value of the information 000.25010.5100.75111

[0191] For example, the relevant information of the first model includes the updated parameters of the first model. The information obtained by processing the relevant information of the first model includes the average or weighted value of the absolute values ​​of the updated parameters of the first model. For example, the updated parameters of the first model include the weight gradient obtained by the first device performing the Qth round of training on the first model. Weight gradient and weight gradient The first device is to calculate the weight gradient Weight gradient and weight gradient The corresponding absolute values ​​are averaged to obtain the average value of the absolute value of the weight gradient of the first model. The second information includes the average value or weighted value of the absolute value of the updated parameter of the first model. Alternatively, the second information indicates the average value or weighted value of the absolute value of the updated parameter of the first model. For example, the second information is indication information, and the corresponding relationship between the value of the indication information and the average value or weighted value of the absolute value of the updated parameter of the first model can be shown in Table 2:

[0192] Table 2

[0193] The value of the indicator information updates the average or weighted value of the absolute value of the parameter 000.5011101.5112

[0194] Implementation 2: The second information is used to indicate information obtained by processing the relevant information obtained by the first device through the Mth round of training of the first model. The relevant information of the first model is the relevant information obtained by the first device through the Qth round of training of the first model. M is an integer greater than or equal to 1 and less than Q, and Q is an integer greater than 1.

[0195] In Implementation 2, the second information includes information obtained by processing the relevant information obtained by the first device during the Mth round of training of the first model; or, the second information indicates information obtained by processing the relevant information obtained by the first device during the Mth round of training of the first model. For information obtained by processing the relevant information obtained by the first device during the Mth round of training of the first model, please refer to the aforementioned description of information obtained by processing the relevant information of the first model.

[0196] Implementation method 2 is similar to implementation method 1. For details, please refer to the introduction of implementation method 1.

[0197] In one possible implementation, the first device is a terminal device, the second device is a network device, and the second information can be carried in downlink control information, RRC message, or MAC CE. In another possible implementation, the first device is a network device, the second device is a terminal device, and the second information can be carried in uplink control information.

[0198] 201b. The second device determines the at least one quantization threshold value according to the second information.

[0199] For example, the at least one quantization threshold value includes a quantization threshold value. The second information includes an average value of the absolute value of the weight gradient of the first model. The quantization threshold value γ1=mean(abs(Δw Q ))*a, a is a control factor used to control the interval of quantization processing, and the value range of a is [0,+∞). abs(Δw Q ) represents the absolute value of the weight gradient obtained by the first device performing the Q-th round of training on the first model.

[0200] For example, the at least one quantization threshold value includes two quantization threshold values, namely a first quantization threshold value and a second quantization threshold value. The first quantization threshold value γ1=mean(abs(Δw Q ))*a, the second quantization threshold value -γ1=-mean(abs(Δw Q ))*a. abs(Δw Q ) represents the absolute value of the weight gradient obtained by the first device performing the Q-th round of training on the first model.

[0201] Optionally, the embodiment shown in FIG2 further includes step 201c. Step 201c may be performed before step 201.

[0202] 201c. The third device sends third information to the second device. Correspondingly, the second device receives the third information from the third device.

[0203] The third information indicates information obtained by processing the relevant information of the second model by the third device. Alternatively, the third information indicates information obtained by processing the second model after the third device performs the Sth round of training. The relevant information of the second model is information obtained by the third device after the Rth round of training of the second model. S is an integer greater than or equal to 1 and less than R, and R is an integer greater than 1. The third information is similar to the second information. For details, please refer to the aforementioned description of the second information.

[0204] It should be noted that the second model can be a model configured by the second device for the third device. The first model and the second model can be the same model, for example, both the first model and the second model are global models configured for the second device. The first model and the second model are used herein to distinguish between models on the first device and the second device, and they can actually be the same model.

[0205] Based on the above step 201c, optionally, the above step 201b specifically includes:

[0206] The second device determines the at least one quantization threshold value according to the second information and the third information.

[0207] For example, the second information includes the average value of the absolute value of the weight gradient of the first model. The third information includes the average value of the absolute value of the weight gradient of the second model. The second device determines the at least one quantization threshold value based on the average value of the absolute value of the weight gradient of the first model and the average value of the absolute value of the weight gradient of the second model. For example, the at least one quantization threshold value includes two quantization threshold values, namely the first quantization threshold value and the second quantization threshold value. The first quantization threshold value γ1=mean(mean(abs(Δw Q )),mean(abs(Δw R )))*a, the second quantization threshold value -γ1=-mean(mean(abs(Δw Q )),mean(abs(Δw R The first device performs the Qth round of training on the first model to obtain N weight gradients through the vector Δw Q The second device performs the Rth round of training on the second model to obtain the N weight gradients through the vector Δw R express.

[0208] It should be noted that steps 201a to 201c described above are merely examples of the technical solution of this application, using the example of the second device determining the at least one quantization threshold value based on the second information from the first device and the third information from the third device. In actual applications, the second device may receive information related to models indicated by multiple devices and determine the at least one quantization threshold value based on the information related to these models, but this application does not impose any specific limitations thereon.

[0209] It should be noted that there is no fixed order for executing step 201c and step 201a. Step 201a may be executed first, then step 201c; or step 201c may be executed first, then step 201a; or, depending on the circumstances, step 201a and step 201c may be executed simultaneously, which is not limited in this application.

[0210] 202. The first device quantizes relevant information of the first model of the first device according to at least one quantization threshold value.

[0211] As can be seen from the above introduction, the relevant information of the first model includes the output parameters or update parameters of the first model. Here, the technical solution of the present application is introduced by taking the relevant information of the first model including N parameters of the first model as an example. N is an integer greater than or equal to 1. Therefore, the above step 202 specifically includes: the first device quantizes the N parameters of the first model according to at least one quantization threshold value to obtain the N parameters after quantization. For example, as shown in Figure 3, the first device performs the Qth round of training on the first model to obtain the relevant information of the first model. Then, the first device quantizes the relevant information of the first model.

[0212] In one possible implementation, the at least one quantization threshold value includes a quantization threshold value γ1. The relevant information of the first model includes N parameters of the first model. The above step 202 specifically includes: if the i-th parameter among the N parameters is greater than the quantization threshold value γ1, the first device quantizes the i-th parameter to a first value, where i is an integer greater than or equal to 1 and less than or equal to N. If the i-th parameter among the N parameters is less than or equal to the quantization threshold value γ1, the first device quantizes the i-th parameter to a third value. Alternatively, the above step 202 specifically includes: if the i-th parameter among the N parameters is greater than or equal to the quantization threshold value γ1, the first device quantizes the i-th parameter to the first value, where i is an integer greater than or equal to 1 and less than or equal to N. If the i-th parameter among the N parameters is less than the quantization threshold value γ1, the first device quantizes the i-th parameter to a third value.

[0213] For example, the first value is +1 and the third value is -1. The N parameters of the first model are the N weight gradients of the first model. The i-th weight gradient in the N weight gradients is expressed as When the weight gradient is greater than the quantization threshold γ1, then the weight gradient Quantized to +1, when the weight gradient is less than or equal to the quantization threshold value γ1, then the weight gradient Quantized to -1. The i-th weight gradient s after quantization i It can be expressed by the following formula 1:

[0214]

[0215] The above shows the quantization process of the i-th parameter among the N parameters of the first model by the first device. The quantization process of other parameters among the N parameters is also applicable and will not be described in detail here.

[0216] It should be noted that, optionally, if the i-th parameter among the N parameters is greater than or equal to the quantization threshold value γ1, the first device quantizes the i-th parameter to a first value, where i is an integer greater than or equal to 1 and less than or equal to N. If the i-th parameter among the N parameters is less than or equal to the quantization threshold value γ1, the first device quantizes the i-th parameter to a third value. In other words, if the i-th parameter is equal to the quantization threshold value γ1, the first device may quantize the i-th parameter to the first value or the third value. In this case, the first device may randomly quantize the i-th parameter to the first value or the third value through random quantization processing.

[0217] In another possible implementation, at least one quantization threshold value includes two quantization threshold values, namely a first quantization threshold value γ1 and a second quantization threshold value -γ1. The relevant information of the first model includes N parameters of the first model. The above step 202 specifically includes: if the i-th parameter among the N parameters is greater than the first quantization threshold value γ1, the first device quantizes the i-th parameter to a first value, where i is an integer greater than or equal to 1 and less than or equal to N; if the i-th parameter among the N parameters is less than or equal to the first quantization threshold value γ1 and greater than or equal to the second quantization threshold value -γ1, the first device quantizes the i-th parameter to a second value; if the i-th parameter among the N parameters is less than the second quantization threshold value -γ1, the first device quantizes the i-th parameter to a third value. Alternatively, the above-mentioned step 202 specifically includes: if the i-th parameter among the N parameters is greater than or equal to the first quantization threshold value γ1, the first device quantizes the i-th parameter to a first value, where i is an integer greater than or equal to 1 and less than or equal to N; if the i-th parameter among the N parameters is less than the first quantization threshold value γ1 and greater than the second quantization threshold value -γ1, the first device quantizes the i-th parameter to a second value; or, if the i-th parameter among the N parameters is less than or equal to the second quantization threshold value -γ1, the first device quantizes the i-th parameter to a third value.

[0218] For example, the first value is +1, the second value is 0, and the third value is -1. The N parameters of the first model are the N weight gradients of the first model. The i-th weight gradient in the N weight gradients is expressed as When the weight gradient is greater than the first quantization threshold γ1, then the weight gradient Quantized to +1. When the weight gradient is less than the second quantization threshold -γ1, then the weight gradient Quantized to -1. When the weight gradient When the weight gradient is less than or equal to the first quantization threshold value γ1 and greater than or equal to the second quantization threshold value -γ1, quantized to 0. Therefore, the i-th weight gradient s after quantization is i It can be expressed by the following formula 2:

[0219]

[0220] The above shows the quantization process of the first device for the i-th parameter among the N parameters of the first model. The quantization process of other parameters in the N parameters is also applicable and will not be described in detail here. In the above implementation, the first device can quantize the parameters of the first model through multiple quantization threshold values, which is conducive to improving the quantization accuracy and improving the convergence speed and performance of the model. Further, it can be seen from the above formula 2 that s i The value may be 0, indicating that when the value of the i-th parameter falls within the interval between the second quantization threshold value and the first quantization threshold value, the first device may not update the i-th parameter. For example, if the i-th parameter is caused by training noise, then the first device does not update the i-th parameter, which is beneficial to improving the accuracy of the first model trained by the second device.

[0221] It should be noted that, optionally, if the i-th parameter among the N parameters is greater than or equal to the first quantization threshold value γ1, the first device quantizes the i-th parameter to a first value, where i is an integer greater than or equal to 1 and less than or equal to N; if the i-th parameter among the N parameters is less than or equal to the first quantization threshold value γ1 and greater than or equal to the second quantization threshold value -γ1, the first device quantizes the i-th parameter to a second value; if the i-th parameter among the N parameters is less than the second quantization threshold value -γ1, the first device quantizes the i-th parameter to a third value. In other words, for the i-th parameter, if the i-th parameter is equal to the first quantization threshold value γ1, the first device may quantize the i-th parameter to the first value or the second value. In this case, the first device may randomly quantize the i-th parameter to the first value or the second value through random quantization processing.

[0222] It should be noted that, optionally, if the i-th parameter among the N parameters is greater than the first quantization threshold value γ1, the first device quantizes the i-th parameter to a first value, where i is an integer greater than or equal to 1 and less than or equal to N; if the i-th parameter among the N parameters is less than or equal to the first quantization threshold value γ1 and greater than or equal to the second quantization threshold value -γ1, the first device quantizes the i-th parameter to a second value; if the i-th parameter among the N parameters is less than or equal to the second quantization threshold value -γ1, the first device quantizes the i-th parameter to a third value. That is, for the i-th parameter, if the i-th parameter is equal to the second quantization threshold value -γ1, the first device may quantize the i-th parameter to the second value or the third value. In this case, the first device may randomly quantize the i-th parameter to the second value or the third value through random quantization processing.

[0223] The above examples illustrate that the at least one quantization threshold value includes one quantization threshold value and two quantization threshold values. In practical applications, the at least one quantization threshold value may include three quantization threshold values, four quantization threshold values, or more quantization threshold values, which are not specifically limited in this application and are not exemplified here.

[0224] Optionally, in step 202, the relevant information of the first model includes N parameters of the first model after quantization error compensation. For details about the N parameters after quantization error compensation, please refer to the following description of step 202a.

[0225] Optionally, the embodiment shown in FIG. 2 further includes step 202 a , which may be performed before step 202 .

[0226] 202a. The first device performs error compensation on the N parameters of the first model according to the quantization errors corresponding to the N parameters, to obtain N parameters after quantization error compensation.

[0227] The N parameters of the first model are obtained by the first device performing a Q-th round of training on the first model. The quantization error corresponding to the i-th parameter of the N parameters is determined by the i-th parameter obtained by the first device performing a Q-1-th round of training on the first model and performing quantization error compensation.

[0228] For example, the i-th parameter among the N parameters of the first model is the i-th weight gradient The i-th weight gradient after quantization error compensation can be expressed as in, represents the i-th weight gradient obtained from the Q-1-th round of training after quantization error compensation, and η is the global learning rate. Express Perform quantification processing.

[0229] It should be noted that the first device can determine the quantization error corresponding to the i-th parameter obtained by the Q+1-th round of training This facilitates the first device to perform quantization error compensation on the N parameters obtained from the Q+1th round of training.

[0230] Based on step 202a, the relevant information of the first model includes N parameters after quantization error compensation. Optionally, step 202 specifically includes: the first device quantizes the N parameters after quantization error compensation according to the at least one quantization threshold value. For the specific quantization process, please refer to the relevant description of step 202.

[0231] It can be seen from this that in the above step 202a, the first device compensates for the quantization errors corresponding to the N parameters of the first model, which is beneficial to improving the accuracy of the second device in updating the first model and improving the performance of model training.

[0232] Optionally, in step 202, the relevant information of the first model includes N parameters of the first model that have been sparsely processed. For details about the N parameters of the first model that have been sparsely processed, please refer to the following description of step 202b.

[0233] Optionally, the embodiment shown in FIG2 further includes step 202b. Step 202b may be performed before step 202.

[0234] 202b. The first device selects N parameters from the K parameters of the first model according to the common sparse mask to obtain N parameters of the first model that have been sparsely processed.

[0235] In one possible implementation, the K parameters of the first model are obtained by the first device performing a round of training on the first model.

[0236] In another possible implementation, the K parameters of the first model are obtained by the first device performing a round of training on the first model and performing quantization error compensation. The process of the first device performing quantization error compensation on the K parameters is similar to the aforementioned step 202a. For details, please refer to the relevant description of the aforementioned step 202a.

[0237] Optionally, the common sparse mask is a bit sequence comprising K bits. The K bits correspond one-to-one to the K parameters. When the value of one of the K bits is 0, it is used to instruct the first device not to select the parameter corresponding to the bit. When the value of one of the K bits is 1, it is used to instruct the first device to select the parameter corresponding to the bit. Alternatively, when the value of one of the K bits is 0, it is used to instruct the first device to select the parameter corresponding to the bit. When the value of one of the K bits is 1, it is used to instruct the first device to select the parameter corresponding to the bit. For example, the K parameters include ten weight gradients of the first model. The bit sequence is 1000111100, and the bit sequence corresponds one-to-one to the ten weight gradients from high to low bits. For example, the first bit of the bit sequence corresponds to the first weight gradient of the ten weight gradients. The second bit of the bit sequence corresponds to the second weight gradient of the ten weight gradients, and so on. The tenth bit of the bit sequence corresponds to the tenth weight gradient of the ten weight gradients. It can be seen that the relevant information of the first model includes the first weight gradient, the fifth weight gradient, the sixth weight gradient, the seventh weight gradient and the eighth weight gradient among the ten weight gradients.

[0238] Two possible implementations of the first device obtaining the common sparse mask are introduced below.

[0239] Implementation 1: The public sparse mask is determined by the first device according to the sparse ratio and the pseudo-random number. The sparse ratio is indicated by the second device to the first device.

[0240] It should be noted that multiple devices need to use the same common sparse mask to enable each of the multiple devices to send parameters with the same index of the model configured on each device to the second device. Furthermore, the multiple devices can send parameters with the same index using the same time-frequency resources. This helps reduce the communication resources required for multiple devices to report model parameters. This improves the utilization of communication resources. This allows the second device to receive parameters with the same index sent by multiple devices on the same time-frequency resource. In other words, it supports the second device to achieve model fusion through the superposition of over-the-air signals.

[0241] It should be noted that the second device can indicate different sparsity ratios to the first device at different training stages. For example, at the beginning of training, the sparsity ratio can be relatively small. This facilitates the second device to obtain more model-related information and achieve rapid model convergence. During the training convergence phase, the sparsity ratio can be relatively large.

[0242] Implementation method 2: Implementation method 2 is introduced below in conjunction with step 201e.

[0243] Optionally, the embodiment shown in Figure 2 further includes step 201e. Step 201e may be performed before step 202b.

[0244] 202b. The second apparatus sends second indication information to the first apparatus. Correspondingly, the first apparatus receives the second indication information from the second apparatus. The second indication information is used to indicate a common sparse mask.

[0245] A possible implementation manner in which the second device determines the common sparse mask will be described later in conjunction with the embodiment shown in FIG4 . For details, please refer to the relevant introduction to the embodiment shown in FIG4 later.

[0246] Based on the above step 202b, optionally, the above step 202 specifically includes: the first device quantizes the N parameters of the first model that have been sparsely processed according to the at least one quantization threshold value. For the specific quantization process, please refer to the relevant introduction in the above step 202. For example, as shown in Figure 3, the first device performs the Qth round of training on the first model to obtain K parameters of the first model. Then, the first device performs quantization error on the K parameters of the first model to obtain K parameters after quantization error compensation. The first device selects N parameters from the K parameters after quantization error compensation according to a common sparse mask, and then quantizes the N parameters according to the at least one quantization threshold value.

[0247] It can be seen from this that in the above step 202b, the first device selects some parameters of the first model according to the common sparse mask, which is beneficial to reducing the overhead of the first device in reporting the parameters of the first model.

[0248] There is no fixed order for executing step 201e, step 201a, step 201b, step 201c, and step 201. Step 201e may be executed first, followed by steps 201a, step 201b, step 201c, and step 201. Alternatively, steps 201a, step 201b, step 201c, and step 201 may be executed first, followed by step 201e. Alternatively, steps 201e, 201a, step 201b, step 201c, and step 201 may be executed simultaneously, depending on the circumstances.

[0249] 203. The first device sends first information to the second device. The first information is used to indicate relevant information of the first model after quantization processing. Correspondingly, the second device receives the first information from the first device.

[0250] In one possible implementation, the first information includes relevant information of the first model after quantization processing. For example, the relevant information of the first model includes N parameters of the first model, and the first information includes the N parameters of the first model after quantization processing.

[0251] In another possible implementation manner, the first information is indication information, and the indication information indicates relevant information of the first model after quantization processing.

[0252] Optionally, the relevant information of the first model includes N parameters of the first model after quantization. A possible implementation of the above step 203 is described below. Optionally, the above step 203 specifically includes step 2003a and step 2003b.

[0253] 2003a. The first device modulates the N parameters of the first model after quantization to obtain N first signals, wherein the N first signals correspond one-to-one to the N parameters.

[0254] 2003b: The first device sends N first signals to the second device. Correspondingly, the second device receives the N first signals from the first device.

[0255] The following describes the above steps 2003a and 2003b in conjunction with the quantization example shown in the above formula 2.

[0256] The first device modulates the i-th parameter among the N parameters of the quantized first model to obtain an i-th first signal. The i-th first signal corresponds to two sequences, each of which includes at least one symbol. The following describes two possible implementations of the first device transmitting the two sequences, thereby facilitating the second device determining the value of the i-th parameter after quantization.

[0257] Implementation method 1: When the i-th parameter after quantization processing is a first value, the transmission power of the first sequence of the two sequences transmitted by the first device is less than the transmission power of the second sequence of the two sequences transmitted by the first device. When the i-th parameter after quantization processing is a second value, the transmission power of the first sequence of the two sequences transmitted by the first device is equal to the transmission power of the second sequence of the two sequences transmitted by the first device. When the i-th parameter after quantization processing is a third value, the transmission power of the first sequence of the two sequences transmitted by the first device is greater than the transmission power of the second sequence of the two sequences transmitted by the first device.

[0258] Optionally, when the i-th parameter after quantization is a first value, the first of the two sequences is an all-0 sequence, and the second is a non-all-0 sequence. When the i-th parameter after quantization is a second value, the two sequences are all-0 sequences. When the i-th parameter after quantization is a third value, the first of the two sequences is a non-all-0 sequence, and the second is an all-0 sequence. For example, the first value is +1, the second value is 0, and the third value is -1. The i-th first signal carries the i-th parameter s i, the two sequences corresponding to the i-th parameter. For various values ​​of the i-th parameter, the corresponding two sequences (i.e., sequence 1 and sequence 2) are shown in Table 3:

[0259] Table 3

[0260] s i +10-1 sequence 100c1 sequence 2c000

[0261] Wherein, c0 and c1 are sequences of a specific length. For example, the length of c0 and the length of c1 are both 1, i.e., each includes one symbol. Optionally, c0 and c1 can both be Zadoff–Chu sequences, which can be abbreviated as ZC sequences.

[0262] Implementation method 2: When the i-th parameter after quantization processing is a first value, the transmission power of the first sequence of the two sequences transmitted by the first device is greater than the transmission power of the second sequence of the two sequences transmitted by the first device. When the i-th parameter after quantization processing is a second value, the transmission power of the first sequence of the two sequences transmitted by the first device is equal to the transmission power of the second sequence of the two sequences transmitted by the first device. When the i-th parameter after quantization processing is a third value, the transmission power of the first sequence of the two sequences transmitted by the first device is less than the transmission power of the second sequence of the two sequences transmitted by the first device.

[0263] Optionally, when the i-th parameter after quantization is a first value, the first of the two sequences is a non-all-0 sequence, and the second sequence is an all-0 sequence. When the i-th parameter after quantization is a second value, the two sequences are all-0 sequences. When the i-th parameter after quantization is a third value, the first of the two sequences is an all-0 sequence, and the second sequence is a non-all-0 sequence. For example, the first value is +1, the second value is 0, and the third value is -1. The i-th first signal carries the i-th parameter s i , the two sequences corresponding to the i-th parameter. For various values ​​of the i-th parameter, the corresponding two sequences (i.e., sequence 1 and sequence 2) are shown in Table 4:

[0264] Table 4

[0265] s i +10-1 sequence 1c000 sequence 200c1

[0266] For details about c0 and c1, please refer to the above introduction and will not be explained here.

[0267] It should be noted that the above shows one possible example of the first value, the second value, and the third value. In actual applications, the first value, the second value, and the third value may also be other values, which are not limited in this application. For example, the first value is 0.7, the second value is 0, and the third value is -0.7.

[0268] As shown in Figure 3, the first device quantizes the N parameters of the first model to obtain the N parameters of the quantized first model. The first device modulates the N parameters of the quantized first model, maps the modulated sequence to the corresponding time-frequency resources, and performs waveform shaping to obtain the N first signals. The first device sends the N first signals to the second device. As can be seen from the above introduction, the first device modulates each of the N parameters of the first model onto two sequences. The first device controls the transmission power used for each of the two sequences, thereby facilitating the second device to determine the value of the parameter. The first device does not need to perform channel estimation and equalization, and thus does not require corresponding pilot overhead.

[0269] As can be seen, the first device receives at least one quantization threshold value from the second device. The first device then quantizes the information related to the first model of the first device based on the at least one quantization threshold value. The first device then sends first information to the second device, indicating the information related to the first model after quantization. This reduces the communication overhead of the first device reporting the information related to the first model, conserving communication resources.

[0270] Optionally, the embodiment shown in FIG2 further includes step 204 and step 205. Step 204 and step 205 may be performed after step 203.

[0271] 204. The second device determines global information of the first model based on the first information.

[0272] The global information of the first model includes the global output parameters of the first model. Alternatively, the global information of the first model includes the global update parameters and / or global learning rate of the first model. The global output parameters of the first model can be understood as the global output data of the first model. The global update parameters of the first model include the global weight parameters or global weight gradient of the first model.

[0273] Optionally, the global information of the first model includes N global parameters of the first model, where the global parameters are output parameters or update parameters. For details on the determination process of the N global parameters, please refer to the relevant description below.

[0274] Optionally, the first information includes N parameters of the first model after quantization processing, and the second device can determine the global learning rate η based on the N parameters of the first model.

[0275] For example, the N parameters of the first model after quantization include the N weight gradients obtained by the first device performing the Qth round of training on the first model and after quantization. Q Represents the N weight gradients of the first model. That is, the vector Δw Q The first device may determine the global learning rate η=mean(abs(Δw q )). Vector Δw q Including vector Δw Q The weight parameters that are not 0 after quantization processing.

[0276] It should be noted that, optionally, the first device may also send sixth information to the second device. The sixth information is used to indicate the average of the absolute values ​​of the non-zero parameters among the N parameters of the first model after quantization. The second device determines the global learning rate based on the sixth information.

[0277] For example, the N parameters of the first model are the N weight gradients obtained by the first device performing the Qth round of training on the first model, specifically through the vector Δw Q represents the N weight gradients of the first model. Then the second device can determine the global learning rate η=mean(abs(Δw q )), mean(abs(Δw q )) is indicated by the first device to the second device through the sixth information, abs(Δw q ) is the vector Δw Q The absolute value of the non-zero weight parameter after quantization.

[0278] It should be noted that, optionally, the global learning rate η is variable. For example, the global learning rate η is a constant that changes with the number of training rounds.

[0279] It should be noted that in the above step 204, the second device determines the global learning rate based on the first information. In practical applications, the second device can determine the global learning rate based on the second information. Optionally, the second device determines the global learning rate based on the second information and the third information, which is not specifically limited in this application.

[0280] In one possible implementation, the first model is a neural network model. The relevant information of the first model includes relevant parameters of neurons in all layers of the neural network model. Based on this implementation, the N global parameters of the first model included in the global information of the first model in step 204 are global parameters of neurons in all layers.

[0281] In this implementation, the at least one quantization threshold value and the global learning rate are uniformly set for neurons in each layer of the neural network model.

[0282] In another possible implementation, the first model is a neural network model. The relevant information of the first model includes relevant parameters of neurons in a P layer of the neural network model, where P is an integer greater than or equal to 1.

[0283] Based on this implementation, the N global parameters of the first model included in the global information of the first model in the above step 204 are global parameters of the neurons in the P layer.

[0284] In this implementation, the at least one quantization threshold value and the global learning rate are uniformly set for the neurons in the P layer of the neural network model. For neurons in other layers of the neural network model other than the P layer, corresponding quantization threshold values ​​and global learning rates should be separately determined.

[0285] Optionally, the embodiment shown in FIG2 further includes step 203 a , and step 203 a may be performed before step 204 .

[0286] 203a. The third device sends fifth information to the second device. The fifth information is used to indicate relevant information of the second model after quantization processing. Correspondingly, the second device receives the fifth information from the third device.

[0287] Specifically, the quantized related information of the second model is obtained by the third device performing quantization processing on the related information of the second model according to the at least one quantization threshold value. For the specific quantization processing process, please refer to the relevant introduction of the aforementioned step 202.

[0288] Optionally, the relevant information of the second model includes N parameters of the second model. For more information about the second model, please refer to the above-mentioned related introduction. Optionally, the above-mentioned step 203a specifically includes step 1 and step 2.

[0289] Step 1: The third device modulates the N parameters of the second model to obtain N second signals. The N second signals carry the N parameters of the second model, and the N second signals correspond one-to-one to the N parameters of the second model.

[0290] Step 2: The third device sends the N second signals to the second device. Correspondingly, the second device receives the N second signals from the third device.

[0291] Steps 1 to 2 are similar to the aforementioned steps 2003a to 2003b. For details, please refer to the relevant introduction of the aforementioned steps 2003a to 2003b, which will not be repeated here.

[0292] Optionally, the i-th first signal among the N first signals corresponds to the first sequence and the second sequence. The first sequence is the first sequence of the two sequences corresponding to the i-th first signal, and the second sequence is the second sequence of the two sequences corresponding to the i-th first signal. The i-th second signal among the N second signals corresponds to the third sequence and the fourth sequence. The third sequence is the first sequence of the two sequences corresponding to the i-th second signal, and the fourth sequence is the second sequence of the two sequences corresponding to the i-th second signal. i is an integer greater than or equal to 1 and less than or equal to N. The time-frequency resources used by the first device to send the first sequence are the same as the time-frequency resources used by the third device to send the third sequence. The time-frequency resources used by the first device to send the second sequence are the same as the time-frequency resources used by the third device to send the fourth sequence. This supports the second device to achieve non-coherent reception of the superimposed transmission of multi-user air signals.

[0293] It should be noted that there is no fixed order for executing step 203 and step 203a. Step 203 may be executed first, then step 203a; or step 203a may be executed first, then step 203; or, depending on the circumstances, step 203 and step 203a may be executed simultaneously. This application does not impose any specific restrictions.

[0294] Based on the above steps 203 and 203a, optionally, the above step 204 specifically includes: the second device determines the global information of the first model according to the first information and the fifth information.

[0295] Specifically, the second device determines the global information of the first model based on the N first signals and the N second signals. A possible implementation of step 204 is described below using an example in which the i-th first signal among the N first signals corresponds to the first sequence and the second sequence, and the i-th second signal among the N second signals corresponds to the third sequence and the fourth sequence. The time-frequency resources used by the first device to send the first sequence are the same as the time-frequency resources used by the third device to send the third sequence. The time-frequency resources used by the first device to send the second sequence are the same as the time-frequency resources used by the third device to send the fourth sequence.

[0296] Optionally, the above step 204 specifically includes steps 204a to 204c.

[0297] 204a. The second device determines a sum of first signal energies of the first sequence and the third sequence received by the second device.

[0298] For example, the first signal energy sum can be expressed as

[0299] 204b. The second device determines a sum of second signal energies of the second sequence and the fourth sequence received by the second device.

[0300] For example, the second signal energy sum can be expressed as

[0301] 204c. The second device determines an i-th global parameter among the N global parameters according to the first signal energy sum and the second signal energy sum.

[0302] Based on the implementation method 1 in the above step 2003b, optionally, the above step 204c specifically includes:

[0303] If the sum of the first signal energy sum and the decision threshold value is less than the second signal energy sum, the second device determines that the value of the i-th global parameter is the first value; or, if the sum of the first signal energy sum and the decision threshold value is greater than or equal to the second signal energy sum, and the sum of the second signal energy sum and the decision threshold value is greater than or equal to the first signal energy sum, the second device determines that the value of the i-th global parameter is the second value; or, if the sum of the second signal energy sum and the decision threshold value is less than the first signal energy sum, the second device determines that the value of the i-th global parameter is the third value.

[0304] For example, the first value is +1, the second value is 0, and the third value is -1. The global information of the first model includes N global weight gradients of the first model, N global weight gradients, the i-th global weight gradient a i It can be expressed as formula 3:

[0305]

[0306] Among them, γ2 is the decision threshold, and the energy of the first signal can be expressed as The second signal energy sum can be expressed as

[0307] Based on the implementation method 2 in the aforementioned step 2003b, optionally, the aforementioned step 204c specifically includes:

[0308] If the first signal energy sum is greater than the sum of the second signal energy sum and the decision threshold value, the second device determines that the value of the i-th global parameter is the first value; or, if the first signal energy sum is less than or equal to the sum of the second signal energy sum and the decision threshold value, and the second signal energy sum is less than or equal to the sum of the first signal energy sum and the decision threshold value, the second device determines that the value of the i-th global parameter is the second value; or, if the second signal energy sum is greater than the sum of the first signal energy sum and the decision threshold value, the second device determines that the value of the i-th global parameter is the third value.

[0309] For example, the first value is +1, the second value is 0, and the third value is -1. The global information of the first model includes N global weight gradients of the first model, N global weight gradients, the i-th global weight gradient a i It can be expressed as formula 4:

[0310]

[0311] Among them, γ2 is the decision threshold, and the sum of the first signal energies can be expressed as The sum of the second signal energies can be expressed as

[0312] The processes of the above steps 204a to 204c show the process by which the second device determines the i-th global parameter. The second device can use a similar process to determine other global parameters among the N global parameters, which will not be elaborated here one by one.

[0313] It should be noted that the second device can determine the decision threshold in combination with the N first signals and / or the N second signals. For example, the first device sends the i-th first signal to the second device, and the third device sends the i-th second signal to the second device. The i-th first signal and the i-th second signal occupy the same time-frequency resource. The superimposed signal y received by the second device on this time-frequency resource i . The same applies to other first signals and second signals, and no further examples will be given here. For example, the decision threshold γ2 = mean(abs(|y 2i | 2 - |y 2i-1 | 2 )), 0 < i ≤ N, i is an integer) * b. Among them, represents the sum of the first signal energies, represents the sum of the second signal energies. For the sum of the first signal energies and the sum of the second signal energies, please refer to the relevant introductions above. b is a control factor used to control the decision threshold and affect the number of non-zero elements in the global parameter and the update of the first model.

[0314] It can be seen from this that the second device can determine the i-th global parameter by receiving the signal energies of two sequences corresponding to the i-th first signal and the signal energies of two sequences corresponding to the i-th second signal. Thus, it supports the second device to achieve non-coherent reception of multi-user air signal superimposed transmission and achieve robustness against fading channels.

[0315] Optionally, the second device can determine the global learning rate according to the first information and the fifth information.

[0316] For example, the N parameters of the quantized first model include the N weight gradients obtained by the first device training the first model for the Q-th round and through quantization processing. Specifically, the vector Δw Q represents the N weight gradients of the first model. That is, the vector Δw QThe N weight gradients obtained by the first device through the Q-th round of training of the first model. The N parameters of the second model after quantization include the N weight gradients obtained by the second device through the R-th round of training of the second model and after quantization. Specifically, the vector Δw R Represents the N weight gradients of the second model. That is, the vector Δw R The second device includes N weight gradients obtained by performing the Q-th round of training on the second model. Therefore, the second device can determine the global learning rate η=mean(mean(abs(Δw q )),mean(abs(Δw r ))). Vector Δw q Including vector Δw Q The weight parameter after quantization is not 0. Vector Δw r Including vector Δw R The weight gradient after quantization is not 0.

[0317] It should be noted that, optionally, the first device may send sixth information to the second device. The sixth information is used to indicate the absolute value of the value of the parameter that is not zero after quantization processing among the N parameters of the first model. The third device sends seventh information to the second device. The seventh information is used to indicate the average of the absolute values ​​of the value of the parameter that is not zero after quantization processing among the N parameters of the second model. The second device determines the global learning rate based on the sixth information and the seventh information.

[0318] For example, the N parameters of the first model after quantization include the N weight gradients obtained by the first device performing the Qth round of training on the first model and after quantization. Q Represents the N weight gradients of the first model. That is, the vector Δw Q The N weight gradients obtained by the first device through the Q-th round of training of the first model. The N parameters of the second model after quantization include the N weight gradients obtained by the second device through the R-th round of training of the second model and after quantization. Specifically, the vector Δw R Represents the N weight gradients of the second model. That is, the vector Δw R The first device indicates the vector Δw to the second device through the sixth information. Q The average value of the absolute value of the weight gradient after quantization is not 0, mean(abs(Δw q )). The third device indicates the vector Δw to the second device through the seventh information R The average value of the absolute value of the weight gradient after quantization is not 0, mean(abs(Δw r)). Then the second device can determine the global learning rate η=mean(mean(abs(Δw q )),mean(abs(Δw r ))).

[0319] 205. The second device sends fourth information to the first device. The fourth information is used to indicate the global information of the first model determined by the second device. Correspondingly, the first device receives the fourth information from the second device.

[0320] The fourth information includes the global information of the first model determined by the second device. Alternatively, the fourth information indicates the global information of the first model determined by the second device. For example, the second device encodes or modulates the global information of the first model to obtain the fourth information, and indicates the global information of the first model to the first device via the fourth information. For information about the global information of the first model, please refer to the aforementioned relevant description.

[0321] For example, the fourth information includes N global weight gradients of the first model determined by the second device. The N global weight gradients are represented by vector A. Therefore, the first device can update the weight parameter of the first model to w Q =w Q-1 +η*A.w Q-1 The first device performs a Q-1 round of updating on the first model to obtain the global weight parameters of the first model. Q is the global weight parameter of the first model obtained by the first device performing the Q-th round of updating on the first model. η is the global learning rate.

[0322] For example, the fourth information includes N global output parameters of the first model determined by the second device. The first device may perform the Q+1th round of training on the first model to obtain N actual output parameters of the first model. The first device trains the first model based on the N actual output parameters and the N global output parameters to obtain weight parameters of the first model.

[0323] Optionally, the embodiment shown in FIG2 further includes step 201d. Step 201d may be performed before step 203.

[0324] 201d. The second device sends first indication information to the first device. The first indication information is used to indicate the number of times, L, that the first device should send the first information to the second device. Accordingly, the first device receives the first indication information from the second device. Where L is an integer greater than or equal to 1.

[0325] Based on the above step 201d, optionally, the above step 203 specifically includes: the first device sends the first information to the second device L times. Correspondingly, the second device receives the first information from the first device L times.

[0326] In this implementation, the second device can instruct the first device to repeatedly send the first information to the second device. As discussed above in step 204, the second device's energy-gradient decision can result in errors due to the randomness of channel noise and incoherent signal superposition. Therefore, the first device repeatedly sending the first information facilitates the second device to select the most frequent decision as the best decision, thereby reducing the probability of error and improving model training performance.

[0327] For example, as shown in Figure 3, the first device quantizes the N parameters of the first model to obtain the N parameters of the quantized first model. The first device modulates the N parameters of the quantized first model. The first device can map the modulated sequence to the corresponding time-frequency resources according to the number of transmissions L, and perform waveform shaping to obtain the corresponding first signal. The first device sends the first signal to the second device. For example, if L is 2, the first device can repeat the mapping of the modulated sequence to the corresponding time-frequency resources twice.

[0328] It should be noted that the number of transmissions L can optionally be set based on at least one of the model training stage, the number of users participating in model training, and the signal-to-noise ratio of the channel. For example, in the later stages of training, when the number of users participating in model training is small and the signal-to-noise ratio is low, the number of transmissions can be larger.

[0329] It should be noted that the above is an example of introducing the technical solution of this application by using the second device to determine the global learning rate in combination with the first information. The second device can determine the global learning rate in combination with the first information and / or the third information, and this application does not limit this.

[0330] It should be noted that the embodiment shown in FIG. 2 above describes a solution in which the second device determines at least one quantization threshold value based on the second information and the third information. In practical applications, the second device may send the third information to the first device. The first device independently determines the at least one quantization threshold value based on the second information and the third information, and this application does not limit this.

[0331] The present application also provides another embodiment, which is similar to the embodiment shown in FIG2 , except that step 204 is replaced by step 2004a. This embodiment also includes steps 2004b and 2004c. Steps 2004b and 2004c can be performed before step 205.

[0332] 2004a: The second device sends the first information to the fourth device. Correspondingly, the fourth device receives the first information from the second device.

[0333] For the first information, please refer to the description of step 203 in the embodiment shown in Figure 2. For example, the second device is a network device, and the fourth device is a server. The server can receive the first information sent from the network device.

[0334] 2004b. The fourth device determines the global information of the first model based on the first information.

[0335] Step 2004b is similar to step 204 in the embodiment shown in FIG. 2 . For details, please refer to the relevant introduction of step 204 in the embodiment shown in FIG. 2 .

[0336] Optionally, this embodiment further includes step 2004d, and step 2004d can be performed before step 2004b.

[0337] 2004d: The second device sends fifth information to the fourth device. Correspondingly, the fourth device receives the fifth information from the second device.

[0338] For the fifth information, please refer to the relevant introduction in step 203a in the embodiment shown in FIG. 2 .

[0339] It should be noted that there is no fixed order for executing step 2004a and step 2004d. Step 2004a may be executed first, then step 2004d; or step 2004d may be executed first, then step 2004a; or step 2004a and step 2004d may be executed simultaneously depending on the situation.

[0340] 2004c. The fourth device sends fourth information to the second device, where the fourth information is used to indicate the determined global information of the second model. Correspondingly, the second device receives the fourth information from the fourth device.

[0341] Regarding the fourth information, please refer to the relevant introduction of step 205 in the embodiment shown in FIG2 , which will not be repeated here.

[0342] It should be noted that the first device may be a first terminal device. The second device may be a network device. The third device may be a second terminal device. The fourth device may be a server. The above embodiment describes a process in which a server obtains relevant information about the models of terminal devices managed by the network device and determines the global information of the first model in combination with the relevant information of these models. In actual applications, the server may obtain relevant information about the models of terminal devices managed by multiple network devices and determine the global information of the first model in combination with the relevant information of these models, but this application does not limit this process.

[0343] A possible implementation manner in which the second device determines the common sparse mask is described below in conjunction with FIG. 4 .

[0344] FIG4 is a schematic diagram of another embodiment of the communication method of the present application. Referring to FIG4 , the method includes:

[0345] 401. A first device sends third indication information to a second device. The third indication information indicates the indices of the N parameters with the largest absolute values ​​among the K parameters obtained by the first device through a round of training of the first model. Accordingly, the second device receives the third indication information from the first device.

[0346] The K parameters of the first model are obtained by the first device through a round of training on the first model. The first device determines N parameters with the largest absolute values ​​of the corresponding values ​​among the K parameters. Then, the first device sends the third indication information to the second device.

[0347] Optionally, the third indication information is a bit sequence, the bit sequence including K bits, where the K bits correspond one-to-one to the K parameters of the first model. When a bit in the bit sequence has a value of 0, it indicates that the first device does not indicate the parameter corresponding to the bit; when a bit in the bit sequence has a value of 1, it indicates that the first device indicates the parameter corresponding to the bit. For relevant examples of bit sequences, please refer to the relevant description of Figure 5 below.

[0348] 402. The third device sends fourth indication information to the second device. The fourth indication information is used to indicate the indexes of the N parameters with the largest absolute values ​​among the K parameters of the second model of the second device. In response, the second device receives the fourth indication information from the first device.

[0349] The K parameters of the second model are obtained by a third device through a round of training of the second model. The third device determines the N parameters with the largest absolute values ​​among the K parameters of the second model. Then, the third device sends the fourth indication information to the second device.

[0350] Optionally, the format of the fourth indication information is similar to that of the third indication information. For details, please refer to the relevant introduction in the aforementioned step 401, which will not be repeated here.

[0351] 403. The second device determines a common sparse mask according to the third indication information and the fourth indication information.

[0352] For details about the common sparse mask, please refer to the relevant introduction in the embodiment shown in FIG2 , which will not be repeated here.

[0353] It should be noted that FIG4 illustrates the process by which the second device determines a common sparse mask based on the third and fourth indication information. In practical applications, the second device may receive indication information from each of multiple devices, indicating the N parameters with the largest absolute values ​​among the K parameters of the model of that device. The second device then determines the common sparse mask based on the indication information from these multiple devices.

[0354] For example, as shown in Figure 5, the network device can receive indication information from each terminal device among multiple terminal devices for indicating the N parameters with the largest absolute values ​​of the corresponding values ​​among the K parameters of the model of the terminal device. As shown in Figure 5, the first terminal device sends a first bit sequence to the network device, and the first bit sequence is 110010100. Each bit in the first bit sequence corresponds to one parameter of the K parameters of the model of the first terminal device, that is, K is equal to 9. For example, the first bit in the first bit sequence corresponds to the first parameter of the K parameters, the second bit corresponds to the second parameter of the K parameters, and so on, the last bit corresponds to the last parameter of the K parameters. The parameters corresponding to the bits with a value of 1 in the first bit sequence are the four parameters with the largest absolute values ​​of the corresponding values ​​among the nine parameters. The first terminal device indicates the indexes of the four parameters to the network device through the first bit sequence.

[0355] The same is true for the second terminal device and the third terminal device. For example, the second terminal device sends a second bit sequence to the network device, and the second bit sequence is 101000101. Each bit in the second bit sequence corresponds to one of the K parameters of the model of the second terminal device, that is, K is equal to 9. The parameters corresponding to the bits with a value of 1 in the second bit sequence are the four parameters with the largest absolute values ​​of the corresponding values ​​among the nine parameters. The second terminal device indicates the indexes of the four parameters to the network device through the second bit sequence. The third terminal device sends a third bit sequence to the network device, and the third bit sequence is 110001001. Each bit in the third bit sequence corresponds to one of the K parameters of the model of the third terminal device, that is, K is equal to 9. The parameters corresponding to the bits with a value of 1 in the third bit sequence are the four parameters with the largest absolute values ​​of the corresponding values ​​among the nine parameters. The third terminal device indicates the indexes of the four parameters to the network device through the third bit sequence. Optionally, the network device determines a common sparse mask based on the first bit sequence, the second bit sequence, and the third bit sequence. As shown in Figure 5, the public sparse mask is a bit sequence, specifically 110001101. The network device uses this bit sequence to instruct the terminal device to report the model parameters corresponding to the bits in the bit sequence with a value of 1. This reduces the overhead of the terminal device reporting the model parameters and saves communication resources.

[0356] The above-mentioned method embodiments can be implemented individually or in combination. The terms and related technologies involved in each embodiment can refer to each other. In other words, technical solutions that are not inconsistent or logically conflicting between different embodiments can be combined with each other, and this application does not limit this.

[0357] The following describes the first apparatus provided in an embodiment of the present application. Refer to Figure 6, which is a schematic diagram of the structure of the first apparatus in an embodiment of the present application. First apparatus 600 can be used to perform the steps performed by the first apparatus in the embodiments shown in Figures 2 and 4. For details, please refer to the relevant description of the above method embodiments.

[0358] The first device 600 includes a transceiver module 601 and a processing module 602 .

[0359] The transceiver module 601 is configured to receive at least one quantization threshold value from a second device;

[0360] A processing module 602 is configured to quantize the relevant information of the first model of the first device 600 according to at least one quantization threshold value;

[0361] The transceiver module 601 is further configured to send first information to the second device, where the first information is used to indicate relevant information of the first model after quantization processing.

[0362] In a possible implementation, the relevant information of the first model includes: output parameters or update parameters of the first model, and the update parameters include weight gradients or weight parameters of the first model.

[0363] In another possible implementation, the transceiver module 601 is further configured to:

[0364] Sending second information to the second device; wherein the second information is used to indicate information obtained by processing the relevant information of the first model; or,

[0365] The second information is used to indicate information obtained by processing relevant information obtained by the first device 600 performing the Mth round of training on the first model. The relevant information of the first model is relevant information obtained by the first device 600 performing the Qth round of training on the first model. M is an integer greater than or equal to 1 and less than Q, and Q is an integer greater than 1.

[0366] In another possible implementation, the relevant information of the first model includes an output parameter of the first model, and the information obtained by processing the relevant information of the first model includes an average of the absolute values ​​of the output parameter of the first model; or

[0367] The relevant information of the first model includes updated parameters of the first model, and information obtained by processing the relevant information of the first model includes an average value of the absolute values ​​of the updated parameters of the first model.

[0368] In another possible implementation, the transceiver module 601 is further configured to:

[0369] Third information is received from the second device, where the third information is used to indicate global information of the first model.

[0370] In another possible implementation, the global information of the first model includes a global output parameter of the first model; or, the global information of the first model includes a global update parameter and / or a global learning rate of the first model.

[0371] In another possible implementation, the relevant information of the first model includes N parameters of the first model, where N is an integer greater than or equal to 1; and the processing module 602 is specifically configured to:

[0372] quantizing the N parameters according to at least one quantization threshold value to obtain the quantized N parameters;

[0373] The transceiver module 601 is specifically used for:

[0374] Modulating the N parameters after quantization to obtain N first signals;

[0375] N first signals are sent to the second device.

[0376] In another possible implementation, the at least one quantization threshold value includes a first quantization threshold value and a second quantization threshold value; and the processing module 602 is specifically configured to:

[0377] If the i-th parameter among the N parameters is greater than the first quantization threshold, quantize the i-th parameter to a first value, where i is an integer greater than or equal to 1 and less than or equal to N; or

[0378] If the i-th parameter among the N parameters is less than or equal to the first quantization threshold value and greater than or equal to the second quantization threshold value, quantize the i-th parameter to the second value; or,

[0379] If the i-th parameter among the N parameters is less than the second quantization threshold value, the i-th parameter is quantized to a third value.

[0380] In another possible implementation, the transceiver module 601 is specifically configured to:

[0381] Modulating the quantized i-th parameter to obtain an i-th first signal, where the i-th first signal corresponds to two sequences;

[0382] When the i-th parameter after quantization processing is a first value, the transmission power of the first device 600 sending the first sequence of the two sequences is less than the transmission power of the first device 600 sending the second sequence of the two sequences; when the i-th parameter after quantization processing is a second value, the transmission power of the first device 600 sending the first sequence of the two sequences is equal to the transmission power of the first device 600 sending the second sequence of the two sequences; when the i-th parameter after quantization processing is a third value, the transmission power of the first device 600 sending the first sequence of the two sequences is greater than the transmission power of the first device 600 sending the second sequence of the two sequences.

[0383] In another possible implementation, when the i-th parameter after quantization processing is a first value, the first sequence of the two sequences is a non-all-0 sequence, and the second sequence is an all-0 sequence; when the i-th parameter after quantization processing is a second value, both sequences are all-0 sequences; when the i-th parameter after quantization processing is a third value, the first sequence of the two sequences is an all-0 sequence, and the second sequence is a non-all-0 sequence.

[0384] In another possible implementation, the transceiver module 601 is specifically configured to:

[0385] The first information is sent to the second device L times, where L is an integer greater than or equal to 1.

[0386] In another possible implementation, the transceiver module 601 is further configured to:

[0387] First indication information is received from the second device, where the first indication information is used to instruct the first device 600 to send the first information to the second device a number of times L.

[0388] In another possible implementation, the relevant information of the first model includes N parameters of the first model after quantization error compensation. The N parameters after quantization error compensation are obtained by the first device 600 performing error compensation on the N parameters according to the quantization errors corresponding to the N parameters obtained by the first device 600 performing the Q-th round of training on the first model. The quantization error corresponding to the i-th parameter among the N parameters is determined based on the i-th parameter obtained by the first device 600 performing the Q-1-th round of training on the first model and the i-th parameter obtained by the Q-1-th round of training after quantization error compensation. i is an integer greater than or equal to 1 and less than or equal to N, N is an integer greater than or equal to 1, and Q is an integer greater than 1.

[0389] In another possible implementation, the relevant information of the first model includes N parameters of the first model obtained through sparse processing; the N parameters of the first model obtained through sparse processing are N parameters selected by the first device 600 from the K parameters of the first model according to a common sparse mask, and the K parameters of the first model are parameters obtained by the first device 600 after one round of training of the first model, where K is an integer greater than or equal to N, K is an integer greater than or equal to 1, and N is an integer greater than or equal to 1.

[0390] In another possible implementation, the common sparse mask is a bit sequence, which includes K bits, and the K bits correspond one-to-one to K parameters; when the value of one of the K bits is 0, it is used to indicate that the first device 600 does not select the parameter corresponding to the bit; when the value of one of the K bits is 1, it is used to indicate that the first device 600 selects the parameter corresponding to the bit.

[0391] In another possible implementation, the common sparse mask is determined by the first device 600 based on the sparse ratio and a pseudo-random number, and the sparse ratio is indicated by the second device to the first device 600.

[0392] In another possible implementation, the transceiver module 601 is further configured to:

[0393] Second indication information is received from a second device, where the second indication information is used to indicate a common sparse mask.

[0394] In another possible implementation, the transceiver module 601 is further configured to:

[0395] Send third indication information to the second device, where the third indication information is used to indicate the indexes of N parameters whose corresponding values ​​among the K parameters are the largest in absolute value.

[0396] In another possible implementation, the first model is a neural network model, and the relevant information of the first model includes relevant parameters of neurons in a P layer of the neural network model, where P is an integer greater than or equal to 1.

[0397] FIG7 is another structural diagram of the first device according to an embodiment of the present application. Referring to FIG7 , the first device 700 can be used to execute the steps executed by the first device according to the embodiment shown in FIG4 . For details, please refer to the relevant introduction of the above method embodiment.

[0398] The first device 700 includes a transceiver module 701. Optionally, the first device 700 further includes a processing module 702.

[0399] The transceiver module 701 is used to send a first indication message to the second device, where the first indication message is used to indicate the indexes of the N parameters with the largest absolute values ​​among the K parameters of the first model of the first device 700, where the K parameters of the first model are the K parameters obtained by the first device 700 through one round of training of the first model, where K is an integer greater than or equal to N, K is an integer greater than or equal to 1, and N is an integer greater than or equal to 1; and receive a second indication message from the second device; the second indication message is used to indicate a common sparse mask, which is determined by the second device based on the first indication message; and the common sparse mask is used to instruct the first device 700 to report some parameters obtained by the first device through training the first model.

[0400] The following describes the second apparatus provided in the embodiments of the present application. Please refer to Figure 8, which is a schematic diagram of the structure of the second apparatus in the embodiments of the present application. Second apparatus 800 can be used to perform the steps performed by the second apparatus in the embodiments shown in Figures 2 and 4. For details, please refer to the relevant description of the above method embodiments.

[0401] The second device 800 includes a transceiver module 801. Optionally, the second device 800 further includes a processing module 802.

[0402] The transceiver module 801 is used to send at least one quantization threshold value to the first device, and the at least one quantization threshold value is used to quantize the relevant information of the first model of the first device; receive the first information sent from the first device, and the first information is used to indicate the relevant information of the first model after quantization.

[0403] In a possible implementation, the relevant information of the first model includes: output parameters or update parameters of the first model, and the update parameters include weight gradients or weight parameters of the first model.

[0404] In another possible implementation, the transceiver module 801 is further configured to:

[0405] receiving second information from the first device; wherein the second information is used to indicate information obtained by processing relevant information of the first model; or

[0406] The second information is used to indicate information obtained by the first device after performing the Mth round of training on the first model and processing, and the relevant information of the first model is the relevant information obtained by the first device after performing the Qth round of training on the first model, where M is an integer greater than or equal to 1 and less than Q, and Q is an integer greater than 1;

[0407] The processing module 802 is configured to determine at least one quantization threshold value according to the second information.

[0408] In another possible implementation, the relevant information of the first model includes an output parameter of the first model, and the information obtained by processing the relevant information of the first model includes an average of the absolute values ​​of the output parameter of the first model; or

[0409] The relevant information of the first model includes updated parameters of the first model, and information obtained by processing the relevant information of the first model includes an average value of the absolute values ​​of the updated parameters of the first model.

[0410] In another possible implementation, the transceiver module 801 is further configured to:

[0411] Receiving third information from a third device; wherein the third information is used to indicate information obtained by processing relevant information of the second model by the third device; or, the third information is used to indicate information obtained by processing the third device after performing an S-th round of training on the second model, and the relevant information of the second model is relevant information obtained by performing an R-th round of training on the second model by the third device, where S is an integer greater than or equal to 1 and less than R, and R is an integer greater than 1;

[0412] The processing module 802 is configured to determine at least one quantization threshold value according to the second information and the third information.

[0413] In another possible implementation, the processing module 802 is further configured to:

[0414] determining global information of the first model based on the first information;

[0415] The transceiver module 801 is further configured to:

[0416] Fourth information is sent to the first device, where the fourth information is used to indicate global information of the first model.

[0417] In another possible implementation, the global information of the first model includes a global output parameter of the first model; or, the global information of the first model includes a global update parameter and / or a global learning rate of the first model.

[0418] In another possible implementation, the transceiver module 801 is further configured to:

[0419] receiving fifth information from the third device, where the fifth information is used to indicate relevant information about the second model of the third device;

[0420] The processing module 802 is specifically configured to:

[0421] Global information of the first model is determined according to the first information and the fifth information.

[0422] In another possible implementation, the relevant information of the first model includes N parameters of the first model, where N is an integer greater than or equal to 1; the relevant information of the second model includes N parameters of the second model;

[0423] The transceiver module 801 is specifically used for:

[0424] Receiving N first signals from a first device, the N first signals carrying N parameters of a first model, the N first signals corresponding one-to-one to the N parameters of the first model;

[0425] The transceiver module 801 is specifically used for:

[0426] receiving N second signals from a third device, the N second signals carrying N parameters of the second model, the N second signals corresponding one-to-one to the N parameters of the second model;

[0427] The processing module 802 is specifically configured to:

[0428] Global information of the first model is determined according to the N first signals and the N second signals.

[0429] In another possible implementation, the i-th first signal among the N first signals corresponds to the first sequence and the second sequence, the i-th second signal among the N second signals corresponds to the third sequence and the fourth sequence, the time-frequency resources used by the first device to send the first sequence are the same as the time-frequency resources used by the third device to send the third sequence, and the time-frequency resources used by the first device to send the second sequence are the same as the time-frequency resources used by the third device to send the fourth sequence; the global information of the first model includes N global parameters of the first model; i is an integer greater than or equal to 1 and less than or equal to N; and the processing module 802 is specifically used to:

[0430] Determine the sum of first signal energies of the first sequence and the third sequence received by the second device 800;

[0431] Determine the sum of second signal energies of the second sequence and the fourth sequence received by the second device 800;

[0432] An i-th global parameter among the N global parameters is determined according to the first signal energy sum and the second signal energy sum.

[0433] In another possible implementation, the processing module 802 is specifically configured to:

[0434] If the first signal energy sum is less than the sum of the second signal energy sum and the decision threshold, then determining the value of the i-th global parameter to be the first value; or,

[0435] If the first signal energy sum is greater than or equal to the sum of the second signal energy sum and the decision threshold, and the second signal energy sum is less than or equal to the sum of the first signal energy sum and the decision threshold, then determining the value of the i-th global parameter to be the second value; or,

[0436] If the second signal energy sum is greater than the sum of the first signal energy sum and the decision threshold, the value of the i-th global parameter is determined to be a third value.

[0437] In another possible implementation, the transceiver module 801 is further configured to:

[0438] The first indication information is sent to the first device, where the first indication information is used to instruct the first device to send the first information to the second device 800 a number of times L, where L is an integer greater than or equal to 1.

[0439] In another possible implementation, the transceiver module 801 is further configured to:

[0440] Send second indication information to the first device, where the second indication information is used to indicate a common sparse mask, and the common sparse mask is used to instruct the first device to report part of the parameters obtained by the first device training the first model.

[0441] In another possible implementation, the transceiver module 801 is further configured to:

[0442] Receive third indication information from the first device, where the third indication information is used to indicate the indexes of N parameters with the largest absolute values ​​among K parameters obtained by the first device through a round of training on the first model;

[0443] receiving fourth indication information from a third device, the fourth indication information being used to indicate the indices of N parameters having the largest absolute values ​​among K parameters of the second model of the third device, where the K parameters of the second model are K parameters obtained by the third device through one round of training of the second model;

[0444] The processing module 802 is further configured to:

[0445] A common sparse mask is determined according to the third indication information and the fourth indication information.

[0446] FIG9 is another schematic diagram of the structure of the second device according to an embodiment of the present application. Referring to FIG9 , the second device 900 includes a device that can be used to execute the steps performed by the second device according to the embodiment shown in FIG4 . For details, please refer to the relevant description of the above method embodiment.

[0447] The second device 900 includes a transceiver module 901 and a processing module 902 .

[0448] The transceiver module 901 is configured to receive first indication information from a first apparatus, where the first indication information indicates the indexes of N parameters having the largest absolute values ​​among K parameters of a first model of the first apparatus, where the K parameters of the first model are K parameters obtained by the first apparatus after performing one round of training on the first model, where K is an integer greater than or equal to N, K is an integer greater than or equal to 1, and N is an integer greater than or equal to 1;

[0449] A processing module 902 is configured to determine a common sparse mask according to the first indication information, where the common sparse mask is used to instruct the first device to report some parameters obtained by training the first model by the first device;

[0450] The transceiver module 901 is further configured to send second indication information to the first device, where the second indication information is used to indicate a common sparse mask.

[0451] In one possible implementation, the transceiver module 901 is further configured to:

[0452] receiving third indication information from a third device, the third indication information being used to indicate the indices of N parameters with the largest absolute values ​​among K parameters of the second model of the third device, where the K parameters of the second model are K parameters obtained by the third device through one round of training of the second model;

[0453] The processing module 902 is specifically configured to:

[0454] A common sparse mask is determined according to the first indication information and the third indication information.

[0455] The present application also provides a terminal device. Figure 10 is a schematic diagram of the structure of a terminal device 1000 provided in an embodiment of the present application. Terminal device 1000 can be used in the system shown in Figure 1. For example, terminal device 1000 can be the terminal device in the system of Figure 1, configured to perform the functions of the first device or the second device in the above-described method embodiment.

[0456] As shown in the figure, the terminal device 1000 includes a processor 1010 and a transceiver 1020. Optionally, the terminal device 1000 also includes a memory 1030. The processor 1010, the transceiver 1020, and the memory 1030 can communicate with each other via internal connection paths to transmit control and / or data signals. The memory 1030 is used to store computer programs, and the processor 1010 is used to call and execute the computer programs from the memory 1030 to control the transceiver 1020 to transmit and receive signals. Optionally, the terminal device 1000 may also include an antenna 1040 for transmitting uplink data or uplink control signaling output by the transceiver 1020 via wireless signals.

[0457] The processor 1010 and the memory 1030 may be combined into a processing device, and the processor 1010 is configured to execute the program code stored in the memory 1030 to implement the above-mentioned functions. In a specific implementation, the memory 1030 may also be integrated into the processor 1010, or independent of the processor 1010. For example, the processor 1010 may correspond to the processing module 602 in Figure 6. Alternatively, the processor 1010 may correspond to the processing module 702 in Figure 7. Alternatively, the processor 1010 may correspond to the processing module 802 in Figure 8. Alternatively, the processor 1010 may correspond to the processor 902 in Figure 9.

[0458] The transceiver 1020 may correspond to the transceiver module 601 in FIG. 6 , or the transceiver 1002 may correspond to the transceiver module 701 in FIG. 7 Alternatively, the transceiver 1002 may correspond to the transceiver module 801 in FIG. 8 Alternatively, the transceiver 1002 may correspond to the transceiver module 901 in FIG. 9 The transceiver 1020 may also be referred to as a transceiver unit. The transceiver 1020 may include a receiver (or receiver, receiving circuit) and a transmitter (or transmitter, transmitting circuit). The receiver is used to receive signals, and the transmitter is used to transmit signals.

[0459] It should be understood that the terminal device 1000 shown in FIG10 is capable of implementing the various processes involving the first apparatus or the second apparatus in the method embodiments shown in FIG2 and FIG4 . The operations and / or functions of the various modules in the terminal device 1000 are respectively for implementing the corresponding processes in the aforementioned apparatus embodiments. For details, please refer to the description of the aforementioned apparatus embodiments; to avoid repetition, detailed descriptions are omitted here.

[0460] The processor 1010 can be used to execute the actions implemented within the first device or the second device described in the previous device embodiment, and the transceiver 1020 can be used to execute the transceiver actions of the first device or the second device described in the previous device embodiment. For details, please refer to the description of the previous device embodiment, which will not be repeated here.

[0461] Optionally, the terminal device 1000 may further include a power supply 1050 for providing power to various devices or circuits in the terminal device.

[0462] In addition, in order to make the functions of the terminal device more complete, the terminal device 1000 may also include one or more of an input unit 1060, a display unit 1070, an audio circuit 1080, a camera 1090 and a sensor 1000, and the audio circuit may also include a speaker 1082, a microphone 1084, etc.

[0463] This application also provides a network device. Please refer to Figure 11, which is a schematic diagram of the structure of a network device 1100 provided in an embodiment of this application. This network device 1100 can be used in the system shown in Figure 1. For example, network device 1100 can be a network device in the system shown in Figure 1, configured to perform the functions of the first device or the second device in the above-described method embodiment. It should be understood that the following is merely an example, and in future communication systems, network devices may have other forms and configurations.

[0464] For example, in a 5G communication system, the network device 1100 may include a CU, a DU, and an AAU. Compared to a network device in an LTE communication system, which consists of one or more radio frequency units, such as a remote radio unit (RRU) and one or more baseband units (BBU):

[0465] The non-real-time portion of the original BBU will be separated and redefined as a CU, responsible for handling non-real-time protocols and services. Some of the BBU's physical layer processing functions will be merged with the original RRU and passive antennas into the AAU. The remaining BBU functions will be redefined as a DU, responsible for handling physical layer protocols and real-time services. In short, the CU and DU are differentiated by the real-time nature of their processing, and the AAU is a combination of the RRU and antenna.

[0466] The CU, DU, and AAU can be deployed separately or together, resulting in a variety of network deployment configurations. One possible deployment configuration, shown in Figure 11, is consistent with traditional 4G network equipment, with the CU and DU deployed on shared hardware. It should be understood that Figure 11 is merely an example and does not limit the scope of protection of this application. For example, the deployment configuration may also include the DU being deployed in the BBU room, the CU being deployed centrally, or the DU being deployed centrally, with the CU being centralized at a higher level.

[0467] The AAU 11100 can implement transceiver functions and be referred to as a transceiver unit 11100, corresponding to the transceiver module 601 in Figure 6 . Alternatively, the AAU 11100 can implement transceiver functions and be referred to as a transceiver unit 11100, corresponding to the transceiver module 701 in Figure 7 . Alternatively, the AAU 11100 can implement transceiver functions and be referred to as a transceiver unit 11100, corresponding to the transceiver module 801 in Figure 8 . Alternatively, the AAU 11100 can implement transceiver functions and be referred to as a transceiver unit 11100, corresponding to the transceiver module 901 in Figure 9 . Alternatively, the transceiver unit 11100 can also be referred to as a transceiver, a transceiver circuit, or a transceiver, and can include at least one antenna 11101 and a radio frequency unit 11102. Alternatively, the transceiver unit 11100 can include a receiving unit and a transmitting unit. The receiving unit can correspond to a receiver (or a receiver or a receiving circuit), and the transmitting unit can correspond to a transmitter (or a transmitter or a transmitting circuit).

[0468] The CU and DU11200 may implement an internal processing function called a processing unit 11200, corresponding to the processing module 602 in Figure 6. Alternatively, the CU and DU11200 may implement an internal processing function called a processing unit 11200, corresponding to the processing module 702 in Figure 7. The CU and DU11200 may implement an internal processing function called a processing unit 11200, corresponding to the processing module 802 in Figure 8. Alternatively, the CU and DU11200 may implement an internal processing function called a processing unit 11200, corresponding to the processing module 902 in Figure 9. Optionally, the processing unit 11200 may control a network device, etc., and may be referred to as a controller. The AAU and the CU and DU may be physically arranged together or physically separated.

[0469] In addition, the network device is not limited to the form shown in Figure 11, and can also be other forms: for example, including a BBU and an adaptive radio unit (ARU), or including a BBU and an active antenna unit (AAU); it can also be customer premises equipment (CPE), or it can be other forms, which are not limited in this application.

[0470] In one example, the processing unit 11200 can be composed of one or more single boards, and multiple single boards can jointly support a wireless access network with a single access standard (such as an LTE network), or can respectively support wireless access networks with different access standards (such as an LTE network, a 5G network, a future network or other networks). The CU and DU 11200 also include a memory 11201 and a processor 11202. The memory 11201 is used to store necessary instructions and data. The processor 11202 is used to control the network device to perform necessary actions, such as controlling the network device to execute the operation process of the first device or the second device in the above method embodiment. The memory 11201 and the processor 11202 can serve one or more single boards. That is, a memory and a processor can be set separately on each single board. Alternatively, multiple single boards can share the same memory and processor. In addition, necessary circuits can also be set on each single board.

[0471] It should be understood that the network device 1100 shown in Figure 11 can implement the functions of the first device or the second device involved in the method embodiments of Figures 2 and 4. The operations and / or functions of the various units in the network device 1100 are respectively for implementing the corresponding processes performed by the network device in the method embodiments of the present application. To avoid repetition, detailed descriptions are appropriately omitted here. The structure of the network device illustrated in Figure 11 is only one possible form and should not constitute any limitation on the embodiments of the present application. The present application does not exclude the possibility of other forms of network device structures that may appear in the future.

[0472] The CU and DU 11200 can be used to perform the actions implemented within the first device or the second device described in the previous method embodiments, while the AAU 11100 can be used to perform the transceiver actions of the first device or the second device described in the previous method embodiments. For details, please refer to the description in the previous method embodiments and will not be repeated here.

[0473] The present application also provides a computer program product, which includes: computer program code, which, when running on a computer, enables the computer to execute the method of any one of the embodiments shown in Figures 2 and 4.

[0474] The present application also provides a computer-readable medium storing a program code. When the program code is run on a computer, the computer executes the method of any one of the embodiments shown in FIG. 2 and FIG. 4 .

[0475] The present application also provides a communication system, which includes a first device and a second device. The first device is configured to execute some or all of the steps executed by the first device in the embodiments shown in Figures 2 and 4 above, and the second device is configured to execute some or all of the steps executed by the second device in the embodiments shown in Figures 2 and 4 above.

[0476] Optionally, the communication system further includes a third device configured to execute part or all of the steps executed by the third device in the embodiments shown in FIG. 2 and FIG. 4 .

[0477] An embodiment of the present application further provides a chip device, including a processor, configured to call a computer program or computer instruction stored in the memory so that the processor executes the method of the embodiments shown in FIG. 2 and FIG. 4 .

[0478] In a possible implementation, the input of the chip device corresponds to the receiving operation in the embodiments shown in FIG. 2 and FIG. 4 , and the output of the chip device corresponds to the sending operation in the embodiments shown in FIG. 2 and FIG. 4 .

[0479] Optionally, the processor is coupled to the memory via an interface.

[0480] Optionally, the chip device further includes a memory, in which computer programs or computer instructions are stored.

[0481] The processor mentioned in any of the above may be a general-purpose central processing unit, a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the method of the embodiments shown in Figures 2 and 4. The memory mentioned in any of the above may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM), etc.

[0482] Those skilled in the art can clearly understand that, for the sake of convenience and brevity of description, the explanation and beneficial effects of the relevant contents in any of the communication devices provided above can refer to the corresponding method embodiments provided above, and will not be repeated here.

[0483] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0484] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0485] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0486] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0487] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A communication method, characterized in that: The method comprises: The first device receives at least one quantization threshold value from the second device; The first device performs quantization processing on the relevant information of the first model of the first device according to the at least one quantization threshold value; The first device sends first information to the second device, where the first information is used to indicate relevant information of the first model after quantization processing.

2. The method according to claim 1, characterized in that The relevant information of the first model includes: output parameters or update parameters of the first model, and the update parameters include weight gradients or weight parameters of the first model.

3. The method according to claim 1 or 2, characterized in that Before the first device receives at least one quantization threshold value from the second device, the method further includes: The first device sends second information to the second device; wherein the second information is used to indicate information obtained by processing relevant information of the first model; or The second information is used to indicate information obtained by processing relevant information obtained by the first device through the Mth round of training of the first model. The relevant information of the first model is relevant information obtained by the first device through the Qth round of training of the first model. M is an integer greater than or equal to 1 and less than Q, and Q is an integer greater than 1.

4. The method according to claim 3, characterized in that The relevant information of the first model includes an output parameter of the first model, and information obtained by processing the relevant information of the first model includes an average of absolute values ​​of the output parameter of the first model; or The relevant information of the first model includes updated parameters of the first model, and information obtained by processing the relevant information of the first model includes an average value of absolute values ​​of the updated parameters of the first model.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: The first device receives third information from the second device, where the third information is used to indicate global information of the first model.

6. The method according to claim 5, characterized in that The global information of the first model includes a global output parameter of the first model; or, the global information of the first model includes a global update parameter and / or a global learning rate of the first model.

7. The method according to any one of claims 1 to 6, characterized in that The relevant information of the first model includes N parameters of the first model, where N is an integer greater than or equal to 1; The first device quantizes the relevant information of the first model of the first device according to the at least one quantization threshold value, including: The first device quantizes the N parameters according to the at least one quantization threshold value to obtain N quantized parameters; The first information includes the N parameters after the quantization process; the first device sends the first information to the second device, including: The first device modulates the N parameters after the quantization process to obtain N first signals; The first device sends the N first signals to the second device.

8. The method according to claim 7, characterized in that The at least one quantization threshold value includes a first quantization threshold value and a second quantization threshold value; The first device quantizes the N parameters according to the at least one quantization threshold value to obtain the N parameters after quantization, including: If the i-th parameter among the N parameters is greater than the first quantization threshold, the first device quantizes the i-th parameter to a first value, where i is an integer greater than or equal to 1 and less than or equal to N; or If the i-th parameter among the N parameters is less than or equal to the first quantization threshold value and greater than or equal to the second quantization threshold value, the first device quantizes the i-th parameter to a second value; or, If the i-th parameter among the N parameters is less than the second quantization threshold, the first device quantizes the i-th parameter to a third value.

9. The method according to claim 7 or 8, characterized in that The first device modulates the N parameters after the quantization process to obtain N first signals, including: The first device modulates the quantized i-th parameter to obtain an i-th first signal, where the i-th first signal corresponds to two sequences; When the i-th parameter after the quantization processing is the first value, the transmission power of the first sequence of the two sequences sent by the first device is less than the transmission power of the second sequence of the two sequences sent by the first device; when the i-th parameter after the quantization processing is the second value, the transmission power of the first sequence of the two sequences sent by the first device is equal to the transmission power of the second sequence of the two sequences sent by the first device; when the i-th parameter after the quantization processing is the third value, the transmission power of the first sequence of the two sequences sent by the first device is greater than the transmission power of the second sequence of the two sequences sent by the first device.

10. The method according to claim 9, characterized in that When the i-th parameter after the quantization processing is the first value, the first sequence of the two sequences is a non-all-0 sequence, and the second sequence is an all-0 sequence; when the i-th parameter after the quantization processing is the second value, the two sequences are both all-0 sequences; when the i-th parameter after the quantization processing is the third value, the first sequence of the two sequences is an all-0 sequence, and the second sequence is a non-all-0 sequence.

11. The method according to any one of claims 1 to 10, characterized in that The first device sending first information to the second device includes: The first device sends the first information to the second device L times, where L is an integer greater than or equal to 1.

12. The method according to claim 11, characterized in that The method further comprises: The first device receives first indication information from the second device, where the first indication information is used to instruct the first device to send the first information to the second device a number of times L.

13. The method according to any one of claims 1 to 12, characterized in that The relevant information of the first model includes N parameters of the first model after quantization error compensation. The N parameters after quantization error compensation are obtained by the first device performing error compensation on the N parameters according to the quantization errors corresponding to the N parameters obtained by the first device performing the Q-1th round of training on the first model. The quantization error corresponding to the i-th parameter among the N parameters is determined based on the i-th parameter obtained by the first device performing the Q-1th round of training on the first model and after quantization error compensation. i is an integer greater than or equal to 1 and less than or equal to N, N is an integer greater than or equal to 1, and Q is an integer greater than 1.

14. The method according to any one of claims 1 to 13, characterized in that The relevant information of the first model includes N parameters of the first model obtained through sparse processing; the N parameters of the first model obtained through sparse processing are N parameters selected by the first device from K parameters of the first model according to a common sparse mask, and the K parameters of the first model are parameters obtained by the first device through one round of training of the first model, where K is an integer greater than or equal to N, K is an integer greater than or equal to 1, and N is an integer greater than or equal to 1.

15. The method according to claim 14, characterized in that The common sparse mask is a bit sequence, which includes K bits, and the K bits correspond one-to-one to the K parameters; when the value of one of the K bits is 0, it is used to instruct the first device not to select the parameter corresponding to the bit; when the value of one of the K bits is 1, it is used to instruct the first device to select the parameter corresponding to the bit.

16. The method according to claim 14 or 15, characterized in that The public sparse mask is determined by the first device according to a sparse ratio and a pseudo-random number, and the sparse ratio is indicated by the second device to the first device.

17. The method according to claim 14 or 15, characterized in that The method further comprises: The first device receives second indication information from the second device, where the second indication information is used to indicate the common sparse mask.

18. The method according to any one of claims 14 to 17, characterized in that The method further comprises: The first device sends third indication information to the second device, where the third indication information is used to indicate the indexes of N parameters whose corresponding values ​​among the K parameters are the largest in absolute value.

19. The method according to any one of claims 1 to 18, characterized in that The first model is a neural network model, and the relevant information of the first model includes relevant parameters of neurons in a P layer of the neural network model, where P is an integer greater than or equal to 1.

20. A communication method, characterized in that: The method comprises: The second device sends at least one quantization threshold value to the first device, where the at least one quantization threshold value is used to quantize relevant information of the first model of the first device; The second device receives first information sent from the first device, where the first information is used to indicate relevant information of the first model after quantization processing.

21. The method according to claim 20, characterized in that The relevant information of the first model includes: output parameters or update parameters of the first model, and the update parameters include weight gradients or weight parameters of the first model.

22. The method according to claim 20 or 21, characterized in that The method further comprises: The second device receives second information from the first device; wherein the second information is used to indicate information obtained by processing relevant information of the first model; or The second information is used to indicate information obtained by the first device through processing of the Mth round of training of the first model, the relevant information of the first model is relevant information obtained by the first device through the Qth round of training of the first model, where M is an integer greater than or equal to 1 and less than Q, and Q is an integer greater than 1; The second device determines the at least one quantization threshold value according to the second information.

23. The method according to claim 22, characterized in that The relevant information of the first model includes an output parameter of the first model, and information obtained by processing the relevant information of the first model includes an average of absolute values ​​of the output parameter of the first model; or The relevant information of the first model includes updated parameters of the first model, and information obtained by processing the relevant information of the first model includes an average value of absolute values ​​of the updated parameters of the first model.

24. The method according to claim 22 or 23, characterized in that The method further comprises: The second device receives third information from the third device; wherein the third information is used to indicate information obtained by processing relevant information of the second model of the third device; or, the third information is used to indicate information obtained by the third device after performing an S-th round of training on the second model, and the relevant information of the second model is relevant information obtained by the third device after performing an R-th round of training on the second model, where S is an integer greater than or equal to 1 and less than R, and R is an integer greater than 1; The second device determines the at least one quantization threshold value according to the second information, including: The second device determines the at least one quantization threshold value according to the second information and the third information.

25. The method according to any one of claims 20 to 24, characterized in that The method further comprises: The second device determines global information of the first model based on the first information; The second device sends fourth information to the first device, where the fourth information is used to indicate global information of the first model.

26. The method according to claim 25, characterized in that The global information of the first model includes a global output parameter of the first model; or, the global information of the first model includes a global update parameter and / or a global learning rate of the first model.

27. The method according to claim 25 or 26, characterized in that The method further comprises: The second device receives fifth information from the third device, where the fifth information is used to indicate relevant information of the second model of the third device; The second device determines the global information of the first model according to the first information, including: The second device determines global information of the first model according to the first information and the fifth information.

28. The method according to claim 27, characterized in that The relevant information of the first model includes N parameters of the first model, where N is an integer greater than or equal to 1; the relevant information of the second model includes N parameters of the second model; the first information includes the N parameters of the first model after quantization; and the fifth information includes the N parameters of the second model after quantization; The second device receives the first information sent by the first device, including: The second device receives N first signals from the first device, the N first signals carrying N parameters of the first model after quantization processing, and the N first signals correspond one-to-one to the N parameters of the first model after quantization processing; The second device receives fifth information from the third device, including: The second device receives N second signals from the third device, the N second signals carrying the N parameters of the second model after the quantization processing, and the N second signals correspond one-to-one to the N parameters of the second model after the quantization processing; The second device determines the global information of the first model according to the first information and the fifth information, including: The second device determines global information of the first model according to the N first signals and the N second signals.

29. The method according to claim 28, characterized in that The i-th first signal among the N first signals corresponds to the first sequence and the second sequence, the i-th second signal among the N second signals corresponds to the third sequence and the fourth sequence, the time-frequency resources used by the first apparatus to send the first sequence are the same as the time-frequency resources used by the third apparatus to send the third sequence, and the time-frequency resources used by the first apparatus to send the second sequence are the same as the time-frequency resources used by the third apparatus to send the fourth sequence; the global information of the first model includes N global parameters of the first model; i is an integer greater than or equal to 1 and less than or equal to N; The second device determines global information of the first model according to the N first signals and the N second signals, including: determining, by the second device, a sum of first signal energies of the first sequence and the third sequence received by the second device; determining, by the second device, a second signal energy sum of the second sequence and the fourth sequence received by the second device; The second device determines an i-th global parameter among the N global parameters according to the first signal energy sum and the second signal energy sum.

30. The method according to claim 29, wherein The second device determines the i-th global parameter among the N global parameters according to the first signal energy sum and the second signal energy sum, including: If the sum of the first signal energy sum and the decision threshold value is less than the second signal energy sum, the second device determines that the value of the i-th global parameter is the first value; or, If the sum of the first signal energy sum and the decision threshold is greater than or equal to the second signal energy sum, and the sum of the second signal energy sum and the decision threshold is greater than or equal to the first signal energy sum, then the second device determines that the value of the i-th global parameter is a second value; or If the sum of the second signal energy sum and the decision threshold value is less than the first signal energy sum, the second device determines that the value of the i-th global parameter is a third value.

31. The method according to any one of claims 20 to 30, characterized in that The method further comprises: The second device sends first indication information to the first device, where the first indication information is used to instruct the first device to send the first information to the second device a number of times L, where L is an integer greater than or equal to 1.

32. The method according to any one of claims 20 to 31, characterized in that The method further comprises: The second device sends second indication information to the first device, where the second indication information is used to indicate a common sparse mask, and the common sparse mask is used to instruct the first device to report part of the parameters obtained by the first device through training the first model.

33. The method according to claim 32, characterized in that The method further comprises: The second device receives third indication information from the first device, where the third indication information is used to indicate the indexes of N parameters with the largest absolute values ​​among K parameters obtained by the first device through a round of training on the first model; The second device receives fourth indication information from the third device, where the fourth indication information is used to indicate the indexes of N parameters with the largest absolute values ​​among the K parameters of the second model of the third device, where the K parameters of the second model are K parameters obtained by the third device through one round of training of the second model; The second device determines the common sparse mask according to the third indication information and the fourth indication information.

34. A communication method, characterized in that: The method comprises: The first device sends first indication information to the second device, where the first indication information is used to indicate the indexes of N parameters with the largest absolute values ​​among K parameters of the first model of the first device, where the K parameters of the first model are K parameters obtained by the first device through one round of training of the first model, where K is an integer greater than or equal to N, K is an integer greater than or equal to 1, and N is an integer greater than or equal to 1; The first device receives second indication information from the second device, where the second indication information is used to indicate a common sparse mask, which is determined by the second device based on the first indication information; the common sparse mask is used to instruct the first device to report part of the parameters obtained by the first device training the first model.

35. A communication method, characterized in that: The method comprises: The second device receives first indication information from the first device, where the first indication information is used to indicate the indexes of N parameters with the largest absolute values ​​among K parameters of the first model of the first device, where the K parameters of the first model are K parameters obtained by the first device through one round of training of the first model, where K is an integer greater than or equal to N, K is an integer greater than or equal to 1, and N is an integer greater than or equal to 1; The second device determines a common sparse mask according to the first indication information, where the common sparse mask is used to instruct the first device to report part of the parameters obtained by the first device training the first model; The second device sends second indication information to the first device, where the second indication information is used to indicate the common sparse mask.

36. The method according to claim 35, characterized in that The method further comprises: The second device receives third indication information from the third device, where the third indication information is used to indicate the indexes of N parameters with the largest absolute values ​​among K parameters of the second model of the third device, where the K parameters of the second model are K parameters obtained by the third device through one round of training of the second model; The second device determines a common sparse mask according to the first indication information, including: The second device determines the common sparse mask according to the first indication information and the third indication information.

37. A first device, characterized in that: The first device includes a transceiver module and a processing module; the transceiver module is used to perform the transceiver operation according to any one of claims 1 to 19, and the processing module is used to perform the processing operation according to any one of claims 1 to 19.

38. A first device, characterized in that The first device includes a transceiver module; the transceiver module is used to perform the transceiver operation as described in claim 34.

39. A second device, characterized in that: The second device includes a transceiver module, and the transceiver module is used to perform the transceiver operation according to any one of claims 20 to 33.

40. The second device according to claim 39, characterized in that The second device further comprises a processing module; the processing module is configured to perform the processing operation according to any one of claims 20 to 33.

41. A second device, characterized in that: The second device includes a transceiver module and a processing module, the transceiver module is used to perform the transceiver operation as described in claim 35 or 36, and the processing module is used to perform the processing operation as described in claim 35 or 36.

42. A device, characterized in that The device includes a processor; the processor is used to execute a computer program or computer instructions in a memory to perform the method as described in any one of claims 1 to 19; or, the processor is used to execute a computer program or computer instructions in the memory to perform the method as described in any one of claims 20 to 33; or, the processor is used to execute a computer program or computer instructions in the memory to perform the method as described in claim 34; or, the processor is used to execute a computer program or computer instructions in the memory to perform the method as described in claim 35 or 36.

43. The device according to claim 42, characterized in that The apparatus further comprises the memory.

44. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by the device, the device performs the method as claimed in any one of claims 1 to 19, or the method as claimed in any one of claims 20 to 33, or the method as claimed in claim 34, or the method as claimed in claim 35 or 36.

45. A computer program product, characterized in that When the computer program product runs on a computer, the computer executes the method according to any one of claims 1 to 19, or the method according to any one of claims 20 to 33, or the method according to claim 34, or the method according to claim 35 or 36.

Images