Communication method and communication apparatus

Abstract

Provided in the present application are a communication method and a communication apparatus. The method comprises: receiving first information, wherein the first information comprises a first index and information for indicating a first mean value, the first mean value is used for calculating a first variance, and the first variance is a variance of an input of a first network layer in an AI model; and sending second information, wherein the second information comprises a second index and information for indicating the first variance, and the first index has an association relationship with the second index. The solution in the embodiments of the present application is conductive to obtaining a more accurate first variance, such that a more accurate global variance can be obtained by means of performing aggregation on the basis of the more accurate first variance, thereby facilitating an improvement in the accuracy of batch normalization processing during distributed training, and improving the performance of the distributed training.

Description

Method and apparatus for communication

[0001] The present application claims priority to the Chinese patent application No. 202410725330.0, filed on June 5, 2024, with the State Intellectual Property Office of China, and the Chinese patent application No. 202410725330.0 has the title of “Method and apparatus for communication”, the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of communication, and more particularly, to a method and apparatus for communication. BACKGROUND

[0003] In a wireless communication network, such as a mobile communication network, the services supported by the network are increasingly diverse, and thus the needs to be met are increasingly diverse. In order to support diverse services, artificial intelligence (AI) technology can be introduced into the wireless communication network, thereby realizing intelligentization of the network. Most machine learning (ML), such as a learning algorithm based on a deep neural network, requires a large amount of data for training. A centralized training manner is usually adopted to train an AI model, i.e., the AI model and a training data set are centralized on a computing node, and the training of the AI model is completed by the computing node. However, the centralized training has some problems. For example, the training data of the centralized training usually comes from user equipment, i.e., the user needs to upload data. It may require a large overhead to upload these data, and meanwhile, collecting and using these data may infringe the privacy of the user. Distributed training can solve the above problems. In the distributed training, the training is usually completed by a plurality of sub-computing nodes and a center computing node. The sub-computing nodes can train the AI model based on local training data, and complete the training by uploading the parameters of the AI model and cooperating with the center computing node.

[0004] However, how to improve the performance of the distributed training is a problem to be solved. SUMMARY

[0005] The present application provides a method and apparatus for communication, so as to improve the accuracy of batch normalization in the distributed training process, thereby facilitating the improvement of the performance of the distributed training.

[0006] In a first aspect, a method for communication is provided, which can be executed by a communication apparatus or a module (e.g., a chip or a circuit, etc.) applied to the communication apparatus. The communication apparatus can be a sub-node in the method embodiment. The sub-node and a center node can cooperate to complete the training of an AI model.

[0007] The child node can be a device on the terminal device side or a device on the network device side. The terminal device side can include at least one of a terminal device or an AI entity on the terminal device side. The AI entity on the terminal device side can be the terminal device itself or an AI entity serving the terminal device, such as a server, for example, an over the top (OTT) server or a cloud server. The network device side can include at least one of a network device or an AI entity on the network device side. The AI entity on the network device side can be the network device itself or an AI entity serving the network device, for example, a radio access network (RAN) intelligent controller (RIC), an operation administration and maintenance (OAM), or a server, for example, an OTT server or a cloud server.

[0008] The method includes: receiving first information, the first information including a first index and information indicating a first mean value, the first mean value being used for calculation of a first variance of an input of a first network layer in an AI model; and sending second information, the second information including a second index and information indicating the first variance, the first index and the second index having a correlation relationship.

[0009] In the scheme of the embodiments of the present application, the child node calculates the variance of the input of the network layer in the local AI model according to the first mean value from the center node, that is, the first variance, instead of calculating the first variance according to the mean value of the input of the network layer in the local AI model based on the local data set, which is beneficial to avoid the influence of the non-independent and identically distributed local data of each child node, obtain a more accurate first variance, and enable the center node to obtain a more accurate global variance based on the more accurate first variance, thereby being beneficial to improve the accuracy of batch normalization processing in the distributed training process and improve the performance of the distributed training.

[0010] At the same time, the index can be used to determine the correlation between the local variance and the global mean value, so as to avoid the child node from calculating the local variance based on the irrelevant global mean value, for example, avoid the child node from calculating the variance of the input of the first network layer based on the local mean value corresponding to other network layers, or avoid the child node from calculating the variance of the input of the first network layer in the current round based on the global mean value in other rounds, thereby being beneficial to ensure the accuracy of the variance.

[0011] Exemplarily, the first network layer can be any BN layer in the AI model. Exemplarily, the input of the first network layer can also be replaced by a feature or a feature channel of the first network layer.

[0012] Optionally, the method can further include: receiving the model parameter of the AI model.

[0013] In the scheme of the embodiments of the application, the sub-node can calculate the variance of the input of the first network layer based on the received model parameter of the AI model before the start of the current round of training, which is conducive to ensuring that each sub-node uses a unified AI model to calculate the variance of the input of the first network layer, or in other words, is conducive to ensuring that each sub-node uses a unified AI model when calculating the variance of the input of the first network layer of the local model, thereby facilitating improvement of the accuracy of the global variance obtained by aggregation.

[0014] In combination with the first aspect, in some implementations of the first aspect, the first index is related to the first mean value.

[0015] The first index and the first mean value are related to the same round and / or the same network layer.

[0016] A round can be understood as starting from the center node sending the model parameter of the global model to each sub-node to the end of the center node aggregating to generate a new global model.

[0017] Exemplarily, the first index and the first mean value are carried in the same signaling.

[0018] In combination with the first aspect, in some implementations of the first aspect, the index values of the first index and the second index are the same, or the first index and the second index have a predefined or preconfigured association relationship.

[0019] In combination with the first aspect, in some implementations of the first aspect, the first index indicates the identity of the first network layer.

[0020] In combination with the first aspect, in some implementations of the first aspect, the first index indicates the round of training related to the first mean value.

[0021] In the scheme of the embodiments of the application, the first index can indicate the identity of the first network layer, which is conducive to avoiding the sub-node from calculating the variance of the input of the first network layer based on the local mean value corresponding to other network layers. The first index can indicate the round of training related to the first mean value, which is conducive to avoiding the sub-node from calculating the variance of the input of the network layer of the current round based on the global mean value of other rounds, thereby facilitating improvement of the accuracy of the variance.

[0022] In combination with the first aspect, in some implementations of the first aspect, the method further includes: sending third information, the third information being used to request the model parameter of the AI model.

[0023] Optionally, sending the third information includes: in a case where the round indicated by the first index is different from the round at the sub-node, sending the third information.

[0024] For example, the index value of the first index includes a number of a round of training related to the first mean value. In a case where the number of the round indicated by the first index is greater than the number of the round at the child node, the third information is sent.

[0025] According to the scheme of the embodiments of the application, the child node can request the model parameter of the AI model from the center node, which is conducive to ensuring that each child node calculates the local variance based on the unified AI model, thereby being conducive to further improving the accuracy of the global variance obtained by aggregation.

[0026] In combination with the first aspect, in some implementations of the first aspect, after the second information is sent, the method further includes: receiving fourth information, the fourth information including a third index and information indicating a second variance, at least one of the first index and the second index having an association relationship with the third index, the first mean value and the second variance being used for training of the AI model.

[0027] Optionally, the second variance can be obtained by aggregating the second variances indicated by the fourth information from the plurality of child nodes. For example, the second variance can be an average of the second variances indicated by the fourth information from the plurality of child nodes. For another example, the second variance can be a weighted average of the second variances indicated by the fourth information from the plurality of child nodes.

[0028] In combination with the first aspect, in some implementations of the first aspect, before the first information is received, the method further includes: sending fifth information, the fifth information including a fourth index and information indicating a second mean value, the second mean value being a mean value of an input of a first network layer in the AI model, at least one of the first index and the second index having an association relationship with the fourth index.

[0029] Optionally, the first mean value can be obtained by aggregating the second mean values indicated by the fifth information from the plurality of child nodes. For example, the first mean value can be an average of the second mean values indicated by the fifth information from the plurality of child nodes. For another example, the first mean value can be a weighted average of the second mean values indicated by the fifth information from the plurality of child nodes.

[0030] The second aspect provides a communication method, which can be executed by a communication device or a module (for example, a chip or a circuit, etc.) applied to the communication device. The communication device can be the center node in the method embodiment. The center node can cooperate with the child node to complete the training of the AI model.

[0031] The central node can be a device at a terminal device side or a device at a network device side. The terminal device side can include at least one of a terminal device or an AI entity at the terminal device side. The AI entity at the terminal device side can be the terminal device itself or an AI entity serving the terminal device, for example, a server such as an OTT server or a cloud server. The network device side can include at least one of a network device or an AI entity at the network device side. The AI entity at the network device side can be the network device itself or an AI entity serving the network device, for example, a RIC, an OAM, or a server such as an OTT server or a cloud server.

[0032] The method includes: sending first information to a plurality of devices, the first information including a first index and information indicating a first mean value, the first mean value being used for calculation of a first variance, the first variance being a variance of an input of a first network layer in an AI model of each device in the plurality of devices; and receiving second information from each device in the plurality of devices, the second information including a second index and information indicating the first variance, the first index and the second index having a correlation relationship.

[0033] The plurality of devices corresponding to the destination of the first information and the plurality of devices corresponding to the source of the second information can be the same or different. The plurality of devices corresponding to the destination of the first information and the plurality of devices corresponding to the source of the second information are at least partially the same. For example, the plurality of devices corresponding to the destination of the first information can include the plurality of devices corresponding to the source of the second information.

[0034] In the scheme of the embodiments of the present application, the central node can send the first mean value to the sub-node, so that the sub-node can calculate the variance of the input of the network layer in the local AI model based on the first mean value, that is, the first variance, instead of calculating the first variance based on the mean value of the input of the network layer in the local AI model obtained based on the local data set of the sub-node, which is beneficial to avoiding the influence of the non-independent and identically distributed local data of each sub-node, obtaining a more accurate first variance, so as to enable the central node to aggregate a more accurate global variance based on the more accurate first variance, thereby facilitating to improve the accuracy of batch normalization processing in the distributed training process and improve the performance of the distributed training. In combination with the second aspect, in some implementation manners of the second aspect, the first index and the first mean value are related.

[0035] In combination with the second aspect, in some implementation manners of the second aspect, the index values of the first index and the second index are the same, or the first index and the second index have a predefined or preconfigured correlation relationship.

[0036] In combination with the second aspect, in some implementation manners of the second aspect, the first index indicates an identifier of the first network layer.

[0037] With reference to the second aspect, in some implementations of the second aspect, the first index is associated with a round of training of the first mean.

[0038] With reference to the second aspect, in some implementations of the second aspect, the method further includes receiving third information from a first device of the plurality of devices, the third information being used to request model parameters of the AI model.

[0039] With reference to the second aspect, in some implementations of the second aspect, the method further includes sending fourth information to the plurality of devices, the fourth information including a third index and information indicating a second variance, at least one of the first index and the second index having an association with the third index, the first mean and the second variance being used for training of the AI model.

[0040] The plurality of devices corresponding to the destination of the first information and the plurality of devices corresponding to the destination of the fourth information can be the same or different. The plurality of devices corresponding to the source of the second information and the plurality of devices corresponding to the destination of the fourth information can be the same or different.

[0041] With reference to the second aspect, in some implementations of the second aspect, the method further includes determining the second variance according to the first variances indicated by the second information from each device of the plurality of devices.

[0042] For example, the second variance can be an average of the second variances indicated by the fourth information from each device of the plurality of devices.

[0043] With reference to the second aspect, in some implementations of the second aspect, the second variance satisfies:

[0044] wherein, denotes the second variance, M denotes the number of the plurality of devices, |B j denotes the number of training samples on the jth device of the plurality of devices, denotes the first variance indicated by the second information from the jth device, M is a positive integer, |B j is a positive integer.

[0045] With reference to the second aspect, in some implementations of the second aspect, before the first information is sent to the plurality of devices, the method further includes receiving fifth information from each device of the plurality of devices, the fifth information including a fourth index and information indicating a second mean, the second mean being a mean of inputs of the first network layer in the AI model of each device of the plurality of devices, at least one of the first index and the second index having an association with the fourth index.

[0046] The first information is sent to the plurality of devices corresponding to the destination of the first information, and the fifth information is sent to the plurality of devices corresponding to the source of the fifth information.

[0047] With reference to the second aspect, in some implementations of the second aspect, the method further includes determining the first information based on the fifth information of each of the plurality of devices, wherein the first mean value is based on the second mean value.

[0048] In a third aspect, a method of communication is provided, which can be performed by a communication device or a module (e.g., a chip or a circuit, etc.) applied to the communication device. The communication device can be a child node in the method embodiment. The child node and the center node can cooperate to complete the training of the AI model.

[0049] The child node can be a device on a terminal device side or a device on a network device side. The terminal device side can include at least one of a terminal device or an AI entity on the terminal device side. The AI entity on the terminal device side can be the terminal device itself, or an AI entity serving the terminal device, such as an over the top (OTT) server or a cloud server. The network device side can include at least one of a network device or an AI entity on the network device side. The AI entity on the network device side can be the network device itself, or an AI entity serving the network device, such as a radio access network (RAN) intelligent controller (RIC), an operation administration and maintenance (OAM), or a server, such as an OTT server or a cloud server.

[0050] The method includes sending seventh information, the seventh information including information indicating a third mean value and a fourth mean value, the third mean value being a mean value of an input of a first network layer in the AI model, and the fourth mean value being a mean value of a square of the input of the first network layer.

[0051] In the scheme of the embodiments of the present application, the sub-node sends the mean of the input of the first network layer of the AI model and the mean of the square of the input to the center node, so that the center node can calculate the global variance based on the mean of the input of the first network layer of the AI model on the sub-node and the mean of the square of the input, which is helpful to avoid the influence of the non-independent and identically distributed local data of each sub-node, so as to enable the center node to aggregate a more accurate global variance, thereby facilitating to improve the accuracy of batch normalization processing in the distributed training process and improve the performance of the distributed training.

[0052] Meanwhile, in the embodiments of the present application, the sub-node and the center node interact less frequently, which is helpful to reduce the interaction overhead.

[0053] Optionally, the method can further include receiving the model parameters of the AI model.

[0054] In the scheme of the embodiments of the present application, the sub-node can calculate the mean of the input of the first network layer and the mean of the square of the input based on the received model parameters of the AI model before the start of the current round of training, which is helpful to ensure that each sub-node uses a unified AI model to calculate the third mean and the fourth mean, thereby facilitating to improve the accuracy of the global mean and the global variance determined based on the third mean and the fourth mean.

[0055] In combination with the third aspect, in some implementations of the third aspect, the method further includes receiving eighth information, the eighth information including information indicating a fifth mean and a third variance, the fifth mean and the third variance being related to the first network layer, and the fifth mean and the third variance being used for training of the AI model.

[0056] Optionally, the fifth mean can be obtained based on the third mean indicated by the seventh information from the plurality of sub-nodes. For example, the fifth mean is an average of the third mean indicated by the seventh information from the plurality of sub-nodes. For another example, the fifth mean is a weighted average of the third mean indicated by the seventh information from the plurality of sub-nodes.

[0057] In combination with the third aspect, in some implementations of the third aspect, the third variance is based on the third mean and the fourth mean.

[0058] In combination with the third aspect, in some implementations of the third aspect, the seventh information further includes a fifth index, the eighth information further includes a sixth index, and the fifth index and the sixth index have an association relationship.

[0059] In combination with the third aspect, in some implementations of the third aspect, at least one of the third mean and the fourth mean is related to the fifth index.

[0060] In the scheme of the embodiments of the present application, the correlation between the mean value and the variance can be determined through the index, so as to avoid the center node from calculating the global variance based on the irrelevant mean value, for example, avoiding the center node from calculating the global mean value and the global variance corresponding to the first network layer based on the mean value of the input of the other network layer and the mean value of the square of the input, or avoiding the center node from calculating the global mean value and the global variance corresponding to the first network layer of the current round based on the mean value of the input of the first network layer of the other round and the mean value of the square of the input, thereby facilitating to ensure the accuracy of the mean value and the variance.

[0061] With reference to the third aspect, in some implementations of the third aspect, the fifth index and the sixth index have the same index value, or the fifth index and the sixth index have a preconfigured or predefined correlation.

[0062] With reference to the third aspect, in some implementations of the third aspect, the fifth index indicates an identifier of the first network layer.

[0063] With reference to the third aspect, in some implementations of the third aspect, the fifth index indicates a round of training related to the third mean value and / or the fourth mean value.

[0064] The fourth aspect provides a communication method, which can be executed by a communication device or a module (for example, a chip or a circuit, etc.) applied to the communication device, and the communication device can be the center node in the method embodiments.

[0065] The center node can be a device on the terminal device side or a device on the network device side. The terminal device side can include at least one of a terminal device or an AI entity on the terminal device side. The AI entity on the terminal device side can be the terminal device itself, or an AI entity serving the terminal device, for example, a server, such as an OTT server or a cloud server. The network device side can include at least one of a network device or an AI entity on the network device side. The AI entity on the network device side can be the network device itself, or an AI entity serving the network device, for example, a RIC, an OAM, or a server, such as an OTT server or a cloud server.

[0066] The method includes: receiving seventh information from each of the plurality of devices, the seventh information including information indicating a third mean value and a fourth mean value, the third mean value being an input of a first network layer in an artificial intelligence, AI, model of each of the plurality of devices, and the fourth mean value being a mean value of a square of the input of the first network layer.

[0067] In the scheme of the embodiments of the present application, the center node can calculate the global variance based on the mean of the input of the first network layer of the AI model on the sub-node and the mean of the square of the input, which is advantageous to avoid the influence of the local data of each sub-node being non-independent and identically distributed, so as to enable the center node to aggregate a more accurate global variance, thereby being advantageous to improve the accuracy of the batch normalization processing in the distributed training process and improve the performance of the distributed training.

[0068] Meanwhile, in the embodiments of the present application, the sub-node and the center node interact less frequently, which is advantageous to reduce the interaction overhead.

[0069] In combination with the fourth aspect, in some implementation manners of the fourth aspect, the method further includes: sending eighth information to the plurality of devices, the eighth information including information indicating a fifth mean and a third variance, the fifth mean and the third variance being related to the first network layer, and the fifth mean and the third variance being used for training of the AI model.

[0070] The plurality of devices corresponding to the source end of the seventh information and the plurality of devices corresponding to the destination end of the eighth information can be the same or different.

[0071] In combination with the fourth aspect, in some implementation manners of the fourth aspect, the method further includes: determining the third variance according to the third mean and the fourth mean indicated by the seventh information from each device of the plurality of devices.

[0072] In combination with the fourth aspect, in some implementation manners of the fourth aspect, the third variance satisfies:

[0073] wherein, denotes the third variance, denotes the sixth mean, the sixth mean being an average of the fourth mean indicated by the seventh information from each device of the plurality of devices, and l denotes the fifth mean, the fifth mean being an average of the third mean indicated by the seventh information from each device of the plurality of devices, and B denotes a training data set on the plurality of devices.

[0074] In combination with the fourth aspect, in some implementation manners of the fourth aspect, the seventh information further includes a fifth index, and the eighth information further includes a sixth index, the fifth index and the sixth index having an association relationship.

[0075] In combination with the fourth aspect, in some implementation manners of the fourth aspect, at least one of the third mean and the fourth mean is related to the fifth index.

[0076] In combination with the fourth aspect, in some implementation manners of the fourth aspect, the index values of the fifth index and the sixth index are the same, or the fifth index and the sixth index have a preconfigured or predefined association relationship.

[0077] In some implementations of the fourth aspect, in combination with the fourth aspect, the fifth index indicates an identity of the first network layer.

[0078] In some implementations of the fourth aspect, in combination with the fourth aspect, the fifth index indicates a round of training related to the third mean and / or the fourth mean.

[0079] In the fifth aspect, a communication apparatus can be a terminal device, or a device, module, circuit, chip or the like configured to be arranged in a terminal device, or an apparatus capable of being used in combination with a terminal device. In one design, the communication apparatus can include a module corresponding to each of the steps / operations / actions in the method described in the first aspect or the third aspect. The module can be hardware circuitry, software, or a combination of hardware circuitry and software. In one design, the communication apparatus can include a processing module and a communication module.

[0080] The sending module is configured to perform the sending actions in the method described in the first aspect or the third aspect, the processing module is configured to perform the processing actions in the method described in the first aspect or the third aspect, and the receiving module is configured to perform the receiving actions in the method described in the first aspect or the third aspect.

[0081] In the sixth aspect, a communication apparatus can be a network device, or a device, module, circuit, chip or the like configured to be arranged in a network device, or an apparatus capable of being used in combination with a network device. In one design, the communication apparatus can include a module corresponding to each of the steps / operations / actions in the method described in the second aspect or the fourth aspect. The module can be hardware circuitry, software, or a combination of hardware circuitry and software. In one design, the communication apparatus can include a processing module and a communication module.

[0082] The receiving module is configured to perform the receiving actions in the method described in the second aspect or the fourth aspect, the processing module is configured to perform the processing actions in the method described in the second aspect or the fourth aspect, and the sending module is configured to perform the sending actions in the method described in the second aspect or the fourth aspect.

[0083] In a seventh aspect, a communication apparatus is provided, which comprises one or more processors coupled with one or more storage media storing instructions that, when executed by the one or more processors, cause the method in the first aspect or any possible implementation of the method in the first aspect to be implemented, cause the method in the second aspect or any possible implementation of the method in the second aspect to be implemented, cause the method in the third aspect or any possible implementation of the method in the third aspect to be implemented, or cause the method in the fourth aspect or any possible implementation of the method in the fourth aspect to be implemented.

[0084] In an eighth aspect, a communication apparatus is provided, which comprises one or more processors configured to process data and / or information, so as to cause the method in the first aspect or any possible implementation of the method in the first aspect to be implemented, cause the method in the second aspect or any possible implementation of the method in the second aspect to be implemented, cause the method in the third aspect or any possible implementation of the method in the third aspect to be implemented, or cause the method in the fourth aspect or any possible implementation of the method in the fourth aspect to be implemented.

[0085] Optionally, the communication apparatus can further comprise a communication interface configured to receive data and / or information, and transmit the received data and / or information to the processor. Optionally, the communication interface is further configured to output the data and / or information processed by the processor.

[0086] In a ninth aspect, a chip is provided, which comprises a processor configured to execute programs or instructions, so as to cause the method in the first aspect or any possible implementation of the method in the first aspect to be implemented, cause the method in the second aspect or any possible implementation of the method in the second aspect to be implemented, cause the method in the third aspect or any possible implementation of the method in the third aspect to be implemented, or cause the method in the fourth aspect or any possible implementation of the method in the fourth aspect to be implemented.

[0087] Optionally, the chip can further comprise a memory configured to store the programs or instructions. Optionally, the chip can further comprise the transceiver.

[0088] Optionally, the chip is an application specific integrated circuit (ASIC) or a system on chip (SoC).

[0089] In a tenth aspect, a computer-readable storage medium is provided, the computer-readable storage medium comprising instructions that, when executed by a processor, cause the method according to the first aspect or any possible implementation of the first aspect to be implemented, cause the method according to the second aspect or any possible implementation of the second aspect to be implemented, cause the method according to the third aspect or any possible implementation of the third aspect to be implemented, or cause the method according to the fourth aspect or any possible implementation of the fourth aspect to be implemented.

[0090] In an eleventh aspect, a computer program product is provided, the computer program product comprising computer program code or instructions, which, when executed by a processor, cause the method according to the first aspect or any possible implementation of the first aspect to be implemented, cause the method according to the second aspect or any possible implementation of the second aspect to be implemented, cause the method according to the third aspect or any possible implementation of the third aspect to be implemented, or cause the method according to the fourth aspect or any possible implementation of the fourth aspect to be implemented.

[0091] In a twelfth aspect, a communication system is provided, the communication system comprising one or more of the following means in combination: a communication device configured to perform the method according to the first aspect or any possible implementation of the first aspect, a communication device configured to perform the method according to the second aspect or any possible implementation of the second aspect, a communication device configured to perform the method according to the third aspect or any possible implementation of the third aspect, or a communication device configured to perform the method according to the fourth aspect or any possible implementation of the fourth aspect. For example, the communication system can comprise the communication device according to the fifth aspect, and / or the communication device according to the sixth aspect. BRIEF DESCRIPTION OF DRAWINGS

[0092] Fig. 1 is a schematic diagram of a communication system suitable for use with embodiments of the application;

[0093] Fig. 2 is a schematic diagram of another communication system suitable for use with embodiments of the application;

[0094] Fig. 3 is a schematic diagram of yet another communication system suitable for use with embodiments of the application;

[0095] Fig. 4 is a schematic diagram of an application framework of a communication system suitable for use with embodiments of the application;

[0096] Fig. 5 is a schematic flowchart of a method of communication according to embodiments of the application;

[0097] Fig. 6 is a schematic diagram of information indicating a global mean according to embodiments of the application;

[0098] Fig. 7 is a schematic diagram of a format of an association according to embodiments of the application;

[0099] FIG. 8 is a schematic flowchart of another method of communication according to embodiments of the present application;

[0100] FIG. 9 is a schematic diagram of an example of information #4 according to embodiments of the present application;

[0101] FIG. 10 is a schematic diagram of an example of information #1 according to embodiments of the present application;

[0102] FIG. 11 is a schematic diagram of an example of information #2 according to embodiments of the present application;

[0103] FIG. 12 is a schematic flowchart of another method of communication according to embodiments of the present application;

[0104] FIG. 13 is a schematic flowchart of another method of communication according to embodiments of the present application;

[0105] FIG. 14 is a schematic block diagram of an apparatus for communication according to embodiments of the present application;

[0106] FIG. 15 is a schematic block diagram of another apparatus for communication according to embodiments of the present application. DETAILED DESCRIPTION

[0107] The technical solutions in the present application will be described below with reference to the accompanying drawings.

[0108] The technical solutions provided by the present application can be applied to various communication systems, for example: a 5th generation (5G) or new radio (NR) system, a long term evolution (LTE) system, an LTE frequency division duplex (FDD) system, an LTE time division duplex (TDD) system, a wireless local area network (WLAN) system, a satellite communication system, a future communication system such as a future mobile communication network system, or a fusion system of multiple systems, etc. The technical solutions provided by the present application can also be applied to device to device (D2D) communication, vehicle-to-everything (V2X) communication, machine to machine (M2M) communication, machine type communication (MTC), and internet of things (IoT) communication system or other communication systems.

[0109] A device in a communication system can send or receive signals to or from another device. Wherein the signals can include information, signaling or data, etc. Wherein the device can also be replaced by an entity, network entity, network element, communication device, communication module, node, communication node, etc. The disclosure is described by taking the device as an example. For example, the communication system can include at least one terminal device and at least one network device. In the communication system, the network device can send a downlink signal to the terminal device, the terminal device can send an uplink signal to the network device, the network device can send a signal to another network device, and the terminal device can send a sidelink signal to another terminal device. It can be understood that the terminal device in the disclosure can be replaced by a first device, and the network device can be replaced by a second device, both of which perform the corresponding communication method in the disclosure.

[0110] In the embodiments of the present application, the terminal device can also be referred to as a user equipment (UE), an access terminal, a user unit, a user station, a mobile station, a mobile station, a remote station, a remote terminal, a mobile device, a user terminal, a terminal, a wireless communication device, a user agent or a user apparatus.

[0111] The terminal device can be a device providing voice / data, for example, a handheld device with wireless connection function, a vehicle-mounted device, etc. At present, some examples of terminals are: mobile phone, tablet computer, notebook computer, palm computer, mobile internet device (MID), wearable device, virtual reality (VR) device, augmented reality (AR) device, wireless terminal in industrial control, wireless terminal in self driving, wireless terminal in telemedicine, wireless terminal in smart grid, wireless terminal in transportation safety, wireless terminal in smart city, wireless terminal in smart home, cellular phone, cordless phone, session initiation protocol (SIP) phone, wireless local loop (WLL) station, personal digital assistant (PDA), handheld device with wireless communication function, computing device or other processing device connected to a wireless modem, wearable device, terminal device in a 5G network, or terminal device in a future evolved public land mobile network (PLMN), etc. The embodiments of the present application are not limited thereto.

[0112] By way of example and not limitation, in the embodiments of the present application, the terminal device can also be a wearable device. The wearable device can also be referred to as a wearable smart device, which is a general term for devices that are designed and developed by applying wearable technology to daily wear, such as glasses, gloves, watches, clothing, and shoes. The wearable device is a portable device that is directly worn on the body or integrated into the user's clothes or accessories. The wearable device is not only a hardware device, but also a device that realizes powerful functions through software support and data interaction and cloud interaction. The general wearable smart device includes a device with full functions and large size, which can realize complete or partial functions without relying on a smart phone, such as a smart watch or smart glasses, and a device that focuses on a certain application function and needs to cooperate with other devices such as a smart phone, such as various smart wristbands and smart jewelry for monitoring vital signs.

[0113] In the embodiments of the present application, the apparatus for implementing the function of the terminal device can be a terminal device, or an apparatus capable of supporting the terminal device to implement the function, for example, a chip system, which can be installed in the terminal device or used in matching with the terminal device. In the embodiments of the present application, the chip system can be composed of a chip, or can include the chip and other discrete devices. In the embodiments of the present application, only the apparatus for implementing the function of the terminal device is taken as an example for description, and the scheme of the embodiments of the present application is not limited in this way.

[0114] The network device in the embodiments of the present application can include a device for communicating with a terminal device, for example, the network device can include an access network device or a radio access network device, for example, the access network device can be a base station. The radio access network device in the embodiments of the present application can refer to a RAN node (or device) for accessing a terminal device to a wireless network. The base station can broadly cover various names in the following or be replaced by the following names, such as: Node B (NodeB), evolved Node B (eNB), next generation Node B (gNB), relay station, access point, transmitting and receiving point (TRP), transmitting point (TP), primary station, secondary station, motor slide retainer (MSR) node, home base station, network controller, access node, wireless node, access point (AP), transmission node, transceiver node, baseband unit (BBU), remote radio unit (RRU), active antenna unit (AAU), remote radio head (RRH), central unit (CU), distributed unit (DU), radio unit (RU), positioning node, etc. The base station can be a macro base station, a micro base station, a relay node, a donor node or the like, or a combination thereof. The base station can also refer to a communication module, modem or chip used in the aforementioned device or apparatus. The base station can also be a mobile switching center and a device assuming the function of a base station in D2D, V2X, M2M communication, a network side device in a future communication network, a device assuming the function of a base station in a future communication system, etc. The base station can support networks of the same or different access technologies. Optionally, the RAN node can also be a server, a wearable device, a vehicle or a vehicle-mounted device, etc. For example, the access network device in the V2X technology can be a road side unit (RSU). The embodiments of the present application do not limit the specific technology and specific device form of the network device.

[0115] The base station can be fixed or mobile. For example, a helicopter or a drone can be configured to act as a mobile base station, and one or more cells can move according to the location of the mobile base station. In other examples, the helicopter or the drone can be configured to serve as a device communicating with another base station.

[0116] In some deployments, the network device mentioned by embodiments of the present application can be a device including a CU, or a DU, or a device including a CU and a DU, or a control plane CU node (central unit-control plane (CU-CP)) and a user plane CU node (central unit-user plane (CU-UP)) and a DU node. For example, the network device can include a gNB-CU-CP, a gNB-CU-UP and a gNB-DU.

[0117] In some deployments, wireless access is assisted for a terminal by multiple RAN nodes cooperating, and different RAN nodes respectively implement part of the functions of a base station. For example, the RAN node can be a CU, a DU, a CU-CP, a CU-UP, or an RU, etc. The CU and the DU can be separately arranged, or can also be included in the same network element, for example, in a BBU. The RU can be included in a radio frequency device or a radio frequency unit, for example, included in an RRU, an AAU or an RRH.

[0118] The RAN node can support one or more types of front-haul interfaces, and different front-haul interfaces respectively correspond to DUs and RUs having different functions.

[0119] If the front-haul interface between the DU and the RU is a common public radio interface (CPRI), the DU is configured to implement one or more of the baseband functions, and the RU is configured to implement one or more of the radio frequency functions.

[0120] If the front-haul interface between the DU and the RU is another interface, compared with the CPRI, part of the baseband functions of the downlink and / or the uplink, such as one or more of precoding, digital beamforming (BF), or inverse fast fourier transform (IFFT) / adding a cyclic prefix (CP) for the downlink, or one or more of digital beamforming (BF), or fast fourier transform (FFT) / removing a cyclic prefix (CP) for the uplink, are moved from the DU to the RU for implementation.

[0121] One possible implementation, the interface can be an enhanced common public radio interface (eCPRI). Under the eCPRI architecture, the split between the DU and the RU is different, corresponding to different categories (Cat) of eCPRI, such as eCPRI Cat A, B, C, D, E, F.

[0122] Taking eCPRI Cat A as an example, for downlink transmission, the split is layer mapping, the DU is configured to implement layer mapping and one or more functions before layer mapping (i.e., one or more of encoding, rate matching, scrambling, modulation, layer mapping), and other functions after layer mapping (e.g., one or more of resource element (RE) mapping, digital beamforming (BF), or inverse fast Fourier transform (IFFT) / adding cyclic prefix (CP)) are implemented in the RU. For uplink transmission, the split is RE demapping, the DU is configured to implement demapping and one or more functions before demapping (i.e., one or more of decoding, de-rate matching, de-scrambling, de-modulation, inverse discrete Fourier transform (IDFT), channel equalization, RE demapping), and other functions after demapping (e.g., one or more of digital BF or fast Fourier transform (FFT) / CP removal) are implemented in the RU. It can be understood that the description of the functions of the DU and the RU corresponding to various types of eCPRI can refer to the eCPRI protocol, which is not described here.

[0123] In one possible design, the processing unit in the BBU for implementing baseband functions is referred to as a base band high (BBH) unit, and the processing unit in the RRU / AAU / RRH for implementing baseband functions is referred to as a base band low (BBL) unit.

[0124] The CU (or CU-CP and CU-UP), DU or RU can also have different names in different systems, but those skilled in the art can understand their meanings. For example, in an open RAN (ORAN) system, the CU can also be referred to as an O-CU (open CU), the DU can also be referred to as an O-DU, the CU-CP can also be referred to as an O-CU-CP, the CU-UP can also be referred to as an O-CU-UP, and the RU can also be referred to as an O-RU. Any of the CUs (or CU-CPs, CU-UPs), DUs and RUs in this application can be implemented by a software module, a hardware module, or a combination of a software module and a hardware module.

[0125] In the embodiments of the present application, the apparatus for implementing the function of the network device can be a network device; or can be an apparatus capable of supporting the network device to implement the function, such as a chip system, a hardware circuit, a software module, or a hardware circuit plus a software module. The apparatus can be installed in the network device or used in matching with the network device. In the embodiments of the present application, only the apparatus for implementing the function of the network device is taken as an example for illustration, and the scheme of the embodiments of the present application is not limited.

[0126] The network device and / or the terminal device can be deployed on land, including indoors, outdoors, handheld, and / or vehicle-mounted; can also be deployed on the water surface (such as a ship, etc.); and can also be deployed in the air (such as an airplane, a balloon, and / or a satellite). The scenario in which the network device and the terminal device are located is not limited in the embodiments of the present application.

[0127] In addition, the terminal device and the network device can be hardware devices, or can be software functions running on special hardware, software functions running on general hardware, such as virtualized functions instantiated on a platform (for example, a cloud platform), or entities including special or general hardware devices and software functions. The specific form of the terminal device and the network device is not limited in the present application.

[0128] In a wireless communication network, e.g., in a mobile communication network, the services supported by the network are increasingly diverse, and thus the requirements to be met are increasingly diverse. For example, the network needs to be able to support ultra-high rates, ultra-low latency, and / or ultra-large connections. This feature makes network planning, network configuration, and / or resource scheduling increasingly complex. In addition, as the functions of the network become increasingly powerful, e.g., support increasingly high frequency spectrums, support high-order multiple input multiple output (MIMO) technology, support beamforming, and / or support new technologies such as beam management, etc., network energy saving has become a hot research topic. These new requirements, new scenarios, and new features bring unprecedented challenges to network planning, operation and maintenance, and efficient operation. In order to meet this challenge, artificial intelligence technology can be introduced into the wireless communication network, thereby realizing network intelligentization.

[0129] In order to support artificial intelligence (AI) technology in the wireless network, AI nodes (which can also be referred to as AI entities) can also be introduced into the network.

[0130] Optionally, the AI entity can be deployed in one or more of the following positions in the communication system: an access network device, a terminal device, or a core network device, etc., or the AI entity can also be deployed separately, e.g., in a position other than any of the above devices, such as a host or a cloud server of an OTT system. The AI entity can communicate with other devices in the communication system, which can be one or more of the following: a network device, a terminal device, or a network element of a core network, etc. Based on the object served by the AI entity, the AI entity can include a network device-side AI entity, a terminal device-side AI entity, or a core network-side AI entity.

[0131] It can be understood that the present application does not limit the number of AI entities. For example, when there are multiple AI entities, the multiple AI entities can be divided based on functions, e.g., different AI entities are responsible for different functions.

[0132] It can also be understood that the AI entity can be a device independent of each other, can be integrated into the same device to realize different functions, or can be a network element in a hardware device, or can be a software function running on a dedicated hardware, or a virtualized function instantiated on a platform (e.g., a cloud platform), and the present application does not limit the specific form of the AI entity.

[0133] An AI entity can be an AI network element or an AI module. An AI entity is configured to implement a corresponding AI function. AI modules deployed in different network elements can be the same or different. AI models in an AI entity can implement different functions according to different parameter configurations. An AI model in an AI entity can be configured based on one or more of the following parameters: a structural parameter (e.g., at least one of a number of neural network layers, a width of a neural network, a connection relationship between layers, a weight of a neuron, an activation function of a neuron, or a bias in an activation function), an input parameter (e.g., a type of an input parameter and / or a dimension of an input parameter), or an output parameter (e.g., a type of an output parameter and / or a dimension of an output parameter). The bias in an activation function can also be referred to as a bias of a neural network.

[0134] One AI entity can have one or more models. One model can infer an output including one parameter or multiple parameters. Learning processes, training processes, or inference processes of different models can be deployed in different entities or devices, or can be deployed in the same entity or device.

[0135] FIG. 1 is a schematic diagram of a communication system applicable to a communication method according to an embodiment of the present application. As shown in FIG. 1, the communication system 100 can include at least one network device, such as the network device 110 shown in FIG. 1, and can include at least one terminal device, such as the terminal device 120 and the terminal device 130 shown in FIG. 1. The network device 110 and the terminal devices (e.g., the terminal device 120 and the terminal device 130) can communicate with each other through wireless links. The communication devices in the communication system, such as the network device 110 and the terminal device 120, can communicate with each other through multi-antenna technology.

[0136] FIG. 2 is a schematic diagram of another communication system applicable to a communication method according to an embodiment of the present application. Compared with the communication system 100 shown in FIG. 1, the communication system 200 shown in FIG. 2 further includes an AI network element 140. The AI network element 140 is configured to perform AI-related operations, such as constructing a training data set or training an AI model.

[0137] In a possible implementation, the network device 110 can send data related to training of the AI model to the AI network element 140, and the AI network element 140 can construct a training data set and train the AI model. For example, the data related to training of the AI model can include data reported by the terminal device. The AI network element 140 can send a result of an operation related to the AI model to the network device 110, and forward the result to the terminal device through the network device 110. For example, the result of the operation related to the AI model can include at least one of the following: a trained AI model, an evaluation result or a test result of the model, and the like. For example, part of the trained AI model can be deployed on the network device 110, and the other part can be deployed on the terminal device. Alternatively, the trained AI model can be deployed on the network device 110. Alternatively, the trained AI model can be deployed on the terminal device.

[0138] It should be understood that FIG. 2 is only used as an example to illustrate that the AI network element 140 is directly connected to the network device 110, and in other scenarios, the AI network element 140 can also be connected to the terminal device. Alternatively, the AI network element 140 can be connected to both the network device 110 and the terminal device. Alternatively, the AI network element 140 can also be connected to the network device 110 through a third-party network element. The embodiments of the present application do not limit the connection relationship between the AI network element and other network elements.

[0139] The AI network element 140 can also be arranged as a module in the network device and / or the terminal device, for example, in the network device 110 or the terminal device shown in FIG. 1. One or more AI modules can be deployed in the network device 110. One or more AI modules can be deployed in the terminal device.

[0140] It should be noted that FIGS. 1 and 2 are only simplified schematic diagrams for illustration, for example, the communication system can further include other devices, such as a wireless relay device and / or a wireless backhaul device, which are not shown in FIGS. 1 and 2. In actual application, the communication system can include multiple network devices and / or multiple terminal devices. The embodiments of the present application do not limit the number of network devices and terminal devices included in the communication system.

[0141] FIG. 3 is a schematic diagram of a possible application framework of a communication system according to an embodiment of the present application. As shown in FIG. 3, the network elements in the communication system are connected through interfaces (e.g., NG, Xn) or air interfaces. One or more AI modules (only one is shown in FIG. 3 for clarity) are provided in one or more of the network element nodes, such as a core network device, an access network node (RAN node), a terminal device, or an operation administration and maintenance (OAM) device. The access network node can be a single RAN node or can include multiple RAN nodes, such as a CU and a DU. The CU and / or the DU can also be provided with one or more AI modules. Optionally, the CU can be further split into a CU-CP and a CU-UP. One or more AI modules are provided in the CU-CP and / or the CU-UP. For example, the CU and the DU are connected through an F1 interface. The CUs are connected through an Xn interface.

[0142] The network device can be a network device provided with one or more AI modules. The network device can be one or more of the core network device, the access network node (RAN node), or the OAM device shown in FIG. 3. For example, the AI module can be a RIC, such as a near-real-time RIC or a non-real-time RIC, as shown in FIG. 4. For example, the near-real-time RIC is provided in the RAN node (e.g., the CU, the CU-CP, the CU-UP, the DU, and / or the RU), and the non-real-time RIC is provided in the OAM, the cloud server, the core network device, or another network device. The RIC can obtain a subset of data from multiple terminal devices from the RAN node (e.g., the CU, the CU-CP, the CU-UP, the DU, and / or the RU), reorganize the subset of data into a training data set #2, and train based on the training data set #2. For example, the near-real-time RIC and the non-real-time RIC can be separately provided as a network element, and the network device can be the near-real-time RIC or the non-real-time RIC.

[0143] FIG. 4 is a schematic diagram of a possible application framework in a communication system. As shown in FIG. 4, the communication system includes a RAN intelligent controller (RIC). For example, the RIC can be an AI module shown in FIG. 3, which is used to implement AI-related functions. The RIC includes a near-real-time RIC (near-RT RIC) and a non-real-time RIC (Non-RT RIC). The non-real-time RIC mainly processes non-real-time information, such as data that is not sensitive to latency, which can be in the order of seconds. The near-RT RIC mainly processes near-real-time information, such as data that is relatively sensitive to latency, which can be in the order of tens of milliseconds.

[0144] Near real-time RIC is used for model training and inference. For example, it is used for training an AI model, and inference with the AI model. Near real-time RIC can obtain network side and / or terminal side information from RAN nodes (e.g., CU, CU-CP, CU-UP, DU, and / or RU) and / or terminals. The information can be used as training data or inference data. Optionally, near real-time RIC can deliver inference results to RAN nodes and / or terminals. Optionally, inference results can be exchanged between CU and DU, and / or between DU and RU. For example, near real-time RIC delivers inference results to DU, which delivers them to RU.

[0145] Non-real-time RIC is also used for model training and inference. For example, it is used for training an AI model, and inference with the AI model. Non-real-time RIC can obtain network side and / or terminal side information from RAN nodes (e.g., CU, CU-CP, CU-UP, DU, and / or RU) and / or terminals. The information can be used as training data or inference data, and inference results can be delivered to RAN nodes and / or terminals. Optionally, inference results can be exchanged between CU and DU, and / or between DU and RU. For example, non-real-time RIC delivers inference results to DU, which delivers them to RU.

[0146] Near real-time RIC and non-real-time RIC can also be separately configured as a network element. Optionally, near real-time RIC and non-real-time RIC can also be part of other devices. For example, near real-time RIC can be configured in RAN nodes (e.g., CU, DU), while non-real-time RIC can be configured in OAM, cloud server, core network device, or other network devices.

[0147] To facilitate understanding of the schemes of the embodiments of the present application, the following explains the terms that can be involved in the embodiments of the present application.

[0148] (1) Neural network (NN):

[0149] Neural network is a specific implementation form of AI or machine learning (ML). According to the universal approximation theorem, neural network can theoretically approximate any continuous function, so that neural network has the ability to learn any mapping.

[0150] Taking the type of the AI model as a neural network as an example, the AI model involved in the present disclosure can be a deep neural network (DNN). A conventional communication system needs to be designed with the help of rich expert knowledge of a communication module, while a deep learning communication system based on a deep neural network (DNN) can automatically discover an implicit pattern structure from a large amount of data sets, establish a mapping relationship between data, and obtain a performance better than a conventional modeling method.

[0151] A neural network can be composed of neurons, each of which performs a weighted summation operation on its input values, and the weighted summation result generates an output through a nonlinear function. A DNN generally has a multi-layer structure, and each layer of the DNN can include multiple neurons. The input layer transmits the received values to the intermediate hidden layer after processing by the neurons. Similarly, the hidden layer transmits the calculation results to the final output layer to generate the final output of the DNN.

[0152] A DNN generally has more than one hidden layer, and the hidden layer often directly affects the ability to extract information and fit a function. Increasing the number of hidden layers of the DNN or expanding the width of each layer can improve the function fitting ability of the DNN. The weighted value in each neuron is the parameter of the DNN network model. The model parameters are optimized through a training process, so that the DNN network has the ability to extract data features and express mapping relationships. A DNN generally uses a supervised learning or unsupervised learning strategy to optimize the model parameters.

[0153] According to the construction method of the network, the DNN can include a feedforward neural network (FNN), a convolutional neural network (CNN), and a recurrent neural network (RNN), etc.

[0154] A CNN is a neural network specially designed to process data with a similar grid structure. For example, time series data (discrete sampling on the time axis) and image data (two-dimensional discrete sampling) can be considered as data with a similar grid structure. A CNN does not use all input information for operation at one time, but uses a fixed-size window to extract part of the information for convolution operation, which greatly reduces the calculation amount of model parameters. In addition, according to the different types of information extracted by the window (such as people and objects in the same image), each window can use different convolution kernel operations, which enables the CNN to better extract the features of the input data.

[0155] RNN is a kind of DNN network using feedback time series information. Its input includes the new input value at the current time and the output value of itself at the previous time. RNN is suitable for obtaining sequence features with correlation in time, and is particularly suitable for speech recognition, channel coding and decoding and other applications.

[0156] The characteristic of FNN network is that the neurons of adjacent layers are completely connected with each other, which makes FNN usually need a large amount of storage space and leads to high computational complexity.

[0157] The above FNN, CNN and RNN are all constructed based on neurons. As described above, each neuron performs weighted summation operation on its input value, and the weighted summation result generates output through a nonlinear function. The weights of the weighted summation operation of the neurons in the neural network and the nonlinear function are called the parameters of the neural network. The parameters of all neurons of a neural network constitute the parameters of the neural network.

[0158] (2) Centralized training:

[0159] Centralized training can also be referred to as centralized training, which means that the machine learning model and the training data are concentrated in one computing node, and all training steps are completed by the computing node.

[0160] (3) Distributed training:

[0161] Distributed training can also be referred to as distributed learning, which means using multiple computing nodes to perform machine learning or deep learning algorithms and systems.

[0162] In distributed learning, multiple sub-computing nodes and a central computing node jointly participate in training. The user's original data resides locally. The sub-computing nodes cooperate with the central computing node to complete the training by uploading the neural network parameters, which avoids direct sharing of the original data, thereby protecting the privacy and security of the user data.

[0163] (4) Federated learning:

[0164] Federated learning means training algorithms on multiple distributed edge devices or computing nodes with local data samples, and allowing differences in data distribution of each computing node.

[0165] Federated learning is an important framework in distributed learning. In federated learning, one or more sub-computing nodes train a neural network on their local data, and the central computing node is responsible for aggregating the neural network parameters to build a global model.

[0166] The federated learning includes a center computing node and a sub-computing node. The center computing node can also be referred to as a center node. The sub-computing node can also be referred to as a sub-node. The local data set of M sub-nodes is represented as {D1, D2, ···, DM}. M}. M is a positive integer. The local data set of the sub-node k is denoted as , where x i is the input of the model, i.e., the i-th sample, y i is the label corresponding to the input x i , and N k is the data quantity of D k . The center node and the sub-node have their respective neural network models f(·, w c ) and f(·, w k ). Where w c is the parameter of the neural network model of the center node, and the neural network model of the center node is also referred to as a global model, w k is the parameter of the neural network model of the sub-node k. The neural network model of the sub-node can also be referred to as a local model. The local data set is used for training the local model. The center node and the sub-node cooperate to solve the following optimization problem together:

[0167] wherein is the proportion of the data quantity of the local data set of the sub-node k to the total data quantity. L k (w; D k ) represents the local loss value of the loss function of the sub-node k. L(w) represents the loss value. w represents the parameter of the neural network model.

[0168] The training process of the federated learning is as follows: first, in the initial stage of the training, the center node sends the parameter w c of the global model to all the sub-nodes. Then, the model of each sub-node is trained using the local data set D k for multiple rounds, which is referred to as local training. The specific process of the local training includes the following steps.

[0169] (1) In the t-th iteration update, the sub-node k first randomly selects a batch of training data B from the local data set D k , where |B k | is the data quantity of B k , i.e., the number of the batch of training data, |B k | is a positive integer, and t is a positive integer.

[0170] (2) The sub-node k calculates the output y i for the input x and the loss value is calculated according to the loss function l(.,.) wherein, denotes the parameters of the local model at the t-th iteration.

[0171] (3) The child node k calculates L k (w) the gradient of the parameter w: denotes L at the t-th iteration. k (w) the gradient of the parameter w.

[0172] (4) The child node k updates the parameters of the local model using the learning rate λ: set t = t + 1, and return to step (1).

[0173] The above steps (1) to (4) are cycled until all samples in the local data set are used, that is, the number of local model updates reaches , the local training of this round is completed, wherein denotes the floor operation on x. Then, the child node sends the trained model parameters to the center node. The center node aggregates the received model parameters to obtain new model parameters w c . After the aggregation is completed, the center node shares the parameters w c of the aggregated global model with the child nodes, and then repeats the above steps until the model converges.

[0174] The batch normalization (BN) layer normalizes the features of the hidden layer output, which can alleviate the overfitting problem, alleviate the gradient vanishing problem, and has a regularization effect, which is beneficial to improve the training performance of the AI model. For a deep neural network model, since the parameters are constantly updated during training, even if the input data has been standardized, the input received by the relatively later layers is still changing dramatically, which usually leads to numerical instability and makes the model difficult to converge. Batch normalization can make the output of the intermediate layer of the model more stable. When constructing a neural network model, one or more batch normalization layers can be set after the network layer. For example, a batch normalization layer is set after the second layer of the neural network model, and a batch normalization layer is set after the fourth layer. The output of the second layer is the input of the corresponding batch normalization layer of the second layer, and the output of the fourth layer is the input of the corresponding batch normalization layer of the fourth layer.

[0175] The calculation formula of batch normalization is:

[0176] wherein, x l denotes the input of the l-th layer of the neural network model, h l (x l) represents the output of the l-th layer of the neural network model, which is the input of the batch normalization layer corresponding to the l-th layer. B h l (x l The mean of ) h l (x l The variance of the fraction is ε, where ε is a constant that guarantees the denominator is not zero, usually taken as 10. -6 γ l and β l These are the trainable parameters for this batch of normalized layers. μ represents the output of this batch of normalized layers. B represents a batch of training data in the dataset. This dataset is used to train this neural network model. In the above formula, μ B and It can be determined using the following formula.

[0177] Where |B| represents the number of training data sets. This represents the input of the i-th sample in the l-th layer.

[0178] Because child nodes have limited training data, local training is prone to overfitting, leading to overfitting of the global model obtained by aggregating data from the central node, ultimately resulting in poor performance of the global model. Therefore, Batch Normalization (BN) layers are typically added to the local model to alleviate the overfitting problem.

[0179] In distributed training, the local data of child nodes are not independent and identically distributed, meaning that the data distribution of each child node is different. This leads to differences in the mean and variance of the batch normalization layer statistics of the local model, ultimately resulting in the performance of distributed training being worse than that of centralized training.

[0180] In the relevant scheme, after local training is complete, child nodes can upload the mean and variance of the batch normalization layer inputs of their local model, as well as the trained model parameters. The central node aggregates the received mean, variance, and model parameters, and sends the aggregated mean, variance, and new global model parameters to the child nodes. The above steps are repeated until the model converges.

[0181] Although the above scheme alleviates the problem of differences in the mean and variance of the input of the batch normalization layer of the local model of each child node, the aggregated variance obtained by the above method is not the same as the variance in centralized training.

[0182] Specifically, for M child nodes, the mean and variance of the input to the batch normalization layer of the local model can be calculated using the following formula.

[0183] in, input of the i-th sample of the l-th layer of the neural network model, output of the l-th layer of the neural network model on the child node m when the input is input of the l-th layer of the neural network model on the child node m when the input is mean of the input of the batch normalization layer corresponding to the l-th layer of the neural network model on the child node m, variance of the input of the batch normalization layer corresponding to the l-th layer of the neural network model on the child node m. m Bm represents a batch of training data on the child node m, i.e., the m-th batch of data. m | represents the number of the batch of training data.

[0184] The center node can aggregate the mean and variance of the feedback from the M child nodes using the following formula:

[0185] wherein, μ l,B represents the mean of the input of the batch normalization layer corresponding to the l-th layer of the aggregated neural network model, represents the variance of the input of the batch normalization layer corresponding to the l-th layer of the aggregated neural network model.

[0186] In centralized training, the variance of the input of the batch normalization layer corresponding to the l-th layer of the neural network model can be calculated by the following formula.

[0187] wherein, B = B1∪B2∪...∪B M , μ' l,B represents the mean of the input of the batch normalization layer corresponding to the l-th layer of the training data B. is the variance of the input of the batch normalization layer corresponding to the l-th layer of the m-th batch of training data B m output of the l-th layer of the neural network model when the input is input of the batch normalization layer corresponding to the l-th layer.

[0188] can be calculated by the following formula.

[0189] ​Most machine learning algorithms, for example, learning algorithms based on DNN, need to be trained based on massive data. The training of an AI model is usually completed by using a centralized training manner. However, the centralized training manner has some problems. For example, the centralized training needs to collect training data, and the training data usually comes from user equipment, that is, the user equipment needs to upload data. The uploading overhead of the data is usually large, and the collection and use of the data may infringe the privacy and security of the user, for example, the location, motion route and photo of the user are leaked. In the distributed training process, a local model is trained based on local data on a subnode, and the parameters of the local model are uploaded to a center node to complete the training in cooperation with the center node. In this process, the local data on the subnode can stay in the local subnode, avoiding the sharing of the local data, thereby avoiding the overhead brought by the uploading of the training data, and being beneficial to protecting the privacy and security of the local data. The distributed training is expected to use the data and computing power of the user equipment while protecting the privacy and security of the user to achieve the same performance as the centralized training.

[0190] In the distributed training process, the local model of the subnode can use the BN layer to resist overfitting to obtain better performance.

[0191] However, in the distributed training, the local data of each sub computing node is not independent and identically distributed, and there is a difference between the mean and variance counted by the BN layer of each sub computing node, which affects the accuracy of batch normalization and thus the performance of distributed training.

[0192] In the related scheme, the subnode can upload the mean and variance corresponding to the BN layer and the model parameters of the trained local model after the local training is completed, the center node aggregates the received mean, variance and model parameters, and sends the aggregated mean, variance and model parameters of the new global model to each subnode, the subnode locally trains the new global model based on the aggregated mean and variance, and repeats the above process until the model converges. However, the variance aggregated by this method is different from the variance of the centralized training. Specifically, as can be seen from the above formulas (6), (8), (9) and (10), there is a difference between the variances calculated in the two ways. That is, the aggregated variance obtained by aggregating the variances of the subnodes in the distributed training is different from the variance of the centralized training, which affects the accuracy of the batch normalization processing in the distributed training process, and thus affects the performance of the distributed training, making it difficult to achieve the performance of the centralized training.

[0193] Therefore, the present application provides a communication method and a communication device, which are beneficial to improve the accuracy of batch normalization processing in the distributed training process, and thus are beneficial to improve the performance of the distributed training.

[0194] It should be understood that, in the present application, indicating includes direct indication (also known as explicit indication) and implicit indication. Among them, directly indicating information A means including the information A; implicitly indicating information A means indicating information A through the correspondence between information A and information B and directly indicating information B. Among them, the correspondence between information A and information B can be predefined, pre-stored, pre-burned, or pre-configured.

[0195] It should be understood that, in the present application, information C is used for determination of information D, which includes that information D is determined based on information C only, and information D is determined based on information C and other information. In addition, information C used for determination of information D can also be determined indirectly, such as the case where information D is determined based on information E, and information E is determined based on information C.

[0196] In addition, in the embodiments of the present application, "network element A sends information A to network element B" can be understood as that the destination of the information A or the intermediate network element in the transmission path between the destination is network element B, which can include direct or indirect sending of information to network element B. "Network element B receives information A from network element A" can be understood as that the source of the information A or the intermediate network element in the transmission path between the source is network element A, which can include direct or indirect receiving of information from network element A. The information can be processed as necessary between the source and the destination of the information transmission, for example, format change, etc., but the destination can understand the valid information from the source. Similar expressions in the present application can be understood similarly, and will not be described here.

[0197] In the scheme of the embodiments of the present application, "and / or" is used to describe the corresponding relationship of the corresponding objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases of only A, only B and A and B existing at the same time, where A and B can be singular or plural.

[0198] In the scheme of the embodiments of the present application, the center node can cooperate with M sub-nodes to complete the training of the AI model. M is an integer greater than 1. For ease of description, the communication between one of the M sub-nodes (such as sub-node #1) and the center node is mainly taken as an example for description in the embodiments of the present application. The sub-node #1 can be any one of the M sub-nodes. The behaviors of other sub-nodes can be referred to the behavior of the sub-node #1.

[0199] As an example, the center node can be an AI entity on the network device side, and the sub-node can be an AI entity on the terminal device side.

[0200] The network device side includes the network device, or other devices in communication with the network device, such as devices controlled by or serving the network device. The AI entity can be the network device itself, or an AI entity in communication with the network device. For example, the center node can be a RIC, an OAM, or a server, such as an OTT server or a cloud server. The near-real-time RIC is disposed in a RAN node, for example, a CU / DU.

[0201] The terminal device side includes the terminal device, or other devices in communication with the terminal device, such as devices controlled by or serving the terminal device. The AI entity can be the terminal device itself, or an AI entity in communication with the terminal device. For example, the sub-node can be a server, such as an OTT server or a cloud server.

[0202] As another example, the center node can be an AI entity on the terminal device side, and the sub-node can be an AI entity on the terminal device side.

[0203] For example, the center node can be a server, and the plurality of sub-nodes can be a plurality of terminal devices in communication with the server.

[0204] Exemplarily, after the AI model training is completed, the AI model can be deployed on the terminal device side, or on the network device side, or on both the terminal device side and the network device side. Exemplarily, the output of the AI model can include discrete values or continuous values. For example, when the AI model is used to perform a classification task, it can output discrete values. For example, when the AI model is used to perform a regression task, it can output continuous values.

[0205] In addition, in some scenarios, the center node can also train the AI model based on the training data set on the center node. In this case, the center node can also be regarded as a sub-node. Or, the center node also undertakes the operations related to model training on the sub-node. The aforementioned M sub-nodes can be regarded as including the center node.

[0206] The sub-node can be referred to as a sub-computing node. The M sub-nodes can be replaced by M devices.

[0207] The center node can also be referred to as a center computing node.

[0208] The AI model on the center node can be referred to as a global model. The AI model on the sub-node can be referred to as a local model on the sub-node. The AI model on the center node and the AI model on the M sub-nodes have the same structure.

[0209] FIG. 5 is a schematic flowchart of a method of communication provided by the present application. Exemplarily, the method 500 shown in FIG. 5 can be applied to the scenario of distributed training. In the method 500, the center node downlink the aggregated global mean corresponding to the BN layer to the subnodes, and the subnodes calculate the variance of the input of the BN layer based on the global mean corresponding to the BN layer. The center node can aggregate the variance of the input of the BN layer fed back by the subnodes to obtain the global variance corresponding to the BN layer.

[0210] As shown in FIG. 5, the method 500 can include the following steps.

[0211] 510. The center node sends information #1 (an example of the first information) to a plurality of subnodes (e.g., subnode #1 and subnode #2 in FIG. 5). The information #1 includes an index #1 (an example of the first index) and information indicating a global mean #1 (an example of the first mean).

[0212] The global mean #1 can be used for the calculation of a local variance #1 (an example of the first variance). The local variance #1 of the subnode is the variance of the input of a network layer #1 (an example of the first network layer) of an AI model on the subnode.

[0213] In other words, the global mean #1 can be used for the calculation of the variance of the input of the network layer #1 of the AI model on the plurality of subnodes.

[0214] The index #1 is related to the global mean #1.

[0215] According to the index #1, it can be determined that the global mean #1 is used for the calculation of the local variance #1.

[0216] The plurality of subnodes in step 510 are part or all of the aforementioned M subnodes.

[0217] The plurality of subnodes can include a subnode #1 (an example of the first device). For example, the subnode #1 receives the information #1 from the center node, and can calculate the local variance #1 of the subnode #1 according to the global mean #1, i.e., the variance of the input of the network layer #1 of the AI model on the subnode #1.

[0218] The global mean #1 is the global mean corresponding to the network layer #1, or the global mean related to the network layer #1. The global mean #1 can be obtained by aggregating one or more local means #1 of the subnodes. The local mean #1 of the subnode is the mean of the input of the network layer #1 of the AI model on the subnode. The determination method of the global mean #1 can be referred to the description hereinafter.

[0219] The plurality of subnodes can be replaced by a plurality of devices.

[0220] 520, the plurality of child nodes send information #2 (an example of second information) to the center node. The information #2 includes index #2 (an example of second index) and information indicating local bias #1. The index #2 and the index #1 have a correlation relationship.

[0221] The information #2 from the plurality of child nodes respectively includes information indicating the local bias #1 of the plurality of child nodes.

[0222] The plurality of child nodes in step 520 are part or all of the aforementioned M child nodes.

[0223] The plurality of child nodes in step 520 can be the same as or different from the plurality of child nodes in step 510.

[0224] The plurality of child nodes in step 520 are at least partially the same as the plurality of child nodes in step 510. For example, the plurality of child nodes in step 510 can include the plurality of child nodes in step 520.

[0225] The plurality of child nodes can include child node #1. Taking the child node #1 as an example, the child node #1 sends the information #2 to the center node. The local bias #1 indicated by the information #2 from the child node #1 is the local bias #1 of the child node #1.

[0226] The index #2 is related to the local bias #1. According to the correlation relationship between the index #2 and the index #1, it can be determined that the local bias #1 is related to the global mean #1, or in other words, the local bias #1 is determined based on the global mean #1. Further, the local bias #1 can be determined based on the global mean #1 and the input of the network layer #1.

[0227] The distributed training usually includes multiple rounds of training, and there can be multiple BN layers in the AI model. In the embodiments of the present application, the correlation between the local bias and the global mean can be determined through the index, which avoids the child node from calculating the local bias based on the irrelevant global mean, for example, avoids the child node from calculating the local bias of the input of the current BN layer based on the global mean corresponding to other BN layers, or avoids the child node from calculating the local bias of the current round based on the global mean of other rounds, thereby facilitating to ensure the accuracy of the local bias.

[0228] Further, the method 500 can further include step 530.

[0229] 530, the center node sends information #3 (an example of fourth information) to the plurality of child nodes. The information #3 includes index #3 (an example of third index) and information indicating global variance #1 (an example of second variance). The index #1 and the index #3 have a correlation relationship, and / or the index #2 and the index #3 have a correlation relationship. The global mean #1 and the global variance #1 can be used for training of the AI model on the child node.

[0230] According to the index #2, the local variance #1 can be determined for the calculation of the global variance #1.

[0231] The center node can determine the global variance #1 according to the local variance #1 indicated by the information #2 from one or more sub-nodes. The local variance #1 indicated by the information #2 from multiple sub-nodes is the local variance #1 of the multiple sub-nodes.

[0232] Specifically, the center node can aggregate the local variance #1 of one or more sub-nodes to obtain the global variance #1, and distribute the global variance #1 to multiple sub-nodes through the information #3. The global variance can also be referred to as an aggregated variance or an aggregated variance.

[0233] The multiple sub-nodes in step 530 are part or all of the aforementioned M sub-nodes.

[0234] The multiple sub-nodes in step 530 can be the same as or different from the multiple sub-nodes in step 510.

[0235] The multiple sub-nodes in step 530 can be the same as or different from the multiple sub-nodes in step 520.

[0236] The multiple sub-nodes can include a sub-node #1. Taking the sub-node #1 as an example, after receiving the information #3 from the center node, the sub-node #1 can train the local model according to the global mean #1 and the global variance #1.

[0237] The global variance #1 is the global variance corresponding to the network layer #1, or the global variance related to the network layer #1. The determination method of the global variance #1 can be referred to the description hereinafter.

[0238] Wherein, the index #3 is related to the global variance #1. According to the association relationship between the index #3 and the index #1 and the association relationship between the index #1 and the index #2, the global variance #1, the global mean #1 and the local variance #1 can be determined. Alternatively, according to the association relationship between the index #3 and the index #2 and the association relationship between the index #1 and the index #2, the global variance #1, the global mean #1 and the local variance #1 can be determined. Alternatively, according to the association relationship between the index #3, the index #1 and the index #2, the global variance #1, the global mean #1 and the local variance #1 can be determined.

[0239] Further, before step 510, the method 500 can further include step 540.

[0240] 540, the plurality of child nodes send information #4 (an example of the fifth information) to the center node. The information #4 includes an index #4 (an example of the fourth index) and information indicating a local mean #1 (an example of the second mean). The local mean #1 of the child node is the mean of the input of the network layer #1 of the AI model on the child node. The index #4 has a correlation relationship with at least one of the index #1, the index #2, or the index #3.

[0241] The information #4 from the plurality of child nodes respectively includes information indicating the local mean #1 of the plurality of child nodes.

[0242] The plurality of child nodes in step 540 are part or all of the aforementioned M child nodes.

[0243] The plurality of child nodes in step 540 can be the same as or different from the plurality of child nodes in step 510.

[0244] The plurality of child nodes in step 540 can be the same as or different from the plurality of child nodes in step 520.

[0245] The plurality of child nodes in step 540 can be the same as or different from the plurality of child nodes in step 530.

[0246] The plurality of child nodes can include a child node #1. Taking the child node #1 as an example, the child node #1 sends the information #4 to the center node. The local mean #1 indicated by the information #4 sent by the child node #1 is the local mean #1 of the child node #1, that is, the mean of the input of the network layer #1 of the AI model on the child node #1.

[0247] The center node can determine the global mean according to the local mean #1 indicated by the information #4 from one or more child nodes. The local mean #1 indicated by the information #4 from the plurality of child nodes is the local mean #1 of the plurality of child nodes.

[0248] Specifically, the center node can aggregate the local mean #1 of one or more child nodes to obtain the global mean #1. The global mean #1 is issued to the plurality of child nodes through the information #1 in step 510. The global mean can also be referred to as an aggregated mean or an aggregated mean.

[0249] Wherein, the index #4 is related to the local mean #1. The index #4 has a correlation relationship with at least one of the index #1, the index #2, or the index #3, according to which it can be determined that the local mean #1 is related to at least one of the global mean #1, the local variance #1, or the global variance #1.

[0250] It should be understood that the numbering of the steps in the embodiments of the present application is only used for convenience of description, and does not limit the execution order of the steps in the embodiments of the present application. For example, step 540 can be performed before step 510.

[0251] The AI model can include one or more BN layers. The network layer #1 can be any of the one or more BN layers. For ease of description, only one BN layer is taken as an example for description in the embodiments of the present application. In the case where the AI model includes multiple BN layers, the other BN layers can all refer to the processing mode related to the network layer #1.

[0252] In the AI model, the input of one network layer can be the output of another network layer, except for the input layer. Accordingly, the input of the network layer #1 can also be understood as the output of the previous layer of the network layer #1. Or in other words, the input of the network layer #1 can also be replaced by the output of the network layer #2, and the network layer #2 can be the previous layer of the BN layer.

[0253] The input of the network layer #1 can include one or more features, or one or more feature channels. The input of the network layer #1 can also be replaced by the feature of the network layer #1, or the feature channel of the network layer #1. For example, the mean of the input of the network layer #1 is the mean of the feature of the network layer #1. The variance of the input of the network layer #1 is the variance of the feature of the network layer #1.

[0254] How to determine the local mean #1 is described below.

[0255] The mean of the input of the network layer #1 of the AI model on the child node is determined based on a batch of training data on the child node.

[0256] Taking the child node m as an example, a batch of training data B m on the child node m has a data amount of |B m |. |B m | is a positive integer. Inputting the batch of training data B m into the AI model on the child node m, accordingly, |B m | inputs of the network layer #1 can be obtained, and the average of the |B m | inputs can be taken as the mean of the input of the network layer #1 of the AI model on the child node m. For example, the child node m can calculate the local mean #1 based on the formula (5) in the foregoing, and the BN layer corresponding to the lth layer can be regarded as the network layer #1.

[0257] How to determine the global mean #1 is described below.

[0258] The center node can aggregate the local mean #1 of one or more child nodes to obtain the global mean #1.

[0259] The one or more child nodes can be part or all of the M child nodes, i.e. the center node can determine the global mean #1 based on the local means #1 of part or all of the child nodes. The one or more child nodes belong to the plurality of child nodes in step 510.

[0260] As one possible implementation, the center node determines the global mean #1 according to the local means #1 of the M child nodes.

[0261] The center node can receive information #4 from the M child nodes, and obtain the local means #1 of the M child nodes. The center node aggregates the local means #1 of the M child nodes to obtain the global mean #1.

[0262] As another possible implementation, the center node determines the global mean #1 according to the local means #1 of part of the M child nodes.

[0263] As one example, the center node only receives information #4 from part of the M child nodes. The center node can aggregate the local means #1 of the received child nodes to obtain the global mean #1.

[0264] For example, child node #1 can not send information #4. For another example, child node #1 sends information #4 to the center node, but the center node does not receive the local mean #1 reported by child node #1. In this case, the center node can not use the local mean #1 of child node #1 when calculating the global mean #1.

[0265] As another example, the center node can decide by itself to determine the global mean #1 according to the local means #1 of part of the M child nodes.

[0266] For example, the center node receives the local mean #1 reported by child node #1, but the center node can decide by itself not to use the local mean #1 of child node #1 to calculate the global mean #1.

[0267] The center node can aggregate the local means of the child nodes in various aggregation manners.

[0268] Illustratively, the center node aggregating the local means #1 of one or more child nodes can include: the center node calculating the average of the local means #1 of the plurality of child nodes as the global mean #1.

[0269] For example, taking all of the plurality of child nodes as the M child nodes, the global mean #1 can satisfy the following formula:

[0270] wherein μ l,Bdenotes the global mean corresponding to the BN layer of the l-th layer of the AI model, i.e., global mean #1. denotes the mean of the input of the BN layer corresponding to the l-th layer of the AI model, i.e., local mean #1.

[0271] Alternatively, the center node aggregates the local mean #1 of the one or more sub-nodes, which can include: the center node calculates a weighted average of the local mean #1 of the plurality of sub-nodes as the global mean #1. For example, in the calculation of the weighted average, the weight corresponding to the local mean #1 of each sub-node can be positively correlated with the data amount of the batch training data of each sub-node. Taking the one or more sub-nodes as all M sub-nodes as an example, the center node can calculate the global mean #1 based on the formula (7) in the foregoing.

[0272] The following describes how to determine the local variance #1.

[0273] The global mean #1 can be used to calculate the local variance #1 of the plurality of sub-nodes.

[0274] Taking the sub-node #1 as an example, after receiving the global mean #1, the sub-node #1 does not calculate the local variance #1 based on the local mean #1, but calculates the local variance #1 based on the global mean #1.

[0275] For example, the local variance #1 can satisfy the following formula:

[0276] wherein, denotes the variance of the input of the BN layer (i.e., network layer #1) of the l-th layer of the AI model of the sub-node m, i.e., the local variance #1 of the sub-node m, μ l,B denotes the global mean corresponding to the BN layer, i.e., global mean #1. denotes the input of the i-th sample of the l-th layer, denotes the output of the l-th layer of the AI model on the sub-node m in the case that the input is denotes the input of the BN layer corresponding to the l-th layer.

[0277] The following describes how to determine the global variance #1.

[0278] The center node can aggregate the local variance #1 of one or more sub-nodes to obtain the global variance #1.

[0279] The one or more child nodes can be part or all of the M child nodes, i.e., the center node can determine the global variance #1 based on the local variance #1 of part or all of the child nodes. The one or more child nodes belong to the plurality of child nodes in step 520. The specific description of the one or more child nodes can refer to the related description of the one or more child nodes in the process of calculating the global mean #1 in the foregoing. It should be understood that the one or more child nodes to which the local variance #1 for calculating the global variance #1 belongs and the one or more child nodes to which the local mean #1 for calculating the global mean #1 belongs can be the same or different.

[0280] Exemplarily, the center node aggregating the local variance #1 of the one or more child nodes can include: the center node calculating the average of the local variance #1 of the plurality of child nodes as the global variance #1.

[0281] Alternatively, the center node aggregating the local variance #1 of the one or more child nodes can include: the center node calculating the weighted average of the local variance #1 of the plurality of child nodes as the global variance #1. For example, in the calculation process of the weighted average, the weight corresponding to the local variance #1 of each child node can be in a positive correlation with the data amount of the batch training data of each child node. Taking all of the plurality of child nodes as M child nodes as an example, the center node can calculate the global variance #1 based on the formula (8) in the foregoing.

[0282] In the scheme of the embodiments of the present application, the child node calculates the local variance #1 related to the global mean #1 based on the global mean #1 from the center node, which is beneficial to avoid the influence of the non-independent and identically distributed local data of each child node, to obtain a more accurate local variance #1, so as to enable the center node to aggregate a more accurate global variance #1 based on the more accurate local variance #1, thereby being beneficial to improve the accuracy of the batch normalization processing in the distributed training process and improve the performance of the distributed training.

[0283] The correlation between the index #1 and the global mean #1 will be described below.

[0284] The distributed training usually needs multiple rounds to complete the training of the model. One round can be understood as starting from the center node sending the model parameters of the global model to each sub-node to the center node aggregating to generate a new global model. Different rounds can be distinguished by different round identifiers. Exemplarily, the identifier of the round can be the number of the current round. For example, the number of the current round can represent the number of rounds that have been experienced so far, i.e., the current round number. The number of the current round can be equal to the current round number. Alternatively, the number of the round and the round number can be in a positive correlation, for example, the larger the round number, the larger the number of the round. The number of the current round can also be irrelevant to the current round number, as long as different rounds can be distinguished. For ease of description, the number of the current round can represent the current round number in the embodiments of the present application.

[0285] Exemplarily, for the center node, each time the model parameters of the global model are sent to the sub-node, or each time the global model is updated, the center node enters a new round, and the center node can update the identifier of the round of the center node, for example, the number of the round at the center node is incremented by one, or the number of the round of the training related to the global model is incremented by one. For the sub-node, each time the model parameters of the global model are received, or each time the local model is updated according to the received model parameters of the global model, the sub-node enters a new round, and the sub-node can update the identifier of the local round, for example, the number of the round at the sub-node is incremented by one, or the number of the round of the training related to the local model is incremented by one. Alternatively, each time the center node sends the model parameters of the global model to the sub-node, the center node can also send the updated identifier of the round to the sub-node, and the sub-node can update the identifier of the local round based on the received identifier of the round.

[0286] The identifier of the round can also be the number of times the global model is updated, or the number of times the center node and the sub-node interact about the model parameters of the global model.

[0287] The index #1 and the global mean #1 are related, which can be understood as the index #1 and the global mean #1 are both related to the same round and / or the same network layer.

[0288] The following is described by taking two rounds as an example.

[0289] For example, in the first round, the center node calculates the global mean a according to the local means a of the plurality of sub-nodes, the center node sends the index a and the global mean a to the sub-nodes, the index a is related to the first round, the sub-nodes calculate the local dispersion a based on the global mean a and report to the center node, the center node calculates the global dispersion a based on the local dispersions a of the plurality of sub-nodes and distributes to the sub-nodes, and the sub-nodes train the local model based on the global dispersion a and the global mean a. After the local training is completed, the model parameters of the trained local model are reported to the center node, the center node aggregates the model parameters of the local model reported by the plurality of sub-nodes to generate a new global model. The center node distributes the parameters of the new global model to the sub-nodes, that is, enters the second round. In the second round, the center node calculates the global mean b according to the local means b of the plurality of sub-nodes, the center node sends the index b and the global mean b to the sub-nodes, the index b is related to the second round, the sub-nodes calculate the local dispersion b based on the global mean b and report to the center node, the center node calculates the global dispersion b based on the local dispersions b of the plurality of sub-nodes and distributes to the sub-nodes, and the sub-nodes train the local model based on the global dispersion b and the global mean b. In the above process, the index a is the index related to the global mean a, and the index b is the index related to the global mean b. The index a can be regarded as an index irrelevant to the global mean b. The index b can be regarded as an index irrelevant to the global mean a.

[0290] The relevance of the index #1 and the global mean #1 can be embodied in various ways.

[0291] For example, the index #1 and the global mean #1 can be carried in the same signaling. That is, the first information can be carried in the same signaling.

[0292] In this case, the index carried in the same signaling with the global mean #1 is the index #1 related to the global mean #1.

[0293] Further, if a plurality of global means corresponding to a plurality of BN layers and a plurality of indexes corresponding to the plurality of global means are carried in the same signaling, the indexes related to the plurality of global means can be indexes having a preset positional relationship with the positions of the plurality of global means.

[0294] For example, the index #1 can be an index located behind the global mean #1 and adjacent to the global mean #1.

[0295] Alternatively, the index #1 and the global mean #1 can be carried in two signalings respectively. That is, the first information can be carried in two signalings. The two signalings are related. The sub-nodes calculate the local dispersion #1 according to the global mean #1 after receiving the two signalings.

[0296] The information indicated by index #1 is explained below based on two examples (Example #1 and Example #2).

[0297] Example #1:

[0298] As an example, index #1 can indicate the round of training related to global mean #1.

[0299] The round of training related to global mean #1 is the round at the center node.

[0300] Optionally, index #1 can indicate the identification of the round of training related to global mean #1.

[0301] For example, index #1 can include the number of the round of training related to global mean #1.

[0302] After receiving information #1, sub-node #1 can determine which round global mean #1 is calculated in according to index #1, and then can calculate local deviation #1 based on the global mean #1 in the round.

[0303] Optionally, index #1 can indicate the identification of the AI model corresponding to global mean #1.

[0304] Index #1 can indicate the round of training related to global mean #1 by indicating the identification of the AI model.

[0305] For example, the identification of the AI model can be the version number of the AI model.

[0306] In the distributed training process, a new identification different from the AI model in the previous round can be assigned to the current AI model, such as setting a new version number, every time a new round is entered. For example, the version number of the AI model in the first round can be V1.1, the version number of the AI model in the second round can be V1.2, and so on. Alternatively, the center node can assign a new identification different from the previous global model, such as setting a new version number, to the new global model every time a new global model is aggregated and generated. For example, the version number of the initial global model can be V1.0, the version number of the global model after the first update can be V1.1, and so on.

[0307] After the new global model is aggregated and generated, the center node can distribute the model parameters of the new global model and the identification of the new global model to each sub-node. The sub-node updates the local model based on the model parameters of the new global model, and accordingly, the identification of the new global model serves as the identification of the updated local model.

[0308] The child node #1 can determine which version of the AI model the global mean #1 corresponds to according to the identifier of the AI model corresponding to the global mean #1, i.e., determine which round the global mean #1 is calculated in.

[0309] Example #2:

[0310] As an example, the index #1 can indicate the network layer #1. That is, the index #1 can indicate the network layer corresponding to the global mean #1.

[0311] Exemplarily, the index #1 can include an identifier of the network layer #1. For example, the identifier of the network layer can be a layer number of the network layer or an index of the network layer. The layer number can be an integer.

[0312] The child node #1 can determine which network layer the global mean #1 is associated with according to the index #1, thereby determining that the global mean #1 is used to calculate the variance of the input of the network layer in the AI model in the child node #1.

[0313] It should be understood that the above is only an example, and the network layer #1 can also be indicated in other ways. Exemplarily, the position of the information indicating the global mean #1 in the information #1 can be used to indicate the network layer #1. For example, the AI model can include a plurality of BN layers, and the information #1 can include information for indicating the global means corresponding to the plurality of BN layers. In the information #1, the order of the fields indicating the global means corresponding to the plurality of BN layers is consistent with the order of the plurality of BN layers in the AI model. For example, the field indicating the global mean corresponding to the first BN layer to the field indicating the global mean corresponding to the last BN layer are arranged in the information #1 in the order from front to back or from back to front.

[0314] FIG. 6 shows an example of information indicating global means in the information #1.

[0315] For example, the AI model includes 3 BN layers, which are arranged in the order from front to back as BN layer L1, BN layer L2 and BN layer L3 in the AI model, and the corresponding global means are global mean a, global mean a' and global mean a", respectively. The information #1 can include information indicating the global means corresponding to the 3 BN layers. As shown in FIG. 6, the global mean a, the global mean a' and the global mean a" are arranged in the information #1 in the order of the corresponding BN layers in the AI model. The leftmost global mean is the global mean a corresponding to the BN layer L1, and the rightmost global mean is the global mean a" corresponding to the BN layer L3.

[0316] Examples 1 and 2 can also be used in combination. That is, the index #1 can indicate the network layer #1 and the round of training related to the global mean #1.

[0317] Optionally, the index #1 can indicate the identity of the network layer #1 and the identity of the round of training related to the global mean #1.

[0318] Exemplarily, the index #1 can include the identity of the network layer #1 and the identity of the round of training related to the global mean #1. Optionally, the index #1 can include two fields (also referred to as information elements), respectively carrying the identity of the network layer #1 and the identity of the round of training related to the global mean #1.

[0319] Exemplarily, the index #1 can include the joint encoding result of the identity of the network layer #1 and the identity of the round of training related to the global mean #1. Optionally, the index #1 can include one field (also referred to as information element), which carries the joint encoding result of the identity of the network layer #1 and the identity of the round of training related to the global mean #1.

[0320] For example, the index value of the index #1 is 01, which can represent that the network layer #1 is the first BN layer and the number of the round of training related to the global mean #1 is 1; the index value of the index #1 is 02, which can represent that the network layer #1 is the second BN layer and the number of the round of training related to the global mean #1 is 1; the index value of the index #1 is 03, which can represent that the network layer #1 is the first BN layer and the number of the round of training related to the global mean #1 is 2.

[0321] Optionally, the index #1 can indicate the identity of the network layer #1 and the identity of the AI model corresponding to the global mean #1.

[0322] The correlation between the index #2 and the local difference #1 is described below.

[0323] The index #2 and the local difference #1 are related, which can be understood as that the index #2 and the local difference #1 are both related to the same round and / or the same network layer.

[0324] The following is described by taking two rounds as an example.

[0325] For example, in the first round, the center node calculates the global mean a according to the local mean a of the plurality of sub-nodes, the center node sends the global mean a to the sub-nodes, the sub-nodes calculate the local variance a based on the global mean a, and report the index a and the local variance a to the center node. The index a is related to the first round. The center node calculates the global variance a based on the local variance a of the plurality of sub-nodes, and sends the global variance a to the sub-nodes, the sub-nodes train the local model based on the global variance a and the global mean a. After the local training is completed, the model parameters of the trained local model are reported to the center node, and the center node aggregates the model parameters of the local model reported by the plurality of sub-nodes to generate a new global model. The center node sends the parameters of the new global model to the sub-nodes, that is, enters the second round. In the second round, the center node calculates the global mean b according to the local mean b of the plurality of sub-nodes, the center node sends the global mean b to the sub-nodes, the sub-nodes calculate the local variance b based on the global mean b, and report the index b and the local variance b to the center node, the index b is related to the second round, the center node calculates the global variance b based on the local variance b of the plurality of sub-nodes, and sends the global variance b to the sub-nodes, the sub-nodes train the local model based on the global variance b and the global mean b. In the above process, the index a is an index related to the local variance a, and the index b is an index related to the local variance b. The index a can be regarded as an index irrelevant to the local variance b. The index b can be regarded as an index irrelevant to the local variance a.

[0326] The correlation between the index #2 and the local variance #1 can be embodied in various ways.

[0327] For example, the index #2 and the local variance #1 can be carried in the same signaling. That is, the second information can be carried in the same signaling.

[0328] Alternatively, the index #2 and the local variance #1 can be carried in two signalings respectively. That is, the second information can be carried in two signalings. The two signalings are related.

[0329] The specific description can refer to the correlation between the index #1 and the global mean #1 in the foregoing, which will not be repeated here.

[0330] The information indicated by the index #2 will be described below based on two examples (example #3 and example #4).

[0331] Example #3:

[0332] As an example, the index #2 can indicate the round of training related to the local variance #1.

[0333] The round of training related to the local variance #1 is the round of the sub-nodes.

[0334] Optionally, the index #2 can indicate an identification of a round of training related to the local variance.

[0335] For example, the index #2 can include a number of a round of training related to the local variance #1.

[0336] After receiving the information #2, the center node can determine, according to the index #2, in which round the local variance #1 is calculated, and then can calculate the global variance #1 based on the local variance #1 in the round.

[0337] Optionally, the index #2 can indicate an identification of an AI model corresponding to the local variance #1.

[0338] The index #2 can indicate a round of training related to the local variance #1 by indicating an identification of an AI model corresponding to the local variance #1.

[0339] The AI model corresponding to the local variance #1, i.e., the local variance #1 is calculated based on the AI model.

[0340] For example, the identification of the AI model can be a version number of the AI model.

[0341] The center node determines, according to the version number of the AI model corresponding to the local variance #1, which version of the AI model the local variance #1 corresponds to, i.e., determines in which round the local variance #1 is calculated.

[0342] Example #4:

[0343] As an example, the index #2 can indicate a network layer #1. That is, the index #2 can indicate a network layer corresponding to the local variance #1.

[0344] Exemplarily, the index #2 can include an identification of the network layer #1.

[0345] The center node can determine, according to the index #2, which network layer the local variance #1 is associated with, and thus determine that the local variance #1 is used to calculate a global variance corresponding to the network layer.

[0346] It should be understood that the above is only an example, and the network layer #1 can also be indicated in other ways. Exemplarily, a position of information indicating the local variance #1 in the information #2 can be used to indicate the network layer #1.

[0347] Example #3 and example #4 can also be used in combination. That is, the index #2 can indicate the network layer #1 and a round of training related to the local variance #1.

[0348] Optionally, the index #2 can indicate an identification of the network layer #1 and an identification of a round of training related to the local variance #1.

[0349] For example, the index #2 can include the identification of the network layer #1 and the identification of the trained round related to the local variance #1. Alternatively, the index #2 can include two fields (also referred to as information elements) carrying the identification of the network layer #1 and the identification of the trained round related to the local variance #1, respectively.

[0350] For another example, the index #2 can include a joint encoding result of the identification of the network layer #1 and the identification of the trained round related to the local variance #1. Alternatively, the index #2 can include one field (also referred to as information element) carrying the joint encoding result of the identification of the network layer #1 and the identification of the trained round related to the local variance #1.

[0351] Alternatively, the index #2 can indicate the identification of the network layer #1 and the identification of the AI model corresponding to the local variance #1.

[0352] The specific description of the index #2 can refer to the index #1 described above, and will not be described here again.

[0353] The correlation between the index #3 and the global variance #1 can refer to the correlation between the index #1 and the global mean #1 described above, and the correlation between the index #2 and the local variance #1 described above. The description of the index #3 can refer to the description of the index #1 and the index #2, and will not be described here again.

[0354] The correlation between the index #4 and the local mean #1 can refer to the correlation between the index #1 and the global mean #1 described above, and the correlation between the index #2 and the local variance #1 described above. The description of the index #4 can refer to the description of the index #1 and the index #2, and will not be described here again.

[0355] The association between the index #1 and the index #2 will be described below.

[0356] Alternatively, the index values of the index #1 and the index #2 are the same.

[0357] If the index values of the two indexes are the same, the two indexes have an association. In this case, the index #1 and the index #2 have an association, that is, the index values of the index #1 and the index #2 are the same.

[0358] Alternatively, the index #1 and the index #2 have a predefined association.

[0359] That is, the association between the index #1 and the index #2 can be predefined.

[0360] Alternatively, the index #1 and the index #2 have a preconfigured association.

[0361] That is, the method 500 can further include that the center node sends information #5 to the plurality of child nodes, the information #5 indicating the association between the two indexes.

[0362] Exemplarily, the two indexes have the correlation relationship if index values of the two indexes satisfy a preset condition #1. The preset condition #1 can be predefined, or can be indicated by the information #5.

[0363] For example, the two indexes have the correlation relationship if a difference between the index values of the two indexes is 1. That is, if the difference between the index values of the two indexes is 1, the two indexes have the correlation relationship. In this case, the index #1 and the index #2 have the correlation relationship, that is, the difference between the index values of the index #1 and the index #2 is 1. For example, after receiving the information a and the index a related to the information a, the child node can add 1 to the index value of the index a to obtain the index value of the index b when sending the information b related to the information a, and sends the information b and the index b. After receiving the information b and the index b, the center node can determine that the index a is related to the index b according to the difference between the index b and the index a being 1, and thus determine that the information b is related to the information a.

[0364] FIG. 7 shows a format diagram of a correlation relationship according to an embodiment of the present application. Exemplarily, the format can be issued by the center node to the child node. Alternatively, the format can also be predefined.

[0365] The child node can determine whether the mean and the variance are related according to the format. For example, as shown in FIG. 7, the child node can fill in the number of the training round related to the current local model at the position of “round index number t of the model”. After receiving the global mean #1 and the index #1, the child node can fill in the number of the training round indicated by the index #1 and the layer number of the network layer #1 corresponding to the global mean #1 at the positions of “round index number t' of the mean” and “layer sequence number l of the mean”, respectively. By comparing “round index number t of the model” and “round index number t' of the mean”, it can be determined whether the number of the training round related to the global mean #1 is related to the number of the round of the current local model, for example, whether they are the same. Similarly, after receiving the global variance #1 and the index #3, the child node can fill in the number of the training round indicated by the index #3 and the layer number of the network layer #1 corresponding to the global variance #1 at the positions of “round index number t” of the variance” and “layer sequence number l' of the variance”, respectively. By comparing “round index number t” of the variance” and “round index number t' of the mean” and “layer sequence number l' of the variance” and “layer sequence number l of the mean”, it can be determined whether the received global variance #1 and the global mean #1 are related, for example, if they are completely the same, it means that they are related, otherwise, they are not related.

[0366] As described above, the index #3 has a correlation relationship with at least one of the index #1 or the index #2.

[0367] As described above, the index #4 has a correlation with at least one of the index #1, the index #2, or the index #3.

[0368] The description of the correlation between two indexes can refer to the index #1 and the index #2, which will not be repeated here.

[0369] Further, the method 500 can further include that the child node #1 receives the model parameters of the AI model from the center node.

[0370] After receiving the model parameters of the AI model from the center node, each child node can update the local model according to the model parameters of the AI model, and the model parameters of the local model after the update are the model parameters of the AI model issued by the center node, or in other words, the local model after the update is the AI model issued by the center node.

[0371] Exemplarily, the model parameters of the AI model can be the model parameters of the AI model used to calculate the local mean in step 540.

[0372] For example, each child node can calculate the mean of the input of the network layer #1 of the AI model issued by the center node based on the local training data, and report to the center node. In other words, after entering a new round, each child node can calculate the mean of the input of the network layer #1 based on the latest global model issued by the center node before starting local training, instead of recalculating the mean of the input of the network layer #1 when the parameters of the local model change after the start of local training.

[0373] In the scheme of the embodiments of the present application, each child node can calculate the mean of the input of the BN layer based on the received model parameters of the AI model before the start of the current training, which is beneficial to ensure that each child node uses a unified AI model to calculate the mean of the input of the BN layer, or in other words, is beneficial to ensure that the local model of each child node adopts a unified AI model when each child node calculates the mean of the input of the BN layer of the local model, thereby being beneficial to improve the accuracy of the global mean and the global variance.

[0374] Exemplarily, the model parameters of the AI model can be the model parameters of the AI model used to calculate the local difference in step 510, or in other words, the model parameters of the AI model can be the model parameters of the AI model corresponding to the global mean #1.

[0375] For example, each sub-node can calculate the variance of the input of the network layer #1 of the AI model issued by the center node based on the local training data and the global mean #1, and report to the center node. In other words, after entering a new round, each sub-node can calculate the variance of the input of the network layer #1 based on the local training data and the latest global model issued by the center node before starting local training, instead of recalculating the variance of the input of the network layer #1 when the parameters of the local model change after the local training starts.

[0376] In the scheme of the embodiments of the present application, each sub-node can calculate the variance of the input of the BN layer based on the received model parameters of the AI model before starting the current round of training, which is beneficial to ensure that each sub-node uses a unified AI model to calculate the variance of the input of the BN layer, or in other words, is beneficial to ensure that each sub-node uses a unified AI model when calculating the variance of the input of the BN layer of the local model, thereby improving the accuracy of the global variance.

[0377] Further, the center node and / or the sub-node can determine whether the AI model on the sub-node is consistent with the global model on the center node. The following takes scheme #1 and scheme #2 as examples for illustration.

[0378] Scheme #1:

[0379] In scheme #1, the sub-node can determine whether the local model is consistent with the global model on the center node.

[0380] In one possible implementation, the sub-node #1 can receive information from the center node, which can be used to determine whether the sub-node #1 needs to obtain the model parameters of the AI model from the center node.

[0381] The AI model can be the latest global model, and the information can be used to determine whether the AI model on the sub-node #1 is the latest global model.

[0382] As an example, the index #1 can be used to determine whether the AI model on the sub-node #1 is the AI model corresponding to the global mean #1, or whether it is the latest global model.

[0383] The AI model corresponding to the global mean #1 is usually the latest global model.

[0384] By way of example, the index #1 can be used to indicate the round of training related to the global mean #1.

[0385] The sub-node #1 can determine whether the local model is the latest global model according to whether the round indicated by the index #1 is the same as the round of the sub-node #1.

[0386] For example, the index #1 can include the number of the round of training related to the global mean #1. If the number of the round indicated by the index #1 is greater than the number of the round of the child node #1, it indicates that the child node #1 does not receive the new global model issued by the center node. Thus, the child node #1 can determine whether the local model is the latest global model according to the number of the round indicated by the index #1 and the number of the round of the child node #1.

[0387] For example, the index #1 can include the number of the round of training related to the global mean #1. If the number of the round indicated by the index #1 is greater than the number of the round of the child node #1, it indicates that the child node #1 does not receive the new global model issued by the center node. Thus, the child node #1 can determine whether the local model is the latest global model according to the number of the round indicated by the index #1 and the number of the round of the child node #1.

[0388] For example, the index #1 can include the number of the round of training related to the global mean #1. If the number of the round indicated by the index #1 is greater than the number of the round of the child node #1, it indicates that the child node #1 does not receive the new global model issued by the center node. Thus, the child node #1 can determine whether the local model is the latest global model according to the number of the round indicated by the index #1 and the number of the round of the child node #1.

[0389] For example, the index #1 can include the number of the round of training related to the global mean #1. If the number of the round indicated by the index #1 is greater than the number of the round of the child node #1, it indicates that the child node #1 does not receive the new global model issued by the center node. Thus, the child node #1 can determine whether the local model is the latest global model according to the number of the round indicated by the index #1 and the number of the round of the child node #1.

[0390] The above is only an example. The center node can also send other information to enable the child node #1 to determine whether the local model is the latest global model.

[0391] Further, optionally, the method 500 can further include that the child node #1 sends information #6 (an example of the third information) to the center node. The information #6 is used to request the model parameters of the AI model.

[0392] For example, in the case where the child node #1 determines that the local model is not the AI model corresponding to the global mean #1, the child node #1 can request the model parameters of the AI model from the center node, and then update the local model based on the model parameters of the AI model, so that the child node #1 can calculate the local deviation #1 based on the AI model corresponding to the global mean #1. Alternatively, in the case where the child node #1 determines that the local model is not the latest global model, the child node #1 can request the model parameters of the latest global model from the center node, and then update the local model based on the model parameters of the latest global model, so that the child node #1 can calculate the local deviation #1 based on the latest global model.

[0393] The following is an example in which the index #1 includes the number of the round of training related to the global mean #1.

[0394] Optionally, in the case where the number of the round indicated by the index #1 is greater than the number of the round of the child node #1, the child node #1 sends the information #6 to the center node.

[0395] After sending the information #6, the child node #1 receives the model parameters of the AI model sent by the center node. The child node #1 updates the local model based on the model parameters, that is, updates the model parameters of the local model to the model parameters of the AI model sent by the center node, and then calculates the local difference #1 based on the updated local model.

[0396] For example, the child node #1 receives the information #1 from the center node. The index #1 in the information #1 indicates that the round number is 2, that is, the global mean #1 indicated by the information #1 is the global mean #1 in the second round. The round number of the child node #1 is 1, that is, the local model on the current child node #1 is based on the model parameters of the global model received in the first round. The child node #1 can determine that the global mean #1 is used to calculate the local difference #1 in the second round, that is, the local difference #1 is calculated based on the global model in the second round, and the current local model is not the global model in the second round. The child node #1 can send the information #6 to the center node to request the model parameters of the global model in the second round. After receiving the information #6, the center node can send the model parameters of the global model in the second round to the child node #1. The child node #1 updates the local model based on the model parameters of the global model in the second round, and updates the round number of the training related to the local model to 2. The child node #1 calculates the local difference #1 according to the global mean #1 based on the updated local model.

[0397] In the case where the round number indicated by the index #1 is the same as the round number of the child node #1, the child node #1 can calculate the local difference #1 based on the current local model.

[0398] In this way, it is beneficial to ensure that each child node calculates the local difference based on a unified AI model, thereby further improving the accuracy of the global variance of the BN layer.

[0399] Scheme #2:

[0400] In scheme #2, the center node can determine whether the local model is consistent with the global model on the center node.

[0401] In one possible implementation, the child node #1 can send information to the center node, which can be used to determine whether the model parameters of the AI model need to be sent to the child node #1.

[0402] The AI model can be the latest global model, and the information can be used to determine whether the AI model on the child node #1 is the latest global model.

[0403] As an example, the index #4 can be used to determine whether the AI model on the child node #1 is the AI model corresponding to the global mean #1 or whether it is the latest global model.

[0404] By way of example, the index #4 can be used to indicate the round of training related to the local mean #1.

[0405] The center node can determine whether the local model on the child node #1 is the latest global model according to whether the round indicated by the index #4 is the same as the round of the center node.

[0406] For example, the index #4 can include the number of the round of training related to the local mean #1. If the number of the round indicated by the index #4 is less than the number of the round of the center node, it means that the child node #1 has not received the new global model issued by the center node. Thus, the center node can determine whether the local model on the child node #1 is the latest global model according to the number of the round indicated by the index #4 and the number of the round of the center node.

[0407] By way of example, the index #4 can be used to indicate the AI model corresponding to the local mean #1.

[0408] The center node can determine whether the local model on the child node #1 is the latest global model according to whether the version of the AI model indicated by the index #4 is the same as the version of the global model.

[0409] For example, the index #4 can include the version number of the AI model corresponding to the local mean #1. If the version number of the global model is the same as the version number of the AI model indicated by the index #4, it means that the child node #1 has not received the new global model issued by the center node. Thus, the center node can determine whether the local model on the child node #1 is the latest global model according to the version number of the AI model indicated by the index #4 and the version number of the global model.

[0410] The above is only an example, and the child node #1 can also send other information to enable the center node to determine whether the local model on the child node #1 is the latest global model.

[0411] As another example, the index #2 can be used to determine whether the AI model on the child node #1 is the AI model corresponding to the global mean #1 or whether it is the latest global model.

[0412] By way of example, the index #2 can be used to indicate the round of training related to the local deviation #1.

[0413] By way of example, the index #2 can be used to indicate the AI model corresponding to the local deviation #1.

[0414] The specific determination method can refer to the description of index #4 in the foregoing, and index #4 is replaced by index #2, and local mean #1 is replaced by local deviation #1.

[0415] Further, the method 500 can include that the center node sends the model parameters of the AI model to the subnode #1 again.

[0416] That is, the center node sends the model parameters of the AI model to the subnode #1 twice, and the two times of the model parameters of the AI model are the same.

[0417] For example, in the case that the center node determines that the local model on the subnode #1 is not the latest global model, the center node can send the model parameters of the latest global model to the subnode, so that the subnode #1 can update the local model based on the model parameters of the latest global model. The following takes the index #4 including the round number of the local mean #1 as an example for description.

[0418] Optionally, in the case that the round number indicated by the index #4 is less than the round number of the center node, the center node sends the model parameters of the AI model to the subnode #1 again.

[0419] In this case, the center node sends the same model parameters of the AI model to the subnode #1 before and after receiving the information #4 from the subnode #1.

[0420] After sending the information #4, the subnode #1 receives the model parameters of the AI model sent by the center node. The subnode #1 updates the local model based on the model parameters of the AI model, that is, updates the model parameters of the local model to the model parameters of the AI model sent by the center node, and then calculates the local deviation #1 based on the updated local model. In this way, the subnode #1 can calculate the local deviation #1 based on the latest global model.

[0421] For example, the center node receives the information #4 from the subnode #1. The index #1 in the information #4 indicates that the round number is 2, that is, the local mean #1 indicated by the information #4 is the local mean #1 in the second round. The round number of the training related to the global model is 3, that is, the current global model is the global model aggregated in the third round. The center node can determine that the local mean #1 is used to calculate the global mean #1 in the second round, and the current global model is not the global model of the second round. The center node can send the model parameters of the global model of the third round to the subnode #1 again. The subnode #1 updates the local model based on the model parameters of the global model of the third round, and updates the round number of the training related to the local model to 3. The subnode #1 calculates the local deviation #1 according to the global mean #1 based on the updated local model.

[0422] In this case, the center node can not use the local mean #1 indicated by the information #4 to calculate the global mean #1. In other words, the center node can ignore the local mean #1 indicated by the information #4.

[0423] Alternatively, the child node #1 can perform the step 540 again after receiving the model parameters of the latest global model after sending the information #4. In this case, the center node can calculate the global mean #1 based on the local mean #1 indicated by the information #4 sent again. Alternatively, the center node can also not use the local mean #1 reported again by the child node #1 to calculate the global mean #1.

[0424] In this way, it is beneficial to ensure that the local means of each child node used to calculate the global mean are all calculated based on the unified AI model, thereby facilitating the improvement of the accuracy of the global mean of the BN layer.

[0425] It should be understood that the above is only an example, and the center node can also use the local mean #1 indicated by the information #4 received before sending the model parameters of the latest global model to the child node #1 again to calculate the global mean #1.

[0426] The following takes the number of rounds of training related to the local difference #1 included in the index #2 as an example.

[0427] Optionally, in the case where the number of rounds indicated by the index #2 is less than the number of rounds of the center node, the center node sends the model parameters of the AI model to the child node #1 again.

[0428] In this case, the center node sends the same model parameters of the AI model to the child node #1 before and after receiving the information #2 from the child node #1.

[0429] In this case, the center node can not use the local difference #1 indicated by the information #2 to calculate the global difference #1. In other words, the center node can ignore the local difference #1 indicated by the information #2.

[0430] Alternatively, the child node #1 can update the local model based on the model parameters of the latest global model after sending the information #2, and then calculate the local difference #1 based on the updated local model. In this way, the child node #1 can calculate the local difference #1 based on the latest global model. The child node #1 can send the information #2 again, and in this case, the center node can calculate the global difference #1 based on the local difference #1 indicated by the information #2 sent again. Alternatively, the center node can also not use the local difference #1 reported again by the child node #1 to calculate the global difference #1.

[0431] In this way, it is beneficial to ensure that the local variance of each sub-node used to calculate the global variance is calculated based on the unified AI model, thereby improving the accuracy of the global variance of the BN layer.

[0432] It should be understood that the above is only an example, and the center node can also use the local variance #1 indicated by the information #2 received before the model parameters of the latest global model are sent to the sub-node #1 again to calculate the global variance #1.

[0433] The above is only an example, and the sub-node #1 can also use other information to enable the center node to determine whether the AI model on the sub-node #1 is the latest global model. Further, the center node can determine whether the model parameters of the latest global model need to be issued again according to the determination result. The center node can determine whether the local mean #1 and / or the local variance #1 reported by the sub-node #1 are determined based on the latest global model according to the determination result, and determine whether to use the local mean #1 and / or the local variance #1 reported by the sub-node #1 to calculate the global mean #1 and / or the global variance #1.

[0434] The global variance #1 and the global mean #1 can be used for training of the AI model on the sub-node.

[0435] For example, the sub-node #1 can perform one or more forward propagation and backward propagation based on the global variance #1 and the global mean #1 to update the model parameters of the local model. The specific training process can be referred to later, and will not be described here.

[0436] It should be understood that only two sub-nodes are taken as examples in FIG. 5, and the number of sub-nodes in the embodiments of the present application is not limited.

[0437] FIG. 8 shows a schematic flowchart of a method of communication according to an embodiment of the present application. The method 800 shown in FIG. 8 can be regarded as a specific implementation of the method 500. The specific description can refer to the method 500, and part of the description is appropriately omitted when describing the method 800 to avoid repetition. In this scheme, the center node aggregates the parameters of the local model uploaded by each sub-node to generate a new global model. The center node distributes the parameters of the new global model to each sub-node. The sub-node can calculate the mean of the features of the BN layer according to the same model, and upload the mean to the center node. The center node aggregates the means of the features of the BN layer uploaded by each sub-node to generate the global mean of the features of the BN layer, and distributes the global mean to each sub-node. The sub-node calculates the variance of the features of the BN layer according to the global mean, and uploads the variance to the center node. The center node aggregates the variances of the features of the BN layer uploaded by each sub-node to generate the global variance of the features of the BN layer. For the convenience of description, the main equipment on the network device side in the method 800 is taken as an example of the center node, and the equipment on the terminal device side is taken as an example of the sub-node. For example, the center node can be a base station. The equipment on the network device side can be a network device, or a module (such as a chip) in the network device, or software (such as a control subsystem) containing the function of the network device, or other devices in communication with the network device, such as an AI network element, which is a server, such as an OTT device or a cloud server, etc., or a combination of the network device and other devices in communication with the network device. The equipment on the terminal device side can be a terminal device, or a module (such as a chip) in the terminal device, or software (such as a control subsystem) containing the function of the terminal device, or other devices in communication with the terminal device, such as an AI network element, which is a server, such as an OTT device or a cloud server, etc., or a combination of the terminal device and other devices in communication with the terminal device. The center node can also be replaced by other equipment other than the equipment on the network device side, and the sub-node can also be replaced by other equipment other than the equipment on the terminal device side. The specific description is referred to the foregoing, which will not be repeated here.

[0438] For the convenience of description, only one sub-node is taken as an example in FIG. 8, and the behavior of other sub-nodes can be referred to the sub-node.

[0439] As shown in FIG. 8, the method 800 can include the following steps.

[0440] 801, the center node sends model configuration information and model training information to the sub-node.

[0441] Exemplarily, the center node can send the same model configuration information and model training information to all sub-nodes in a broadcast manner.

[0442] Exemplarily, the model configuration information can include at least one of the following: a type of the initial AI model, structure information of the initial AI model, or a model parameter of the initial AI model. For example, the type of the initial AI model can include at least one of the following: a DNN, a CNN, or a Transformer model.

[0443] For example, the structure information of the initial AI model can include at least one of the following: a number of hidden layers of the DNN, a number of neurons per layer or partial layer, and a position of an activation function or a batch normalization layer.

[0444] The type of the initial AI model and the structure information of the initial AI model can be embodied in a configuration text manner, or can be a code script that can compile a corresponding initial AI model.

[0445] The model parameter of the initial AI model can be generated by the center node through a certain strategy. For example, the model parameter of the initial AI model can be randomly generated or pre-trained by the center node. Alternatively, the model parameter of the initial AI model can also be obtained by the center node from other entities. The other entities can be other devices in communication with the center node, such as servers, OTT devices, cloud servers, and the like.

[0446] Exemplarily, the model training information can include at least one of the following: an optimizer used by the sub-node to perform gradient update, a regularization penalty term (such as an L2 penalty term), an initial learning rate, a gradient update batch size, or a data augmentation method.

[0447] For example, the optimizer can be any of the following: stochastic gradient descent (SGD), root mean square propagation (RMSprop), or adaptive moment estimation (Adam).

[0448] The data augmentation method can be one or multiple.

[0449] The center node issues model configuration information, which can indicate the model parameter of the initial AI model. For example, the number of rounds at the center node can be set to 1, and the center node enters the first round of distributed training. That is, the number of rounds of training related to the initial AI model is 1.

[0450] Further, the sub-node can also send the data amount of the local training sample set to the center node.

[0451] Further, the center node can also send the format of the correlation between the mean and variance of the batch normalization layer to the sub-node.

[0452] Alternatively, the format of the association between the mean and the variance of the batch normalization layer can also be predefined.

[0453] After receiving the model configuration information, the child node can update the local model according to the model parameters of the initial AI model, and set the number of the local round to 1. The child node enters the first round of distributed training. That is, the number of the round of training related to the local model is 1.

[0454] 802, the child node performs training of the local model.

[0455] In one round, the child node performs training of the local model based on the local training data set to update the parameters of the local model. The update process is an iteration process. During the training process of one round, the parameters of the local model can be updated once or multiple times.

[0456] 803, the child node feeds back the updated model parameters of the local model to the center node.

[0457] As an example, the child node can feed back the updated model parameters of the local model to the center node under the condition #1.

[0458] The condition #1 can be understood as an upload condition of the model parameters. After updating the model parameters of the local model, the child node #1 can determine whether the condition #1 is met, and upload the updated model parameters of the local model if the condition #1 is met.

[0459] Exemplarily, the condition #1 can be a condition related to the number of updates of the local model.

[0460] For example, the condition #1 can be that the number of updates t of the local model is n times of a constant T. T is a positive integer. n is a positive integer. T can be predefined, configured by the center node, or determined by the child node. n can be predefined, configured by the center node, or determined by the child node.

[0461] Exemplarily, the condition #1 can be a condition related to the updated model parameters of the local model.

[0462] For example, the condition #1 can be that the difference between the updated model parameters of the local model and the model parameters of the local model before the start of the current round of training is greater than or equal to a threshold value #1.

[0463] The model parameters of the local model before the start of the current round of training are the model parameters of the global model issued by the center node in the current round.

[0464] The threshold #1 can be predefined, configured by the center node, or determined by the sub node.

[0465] Exemplarily, the condition #1 can be a termination condition of the local model training in a round. When the termination condition is met, the training of the local model in the round can be terminated. That is, in the case that the training of the local model is terminated, the model parameters of the updated local model are fed back to the center node. The termination condition can be predefined, configured by the center node (for example, indicated by the model training information), or determined by the sub node.

[0466] As another example, the center node can send indication information to each sub node to inform each sub node to feed back the model parameters of the updated local model.

[0467] After receiving the indication information, the sub node can feed back the model parameters of the current local model to the center node.

[0468] The above is only an example, and the embodiments of the present application do not limit the triggering manner of the sub node reporting the model parameters.

[0469] 804, the center node aggregates the model parameters fed back by the sub nodes to update the global model.

[0470] After receiving the model parameters fed back by one or more sub nodes, the center node performs an aggregation operation to generate a new global model. For example, the aggregation operation can be to calculate the average of the model parameters fed back by the plurality of sub nodes. For another example, the aggregation operation can be to calculate the weighted average of the model parameters fed back by the plurality of sub nodes.

[0471] The one or more sub nodes can be determined in various ways. For example, the receiving process can end after a period of time, and the model parameters fed back by the sub nodes received in the period of time can be used to aggregate the global model. Alternatively, the center node ends after receiving a certain number of model parameters fed back by the sub nodes. The certain number of model parameters fed back by the sub nodes can be used to aggregate the global model.

[0472] The embodiments of the present application do not limit the specific implementation manner of the aggregation operation.

[0473] 805, the center node sends the model parameters of the updated global model to the sub node.

[0474] After updating the global model, or after the center node issues the model parameters of the updated global model, the number of the round at the center node is incremented by 1, that is, the number of the training round related to the updated global model is incremented by 1.

[0475] For example, the center node can broadcast the model parameters of the updated global model to the child nodes.

[0476] 806. The child node updates the local model according to the received model parameters.

[0477] After the child node receives the model parameters of the global model from the center node, or after the child node updates the local model according to the received model parameters, the number of the local round is incremented by 1, i.e., the number of the round of training related to the local model is incremented by 1.

[0478] 807. The child node calculates the mean of the input of the BN layer (e.g., network layer #1) of the local model (e.g., local mean #1).

[0479] The child node calculates the mean of the input of one or more BN layers in the local model.

[0480] 808. The child node sends information indicating the mean of the input of the BN layer and the related index (e.g., index #4) to the center node.

[0481] The index related to a mean can be used to indicate the number of the round of training related to the mean and the identification of the BN layer related to the mean. According to the index, it can be determined which round and which BN layer the mean corresponds to, or in other words, according to the index, it can be determined which round of which BN layer the mean is used for the calculation of the global mean.

[0482] FIG. 9 shows a schematic diagram of the format of an information #4. FIG. 9 only takes an AI model including two BN layers as an example. The means of the input of the two BN layers are mean b and mean b', respectively.

[0483] As shown in FIG. 9, the information #4 includes two means, i.e., mean b and mean b'. The index adjacent to a mean after the mean is the index related to the mean. As shown in FIG. 9, the index value of the index related to mean b includes the number of the round c and the layer number L1, and the index value of the index related to mean b' includes the number of the round c and the layer number L2.

[0484] 809. The center node aggregates the means fed back by the child nodes to obtain the global mean.

[0485] Taking mean b in FIG. 9 as an example, the center node can determine, according to information #4, that the number of the round of training related to mean b is c and the layer number related to mean b is L1, i.e., mean b is used for the calculation of the global mean corresponding to the BN layer with the layer number L1 in the c-th round.

[0486] The center node can aggregate the means of the input of each BN layer of each child node according to the index to obtain the global mean corresponding to each BN layer.

[0487] 810, the center node sends information indicating a global mean (e.g., global mean #1) and a related index (e.g., index #1) to the child node.

[0488] The index related to a global mean can be used to indicate the number of the round of training related to the global mean and the identification of the BN layer related to the global mean. According to the index, it can be determined which round and which BN layer the global mean corresponds to, or in other words, according to the index, it can be determined which round of which BN layer the global mean is used for the calculation of the variance of the input.

[0489] FIG. 10 shows a schematic diagram of the format of information #1. FIG. 10 only takes an example in which the AI model includes two BN layers. The global means corresponding to the two BN layers are global mean a and global mean a', respectively.

[0490] As shown in FIG. 10, information #1 includes two means, i.e., global mean a and global mean a'. The index adjacent to a mean after the mean is the index related to the mean. As shown in FIG. 10, the index value of the index related to global mean a includes the number of rounds c' and the layer number L1, and the index value of the index related to global mean b includes the number of rounds c' and the layer number L2.

[0491] 811, the child node determines whether it needs to obtain the model parameters of the latest global model according to the index.

[0492] If yes, step 812 is performed, and if no, step 815 is performed.

[0493] Exemplarily, the child node can determine whether it needs to obtain the model parameters of the latest global model according to the number of rounds indicated by index #1. Taking FIG. 9 and FIG. 10 as an example, the child node sends information #4 to the center node, wherein index #4 indicates the number of rounds c, i.e., the child node is in the cth round, and the number of rounds of training related to the local model of the child node is c. The center node sends information #1 to the child node, wherein index #1 indicates the number of rounds c', i.e., the center node is in the c'th round, and the number of rounds of training related to the global model is the number c'. If c and the number c' are the same, the child node can determine that the current local model is the latest global model, and step 815 is performed. If c is less than the number c', the child node can determine that the current local model is not the latest global model, and then step 812 is performed.

[0494] 812, the child node sends information (e.g., information #6) to the center node to request the model parameters of the AI model.

[0495] The child node can send information to the center node to request the center node to issue the model parameters of the latest global model.

[0496] 813, the center node sends the model parameters of the latest global model to the child node.

[0497] The model parameters are the model parameters issued in step 805.

[0498] 814, the child node updates the local model according to the received model parameters.

[0499] After the child node receives the model parameters of the global model from the center node, or after the child node updates the local model according to the received model parameters, the number of the local round is incremented by 1, i.e. the number of the training round related to the local model is incremented by 1.

[0500] 815, the child node calculates the variance of the input of the BN layer of the local model according to the global mean (e.g. local variance #1).

[0501] Exemplarily, the child node can perform step 815 in the case that the number of the round indicated by the index related to the global mean is equal to the number of the training round related to the local model.

[0502] Alternatively, the child node can perform step 815 in the case that the child node receives the model parameters issued by the center node after sending information #6.

[0503] The child node can determine the association between the global mean issued by the center node and the variance of the input of each BN layer of the local model according to index #1, and further calculate the variance of the input of each BN layer in the local model according to the global mean corresponding to each BN layer.

[0504] Taking the global mean a in FIG. 10 as an example, the center node can determine that the number of the training round related to the global mean a is c', and the layer number related to the global mean a is L1 according to information #1, i.e. the global mean a is used for calculating the variance of the input of the BN layer with layer number L1 in the local model of the c'th round.

[0505] It should be understood that the above steps 811 to 814 are optional steps. If the method 800 includes steps 811 to 814, it is beneficial to ensure that each child node can calculate the local variance based on the parameters of the latest global model in step 815, i.e. it is beneficial to ensure that each child node calculates the local variance based on a unified AI model. If the method 800 does not include steps 811 to 814, the child node may calculate the local variance based on an outdated model, which is obtained according to the global model issued in the previous round.

[0506] 816, the child node sends information indicating the variance of the input of the BN layer and the related index (e.g. index #2) to the center node.

[0507] The index associated with a variance can be used to indicate the number of the round of training associated with the variance and the identity of the BN layer associated with the variance. According to the index, it can be determined which round and which BN layer the variance corresponds to, or in other words, according to the index, it can be determined which round and which BN layer the variance is used for the calculation of the global variance corresponding to.

[0508] FIG. 11 shows a schematic diagram of the format of information #2. FIG. 11 takes an example in which the AI model includes two BN layers. The variances of the inputs of the two BN layers are variance d and variance d', respectively.

[0509] As shown in FIG. 11, information #2 includes two variances, namely variance d and variance d'. The index adjacent to a variance after the variance is the index associated with the variance. As shown in FIG. 11, the index value of the index associated with variance d includes the number of the round c" and the layer number LI, and the index value of the index associated with variance d' includes the number of the round c" and the layer number L2.

[0510] 817, the center node aggregates the variances fed back by the sub-nodes according to the index to obtain a global variance (such as global variance #1).

[0511] Taking variance a in FIG. 11 as an example, the center node can determine, according to information #2, that the number of the round of training associated with variance d is c", and the layer number associated with variance d is LI, that is, variance d is used for the calculation of the global variance corresponding to the BN layer with the layer number LI in the c" round.

[0512] The center node can aggregate the variances of the inputs of the respective BN layers of the respective sub-nodes according to the index to obtain the global variances corresponding to the respective BN layers.

[0513] 818, the center node sends information indicating the global variance and the related index (such as index #3) to the sub-nodes.

[0514] The index associated with a global variance can be used to indicate the number of the round of training associated with the global variance and the identity of the BN layer associated with the global variance. According to the index, it can be determined which round and which BN layer the global variance corresponds to, or in other words, according to the index, it can be determined which round and which BN layer the global variance is used as the input in the training process.

[0515] The format of information #3 can refer to information #1, which will not be described here again.

[0516] Returning to step 802, steps 802 to 818 are repeatedly executed until the global model training is completed, for example, until the global model converges, such as the model performance reaches the requirement.

[0517] In step 802, after receiving the global mean and the global variance, the child node can train the local model based on the global mean and the global variance.

[0518] The training process after receiving the global mean and the global variance is exemplarily described below.

[0519] Exemplarily, in one round, the training process can be divided into two stages: stage 1 and stage 2. In stage 1, the child node can perform one or more forward propagation and backward propagation to update the model parameters of the local model using the global mean and the global variance corresponding to each BN layer. Taking the formula (2) in the foregoing as an example, in stage 1, the mean and the variance in the formula (2) are respectively replaced by the global mean and the global variance to calculate the input of each BN layer, and one or more forward propagation and backward propagation are performed based thereon to update the model parameters of the local model. In stage 2, the mean and the variance of the input of each BN layer can be calculated based on the local model updated in stage 1. Taking one BN layer as an example, the input of the BN layer based on a batch of training data is obtained by inputting the batch of training data into the updated local model, and then the mean and the variance of the input of the BN layer are calculated. The child node can perform one or more forward propagation and backward propagation based on the mean and the variance of the input of each BN layer obtained by calculation to update the parameters of the local model. For example, in stage 2, the output of each BN layer can be calculated by using the formula (2), and one or more forward propagation and backward propagation are performed based thereon to update the model parameters of the local model. After the training process, steps 803 to 818 are continued to be performed.

[0520] FIG. 12 is a schematic flowchart of another method of communication provided in the present application. Exemplarily, the method 1200 shown in FIG. 12 can be applied to the scenario of distributed training. In the method 1200, the center node aggregates the mean of the input of the BN layer and the mean of the square of the input of the BN layer fed back by the child node, and obtains the global variance corresponding to the BN layer according to the aggregation result.

[0521] As shown in FIG. 12, the method 1200 can include the following steps.

[0522] 1210, a plurality of child nodes (such as child node #1 and child node #2 in the figure) send information #7 (an example of the seventh information) to the center node. The information #7 includes information of the local mean #2 (an example of the third mean) and the local mean #3 (an example of the fourth mean). The local mean #2 is the input of the network layer #1 in the AI model on the child node, and the local mean #3 is the mean of the square of the input.

[0523] The description of the network layer #1 in the method 1200 can refer to the description of the network layer #1 in the method 500, and the description of the local mean #2 in the method 1200 can refer to the description of the local mean #1 in the method 500, which will not be repeated herein.

[0524] The local mean #2 and the local mean #3 can be used for calculation of the global variance #2 (an example of the third variance).

[0525] The information #7 from the plurality of child nodes respectively includes information indicating the local mean #2 and the local mean #3 of the plurality of child nodes. Alternatively, the local mean #2 and the local mean #3 indicated by the information #7 from the plurality of child nodes are respectively the local mean #2 and the local mean #3 of the plurality of child nodes.

[0526] The plurality of child nodes in the step 1210 are part or all of the aforementioned M child nodes.

[0527] The plurality of child nodes can include the child node #1, and the child node #1 sends the information #7 to the center node. The local mean #2 indicated by the information #7 from the child node #1 is the local mean #2 of the child node #1. The local mean #3 indicated by the information #7 from the child node #1 is the local mean #3 of the child node #1.

[0528] The plurality of child nodes can also be replaced by a plurality of devices.

[0529] Further, the method 1200 can further include the step 1220.

[0530] 1220, the center node determines the global variance #2 according to the local mean #2 and the local mean #3 indicated by the information #7 from one or more child nodes.

[0531] The global variance #2 is the global variance corresponding to the network layer #1, or the global variance related to the network layer #1. The determination manner of the global variance #2 can refer to the description hereinafter.

[0532] The one or more child nodes in the step 1220 are part or all of the aforementioned M child nodes.

[0533] The one or more child nodes in the step 1220 belong to the plurality of child nodes in the step 1210.

[0534] Further, the method 1200 can further include the step 1230.

[0535] 1230, the center node sends information #8 (an example of the eighth information) to the plurality of child nodes. The information #8 includes information indicating the global mean #2 (an example of the fifth mean) and the global variance #2.

[0536] The global mean #2 is the global mean corresponding to the network layer #1, or the global mean related to the network layer #1. The description of the global mean #2 can refer to the global mean #1 in the method 500.

[0537] The global mean #2 and the global variance #2 can be used for training the AI model on the child node.

[0538] The plurality of child nodes in step 1230 is part or all of the aforementioned M child nodes.

[0539] The plurality of child nodes in step 1230 can be the same as or different from the plurality of child nodes in step 1210.

[0540] The plurality of child nodes can include a child node #1. Taking the child node #1 as an example, after receiving the information #8 from the center node, the child node #1 can train the local model according to the global mean #2 and the global variance #2.

[0541] The determination of the local mean #3 is described below.

[0542] The mean of the square of the input of the network layer #1 of the AI model on the child node is determined based on a batch of training data on the child node.

[0543] Taking a child node m as an example, a batch of training data B m on the child node m has a data amount of |B m |. |B m | is a positive integer. When the batch of training data B m is input into the AI model on the child node m, correspondingly, |B m | inputs of the network layer #1 can be obtained, and the average of the squares of the |B m | inputs can be used as the mean of the square of the input of the network layer #1 of the AI model on the child node m.

[0544] The determination of the global mean #2 is described below.

[0545] The center node can aggregate the local mean #2 of one or more child nodes to obtain the global mean #2.

[0546] The one or more child nodes can be part or all of the M child nodes, that is, the center node can determine the global mean #2 based on the local mean #2 of part or all of the child nodes.

[0547] The specific determination method of the global mean #2 can refer to the determination method of the global mean #1 in the method 500, and the global mean #1 is replaced by the global mean #2, the local mean #1 is replaced by the local mean #2, and the information #4 is replaced by the information #7.

[0548] The following describes how to determine the global variance #2.

[0549] The center node can aggregate the local mean #2 and the local mean #3 of one or more sub-nodes, and calculate the global variance #2 according to the aggregation result of the local mean #2 and the aggregation result of the local mean #3. The aggregation result of the local mean #2 can be the global mean #2.

[0550] The one or more sub-nodes can be part or all of the M sub-nodes.

[0551] Illustratively, the center node aggregating the local mean #2 of the one or more sub-nodes can include: the center node calculating the average of the local mean #2 of the plurality of sub-nodes, and taking it as the aggregation result of the local mean #2.

[0552] Alternatively, the center node aggregating the local mean #2 of the one or more sub-nodes can include: the center node calculating the weighted average of the local mean #2 of the plurality of sub-nodes, and taking it as the aggregation result of the local mean #2. For example, in the calculation of the weighted average, the weight corresponding to the local mean #2 of each sub-node can be in a positive correlation with the data amount of the batch training data of each sub-node.

[0553] Illustratively, the center node aggregating the local mean #3 of the one or more sub-nodes can include: the center node calculating the average of the local mean #3 of the plurality of sub-nodes, and taking it as the aggregation result of the local mean #3.

[0554] Alternatively, the center node aggregating the local mean #3 of the one or more sub-nodes can include: the center node calculating the weighted average of the local mean #3 of the plurality of sub-nodes, and taking it as the aggregation result of the local mean #3. For example, in the calculation of the weighted average, the weight corresponding to the local mean #3 of each sub-node can be in a positive correlation with the data amount of the batch training data of each sub-node.

[0555] According to the aggregation result of the local mean #2 and the aggregation result of the local mean #3, the global variance #2 can be calculated, according to the square of the aggregation result of the local mean #2 and the aggregation result of the local mean #3.

[0556] Optionally, the global variance #2 satisfies the following formula:

[0557] wherein, represents the global variance corresponding to the BN layer (i.e. network layer #1) of the lth layer of the AI model, i.e. the global variance #2. The mean of the square of the input of the BN layer, i.e., the aggregation result of the local means #3 indicated by the information #7 from one or more child nodes, E[h l (x) represents the global mean corresponding to the BN layer, i.e., the global mean #2, i.e., the aggregation result of the local means #2 indicated by the information #7 from one or more child nodes. B represents a training data set on multiple child nodes.

[0558] In the scheme of the embodiments of the present application, the center node calculates the global variance #2 based on the mean of the input of the BN layer of the AI model on the child node and the mean of the square of the input, which is helpful to avoid the influence of the non-independent and identically distributed local data of each child node, so as to enable the center node to aggregate a more accurate global variance #2, thereby facilitating to improve the accuracy of the batch normalization processing in the distributed training process and improve the performance of the distributed training.

[0559] Meanwhile, in the embodiments of the present application, the number of interactions between the child node and the center node is less, which is helpful to reduce the interaction overhead.

[0560] Optionally, the information #7 further includes an index #5 (an example of a fifth index).

[0561] The index #5 is related to at least one of the local mean #2 or the local mean #3.

[0562] According to the index #5, it can be determined that the local mean #2 and the local mean #3 are used to calculate the global variance #2.

[0563] The correlation between the index #5 and the local mean #2 is described below.

[0564] The index #5 and the local mean #2 are related, which can be understood as that the index #5 and the local mean #2 are both related to the same round and / or the same network layer.

[0565] The correlation between the index #5 and the local mean #2 can be embodied in various ways.

[0566] Exemplarily, the index #5 and the local mean #2 can be carried in the same signaling. That is, the information #7 can be carried in the same signaling.

[0567] Alternatively, the index #5 and the local mean #2 can be carried in two signalings respectively. That is, the information #7 can be carried in two signalings. The two signalings are related.

[0568] The information indicated by the index #1 is described below based on two examples (example #5 and example #6).

[0569] Example #5:

[0570] As an example, the index #5 can indicate the round of training related to the local mean #2 and / or the local mean #3.

[0571] After receiving the information #7, the center node can determine, according to the index #5, in which round the local mean #2 and the local mean #3 are calculated, and then can calculate the global variance #2 based on the local mean #2 and the local mean #3 in the round.

[0572] Optionally, the index #5 can indicate the identification of the round of training related to the local mean #2 and / or the local mean #3.

[0573] For example, the index #5 can include the number of the round of training related to the local mean #2 and / or the local mean #3.

[0574] Optionally, the index #5 can indicate the identification of the AI model corresponding to the local mean #2 and / or the local mean #3.

[0575] The index #5 can indicate the round of training related to the local mean #2 and / or the local mean #3 by indicating the identification of the AI model corresponding to the local mean #2 and / or the local mean #3.

[0576] After receiving the information #7, the center node can determine, according to the index #5, which version of the AI model the local mean #2 and the local mean #3 correspond to, i.e., determine which round the local mean #2 and the local mean #3 correspond to.

[0577] Example #6:

[0578] As an example, the index #5 can indicate the network layer #1. That is, the index #5 can indicate the network layer corresponding to the local mean #2 or the local mean #3.

[0579] Exemplarily, the index #5 can include the identification of the network layer #5.

[0580] The center node can determine, according to the index #5, which network layer the local mean #2 and the local mean #3 are associated with, and thus determine that the local mean #2 and the local mean #3 are used to calculate the global variance corresponding to the network layer.

[0581] It should be understood that the above is only an example, and the network layer #1 can also be indicated in other ways. Exemplarily, the position of the information indicating the local mean #2 or the local mean #3 in the information #7 can be used to indicate the network layer #1.

[0582] Example #5 and example #6 can also be used in combination. That is, the index #5 can indicate the network layer #1, and the round of training related to the local mean #2 or the local mean #3.

[0583] Optionally, the index #5 can indicate the identity of the network layer #1, and the identity of the round of training related to the local mean #2 or the local mean #3.

[0584] Illustratively, the index #5 can include the identity of the network layer #1 and the identity of the round of training related to the local mean #2 or the local mean #3. Optionally, the index #5 can include two fields (or information elements) respectively carrying the identity of the network layer #1 and the identity of the round of training related to the local mean #2 or the local mean #3.

[0585] Illustratively, the index #5 can include the identity of the network layer #1 and the identity of the round of training related to the local mean #2 or the local mean #3. Optionally, the index #5 can include two fields (or information elements) respectively carrying the identity of the network layer #1 and the identity of the round of training related to the local mean #2 or the local mean #3.

[0586] Optionally, the index #5 can indicate the identity of the network layer #1 and the identity of the AI model corresponding to the local mean #2 or the local mean #3.

[0587] The specific description of the relevance of at least one of the local mean #2 or the local mean #3 to the index #5 can refer to the description of the relevance of the index #1 and the global mean #1 in the method 500, and the description of the relevance of the index #2 and the local variance #1, and the description of the index #5 can refer to the description of the index #1 and the index #2 in the method 500, which will not be repeated here.

[0588] Optionally, the information #8 further includes an index #6 (an example of a sixth index).

[0589] The index #6 is related to at least one of the global mean #2 or the global variance #2.

[0590] The specific description of the relevance of at least one of the global mean #2 or the global variance #2 to the index #6 can refer to the description of the relevance of the index #1 and the global mean #1 in the method 500, and the description of the relevance of the index #2 and the local variance #1, and the description of the index #6 can refer to the description of the index #1 and the index #2 in the method 500, which will not be repeated here.

[0591] Optionally, the index #5 and the index #6 have a correlation relationship.

[0592] According to the correlation relationship of the index #5 and the index #6, it can be determined that at least one of the local mean #2 or the local mean #3 is related to the global variance #2, or in other words, the global variance #2 is determined based on the local mean #2 and the local mean #3.

[0593] The correlation relationship of the index #5 and the index #6 will be described below.

[0594] Optionally, the index value of the index #5 is the same as the index value of the index #6.

[0595] Optionally, the index #5 and the index #6 have a predefined association relationship.

[0596] Optionally, the index #5 and the index #6 have a preconfigured association relationship.

[0597] That is, the method 1200 can further include that the center node sends information #9 to the plurality of sub-nodes, the information #9 indicating an association relationship between two indexes.

[0598] Exemplarily, the association relationship between the two indexes can be that the index values of the two indexes satisfy a preset condition #2. The preset condition #2 can be predefined, or can also be indicated by the information #9.

[0599] The description of the association relationship between the two indexes can refer to the index #1 and the index #2 in the method 500, which will not be repeated here.

[0600] Further, the method 1200 can further include that the sub-node #1 receives model parameters of the AI model from the center node.

[0601] Exemplarily, the model parameters of the AI model can be model parameters of the AI model used to calculate the local mean #2 and the local mean #3.

[0602] For example, each sub-node can calculate and report to the center node the mean of the input of the network layer #1 of the AI model and the mean of the square of the input based on the local training data, after the center node issues the latest global model. In other words, after entering a new round, each sub-node can calculate the mean of the input of the network layer #1 and the mean of the square of the input based on the latest global model issued by the center node before starting local training, instead of calculating the mean of the input of the network layer #1 and the mean of the square of the input after the parameters of the local model change after the local training starts.

[0603] In the scheme of the embodiments of the present application, each sub-node can calculate the mean of the input of the BN layer and the mean of the square of the input based on the received model parameters of the AI model before the start of the current round of training, which is conducive to ensuring that each sub-node uses a unified AI model to calculate the mean of the input of the BN layer and the mean of the square of the input, or in other words, is conducive to ensuring that the local model of each sub-node adopts a unified AI model when each sub-node calculates the mean of the input of the BN layer and the mean of the square of the input of the local model, thereby facilitating the accuracy of the global mean and the global variance.

[0604] Further, the center node and / or the sub-node can determine whether the AI model on the sub-node is consistent with the global model on the center node. The following takes scheme #3 as an example for illustration.

[0605] Scheme #3:

[0606] In scheme #3, the center node can determine whether the local model is consistent with the global model on the center node.

[0607] In a possible implementation, the sub-node #1 can send information to the center node, which can be used to determine whether the model parameters of the AI model need to be sent to the sub-node #1.

[0608] The AI model can be the latest global model, and the information can be used to determine whether the AI model on the sub-node #1 is the latest global model.

[0609] As an example, the index #5 can be used to determine whether the AI model on the sub-node #1 is the AI model corresponding to the global mean #2 and / or the global variance #2, or whether it is the latest global model.

[0610] Exemplarily, the index #5 can be used to indicate the round of training related to the local mean #2 and / or the local mean #3.

[0611] The center node can determine whether the local model on the sub-node #1 is the latest global model according to whether the round indicated by the index #5 is the same as the round of the center node.

[0612] Exemplarily, the index #5 can be used to indicate the AI model corresponding to the local mean #2 and / or the local mean #3.

[0613] The center node can determine whether the local model on the sub-node #1 is the latest global model according to whether the version of the AI model indicated by the index #5 is the same as the version of the global model.

[0614] The specific determination manner can refer to the related description of the index #4 in the method 500, which will not be described here.

[0615] The above is only an example, and the sub-node #1 can also send other information to enable the center node to determine whether the local model on the sub-node #1 is the latest global model.

[0616] Further, the method 500 can include that the center node sends the model parameters of the AI model to the sub-node #1 again.

[0617] That is, the center node sends the model parameters of the AI model to the sub-node #1 twice, and the two times of the model parameters of the AI model are the same.

[0618] For example, in the case where the center node determines that the local model on the sub node #1 is not the latest global model, the center node can send the model parameters of the latest global model to the sub node, so that the sub node #1 can update the local model based on the model parameters of the latest global model.

[0619] Optionally, the index #5 can include the number of the round of training related to the local mean #2 and / or the local mean #3, in the case where the number of the round indicated by the index #5 is less than the number of the round of the center node, the center node again sends the model parameters of the AI model to the sub node #1.

[0620] In this case, the center node sends the same model parameters of the AI model to the sub node #1 before and after receiving the information #7 from the sub node #1.

[0621] After sending the information #7, the sub node #1 receives the model parameters of the AI model sent by the center node. The sub node #1 updates the local model based on the model parameters of the AI model, that is, updates the model parameters of the local model to the model parameters of the AI model sent by the center node.

[0622] In this case, the center node can not use the local mean #2 and the local mean #3 indicated by the information #7 to calculate the global mean #2 and / or the global variance #2. In other words, the center node can ignore the information #7 from the sub node #1.

[0623] Alternatively, the sub node #1 can calculate the local mean #2 and the local mean #3 based on the model parameters of the latest global model after receiving the model parameters of the latest global model after sending the information #7, and again perform step 1210. In this case, the center node can calculate the global mean #2 and / or the global variance #2 based on the local mean #2 and the local mean #3 indicated by the information #7 sent again. Alternatively, the center node can also not use the local mean #2 and the local mean #3 reported again by the sub node #1 to calculate the global mean #2 and / or the global variance #2.

[0624] In this way, it is beneficial to ensure that the mean of the input of the BN layer of each sub node used to calculate the global mean and the global variance and the mean of the square of the input of the BN layer are calculated based on the unified AI model, thereby facilitating the accuracy of the global mean and the global variance corresponding to the BN layer.

[0625] It should be understood that the above is only an example, and the center node can also use the local mean #2 and the local mean #3 indicated by the information #7 received before again sending the model parameters of the latest global model to the sub node #1.

[0626] The above is only an example. The child node #1 can also use other information to enable the center node to determine whether the AI model on the child node #1 is the latest global model. Further, the center node can determine whether the model parameters of the latest global model need to be re-deployed according to the determination result. The center node can determine whether the local mean #2 and the local mean #3 reported by the child node #1 are determined based on the latest global model according to the determination result, and determine whether the local mean #2 and the local mean #3 reported by the child node #1 are used to calculate the global mean #1 and the global variance #1.

[0627] Alternatively, the child node can also determine whether the local model is the latest global model. For example, the index #6 can be used to determine whether the AI model on the child node #1 is the AI model corresponding to the global mean #2 and / or the global variance #2, or whether it is the latest global model.

[0628] The global variance #2 and the global mean #2 can be used for training of the AI model on the child node.

[0629] For example, the child node #1 can perform one or more forward propagation and backward propagation based on the global variance #2 and the global mean #2 to update the model parameters of the local model. The specific training process can be referred to in the foregoing, and will not be described here.

[0630] It should be understood that two child nodes are taken as examples in FIG. 12, and the number of child nodes in the embodiments of the present application is not limited.

[0631] FIG. 13 shows a schematic flowchart of a method of communication according to an embodiment of the present application. The method 1300 shown in FIG. 13 can be regarded as a specific implementation of the method 1200. The specific description can be referred to the method 1200, and part of the description is appropriately omitted when the method 1300 is described. In this scheme, the center node aggregates the parameters of the local model uploaded by each child node to generate a new global model. The center node deploys the parameters of the new global model to each child node. The child node can calculate the mean of the features of the BN layer and the mean of the square of the features of the BN layer according to the same global model, and upload them to the center node. The center node aggregates the means of the features of the BN layer uploaded by each child node to generate the global mean of the features of the BN layer. The center node calculates the global variance of the features of the BN layer according to the global mean and the means of the square of the features of the BN layer uploaded by each child node. For ease of description, the method 1300 mainly takes the center node as the network device side device and the child node as the terminal device side device as an example for description. The center node can also be replaced by other devices other than the network device side device, and the child node can also be replaced by other devices other than the terminal device side device. The specific description is referred to the foregoing, and will not be described here.

[0632] For ease of description, one child node is taken as an example in FIG. 13, and the behaviors of other child nodes can be referred to the child node.

[0633] As shown in FIG. 13, the method 1300 can include the following steps. The descriptions of steps 1301 to 1306 are consistent with steps 801 to 806, and the specific descriptions can be referred to steps 801 to 806.

[0634] 1301, the center node sends model configuration information and model training information to the child node.

[0635] 1302, the child node performs training of the local model.

[0636] 1303, the child node feeds back the model parameters of the updated local model to the center node.

[0637] 1304, the center node aggregates the model parameters fed back by the child node to update the global model.

[0638] 1305, the center node issues the model parameters of the updated global model to the child node.

[0639] 1306, the child node updates the local model according to the received model parameters.

[0640] 1307, the child node calculates the mean (such as local mean #2) of the input of a BN layer (such as network layer #1) of the local model and the mean (such as local mean #3) of the square of the input.

[0641] The child node calculates the mean of the input of one or more BN layers in the local model and the mean of the square of the input.

[0642] 1308, the child node sends information indicating the mean of the input of the BN layer and information indicating the mean of the square of the input to the center node, as well as the related index (such as index #5).

[0643] 1309, the center node aggregates the mean of the input of the BN layer and the mean of the square of the input fed back by the child node according to the index to determine the global mean (such as global mean #2) and the global variance (global variance #2).

[0644] The center node can aggregate the mean of the input of each BN layer of each child node according to the index to obtain the global mean corresponding to each BN layer. The center node can aggregate the mean of the square of the input of each BN layer of each child node according to the index to obtain the aggregation result of the mean of the square of the input of each BN layer. The center node determines the global variance according to the global mean corresponding to each BN layer and the aggregation result of the mean of the square of the input of each BN layer.

[0645] 1310, the central node sends information indicating the global mean and information indicating the global variance to the child node, and an associated index (such as index #6).

[0646] Returning to step 1302, steps 1302 to 1310 are repeatedly executed until the global model training is completed, for example, until the global model converges. In step 1302, after receiving the global mean and the global variance, the child node can train the local model based on the global mean and the global variance. The specific description of the training process can refer to step 802 described above.

[0647] It can be understood that in some embodiments described above, the information names involved are only examples and do not limit the protection scope of the embodiments of the present application.

[0648] It can also be understood that the formulas involved in various embodiments of the present application are only exemplary and do not limit the protection scope of the embodiments of the present application. In the process of calculating the above-mentioned parameters, the calculation can also be performed according to the above-mentioned formulas, or the calculation can be performed based on the deformation of the above-mentioned formulas, or the calculation can be performed according to other ways to meet the results of formula calculation.

[0649] It can also be understood that some optional features in various embodiments of the present application can not depend on other features in some scenarios, or can be combined with other features in some scenarios, without limitation.

[0650] It can also be understood that the schemes in various embodiments of the present application can be reasonably combined, and the explanations or descriptions of various terms appearing in the embodiments can be mutually referenced or explained in various embodiments, without limitation.

[0651] It can also be understood that the size of various numerical serial numbers in various embodiments of the present application does not mean the order of execution, but is only a distinction for convenience of description, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0652] It can also be understood that the methods and operations implemented by the device in various method embodiments described above can also be implemented by components (such as chips or circuits) of the device.

[0653] Corresponding to the methods given in the above method embodiments, the embodiments of the present application also provide corresponding devices, which include modules for executing the corresponding modules of the above various method embodiments. The module can be software, hardware, or a combination of software and hardware. It can be understood that the technical features described in the above method embodiments are also applicable to the following device embodiments.

[0654] FIG. 14 is a schematic diagram of an apparatus 1900 for communication, according to an embodiment of the present application. The apparatus 1900 includes a transceiver 1910 and a processing unit 1920. The transceiver 1910 can be configured to implement corresponding communication functions. The transceiver 1910 can also be referred to as a communication interface or a communication unit, etc. The processing unit 1920 can be configured to implement corresponding processing or control functions, such as configuring resources.

[0655] Optionally, the apparatus 1900 further includes a storage unit, which can be configured to store instructions and / or data. The processing unit 1920 can read the instructions and / or data in the storage unit, so that the apparatus implements the actions of the device or network element in each of the foregoing method embodiments.

[0656] The apparatus 1900 can be a central node, or can be applied to or matched with a central node, and can implement a communication apparatus that implements the communication method performed by the central node. Alternatively, the apparatus 1900 can be a sub-node, or can be applied to or matched with a sub-node, and can implement a communication apparatus that implements the communication method performed by the sub-node.

[0657] When the apparatus 1900 is applied to a central node, the apparatus 1900 can implement the steps or processes performed by the central node in the foregoing method embodiments. The transceiver 1910 can be configured to perform the transceiving-related operations of the central node in the foregoing method embodiments, and the processing unit 1920 can be configured to perform the processing-related operations of the central node in the foregoing method embodiments.

[0658] When the apparatus 1900 is applied to a sub-node, the apparatus 1900 can implement the steps or processes performed by the sub-node in the foregoing method embodiments. The transceiver 1910 can be configured to perform the transceiving-related operations of the sub-node in the foregoing method embodiments, and the processing unit 1920 can be configured to perform the processing-related operations of the sub-node in the foregoing method embodiments.

[0659] It should be understood that the specific processes by which each unit performs the corresponding steps described above have been described in detail in the foregoing method embodiments, and thus will not be described again here for brevity.

[0660] It should also be understood that the apparatus 1900 is embodied in the form of a functional block diagram. The term "unit" herein can refer to an ASIC, an electronic circuit, a processor (for example, a shared processor, a dedicated processor, or a group processor, etc.) and a memory for executing one or more software or firmware programs, a combination of logical circuit and / or other suitable components supporting the described functions. In an alternative example, those skilled in the art can understand that the apparatus 1900 can be embodied as a child node in the above embodiments, and can be used to execute the processes and / or steps corresponding to the child node in the above method embodiments; or the apparatus 1900 can be embodied as a center node in the above embodiments, and can be used to execute the processes and / or steps corresponding to the center node in the above method embodiments, which will not be repeated here to avoid repetition.

[0661] The apparatus 1900 of each of the above schemes has the function of implementing the corresponding steps performed by the device (for example, the child node, and for example, the center node) in the above method. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions; for example, the transceiver unit can be replaced by a transceiver (for example, the transmitting unit in the transceiver unit can be replaced by a transmitter, and the receiving unit in the transceiver unit can be replaced by a receiver), and other units, such as the processing unit, can be replaced by a processor, which respectively performs the transceiving operations and related processing operations in each method embodiment.

[0662] In addition, the transceiver unit 1910 described above can also be a transceiver circuit (for example, which can include a receiving circuit and a transmitting circuit), and the processing unit 1920 can be a processing circuit. The processing circuit can include one or more processors, or a circuit for processing or control functions in one or more processors, etc.

[0663] It should be noted that the apparatus in FIG. 14 can be a network element or device in the above embodiments, or a chip or chip system, for example, a SoC. Among them, the transceiver unit can be an input / output circuit, a communication interface; the processing unit is a processor or microprocessor integrated on the chip or an integrated circuit. Here, no limitation is made.

[0664] FIG. 15 is a schematic diagram of another apparatus 2000 for communication provided by an embodiment of the present application. The apparatus 2000 includes a processor 2010, which is used to execute computer programs or instructions stored in a memory 2020, or read data / signaling stored in the memory 2020, to perform the methods in the above method embodiments. Optionally, the processor 2010 is one or more.

[0665] Optionally, as shown in FIG. 15, the apparatus 2000 further includes a memory 2020, which is configured to store computer programs or instructions and / or data. The memory 2020 can be integrated with the processor 2010, or can be separately arranged. Optionally, the memory 2020 is one or more.

[0666] Optionally, as shown in FIG. 15, the apparatus 2000 further includes a transceiver circuit 2030, which is configured to receive and / or send signals. For example, the processor 2010 is configured to control the transceiver circuit 2030 to receive and / or send signals. The processor 2010 can also be replaced by a processing circuit.

[0667] The apparatus 2000 can be a network element or device in the foregoing embodiments, or can be a chip or chip system. When the apparatus 2000 is a network element or device in the foregoing embodiments, the transceiver circuit 2030 can be a transceiver. When the apparatus 2000 is a chip or chip system, the transceiver circuit 2030 can be an interface circuit or an input / output interface.

[0668] As a kind of scheme, the apparatus 2000 can be applied to the center node, and specifically, the apparatus 2000 can be the center node, or can be the apparatus capable of supporting the center node, and the apparatus capable of supporting the center node is used to implement the function of the center node in any of the examples described above. The apparatus 2000 is used to implement the operation performed by the center node in each method embodiment described above.

[0669] For example, the processor 2010 is used to execute the computer programs or instructions stored in the memory 2020 to implement the related operation of the center node in each method embodiment described above.

[0670] As another scheme, the apparatus 2000 can be applied to the sub node, and specifically, the apparatus 2000 can be the sub node, or can be the apparatus capable of supporting the sub node, and the apparatus capable of supporting the sub node is used to implement the function of the sub node in any of the examples described above. The apparatus 2000 is used to implement the operation performed by the sub node in each method embodiment described above.

[0671] For example, the processor 2010 is used to execute the computer programs or instructions stored in the memory 2020 to implement the related operation of the sub node in each method embodiment described above.

[0672] It should be appreciated that a processor referenced in the embodiments of the present application can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), ASICs, field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0673] It should also be understood that the memory referenced in the embodiments of the present application can be a volatile memory and / or a non-volatile memory. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM). For example, the RAM can be used as an external cache. As an example but not limitation, the RAM includes the following various forms: static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM) and direct memory bus random access memory (direct rambus RAM, DR RAM).

[0674] It should be noted that when the processor is a general-purpose processor, a DSP, an ASIC, an FPGA or other programmable logic device, a discrete gate or transistor logic component, a discrete hardware component, the memory (storage module) can be integrated in the processor.

[0675] It should also be noted that the memory described herein is intended to include, but not limited to, these and any other suitable types of memory.

[0676] The embodiments of the present application further provide a computer readable storage medium, which stores computer instructions for implementing the method executed by the communication device in each of the above method embodiments.

[0677] For example, the computer program is executed by a computer, so that the computer can implement the method executed by the child node in each of the above method embodiments.

[0678] For another example, the computer program is executed by a computer, so that the computer can implement the method executed by the center node in each of the above method embodiments.

[0679] The embodiments of the present application further provide a computer program product, which contains instructions, and the instructions are executed by a computer to implement the method executed by the device (such as the child node, or the center node) in each of the above method embodiments.

[0680] The embodiments of the present application further provide a communication system, which includes the above-mentioned child node and center node. The child node and the center node can implement the method of communication shown in any of the above examples.

[0681] Optionally, the system further includes a device in communication with the above-mentioned child node and / or center node.

[0682] The explanations and beneficial effects of the related contents in any of the above-mentioned devices can refer to the corresponding method embodiments provided above, and will not be repeated here.

[0683] In several embodiments provided by the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. For example, the described embodiments of the apparatus are merely schematic, and the division of the units is merely a logical function division, and there can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0684] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable apparatus. For example, the computer can be a personal computer, a server, a network device, or the like. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media sets. The available media can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD), etc. For example, the foregoing available media includes but is not limited to: a variety of media that can store program codes such as a U disk, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk, etc.

[0685] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of communication, comprising: The method applied to a child node comprises: receiving first information, the first information comprising a first index and information indicating a first mean value, the first mean value being used for calculation of a first variance of an input of a first network layer in an artificial intelligence AI model; sending second information, the second information comprising a second index and information indicating the first variance, the first index and the second index having a correlation relationship.

2. The method of claim 1, wherein, The first index and the first mean value are related.

3. The method according to claim 1 or 2, characterized in that, Index values of the first index and the second index are the same, or the first index and the second index have a predefined or preconfigured correlation relationship.

4. The method according to any one of claims 1 to 3, characterized in that, The first index indicates an identity of the first network layer.

5. The method according to any one of claims 1 to 4, characterized in that, The first index indicates a round of training related to the first mean value.

6. The method according to any one of claims 1 to 5, characterized in that, The method further comprises: sending third information, the third information being used for requesting model parameters of the AI model.

7. The method according to any one of claims 1 to 6, characterized in that, After the second information is sent, the method further comprises: receiving fourth information, the fourth information comprising a third index and information indicating a second variance, at least one of the first index and the second index having a correlation relationship with the third index, the first mean value and the second variance being used for training of the AI model.

8. The method according to any one of claims 1 to 7, characterized in that, Before the first information is received, the method further comprises: sending fifth information, the fifth information comprising a fourth index and information indicating a second mean value, the second mean value being a mean value of an input of the first network layer in the AI model, at least one of the first index and the second index having a correlation relationship with the fourth index.

9. A method of communication, comprising: The method applied to a center node comprises: sending first information to a plurality of devices, the first information comprising a first index and information indicating a first mean value, the first mean value being used for calculation of a first variance of an input of a first network layer in an artificial intelligence AI model in each device of the plurality of devices; receiving second information from each device of the plurality of devices, the second information comprising a second index and information indicating the first variance, the first index and the second index having a correlation relationship.

10. The method of claim 9, wherein, The first index and the first mean value are related.

11. The method according to claim 9 or 10, characterized in that, Index values of the first index and the second index are the same, or the first index and the second index have a predefined or preconfigured correlation relationship.

12. The method according to any one of claims 9 to 11, characterized in that, The first index indicates an identity of the first network layer.

13. The method according to any one of claims 9 to 12, characterized in that, The first index indicates a round of training related to the first mean value.

14. The method according to any one of claims 9 to 13, characterized in that, The method further comprises: receiving third information from a first device of the plurality of devices, the third information being used for requesting model parameters of the AI model.

15. The method according to any one of claims 9 to 14, characterized in that, The method further comprises: sending fourth information to the plurality of devices, the fourth information comprising a third index and information indicating a second variance, at least one of the first index and the second index having a correlation relationship with the third index, the first mean value and the second variance being used for training of the AI model.

16. The method of claim 15, wherein, The method further comprises: determining the second variance according to the first variance indicated by the second information from each device of the plurality of devices.

17. The method of claim 16, wherein, The second variance satisfies: wherein denotes the second variance, M denotes the number of the plurality of devices, |B j denotes the number of training samples on the jth device of the plurality of devices, denotes the first variance indicated by the second information from the jth device, M is a positive integer, |B j | is a positive integer.

18. The method according to any one of claims 9 to 17, characterized in that, Before the sending the first information to the plurality of devices, the method further includes: receiving fifth information from each of the plurality of devices, the fifth information comprising a fourth index and information indicating a second mean value, the second mean value being a mean value of an input of the first network layer in the AI model of each of the plurality of devices, at least one of the first index and the second index having a correlation relationship with the fourth index.

19. The method of claim 18, wherein, Further comprising: determining the first information based on the fifth information of each of the plurality of devices, wherein the first mean value is based on the second mean value.

20. A method of communication, comprising: Applicable to a child node, the method comprising: sending seventh information, the seventh information comprising information indicating a third mean value and a fourth mean value, the third mean value being a mean value of an input of a first network layer in an artificial intelligence AI model, the fourth mean value being a mean value of a square of the input of the first network layer.

21. The method of claim 20, wherein, Further comprising: receiving eighth information, the eighth information comprising information indicating a fifth mean value and a third variance, the fifth mean value and the third variance being related to the first network layer, the fifth mean value and the third variance being used for training of the AI model.

22. The method of claim 21, wherein, The third variance is based on the third mean value and the fourth mean value.

23. The method of claim 21 or 22, wherein, The seventh information further comprises a fifth index, and the eighth information further comprises a sixth index, the fifth index and the sixth index having a correlation relationship.

24. The method of claim 23, wherein, At least one of the third mean value and the fourth mean value is related to the fifth index.

25. The method of claim 23 or 24, wherein, The index values of the fifth index and the sixth index are the same, or the fifth index and the sixth index have a pre-configured or pre-defined correlation relationship.

26. The method of any one of claims 23-25, wherein, The fifth index indicates an identity of the first network layer.

27. The method of any one of claims 23-26, wherein, The fifth index indicates a round of training related to the third mean value and / or the fourth mean value.

28. A method of communication, comprising: Applicable to a center node, the method comprising: receiving seventh information from each of the plurality of devices, the seventh information comprising information indicating a third mean value and a fourth mean value, the third mean value being an input of a first network layer in an artificial intelligence AI model of each of the plurality of devices, the fourth mean value being a mean value of a square of the input of the first network layer.

29. The method of claim 28, wherein, Further comprising: sending eighth information to the plurality of devices, the eighth information comprising information indicating a fifth mean value and a third variance, the fifth mean value and the third variance being related to the first network layer, the fifth mean value and the third variance being used for training of the AI model.

30. The method of claim 29, wherein, Further comprising: determining the third variance according to the third mean value and the fourth mean value indicated by the seventh information from each of the plurality of devices.

31. The method of claim 29 or 30, wherein, The third variance satisfies: wherein, representing the third variance, denotes a sixth mean value, which is an average of the fourth mean values indicated by the seventh information from each of the plurality of devices, E[h l (x) denotes the fifth mean value, which is an average of the third mean values indicated by the seventh information from each of the plurality of devices, B denotes a training data set on the plurality of devices.

32. The method of any one of claims 29-31, wherein, The seventh information further comprises a fifth index, and the eighth information further comprises a sixth index, the fifth index and the sixth index having a correlation relationship.

33. The method of claim 32, wherein, At least one of the third mean value and the fourth mean value is related to the fifth index.

34. The method of claim 32 or 33, wherein, The index values of the fifth index and the sixth index are the same, or the fifth index and the sixth index have a pre-configured or pre-defined correlation relationship.

35. The method of any one of claims 32-34, wherein, The fifth index indicates an identity of the first network layer.

36. The method of any one of claims 32-35, wherein, The fifth index indicates a round of training related to the third mean value and / or the fourth mean value.

37. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed by a processor, cause the method of any of claims 1-8, 9-19, 20-27, or 28-36 to be implemented.

38. A communications device, characterized by The communication device includes a processor coupled with a storage medium storing instructions that, when executed by the processor, cause the communication device to perform the method of any of claims 1-8, 9-19, 20-27, or 28-36.

39. A communications device, characterized by A module that includes performing the method of any of claims 1-8, 9-19, 20-27, or 28-36.

40. A communications device, characterized by One or more processors for processing data and / or information to cause the method of any of claims 1-8, 9-19, 20-27, or 28-36 to be implemented.

41. A chip, comprising: A processor for executing programs or instructions to cause the method of any of claims 1-8, 9-19, 20-27, or 28-36 to be implemented.

42. A computer program product, characterised in that, The computer program product includes computer program code or instructions that, when executed, cause the method of any of claims 1-8, 9-19, 20-27, or 28-36 to be implemented.

43. A communication system, characterized by A communication device that performs the method of any of claims 1-8, and a communication device that performs the method of any of claims 9-19.

44. A communication system, characterized by A communication device that performs the method of any of claims 20-27, and a communication device that performs the method of any of claims 28-36.

Images