Method for monitoring or training ai model, and communication apparatus

Abstract

Provided in the present application are a method for monitoring or training an AI model, and a communication apparatus. The method comprises: upon monitoring or training of the performance of one or more AI modules of a communication link, acquiring an input and / or output of a first module that affects inputs and / or outputs of the one or more AI modules, and monitoring or training the one or more AI modules according to the input and / or output of the first module. Since the effect of related modules of a communication link on inputs and / or outputs of one or more AI modules that are monitored or need to be trained is taken into consideration, the accuracy of monitoring of the performance of AI modules can be improved, or the reasoning performance of the AI modules can be improved.

Description

Method and communication device for monitoring or training AI models

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on December 30, 2022, with application number 202211735507.2 and application name “Method and communication device for monitoring or training AI models”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of artificial intelligence, and more specifically, to a method and communication device for monitoring or training an AI model. Background Art

[0003] Artificial intelligence (AI) has been widely applied to communication links to improve communication performance. When AI models are used to implement multiple modules in a communication link, the performance of the communication link is affected by the combined effects of these multiple AI models.

[0004] There is an urgent need for a performance monitoring or training method for AI models to improve the accuracy of performance monitoring of AI models and improve the reasoning performance of AI models.

[0005] Summary of the Invention

[0006] The present application provides a method and communication device for monitoring and / or training an AI model, in order to improve the accuracy of performance monitoring of the AI ​​model, or to improve the reasoning performance of the AI ​​model.

[0007] In a first aspect, a method for monitoring and / or training an AI model is provided, the method comprising:

[0008] The first network element obtains an input and / or output of a first module, where the first module is an AI module or a non-AI module of a communication link;

[0009] The first network element monitors and / or trains one or more first AI modules of the communication link based on the input and / or output of the first module,

[0010] The input and / or output of the one or more first AI modules is determined based on the input and / or output of the first module, or the input and / or output of the first module is determined based on the input and / or output of the one or more first AI modules.

[0011] The technical solution provided by this application monitors the performance of one or more AI modules (i.e., a first AI module) on a communication link by obtaining the input and / or output of the first module that affects the input and / or output of the one or more AI modules, and monitors and / or trains the one or more AI modules based on the input and / or output of the first module. By considering the impact of related modules on the communication link on the input and / or output of the one or more AI modules, the accuracy of AI module performance monitoring can be improved, or the reasoning performance of the AI ​​model can be enhanced.

[0012] With reference to the first aspect, in certain implementations of the first aspect, the input and / or output of the one or more first AI modules is determined based on the input and / or output of the first module, or the input and / or output of the first module is determined based on the input and / or output of the one or more first AI modules, including:

[0013] The identifiers of the inputs and / or outputs of the one or more first AI modules correspond to the identifiers of the inputs and / or outputs of the first module; or

[0014] The execution time of each of the one or more first AI modules corresponds to the execution time of the first module.

[0015] Based on this implementation, if the input and / or output of a first module is used to monitor one or more first AI modules, the input and / or output of the first module corresponds to, or is associated with, the input and / or output of each of the one or more first AI modules, or corresponds to the same communication process, thereby improving the accuracy of performance monitoring of the one or more first AI modules or enhancing the inference performance of the AI ​​model.

[0016] For details about the same communication process, please refer to the description and examples of the embodiments in the specification.

[0017] Exemplarily, the input identifier of a module of the communication link may be, for example, the identifier of the resources used when the input of the module is transmitted in the air interface (for example, the identifier of the time-frequency resources), and the output identifier of the module may be, for example, the identifier of the resources used when the output of the module is transmitted in the air interface.

[0018] In combination with the first aspect, in some implementations of the first aspect, the first module is deployed in the first network element or the second network element, and the one or more first AI modules are deployed in the first network element; or,

[0019] The first module is deployed in the first network element or the second network element, and the one or more first AI modules are deployed in the second network element; or

[0020] The first module is deployed in the first network element or the second network element. The one or more first AI modules include a first sub-AI module and a second sub-AI module. The first sub-AI module is deployed in the first network element, and the second sub-AI module is deployed in the second network element. The first sub-AI module and the second sub-AI module are used in conjunction with each other.

[0021] Based on this implementation method, the method for monitoring or training AI modules provided in this application can be applicable to a variety of scenarios, for example, the first network element monitors and / or trains the AI ​​module deployed on this end; or the first network element provides monitoring parameters for the second network element to monitor and / or train the performance of the AI ​​module deployed on the second network element; or the first network element monitors and / or trains the dual-end model, etc., which can improve the accuracy of AI model monitoring in these scenarios, or enhance the reasoning performance of the AI ​​model.

[0022] In combination with the first aspect, in some implementations of the first aspect, the first module is deployed in the second network element;

[0023] The first network element obtaining the input and / or output of the first module includes:

[0024] The first network element receives first information from the second network element, where the first information indicates an input and / or output of the first module.

[0025] Based on this implementation method, if the first module is deployed in the second network element, the first network element obtains the input and / or output of the first module through the report of the second network element, and monitors or trains one or more AI modules on this side or deployed in the second network element according to the input and / or output of the first module, so as to improve the accuracy of AI model monitoring or enhance the reasoning performance of the AI ​​model.

[0026] In conjunction with the first aspect, in some implementations of the first aspect, before the first network element receives the first information from the second network element, the method further includes:

[0027] The first network element sends first indication information to the second network element, where the first indication information indicates one or more monitoring parameters, where the one or more monitoring parameters include input and / or output of the first module.

[0028] Based on this implementation method, the first network element instructs the second network element to report the input and / or output of the corresponding module of the communication link according to the monitoring or training requirements of the AI ​​module, so as to monitor the performance of the AI ​​module or train the AI ​​model.

[0029] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes:

[0030] The first network element obtains an input and / or output of a second module, where the second module is an AI module or a non-AI module of the communication link;

[0031] The first network element sends second information to the second network element, where the second information indicates an input and / or output of the second module, where the input and / or output of the second module is determined based on the input and / or output of the second AI module of the communication link, or where the input and / or output of the second AI module of the communication link is determined based on the input and / or output of the second module.

[0032] Based on this implementation, the first network element obtains one or more monitoring parameters (e.g., including the input and / or output of the second module) and provides them to the second network element. The one or more monitoring parameters are used by the second network element to monitor or train one or more AI modules of the second network element to improve the accuracy of AI model monitoring or training.

[0033] In combination with the first aspect, in some implementations of the first aspect, before the first network element sends the second information to the second network element, the method further includes:

[0034] The first network element receives second indication information from the second network element, where the second indication information indicates one or more monitoring parameters, and the one or more monitoring parameters include input and / or output of the second module.

[0035] Based on this implementation method, the first network element provides corresponding monitoring parameters to the second network element based on the instructions of the second network element, which are used for the second network element to monitor or train the AI ​​model, and can provide on-demand feedback based on the monitoring requirements or training requirements of the second network element.

[0036] In conjunction with the first aspect, in some implementations of the first aspect, the first indication information indicates one or more monitoring parameters, including:

[0037] The first indication information indicates a first index, where the first index belongs to one of the indexes included in a predefined correspondence relationship, where the correspondence relationship indicates a correspondence between a monitoring parameter and / or a combination of monitoring parameters and an index, where the combination of monitoring parameters includes two or more monitoring parameters; or

[0038] The first indication information indicates the identifiers of the one or more monitoring parameters, where the identifiers of the one or more monitoring parameters belong to one or more identifiers in a predefined set of identifiers of monitoring parameters.

[0039] In this implementation, the first indication information may indicate a monitoring parameter (eg, an input and / or output of a non-AI module or an AI module of a communication link) through various implementations.

[0040] In combination with the first aspect, in certain implementations of the first aspect, the first indication information further includes identifiers of the one or more first AI modules, wherein the one or more monitoring parameters indicated by the first indication information are used for monitoring the one or more first AI modules.

[0041] Based on this implementation, the monitoring parameters indicated by the first indication information are associated with the identifiers of one or more first AI modules, and the monitoring parameters to be fed back indicated by the first indication information are used for monitoring or training the one or more first AI modules.

[0042] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes:

[0043] The first network element sends third indication information to the second network element;

[0044] The third indication information indicates one of the following information:

[0045] One or more values ​​of the execution time of the third module of the communication link; or,

[0046] one or more identifications of inputs to a third module of the communication link; or,

[0047] One or more identifications of outputs of a third module of the communication link.

[0048] Based on this implementation method, when the monitoring parameters that need to be fed back (for example, the input and / or output of the first module) are generated (or obtained, generated) without the interaction between the first network element and the second network element, when the first network element instructs the second network element to feed back the monitoring parameters, the first network element can indicate the time or identification information corresponding to any other monitoring parameters that belong to the same communication process as the monitoring parameters. Thus, the second network element can determine the monitoring parameters in the corresponding communication process based on the time or identification information, which can improve the accuracy of model monitoring or enhance the reasoning performance of the AI ​​model.

[0049] In conjunction with the first aspect, in some implementations of the first aspect, the first indication information indicates one or more monitoring parameters, including:

[0050] The first indication information indicates a plurality of monitoring parameters, where the plurality of monitoring parameters include a first monitoring parameter and a second monitoring parameter;

[0051] And, the third indication information indicates a first value of the execution time of the third module of the communication link, and a first value of the execution time of the fourth module of the communication link;

[0052] The first value of the execution time of the third module is used to determine the first value of the first monitoring parameter, and the first value of the execution time of the fourth module is used to determine the first value of the second monitoring parameter, and the first value of the first monitoring parameter corresponds to the first value of the second monitoring parameter;

[0053] or,

[0054] The third indication information indicates the first identifier of the input or the first identifier of the output of the third module of the communication link, and the first identifier of the input or the first identifier of the output of the fourth module of the communication link;

[0055] Among them, the first identifier of the input or the first identifier of the output of the third module is used to determine the first value of the first monitoring parameter, and the first identifier of the input or the first identifier of the output of the fourth module is used to determine the first value of the second monitoring parameter, and the first value of the first monitoring parameter corresponds to the first value of the second monitoring parameter.

[0056] Based on this implementation method, when there are multiple monitoring parameters that need to be fed back, and some monitoring parameters (for example, the first monitoring parameter) are not generated through the interaction between the first network element and the second network element, while some monitoring parameters (for example, the second monitoring parameter) need to be generated through the interaction between the first network element and the second network element, then when the first network element instructs the second network element to feed back the multiple monitoring parameters, the first network element can indicate the corresponding time of execution of a certain module before the interaction or the input and / or output identifier of the module. In addition, it is also necessary to indicate the corresponding time of execution of a certain module after the interaction or the input and / or output identifier of the module. Then, the second network element can determine the multiple monitoring parameters that belong to the same communication process as the time, input and / or output identifier, which can improve the accuracy of model monitoring or improve the reasoning performance of the AI ​​model.

[0057] In conjunction with the first aspect, in some implementations of the first aspect, the first indication information indicates one or more monitoring parameters, including:

[0058] The first indication information indicates a plurality of monitoring parameters, where the plurality of monitoring parameters include a first monitoring parameter and a second monitoring parameter;

[0059] Furthermore, the third indication information indicates a first value of the execution time of the third module of the communication link and first time information, and the first value of the execution time of the third module and the first time information indicate a first value of the execution time of the fourth module of the communication link.

[0060] Among them, the first value of the execution time of the third module is used to determine the first value of the first monitoring parameter, and the first value of the execution time of the fourth module is used to determine the first value of the second monitoring parameter, and the first value of the first monitoring parameter corresponds to the first value of the second monitoring parameter.

[0061] Based on this implementation method, when there are multiple monitoring parameters that need to be fed back, and some monitoring parameters (for example, the first monitoring parameter) are not generated through the interaction between the first network element and the second network element, while some monitoring parameters (for example, the second monitoring parameter) need to be generated through the interaction between the first network element and the second network element, then when the first network element instructs the second network element to feed back the multiple monitoring parameters, it can indicate the corresponding time of execution of a certain module before the interaction or the corresponding time of execution of a certain module after the interaction, as well as a time information (i.e., the first time information). Then, based on these time-related information, the second network element can determine the multiple monitoring parameters that belong to the same communication process as the time, which can improve the accuracy of model monitoring or improve the reasoning performance of the AI ​​model.

[0062] With reference to the first aspect, in certain implementations of the first aspect, the first time information indicates a delay, and the delay is a delay between a first value of the execution time of the fourth module and a first value of the execution time of the third module; or

[0063] The first time information indicates a time difference, where the time difference is a time difference between a first value of the execution time of the third module and a first value of the execution time of the fourth module.

[0064] Based on this implementation method, the first indication information can specifically indicate the corresponding execution time of a module (for example, module A) before the interaction and a delay information, so that the second network element can determine the corresponding execution time of the module (module A) before the interaction and the corresponding execution time of a module (for example, module B) after the interaction; or, the first indication information can specifically indicate the corresponding execution time of a module (for example, module B) after the interaction and a time difference information, so that the second network element can determine the corresponding execution time of a module (for example, module A) before the interaction and the corresponding execution time of the module (module B) after the interaction, so that the second network element can determine multiple monitoring parameters belonging to the same communication process to provide the accuracy of AI model performance monitoring or model training.

[0065] With reference to the first aspect, in certain implementations of the first aspect, the first time information indicates a time T, where T is a real number; or

[0066] The first time information indicates a time interval [T1, T2], where T1 and T2 are both real numbers.

[0067] Based on this implementation, the first time information can be a moment or a time interval. When the first time information indicates a time interval, if multiple communication processes are involved within that time interval, the second network element needs to group the multiple monitoring parameters generated by each communication process and then feed back the multiple groups of monitoring parameters generated by the multiple communication processes to the first network element for model performance monitoring or model training. With a single instruction, multiple groups of monitoring data can be obtained, reducing the feedback overhead of monitoring data.

[0068] In combination with the first aspect, in certain implementations of the first aspect, when the first time information indicates the time interval [T1, T2], there are multiple first values ​​of the second monitoring parameter, and the first value of the first monitoring parameter corresponds to the multiple first values ​​of the second monitoring parameter.

[0069] Based on this implementation, when a certain monitoring parameter corresponds to a time interval, multiple monitoring parameters may be generated within the time interval, and the multiple monitoring parameters and other associated monitoring parameters all belong to one communication process.

[0070] In a second aspect, the present application provides a communication device. In one design, the communication device may include a module for executing the method / operation / step / action described in the first aspect. The module may be a hardware circuit, software, or a combination of a hardware circuit and software. In one design, the communication device may include a processing module and a communication module. In one example, the communication device is a network device, such as an access network device. In another example, the communication device is a terminal device.

[0071] In a third aspect, the present application provides a communication device, comprising a processor configured to implement the method described in the first aspect or any implementation of the first aspect. The processor is coupled to a memory configured to store instructions and data. When the processor executes the instructions stored in the memory, the method described in the first aspect or any implementation of the first aspect can be implemented. Optionally, the communication device may further include a memory. Optionally, the communication device may further include a communication interface configured to enable the device to communicate with other devices. Exemplarily, the communication interface may be a transceiver, hardware circuit, bus, module, pin, or other type of communication interface. In one example, the communication device may be a network device, such as an access network device, or may be a device, module, or chip disposed in the network device, or may be a device capable of being used in conjunction with the network device. In another example, the communication device may be a terminal device, or may be a device, module, or chip disposed in the terminal device, or may be a device capable of being used in conjunction with the terminal device.

[0072] In a fourth aspect, the present application provides a communication system, comprising a first network element, and optionally, a second network element.

[0073] Exemplarily, the communication system includes a terminal device and an access network device, for example, the first network element is an access network device and the second network element is a terminal device; or the first network element is a terminal device and the second network element is an access network device. Alternatively, the second network element and the second network element are both network devices, or the second network element and the second network element are both terminal devices.

[0074] In a fifth aspect, the present application provides a communication system, comprising the communication device as described in the third aspect.

[0075] In a sixth aspect, the present application further provides a computer program, which, when executed on a computer, enables the computer to execute the method provided in the first aspect or any implementation of the first aspect.

[0076] In a seventh aspect, the present application also provides a computer program product, comprising instructions, which, when executed on a computer, enable the computer to execute the method provided in the above-mentioned first aspect or any implementation of the first aspect.

[0077] In an eighth aspect, the present application also provides a computer-readable storage medium, in which a computer program or instruction is stored. When the computer program or instruction is run on a computer, the computer executes the method provided in the above-mentioned first aspect or any implementation of the first aspect.

[0078] In the ninth aspect, the present application also provides a chip, which is used to read a computer program stored in a memory and execute the method provided by the above-mentioned first aspect or any implementation of the first aspect; or, the chip includes a circuit for executing the method provided by the above-mentioned first aspect or any aspect of the first aspect.

[0079] In a tenth aspect, the present application further provides a chip system, comprising a processor configured to support a device in implementing the method provided in the first aspect or any implementation of the first aspect. In one possible design, the chip system further comprises a memory configured to store programs and data necessary for the device. The chip system may consist of a chip alone, or may include a chip and other discrete components.

[0080] The technical effects of the solutions provided by any of the second to tenth aspects or any of their implementation methods can be referred to the corresponding description in the first aspect and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] FIG1 is a schematic diagram of the architecture of a communication system applicable to an embodiment of the present application.

[0082] FIG2 is a schematic diagram of the architecture of another communication system applicable to an embodiment of the present application.

[0083] FIG3 is a schematic diagram of one possible way in which AI may be applied to a communication link.

[0084] FIG4 is a schematic diagram of another possible way in which AI may be applied to a communication link.

[0085] Figure 5 is a schematic diagram of an explainable AI model.

[0086] Figure 6 is a schematic flowchart of the method for monitoring or training an AI model provided in this application.

[0087] FIG7 is a schematic diagram of a communication link.

[0088] FIG8 is an example of a monitoring or training AI module provided in this application.

[0089] FIG9 is a schematic diagram of the first network element performing performance analysis on the AI ​​module.

[0090] FIG10 is another schematic diagram of the first network element performing performance analysis on the AI ​​module.

[0091] FIG11 is another example of a monitoring or training AI module provided in this application.

[0092] FIG12 is another example of a monitoring or training AI module provided in this application.

[0093] FIG13 is a schematic diagram of a communication device 1000 provided in this application.

[0094] FIG14 is a schematic diagram of another communication device 1100 provided in this application. DETAILED DESCRIPTION

[0095] The technical solution in this application will be described below with reference to the accompanying drawings.

[0096] The technical solutions provided in this application can be applied to various communication systems, such as long-term evolution (LTE) systems, fifth-generation (5G) communication systems, worldwide interoperability for microwave access (WiMAX) or wireless local area network (WLAN) systems, satellite communication systems, or future communication systems such as 6G communication systems, or integrated systems of multiple systems. The 5G communication system can also be referred to as a new radio (NR) system.

[0097] Exemplarily, the communication system may include a terminal device and a network device.

[0098] In an embodiment of the present application, a terminal device may be an entity for receiving or transmitting signals, such as a mobile phone. The terminal device includes a handheld device with wireless connection function, other processing devices connected to a wireless modem, or a vehicle-mounted device. The terminal device may be a portable, pocket-sized, handheld, computer-built-in, or vehicle-mounted mobile device. The terminal device can be widely used in various scenarios, such as cellular communications, wireless fidelity (WiFi) systems, device-to-device (D2D), vehicle-to-everything (V2X), peer-to-peer (P2P), machine-to-machine (M2M), machine-type communication (MTC), Internet of Things (IoT), virtual reality (VR), augmented reality (AR), industrial control, autonomous driving, telemedicine, smart grid, smart furniture, smart office, smart wearable, smart transportation, smart city, drones, robots, remote sensing, passive sensing, positioning, navigation and tracking, autonomous delivery and mobility, etc.Some examples of the terminal device 120 include: user equipment (UE) of the 3GPP standard, a station (STA) in a WiFi system, a fixed device, a mobile device, a handheld device, a wearable device, a cellular phone, a smart phone, a session initialization protocol (SIP) phone, a laptop, a personal computer, a smart book, a vehicle, a satellite, a global positioning system (GPS) device, a target tracking device, a drone, a helicopter, an aircraft, a ship, a remote control device, a smart home device, an industrial device, a personal communication service (PCS) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), a wireless network camera, a tablet computer, a handheld computer, a mobile internet device (MID), a wearable device such as a smart watch, a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a terminal in a vehicle network system, a wireless terminal in a self-driving car, a smart grid, and the like. The terminal device may be a wireless terminal in the above scenarios or a device provided in a wireless device, such as a communication module, a modem or a chip in the above device. The terminal device may also be referred to as a terminal, UE, mobile station (MS), mobile terminal (MT), etc. The terminal device may also be a terminal device in a future wireless communication system. In addition, the terminal device may also include a location reference device, such as an automated guided vehicle (AGV) or a device with similar functions. For the sake of convenience, the terminal device will be described below using UE as an example.

[0099] In this application, a communication device for implementing the functions of a terminal device may be a terminal device, a terminal device having some of the functions of the above communication device, or a device capable of supporting the functions of the above terminal device, such as a chip system, which may be installed in the terminal device or used in conjunction with the terminal device. In this application, a chip system may be composed of a chip or may include a chip and other discrete devices.

[0100] A network device may be a device that provides wireless communication function services, can communicate with a terminal device, and is usually located on the network side. The network device may be referred to as an access network device or a wireless access network device. For example, the network device may be a base station. For example, the access network device in the embodiment of the present application includes but is not limited to a next-generation base station (gNodeB, gNB) in a 5G communication system, a base station in a sixth-generation (6G) mobile communication system, a base station in a future mobile communication system, an access point (AP) in a WiFi system, an evolved node B (eNB) in an LTE system, a radio network controller (RNC), a node B (NB), a base station controller (BSC), a home base station (e.g., home evolved NodeB, or home Node B, HNB), a base band unit (BBU), a transmission reception point (TRP), a transmitting point (TP), a base transceiver station (BTS), a satellite, a drone, and the like. In one network structure, a network device may include a centralized unit (CU) node, or a distributed unit (DU) node, or a RAN device including a CU node and a DU node, or a RAN device including a control plane CP node, a user plane CU node, and a DU node. Alternatively, the network device may also be a wireless controller, relay station, vehicle-mounted device, or wearable device in a cloud radio access network (CRAN) scenario. In addition, a base station may be a macro base station, a micro base station, a relay node, a donor node, or a combination thereof. A base station may also refer to a communication module, modem, or chip used to be set in the aforementioned device or apparatus. A base station may also be a mobile switching center and a device that performs base station functions in device-to-device (D2D), vehicle-to-everything (V2X), or machine-to-machine (M2M) communications, a network-side device in a 6G network, or a device that performs base station functions in future communication systems. A base station may support networks with the same or different access technologies, without limitation. A base station may be fixed or mobile.For example, a helicopter or drone can be configured to act as a mobile base station, and one or more cells can move according to the location of the mobile base station. In other examples, a helicopter or drone can be configured to act as a device that communicates with another base station.

[0101] In the present application, the device for implementing the functions of the above-mentioned network device can be an access network device, or a network device with some of the functions of the access network, or a device capable of supporting the implementation of the access network functions, such as a chip system, a hardware circuit, a software module, or a hardware circuit and a software module. The device can be installed in the access network device or used in combination with the access network device. In the method of the present application, the communication device for implementing the functions of the access network device is described as an access network device.

[0102] Optionally, the communication system also includes at least one AI node.

[0103] Optionally, the AI ​​node can be deployed in one or more of the following locations in the communication system: access network equipment, terminal equipment, or core network equipment. Alternatively, the AI ​​node can be deployed separately, for example, in a location other than any of the above devices, such as a host or cloud server in an over-the-top (OTT) system. The AI ​​node can communicate with other devices in the communication system, such as one or more of the following: network equipment, terminal equipment, or core network elements.

[0104] Optionally, the AI ​​node is used to perform AI-related operations. As an example, the AI-related operations may include: one or more of: model failure testing, model performance testing, model training, model reasoning, or data collection.

[0105] For example, a network device may forward data related to an AI model reported by a terminal device to an AI node, which may then execute AI-related operations. For another example, a network device or a terminal device may forward data related to an AI model to an AI node, which may then execute AI-related operations. For another example, an AI node may transmit the output of an AI-related operation, such as one or more of a trained neural network model, model evaluation, or test results, to the network device and / or the terminal device. For example, the AI ​​node may directly transmit the output of an AI-related operation to the network device and the terminal device. For another example, the AI ​​node may transmit the output of an AI-related operation to the terminal device via the network device. For another example, the AI ​​node may transmit the output of an AI-related operation to the network device via the terminal device. It is understood that when the AI ​​node is located within a network device or terminal device, the transmission herein may be understood as transmission between modules within the network device or terminal device.

[0106] It is understood that this application does not limit the number of AI nodes. For example, when there are multiple AI nodes, the multiple AI nodes can be divided based on function, such as different AI nodes are responsible for different functions.

[0107] It can also be understood that AI nodes can be independent devices, or they can be integrated into the same device to implement different functions, or they can be network elements in hardware devices, or they can be software functions running on dedicated hardware, or they can be virtualized functions instantiated on a platform (for example, a cloud platform). This application does not limit the specific form of the above-mentioned AI nodes.

[0108] In addition, the network device in this application, or all or part of the terminal device, can be dedicated hardware, software functions running on dedicated hardware, software functions running on general-purpose hardware, or virtualization functions instantiated on a platform (for example, a cloud platform). This application does not limit the specific form of the above-mentioned network device or terminal device.

[0109] To facilitate understanding, the following is a brief introduction to the relevant technologies or concepts involved in this application.

[0110] AI model: an algorithm or computer program that can realize AI functions. The AI ​​model characterizes the mapping relationship between the input and output of the model, or the AI ​​model refers to a function model that maps input of a certain dimension to output of a certain dimension, and its model parameters are obtained through machine learning training. For example, f(x)=ax2+b is a quadratic function model, which can be regarded as an AI model. a and b are parameters of the AI ​​model, which can be obtained through machine learning training. Exemplarily, the AI ​​models mentioned in the embodiments below of this application are not limited to neural networks, linear regression models, decision tree models, support vector machines (SVM), Bayesian networks, Q learning models or other machine learning (ML) models.

[0111] Training dataset: Data used for model training, validation, and testing in machine learning. The quantity and quality of data will affect the effectiveness of machine learning. Training data can include the input of the AI ​​model, or the input and target output of the AI ​​model. The target output is the target value of the AI ​​model's output, which can also be called the true output value, output truth value, label, or label sample.

[0112] Model training: The process of selecting a suitable loss function and using an optimization algorithm to train the model parameters so that the value of the loss function is less than the threshold, or the value of the loss function meets the target requirements.

[0113] AI model design mainly includes the data collection phase (for example, collecting training data and / or inference data), the model training phase, and the model inference phase. It can also include the application phase of inference results. In the aforementioned data collection phase, the data source is used to provide the training data set and inference data. In the model training phase, the AI ​​model is obtained by analyzing or training the training data provided by the data source. The AI ​​model represents the mapping relationship between the model input and output. Learning the AI ​​model through the model training node is equivalent to learning the mapping relationship between the model input and output using the training data. In the model inference phase, the AI ​​model trained in the model training phase is used to perform inference based on the inference data provided by the data source to obtain the inference result. This phase can also be understood as: inputting the inference data into the AI ​​model and obtaining the output through the AI ​​model, which is the inference result. The inference result can indicate: the configuration parameters used (executed) by the execution object and / or the operation performed by the execution object. Inference results are published during the application phase. For example, inference results can be centrally planned by an actor entity. For example, the actor entity can send the inference results to one or more execution targets (e.g., core network equipment, access network equipment, or terminal devices) for execution. Furthermore, the actor entity can provide feedback on model performance to the data source to facilitate subsequent model updates and training.

[0114] Loss function: It is used to measure the difference or gap between the model's predicted value and the true value.

[0115] Model application: Use the trained model to solve practical problems.

[0116] It is understood that the AI ​​model can be implemented as a hardware circuit, software, or a combination of software and hardware, without limitation. Non-limiting examples of software include: program code, program, subroutine, instruction, instruction set, code, code segment, software module, application, or software application.

[0117] In the embodiments of the present application, "indication" may include direct indication, indirect indication, explicit indication, and implicit indication. When describing a certain indication information as indicating A, it can be understood that the indication information carries A, and it can directly indicate A or indirectly indicate A. Indirect indication can refer to the indication information directly indicating B, as well as the correspondence between B and A, so as to achieve the purpose of indicating A through the indication information. The correspondence between B and A can be predefined by the protocol, pre-stored, or obtained through configuration between network elements.

[0118] The wireless communication system applicable to this application may include one or more network-side devices and one or more terminal devices. The network-side device may include the aforementioned access network device, and optionally, may also include core network devices, without limitation. The device may also be replaced by a network element, entity, network entity, communication device, communication module, node, communication node, etc. This application uses a device or network element as an example for description.

[0119] Referring to Figure 1 , Figure 1 is a schematic diagram of the architecture of a communication system applicable to an embodiment of the present application. Exemplarily, the communication system includes a network device 110, a terminal device 120, and a terminal device 130. Terminal devices 120 and 130 can access network device 110 and communicate with network device 110. Optionally, network device 110 can be an access network device. Exemplarily, the communication system may also include an AI entity, which is located within network device 110, i.e., a module of network device 110, not shown in Figure 1 .

[0120] Referring to FIG2 , FIG2 is a schematic diagram of the architecture of another communication system applicable to an embodiment of the present application. The communication system includes a network device 110, a terminal device 120, a terminal device 130, and an AI entity 140. The network device 110 can forward the data related to the AI ​​model reported by the terminal device to the AI ​​entity 140, and the AI ​​entity 140 performs AI-related operations such as training data set construction and model training, and provides one or more of the outputs of the AI-related operations, such as the trained AI model, model evaluation, and test results, to the network device 110.

[0121] It should be understood that Figures 1 and 2 are only schematic diagrams, and the communication system may also include other devices, such as core network devices, wireless relay devices, wireless backhaul devices, and / or devices for implementing artificial intelligence functions, etc., which are not all drawn in Figures 1 and 2. In addition, in actual applications, the wireless communication system may include multiple network devices (such as access network devices, core network devices, etc.) at the same time, and may also include multiple terminal devices at the same time, without limitation. An access network device can serve one or more terminal devices at the same time. A terminal device can also access one or more access network devices at the same time. The embodiment of the present application does not limit the number of terminal devices and network devices included in the wireless communication system. The number of devices shown in Figures 1 and 2 is only an example, and each device is not limited to one or more, and is not limited, depending on the specific technical solution.

[0122] Possible ways to apply AI to communication links to improve communication performance include: replacing some or all modules in the communication link with AI models while retaining the physical meaning of different modules in the communication link, as shown in Figure 3. Or replacing some or all modules in the communication link with a single AI model without retaining the physical meaning of different modules in the communication link, as shown in Figure 4.

[0123] As shown in Figure 3, the transmitter and receiver are composed of multiple modules, each of which retains the same physical meaning as in traditional communication links, such as channel compression, channel recovery, and precoding. The difference between Figure 3 and traditional communication links is that some or all of the modules in Figure 3 are implemented using AI models. For example, the channel compression module is based on an AI model, and the precoding calculation module is based on an AI model. To ensure the performance of the communication link, the performance of the AI ​​models in different modules can be monitored, allowing for timely model switching or updates.

[0124] The receiving end in Figure 4 is implemented by an AI model, and the specific implementation process is no longer divided into the multiple modules shown in Figure 3. However, black-box AI cannot explain the reasons for its output, and therefore its reliability cannot be verified. Therefore, to ensure the reliability of the AI ​​model, an explainable AI model can be used. An explainable AI model requires that the AI ​​model can also output a set of intermediate variables with physical meaning. Applying this set of intermediate variables to the various modules of a traditional communication link can produce the AI ​​model's output. Therefore, this set of intermediate variables can explain why the black-box AI obtains its output, as shown in Figure 5. Figure 5 is a schematic diagram of an explainable AI model. The AI ​​model in Figure 4 is equivalent to being composed of the multiple virtual modules within the dashed box in Figure 5.

[0125] Intermediate variables that can explain the output of AI models can also be used to monitor the performance of AI models.

[0126] When an AI model in a communication link is used to implement multiple modules in the communication link (including the virtual module in Figure 5), the performance of the communication link is affected by the combined effects of these multiple AI models, and the performance of each AI model is affected by the performance of the remaining modules. This poses challenges to AI model performance monitoring or training. Taking AI model performance monitoring as an example, known model monitoring solutions independently monitor each AI model in the communication link. For example, direct or indirect performance indicators are defined for the AI ​​model, and model performance is monitored based on the values ​​of the defined performance indicators. Direct performance indicators are metrics that can be calculated based on model inputs or outputs or data labels to represent model accuracy, while indirect performance indicators include metrics that represent communication performance when the AI ​​model is applied to the communication link. However, direct performance indicators are difficult to calculate for AI models with difficult data labels. Indirect performance indicators, on the other hand, are difficult to use directly to assess model performance, as the performance of the communication link is affected by multiple AI models. Practice has shown that independently monitoring or training each AI model is not conducive to accurately assessing communication link performance, hindering performance improvement of the communication link.

[0127] The technical solution provided by this application is introduced below.

[0128] See Figure 6, which is a schematic flowchart of the method for monitoring or training an AI model provided in this application.

[0129] 210. A first network element obtains an input and / or output of a first module, where the first module is an AI module or a non-AI module of a communication link.

[0130] In this application, a communication link may refer to a link between a transmitter and a receiver of signals and / or data.

[0131] Optionally, the first network element may be one of the transmitting end and the receiving end of the communication link. Alternatively, the first network element may be an AI entity corresponding to the transmitting end or the receiving end. For example, the AI ​​entity corresponding to the transmitting end may be deployed separately from the transmitting end, or may be a module within the transmitting end. Similarly, the AI ​​entity corresponding to the receiving end may be deployed separately from the receiving end, or may be a module within the receiving end, without limitation.

[0132] Optionally, the first module is not limited to one or more. The embodiments of this application are described using a single first module as an example. Those skilled in the art can understand the embodiments of multiple first modules by referring to the description of a single first module. Taking a single first module as an example, the first module can be any module of a communication link. Furthermore, the first module can be an AI module or a non-AI module.

[0133] 220. The first network element monitors and / or trains one or more first AI modules of the communication link based on the input and / or output of the first module.

[0134] The input and / or output of the one or more first AI modules is determined based on the input and / or output of the first module, or the input and / or output of the first module is determined based on the input and / or output of the one or more first AI modules.

[0135] Optionally, the input and / or output of each of the one or more first AI modules is determined based on the input and / or output of the first module, or the input and / or output of the first module is determined based on the input and / or output of the one or more first AI modules, including:

[0136] The identifiers of the inputs and / or outputs of the one or more first AI modules correspond to the identifiers of the inputs and / or outputs of the first module; or,

[0137] The execution time values ​​of the one or more first AI modules correspond to the execution time value of the first module.

[0138] It should be understood that the above-mentioned “multiple first AI modules” refer to multiple AI modules of the communication link, and each of the multiple AI modules is referred to as a first AI module.

[0139] Furthermore, the correspondence between the identifiers of the inputs and / or outputs of one or more first AI modules and the identifiers of the inputs and / or outputs of the first module, or the correspondence between the execution time values ​​of one or more first AI modules and the execution time value of the first module, may indicate that the inputs and / or outputs of the one or more first AI modules and the inputs and / or outputs of the first module belong to the same communication process. For example, based on the connection relationship between the modules included in the communication link, the inputs and / or outputs of each module generated during the sequential execution of these modules correspond based on the communication process. For example, if the output of the first module at time t1 is the input of AI module a at time t2, then the output of the first module at time t1 corresponds to the input of AI module a at time t2, or in other words, the output of the first module at time t1 and the input of AI module a at time t2 belong to the same communication process. As another example, based on the connection relationship between the inputs and / or outputs of each module in the communication link, the input y and / or output z of AI module a is affected by the output x of the first module. AI module a is an example of an AI module in the communication link, and the first module is an example of any module (AI module or non-AI module) in the communication link. Furthermore, AI module a and the first module may be adjacent or non-adjacent in terms of their connection relationship. Therefore, the input y and / or output z of AI module a corresponds to the output x of the first module, or in other words, they belong to the same communication process. Here, x is the identifier of an output of the first module, y is the identifier of an input of AI module a, and z is the identifier of an output of AI module a. The identifiers of the inputs and / or outputs of these modules may also be represented in other ways, without limitation. In addition, optionally, taking the first module as an example, the identifier of the input of the first module may include a resource identifier used by the input of the first module to be transmitted in the air interface, and the resource identifier includes one or more of the following: an identifier of a time domain resource position, an identifier of a frequency domain resource position, an identifier of a time-frequency resource position, an identifier of a spatial domain resource position, such as an antenna port or beam identifier; the identifier of the output of the first module may include a resource identifier used by the output of the first module to be transmitted in the air interface, and the resource identifier includes one or more of the following: an identifier of a time domain resource position, an identifier of a frequency domain resource position, an identifier of a time-frequency resource position, an identifier of a spatial domain resource position, such as an antenna port or beam identifier. Exemplarily, the time-frequency resource positions used by different inputs (or outputs) of the first module to be transmitted in the air interface are generally different, and thus can be used to identify different inputs (or outputs) of the first module.

[0140] Optionally, the correspondence between the identifiers of the inputs and / or outputs of one or more first AI modules and the identifiers of the inputs and / or outputs of the first module, or the correspondence between the execution time values ​​of one or more first AI modules and the execution time value of the first module, may mean that the inputs and / or outputs of the one or more first AI modules and the inputs and / or outputs of the first module belong to the same inference process.

[0141] For example, taking the UE performing CSI prediction as an example, the UE feeds back information used for CSI prediction (such as channel estimation results) and historically predicted CSI to the base station. The information used for CSI prediction is the output of the channel estimation module on the UE side, and the historically predicted CSI is the output of the CSI prediction module on the UE side. The base station receives the information fed back by the UE and uses this information as the input of the precoding module to obtain the output of the precoding module. There may be multiple pieces of information fed back by the UE for CSI prediction, and there may also be multiple pieces of historically predicted CSI. These multiple pieces of information used for CSI prediction and these multiple pieces of historically predicted CSI may belong to multiple different communication processes, but they are all used as inputs to the precoding module. Therefore, the multiple outputs of the channel estimation module on the UE side and the multiple outputs of the CSI prediction module belong to the same inference process as the output of the precoding module on the base station side.

[0142] Based on the description of the first module and the first AI module, it can be seen that in an embodiment of the present application, one or more first AI modules of the communication link are monitored and / or trained based on the input and / or output of one or more first modules.

[0143] The technical solution provided by this application monitors the performance of one or more AI modules (i.e., a first AI module) on a communication link by obtaining the input and / or output of the first module that affects the input and / or output of the one or more AI modules, and monitoring and / or training the one or more AI modules based on the input and / or output of the first module. By considering the influence of related modules on the communication link on the input and / or output of the one or more AI modules, the accuracy of AI module performance monitoring or model training can be improved.

[0144] In this application, monitoring parameters can be used for performance monitoring of AI models and / or for training of AI models, without limitation. In some embodiments below, model monitoring is only described as an example. It should be understood that in each embodiment of monitoring the AI ​​model according to the monitoring parameters, it can also be replaced by training the AI ​​model according to the monitoring parameters. Since the impact of the relevant modules on the communication link on the input and / or output of the AI ​​module to be monitored or trained is taken into account when performing model monitoring or model training, the accuracy of the performance monitoring or model training of the AI ​​module can be improved.

[0145] Optionally, taking one of the transmitting end and the receiving end of the communication link as the first network element and the other as the second network element as an example, the technical solution provided in this application can be applied to AI model performance monitoring or model training in a variety of scenarios. For example, the first network element obtains the input and / or output of the first module, and monitors and / or trains the AI ​​module deployed on the second network element based on the input and / or output of the first module, or monitors and / or trains the AI ​​module deployed on the first network element, or the first network element monitors and / or trains the dual-end model deployed on the first network element and the second network element. Optionally, the first module may be deployed on the first network element or the second network element. Alternatively, the first network element can also obtain the input and / or output of the first module deployed on the first network element, and provide it to the second network element for monitoring and / or training of the AI ​​module of the second network element.

[0146] For clarity in describing the solutions, an example of a communication link is given below in conjunction with FIG7 , and these technical solutions are explained based on this example.

[0147] Refer to Figure 7, which is a schematic diagram of a communication link. As shown in Figure 7, the communication link includes a transmitter and a receiver. The transmitter is an example of a first network element, and the receiver is an example of a second network element. The transmitter side is deployed with a modulation module, a precoding module, a transmission module, and a channel recovery module. The receiver side is deployed with a receiving module, an equalization module, and a demodulation module. In addition, the receiver is also deployed with a channel estimation module and a channel compression module. As described above, these modules deployed on the transmitter side and the receiver side can all or some of them be AI modules, without limitation.

[0148] Examples 1, 2, and 3 are given below to illustrate the technical solution of this application.

[0149] Example 1

[0150] The first network element obtains the input and / or output of the first module to monitor and / or train the AI ​​module of the first network element. Optionally, the first module is deployed in the first network element or the second network element.

[0151] See Figure 8, which is an example of a monitoring or training AI module provided in this application.

[0152] 301. Optionally, the first network element determines to monitor and / or train one or more AI modules.

[0153] 302. The first network element obtains input and / or output of the first module.

[0154] In a possible case, the first module is deployed in a first network element, and the first network element obtains the input and / or output of the first module.

[0155] The input and / or output of the one or more AI modules are determined based on the input and / or output of the first module, or the input and / or output of the first module are determined based on the input and / or output of the one or more AI modules.

[0156] In another possible scenario, the first module is deployed in a second network element, and the first network element can obtain the input and / or output of the first module from the second network element. For example, the first network element instructs the second network element to provide the input and / or output of the first module, and receives the input and / or output of the first module sent by the second network element.

[0157] Specifically, the first network element sends first indication information to the second network element, where the first indication information indicates one or more monitoring parameters, where the one or more monitoring parameters include input and / or output of the first module.

[0158] Exemplarily, the first indication information may indicate one or more monitoring parameters in the following manner 1 or manner 2.

[0159] Method 1

[0160] The first indication information indicates an index (e.g., referred to as a first index), where the first index belongs to one of the indexes included in the predefined correspondence relationship, where the correspondence relationship indicates a correspondence between a monitoring parameter and / or a combination of monitoring parameters and the index. The combination of monitoring parameters includes two or more monitoring parameters.

[0161] The corresponding relationship is explained using Table 1 as an example.

[0162] Table 1

[0163] Taking the communication link shown in Figure 7, where the first network element is a base station and the second network element is a UE as an example, for the performance monitoring of the AI ​​module of the communication link between the base station and the UE, multiple indexes can be predefined. For example, for the monitoring of the precoding module, index 0 can be defined, corresponding to the received signal-to-interference ratio, which can be used for coarse monitoring of the precoding module; index 1 can also be defined, corresponding to the channel estimation result and the received signal-to-interference ratio, for fine monitoring of the precoding module. Furthermore, the base station can send the corresponding index to the UE according to the monitoring requirements. Optionally, the base station can also send the identifier of the AI ​​module to be monitored, for example, the identifier of the precoding module, to indicate that the monitoring parameters corresponding to the index are used for monitoring the AI ​​module, without limitation.

[0164] In addition, as an example, the index can also be associated with the ID of a specific AI module, indicating that the index is used for monitoring the specific AI module. For example, index 0 is associated with the ID of the precoding module, indicating that index 0 is used for monitoring the precoding module; for another example, index 1 is associated with the ID of the precoding module, indicating that index 1 is used for monitoring the precoding module. As another example, the index may not be associated with the ID of a specific AI module. For example, index 0 is defined to correspond to the received signal-to-interference ratio, which can be used for coarse monitoring of the precoding module, channel compression module or channel recovery module; index 1 is defined to correspond to the channel estimation result and the received signal-to-interference ratio, which can be used for fine monitoring of the precoding module, channel compression module or channel recovery module; index 2 is defined to correspond to the demodulation accuracy, channel estimation result and received signal-to-interference ratio; index 3 is defined to correspond to the demodulation accuracy, the input of the demodulation module, the channel estimation result and the received signal-to-interference ratio.

[0165] Method 2

[0166] The first indication information indicates the identifiers of the one or more monitoring parameters, where the identifiers of the one or more monitoring parameters belong to one or more identifiers of a predefined set of monitoring parameter identifiers, wherein the one or more monitoring parameters include inputs and / or outputs of the first module.

[0167] Taking Figure 7 as an example, a predetermined set of monitoring parameters may include inputs and / or outputs of multiple modules (AI modules or non-AI modules) of the communication link, such as one or more of the following: the modulated bit, the modulation symbol, the received signal-to-interference ratio (SIR) at the receiving end, the log-likelihood information of the symbol to be demodulated, the demodulation information, the channel estimation result, the compression information of the channel estimation result, and the channel recovery information. Corresponding identifiers are defined for each of these monitoring parameters. The first indication information indicates the monitoring parameter corresponding to the identifier by indicating the corresponding identifier. For example, assuming that the above-mentioned monitoring parameters and their corresponding identifiers are: the received signal-to-interference ratio (A) at the receiving end, the log-likelihood information of the symbol to be demodulated (B), the demodulation information (C), the channel estimation result (D), the compression information (E), and the channel recovery information (F), if the first network element needs to instruct the second network element to feedback the received signal-to-interference ratio, the channel estimation result, and the compression information, then the first indication information indicates identifiers A, D, and E. If the first network element requires the second network element to feedback the received signal-to-interference ratio, then the first indication information indicates identifier A.

[0168] Optionally, for the communication link shown in FIG7 , the internal implementation of different modules can be implemented by different manufacturers (including the manufacturer's own optimized algorithms, or AI models), and is not mandatory to disclose. Therefore, the module performance can be evaluated based on the input and / or output of different modules, or the results of processing the input and / or output of different modules. According to the connection relationship between the modules of the communication link as shown in FIG7 , a set of monitoring parameters (or intermediate variables) that can be used for model monitoring can be determined. Among them, the monitoring parameters may include the following types: the input and / or output of a module of the communication link (AI or non-AI implementation); the result of processing the input and / or output of a module of the communication link.

[0169] It should be understood that although the monitoring parameters are used to monitor the AI ​​module, the monitoring parameters can include both the input and / or output of the AI ​​module and the input and / or output of the non-AI module. Taking Figure 7 as an example, it is assumed that the channel compression module, channel recovery module and precoding module adopt the AI ​​model, and the remaining modules such as the channel estimation module and the receiving module adopt traditional algorithms. Among them, the recovery accuracy of the channel recovery module and the performance of the precoding module will be affected by the channel estimation, so the input / output of the channel estimation module can also be used for the performance evaluation of the AI ​​module. In addition, the monitoring parameters may also include the results of the processing of the input / output of one or more modules. For example, the output of the receiving module can be used to estimate the received signal-to-interference ratio of the receiving end, which can be used for the performance evaluation of the precoding module; for example, the correlation between the output of the channel recovery module and the input of the channel compression module is calculated, which can be used for the evaluation of channel compression and recovery accuracy.

[0170] Optionally, the above-mentioned set of monitoring parameters that can be used for AI module monitoring can be predefined, preconfigured or pre-stored by the protocol, without limitation.

[0171] When the first network element determines to monitor the performance of one or more AI modules of the communication link, optionally, if the monitoring parameter is generated by the second network element (that is, the monitoring parameter corresponds to the input and / or output of a module, and the module is deployed on the second network element), the first network element can instruct the second network element to feedback the monitoring parameter. Specifically, the above-mentioned method 1 or method 2 can be used to indicate the monitoring parameter through the first indication information. For example, taking the first network element as a base station and the second network element as a UE as an example, for the precoding module, the data label (that is, the optimal precoding matrix) is usually not available, so the most direct performance indicator is the receive end signal-to-interference ratio (or receive end signal-to-interference-noise ratio). However, the level of the receive end signal-to-interference ratio is also affected by the UE side channel compression and the base station side channel recovery accuracy. Therefore, monitoring the performance of the precoding module based only on the receive end signal-to-interference ratio cannot exclude the influence of the channel estimation, compression and recovery modules. Based on the above analysis, for monitoring the precoding module, the base station can instruct the UE to feedback multiple monitoring parameters, including channel estimation results and receive signal-to-interference ratio.

[0172] In this embodiment of the present application, to effectively evaluate the performance of the AI ​​module, the multiple monitoring parameters to be fed back must be generated during the same communication process. Therefore, when the first network element instructs the second network element to feed back the monitoring parameters, it must indicate the communication process, and the second network element then feeds back the monitoring parameters generated during the communication process.

[0173] Taking the UE-side feedback channel estimation results and received signal-to-interference ratio for monitoring the precoding module as an example, a communication process includes: the UE inputs the channel estimation results into the channel compression module to generate compressed information, and then the compressed information is fed back to the base station and input into the channel recovery module to obtain the channel recovery information and use it for precoding calculation, and then the precoding is applied to downlink transmission. The receiving end can calculate the received signal-to-interference ratio when applying the precoding based on the received signal.

[0174] Optionally, the specific manner in which the first network element indicates the communication process may vary depending on whether the generation of the monitoring parameter to be fed back involves interaction between the second network element and the first network element.

[0175] (1) The generation of the monitoring parameters to be fed back does not involve the interaction between the second network element and the first network element.

[0176] Exemplarily, if the first network element instructs the second network element to feedback the input and / or output of the first module, that is, the monitoring parameter is specifically the input and / or output of the first module, the first network element indicates the specific communication process through the third indication information. Optionally, the third indication information includes one or more of the following:

[0177] The third indication information indicates one or more values ​​of the execution time of the third module of the communication link; or,

[0178] The third indication information indicates one or more identifiers of inputs of the third module of the communication link; or,

[0179] The third indication information indicates one or more identifiers of outputs of the third module of the communication link.

[0180] Taking the communication link between the base station and the UE shown in Figure 7 as an example, for multiple modules running on the UE side (excluding interactions with the base station), the UE only needs to cache the multiple monitoring parameters in sequence according to the connection relationship between the modules shown in Figure 7 to ensure that the multiple monitoring parameters are generated in the same communication process. For example, in Figure 7, when the UE runs the channel estimation module and the channel compression module in sequence, the channel estimation results and compression information generated belong to the same communication process. Therefore, the base station only needs to indicate the identifier or time information corresponding to the value of a certain monitoring parameter, and the UE can determine the values ​​of the remaining monitoring parameters that belong to the same communication process as the value of the monitoring parameter.

[0181] For example, the base station indicates the time corresponding to the execution of a module (such as module A, an example of the third module mentioned above) during the operation process on the UE side, or the ID of a certain output of the module (such as module A), so that the UE can determine the monitoring parameters belonging to the same communication process as the time or the output ID, such as the input and / or output of the first module. For example, when the base station instructs the UE to feedback the channel estimation result for performance monitoring of the channel recovery module, the base station can indicate the feedback time of the compressed information corresponding to the monitoring parameter to be fed back by the UE, and then the UE can determine the channel estimation result belonging to the same communication process as the feedback time of the compressed information, and feed it back to the base station for performance monitoring of the channel recovery module. When the base station indicates N time information, the UE feeds back the monitoring parameter corresponding to each of the N time information. For example, if the base station instructs the UE to feedback the channel estimation result, the base station can indicate N feedback times of the compressed information, and then the UE needs to feed back the channel estimation results corresponding to each of the N feedback times to the base station, that is, feed back N channel estimation results, and the N channel estimation results correspond to the N feedback times, where N is a positive integer.

[0182] Optionally, the first network element may instruct the UE to feedback multiple sets of monitoring parameters within a time period. For example, if the time period involves multiple communication processes, the UE needs to group the multiple monitoring parameters generated by each communication process into one group, and then feedback the multiple sets of monitoring parameters generated by the multiple communication processes to the first network element.

[0183] (2) The generation of the monitoring parameters to be fed back involves the interaction between the second network element and the first network element.

[0184] For example, if the first network element instructs the second network element to provide feedback on the input and / or output of the first module, that is, the monitoring parameter to be fed back is the input and / or output of the first module, the first network element indicates the specific communication process through third indication information. The third indication information can be implemented in the following two ways: a) or b).

[0185] a) Optionally, as a specific implementation, the third indication information includes one or more of the following:

[0186] The third indication information indicates a first value of the execution time of the third module and the first value of the execution time of the fourth module of the communication link, wherein the first value of the execution time of the third module is used to determine the first value of the first monitoring parameter, and the first value of the execution time of the fourth module is used to determine the first value of the second monitoring parameter, and the first value of the first monitoring parameter corresponds to the first value of the second monitoring parameter; or,

[0187] The third indication information indicates the first identifier of the input or the first identifier of the output of the third module of the communication link, and the first identifier of the input or the first identifier of the output of the fourth module, wherein the first identifier of the input or the first identifier of the output of the third module is used to determine the first value of the first monitoring parameter, and the first identifier of the input or the first identifier of the output of the fourth module is used to determine the first value of the second monitoring parameter, and the first value of the first monitoring parameter corresponds to the first value of the second monitoring parameter.

[0188] The first value of the execution time of the third module can be understood as the value of the execution time of a module (for example, module A) of the communication link before the first network element and the second network element interact. The first value of the execution time of the fourth module can be understood as the value of the execution time of a module (for example, module B) of the communication link after the first network element and the second network element interact.

[0189] Similarly, the first identifier of the input or the first identifier of the output of the third module can be understood as the identifier of the input or output of a module in the communication link before the first network element and the second network element interact. The first identifier of the input or the first identifier of the output of the fourth module can be understood as the identifier of the input or output of a module in the communication link after the first network element and the second network element interact.

[0190] The second network element is based on the third indication information. If the value of the monitoring parameter to be fed back is generated before the interaction, the value of the monitoring parameter can be determined according to the first value of the execution time of the third module, or the first identifier of the input or the first identifier of the output of the third module; if the value of the monitoring parameter to be fed back is generated after the interaction, the value of the monitoring parameter can be determined according to the first value of the execution time of the fourth module, or the first identifier of the input or the first identifier of the output of the fourth module.

[0191] It should be understood that the first monitoring parameter is an example of the monitoring parameter to be fed back, and the value of the first monitoring parameter is generated before the interaction. The second monitoring parameter is an example of the monitoring parameter to be fed back, and the value of the second monitoring parameter is generated after the interaction. It should be noted that in this application, "monitoring parameter" and "value of monitoring parameter" are sometimes used interchangeably, and should not impose any limitations on the understanding of the solution. For example, the monitoring parameter A corresponding to a certain communication process can also be understood as the specific value of the monitoring parameter A in the communication process. Alternatively, it can also be understood that after the connection relationship between the modules of the communication link is determined, if the monitoring parameter A needs to be fed back, it actually means that the value corresponding to the monitoring parameter A in a specific communication process needs to be fed back.

[0192] In other words, the first value of the execution time of the third module, or the first identifier of the input or the first identifier of the output of the third module, and the first value of the execution time of the fourth module, or the first identifier of the input or the first identifier of the output of the fourth module correspond to the same communication process. The first monitoring parameter (or the value of the first monitoring parameter) generated in the communication process needs to be determined based on the information related to the third module (specifically, such as the first value of the execution time of the third module, or the first identifier of the input or the first identifier of the output of the third module), and the second monitoring parameter (or the value of the second monitoring parameter) generated in the communication process needs to be determined based on the information related to the fourth module (specifically, such as the first value of the execution time of the fourth module, or the first identifier of the input or the first identifier of the output of the fourth module).

[0193] Taking Figure 7 as an example, the base station instructs the UE to feedback multiple monitoring parameters, including channel estimation results and received signal-to-interference ratio (SIIR), for performance monitoring of the precoding module. The base station can instruct the UE to feedback the compressed information corresponding to the channel estimation result to be fed back, at time t1, and at time t2 when the precoding matrix obtained based on the fed-back channel information begins to be applied to downlink transmission. Furthermore, the UE can determine the channel estimation result belonging to the same communication process as the feedback time t1 of the compressed information, and the received signal-to-interference ratio after time t2 when the precoding matrix begins to be applied to downlink transmission, and feed back the channel estimation result and SIIR to the base station. When the base station indicates N time groups {t1, t2}, the UE needs to feedback the monitoring parameters corresponding to each of the N time groups. In this example, the channel estimation result is an example of the first monitoring parameter, and the received signal-to-interference ratio is an example of the second monitoring parameter. The channel compression module is an example of a pre-interaction module, and the precoding module is an example of a post-interaction module.

[0194] b) Optionally, as another specific implementation, the third indication information indicates the communication process in the following manner:

[0195] The third indication information indicates a first value and first time information of the execution time of the third module of the communication link. The first value and first time information of the execution time of the third module can indicate or be used to determine a first value of the execution time of the fourth module of the communication link. The first value of the execution time of the third module is used to determine a first value of the first monitoring parameter, and the first value of the execution time of the fourth module is used to determine a first value of the second monitoring parameter. The first value of the first monitoring parameter corresponds to the first value of the second monitoring parameter.

[0196] Optionally, the first time information indicates a time delay or a time difference.

[0197] In one example, the first time information indicates a delay, which is the delay between the first value of the execution time of the fourth module of the communication link and the first value of the execution time of the third module. In other words, the third indication information indicates the first value of the execution time of a module (specifically the third module) before the first network element and the second network element interact, as well as the delay information, and the delay information is used to determine the first value of the execution time of a module (specifically the fourth module) after the first network element and the second network element interact.

[0198] In another example, the first time information indicates a time difference, which is the time difference between the first value of the execution time of the third module of the communication link and the first value of the execution time of the fourth module. That is, the third indication information indicates the first value of the execution time of a certain module (specifically the third module) after the interaction between the first network element and the second network element, as well as the time difference information, and the time difference information is used to determine the first value of the execution time of a certain module (specifically the fourth module) before the interaction between the first network element and the second network element.

[0199] Continuing with Figure 7 as an example, for example, the base station instructs the UE to feedback multiple monitoring parameters, which include channel estimation results and received signal-to-interference ratios, for performance monitoring of the precoding module. The base station can instruct the UE to feedback the feedback time t1 of the compressed information corresponding to the channel estimation result to be fed back, and the delay T of the moment when the precoding matrix obtained based on the feedback channel information begins to be applied to the downlink transmission compared to the feedback time t1 of the compressed information. That is, the base station indicates t1+T to the UE. Furthermore, the UE can determine the channel estimation result that belongs to the same communication process as the feedback time t1 of the compressed information, and the received signal-to-interference ratio after a delay of T compared to the feedback time t1 of the compressed information, and feed back these two monitoring parameters to the base station.

[0200] Optionally, the delay T can be a numerical value, indicating that the compressed information fed back at time t1 and the received signal-to-interference ratio at time t1+T are the same communication process; or, the delay T can be a real number interval [T1, T2], indicating that the compressed information fed back at time t1 is applied to the signal transmission for a period of time [t1+T1, t1+T2]. Therefore, the compressed information fed back at time t1 and the signal reception within the time period [t1+T1, t1+T2] are the same communication process. At this time, the UE needs to feedback the monitoring parameters (for example, received signal-to-interference ratio) within a period of time [t1+T1, t1+T2]. The feedback form can be full feedback, or feedback of the average value of the monitoring parameters within the period of time, etc. When the base station indicates N time information, the UE needs to feedback the monitoring parameters corresponding to each of the N time information.

[0201] For another example, the base station indicates the time t1 corresponding to the execution of a certain module A after the interaction, or the ID of a certain input of module A. In addition, the base station indicates the time difference between the time t1 corresponding to the execution of a certain module B before the interaction and the time t1 corresponding to the execution of module A. The UE can then determine the value of the monitoring parameter (e.g., the first monitoring parameter) that belongs to the same communication process as the time t1 or the input ID of module A.

[0202] Optionally, the time difference T can be a numerical value, indicating that the module B executed at time t1-T and the module A executed at time t1 are the same communication process; or, the time difference T can be a real number interval [T1, T2], indicating that the result of executing a module B in the time period [t1-T1, t1-T2] jointly affects the execution of module A at time t1, so the relevant monitoring parameters generated before the interaction in the time period [t1-T1, t1-T2] (for example, the first monitoring parameters) and the relevant monitoring parameters corresponding to time t1 (for example, the second monitoring parameters) are the same communication process. At this time, the UE needs to feedback the relevant monitoring parameters generated before the interaction within a period of time [t1-T1, t1-T2], that is, there are multiple first monitoring parameters, and the feedback form can be all feedback, or feedback of the average of the multiple first monitoring parameters, etc., without limitation.

[0203] 303. The first network element monitors and / or trains one or more AI modules based on the input and / or output of the first module. Optionally, the one or more AI modules are deployed on the first network element or the second network element. Alternatively, the one or more AI modules include a first sub-AI module and a second sub-AI module, wherein the first sub-AI module is deployed on the first network element and the second sub-AI module is deployed on the second network element, and the first sub-AI module and the second sub-AI module are used in combination. It should be understood that the first sub-AI module and the second sub-AI module here refer to two sub-models in a dual-end model. An adaptive encoding (auto encoder, AE) model can generally refer to a network structure composed of two sub-models. An AE model can also be called a bilateral model, or a dual-end model, or a collaborative model. The encoder and decoder of an AE are usually trained together and can be used in combination with each other. For example, AI-based CSI feedback is a dual-end model. For example, the UE side compresses and quantizes the CSI through an encoder, and the access network device recovers the CSI through a decoder. For access network equipment, the input of the AI ​​model is the CSI fed back by the UE side, and the output is the recovered CSI. The training of the model requires the CSI measured on the UE side as the true value label of the recovered CSI.

[0204] Specifically, the first network element performs a performance analysis on the AI ​​module in the corresponding communication process based on the input and / or output of the first module. If the performance of the AI ​​module is lower than the threshold during one or more communication processes, it can be determined that the performance of the monitored AI module does not meet the requirements and a model update or switch is required. If the AI ​​module is located in the second network element, the first network element can instruct the second network element to update or switch the AI ​​module through an indication message. In addition, if the first network element finds during the performance analysis that there are performance risks in the other related AI modules associated with the feedback monitoring parameters, it can also send an indication message to trigger performance monitoring of the related AI module, or send an indication message to remind the second network element that there is a performance risk in the related AI module.

[0205] Refer to Figure 9, which is a schematic diagram of the performance analysis of the AI ​​module by the first network element. Taking the performance monitoring of the precoding module as an example, assuming that the first network element is a base station and the second network element is a UE, the base station can use the following method to analyze the performance of the precoding module: When the monitoring parameters fed back by the UE show that the receiving end signal-to-interference ratio is poor (the receiving end signal-to-interference ratio can be determined by the real channel information and the precoding matrix obtained by the precoding module), the base station can further obtain the expected receiving end signal-to-interference ratio on the base station side based on the channel recovery information obtained by the channel recovery module and the precoding matrix obtained by the precoding module. If the fed-back receiving end signal-to-interference ratio is basically consistent with the expected receiving end signal-to-interference ratio of the base station, it is considered that the performance of the precoding module is poor and a reasonable precoding matrix is ​​not obtained to achieve effective interference suppression. If the fed-back receiving end signal-to-interference ratio is inconsistent with the expected receiving end signal-to-interference ratio of the base station, it is considered that the channel recovery information obtained by the channel recovery module and the real channel information are significantly different, resulting in the precoding module not obtaining a reasonable precoding matrix to achieve effective interference suppression. Therefore, the base station can further compare the feedback channel estimation result and the channel recovery information obtained by the channel recovery module. If the two are highly similar, it is considered that the UE side channel estimation result and the actual channel information are significantly different, that is, the performance of the UE side channel estimation module is poor; conversely, if the feedback channel estimation result and the channel recovery information obtained by the channel recovery module are low in similarity, it is considered that the performance of the channel compression and channel recovery modules is poor.

[0206] Optionally, the first network element may also perform performance analysis on multiple AI modules during the corresponding communication process based on the input and / or output of the first module. If the multiple monitored AI modules include an AI module that fails to meet the requirements, the unsatisfactory AI module may be updated or switched. For example, if the performance of an AI module falls below a threshold during one or more communication processes, it may be determined that the performance of the AI ​​module fails to meet the requirements and requires a model update or switch. If the model is deployed on a second network element, the first network element may send an indication message instructing the second network element to switch or update the AI ​​module, or may alert the second network element that the AI ​​module presents a performance risk.

[0207] See Figure 10, which is another schematic diagram of the first network element performing performance analysis on an AI module. Taking the example of a base station instructing a UE to provide feedback on demodulation accuracy, demodulation module input, channel estimation results, and received signal-to-interference ratio (SIR), the base station can analyze the AI ​​model performance using the following method: When the monitoring parameters fed back by the UE indicate poor demodulation accuracy, the base station can further compare the input of the receiving-end demodulation module with the output of the base station-side modulation module. If the difference between the input of the receiving-end demodulation module and the output of the base station-side modulation module is small, the modulation and demodulation module performance is considered poor, and a poor modulation and demodulation effect is not achieved. If the difference between the input of the receiving-end demodulation module and the output of the base station-side modulation module is large, it is considered that poor performance of intermediate processes within the modulation and demodulation module is the cause of poor demodulation accuracy. Therefore, the base station can further analyze the performance of the precoding module, channel estimation module, compression module, and feedback module based on the channel estimation results and received signal-to-interference ratio. If the performance of these AI modules is normal, the UE-side equalization module performance is considered poor. Conversely, if the feedback channel estimation results and the channel recovery information obtained by the channel recovery module have low similarity, the performance of the channel compression module and channel recovery module is considered poor.

[0208] In this example, the second network element feeds back the monitoring parameters once, which can be used for performance monitoring of multiple AI modules, thereby reducing the feedback overhead of AI module performance monitoring.

[0209] The above is an example of the application of the technical solution provided in this application in the performance monitoring of the AI ​​model in combination with Figures 9 and 10. The following is an example of AI model training based on monitoring parameters.

[0210] Take the module connection relationship of the communication link shown in Figure 7 as an example, and take the first network element as a base station, the second network element as a UE, and the base station as an example to illustrate the model training.

[0211] Taking precoding module training as an example, the training data includes input and labels. The input to the precoding module is the channel recovery information output by the base station-side channel recovery module, and the label output by the precoding module includes the channel estimation result fed back by the UE. The channel recovery information and the channel estimation result are correlated. The training process is as follows: the channel recovery information is input to the precoding module, the precoding result is output, the signal-to-interference ratio is calculated based on the precoding result and the channel estimation result, and if the signal-to-interference ratio is below the threshold, the precoding module weights are updated to improve the signal-to-interference ratio until the signal-to-interference ratio meets the requirements.

[0212] Taking the joint training of the precoding module and channel recovery module as an example, the training data includes input and labels. The channel recovery module and precoding module are considered as a whole. The input of the model is the channel compression information fed back by the UE-side channel compression module. The label of the model includes the channel estimation result fed back by the UE, where the channel compression information and the channel estimation result are correlated. The training process is as follows: the channel compression information is input into the channel recovery module to obtain the channel recovery information, and then the channel recovery information is input into the precoding module, and the precoding result is output. The signal-to-interference ratio is calculated based on the precoding result and the channel estimation result. The difference between the channel recovery information and the channel estimation result is calculated and multiplied by -1, which is recorded as E (a negative number; a higher value indicates higher recovery accuracy). If the weighted sum of the signal-to-interference ratio and E is lower than the threshold, the weights of the precoding module and the channel recovery module are updated to increase the weighted sum of the signal-to-interference ratio and E until the weighted sum of the signal-to-interference ratio and E meets the requirement.

[0213] Combined with FIG10 , as shown in the performance analysis in FIG10 , if the analysis results determine that the performance of the modulation and demodulation module is poor and an AI model update is required, the following steps can be followed:

[0214] Taking the training of the modulation module and demodulation module as an example, the training data includes input and label. The input of the modulation module is the symbol to be modulated on the base station side (including the symbol to be modulated on the base station side associated with the UE feedback), and the label is the same as the input. The base station inputs the symbol to be modulated into the modulation module, and then adds random noise to the output of the modulation module and inputs it into the demodulation module designed by the base station side to obtain the recovered symbol. The base station calculates the difference between the recovered symbol and the input symbol. If the difference is higher than a certain threshold, the weights of the modulation module and the demodulation module are updated to reduce the difference until the difference is lower than the threshold. After completing the training or updating of the modulation module, the input in the training data and the output of the corresponding modulation module are sent to the UE for the UE to update or train its own modulation module.

[0215] If it is determined based on the analysis results that the channel compression module or precoding module performance is poor and needs to be updated, the above-mentioned example description of channel compression module or precoding module training can be referred to.

[0216] Example 2

[0217] The first network element obtains the input and / or output of the module deployed on the first network element and provides it to the second network element for monitoring and / or training of the AI ​​module of the second network element.

[0218] See Figure 11, which is another example of the monitoring or training AI module provided in this application.

[0219] 501. A first network element receives second indication information from a second network element, where the second indication information indicates one or more monitoring parameters, where the one or more monitoring parameters include input and / or output of a second module.

[0220] The second module may be any module of the communication link, and may be a non-AI module or an AI module, without limitation.

[0221] 502. The first network element obtains input and / or output of the second module.

[0222] 503. The first network element sends second information to the second network element, where the second information indicates the input and / or output of the second module.

[0223] The input and / or output of the second module is used to monitor or train one or more AI modules deployed in the second network element. For clarity, the one or more AI modules are collectively referred to as second AI modules. The input and / or output of the one or more second AI modules are determined based on the input and / or output of the second module, or the input and / or output of the second module are determined based on the input and / or output of the one or more second AI modules.

[0224] 504. The second network element monitors and / or trains one or more second AI modules based on the input and / or output of the second module.

[0225] In this example, the first network element, based on an instruction from the second network element, obtains one or more monitoring parameters (e.g., which may include inputs and / or outputs of the second module) and provides them to the second network element. The one or more monitoring parameters are used by the second network element to monitor and / or train one or more AI modules of the second network element.

[0226] For the specific implementation of the second network element indicating the monitoring parameters to be fed back to the first network element through the second indication information, and the first network element feeding back the monitoring parameters to the second network element through the second information, please refer to the description or examples of Example 1 above and will not be repeated here.

[0227] Example 3

[0228] Taking the first network element as a base station and the second network element as a UE as an example, the base station instructs the UE to provide one or more monitoring parameters, which are used for performance monitoring or training of one or more AI models of the uplink communication link.

[0229] See Figure 12, which is another example of the monitoring or training AI module provided in this application.

[0230] 601. A base station sends first indication information to a UE, where the first indication information indicates one or more monitoring parameters.

[0231] The one or more monitored parameters include inputs and / or outputs of one or more modules of the communication link.

[0232] For the specific implementation of the first indication information indicating one or more monitoring parameters, please refer to the description or examples in Example 1.

[0233] Exemplarily, the first indication information may indicate one of a plurality of predefined feedback options. For example, feedback option 1: feedback of channel recovery results for coarse monitoring of the channel compression module and the channel recovery module; or feedback option 2: feedback of channel recovery results and modulation symbols for performance monitoring of the precoding module, the channel compression module, the channel recovery module, and the equalization module.

[0234] For example, the monitoring parameters that the base station instructs the UE to feedback include channel recovery results and modulation symbols. The base station can instruct the UE to feedback the modulation symbols within which the data frame is to be fed back. The UE can then determine the channel recovery results from the same communication process as the modulation symbols to be fed back and feed them back to the base station for performance monitoring of the AI ​​model. When the base station indicates N time instants, the UE needs to feedback the monitoring parameters corresponding to each of the N time instants.

[0235] 602. The base station receives first information from the UE, where the first information indicates values ​​of the one or more monitoring parameters.

[0236] As described above, when there are multiple monitoring parameters, multiple monitoring parameters belonging to the same communication process are grouped together. In addition, when there are multiple communication processes, the first information indicates multiple groups of monitoring parameter values, each group of monitoring parameter values ​​corresponding to one communication process.

[0237] 603. The base station monitors or trains one or more AI modules of the uplink communication link according to the values ​​of the one or more monitoring parameters.

[0238] Optionally, if any of the monitored AI modules does not meet performance requirements, the AI ​​model may be switched or updated. If the unsatisfactory AI model is deployed on the UE side, the base station may issue an indication to the UE to update or switch the AI ​​model, or issue an indication to the UE to notify it of an AI model with performance risks.

[0239] Taking feedback option 2 in step 601, where the base station instructs the UE to provide feedback, as an example, the base station can analyze the AI ​​model performance using the following method: When the demodulation accuracy of a frame on the base station side is poor, the base station can further analyze the strength of the received signal. If the received signal strength differs significantly from the expected received signal strength calculated by the base station based on the estimated uplink channel, the UE-side precoding performance is poor; otherwise, the base station-side equalization module performance is considered poor. If the UE-side precoding performance is poor, the channel recovery result fed back by the UE is further compared with the channel estimated by the base station based on the uplink reference signal. If the UE-side channel recovery result differs significantly from the base station-side estimated uplink channel but is close to the historical uplink channel, the channel compression feedback delay is considered to be large. If the UE-side channel recovery result differs significantly from the base station-side estimated uplink channel and is not close to the historical uplink channel, the channel compression recovery module performance is considered to be poor. If the UE-side channel recovery result is close to the base station-side estimated uplink channel, the UE-side precoding module performance is considered to be poor.

[0240] In this example, the base station instructs the UE to feedback multiple monitoring parameters related to the same uplink communication process, enabling simultaneous performance monitoring of multiple AI models. Each monitoring parameter may be affected by the performance of multiple AI models. Although each monitoring parameter is only fed back once, it can be used to monitor the performance of multiple AI models, reducing the feedback overhead for AI model performance monitoring.

[0241] The above examples 1 to 3 respectively illustrate the application of the technical solutions of the present application in different situations. It is understandable that these examples can also be combined with each other. For example, the combination of example 1 and example 2 is as follows: the first network element obtains the input and / or output of the first module and monitors or trains one or more first AI modules; in addition, the first network element can also provide the input and / or output of the second module to the second network element based on the instruction of the second network element, so that the second network element can monitor or train one or more second AI modules. The combination of example 2 and example 3 is as follows: the first network element provides the input and / or output of the second module to the second network element based on the instruction of the second network element, so that the second network element can monitor or train one or more second AI modules; in addition, the first network element instructs the second network element to feedback the input and / or output of one or more modules of the communication link, so that the first network element can monitor or train one or more AI modules of the uplink communication link.

[0242] Based on the technical solution provided in this application and the above examples, those skilled in the art may also conceive of the use of this technical solution in other application scenarios, for example, based on the input / or output (i.e., one or more monitoring parameters) of one or more modules of the communication link, performance monitoring or training of one or more AI modules of the downlink communication link, etc. The embodiments of this application will not be listed one by one.

[0243] The above describes in detail the embodiment of the method for monitoring or training AI models provided by this application. The following introduces the communication device provided by this application.

[0244] Referring to FIG. 13 , the present application provides a communication device 1000 .

[0245] As shown in Figure 13, the communication device 1000 includes a processing module 1001 and a communication module 1002. The communication device 1000 can be a terminal device, or a communication device applied to a terminal device or used in conjunction with a terminal device and capable of implementing a method executed on the terminal device side, such as a chip, a chip system, or a circuit. Alternatively, the communication device 1000 can be a network device, or a communication device applied to a network device or used in conjunction with a network device and capable of implementing a method executed on the network device side, such as a chip, a chip system, or a circuit. Exemplarily, the network device can be the access network device in the method embodiment of the present application.

[0246] The communication module may also be referred to as a transceiver module, transceiver, transceiver, or transceiver device. The processing module may also be referred to as a processor, processing board, processing unit, or processing device. Optionally, the communication module is used to perform the sending and receiving operations on the terminal device side or the network device side in the above method. The device in the communication module that implements the receiving function can be considered a receiving unit, and the device in the communication module that implements the sending function can be considered a sending unit. That is, the communication module includes a receiving unit and a sending unit.

[0247] When the communication device 1000 is applied to a terminal device, the processing module 1001 can be used to implement the processing function of the terminal device described in each embodiment described in Figures 3 to 12, and the communication module 1002 can be used to implement the transceiver function of the terminal device described in each embodiment described in Figures 3 to 12.

[0248] When the communication device 1000 is applied to a network device, the processing module 1001 can be used to implement the processing function of the network device (for example, a positioning device or an access network device) in each embodiment described in Figures 3 to 12, and the communication module 1002 can be used to implement the transceiver function of the network device in each embodiment described in Figures 3 to 12.

[0249] In addition, it should be noted that the aforementioned communication module and / or processing module can be implemented through virtual modules, for example, the processing module can be implemented through a software functional unit or a virtual device, and the communication module can be implemented through a software function or a virtual device. Alternatively, the processing module or the communication module can also be implemented through a physical device, for example, if the device is implemented using a chip / chip circuit, the communication module can be an input / output circuit and / or a communication interface, performing input operations (corresponding to the aforementioned receiving operations) and output operations (corresponding to the aforementioned sending operations); the processing module is an integrated processor or microprocessor or integrated circuit.

[0250] The division of modules in this application is illustrative and represents only a logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the examples of this application may be integrated into a single processor, exist physically as separate modules, or two or more modules may be integrated into a single module. The aforementioned integrated modules may be implemented in either hardware or software functional modules.

[0251] 14 , the present application further provides a communication device 1100. Optionally, the communication device 1100 may be a chip or a chip system. Optionally, in the present application, a chip system may be composed of a chip or may include a chip and other discrete devices.

[0252] The communication device 1100 can be used to implement the functions of any network element (for example, a terminal device, a network device, or an AI entity) in the communication system described in the foregoing examples. The communication device 1100 may include at least one processor 1110. Optionally, the processor 1110 is coupled to a memory, and the memory may be located within the device, or the memory may be integrated with the processor, or the memory may be located outside the device. For example, the communication device 1100 may also include at least one memory 1120. The memory 1120 stores the necessary computer programs, computer programs or instructions and / or data for implementing any of the above examples; the processor 1110 may execute the computer program stored in the memory 1120 to complete the method in any of the above examples.

[0253] The communication device 1100 may also include a communication interface 1130, through which the communication device 1100 can exchange information with other devices. Exemplarily, the communication interface 1130 may be a transceiver, a circuit, a bus, a module, a pin, or another type of communication interface. When the communication device 1100 is a chip-type device or circuit, the communication interface 1130 in the device 1100 may also be an input-output circuit that can input information (or receive information) and output information (or send information). The processor is an integrated processor or microprocessor or integrated circuit or logic circuit, and the processor can determine output information based on the input information.

[0254] Coupling in this application refers to an indirect coupling or communication connection between devices, units, or modules, which can be electrical, mechanical, or other forms, and is used for information exchange between devices, units, or modules. Processor 1110 may operate in conjunction with memory 1120 and communication interface 1130. This application does not limit the specific connection medium between the processor 1110, memory 1120, and communication interface 1130.

[0255] Optionally, as shown in FIG13 , the processor 1110, the memory 1120, and the communication interface 1130 are interconnected via a bus 1140. The bus 1140 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be classified as an address bus, a data bus, a control bus, etc. For ease of illustration, FIG13 shows only one thick line, but this does not mean that there is only one bus or only one type of bus.

[0256] In this application, a processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in this application. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in this application may be directly executed by a hardware processor, or by a combination of hardware and software modules within the processor.

[0257] In this application, the memory may be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), or a volatile memory, such as a random-access memory (RAM). The memory is any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory in this application may also be a circuit or any other device that can implement a storage function, for storing program instructions and / or data.

[0258] In one possible implementation, the communication device 1100 can be applied to a network device side, such as an access network device in an embodiment of the present application, or a host or cloud device in an over-the-top (OTT) system. Specifically, the communication device 1100 can be a network device, or a device that can support the network device to implement the corresponding functions of the network device side in any of the above-mentioned examples. The memory 1120 stores computer programs (or instructions) and / or data that implement the functions of the network device side in any of the above-mentioned examples. The processor 1110 can execute the computer program stored in the memory 1120 to complete the method executed on the network device side in any of the above-mentioned examples. The communication interface in the communication device 1100 can be used to interact with a terminal device, send information to the terminal device, or receive information from the terminal device.

[0259] In another possible implementation, the communication device 1100 can be applied to a terminal device. Specifically, the communication device 1100 can be a terminal device, or a device that can support the terminal device and implement the functions of the terminal device in any of the above-mentioned examples. The memory 1120 stores a computer program (or instruction) and / or data that implements the functions of the terminal device in any of the above-mentioned examples. The processor 1110 can execute the computer program stored in the memory 1120 to complete the method executed by the terminal device in any of the above-mentioned examples. The communication interface in the communication device 1100 can be used to interact with the network device side (for example, an access network device) to send information to the network device side or receive information from the network device side.

[0260] Since the communication device 1100 provided in this example can be applied to a network device side (e.g., an access network device) to implement the method executed by the network device side, or applied to a terminal device to implement the method executed by the terminal device, the technical effects that can be achieved can be referred to the description of the above method embodiments and will not be repeated here.

[0261] Based on the above examples, the present application provides a communication system. In one example, the communication system includes a first network element and a second network element. Exemplarily, the first network element is an access network device, and the second network element is a terminal. Exemplarily, both the first network element and the second network element are network devices, or both the first network element and the second network element are terminal devices. The communication system can implement the methods for monitoring or training AI models provided in the embodiments shown in Figures 3 to 10.

[0262] The technical solutions provided in this application can be implemented in whole or in part through software, hardware, firmware, or any combination thereof. When implemented using software, they can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in this application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a terminal device, an access network device, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital video disc (DVD)), or a semiconductor medium.

[0263] In this application, under the premise of no logical contradiction, the examples can reference each other, for example, the methods and / or terms between method embodiments can reference each other, for example, the functions and / or terms between device embodiments can reference each other, for example, the functions and / or terms between device examples and method examples can reference each other.

[0264] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0265] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0266] At least one (item) involved in the embodiments of the present application indicates one (item) or more (items). More (items) refers to two (items) or more than two (items). "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. In addition, it should be understood that although the terms first, second, etc. may be used to describe each object in this disclosure, these objects should not be limited to these terms. These terms are only used to distinguish each object from each other.

[0267] The term "comprising" and any variations thereof mentioned in the embodiments of the present application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices. It should be noted that, in this application, words such as "exemplarily" or "for example" are used to indicate examples, illustrations or descriptions. Any method or design described in this application as "exemplarily" or "for example" should not be interpreted as being more preferred or more advantageous than other methods or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete manner.

[0268] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0269] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for monitoring or training an artificial intelligence (AI) model, characterized in that, Applied to a first network element or a chip for a first network element, the method includes: Obtaining the input and / or output of a first module, where the first module is an AI module or a non-AI module of a communication link; Monitoring and / or training one or more first AI modules of the communication link based on the input and / or output of the first module, where the input and / or output of the one or more first AI modules is determined based on the input and / or output of the first module, or the input and / or output of the first module is determined based on the input and / or output of the one or more first AI modules.

2. The method according to claim 1, characterized in that, The input and / or output of the one or more first AI modules is determined based on the input and / or output of the first module, or the input and / or output of the first module is determined based on the input and / or output of the one or more first AI modules, including: The identifier of the input and / or output of each of the one or more first AI modules corresponds to the identifier of the input and / or output of the first module; or The value of the execution time of each of the one or more first AI modules corresponds to the value of the execution time of the first module.

3. The method according to claim 1 or 2, characterized in that, The first module is deployed in the first network element or the second network element, and the one or more first AI modules are deployed in the first network element; or The first module is deployed in the first network element or the second network element, and the one or more first AI modules are deployed in the second network element; or The first module is deployed in the first network element or the second network element, and the one or more first AI modules include a first sub-AI module and a second sub-AI module. The first sub-AI module is deployed in the first network element, the second sub-AI module is deployed in the second network element, and the first sub-AI module and the second sub-AI module are used in a matching manner.

4. The method according to any one of claims 1 to 3, characterized in that, The first module is deployed in the second network element; The obtaining the input and / or output of the first module includes: Receiving first information from the second network element, where the first information indicates the input and / or output of the first module.

5. The method according to claim 4, wherein Before receiving the first information from the second network element, the method further includes: Sending first indication information to the second network element, where the first indication information indicates one or more monitoring parameters, and the one or more monitoring parameters include the input and / or output of the first module.

6. The method according to any one of claims 1 to 5, characterized in that The method further includes: Obtaining the input and / or output of a second module, where the second module is an AI module or a non-AI module of the communication link; Sending second information to the second network element, where the second information indicates the input and / or output of the second module, and the input and / or output of the second module is determined based on the input and / or output of a second AI module of the communication link, or the input and / or output of the second AI module of the communication link is determined based on the input and / or output of the second module.

7. The method according to claim 6, wherein Before sending the second information to the second network element, the method further includes: Receive second indication information from the second network element, where the second indication information indicates one or more monitoring parameters, and the one or more monitoring parameters include the input and / or output of the second module.

8. The method according to any one of claims 5 to 7, characterized in that, The first indication information indicates one or more monitoring parameters, including: The first indication information indicates a first index, where the first index belongs to one of the indexes included in a predefined corresponding relationship, and the corresponding relationship indicates the corresponding relationship between the monitoring parameter and / or the combination of monitoring parameters and the index, and the combination of monitoring parameters includes two or more monitoring parameters; or, The first indication information indicates the identifiers of the one or more monitoring parameters, and the identifiers of the one or more monitoring parameters belong to one or more identifiers in a predefined set of monitoring parameter identifiers.

9. The method according to claim 8, wherein The first indication information further includes the identifiers of the one or more first AI modules, where the one or more monitoring parameters indicated by the first indication information are used for monitoring the one or more first AI modules.

10. The method according to any one of claims 5 to 7, characterized in that The method further includes: Send third indication information to the second network element; Wherein, the third indication information indicates one of the following information: One or more values of the execution time of the third module of the communication link, and the values of the execution time of the third module correspond to the values of the execution time of the first module; or, One or more identifiers of the input of the third module of the communication link, and the identifiers of the input of the third module correspond to the identifiers of the input and / or output of the first module; or, One or more identifiers of the output of the third module of the communication link, and the identifiers of the output of the third module correspond to the identifiers of the input and / or output of the first module.

11. The method according to claim 10, characterized in that The first indication information indicates one or more monitoring parameters, including: The first indication information indicates a plurality of monitoring parameters, and the plurality of monitoring parameters include a first monitoring parameter and a second monitoring parameter; And, the third indication information indicates a first value of the execution time of the third module of the communication link and a first value of the execution time of the fourth module of the communication link; Wherein, the first value of the execution time of the third module is used to determine the first value of the first monitoring parameter, the first value of the execution time of the fourth module is used to determine the first value of the second monitoring parameter, and the first value of the first monitoring parameter corresponds to the first value of the second monitoring parameter; Or, The third indication information indicates a first identifier of the input or output of the third module of the communication link and a first identifier of the input or output of the fourth module of the communication link; Wherein, the first identifier of the input or output of the third module is used to determine the first value of the first monitoring parameter, the first identifier of the input or output of the fourth module is used to determine the first value of the second monitoring parameter, and the first value of the first monitoring parameter corresponds to the first value of the second monitoring parameter.

12. The method according to claim 10, wherein The first indication information indicates one or more monitoring parameters, including: The first indication information indicates a plurality of monitoring parameters, and the plurality of monitoring parameters include a first monitoring parameter and a second monitoring parameter; Moreover, the third indication information indicates a first value of the execution time of a third module of the communication link and first time information, and the first value of the execution time of the third module and the first time information indicate a first value of the execution time of a fourth module of the communication link; Wherein, the first value of the execution time of the third module is used to determine a first value of the first monitoring parameter, the first value of the execution time of the fourth module is used to determine a first value of the second monitoring parameter, and the first value of the first monitoring parameter corresponds to the first value of the second monitoring parameter.

13. The method according to claim 12, characterized in that, The first time information indicates a time delay, and the time delay is the time delay of the first value of the execution time of the fourth module relative to the first value of the execution time of the third module; or, The first time information indicates a time difference, and the time difference is the time difference of the first value of the execution time of the third module relative to the first value of the execution time of the fourth module.

14. The method according to claim 12 or 13, characterized in that, The first time information indicates a moment T, and T is a real number; or, The first time information indicates a time interval [T1, T2], and both T1 and T2 are real numbers.

15. The method according to claim 14, wherein When the first time information indicates the time interval [T1, T2], there are multiple first values of the second monitoring parameter, and the first value of the first monitoring parameter corresponds to the multiple first values of the second monitoring parameter.

16. A communication device, characterized in that, It includes a module for implementing the method according to any one of claims 1-15.

17. A communication device, characterized in that, It includes: A processor, the processor is coupled to a memory, and the processor is used to call computer program instructions stored in the memory to execute the method according to any one of claims 1-15.

18. A communication device, characterized in that, It includes a processor and a communication interface. The communication interface is used to receive data and / or information, and transmit the received data and / or information to the processor, and the processor processes the data and / or information; moreover, the communication interface is further used to output the data and / or information after being processed by the processor, so that the communication device executes the method according to any one of claims 1-15.

19. A computer-readable storage medium, characterized in that, Instructions are stored on the computer-readable storage medium. When the instructions run on a computer, the computer is made to execute the method according to any one of claims 1-15.

20. A computer program product, characterized in that, Instructions are stored on the computer-readable storage medium. When the instructions run on a computer, the computer is made to execute the method according to any one of claims 1-15.

21. A communication system, characterized in that, It includes the communication device according to claim 16 or 17.

Images