Method for monitoring or training an AI model and communication device

Abstract

The present application provides a method and a communication device for monitoring or training an AI model. When the performance of one or more AI modules in a communication link needs to be monitored or trained, the input and / or output of a first module that affects the input and / or output of the one or more AI modules is obtained, and the one or more AI modules are monitored or trained based on the input and / or output of the first module. Since the influence of related modules in the communication link on the input and / or output of the one or more AI modules being monitored or trained is taken into account, the performance monitoring accuracy of the AI ​​module or the inference performance of the AI ​​module can be improved.

Description

[Technical Field]

[0001] This application claims priority to Chinese Patent Application No. 202211735507.2, entitled "Method for monitoring or training an AI model and communication device," filed with the State Intellectual Property Office of the People's Republic of China on December 30, 2022, which is incorporated herein by reference in its entirety.

[0002] TECHNICAL FIELD Embodiments of the present application relate to the field of artificial intelligence, and more particularly to a method and a communication device for monitoring or training an AI model. [Background technology]

[0003] Currently, artificial intelligence (AI) is widely applied to communication links to improve communication performance. When AI models are used to implement multiple modules in a communication link, the performance of the communication link is comprehensively affected by multiple AI models.

[0004] In order to improve the accuracy of AI model performance monitoring and improve the inference performance of AI models, there is an urgent need for a method for monitoring or training AI model performance. Summary of the Invention [Means for solving the problem]

[0005] The present application provides a method and a communication device for monitoring and / or training an AI model to improve the accuracy of AI model performance monitoring or to improve the AI ​​model inference performance.

[0006] According to a first aspect, there is provided a method for supervising and / or training an AI model, the method comprising: a first network element obtaining inputs and / or outputs of a first module, the first module being an AI or non-AI module of a communication link; a first network element monitoring and / or training one or more first AI modules of the communication link based on the inputs and / or outputs of the first modules; Including, The inputs and / or outputs of one or more first AI modules are determined based on the inputs and / or outputs of the first modules, or the inputs and / or outputs of the first modules are determined based on the inputs and / or outputs of one or more first AI modules.

[0007] According to the technical solution provided in the present application, when the performance of one or more AI modules (i.e., a first AI module) of a communication link is monitored, the input and / or output of the first module that affects the input and / or output of the one or more AI modules is obtained, and the one or more AI modules are monitored and / or trained based on the input and / or output of the first module. Since the influence of related modules of the communication link on the input and / or output of the one or more AI modules is taken into account, the performance monitoring accuracy of the AI ​​module can be improved, or the AI ​​model inference performance can be improved.

[0008] In relation to the first aspect, in some implementations of the first aspect, the input and / or output of one or more first AI modules is determined based on the input and / or output of the first module, or the input and / or output of the first module is determined based on the input and / or output of the one or more first AI modules. the identifiers of the corresponding inputs and / or outputs of one or more first AI modules correspond to the identifiers of the inputs and / or outputs of the first module; or the corresponding run-time values ​​of the one or more first AI modules correspond to the run-time values ​​of the first modules; Includes.

[0009] Based on this implementation, when the input and / or output of the first module is used to monitor one or more first AI modules, the input and / or output of the first module corresponds to, or is associated with, or corresponds to the same communication process as, the respective input and / or output of the one or more first AI modules, so as to improve the performance monitoring accuracy of the one or more first AI modules or to improve the AI ​​model inference performance.

[0010] For the same communication process, please refer to the embodiment descriptions and examples herein.

[0011] For example, the identifier of an input of a module of a communication link may be, for example, an identifier of a resource (e.g., an identifier of a time-frequency resource) to be used when the input of the module is transmitted over the interface, and the identifier of an output of the module may be, for example, an identifier of a resource to be used when the output of the module is transmitted over the air interface.

[0012] In relation to the first aspect, in some implementations of the first aspect, the first module is deployed on the first network element or the second network element, and the one or more first AI modules are deployed on the first network element; or The first module is deployed on the first network element or the second network element, and the one or more first AI modules are deployed on the second network element; or A first module is deployed on a first network element or a second network element, and one or more first AI modules include a first sub-AI module and a second sub-AI module, the first sub-AI module is deployed on the first network element and the second sub-AI module is deployed on the second network element, and the first sub-AI module and the second sub-AI module are used in a matching method.

[0013] Based on this implementation, the AI ​​module monitoring or training method provided in the present application may be applicable to multiple scenarios, such as a scenario in which a first network element monitors and / or trains an AI module deployed at a local end, a scenario in which a first network element provides monitoring parameters to a second network element to monitor and / or train the performance of an AI module deployed on the second network element, or a scenario in which a first network element monitors and / or trains a dual-end model, thereby improving the accuracy of AI model monitoring and / or the performance of the AI ​​model inference in these scenarios.

[0014] In relation to the first aspect, in some implementations of the first aspect, the first module is deployed on the second network element.

[0015] The step of the first network element obtaining the input and / or output of the first module includes: receiving first information from a second network element by a first network element, the first information indicating inputs and / or outputs of the first module; Includes.

[0016] Based on this implementation, when the first module is deployed on a second network element, the first network element obtains the input and / or output of the first module using the report of the second network element, and monitors or trains one or more AI modules deployed on this side or on the second network element based on the input and / or output of the first module, to improve the AI ​​model monitoring accuracy or improve the AI ​​model inference performance.

[0017] In relation to the first aspect, in some implementations of the first aspect, before the first network element receives the first information from the second network element, the method includes: a first network element transmitting first indication information to a second network element, the first indication information indicating one or more monitored parameters, the one or more monitored parameters including an input and / or an output of the first module; Further includes:

[0018] Based on this implementation, the first network element indicates to report the input and / or output of the corresponding module of the communication link to the second network element based on the monitoring or training requirements of the AI ​​module, so as to monitor the performance of the AI ​​module or to train the AI ​​model.

[0019] In relation to the first aspect, in some implementations of the first aspect, the method includes: a first network element obtaining inputs and / or outputs of a second module, the second module being an AI or non-AI module of the communication link; a step of the first network element transmitting second information to the second network element, the second information indicating inputs and / or outputs of the second module, the inputs and / or outputs of the second module being determined based on the inputs and / or outputs of a second AI module of the communication link, or the inputs and / or outputs of the second AI module of the communication link being determined based on the inputs and / or outputs of the second module; Further includes:

[0020] Based on this implementation, the first network element obtains one or more monitoring parameters (e.g., including the input and / or output of the second module) and provides the one or more monitoring parameters to the second network element, which are used by the second network element to monitor or train one or more AI modules of the second network element to improve the accuracy of the monitoring or training of the AI ​​model.

[0021] In relation to the first aspect, in some implementations of the first aspect, before the first network element transmits the second information to the second network element, the method includes: receiving, by the first network element, a second indication from the second network element, the second indication indicating one or more monitored parameters, the one or more monitored parameters including an input and / or an output of the second module; Further includes:

[0022] Based on this implementation, the first network element provides corresponding monitoring parameters to the second network element based on the instruction of the second network element, so that the second network element monitors or trains the AI ​​model, and on-demand feedback can be performed based on the monitoring or training requirements of the second network element.

[0023] In relation to the first aspect, in some implementations of the first aspect, the first indication information indicating one or more monitoring parameters may include: the first indication information indicates a first index, the first index being one of the indexes included in a predefined correspondence relationship, the correspondence relationship indicating a correspondence relationship between the monitoring parameter and / or a combination of the monitoring parameters and the index, the combination of the monitoring parameters including two or more monitoring parameters; or The first indication indicates an identifier of one or more monitoring parameters, and the identifier of the one or more monitoring parameters is one or more identifiers within an identifier of a predetermined group of monitoring parameters. Includes.

[0024] In this implementation, the first indication may indicate the monitored parameters (eg, inputs and / or outputs of a non-AI module or an AI module of a communication link) using multiple implementations.

[0025] In relation to the first aspect, in some implementation forms of the first aspect, the first instruction information further includes identifiers of one or more first AI modules, and the one or more monitoring parameters indicated by the first instruction information are used to monitor the one or more first AI modules.

[0026] Based on this implementation, the monitoring parameters indicated by the first instruction information are associated with identifiers of one or more first AI modules, and the monitoring parameters to be fed back indicated by the first instruction information are used to monitor or train the one or more first AI modules.

[0027] In relation to the first aspect, in some implementations of the first aspect, the method includes: sending third indication information from the first network element to the second network element; further comprising The third indication information is the following information: one or more values ​​of the execution time of the third module of the communication link; or one or more identifiers of the inputs of the third module of the communication link; or one or more identifiers of the outputs of the third module of the communication link; Show one of the following.

[0028] Based on this implementation, when a monitoring parameter (e.g., an input and / or output of a first module) that needs to be fed back is generated (or obtained or generated) without interaction between the first network element and the second network element, and when the first network element indicates to the second network element to feed back the monitoring parameter, the first network element may indicate information about the time or identifier corresponding to any other monitoring parameter that belongs to the same communication process as the monitoring parameter. Thus, the second network element may determine the monitoring parameter in the corresponding communication process based on the information about the time or identifier, thereby improving model monitoring accuracy or improving AI model inference performance.

[0029] In relation to the first aspect, in some implementations of the first aspect, the first indication information indicating one or more monitoring parameters may include: the first indication indicates a plurality of monitoring parameters, the plurality of monitoring parameters including a first monitoring parameter and a second monitoring parameter; the third instruction indicates a first value of an execution time of a third module of the communication link and a first value of an execution time of a fourth module of the communication link; a first value of the execution time of the third module is used to determine a first value of a first monitoring parameter, and a first value of the execution time of the fourth module is used to determine a first value of a second monitoring parameter, the first value of the first monitoring parameter corresponding to the first value of the second monitoring parameter; or the third indication indicates a first identifier of an input or a first identifier of an output of a third module of the communication link and a first identifier of an input or a first identifier of an output of a fourth module of the communication link; The first identifier of the input or the first identifier of the output of the third module is used to determine a first value of a first monitoring parameter, and the first identifier of the input or the first identifier of the output of the fourth module is used to determine a first value of a second monitoring parameter, and the first value of the first monitoring parameter corresponds to the first value of the second monitoring parameter. Includes.

[0030] In this implementation, there are multiple monitoring parameters that need to be fed back, and some monitoring parameters (e.g., the first monitoring parameters) are generated without interaction between the first network element and the second network element, while some monitoring parameters (e.g., the second monitoring parameters) need to be generated with interaction between the first network element and the second network element. When the first network element indicates to the second network element to feed back the multiple monitoring parameters, the first network element may indicate the execution time of the module or the identifiers of the module's inputs and / or outputs before the interaction, and may further indicate the execution time of the module or the identifiers of the module's inputs and / or outputs after the interaction. Furthermore, the second network element may determine the multiple monitoring parameters that belong to the same communication process as the time and identifiers of the inputs and / or outputs. In this way, the accuracy of model monitoring can be improved, or the inference performance of the AI ​​model can be improved.

[0031] In relation to the first aspect, in some implementations of the first aspect, the first indication information indicating one or more monitoring parameters includes: the first indication indicates a plurality of monitoring parameters, the plurality of monitoring parameters including a first monitoring parameter and a second monitoring parameter; the third instruction information indicates a first value of an execution time and first time information of a third module of the communication link, and the first value of the execution time and the first time information of the third module indicates a first value of an execution time of a fourth module of the communication link; A first value of the execution time of the third module is used to determine a first value of a first monitoring parameter, and a first value of the execution time of the fourth module is used to determine a first value of a second monitoring parameter, the first value of the first monitoring parameter corresponding to the first value of the second monitoring parameter. Includes.

[0032] In this implementation, there are multiple monitoring parameters that need to be fed back, and some monitoring parameters (e.g., the first monitoring parameters) are generated without interaction between the first network element and the second network element, while some monitoring parameters (e.g., the second monitoring parameters) need to be generated with interaction between the first network element and the second network element. When the first network element indicates to the second network element to feed back the multiple monitoring parameters, the first network element may indicate the execution time of the module before the interaction or the execution time and time information of the module after the interaction (i.e., the first time information). Furthermore, the second network element may determine the multiple monitoring parameters that belong to the same communication process as the time based on the time-related information. In this way, the accuracy of model monitoring or the inference performance of the AI ​​model can be improved.

[0033] In relation to the first aspect, in some implementations of the first aspect, the first time information indicates a delay, and the delay is a delay between a first value of the execution time of the fourth module and a first value of the execution time of the third module; or The first time information indicates a time difference, and the time difference is a time difference between a first value of the execution time of the third module and a first value of the execution time of the fourth module.

[0034] Based on this implementation, the first instruction information may specifically indicate the execution time and delay information of a module (e.g., module A) before the interaction, so that the second network element can determine the execution time of the module (e.g., module A) before the interaction and the execution time of the module (e.g., module B) after the interaction; or the first instruction information may specifically indicate the execution time and time difference information of the module (e.g., module B) after the interaction, so that the second network element can determine the execution time of the module (e.g., module A) before the interaction and the execution time of the module (e.g., module B) after the interaction. In this way, the second network element determines multiple monitoring parameters belonging to the same communication process to improve AI model performance monitoring or model training accuracy.

[0035] In relation to the first aspect, in some implementations of the first aspect, the first time information indicates a time point T, where T is a real number; or The first time information indicates a time interval [T1, T2], where T1 and T2 are both real numbers.

[0036] Based on this implementation, the first time information may be a time point or a time interval. When the first time information indicates a time interval, if multiple communication processes are included in the time interval, the second network element needs to classify the multiple monitoring parameters generated in each communication process into one group and further feed back the multiple groups of monitoring parameters generated in the multiple communication processes to the first network element for model performance monitoring or model training. Multiple groups of monitoring data may be obtained using one instruction, thereby reducing the feedback overhead of the monitoring data.

[0037] In relation to the first aspect, in some implementation forms of the first aspect, when the first time information indicates a time interval [T1, T2], the second monitoring parameter has a plurality of first values, and the first values ​​of the first monitoring parameter correspond to the plurality of first values ​​of the second monitoring parameter.

[0038] Based on this implementation, when a monitoring parameter corresponds to one time interval, multiple monitoring parameters may be generated within the time interval, and the multiple monitoring parameters and another related monitoring parameter all belong to one communication process.

[0039] According to a second aspect, the present application provides a communications device. In one design, the communications device may include modules configured to perform the methods / operations / steps / actions described in the first aspect. The modules may be hardware circuits, software, or may be implemented by hardware circuits in combination with software. In one design, the communications device may include a processing module and a communications module. In one example, the communications device is a network device, such as an access network device. In another example, the communications device is a terminal device.

[0040] According to a third aspect, the present application provides a communications device. The communications device includes a processor configured to perform a method according to the first aspect or any one of the implementations of the first aspect. The processor is coupled to a memory. The memory is configured to store instructions and data. When the processor executes the instructions stored in the memory, the method according to the first aspect or any one of the implementations of the first aspect can be performed. Optionally, the communications device may further include a memory. Optionally, the communications device may further include a communications interface. The communications interface is used by the device to communicate with another device. For example, the communications interface may be a transceiver, a hardware circuit, a bus, a module, a pin, or another type of communications interface. In one example, the communications device may be a network device, such as an access network device, or a device, module, chip, etc., disposed in the network device, or a device that can be used in a matching manner with the network device. In another example, the communications device may be a terminal device, or a device, module, chip, etc., disposed in the terminal device, or a device that can be used in a matching manner with the terminal device.

[0041] According to a fourth aspect, the present application provides a communication system including a first network element, and optionally further including a second network element.

[0042] For example, the communication system includes a terminal device and an access network device. For example, the first network element is an access network device and the second network element is a terminal device, or the first network element is a terminal device and the second network element is an access network device. Alternatively, both the second network element and the second network element are network devices, or both the second network element and the second network element are terminal devices.

[0043] According to a fifth aspect, the present application provides a communication system including a communication device according to the third aspect.

[0044] According to a sixth aspect, the present application further provides a computer program, which, when run on a computer, enables the computer to perform the method according to the first aspect or any one of the implementations of the first aspect.

[0045] According to a seventh aspect, the present application further provides a computer program product comprising instructions, which, when executed on a computer, enable the computer to perform the method according to the first aspect or any one of the implementations of the first aspect.

[0046] According to an eighth aspect, the present application further provides a computer-readable storage medium, which stores a computer program or instructions, and when the computer program or instructions are executed on a computer, the computer is enabled to perform the method according to the first aspect or any one of the implementation forms of the first aspect.

[0047] According to a ninth aspect, the present application further provides a chip, wherein the chip is configured to read a computer program stored in a memory to perform the method according to the first aspect or any one of the implementations of the first aspect, or the chip includes circuitry configured to perform the method according to the first aspect or any one of the implementations of the first aspect.

[0048] According to a tenth aspect, the present application further provides a chip system. The chip system includes a processor configured to support an apparatus in performing a method according to the first aspect or any one of the implementations of the first aspect. In a possible design, the chip system further includes a memory configured to store programs and data required by the apparatus. The chip system may include a chip, or may include a chip and another discrete component.

[0049] For the technical effects of the solutions provided in any one of the second to tenth aspects or the implementation forms of the second to tenth aspects, please refer to the corresponding description of the first aspect, and the details will not be described again. [Brief explanation of the drawings]

[0050] [Figure 1] 1 is a diagram of an architecture of a communication system applicable to an embodiment of the present application; [Figure 2] FIG. 2 is a diagram of another communication system architecture applicable to an embodiment of the present application. [Figure 3] 1 is a diagram of a possible way of applying AI to a communication link. [Figure 4] FIG. 1 is a diagram of another possible way of applying AI to a communication link. [Figure 5] A diagram of an explainable AI model. [Figure 6] 1 is a schematic flowchart of a method for supervising or training an AI model according to the present application. [Figure 7] FIG. 1 is a diagram of a communication link. [Figure 8] FIG. 1 illustrates an example of monitoring or training an AI module according to the present application. [Figure 9] 1 is a diagram of an analysis of the performance of an AI module by a first network element. [Figure 10] FIG. 10 is another diagram of the analysis of the performance of the AI ​​module by the first network element. [Figure 11]FIG. 10 illustrates another example of supervising or training an AI module according to the present application. [Figure 12] FIG. 10 illustrates another example of supervising or training an AI module according to the present application. [Figure 13] 1 is a diagram of a communication device 1000 according to the present application. [Figure 14] 11 is a diagram of another communication device 1100 according to the present application. DETAILED DESCRIPTION OF THE INVENTION

[0051] The following describes the technical solutions of the present application with reference to the accompanying drawings.

[0052] The technical solutions provided in this application may be applied to various communication systems, such as long term evolution (LTE) systems, fifth generation (5G) communication systems, worldwide interoperability for microwave access (WiMAX) systems or wireless local area network (WLAN) systems, satellite communication systems, future communication systems such as 6G communication systems, or systems that integrate multiple systems. 5G communication systems are sometimes called new radio (NR) systems.

[0053] For example, a communication system may include terminal devices and network devices.

[0054] In an embodiment of the present application, a terminal device may be an entity configured to receive or transmit signals, such as a mobile phone. Terminal devices include handheld devices with wireless connectivity, other processing devices connected to a wireless modem, in-vehicle devices, etc. Terminal devices may be portable, pocket-sized, handheld, computer-integrated, or in-vehicle mobile devices. Terminal devices can be widely used in various scenarios, such as cellular communication, wireless fidelity (Wi-Fi) systems, device to device (D2D), vehicle to everything (V2X), peer to peer (P2P), machine to machine (M2M), machine type communication (MTC), internet of things (IoT), virtual reality (VR), augmented reality (AR), industrial control, and autonomous driving, telemedicine, smart grid, smart furniture, smart office, smart wearable, smart transportation, smart city, unmanned aerial vehicles, robots, remote sensing, passive sensing, positioning, navigation and tracking, and self-delivery and mobility.Some examples of the terminal device 120 include a user equipment (UE) of a 3GPP standard, a station (STA) of a Wi-Fi system, a fixed device, a mobile device, a handheld device, a wearable device, a mobile phone, a smartphone, a session initiation protocol (SIP) phone, a notebook computer, a personal computer, a smartbook, a vehicle, a satellite, a global positioning system (GPS) device, a target tracking device, an unmanned aerial vehicle, a helicopter, an aircraft, a ship, a remote control device, a smart home device, an industrial device, a personal communication service (PCS) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), a wireless network camera, a tablet computer, a palmtop computer, a mobile internet device (MID), a wearable device such as a smart watch, a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a terminal in an internet of vehicles system, and a self-driving (self-driving) device. Examples of such wireless terminals include wireless terminals for smart driving, wireless terminals for smart grids, wireless terminals for transportation safety, wireless terminals such as smart fuels in smart cities, terminal devices for high-speed railways, and wireless terminals for smart homes, such as smart speakers, smart coffee machines, and smart printers. The terminal devices may be wireless devices in the various scenarios mentioned above, or devices located within wireless devices, such as communication modules, modems, or chips within the devices mentioned above.The terminal device may also be referred to as a terminal, UE, mobile station (MS), mobile terminal (MT), etc. Alternatively, the terminal device may be a terminal device in a future wireless communication system. In addition, the terminal device may further include a position reference device, for example, an automated guided vehicle (AGV) or a device having a similar function. For ease of explanation, an example in which the terminal device is a UE is used for explanation hereinafter.

[0055] In the present application, a communication device configured to perform the functions of a terminal device may be a terminal device, a terminal device having some functions of the aforementioned communication device, or a device capable of supporting the performance of the functions of the aforementioned terminal device, such as a chip system. The device may be installed in the terminal device or used in a matching manner with the terminal device. In the present application, a chip system may include a chip, or may include a chip and another discrete component.

[0056] The network device may be a device that provides wireless communication function services or communicates with terminal devices, and is usually located on a network side. The network device may also be referred to as an access network device or a radio access network device. For example, the network device may be a base station. For example, the access network device in the embodiment of the present application includes, but is not limited to, a next-generation base station (gNodeB, gNB) in a 5G communication system, a base station in a 6th generation (6G) mobile communication system, a base station in a future mobile communication system, an access point (AP) in a Wi-Fi system, an evolved NodeB (eNB) in an LTE system, a radio network controller (RNC), a NodeB (NodeB, NB), a base station controller (BSC), a home base station (e.g., home evolved NodeB or home NodeB, HNB), a baseband unit (BBU), a transmission reception point (TRP), a transmitting point (TP), a base transceiver station (BTS), a satellite, an unmanned aerial vehicle, etc. In the network structure, the network device may include a central unit (CU) node, may include a distributed unit (DU) node, may be a RAN device including a CU node and a DU node, may be a RAN device including a control plane CU node, a user plane CU node, and a DU node, or may be a radio controller, a relay station, an in-vehicle device, a wearable device, etc. in a cloud radio access network (CRAN) scenario.Additionally, a base station may be a macro base station, a micro base station, a relay node, a donor node, or a combination thereof. Alternatively, a base station may be a communication module, modem, or chip disposed in the aforementioned device or apparatus. Alternatively, a base station may be a mobile switching center, a device performing base station functions for device-to-device (D2D), vehicle-to-everything (V2X), or machine-to-machine (M2M) communications, a network-side device for a 6G network, a device performing base station functions for a future communications system, etc. A base station may support networks of the same or different access technologies. This is not limiting. A base station may be fixed or mobile. For example, a helicopter or unmanned aerial vehicle may be configured as a mobile base station, and one or more cells may move based on the location of the mobile base station. In another example, a helicopter or unmanned aerial vehicle may be configured as a device for communicating with another base station.

[0057] In this application, the device configured to implement the functions of the aforementioned network device may be an access network device, may be a network device having some functions of an access network, or may be a device that can support the implementation of the functions of an access network, such as a chip system, a hardware circuit, a software module, or a combination of a hardware circuit and a software module. The device may be installed in the access network device or used together with the access network device in a matching manner. In the method of this application, an example in which the communication device configured to implement the functions of the access network device is an access network device is used for explanation.

[0058] Optionally, the communication system further comprises at least one AI node.

[0059] Optionally, the AI ​​node may be deployed in one or more of the following locations within a communications system: an access network device, a terminal device, a core network device, etc. Alternatively, the AI ​​node may be independently deployed, or may be deployed in a location other than any one of the above devices, for example, in a host or cloud server within an over-the-top (OTT) system. The AI ​​node may communicate with another device within the communications system. The other device may be, for example, one or more of a network device, a terminal device, a network element of a core network, etc.

[0060] Optionally, the AI ​​node is configured to perform AI-related operations, which may include, for example, one or more of model fault testing, model performance testing, model training, model inference, data collection, etc.

[0061] For example, a network device may forward data related to an AI model reported by a terminal device to an AI node, and the AI ​​node performs an AI-related operation. In another example, a network device or a terminal device may forward data related to an AI model to an AI node, and the AI ​​node performs an AI-related operation. In another example, an AI node may transmit one or more outputs of an AI-related operation, such as a trained neural network model, model evaluation results, model test results, etc., to a network device and / or a terminal device. For example, an AI node may transmit an output of an AI-related operation directly to a network device and a terminal device. In another example, an AI node may use a network device to transmit an output of an AI-related operation to a terminal device. In another example, an AI node may use a terminal device to transmit an output of an AI-related operation to a network device. It will be understood that when an AI node is located within a network device or a terminal device, transmission herein may be understood to refer to transmission between modules within the network device or the terminal device.

[0062] It should be understood that the number of AI nodes is not limited in this application. For example, if there are multiple AI nodes, the multiple AI nodes may be divided based on their functions. For example, different AI nodes are responsible for different functions.

[0063] It will be further understood that an AI node may be an independent device, may be integrated into the same device to perform different functions, may be a network element within a hardware device, may be a software function running on dedicated hardware, or may be a virtualized function instantiated on a platform (e.g., a cloud platform). The specific form of an AI node is not limited by this application.

[0064] In addition, all or part of the network devices or terminal devices in this application may be dedicated hardware, software functions running on dedicated hardware, software functions running on general-purpose hardware, or virtualized functions instantiated on a platform (e.g., a cloud platform). The specific form of the network devices or terminal devices is not limited in this application.

[0065] For ease of understanding, the following provides a brief description of related art or concepts in this application.

[0066] AI model: An AI model is an algorithm or computer program that can perform an AI function. An AI model represents a mapping relationship between the input and output of the model, or is a function model that maps a dimensional input to a dimensional output. Model parameters of an AI model are obtained through machine learning and training. For example, f(x)=ax²+b is a quadratic function model and can be considered as an AI model, where a and b are parameters of the AI ​​model and can be obtained through machine learning and training. For example, the AI ​​models referred to in the following embodiments of the present application are not limited to neural networks, linear regression models, decision tree models, support vector machines (SVMs), Bayesian networks, Q-learning models, or other machine learning (ML) models.

[0067] Training Dataset: A training dataset is data used for model training, validation, and testing in machine learning. The quantity and quality of data affect the effectiveness of machine learning. Training data may include inputs for an AI model, or may include inputs and target outputs for an AI model. Target outputs are target values ​​for the output of an AI model, and may also be called output truth values, output ground truth values, labels, or label samples.

[0068] Model training: Model training is the process of selecting an appropriate loss function and training the model parameters by using an optimization algorithm so that the value of the loss function is less than a threshold or the value of the loss function meets the target requirement.

[0069] AI model design: AI model design mainly includes a data collection phase (e.g., collecting training data and / or inference data), a model training phase, and a model inference phase, and may further include an inference result application phase. In the aforementioned data collection phase, a data source is used to provide a training dataset and inference data. In the model training phase, the training data provided by the data source is analyzed or trained to obtain an AI model. The AI ​​model represents a mapping relationship between the model's input and output. Obtaining an AI model by learning using a model training node is equivalent to obtaining a mapping relationship between the model's input and output by learning using training data. In the model inference phase, the trained AI model in the model training phase is used to perform inference based on the inference data provided by the data source and obtain an inference result. This phase may also be understood as inputting inference data into the AI ​​model to obtain an output using the AI ​​model. The output is the inference result. The inference result may indicate the configuration parameters used (executed) by the actor object and / or the operation performed by the actor object. The inference result is published in the inference result application phase. For example, the inference results may be jointly planned by actor entities. For example, the actor entities may send the inference results to one or more actor objects (e.g., a core network device, an access network device, or a terminal device) for execution. In another example, the actor entities may further feed back the model performance to a data source to facilitate subsequent model update training.

[0070] Loss Function: A loss function is used to measure the difference between the model's predicted value and the true value.

[0071] Model Application: Model application is the use of a trained model to solve a real-world problem.

[0072] It may be understood that the AI ​​model may be implemented using hardware circuitry, software, or a combination of software and hardware. Non-limiting examples of software include program code, programs, subprograms, instructions, instruction sets, code, code segments, software modules, applications, or software applications.

[0073] In the embodiments of the present application, an "indication" may include a direct indication, an indirect indication, an explicit indication, and an implicit indication. When indication information is described as indicating A, it may be understood that the indication information holds A, directly indicates A, or indirectly indicates A. An indirect indication may mean that the indication information directly indicates B and the correspondence between B and A to indicate A using the indication information. The correspondence between B and A may be predefined in a protocol, may be pre-stored, or may be obtained using a configuration between network elements.

[0074] A wireless communication system applicable to the present application may include one or more network side devices and one or more terminal devices. The network side devices may include the aforementioned access network devices, and may optionally further include core network devices. This is not limited thereto. The device may also be replaced with a network element, an entity, a network entity, a communication device, a communication module, a node, a communication node, etc. In the present application, a device or a network element is used as an example for explanation.

[0075] 1 is a diagram of a communication system architecture applicable to one embodiment of the present application. For example, the communication system includes a network device 110, a terminal device 120, and a terminal device 130. The terminal devices 120 and 130 may access and communicate with the network device 110. Optionally, the network device 110 may be an access network device. For example, the communication system may further include an AI entity. The AI ​​entity is located within the network device 110, in other words, a module of the network device 110, and is not shown in FIG. 1.

[0076] 2 is a diagram of another communication system architecture applicable to an embodiment of the present application. The communication system includes a network device 110, a terminal device 120, a terminal device 130, and an AI entity 140. The network device 110 may forward data reported by the terminal device and related to an AI model to the AI ​​entity 140. The AI ​​entity 140 performs AI-related operations such as training dataset construction and model training, and provides one or more outputs of the AI-related operations, such as a trained AI model, model evaluation, and test results, to the network device 110.

[0077] It should be understood that Figures 1 and 2 are merely diagrams. The communication system may further include other devices, such as core network devices, wireless relay devices, wireless backhaul devices, and / or devices configured to perform artificial intelligence functions, all of which are not shown in Figures 1 and 2. In addition, in actual applications, the wireless communication system may include multiple network devices (e.g., access network devices and core network devices) or multiple terminal devices. This is not limited. One access network device may simultaneously provide service to one or more terminal devices. One terminal device may also simultaneously access one or more access network devices. The number of terminal devices and the number of network devices included in the wireless communication system are not limited in the embodiments of the present application. The number of devices shown in Figures 1 and 2 is merely an example. Each device is not limited to one or more, and is not limited according to a specific technical solution.

[0078] Possible ways of applying AI to communication links to improve communication performance include replacing some or all of the modules in the communication link with AI models, while preserving the physical meaning of the different modules in the communication link, as shown in Figure 3. Alternatively, some or all of the modules in the communication link are replaced with AI models, while not necessarily preserving the physical meaning of the different modules in the communication link, as shown in Figure 4.

[0079] As shown in FIG. 3, the transmitting end and the receiving end include multiple modules, each of which has the same physical meaning as a conventional communication link, e.g., channel compression, channel recovery, and precoding. The difference between FIG. 3 and a conventional communication link is that some or all of the modules in FIG. 3 are implemented using AI models. For example, the channel compression module is an AI model-based channel compression module, and the precoding calculation module is an AI model-based precoding module. To ensure the performance of the communication link, the AI ​​model performance in different modules can be monitored, and operations such as model switching or updating can be performed in a timely manner.

[0080] The receiving end in Figure 4 is implemented by an AI model, and the specific implementation process is not divided into multiple modules as shown in Figure 3. However, black-box AI cannot explain why it obtains an output. Therefore, its reliability cannot be verified. Therefore, to ensure the reliability of an AI model, an explainable AI model can be used. An explainable AI model requires that the AI ​​model can output a group of intermediate variables that have physical meaning. The output of the AI ​​model can be obtained by applying the group of intermediate variables to each module in a conventional communication link. Therefore, the group of intermediate variables can explain why the black-box AI obtains an output, as shown in Figure 5. Figure 5 is a diagram of an explainable AI model. The AI ​​model in Figure 4 corresponds to multiple virtual modules within the dashed box in Figure 5.

[0081] The intermediate variables output by an explainable AI model can also be used to monitor the performance of the AI ​​model.

[0082] When an AI model in a communication link is used to implement multiple modules (including the virtual module in FIG. 5 ) in the communication link, the performance of the communication link is comprehensively affected by the multiple AI models, and the performance of each AI model is affected by the performance of the other modules. This poses challenges to performance monitoring or training of the AI ​​models. AI model performance monitoring is used as an example. In known model monitoring solutions, each AI model in the communication link is monitored independently. For example, direct or indirect performance indicators are defined for the AI ​​model, and model performance is monitored based on the values ​​of the defined performance indicators. Direct performance indicators are indicators of model accuracy that can be obtained by calculation based on model inputs or outputs or data labels. Indirect performance indicators include indicators of communication performance when an AI model is used in the communication link. However, for AI models for which data labels are difficult to obtain, it is difficult to calculate direct performance indicators. However, for indirect performance indicators, because the performance of the communication link is affected by multiple AI models, it is difficult to directly use indirect performance indicators to determine model performance. In practice, we show that solutions that monitor or train each AI model independently do not help to accurately determine the performance of the communication link, and affect the performance improvement of the communication link.

[0083] The following describes the technical solutions provided in this application.

[0084] FIG. 6 is a schematic flow chart of a method for supervising or training an AI model according to the present application.

[0085] 210: A first network element obtains an input and / or an output of a first module, where the first module is an AI module or a non-AI module of a communication link.

[0086] In this application, a communication link may be a link between a sending end and a receiving end of signals and / or data.

[0087] Optionally, the first network element may be one of the transmitting end and the receiving end of a communication link. Alternatively, the first network element may be an AI entity corresponding to the transmitting end or the receiving end. For example, the AI ​​entity corresponding to the transmitting end may be deployed separately from the transmitting end or may be a module within the transmitting end. Similarly, the AI ​​entity corresponding to the receiving end may be deployed separately from the receiving end or may be a module within the receiving end. This is not a limitation.

[0088] Optionally, the number of first modules is not limited to one or more. In this embodiment of the present application, one first module is used as an example for explanation. Those skilled in the art can understand an embodiment of multiple first modules by referring to the description of one first module. One first module is used as an example, and the first module may be any module of the communication link. In addition, the first module may be an AI module or a non-AI module.

[0089] 220: The first network element monitors and / or trains one or more first AI modules of the communication link based on the inputs and / or outputs of the first modules.

[0090] The inputs and / or outputs of one or more first AI modules are determined based on the inputs and / or outputs of the first modules, or the inputs and / or outputs of the first modules are determined based on the inputs and / or outputs of one or more first AI modules.

[0091] Optionally, the input and / or output of each of the one or more first AI modules is determined based on the input and / or output of the first module, or the input and / or output of the first module is determined based on the input and / or output of the one or more first AI modules, the identifiers of the corresponding inputs and / or outputs of one or more first AI modules correspond to the identifiers of the inputs and / or outputs of the first module; or the corresponding run-time values ​​of the one or more first AI modules correspond to the run-time values ​​of the first modules; Includes.

[0092] It should be understood that the above "plurality of first AI modules" refers to a plurality of AI modules of a communication link, and each of the plurality of AI modules is referred to as a first AI module.

[0093] In addition, the identifiers of the inputs and / or outputs of one or more first AI modules corresponding to the identifiers of the inputs and / or outputs of the first modules, or the corresponding execution time values ​​of one or more first AI modules corresponding to the execution time values ​​of the first modules, may mean that the inputs and / or outputs of one or more first AI modules and the inputs and / or outputs of the first modules belong to the same communication process. For example, based on the connection relationships between the modules included in the communication link, the inputs and / or outputs of modules generated in the process of sequentially operating these modules correspond to each other based on the communication process. For example, if the output of the first module at time t1 is the input of AI module a at time t2, the output of the first module at time t1 corresponds to the input of AI module a at time t2, or the output of the first module at time t1 and the input of AI module a at time t2 belong to the same communication process. In another example, according to the connection relationships between the inputs and / or outputs of the modules of the communication link, the input y and / or output z of AI module a are affected by the output x of the first module. AI module a is an example of an AI module on the communication link. The first module is an example of any module (AI module or non-AI module) on a communication link, and AI module a and the first module may or may not be adjacent in terms of connection relationship. In this case, the input y and / or output z of AI module a correspond to the output x of the first module or belong to the same communication process, where x is the identifier of the output of the first module, y is the identifier of the input of AI module a, and z is the identifier of the output of AI module a. The identifiers of the inputs and / or outputs of these modules may be expressed in other manners. This is not limiting. Additionally, optionally, the first module is used as an example. The identifier of the input of the first module may include a resource identifier used to transmit the input of the first module over the air interface.The resource identifier includes one or more of a time-domain resource location identifier, a frequency-domain resource location identifier, a time-frequency resource location identifier, and a spatial-domain resource location identifier such as an antenna port or beam identifier. The identifier of the output of the first module may include a resource identifier used to transmit the output of the first module over the air interface. The resource identifier includes one or more of a time-domain resource location identifier, a frequency-domain resource location identifier, a time-frequency resource location identifier, and a spatial-domain resource location identifier such as an antenna port or beam identifier. For example, the time-frequency resource locations used to transmit different inputs (or outputs) of the first module over the air interface are generally different and may therefore be used to identify different inputs (or outputs) of the first module.

[0094] Optionally, the correspondence of the input and / or output identifiers of one or more first AI modules to the input and / or output identifiers of the first module, or the correspondence of the corresponding execution time values ​​of one or more first AI modules to the execution time values ​​of the first module, may mean that the inputs and / or outputs of the one or more first AI modules and the inputs and / or outputs of the first module belong to the same inference process.

[0095] For example, an example in which the UE performs CSI prediction is used. The UE feeds back information used for CSI prediction and historically predicted CSI (e.g., channel estimation results) to the base station. The information used for CSI prediction is the output of a channel estimation module on the UE side, and the historically predicted CSI is the output of a CSI prediction module on the UE side. The base station receives the information fed back by the UE and uses the information as input to a precoding module to obtain the output of the precoding module. There may be multiple pieces of information fed back by the UE for CSI prediction, and there may also be multiple historically predicted CSIs. The multiple pieces of information used for CSI prediction and the multiple historically predicted CSIs may belong to multiple different communication processes, but they are all used as input to the precoding module. Therefore, the multiple outputs of the channel estimation module and the multiple outputs of the CSI prediction module on the UE side, and the output of the precoding module on the base station side, belong to the same inference process.

[0096] It can be seen from the description of the first modules and first AI modules that in this embodiment of the present application, one or more first AI modules of the communication link are monitored and / or trained based on the inputs and / or outputs of the one or more first modules.

[0097] According to the technical solution provided in the present application, when the performance of one or more AI modules (i.e., a first AI module) of a communication link is monitored, the inputs and / or outputs of the first module that affect the inputs and / or outputs of the one or more AI modules are obtained, and the one or more AI modules are monitored and / or trained based on the inputs and / or outputs of the first module. Since the influence of related modules of the communication link on the inputs and / or outputs of the one or more AI modules is taken into account, the accuracy of performance monitoring or model training of the AI ​​modules can be improved.

[0098] In the present application, the monitoring parameters may be used for AI model performance monitoring and / or AI model training. This is not limiting. In the following embodiments, only model monitoring is used as an example for explanation. It should be understood that in embodiments in which an AI model is monitored based on the monitoring parameters, the AI ​​model may alternatively be trained based on the monitoring parameters. During model monitoring or model training, the influence of related modules of a communication link on the input and / or output of the AI ​​module being monitored or trained is taken into account. Therefore, the accuracy of AI module performance monitoring or model training can be improved.

[0099] Optionally, an example is used in which one of the transmitting end and the receiving end of the communication link is a first network element and the other is a second network element. The technical solutions provided in the present application can be applied to AI model performance monitoring or model training in multiple scenarios. For example, the first network element obtains inputs and / or outputs of the first module and monitors and / or trains an AI module deployed on a second network element based on the inputs and / or outputs of the first module, or monitors and / or trains an AI module deployed on the first network element; or the first network element monitors and / or trains a dual-ended model deployed on the first network element and the second network element. Optionally, the first module may be deployed on the first network element or the second network element. Alternatively, the first network element may further obtain inputs and / or outputs of the first module deployed on the first network element and provide inputs and / or outputs for the second network element to monitor and / or train the AI ​​module of the second network element.

[0100] To clarify the solution description, an example of a communication link is provided below with reference to FIG. 7, and these technical solutions are described based on the example.

[0101] FIG. 7 is a diagram of a communication link. As shown in FIG. 7, the communication link includes a transmitting end and a receiving end. The transmitting end is an example of a first network element, and the receiving end is an example of a second network element. A modulation module, a precoding module, a transmission module, and a channel recovery module are disposed at the transmitting end. A reception module, an equalization module, and a demodulation module are disposed at the receiving end. In addition, a channel estimation module and a channel compression module are further disposed at the receiving end. As mentioned above, all or some of the modules disposed at the transmitting end and the receiving end may be AI modules. This is not a limitation.

[0102] The following provides Example 1, Example 2 and Example 3 to illustrate the technical solutions of the present application.

[0103] Example 1 The first network element obtains the inputs and / or outputs of the first module to monitor and / or train the AI ​​module of the first network element. Optionally, the first module is deployed on the first network element or the second network element.

[0104] FIG. 8 illustrates an example of supervising or training an AI module according to the present application.

[0105] 301: Optionally, a first network element determines to monitor and / or train one or more AI modules.

[0106] 302: A first network element obtains an input and / or an output of a first module.

[0107] If possible, the first module is deployed on a first network element, and the first network element obtains the input and / or output of the first module.

[0108] The inputs and / or outputs of one or more AI modules are determined based on the inputs and / or outputs of the first module, or the inputs and / or outputs of the first module are determined based on the inputs and / or outputs of one or more AI modules.

[0109] In another possible case, the first module may be deployed on a second network element, and the first network element may obtain the inputs and / or outputs of the first module from the second network element, e.g., the first network element may indicate to the second network element to provide the inputs and / or outputs of the first module and receive the inputs and / or outputs of the first module transmitted by the second network element.

[0110] Specifically, the first network element sends first instruction information to the second network element, the first instruction information indicating one or more monitoring parameters, the one or more monitoring parameters including inputs and / or outputs of the first module.

[0111] For example, the first instruction information may indicate one or more monitoring parameters in the following manner 1 or 2.

[0112] Method 1 The first indication information indicates an index (e.g., referred to as a first index), the first index being one of the indexes included in a predetermined correspondence relationship, the correspondence relationship indicating a correspondence relationship between the index and the monitoring parameter and / or a combination of the monitoring parameters, the combination of the monitoring parameters including two or more monitoring parameters.

[0113] Table 1 is used as an example to illustrate the correspondence.

[0114] [Table 1]

[0115] For example, in the communication link shown in FIG. 7, the first network element is a base station and the second network element is a UE. Multiple indexes may be predefined for performance monitoring of the AI ​​module of the communication link between the base station and the UE. For example, for monitoring the precoding module, an index 0 may be defined that corresponds to a received signal-to-interference ratio and can be used for coarse monitoring of the precoding module, and an index 1 may be defined that corresponds to a channel estimation result and the received signal-to-interference ratio and can be used for fine monitoring of the precoding module. Furthermore, the base station may deliver the corresponding index to the UE based on the monitoring requirements. Optionally, the base station may further deliver an identifier of the AI ​​module to be monitored, for example, the identifier of the precoding module, to indicate that the monitoring parameter corresponding to the index is used to monitor the AI ​​module. This is not a limitation.

[0116] In addition, in one example, the index may be further associated with the ID of a specific AI module and indicate that the index is used to monitor the specific AI module. For example, index 0 may be associated with the ID of the precoding module and indicate that index 0 is used to monitor the precoding module. In another example, index 1 may be associated with the ID of the precoding module and indicate that index 1 is used to monitor the precoding module. In another example, the index may not be associated with the ID of a specific AI module. For example, index 0 may be defined to correspond to the received signal-to-interference ratio and be used for coarse monitoring of the precoding module, channel compression module, or channel recovery module; index 1 may be defined to correspond to the channel estimation result and the received signal-to-interference ratio and be used for fine monitoring of the precoding module, channel compression module, or channel recovery module; index 2 may be defined to correspond to the demodulation accuracy, the channel estimation result, and the received signal-to-interference ratio; and index 3 may be defined to correspond to the demodulation accuracy, the input of the demodulation module, the channel estimation result, and the received signal-to-interference ratio.

[0117] Method 2 The first instruction indicates an identifier of one or more monitoring parameters, the identifier of the one or more monitoring parameters being one or more identifiers within a predetermined group of identifiers of monitoring parameters, the one or more monitoring parameters including an input and / or an output of the first module.

[0118] 7 is used as an example. The group of predetermined monitoring parameters may include one or more of the inputs and / or outputs of multiple modules (AI modules or non-AI modules) of the communication link, such as modulated bits, modulated symbols, a received signal-to-interference ratio (SIR) at the receiving end, log-likelihood information of symbols to be demodulated, demodulation information, channel estimation results, compressed information of the channel estimation results, and channel recovery information. Corresponding identifiers are defined for these monitoring parameters. The first indication information indicates the corresponding identifiers to indicate the monitoring parameters corresponding to the identifiers. For example, assume that the aforementioned monitoring parameters and the identifiers corresponding to the monitoring parameters are the received signal-to-interference ratio (SIR) at the receiving end (A), log-likelihood information of symbols to be demodulated (B), demodulation information (C), channel estimation results (D), compressed information (E), and channel recovery information (F). If the first network element needs to instruct the second network element to feed back the received signal-to-interference ratio, the channel estimation results, and compressed information, the first indication information indicates identifiers A, D, and E. When the first network element needs to indicate to the second network element to feed back the received signal-to-interference ratio, the first indication information indicates the identifier A.

[0119] Optionally, for the communication link shown in FIG. 7, the internal implementations of different modules may be implemented by different vendors (including algorithms optimized by the vendors or AI models). This is not mandatory. Therefore, module performance may be evaluated based on the inputs and / or outputs of different modules, or the results of processing inputs and / or outputs of different modules. According to the inter-module connection relationships of the communication link shown in FIG. 7, a group of monitoring parameters (also called intermediate variables) that can be used for model monitoring may be determined. The monitoring parameters may include the following types: inputs and / or outputs of the modules of the communication link (implemented by AI or non-AI), and results obtained after the inputs and / or outputs of the modules of the communication link are processed.

[0120] Although the monitoring parameters are used to monitor the AI ​​module, it should be understood that the monitoring parameters may include the input and / or output of the AI ​​module, or may further include the input and / or output of a non-AI module. Figure 7 is used as an example. Assume that the channel compression module, channel recovery module, and precoding module use AI models, while other modules, such as the channel estimation module and receiving module, use conventional algorithms. The recovery accuracy of the channel recovery module and the performance of the precoding module are affected by channel estimation. Therefore, the input / output of the channel estimation module may also be used to evaluate the performance of the AI ​​module. In addition, the monitoring parameters may further include results obtained after the input / output of one or more modules are processed. For example, the received signal-to-interference ratio at the receiving end may be estimated using the output of the receiving module and used to evaluate the performance of the precoding module. In another example, the correlation between the output of the channel recovery module and the input of the channel compression module may be calculated and used to evaluate the channel compression and recovery accuracy.

[0121] Optionally, the above group of monitoring parameters that can be used to monitor the AI ​​module may be predefined, preconfigured, or prestored in a protocol, without being limited in this regard.

[0122] When the first network element determines to monitor the performance of one or more AI modules of the communication link, optionally, if the monitoring parameters are generated by the second network element (i.e., the monitoring parameters correspond to the input and / or output of the module, and the module is deployed on the second network element), the first network element may indicate to the second network element to feed back the monitoring parameters. Specifically, the aforementioned method 1 or method 2 may be used, and the monitoring parameters are indicated by the first indication information. For example, the first network element is a base station, and the second network element is a UE. For a precoding module, the data label (i.e., the optimal precoding matrix) usually cannot be obtained. Therefore, the most direct performance indicator is the signal-to-interference ratio (SIR) at the receiving end (or the SIR at the receiving end). However, the SIR at the receiving end is also affected by the channel compression at the UE side and the channel recovery accuracy at the base station side. Therefore, simply monitoring the performance of the precoding module based on the SIR at the receiving end cannot eliminate the influence of the channel estimation, compression, and recovery modules. Based on the foregoing analysis, to monitor the precoding module, the base station may indicate to the UE to feed back several monitoring parameters, including a channel estimation result and a received signal-to-interference ratio.

[0123] In this embodiment of the present application, in order to effectively evaluate the performance of the AI ​​module, multiple monitoring parameters that need to be fed back are generated in the same communication process. Therefore, when instructing the second network element to feed back the monitoring parameters, the first network element needs to indicate the communication process. Furthermore, the second network element feeds back the monitoring parameters generated in the communication process.

[0124] An example is used in which the UE side feeds back channel estimation results and received signal-to-interference ratios to monitor the precoding module. One communication process includes the UE inputting the channel estimation results to a channel compression module to generate compression information. The compression information is then fed back to the base station and input to a channel recovery module to obtain channel recovery information for precoding calculation. The precoding is used for downlink transmission. The receiving end can calculate the received signal-to-interference ratio when precoding is used based on the received signal.

[0125] Optionally, the specific manner in which the first network element indicates the communication process may differ based on whether the generation of the monitoring parameter to be fed back involves an interaction between the second network element and the first network element.

[0126] (1) The generation of the monitoring parameters to be fed back does not involve any interaction between the second network element and the first network element.

[0127] For example, when the first network element indicates to the second network element to feed back the input and / or output of the first module, i.e., when the monitored parameters are particularly the input and / or output of the first module, the first network element uses the third indication information to indicate the specific communication process. Optionally, the third indication information includes one or more of the following: the third indication indicates one or more values ​​of an execution time of a third module of the communication link; or the third indication indicates one or more identifiers of an input of the third module of the communication link; or The third indication indicates one or more identifiers of the output of the third module of the communication link.

[0128] The communication link between the base station and the UE shown in FIG. 7 is used as an example. In the case of multiple modules operating on the UE side (not including interaction with the base station), the UE only needs to sequentially buffer multiple monitoring parameters based on the connection relationships between the modules shown in FIG. 7 to ensure that multiple monitoring parameters are generated in the same communication process. For example, in FIG. 7, in the process in which the UE sequentially executes the channel estimation module and the channel compression module, the generated channel estimation results and compression information belong to the same communication process. Therefore, the base station only needs to indicate the identifier or time information corresponding to the value of the monitoring parameter, and the UE can determine the value of another monitoring parameter that belongs to the same communication process as the value of the monitoring parameter.

[0129] For example, the base station may indicate a time point corresponding to the execution of a module (e.g., module A, an example of a third module) in the execution process on the UE side or an ID of the output of the module (e.g., module A). As a result, the UE may determine the monitoring parameters belonging to the same communication process as the time point or ID of the output, e.g., the input and / or output of the first module. For example, when the base station indicates to the UE to feed back a channel estimation result, the monitoring parameters are used to monitor the performance of the channel recovery module. The base station may indicate a feedback time point of the compression information corresponding to the monitoring parameters that need to be fed back by the UE. Furthermore, the UE may determine a channel estimation result belonging to the same communication process as the feedback time point of the compression information and feed back the channel estimation result to the base station for performance monitoring of the channel recovery module. When the base station indicates N time point information, the UE feeds back monitoring parameters corresponding to the N time point information. For example, when the base station indicates to the UE to feed back a channel estimation result, the base station may indicate N feedback time points of the compression information. Furthermore, the UE needs to feed back channel estimation results corresponding to N feedback time points respectively to the base station, i.e., feed back N channel estimation results, where the N channel estimation results correspond to N feedback time points respectively, and N is a positive integer.

[0130] Optionally, the first network element may indicate to the UE to feed back multiple groups of monitoring parameters within one period, for example, if multiple communication processes are included in the period, the UE needs to classify multiple monitoring parameters generated in each communication process into one group, and further feed back the multiple groups of monitoring parameters generated in the multiple communication processes to the first network element.

[0131] (2) The generation of the monitoring parameters to be fed back involves an interaction between the second network element and the first network element.

[0132] For example, when the first network element indicates to the second network element to feed back the input and / or output of the first module, i.e., when the monitoring parameters to be fed back are the input and / or output of the first module, the first network element uses the third indication information to indicate a specific communication process. The third indication information can be implemented in the following two ways (a) and (b).

[0133] (a) Optionally, in certain implementations, the third instruction information includes one or more of the following: the third instruction information indicates a first value of an execution time of a third module and a first value of an execution time of a fourth module of the communication link, the first value of the execution time of the third module being used to determine a first value of a first monitoring parameter, the first value of the execution time of the fourth module being used to determine a first value of a second monitoring parameter, and the first value of the first monitoring parameter corresponding to the first value of the second monitoring parameter; or The third instruction information indicates a first identifier of an input or a first identifier of an output of a third module of the communication link and a first identifier of an input or a first identifier of an output of a fourth module, the first identifier of the input or the first identifier of the output of the third module being used to determine a first value of a first monitoring parameter, the first identifier of the input or the first identifier of the output of the fourth module being used to determine a first value of a second monitoring parameter, and the first value of the first monitoring parameter corresponding to the first value of the second monitoring parameter.

[0134] The first value of the execution time of the third module may be understood as the value of the execution time of a module (e.g., module A) of the communication link before the first network element interacts with the second network element, and the first value of the execution time of the fourth module may be understood as the value of the execution time of a module (e.g., module B) of the communication link after the first network element interacts with the second network element.

[0135] Similarly, the first identifier of the input or the first identifier of the output of the third module may be understood as the identifier of the input or the output of the module of the communication link before the first network element interacts with the second network element, and the first identifier of the input or the first identifier of the output of the fourth module may be understood as the identifier of the input or the output of the module of the communication link after the first network element interacts with the second network element.

[0136] If the second network element determines, based on the third instruction information, that the value of the monitoring parameter that needs to be fed back is generated before the interaction, the value of the monitoring parameter may be determined based on a first value of the execution time of the third module or a first identifier of the input or a first identifier of the output of the third module, or if the value of the monitoring parameter that needs to be fed back is generated after the interaction, the value of the monitoring parameter may be determined based on a first value of the execution time of the fourth module or a first identifier of the input or a first identifier of the output of the fourth module.

[0137] It should be understood that the first monitoring parameter is an example of a monitoring parameter to be fed back, and the value of the first monitoring parameter is generated before the interaction. The second monitoring parameter is an example of a monitoring parameter to be fed back, and the value of the second monitoring parameter is generated after the interaction. It should be noted that in this application, the terms "monitoring parameter" and "value of the monitoring parameter" may be used interchangeably and do not limit the understanding of the solution. For example, a monitoring parameter A corresponding to a communication process may be understood as a specific value of the monitoring parameter A in the communication process. Alternatively, it should be understood that if the monitoring parameter A needs to be fed back after the connection relationship between modules of the communication link is determined, a value corresponding to the monitoring parameter A in the specific communication process actually needs to be fed back.

[0138] In other words, the first value of the execution time of the third module, or the first identifier of the input or the first identifier of the output of the third module, and the first value of the execution time of the fourth module, or the first identifier of the input or the first identifier of the output of the fourth module, correspond to the same communication process. The first monitoring parameter (or the value of the first monitoring parameter) generated in the communication process needs to be determined based on related information of the third module (specifically, for example, the first value of the execution time of the third module, or the first identifier of the input or the first identifier of the output of the third module). The second monitoring parameter (or the value of the second monitoring parameter) generated in the communication process needs to be determined based on related information of the fourth module (specifically, for example, the first value of the execution time of the fourth module, or the first identifier of the input or the first identifier of the output of the fourth module).

[0139] FIG. 7 is used as an example. The base station indicates to the UE to feed back multiple monitoring parameters, including a channel estimation result and a received signal-to-interference ratio (SIR), used to monitor the performance of the precoding module. The base station may indicate a feedback time t1 of compression information corresponding to the channel estimation result that needs to be fed back by the UE and a time t2 at which a precoding matrix obtained based on the fed-back channel information begins to be used for downlink transmission. Furthermore, the UE may determine a received signal-to-interference ratio (SIR) after time t2 at which the channel estimation result and the precoding matrix belonging to the same communication process as the feedback time t1 of the compression information begin to be used for downlink transmission, and feed back the channel estimation result and the SIR to the base station. When the base station indicates N time point groups {t1, t2}, the UE needs to feed back monitoring parameters corresponding to the N time point groups, respectively. In this example, the channel estimation result is an example of a first monitoring parameter, and the SIR is an example of a second monitoring parameter. The channel compression module is an example of a pre-interaction module, and the precoding module is an example of a post-interaction module.

[0140] (b) Optionally, in another specific implementation, the third instruction information indicates a communication process in the following manner:

[0141] The third indication information indicates a first value of an execution time and first time information of a third module of the communication link. The first value of the execution time and the first time information of the third module may indicate or be used to determine a first value of an execution time of a fourth module of the communication link. The first value of the execution time of the third module is used to determine a first value of a first monitoring parameter, and the first value of the execution time of the fourth module is used to determine a first value of a second monitoring parameter, and the first value of the first monitoring parameter corresponds to the first value of the second monitoring parameter.

[0142] Optionally, the first time information indicates a delay or a time difference.

[0143] In one example, the first time information indicates a delay. The delay is a delay between a first value of an execution time of a fourth module of the communication link and a first value of an execution time of a third module. In other words, the third instruction information indicates a first value of an execution time of a module (specifically, the third module) before the first network element interacts with the second network element and the delay information. The delay information is used to determine a first value of an execution time of a module (specifically, the fourth module) after the first network element interacts with the second network element.

[0144] In another example, the first time information indicates a time difference. The time difference is a time difference between a first value of an execution time of a third module and a first value of an execution time of a fourth module of the communication link. That is, the third instruction information indicates a first value of an execution time of a module (specifically, the third module) after the first network element interacts with the second network element and the time difference information. The time difference information is used to determine a first value of an execution time of a module (specifically, the fourth module) before the first network element interacts with the second network element.

[0145] FIG. 7 is still used as an example. For example, the base station instructs the UE to feed back multiple monitoring parameters. The multiple monitoring parameters include a channel estimation result and a received signal-to-interference ratio (SIR) used to monitor the performance of the precoding module. The base station may indicate a feedback time t1 of the compressed information corresponding to the channel estimation result that needs to be fed back by the UE, and a delay T between the time when the precoding matrix obtained based on the fed back channel information begins to be used for downlink transmission and the feedback time t1 of the compressed information. That is, the base station indicates t1+T to the UE. Furthermore, the UE may determine a channel estimation result belonging to the same communication process as the feedback time t1 of the compressed information and a received SIR delayed by T compared to the feedback time t1 of the compressed information, and feed back the two monitoring parameters to the base station.

[0146] Optionally, the delay T may be a value indicating that the compressed information fed back at time t1 and the signal-to-interference ratio received at time t1+T belong to the same communication process, or the delay T may be a real interval [T1, T2] indicating that the compressed information fed back at time t1 is used to transmit a signal within the period [t1+T1, t1+T2]. Therefore, the compressed information fed back at time t1 and the signal received in the period [t1+T1, t1+T2] belong to the same communication process. In this case, the UE needs to feed back a monitoring parameter (e.g., a received signal-to-interference ratio) in the period [t1+T1, t1+T2]. The feedback format may be full feedback, feedback of an average value of the monitoring parameter in the period, etc. When the base station indicates N time point information, the UE needs to feed back monitoring parameters corresponding to the N time point information, respectively.

[0147] In another example, the base station indicates the execution time t1 of module A after the interaction or the ID of the input of module A. In addition, the base station indicates the time difference between the execution time of module B before the interaction and the execution time t1 of module A. Furthermore, the UE may determine the value of a monitoring parameter (e.g., a first monitoring parameter) belonging to the same communication process as the time t1 or the ID of the input of module A.

[0148] Optionally, the time difference T may be a value indicating that module B executed at time t1-T and module A executed at time t1 belong to the same communication process, or may be a real interval [T1, T2] indicating that the result of executing module B in a certain period [t1-T1, t1-T2] jointly affects the execution of module A at time t1. Therefore, the related monitoring parameter (e.g., first monitoring parameter) generated before the interaction in the time period [t1-T1, t1-T2] and the related monitoring parameter (e.g., second monitoring parameter) corresponding to time t1 belong to the same communication process. In this case, the UE needs to feed back the related monitoring parameter generated before the interaction in the certain period [t1-T1, t1-T2], i.e., there are multiple first monitoring parameters. The feedback form may be full feedback, feedback of the average value of multiple first monitoring parameters, etc., but is not limited thereto.

[0149] 303: The first network element monitors and / or trains one or more AI modules based on the input and / or output of the first module. Optionally, the one or more AI modules are deployed on the first network element or the second network element. Alternatively, the one or more first AI modules include a first sub-AI module and a second sub-AI module, where the first sub-AI module is deployed on the first network element and the second sub-AI module is deployed on the second network element, and the first sub-AI module and the second sub-AI module are used in the matching scheme. It should be understood that the first sub-AI module and the second sub-AI module here are two sub-models in a dual-end model. An autoencoder (AE) model can generally refer to a network structure including two sub-models. The AE model may also be referred to as a bilateral model, a dual-end model, or a cooperative model. The encoder and decoder of the AE are usually trained together and may be used in the matching scheme. For example, AI-based CSI feedback is a dual-end model. For example, the UE side uses an encoder to compress and quantize the CSI, and the access network device uses a decoder to restore the CSI. For the access network device, the input of the AI ​​model is the CSI fed back by the UE side, and the output is the restored CSI. However, for model training, the CSI measured by the UE side needs to be used as the truth value label of the restored CSI.

[0150] Specifically, the first network element analyzes the performance of the AI ​​module in the corresponding communication process based on the input and / or output of the first module. If the performance of the AI ​​module in one or more communication processes is lower than a threshold, it may be determined that the performance of the monitored AI module does not meet the requirements, and a model update or switchover needs to be performed. If the AI ​​module is located on the second network element, the first network element may use instruction information to instruct the second network element to update or switch the AI ​​module. In addition, if the first network element finds in the performance analysis process that a performance risk exists in the remaining related AI modules associated with the fed-back monitoring parameters, the first network element may also send instruction information to trigger performance monitoring of the related AI module or distribute instruction information to notify the second network element that a performance risk exists in the related AI module.

[0151] FIG. 9 is a diagram illustrating the analysis of the performance of an AI module by a first network element. Performance monitoring of a precoding module is used as an example. Assume that the first network element is a base station and the second network element is a UE. The base station can analyze the performance of the precoding module using the following method: When the monitoring parameters fed back by the UE indicate that the signal-to-interference ratio of the receiving end is low (the signal-to-interference ratio of the receiving end can be determined based on actual channel information and the precoding matrix obtained by using the precoding module), the base station can further obtain a signal-to-interference ratio of the receiving end predicted at the base station side based on the channel recovery information obtained by the channel recovery module and the precoding matrix obtained by the precoding module. If the fed-back signal-to-interference ratio of the receiving end is basically consistent with the signal-to-interference ratio of the receiving end predicted by the base station, it is considered that the performance of the precoding module is relatively low and an appropriate precoding matrix for effectively suppressing interference has not been obtained. If the fed-back signal-to-interference ratio of the receiving end is not consistent with the signal-to-interference ratio of the receiving end predicted by the base station, it is considered that the channel recovery information obtained by the channel recovery module is significantly different from the actual channel information. Therefore, the precoding module does not obtain an appropriate precoding matrix to effectively suppress interference. Therefore, the base station may further compare the fed-back channel estimation result with the channel recovery information obtained by the channel recovery module. If the similarity between the fed-back channel estimation result and the channel recovery information is high, the channel estimation result on the UE side is significantly different from the actual channel information, that is, the performance of the channel estimation module on the UE side is considered to be relatively low. Conversely, if the similarity between the fed-back channel estimation result and the channel recovery information obtained by the channel recovery module is low, the performance of the channel compression module and the channel recovery module is considered to be relatively low.

[0152] Optionally, the first network element may also analyze the performance of multiple AI modules in the corresponding communication process based on the input and / or output of the first module. If the multiple monitored AI modules include an AI module that does not meet the requirements, the AI ​​module that does not meet the requirements may be updated or switched. For example, if the performance of an AI module in one or more communication processes is lower than a threshold, it may be determined that the performance of the AI ​​module does not meet the requirements, and a model update or switch needs to be performed. When the model is deployed on the second network element, the first network element may send instruction information to the second network element indicating to switch or update the AI ​​module, or prompt the second network element that the AI ​​module has a performance risk.

[0153] Figure 10 is another diagram of the analysis of the performance of an AI module by a first network element. An example is shown in which the base station feeds back to the UE the demodulation accuracy, the input of the demodulation module, the channel estimation result, and the received signal-to-interference ratio. The base station may analyze the performance of the AI ​​model using the following method: If the monitoring parameters fed back by the UE indicate low demodulation accuracy, the base station may further compare the input of the demodulation module at the receiving end with the output of the modulation module at the base station. If the difference between the input of the demodulation module at the receiving end and the output of the modulation module at the base station is small, the performance of the modulation and demodulation module is considered to be relatively low and the modulation and demodulation effect is not good. If the difference between the input of the demodulation module at the receiving end and the output of the modulation module at the base station is large, the demodulation accuracy is considered to be low due to the poor performance of the intermediate process of the modulation and demodulation module. Therefore, the base station may further analyze the performance of the precoding module, channel estimation module, compression module, and feedback module based on the channel estimation result and the received signal-to-interference ratio. If the performance of these AI modules is normal, the performance of the equalization module at the UE is considered to be relatively low. Conversely, if the similarity between the fed back channel estimation result and the channel recovery information obtained by the channel recovery module is low, the performance of the channel compression module and the channel recovery module is considered to be relatively poor.

[0154] In this example, the second network element feeds back the monitoring parameters once, which may be used for performance monitoring of multiple AI modules, thereby reducing the feedback overhead of performance monitoring of the AI ​​modules.

[0155] 9 and 10, the above describes an example of applying the technical solution provided in the present application to AI model performance monitoring. The following describes an example of performing AI model training based on monitoring parameters.

[0156] The connection relationship of the modules of the communication link shown in Figure 7 is used as an example for explanation. In addition, an example in which the first network element is a base station, the second network element is a UE, and the base station performs model training is used for explanation.

[0157] The training of a precoding module is used as an example. The training data includes input and labels. The input of the precoding module is the channel recovery information output by the channel recovery module on the base station side, and the label output by the precoding module includes the channel estimation result fed back by the UE. The channel recovery information is associated with the channel estimation result. The training process is as follows: the channel recovery information is input to the precoding module, the precoding result is output, and the signal-to-interference ratio is calculated based on the precoding result and the channel estimation result. If the signal-to-interference ratio is lower than the threshold, the weights of the precoding module are updated to increase the signal-to-interference ratio until the signal-to-interference ratio meets the requirement.

[0158] Furthermore, an example in which the precoding module and the channel recovery module are trained together is used as an example. The training data includes inputs and labels. The channel recovery module and the precoding module are considered as a whole. The input of the model is the channel compression information fed back by the channel compression module on the UE side. The label of the model includes the channel estimation result fed back by the UE. The channel compression information is associated with the channel estimation result. The training process is as follows: the channel compression information is input to the channel recovery module to obtain the channel recovery information, and then the channel recovery information is input to the precoding module to output the precoding result. A signal-to-interference ratio is calculated based on the precoding result and the channel estimation result, and the difference between the channel recovery information and the channel estimation result is multiplied by -1, which is represented as E (a negative number, with a larger value indicating higher recovery accuracy). If the weighted sum of the signal-to-interference ratio and E is less than a threshold, the weights of the precoding module and the weights of the channel recovery module are updated to increase the weighted sum of the signal-to-interference ratio and E until the weighted sum of the signal-to-interference ratio and E meets the requirement.

[0159] Referring to Figure 10, in the performance analysis shown in Figure 10, if it is determined based on the analysis results that the performance of the modulation module and demodulation module is relatively low and the AI ​​model needs to be updated, the following steps may be performed.

[0160] The training of the modulation module and the demodulation module is used as an example. The training data includes input and label. The input of the modulation module is a modulated symbol on the base station side (including a modulated symbol associated with the UE feedback on the base station side), and the label is the same as the input. The base station inputs the modulated symbol to the modulation module, and then adds random noise to the output of the modulation module to input it to the demodulation module designed by the base station side to obtain a restored symbol. The base station calculates the difference between the restored symbol and the input symbol. If the difference is greater than a threshold, the weights of the modulation module and the demodulation module are updated to reduce the difference until the difference is less than the threshold. After the modulation module is trained or updated, the input in the training data and the output of the corresponding modulation module are delivered to the UE, so that the UE updates or trains the modulation module of the UE.

[0161] If, based on the analysis results, it is determined that the channel compression module or precoding module has relatively poor performance and needs to be updated, please refer to the above exemplary description of training the channel compression module or precoding module.

[0162] Example 2 A first network element obtains inputs and / or outputs of a module deployed on the first network element and provides inputs and / or outputs for a second network element to monitor and / or train an AI module of the second network element.

[0163] FIG. 11 illustrates another example of supervising or training an AI module according to the present application.

[0164] 501: A first network element receives second instruction information from a second network element, the second instruction information indicating one or more monitoring parameters, the one or more monitoring parameters including an input and / or an output of a second module.

[0165] The second module may be any module in the communication link, and the second module may be a non-AI module or an AI module, without limitation.

[0166] 502: A first network element obtains an input and / or an output of a second module.

[0167] 503: The first network element sends second information to the second network element, where the second information indicates the input and / or output of the second module.

[0168] The inputs and / or outputs of the second module are used to monitor or train one or more AI modules. The one or more AI modules are deployed on a second network element. For clarity, all of the one or more AI modules are referred to as second AI modules. The inputs and / or outputs of the one or more second AI modules are determined based on the inputs and / or outputs of the second module, or the inputs and / or outputs of the second module are determined based on the inputs and / or outputs of the one or more second AI modules.

[0169] 504: The second network element monitors and / or trains one or more second AI modules based on the inputs and / or outputs of the second modules.

[0170] In this example, the first network element obtains one or more monitoring parameters (which may include, for example, inputs and / or outputs of the second module) based on instructions from the second network element and provides the one or more monitoring parameters to the second network element, which are used by the second network element to monitor and / or train one or more AI modules of the second network element.

[0171] For a specific implementation form of using the second indication information to indicate the monitoring parameters that need to be fed back by the second network element to the first network element, and using the second information by the first network element to feed back the monitoring parameters to the second network element, please refer to the description or examples in Example 1. Details will not be described again.

[0172] Example 3 In the example, the first network element is a base station and the second network element is a UE. The base station indicates to the UE to provide one or more monitoring parameters used to monitor or train the performance of one or more AI models of an uplink communication link.

[0173] FIG. 12 illustrates another example of supervising or training an AI module according to the present application.

[0174] 601: A base station sends first indication information to a UE, where the first indication information indicates one or more monitoring parameters.

[0175] The one or more monitored parameters include inputs and / or outputs of one or more modules of the communication link.

[0176] For a specific implementation in which the first indication information indicates one or more monitoring parameters, please refer to the description or examples in Example 1.

[0177] For example, the first indication information may indicate one of a plurality of predetermined feedback options, such as feedback option 1, in which the channel recovery result is fed back and used for coarse monitoring of the channel compression module and the channel recovery module, or feedback option 2, in which the channel recovery result and modulation symbols are fed back and used for performance monitoring of the precoding module, the channel compression module, the channel recovery module, and the equalization module.

[0178] For example, the monitoring parameters that the base station indicates to the UE to feed back include channel recovery results and modulation symbols. The base station may indicate to the UE to feed back modulation symbols in a specific data frame. Furthermore, the UE may determine channel recovery results belonging to the same communication process as the modulation symbols to be fed back and feed back the channel recovery results to the base station for AI model performance monitoring. When the base station indicates N time point information, the UE needs to feed back monitoring parameters corresponding to the N time point information, respectively.

[0179] 602: The base station receives first information from the UE, where the first information indicates values ​​of one or more monitored parameters.

[0180] As described above, when there are multiple monitoring parameters, the multiple monitoring parameters belonging to the same communication process are classified into one group. In addition, when there are multiple communication processes, the first information indicates the values ​​of the multiple groups of monitoring parameters, and the value of each group of monitoring parameters corresponds to one communication process.

[0181] 603: The base station monitors or trains one or more AI modules of the uplink communication link based on the values ​​of the one or more monitored parameters.

[0182] Optionally, if there is an AI module in the monitored AI module whose performance does not meet the requirements, the AI ​​model may be switched or updated. If an AI model that does not meet the requirements is deployed on the UE side, the base station may deliver instruction information to the UE to instruct it to update or switch the AI ​​model, or deliver instruction information to notify the UE of an AI model with a performance risk.

[0183] An example is used in which the base station indicates to the UE to feed back feedback option 2 in step 601. The base station may analyze the AI ​​model performance using the following method: when the demodulation accuracy of the frame on the base station side is low, the base station may further analyze the received signal strength. If the received signal strength is significantly different from the predicted received signal strength calculated by the base station side based on the estimated uplink channel, the precoding performance on the UE side is low. Otherwise, if the received signal strength is not significantly different from the predicted received signal strength calculated by the base station side based on the estimated uplink channel, the performance of the equalization module on the base station side is considered to be low. If the precoding performance on the UE side is relatively low, the channel recovery result fed back by the UE is further compared with the channel estimated by the base station side based on the uplink reference signal. If the channel recovery result on the UE side is significantly different from the uplink channel estimated by the base station side but close to the past uplink channel, the channel compression feedback delay is considered to be relatively large. If the channel recovery result on the UE side is significantly different from the uplink channel estimated by the base station side but not close to the past uplink channel, the performance of the channel compression and recovery module is considered to be relatively low. If the channel recovery result at the UE side is close to the uplink channel estimated by the base station side, the performance of the precoding module at the UE side is considered to be relatively poor.

[0184] In this example, the base station indicates to the UE to feed back multiple monitoring parameters belonging to the same uplink communication process, and simultaneously monitors the performance of multiple AI models, and each monitoring parameter may be affected by the performance of multiple AI models. Although each monitoring parameter is fed back only once, the monitoring parameter may be used to monitor the performance of multiple AI models, thereby reducing the feedback overhead of AI model performance monitoring.

[0185] The above examples 1 to 3 separately describe the application of the technical solutions of the present application in different cases. It should be understood that these examples may alternatively be combined with each other. For example, when example 1 is combined with example 2, a first network element obtains inputs and / or outputs of a first module and monitors or trains one or more first AI modules. In addition, the first network element may further provide inputs and / or outputs of a second module for the second network element based on instructions from the second network element, so that the second network element monitors or trains one or more second AI modules. When example 2 is combined with example 3, the first network element provides inputs and / or outputs of a second module for the second network element based on instructions from the second network element, so that the second network element monitors or trains one or more second AI modules. Additionally, the first network element indicates to the second network element to feed back the inputs and / or outputs of one or more modules of the communication link, so that the first network element monitors or trains one or more AI modules of the uplink communication link.

[0186] Based on the technical solutions provided in the present application and the foregoing examples, those skilled in the art may further understand the use of the technical solutions in another application scenario, for example, monitoring the performance of one or more AI modules of a downlink communication link or training one or more AI modules based on the input / output (i.e., one or more monitoring parameters) of one or more modules of the communication link. In the embodiments of the present application, details are not listed one by one.

[0187] The above describes in detail the embodiments of the method for monitoring or training an AI model provided in the present application. The following describes the communication device provided in the present application.

[0188] See Figure 13. The present application provides a communication device 1000.

[0189] As shown in FIG. 13 , the communication device 1000 includes a processing module 1001 and a communication module 1002. The communication device 1000 may be a terminal device, or a communication device, such as a chip, chip system, or circuit, used in a terminal device or used in a matching manner with a terminal device and capable of implementing a method executed on the terminal device side. Alternatively, the communication device 1000 may be a network device, or a communication device, such as a chip, chip system, or circuit, used in a network device or used in a matching manner with a network device and capable of implementing a method executed on the network device side. For example, the network device may be an access network device in an embodiment of the method of the present application.

[0190] The communication module may also be referred to as a transceiver module, a transceiver, a transceiver device, a transceiver apparatus, etc. The processing module may also be referred to as a processor, a processing board, a processing unit, a processing device, etc. Optionally, the communication module is configured to perform transmitting and receiving operations at the terminal device side or the network device side in the manner described above. A component configured to perform a receiving function in the communication module may be considered a receiving unit, and a component configured to perform a transmitting function in the communication module may be considered a transmitting unit. In other words, the communication module includes a receiving unit and a transmitting unit.

[0191] When the communication apparatus 1000 is used in a terminal device, the processing module 1001 may be configured to perform the processing functions of the terminal device in the embodiments of Figures 3 to 12, and the communication module 1002 may be configured to perform the receiving and transmitting functions of the terminal device in the embodiments of Figures 3 to 12.

[0192] When the communication apparatus 1000 is used in a network device, the processing module 1001 may be configured to perform the processing functions of the network device (e.g., a positioning device or an access network device) in the embodiments of Figures 3 to 12, and the communication module 1002 may be configured to perform the receiving and transmitting functions of the network device in the embodiments of Figures 3 to 12.

[0193] Additionally, it should be noted that the communication module and / or the processing module may be implemented using virtual modules. For example, the processing module may be implemented using a software functional unit or a virtual device, and the communication module may be implemented using a software function or a virtual device. Alternatively, the processing module or the communication module may be implemented using an entity device. For example, if the device is implemented using a chip / chip circuit, the communication module may be an input / output circuit and / or a communication interface, and may perform input operations (corresponding to the receiving operations described above) and output operations (corresponding to the transmitting operations described above). The processing module may be an integrated processor, a microprocessor, or an integrated circuit.

[0194] The division into modules in this application is merely an example, and is merely a division into logical functions, and other divisions may be used in actual implementation. In addition, the functional modules in the examples of this application may be integrated into one processor, each module may exist physically alone, or two or more modules may be integrated into one module. The integrated module may be implemented in the form of hardware or in the form of a software functional module.

[0195] See Figure 14. The present application further provides a communication device 1100. Optionally, the communication device 1100 may be a chip or a chip system. Optionally, in the present application, a chip system may include a chip, or may include a chip and other discrete components.

[0196] The communications device 1100 may be configured to perform the functions of any network element (e.g., a terminal device, a network device, or an AI entity) in the communications system described in the previous examples. The communications device 1100 may include at least one processor 1110. Optionally, the processor 1110 is coupled to a memory. The memory may be located within the device. Alternatively, the memory may be integrated with the processor. Alternatively, the memory may be located external to the device. For example, the communications device 1100 may further include at least one memory 1120. The memory 1120 stores computer programs, computer programs or instructions, and / or data necessary to implement any of the previous examples. The processor 1110 may execute computer programs stored in the memory 1120 to accomplish the method of any of the previous examples.

[0197] The communication device 1100 may further include a communication interface 1130, and the communication device 1100 may exchange information with another device via the communication interface 1130. For example, the communication interface 1130 may be a transceiver, a circuit, a bus, a module, a pin, or another type of communication interface. When the communication device 1100 is a chip-type device or circuit, the communication interface 1130 in the device 1100 may alternatively be an input / output circuit, which may input information (also referred to as received information) and output information (also referred to as transmitted information). The processor may be an integrated processor, a microprocessor, an integrated circuit, or a logic circuit. The processor may determine output information based on the input information.

[0198] The term "coupling" in this application may refer to an electrical, mechanical, or other form of indirect coupling or communication connection between devices, units, or modules, and is used for information exchange between the devices, units, or modules. The processor 1110 may operate in cooperation with the memory 1120 and the communication interface 1130. The specific connection medium between the processor 1110, the memory 1120, and the communication interface 1130 is not limited in this application.

[0199] Optionally, as shown in Figure 13, the processor 1110, memory 1120, and communication interface 1130 are connected to each other using a bus 1140. The bus 1140 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be classified into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line represents a bus in Figure 13, but this does not mean that there is only one bus or only one type of bus.

[0200] In this application, a processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, which may implement or perform the methods, steps, and logic block diagrams disclosed in this application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed with reference to this application may be implemented directly by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor.

[0201] In this application, memory may be non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), or volatile memory, such as random access memory (RAM). Memory is any medium capable of holding or storing program code in the form of instructions or data structures and accessible by a computer, but is not limited to such. Memory in this application may alternatively be a circuit or any other device capable of performing storage functions and configured to store program instructions and / or data.

[0202] In a possible implementation, the communication device 1100 may be used on a network device side, such as an access network device in an embodiment of the present application, or a host or cloud device in an over-the-top (OTT) system. Specifically, the communication device 1100 may be a network device or a device capable of supporting a network device in performing corresponding functions on the network device side of any of the aforementioned examples. The memory 1120 stores computer programs (or instructions) and / or data for performing functions on the network device side of any of the aforementioned examples. To accomplish a method performed by the network device side of any of the aforementioned examples, the processor 1110 may execute a computer program stored in the memory 1120. A communication interface in the communication device 1100 may be configured to interact with a terminal device, transmit information to the terminal device, or receive information from the terminal device.

[0203] In another possible implementation, the communication device 1100 may be used in a terminal device. Specifically, the communication device 1100 may be a terminal device or a device capable of supporting a terminal device in performing the functions of the terminal device in any of the foregoing examples. The memory 1120 stores computer programs (or instructions) and / or data for performing the functions of the terminal device in any of the foregoing examples. To accomplish the method performed by the terminal device in any of the foregoing examples, the processor 1110 may execute the computer program stored in the memory 1120. The communication interface in the communication device 1100 may be configured to interact with a network device side (e.g., an access network device) and transmit information to or receive information from the network device side.

[0204] The communication device 1100 provided in this example may be used in a network device (e.g., an access network device) to complete a method performed by the network device, or may be used in a terminal device to complete a method performed by the terminal device. Therefore, for technical effects that can be achieved by this embodiment, please refer to the aforementioned method embodiment. Details will not be repeated here.

[0205] Based on the above example, the present application provides a communication system. In one example, the communication system includes a first network element and a second network element. For example, the first network element is an access network device, and the second network element is a terminal. For example, both the first network element and the second network element are network devices, or both the first network element and the second network element are terminal devices. The communication system may implement the AI ​​model monitoring or training method provided in the embodiments shown in Figures 3 to 10.

[0206] All or part of the technical solutions provided in this application may be implemented using software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the technical solutions may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded into a computer and executed, all or part of the procedures or functions described in this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, a terminal device, an access network device, or another programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, or digital subscriber line (DSL)) or wireless (e.g., infrared, radio, or microwave) methods. The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that consolidates one or more available media. The available medium may be magnetic media (e.g., floppy disks, hard disk drives, or magnetic tapes), optical media (e.g., digital video discs (DVDs)), or semiconductor media, etc.

[0207] In this application, cross-references may be made between examples without logical contradiction, for example, between methods and / or terms in method embodiments, between functions and / or terms in apparatus embodiments, and between functions and / or terms in apparatus examples and method examples.

[0208] The units described as separate parts may or may not be physically separate, and the parts shown as units may or may not be physical units, and may be located in one place or distributed over multiple network units. Some or all of the units may be selected based on actual requirements to achieve the objectives of the solutions of the embodiments.

[0209] In addition, the functional units in the embodiments of the present application may be integrated into one processing unit, or each of the units may exist physically alone, or two or more units may be integrated into one unit.

[0210] In the embodiments of the present application, "at least one (item)" refers to one or more (items). "Multiple (items)" refers to two (items) or three or more (items). The term "and / or" describes an association relationship between associated objects and indicates that three relationships may exist. For example, A and / or B may refer to the following three cases: a case where only A exists, a case where both A and B exist, and a case where only B exists. The character " / " generally indicates an "or" relationship between associated objects. In addition, although terms such as "first" and "second" may be used to describe objects in the present disclosure, it should be understood that these objects are not limited by these terms. These terms are used merely to distinguish between objects.

[0211] The term "comprises" and any other variations thereof referred to in the embodiments of this application are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally further include other unlisted steps or units, or may optionally further include other inherent steps or units of the process, method, product, or device. It should be noted that in this application, words such as "example" or "for example" represent providing an example, illustration, or explanation. A method or design solution described in this application as an "example" or "for example" should not be described as preferred or advantageous over another method or design solution. Rather, the use of words such as "example" or "for example" is intended to present a relative concept in a concrete manner.

[0212] When a function is implemented in the form of a software functional unit and sold or used as an independent product, the function may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application may essentially be implemented in the form of a software product, or a portion of the technical solution may be implemented in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for instructing a computer device (which may be a personal computer, a server, or a network device) to perform all or part of the steps of the method described in the embodiments of the present application. The aforementioned storage medium includes any medium capable of storing program code, such as a USB flash drive, a removable hard disk drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or a compact disk.

[0213] The above description is merely a specific implementation form of the present application and does not limit the protection scope of the present application. Any variations or replacements that can be easily conceived by those skilled in the art within the technical scope disclosed in the present application shall fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims. [Explanation of symbols]

[0214] 110 Network Devices 120 Terminal Devices 130 Terminal Devices 140 AI entities 1000 Communication Equipment 1001 Processing Module 1002 communication module 1100 Communication equipment 1110 processor 1120 memory 1130 Communication Interface 1140 Bus

Claims

1. A method for monitoring or training an artificial intelligence (AI) model applied to a first network element or a chip of said first network element, said method comprising: obtaining inputs and / or outputs of a first module, said first module being an AI or non-AI module of a communications link; monitoring and / or training one or more first AI modules of the communication link based on the inputs and / or outputs of the first modules; Including, the inputs and / or outputs of the one or more first AI modules are determined based on the inputs and / or outputs of the first modules, or the inputs and / or outputs of the first modules are determined based on the inputs and / or outputs of the one or more first AI modules; method.

2. The inputs and / or outputs of the one or more first AI modules are determined based on the inputs and / or outputs of the first modules, or the inputs and / or outputs of the first modules are determined based on the inputs and / or outputs of the one or more first AI modules. the identifiers of the corresponding inputs and / or outputs of the one or more first AI modules correspond to the identifiers of the inputs and / or outputs of the first module; or the corresponding execution time values ​​of the one or more first AI modules correspond to the execution time values ​​of the first modules; 2. The method of claim 1, comprising:

3. the first module is deployed on the first network element or a second network element, and the one or more first AI modules are deployed on the first network element; or the first module is deployed on the first network element or the second network element, and the one or more first AI modules are deployed on the second network element; or the first module is deployed on the first network element or the second network element, the one or more first AI modules include a first sub-AI module and a second sub-AI module, the first sub-AI module is deployed on the first network element and the second sub-AI module is deployed on the second network element, and the first sub-AI module and the second sub-AI module are used in a matching method; 3. The method according to claim 1 or 2.

4. the first module is deployed on a second network element; The step of obtaining the input and / or output of the first module comprises: receiving first information from the second network element, the first information indicating the inputs and / or outputs of the first module; Including, 4. The method according to any one of claims 1 to 3.

5. Before receiving the first information from the second network element, the method further comprises: sending a first indication to the second network element, the first indication indicating one or more monitored parameters, the one or more monitored parameters including the inputs and / or outputs of the first module; 5. The method of claim 4, further comprising:

6. The method comprises: obtaining inputs and / or outputs of a second module, said second module being an AI or non-AI module of said communication link; sending second information to a second network element, the second information indicating the inputs and / or outputs of the second module, the inputs and / or outputs of the second module being determined based on the inputs and / or outputs of a second AI module of the communication link, or the inputs and / or outputs of a second AI module of the communication link being determined based on the inputs and / or outputs of the second module; 6. The method of claim 1, further comprising:

7. Before transmitting the second information to the second network element, the method further comprises: receiving second indications from the second network element, the second indications indicating one or more monitored parameters, the one or more monitored parameters including the inputs and / or outputs of the second module; 7. The method of claim 6, further comprising:

8. the first indication is indicative of the one or more monitored parameters; the first indication information indicates a first index, the first index being one of the indexes included in a predefined correspondence relationship, the correspondence relationship indicating a correspondence relationship between a monitoring parameter and / or a combination of monitoring parameters and an index, the combination of monitoring parameters including two or more monitoring parameters; or the first indication indicates an identifier of the one or more monitoring parameters, the identifier of the one or more monitoring parameters being one or more identifiers within an identifier of a predetermined group of monitoring parameters; 8. The method of any one of claims 5 to 7, comprising:

9. 9. The method of claim 8, wherein the first instruction information further includes identifiers of the one or more first AI modules, and the one or more monitoring parameters indicated by the first instruction information are used to monitor the one or more first AI modules.

10. The method comprises: sending third indication information to the second network element; further comprising The third instruction information is the following information: one or more values ​​of an execution time of a third module of the communication link, the value of the execution time of the third module corresponding to the value of the execution time of the first module; or one or more identifiers of inputs of the third module of the communication link, the identifiers of the inputs of the third module corresponding to the identifiers of the inputs and / or outputs of the first module; or one or more identifiers of the outputs of the third module of the communication link, the identifiers of the outputs of the third module corresponding to the identifiers of the inputs and / or outputs of the first module; Indicates one of the 8. The method according to any one of claims 5 to 7.

11. the first indication is indicative of the one or more monitored parameters; the first indication indicates a plurality of monitoring parameters, the plurality of monitoring parameters including a first monitoring parameter and a second monitoring parameter; and the third indication indicates a first value of the execution time of the third module of the communication link and a first value of the execution time of a fourth module of the communication link; the first value of the execution time of the third module is used to determine a first value of the first monitoring parameter, and the first value of the execution time of the fourth module is used to determine a first value of the second monitoring parameter, and the first value of the first monitoring parameter corresponds to the first value of the second monitoring parameter; or the third indication indicates a first identifier of the input or a first identifier of the output of the third module of the communication link and a first identifier of the input or a first identifier of the output of the fourth module of the communication link; The first identifier of the input or the first identifier of the output of the third module is used to determine a first value of the first monitoring parameter, and the first identifier of the input or the first identifier of the output of the fourth module is used to determine a first value of the second monitoring parameter, and the first value of the first monitoring parameter corresponds to the first value of the second monitoring parameter.

11. The method of claim 10, comprising:

12. the first indication is indicative of the one or more monitored parameters; the first indication information indicates a plurality of monitoring parameters, the plurality of monitoring parameters including a first monitoring parameter and a second monitoring parameter; the third instruction information indicates a first value of the execution time and first time information of the third module of the communication link, and the first value of the execution time and the first time information of the third module indicates a first value of the execution time of a fourth module of the communication link; The first value of the execution time of the third module is used to determine a first value of the first monitoring parameter, and the first value of the execution time of the fourth module is used to determine a first value of the second monitoring parameter, and the first value of the first monitoring parameter corresponds to the first value of the second monitoring parameter.

11. The method of claim 10, comprising:

13. the first time information indicates a delay, the delay being a delay between the first value of the execution time of the fourth module and the first value of the execution time of the third module; or the first time information indicates a time difference, and the time difference is a time difference between the first value of the execution time of the third module and the first value of the execution time of the fourth module; The method of claim 12.

14. the first time information indicates a time point T, where T is a real number; or the first time information indicates a time interval [T1, T2], where T1 and T2 are both real numbers; 14. The method of claim 12 or 13.

15. 15. The method of claim 14, wherein when the first time information indicates the time interval [T1, T2], the second monitoring parameter has a plurality of first values, and the first values ​​of the first monitoring parameter correspond to the plurality of first values ​​of the second monitoring parameter.

16. A communication device comprising a module configured to implement the method according to any one of claims 1 to 15.

17. 1. A communication device comprising a processor, The processor is coupled to a memory, the processor being configured to invoke computer program instructions stored in the memory to perform the method of any one of claims 1 to 15. Communication equipment.

18. 16. A communications device comprising a processor and a communications interface, wherein the communications interface is configured to receive data and / or information and to transmit the received data and / or information to the processor, wherein the processor processes the data and / or information, and wherein the communications interface is further configured to output the data and / or information processed by the processor, such that the communications device performs a method according to any one of claims 1 to 15.

19. 16. A computer-readable storage medium having stored thereon instructions that, when executed on a computer, enable the computer to perform the method of any one of claims 1 to 15.

20. 16. A computer program product, wherein the computer-readable storage medium stores instructions that, when executed on a computer, enable the computer to perform the method of any one of claims 1 to 15.

21. A communication system comprising a communication device according to claim 16 or 17.

Images