Monitoring method and device of AI / ML model

Abstract

The embodiment of the invention provides a monitoring method and device for an AI / ML model. The method comprises the steps that network equipment receives an uplink reference signal sent by terminal equipment; and performing performance monitoring on the AI / ML model by using the uplink reference signal.

Description

AI / ML model monitoring method and device Technical Field

[0001] The embodiments of the present application relate to the field of communication technologies. Background Art

[0002] Multi-antenna technology has been widely adopted in LTE, LTE-A, and 5G NR systems. In particular, massive MIMO (massive MIMO) technology has been well supported in the 5G standard. It is foreseeable that massive MIMO will be even more widely used in the 5G-Advanced and 6G phases. Performance-enhancing massive and ultra-large-scale antenna technologies are also a research focus for next-generation mobile communication systems.

[0003] With the advancement of artificial intelligence (AI) and machine learning (ML) technologies, applying AI / ML to wireless communications to overcome the difficulties of traditional methods has become a current technological trend. The application of AI / ML models to wireless communication systems, particularly in air interface transmission, is a new technology in the 5G-Advanced and 6G phases.

[0004] For example, for channel state information (CSI) measurement and feedback, the terminal device measures CSI and uses an AI / ML model to generate CSI feedback information. This information is then sent to the network via the air interface. The network then receives the CSI feedback information and uses the corresponding AI / ML model to recover the original CSI. In this example, the AI / ML model can reduce CSI feedback overhead or improve feedback quality, thereby improving communication quality.

[0005] For example, applying AI / ML technologies to beam management can improve beam management quality, accelerate beam scanning, and reduce or eliminate beam failures. Applying AI / ML technologies to positioning can enhance positioning accuracy. Other applications, such as channel estimation, beamforming, mobility management, and modem, can all leverage AI / ML to improve performance, reduce overhead, and deliver gains in reliability and latency.

[0006] It should be noted that the above introduction to the technical background is merely intended to provide a clear and complete description of the technical solutions of this application and facilitate understanding by those skilled in the art. Simply because these solutions are described in the background technology section of this application, it should not be assumed that the above technical solutions are well known to those skilled in the art.

[0007] Summary of the Invention

[0008] However, the inventors have discovered that AI / ML models may experience performance degradation and require performance monitoring. However, there is currently no solution for how to monitor this and what parameters to use.

[0009] To address at least one of the above problems, embodiments of the present application provide a method and apparatus for monitoring an AI / ML model.

[0010] According to one aspect of an embodiment of the present application, a method for monitoring an AI / ML model is provided, wherein a network device monitors the performance of the AI / ML model. The method includes:

[0011] The network device receives an uplink reference signal sent by the terminal device; and

[0012] The network device uses the uplink reference signal to monitor the performance of the AI / ML model.

[0013] According to another aspect of an embodiment of the present application, a monitoring device for an AI / ML model is provided, comprising:

[0014] a receiving unit, configured to receive an uplink reference signal sent by a terminal device; and

[0015] A monitoring unit is configured to perform performance monitoring on an AI / ML model using the uplink reference signal.

[0016] According to another aspect of an embodiment of the present application, a method for monitoring an AI / ML model is provided, wherein a terminal device monitors the performance of the AI / ML model, the method comprising:

[0017] The terminal device receives a downlink reference signal sent by the network device; and

[0018] The terminal device uses the downlink reference signal to monitor the performance of the AI / ML model.

[0019] According to another aspect of an embodiment of the present application, a device for monitoring an AI / ML model is provided, the device comprising:

[0020] a receiving unit, configured to receive a downlink reference signal sent by a network device; and

[0021] A monitoring unit is configured to perform performance monitoring on the AI / ML model using the downlink reference signal.

[0022] According to another aspect of an embodiment of the present application, a communication system is provided, including:

[0023] a terminal device that receives a downlink reference signal sent by a network device; and uses the downlink reference signal to monitor the performance of the AI / ML model; and / or

[0024] A network device that receives an uplink reference signal sent by a terminal device; and uses the uplink reference signal to monitor the performance of the AI / ML model.

[0025] One of the beneficial effects of the embodiments of the present application is that: a network device receives an uplink reference signal sent by a terminal device; and the network device uses the uplink reference signal to monitor the performance of the AI / ML model. Thus, appropriate label data can be obtained at a low cost, thereby supporting the monitoring of the AI / ML model.

[0026] With reference to the following description and accompanying drawings, specific embodiments of the present application are disclosed in detail, indicating the manner in which the principles of the present application can be employed. It should be understood that the embodiments of the present application are not limited in scope. Within the spirit and scope of the appended claims, the embodiments of the present application include many variations, modifications and equivalents.

[0027] Features described and / or illustrated with respect to one embodiment may be used in the same or similar manner in one or more other embodiments, combined with features in other embodiments, or substituted for features in other embodiments.

[0028] It should be emphasized that the term "include / comprising" when used herein refers to the presence of features, integers, steps or components, but does not exclude the presence or addition of one or more other features, integers, steps or components. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The elements and features described in one figure or one embodiment of the present application can be combined with the elements and features shown in one or more other figures or embodiments. In addition, in the accompanying drawings, similar reference numerals represent corresponding parts in several figures and can be used to indicate corresponding parts used in more than one embodiment.

[0030] FIG1 is a schematic diagram of a communication system according to an embodiment of the present application;

[0031] FIG2 is a schematic diagram of a monitoring method for an AI / ML model according to an embodiment of the present application;

[0032] FIG3 is a schematic diagram of an AI / ML model according to an embodiment of the present application;

[0033] FIG4 is an example diagram of monitoring performed by a network device according to an embodiment of the present application;

[0034] FIG5 is a schematic diagram of antenna port switching according to an embodiment of the present application;

[0035] FIG6 is an example diagram of signal extraction according to an embodiment of the present application;

[0036] FIG7 is a signaling example diagram of AI / ML model monitoring according to an embodiment of the present application;

[0037] FIG8 is an example diagram of a reference signal according to an embodiment of the present application;

[0038] FIG9 is a schematic diagram of the application of the bilateral model in the gNB and UE according to an embodiment of the present application;

[0039] FIG10 is a schematic diagram of the model of the embodiment of the present application applied to the gNB

[0040] FIG11 is a schematic diagram of a monitoring method for an AI / ML model according to an embodiment of the present application;

[0041] FIG12 is a signaling example diagram of AI / ML model monitoring according to an embodiment of the present application;

[0042] FIG13 is another signaling example diagram of AI / ML model monitoring according to an embodiment of the present application;

[0043] FIG14 is an example diagram of a terminal device performing model training according to an embodiment of the present application;

[0044] FIG15 is a schematic diagram of a UE application of a model according to an embodiment of the present application;

[0045] FIG16 is a schematic diagram of a monitoring device for an AI / ML model according to an embodiment of the present application;

[0046] FIG17 is a schematic diagram of a monitoring device for an AI / ML model according to an embodiment of the present application;

[0047] FIG18 is a schematic diagram of a network device according to an embodiment of the present application;

[0048] Figure 19 is a schematic diagram of a terminal device according to an embodiment of the present application. DETAILED DESCRIPTION

[0049] The above and other features of the present application will become apparent through the following description with reference to the accompanying drawings. In the description and the accompanying drawings, specific embodiments of the present application are disclosed in detail, which illustrate some embodiments in which the principles of the present application can be adopted. It should be understood that the present application is not limited to the described embodiments. On the contrary, the present application includes all modifications, variations and equivalents that fall within the scope of the appended claims.

[0050] In the embodiments of the present application, the terms "first", "second", etc. are used to distinguish different elements from the name, but do not indicate the spatial arrangement or temporal order of these elements, and these elements should not be limited by these terms. The term "and / or" includes any one and all combinations of one or more of the associated listed terms. The terms "comprising", "including", "having", etc. refer to the presence of the stated features, elements, components or components, but do not exclude the presence or addition of one or more other features, elements, components or components.

[0051] In the embodiments of this application, the singular forms "a," "the," etc. include plural forms and should be broadly understood to mean "a" or "a type" rather than being limited to "one." Furthermore, the term "said" should be understood to include both singular and plural forms, unless the context clearly indicates otherwise. Furthermore, the term "according to" should be understood to mean "at least in part based on...", and the term "based on" should be understood to mean "at least in part based on...", unless the context clearly indicates otherwise.

[0052] In the embodiments of the present application, the term "communication network" or "wireless communication network" may refer to a network that complies with any of the following communication standards, such as Long Term Evolution (LTE), enhanced Long Term Evolution (LTE-A, LTE-Advanced), Wideband Code Division Multiple Access (WCDMA), High-Speed ​​Packet Access (HSPA), etc.

[0053] Furthermore, communication between devices in the communication system may be carried out according to communication protocols of any stage, for example, including but not limited to the following communication protocols: 1G (generation), 2G, 2.5G, 2.75G, 3G, 4G, 4.5G and 5G, New Radio (NR), future 6G, etc., and / or other communication protocols currently known or to be developed in the future.

[0054] In the embodiments of the present application, the term "network device" refers to, for example, a device in a communication system that connects a terminal device to the communication network and provides services to the terminal device. Network devices may include, but are not limited to, the following devices: base station (BS), access point (AP), transmission reception point (TRP), broadcast transmitter, mobile management entity (MME), gateway, server, radio network controller (RNC), base station controller (BSC), etc.

[0055] Among them, base stations may include but are not limited to: NodeB (NodeB or NB), evolved NodeB (eNodeB or eNB) and 5G base station (gNB), IAB host, etc., and may also include remote radio head (RRH, Remote Radio Head), remote radio unit (RRU, Remote Radio Unit), relay (relay) or low-power node (such as femeto, pico, etc.). The term "base station" can include some or all of their functions. Each base station can provide communication coverage for a specific geographical area. The term "cell" can refer to a base station and / or its coverage area, depending on the context in which the term is used.

[0056] In the embodiments of the present application, the term "user equipment" (UE) or "terminal equipment" (TE) refers to, for example, a device that accesses a communication network through a network device and receives network services. A terminal device can be fixed or mobile and may also be referred to as a mobile station (MS), a terminal, a subscriber station (SS), an access terminal (AT), a station, and so on.

[0057] Among them, terminal devices may include but are not limited to the following devices: cellular phones, personal digital assistants (PDAs), wireless modems, wireless communication devices, handheld devices, machine-type communication devices, laptop computers, cordless phones, smart phones, smart watches, digital cameras, etc.

[0058] For another example, in scenarios such as the Internet of Things (IoT), the terminal device can also be a machine or device for monitoring or measurement, including but not limited to: machine type communication (MTC) terminal, vehicle-mounted communication terminal, device-to-device (D2D) terminal, machine-to-machine (M2M) terminal, and so on.

[0059] In addition, the term "network side" or "network device side" refers to one side of the network, which can be a base station or one or more network devices as described above. The term "user side" or "terminal side" or "terminal device side" refers to the user or terminal side, which can be a UE or one or more terminal devices as described above. Unless otherwise specified herein, "device" can refer to either network equipment or terminal equipment.

[0060] The following describes the scenarios of the embodiments of the present application through examples, but the present application is not limited thereto.

[0061] FIG1 is a schematic diagram of a communication system according to an embodiment of the present application, schematically illustrating a situation using a terminal device and a network device as an example. As shown in FIG1 , a communication system 100 may include a network device 101 and terminal devices 102 and 103. For simplicity, FIG1 illustrates only two terminal devices and one network device as an example, but the embodiments of the present application are not limited thereto.

[0062] In the embodiment of the present application, existing services or future services can be transmitted between the network device 101 and the terminal devices 102 and 103. For example, these services may include but are not limited to: enhanced mobile broadband (eMBB), massive machine type communication (mMTC), and ultra-reliable and low-latency communication (URLLC), etc.

[0063] It is worth noting that FIG1 shows that both terminal devices 102 and 103 are within the coverage range of network device 101, but the present application is not limited thereto. Both terminal devices 102 and 103 may not be within the coverage range of network device 101, or one terminal device 102 may be within the coverage range of network device 101 while the other terminal device 103 is outside the coverage range of network device 101.

[0064] In the embodiments of the present application, the high-layer signaling may be, for example, radio resource control (RRC) signaling; for example, an RRC message, including, for example, an MIB, system information, or a dedicated RRC message; or an RRC information element (RRC IE). The high-layer signaling may also be, for example, MAC (Medium Access Control) signaling; or a MAC control element (MAC CE). However, the present application is not limited thereto.

[0065] In embodiments of the present application, one or more AI / ML models may be configured and run in a network device and / or a terminal device. The AI / ML models may be used for various signal processing functions in wireless communications, such as CSI estimation and reporting, beam management, and beam prediction, etc.; the present application is not limited thereto.

[0066] Although the application of AI / ML can bring performance gains, unlike traditional methods based on communication theory and signal processing theory, its performance depends largely on the data set used for training and the data that needs to be processed in its application.

[0067] For example, it's common for the statistical characteristics of the training dataset to differ from the data characteristics of the application environment. This is especially true for various wireless environments, as well as the radio frequency and antenna components used on both the transmitter and receiver sides. These factors can be summarized as the equivalent wireless channel between the network device and the terminal device. This diversity of wireless channels makes it difficult to traverse all channels when building a neural network model. Consequently, when the model is applied to a real-world wireless environment, it often encounters wireless channels that were not encountered during training.

[0068] Especially for mobile communications, the terminal-side model experiences a wide variety of environments as it moves, including urban and suburban areas, densely populated areas with tall buildings, intertwined road networks, various indoor environments, and varying speeds. All of these can lead to model mismatch. Model mismatch refers to the statistical inconsistency between the data used to build the model and the data processed during application. This inconsistency can lead to a degradation in model performance, even to the point where performance falls short of traditional signal processing methods, or even becomes unacceptable.

[0069] To address this issue, model performance monitoring is necessary. If performance issues are identified, traditional signal processing methods can be switched to. Furthermore, the model can be continuously retrained or fine-tuned based on wireless data within the application environment to continuously optimize network parameters and even structure, and existing AI / ML models can be replaced when appropriate.

[0070] All of these operations require a reference signal or data, which can be called a ground truth label, or simply label data, or label signal, or sometimes referred to as meta data or signal. For a given model input, the expected model output is defined as a label. The difference between the model's actual output and the label is used to judge the model's output performance or train the model.

[0071] However, the inventors have discovered that, in practical communication systems, obtaining absolutely accurate label signals or label data is often difficult or requires significant overhead. Therefore, obtaining practical and low-cost label signals and auxiliary label data signals is an urgent problem to be solved.

[0072] The following describes an embodiment of the present application. The embodiment of the present application can be specifically applied to various scenarios such as CSI generation and reporting, CSI estimation, beam management, and positioning. The following embodiment is described using CSI generation and reporting as an example, but the present application is not limited thereto.

[0073] Embodiments of the first aspect

[0074] An embodiment of the present application provides a method for monitoring an AI / ML model, in which a network device monitors the performance of the AI / ML model, which is described below from the perspective of the network device.

[0075] FIG2 is a schematic diagram of a monitoring method for an AI / ML model according to an embodiment of the present application. As shown in FIG2 , the method includes:

[0076] 201. The network device receives an uplink reference signal sent by the terminal device; and

[0077] 202. The network device uses the uplink reference signal to monitor the performance of the AI / ML model.

[0078] It is worth noting that FIG2 above is merely a schematic illustration of an embodiment of the present application, and the present application is not limited thereto. For example, the execution order of the various operations may be appropriately adjusted, and other operations may be added or some operations may be reduced. Those skilled in the art may make appropriate modifications based on the above description, and are not limited to the description of FIG2 above.

[0079] In some embodiments, the AI / ML model is used to generate and report channel state information (CSI); the AI / ML model includes a CSI generation unit deployed in the terminal device and a CSI reconstruction unit deployed in the network device. For example, the CSI generation unit and the CSI reconstruction unit have corresponding model identifiers and / or version identifiers.

[0080] Figure 3 is a schematic diagram of an AI / ML model in an embodiment of the present application. For example, AI / ML can be used for CSI feedback to reduce feedback overhead or improve performance. A two-sided model can be used, where the CSI generation unit is deployed on the terminal side and the CSI reconstruction unit is deployed on the network side.

[0081] After model deployment, it's necessary to monitor model performance. This requires corresponding signals as label signals or to aid in generating labeled data. Furthermore, if model performance is poor, small-scale online training and parameter adjustment can be performed, a process also known as fine tuning. Alternatively, larger-scale training, online training, model adjustments, and updates can be performed. To enable continued model updates after deployment, label signals or reference signals are also required to assist in generating the necessary loss function for training.

[0082] In some embodiments, a network model including a first neural network and a second neural network is set in the network device, and the AI / ML model is monitored by the network model; the first neural network is the same as the CSI generation part of the AI / ML model, and the second neural network is the same as the CSI reconstruction part of the AI / ML model, or the CSI reconstruction part of the AI / ML model is used as the second neural network.

[0083] In some embodiments, the uplink reference signal is input into the network model to obtain label data; and the network device compares the label data with the output data of the AI / ML model, and performs performance monitoring on the AI / ML model based on the comparison result.

[0084] In some embodiments, the uplink reference signal includes at least one of the following: a sounding reference signal (SRS), a positioning sounding reference signal (Positioning SRS), a demodulation reference signal (DMRS), and a positioning reference signal (PRS). The present application is not limited thereto, and the following description will be made using SRS as an example.

[0085] In some embodiments, the network device configures and sends a CSI-RS for measuring and reporting the channel state information (CSI) to the terminal device; wherein the bandwidth of the uplink reference signal is the same as or similar to the bandwidth of the CSI-RS, or the bandwidth of the uplink reference signal is greater than the bandwidth of the CSI-RS.

[0086] In some embodiments, the network device performs channel estimation on the uplink reference signal according to the port information of the CSI-RS to obtain a channel coefficient corresponding to the CSI-RS port information, and inputs the channel coefficient into the first neural network for performance monitoring of the AI / ML model.

[0087] Figure 4 is an example diagram of monitoring by a network device in an embodiment of the present application. As shown in Figure 4, for a terminal device using a CSI generation unit of a certain model ID, the network device is not only provided with a CSI reconstruction unit of the model ID (also serving as a second neural network), but also provided with a CSI generation unit (first neural network) identical to the CSI generation unit of the terminal device. The network device can monitor the performance of the AI / ML model using an uplink RS. The bandwidth of the uplink RS configured by the base station is approximately equal to or greater than the bandwidth of the CSI-RS configured by it. The following describes the model monitoring process using SRS as an example.

[0088] For example, the network device configures the terminal device to send SRS, and a periodic, semi-periodic, or dynamic scheduling method can be used. For a certain terminal device, the bandwidth of its configured SRS must be greater than or approximately equal to its configured CSI-RS bandwidth. For the received SRS signal, the network device can extract the SRS based on the frequency domain density interval of its scheduled CSI-RS, and perform channel estimation on the SRS received by the receiving antenna corresponding to each port according to the number of CSI-RS ports to obtain the channel coefficient of the corresponding port number. In the frequency domain direction, select the channel coefficient that is consistent with the input dimension of the CSI generation unit, and send the channel coefficient to the input end of the CSI generation unit (first neural network) in the order of sending ports.

[0089] In some embodiments, if the performance monitoring result of the AI / ML model does not meet the requirements, the network device notifies the terminal devices in the cell to stop using the AI / ML model through cell-specific signaling. The cell-specific signaling may include a model identifier and / or version identifier of the AI / ML model.

[0090] For example, the network side uses the first neural network and the second neural network to simulate the joint operation of the CSI generation unit on the terminal side and the CSI reconstruction unit on the network side. As shown in Figure 4, the signal similarity between the input of the CSI generation unit (first neural network) and the output of the CSI reconstruction unit can be detected using indicators such as cosine similarity or mean square error. The network side monitors the performance of the model based on this indicator. When the network side determines that the performance of the model cannot meet the requirements, it notifies the terminals in the cell through cell-specific signaling (such as SIB or Cell-specific RRC), so that the terminals using the model ID stop using the model.

[0091] The above is a schematic illustration of the use of AI / ML models by multiple terminal devices within a cell. The following is an illustration of the use of AI / ML models by one terminal device, which is also applicable to multiple terminal devices.

[0092] In some embodiments, the network device receives terminal side antenna port information reported by the terminal device; and the network device performs channel estimation on the uplink reference signal according to the terminal side antenna port information to obtain a corresponding channel coefficient, and inputs the channel coefficient into the first neural network for performance monitoring of the AI / ML model.

[0093] For example, a network device and a terminal device each have a CSI reconstruction unit and a CSI generation unit based on a bilateral model. To monitor the performance of the model, the network also deploys a CSI generation unit (a first neural network) corresponding to the model ID and uses SRS as a monitoring signal.

[0094] The network device may also configure the terminal device to report the number of terminal-side antenna ports (number of receiver radio frequency chains). The network device may refer to the terminal-side antenna port information to configure the terminal device to perform SRS antenna switching.

[0095] Figure 5 is a schematic diagram of antenna port switching in an embodiment of the present application. Figure 5 illustrates the scenarios of one receive port, two receive ports, and four receive ports. To better simulate the signal input characteristics of the CSI generation unit of the terminal device, the network device can configure the terminal device to perform corresponding SRS antenna switching for 1T4R or 2T4R terminal devices.

[0096] In some embodiments, when the performance monitoring result of the AI / ML model does not meet the requirements, the network device notifies the terminal device through user-specific (UE-specific) signaling to stop using the AI / ML model, or deactivate the CSI reporting of the terminal device, or activate non-AI CSI reporting.

[0097] For example, the network side can use the first neural network and the second neural network to simulate the joint operation of the CSI generation unit on the terminal side and the CSI reconstruction unit on the network side. As shown in Figure 4, the network device detects the signal similarity between the input of the CSI generation unit (first neural network) and the output of the CSI reconstruction unit, using indicators such as cosine similarity or mean square error. The network side monitors the performance of the model based on the indicator. When the network side determines that the performance of the model cannot meet the requirements, it deactivates the CSI reporting of the terminal device through UE specific RRC signaling, or MAC-CE, or DCI (if the CSI reporting using the model is in an enabled state), or instructs the terminal device to deactivate the model.

[0098] In some embodiments, the network device extracts uplink reference signals at intervals according to the frequency domain density of the CSI-RS.

[0099] Figure 6 illustrates an example of signal extraction in accordance with an embodiment of the present application. For example, since the SRS signal density is typically higher than the CSI-RS signal density, channel information can be extracted from the SRS signal at intervals equal to the CSI-RS signal density. As shown in Figure 6, data can be extracted from a combo-2 SRS signal, resulting in multiple input data samples.

[0100] The following describes the signaling process for AI / ML model monitoring.

[0101] In some embodiments, the network device sends configuration information and / or indication information to the terminal device to schedule the terminal device to send the uplink reference signal.

[0102] In some embodiments, the network device sends first indication information to the terminal device for instructing to turn on or off the AI / ML model based on the performance monitoring result, or sends second indication information for instructing to switch from the AI / ML model to another AI / ML model.

[0103] In some embodiments, the network device sends an AI / ML-related capability query to the terminal device; and the network device receives an AI / ML-related capability response from the terminal device. The AI / ML-related capabilities include at least one of the following: signal processing module information, AI / ML support information, AI / ML model identification information, version information, data configuration information, AI / ML supported training capability information, AI / ML upgrade capability information, and performance monitoring capability information.

[0104] FIG7 is a signaling example diagram of AI / ML model monitoring according to an embodiment of the present application, schematically illustrating a situation where a network device performs monitoring and stops the model when performance is poor.

[0105] As shown in Figure 7, the network queries the terminal for CSI-related AI capabilities, including identification information such as the CSI generator ID and version. The terminal responds with AI / ML capabilities, including model identification information. After the network determines that a matching model exists, it can configure and instruct the terminal to use that model.

[0106] As shown in Figure 7, the network can configure CSI reporting and enable the AI / ML model to perform CSI reporting. Furthermore, the network can send CSI-RS, and the terminal can perform CSI measurements and generate CSI using the CSI generation unit before reporting. The network uses the CSI reconstruction unit to reconstruct the received CSI.

[0107] As shown in Figure 7, the network can also configure the terminal to send an SRS, which the network uses to monitor the performance of the AI / ML model. If the performance does not meet the requirements, the network sends a model stop indication, instructing the terminal to stop using the AI / ML model.

[0108] It is worth noting that FIG7 above is merely a schematic illustration of an embodiment of the present application, and the present application is not limited thereto. For example, the execution order of the various operations may be appropriately adjusted, and other operations may be added or some operations may be reduced. Those skilled in the art may make appropriate modifications based on the above description, and are not limited to the description of FIG7 above.

[0109] The above schematically illustrates the case of stopping the model. If you switch to other AI / ML models or switch to non-AI / ML CSI reporting, you can make corresponding adjustments and will not be repeated here. The following will explain the training of the AI / ML model.

[0110] In some embodiments, the network device uses the uplink reference signal to train the AI / ML model.

[0111] For example, in addition to being used for model monitoring, label data can also be used for model retraining, such as fine tuning or online training. When a model ID is determined through monitoring or other means to require further training or parameter optimization, the uplink reference signal can be used for training.

[0112] For example, when the uplink service is not busy, the network equipment schedules the uplink RS more frequently for model training.

[0113] In some embodiments, the network device sends a reference signal dedicated configuration to the terminal device to configure the terminal device to send the uplink reference signal according to the reference signal dedicated configuration. The uplink reference signal may be sent in consecutive symbols and consecutive time slots.

[0114] Figure 8 is an example diagram of a reference signal according to an embodiment of the present application. As shown in Figure 8 , a training-specific SRS configuration can be used, which has more RS symbols and RS RBs in the time-frequency domain. For example, SRS bursts can be supported, where SRS is transmitted in consecutive symbols and consecutive slots.

[0115] In some embodiments, if the training result of the AI / ML model meets the requirements, the network device sends a model update instruction and / or model update information to the terminal device. The model update information includes at least one of the following: a trained information generator, generator model update configuration information, generator model update layer number information, generator model update parameter information, model identifier, or version identifier.

[0116] For example, similar to model monitoring, a loss function can be set based on signal similarity and iteratively trained to meet performance requirements. Once the model has been trained, verified, and tested and has achieved satisfactory performance, the network can configure the terminal to deactivate or remove the previous SRS training-specific configuration or the legacy high-density SRS configuration.

[0117] In some embodiments, the model update information is sent via RRC signaling or MAC CE.

[0118] For example, after model training is complete, the network device sends the model's CSI generator to the terminal device, notifying the terminal device to update its CSI generator. For fine tuning, during training, the network device may only update one or a few layers of the model network close to the output. For example, if the CSI generator model is small, it can be sent to the terminal device via RRC signaling or MAC CE, including information about the updated number of network layers.

[0119] In addition, the network device can also continuously monitor the SRS to determine the performance of the disabled model or the performance of other models stored in it. When it determines that the performance of a model ID is good, it can negotiate with the terminal device to enable the model.

[0120] Figure 9 is a schematic diagram illustrating the application of a bilateral model on a gNB and a UE according to an embodiment of the present application. As shown in Figure 9 , model monitoring and model training can be implemented on the gNB side, on the UE side, or on both sides simultaneously. The following describes the implementation of model monitoring and / or model training on the gNB side. For implementation of model monitoring and / or model training on the UE side, refer to the embodiments of the second aspect described below.

[0121] As shown in Figure 9, the gNB and UE store a set of models corresponding to different channel conditions. In some embodiments, the gNB can identify the UE's channel condition or determine changes in the communication channel based on information reported by the UE. If the monitored performance is poor, the gNB selects a model it deems to be performing well and notifies the UE to apply the new model.

[0122] To ensure continuous communication performance, the gNB can first notify the UE of the new model ID. After receiving a response from the UE indicating successful reception and that the model can be used, the gNB can deactivate the model currently used by the UE and activate the new model.

[0123] Figure 10 is a schematic diagram of a model application on a gNB according to an embodiment of the present application. As shown in Figure 10 , model monitoring and model training can be implemented on the gNB side. The gNB can periodically use monitoring results and collected data for model training and transmit relevant information to the core network side.

[0124] The scenarios of FIG9 and FIG10 are applicable not only to the generation and reporting of CSI, but also to beam management, positioning and other processing processes.

[0125] The above embodiments are merely exemplary of the present invention, but the present invention is not limited thereto. Appropriate modifications may be made based on the above embodiments. For example, the above embodiments may be used alone, or one or more of the above embodiments may be combined.

[0126] As can be seen from the above embodiment, the network device receives the uplink reference signal sent by the terminal device; and the network device uses the uplink reference signal to monitor the performance of the AI / ML model. In this way, suitable label data can be obtained at a low cost, thereby supporting the monitoring of the AI / ML model.

[0127] Embodiments of the second aspect

[0128] The present application embodiment provides a method for monitoring an AI / ML model, wherein a terminal device monitors the performance of the AI / ML model. The following description is from the terminal device side, and the contents that are the same as the embodiment of the first aspect are not repeated.

[0129] FIG11 is a schematic diagram of a method for monitoring an AI / ML model according to an embodiment of the present application. As shown in FIG11 , the method includes:

[0130] 1101. A terminal device receives a downlink reference signal sent by a network device.

[0131] 1102. The terminal device uses the downlink reference signal to monitor the performance of the AI / ML model.

[0132] It is worth noting that FIG11 above is merely a schematic illustration of an embodiment of the present application, and the present application is not limited thereto. For example, the execution order of the various operations may be appropriately adjusted, and other operations may be added or some operations may be reduced. Those skilled in the art may make appropriate modifications based on the above description, and are not limited to the description of FIG11 above.

[0133] In some embodiments, the AI / ML model is used to generate and report channel state information (CSI); the AI / ML model includes a CSI generation unit deployed in the terminal device and a CSI reconstruction unit deployed in the network device. For example, the CSI generation unit and the CSI reconstruction unit have corresponding model identifiers and / or version identifiers.

[0134] In some embodiments, a network model including a first neural network and a second neural network is set in the terminal device, and the AI / ML model is monitored by the network model; the first neural network is the same as the CSI generation part of the AI / ML model or the CSI generation part of the AI / ML model is used as the first neural network, and the second neural network is the same as the CSI reconstruction part of the AI / ML model.

[0135] In some embodiments, the downlink reference signal is input into the network model to obtain label data; and the terminal device compares the label data with the output data of the AI / ML model, and monitors the performance of the AI / ML model based on the comparison result.

[0136] In some embodiments, the downlink reference signal is a first channel state information reference signal (CSI-RS) for AI / ML model monitoring and / or training. The terminal device receives a second CSI-RS sent by the network device for channel state information (CSI) measurement; wherein the bandwidth of the first CSI-RS is the same as the bandwidth of the second CSI-RS. The downlink reference signal is not limited to this and may also be other reference signals.

[0137] For example, a terminal device may include not only a CSI generation unit but also a CSI reconstruction unit. The terminal device can use CSI-RS as a label signal for model monitoring to determine the performance of the model during operation. For more information on how to determine this, please refer to related technologies.

[0138] In some embodiments, the terminal device receives indicator information for the performance monitoring sent by the network device; the indicator information includes a threshold value and / or a count value.

[0139] For example, the network can configure performance metrics and counter indicators for monitoring on the terminal side, or set predefined performance indicators. The terminal side counts events that do not meet the performance metrics. When the counter or counting condition meets the configured conditions, the model performance is determined to be unsatisfactory. The terminal side requests the network to stop using the model or deactivate the current CSI reporting.

[0140] In some embodiments, the threshold value includes a first threshold value and a second threshold value, and the counter value includes a first counter value and a second counter value; the first threshold value and the first counter value are used to stop the AI / ML model, and the second threshold value and the second counter value are used to start the AI / ML model.

[0141] In some embodiments, the terminal device sends third indication information to the network device for instructing to turn on or off the AI / ML model based on the performance monitoring result, or sends fourth indication information for instructing to switch from the AI / ML model to another AI / ML model.

[0142] For example, the network side configures the CSI reporting method to use the legacy method (non-AI / ML method). In addition to normal measurement reporting, the terminal side uses the CSI generation unit and CSI reconstruction unit corresponding to another model ID (including version) to monitor the performance of the model using CSI-RS. When the performance can reach the configured or predefined indicators, the terminal side sends a request to use the AI / ML model and attaches the model ID (version information) to the network side. The network side determines whether to enable the AI / ML model.

[0143] In order to better complete the monitoring of the model, the network side configures the terminal side with relevant model monitoring indicator information.

[0144] For example, the threshold may include similarity metrics such as cosine similarity, generalized cosine similarity, mean square error, etc.;

[0145] For another example, a counter or timer indicator is also included to determine model failure based on multiple threshold-exceeding events.

[0146] For another example, to avoid the ping-pong effect during model restarts or model switching, dual performance indicator parameters can be configured. For example, the transition from a two-sided model to a stop state corresponds to level 1 performance indicators, while the transition from a stopped state to a request for a two-sided model run corresponds to level 2 performance indicators. Level 2 performance requirements are generally higher.

[0147] In some embodiments, when the performance monitoring result of the AI / ML model does not meet the requirements, the terminal device sends a request message to the network device requesting to stop using the AI / ML model or deactivate CSI measurement reporting, or sends a request message to request switching the model, or sends a request message to activate non-AI CSI reporting.

[0148] For example, the request information includes the model identifier and / or version identifier of the AI / ML model.

[0149] The following describes the signaling process for AI / ML model monitoring.

[0150] In some embodiments, the terminal device receives configuration information and / or indication information sent by the network device to schedule the terminal device to receive the downlink reference signal.

[0151] In some embodiments, the terminal device receives an AI / ML-related capability query sent by the network device; and the terminal device feeds back an AI / ML-related capability response to the network device. The AI / ML-related capabilities include at least one of the following: signal processing module information, AI / ML support information, AI / ML model identification information, version information, data configuration information, AI / ML supported training capability information, AI / ML upgrade capability information, and performance monitoring capability information.

[0152] Figure 12 is a signaling example diagram of AI / ML model monitoring in an embodiment of the present application, which schematically shows a situation where a terminal device performs monitoring and stops the model when performance is poor.

[0153] As shown in Figure 12, the network queries the terminal for CSI-related AI capabilities, including information such as the ID and version of the CSI generator, and the terminal's performance monitoring capabilities. The terminal responds with AI / ML capabilities, including information such as model identification and terminal performance monitoring capabilities. After the network determines that a matching model exists, it can configure and instruct the terminal to use that model and configure the terminal for model monitoring.

[0154] As shown in Figure 12, the network can configure CSI reporting and enable the AI / ML model to perform CSI reporting. Furthermore, the network can send a second CSI-RS, and the terminal can perform CSI measurement and generate CSI using the CSI generation unit before reporting. The network uses the CSI reconstruction unit to reconstruct the received CSI.

[0155] As shown in Figure 12, the network side can also configure the terminal side to receive a first CSI-RS, and the terminal side uses this first CSI-RS to monitor the performance of the AI / ML model. If the performance does not meet the requirements, the terminal side determines that the AI / ML model is not working properly when the corresponding counter counts to the configured value. The terminal side sends a model stop request. After receiving the request, the network side can instruct the terminal side to stop using the AI / ML model and / or deactivate CSI reporting.

[0156] It is worth noting that FIG12 above is merely a schematic illustration of an embodiment of the present application, and the present application is not limited thereto. For example, the execution order of the various operations may be appropriately adjusted, and other operations may be added or some operations may be reduced. Those skilled in the art may make appropriate modifications based on the above description, and are not limited to the description of FIG12 above.

[0157] Figure 13 is another signaling example diagram of AI / ML model monitoring in an embodiment of the present application, which schematically shows a situation where a terminal device performs monitoring and switches models when performance is poor.

[0158] As shown in FIG13 , after the network side determines that there is a paired model, it can configure and instruct the use of the model, and can configure the terminal device to perform model monitoring.

[0159] As shown in Figure 13, the network can configure CSI reporting and enable the AI / ML model to perform CSI reporting. Furthermore, the network can send a second CSI-RS, and the terminal can perform CSI measurement and generate CSI using the CSI generation unit before reporting. The network uses the CSI reconstruction unit to reconstruct the received CSI.

[0160] As shown in Figure 13, the network side can also configure the terminal side to receive a first CSI-RS, and the terminal side uses this first CSI-RS to monitor the performance of the AI / ML model. If the performance does not meet the requirements, the terminal side determines that the AI / ML model is not working properly when the corresponding counter counts to the configured value. The terminal side sends a model stop request. After receiving the request, the network side can instruct the terminal side to stop using the AI / ML model and / or deactivate CSI reporting.

[0161] In addition, as shown in Figure 13, the network side can activate non-AI / ML CSI reporting, and the terminal side uses non-AI / ML methods to measure and report CSI.

[0162] In addition, the terminal side can monitor the performance of another AI / ML model. When the performance of the AI / ML model meets the requirements, the terminal side can send a request to enable the AI / ML model (for example, including the model ID), and the network side activates the AI / ML model to generate and report CSI.

[0163] It is worth noting that FIG13 above is merely a schematic illustration of an embodiment of the present application, and the present application is not limited thereto. For example, the execution order of the various operations may be appropriately adjusted, and other operations may be added or some operations may be reduced. Those skilled in the art may make appropriate modifications based on the above description, and are not limited to the description of FIG13 above.

[0164] The following explains the training of AI / ML models.

[0165] In some embodiments, the terminal device uses the downlink reference signal to train the AI / ML model.

[0166] For example, in addition to being used for model monitoring, label data can also be used for model retraining, such as fine tuning or online training. When a model ID is determined through monitoring or other means to require further training or parameter optimization, the downlink reference signal can be used for training.

[0167] For example, the network side configures CSI-RS resources and sends CSI-RS, but does not need to report CSI information. Alternatively, the network side configures a dedicated CSI-RS resource configuration for training, which has a high time-frequency density CSI-RS.

[0168] In some embodiments, the terminal device receives a reference signal dedicated configuration sent by the network device; and the terminal device receives the downlink reference signal according to the reference signal dedicated configuration. For example, the downlink reference signal is sent in consecutive symbols and consecutive time slots.

[0169] For example, the terminal may only use CSI-RS for training, or even report the CSI-RS configuration. This CSI-RS configuration is typically periodic or semi-persistent, or a dedicated IE may be used to indicate its dedicated use for training. As shown in Figure 8, a training-specific CSI-RS configuration may be used, with more RS symbols and RS RBs in the time-frequency domain. For example, CSI-RS can be transmitted in consecutive symbols and slots.

[0170] In some embodiments, if the training result of the AI / ML model meets the requirements, the terminal device sends a model update request and / or model update information to the network device. The model update information includes at least one of the following: a trained information reconstruction unit, model update configuration information of the reconstruction unit, number of model update layers of the reconstruction unit, model update parameter information of the reconstruction unit, and a model identifier or version identifier.

[0171] For example, after the terminal device's model is trained, the terminal device notifies the base station of the training completion and requests to upload the CSI reconstruction unit, model ID or version information, etc. The network side can stop sending the CSI-RS and configure the terminal device to upload the model update information.

[0172] For another example, the base station can notify the terminal device to use the trained new model ID corresponding to the two-sided model for testing. After the test is completed, the base station sends a model update instruction to the terminal. In another example, the network side directly uses the model sent by the terminal side to communicate with the terminal, but does not update the model for communication with other terminals.

[0173] In some embodiments, the model update information is sent via RRC signaling or MAC CE.

[0174] Furthermore, for fine tuning, during training, the network device can update only one or several layers of its model network close to the output, corresponding to the CSI generation and reconstruction units. Therefore, when a terminal requests a model update, it can report the number of layers corresponding to the updated model and other updated model parameters. The network then configures resources and schedules the upload of the updated model accordingly.

[0175] FIG14 is an example diagram of model training performed by a terminal device according to an embodiment of the present application.

[0176] As shown in Figure 14, the network queries the terminal for CSI-related AI capabilities, including information such as the ID and version of the CSI generator, and the terminal's upgrade capabilities. The terminal responds with AI / ML capabilities, including information such as model identification and terminal upgrade capabilities. The network can configure a training CSI-RS and indicate that CSI reporting is not required.

[0177] As shown in Figure 14, the terminal can perform model training and determine whether training is complete based on performance indicators. After training is complete, the network can be notified of the completion. The network can configure resources for the terminal to upload model update information, allowing the terminal to upload model update information.

[0178] In some embodiments, as shown in Figure 9 , model monitoring and model training can be implemented on the UE side. For terminal-side monitoring, if the UE detects poor monitoring performance and can detect changes in the channel state based on CSI-RS reception and finds a model that better matches the current channel from its stored models, the UE sends a request to the gNB to use the new model and sends the ID of the new model to the gNB.

[0179] If the gNB needs to use the current model ID to communicate with other terminals, it can reject the request. In this case, the UE can request to deactivate the model based on the monitoring status and fall back to non-AI / ML processing. It is worth noting that the model ID can contain version information, and models with different version IDs can be considered different models.

[0180] On the other hand, the UE can send monitored data information and / or channel state information, as well as channel state information configured by other network devices, to the eNB through UCI based on the eNB's configuration information for data collection and training of the model.

[0181] Figure 15 is a schematic diagram of a UE application model according to an embodiment of the present application. As shown in Figure 15 , model monitoring and model training can be implemented on the UE side. In this case, the gNB can configure measurement indicators for model monitoring performance. These measurement indicators may include, for example, measurement signal thresholds or timer or counter values ​​for abnormal situations.

[0182] For example, the UE has the ability to automatically stop the AI / ML model. When the UE detects that the corresponding measurement indicator exceeds the configured value, the terminal automatically stops using the AI / ML model and falls back to the conventional signal processing method (non-AI / ML). The terminal automatically falls back to the conventional signal processing method and reports the stop and fallback behavior to the eNB.

[0183] For another example, the UE does not have the ability to automatically deactivate an AI / ML model. When the UE detects that a corresponding measurement metric exceeds the configured value, it has no right to decide to deactivate the model and must first send a request to the gNB, which will then deactivate the model.

[0184] Furthermore, based on the model monitoring status, the terminal can report status information corresponding to the model ID to the OTT server and collect and report CSI-RS measurement information. This reporting can be transmitted via a data channel, and the relevant configuration needs to be configured by the network. Specifically, the network configures the terminal to periodically send its model monitoring information and / or channel status information to the OTT server, allowing the server to collect model-related data for training.

[0185] For the scenarios of Figures 9 and 15, it is not only applicable to the generation and reporting of CSI, but also to processing processes such as beam management and positioning.

[0186] The above embodiments are merely exemplary of the present invention, but the present invention is not limited thereto. Appropriate modifications may be made based on the above embodiments. For example, the above embodiments may be used alone, or one or more of the above embodiments may be combined.

[0187] As can be seen from the above embodiment, the terminal device receives the downlink reference signal sent by the network device; the terminal device uses the downlink reference signal to monitor the performance of the AI / ML model. This allows for obtaining appropriate label data at a low cost, thereby supporting the monitoring of the AI / ML model.

[0188] Embodiments of the third aspect

[0189] The present application provides an AI / ML model monitoring device. The device may be, for example, a network device, or one or more components or assemblies configured on the network device. The same contents as those in the first and second aspects of the embodiments are not repeated here.

[0190] FIG16 is a schematic diagram of a monitoring device for an AI / ML model according to an embodiment of the present application. As shown in FIG16 , the monitoring device 1600 for an AI / ML model includes:

[0191] A receiving unit 1601, which receives an uplink reference signal sent by a terminal device; and

[0192] A monitoring unit 1602 is configured to perform performance monitoring on the AI / ML model using the uplink reference signal.

[0193] In some embodiments, as shown in FIG16 , the AI / ML model monitoring device 1600 may further include:

[0194] The sending unit 1603 sends configuration information and / or indication information to the terminal device to schedule the terminal device to send the uplink reference signal.

[0195] In some embodiments, the sending unit 1603 further sends first indication information to the terminal device for instructing to turn on or off the AI / ML model based on the performance monitoring result, or sends second indication information for instructing to switch from the AI / ML model to another AI / ML model.

[0196] In some embodiments, the sending unit 1603 further sends an AI / ML-related capability query to the terminal device; and the receiving unit 1601 further receives an AI / ML-related capability response fed back by the terminal device.

[0197] In some embodiments, the AI / ML-related capabilities include at least one of the following: signal processing module information, AI / ML support information, AI / ML model identification information, version information, data configuration information, AI / ML supported training capability information, AI / ML upgrade capability information, and performance monitoring capability information.

[0198] In some embodiments, the AI / ML model is used to generate and report channel state information (CSI); the AI / ML model has a CSI generation unit deployed in the terminal device and a CSI reconstruction unit deployed in the network device;

[0199] The CSI generation unit and the CSI reconstruction unit have corresponding model identifiers and / or version identifiers.

[0200] In some embodiments, the monitoring unit 1602 has a network model for monitoring the AI / ML model, including a first neural network and a second neural network; the first neural network is the same as the CSI generation part of the AI / ML model, and the second neural network is the same as the CSI reconstruction part of the AI / ML model, or the CSI reconstruction part of the AI / ML model is used as the second neural network.

[0201] In some embodiments, the uplink reference signal is input into the network model to obtain label data; and the monitoring unit 1602 compares the label data with the output data of the AI / ML model, and performs performance monitoring on the AI / ML model based on the comparison result.

[0202] In some embodiments, the uplink reference signal includes at least one of the following: a sounding reference signal (SRS), a positioning sounding reference signal (Positioning SRS), and a demodulation reference signal (DMRS);

[0203] The receiving unit 1601 extracts the uplink reference signal at intervals according to the frequency domain density of the CSI-RS.

[0204] In some embodiments, the network device configures and sends a CSI-RS for measuring and reporting the channel state information (CSI) to the terminal device;

[0205] The bandwidth of the uplink reference signal is the same as or similar to the bandwidth of the CSI-RS, or the bandwidth of the uplink reference signal is greater than the bandwidth of the CSI-RS.

[0206] In some embodiments, the monitoring unit 1602 performs channel estimation on the uplink reference signal according to the port information of the CSI-RS to obtain a channel coefficient corresponding to the CSI-RS port information, and inputs the channel coefficient into the first neural network for performance monitoring of the AI / ML model.

[0207] In some embodiments, when the performance monitoring result of the AI / ML model does not meet the requirements, the terminal devices in the cell are notified through cell-specific signaling to stop using the AI / ML model;

[0208] The cell-specific signaling includes a model identifier and / or a version identifier of the AI / ML model.

[0209] In some embodiments, the monitoring unit 1602 performs channel estimation on the uplink reference signal according to the terminal side antenna port information reported by the terminal device to obtain a corresponding channel coefficient, and inputs the channel coefficient into the first neural network for performance monitoring of the AI / ML model.

[0210] In some embodiments, when the performance monitoring result of the AI / ML model does not meet the requirements, the terminal device is notified through user-specific (UE-specific) signaling to stop using the AI / ML model, or to deactivate the CSI reporting of the terminal device, or to activate non-AI CSI reporting.

[0211] In some embodiments, as shown in FIG16 , the apparatus 1600 may further include:

[0212] A training unit 1604 is configured to train the AI / ML model using the uplink reference signal.

[0213] In some embodiments, the training unit 1604 further sends a reference signal dedicated configuration to the terminal device to configure the terminal device to send the uplink reference signal according to the reference signal dedicated configuration;

[0214] The uplink reference signal is sent in consecutive symbols and consecutive time slots.

[0215] In some embodiments, the training unit 1604 sends a model update instruction and / or model update information to the terminal device if the training result of the AI / ML model meets the requirements;

[0216] The model update information includes at least one of the following: a trained information generation unit, generation unit model update configuration information, generation unit model update layer number information, generation unit model update parameter information, model identifier or version identifier;

[0217] The model update information is sent via RRC signaling or MAC CE.

[0218] The above embodiments are merely exemplary of the present invention, but the present invention is not limited thereto. Appropriate modifications may be made based on the above embodiments. For example, the above embodiments may be used alone, or one or more of the above embodiments may be combined.

[0219] It is worth noting that the above only describes the components or modules related to the present application, but the present application is not limited thereto. The AI / ML model monitoring device 1600 may also include other components or modules. For the specific content of these components or modules, please refer to the relevant technology.

[0220] In addition, for the sake of simplicity, FIG16 only illustrates the connection relationship or signal direction between various components or modules. However, it should be clear to those skilled in the art that various related technologies such as bus connection can be used. The above-mentioned components or modules can be implemented by hardware facilities such as processors, memories, transmitters, and receivers; the implementation of this application is not limited to this.

[0221] As can be seen from the above embodiment, the network device receives the uplink reference signal sent by the terminal device; and the network device uses the uplink reference signal to monitor the performance of the AI / ML model. In this way, suitable label data can be obtained at a low cost, thereby supporting the monitoring of the AI / ML model.

[0222] Embodiments of the fourth aspect

[0223] The present application provides an AI / ML model monitoring device. The device may be, for example, a terminal device, or one or more components or assemblies configured on the terminal device. The same contents as those in the first to third aspects of the embodiments are not repeated here.

[0224] FIG17 is a schematic diagram of a monitoring device for an AI / ML model according to an embodiment of the present application. As shown in FIG17 , the monitoring device 1700 for an AI / ML model includes:

[0225] A receiving unit 1701, which receives a downlink reference signal sent by a network device; and

[0226] A monitoring unit 1702 is configured to monitor the performance of the AI / ML model using the downlink reference signal.

[0227] In some embodiments, as shown in FIG17 , the AI / ML model monitoring device 1700 may further include:

[0228] A training unit 1703 trains the AI / ML model using the downlink reference signal.

[0229] The above embodiments are merely exemplary of the present invention, but the present invention is not limited thereto. Appropriate modifications may be made based on the above embodiments. For example, the above embodiments may be used alone, or one or more of the above embodiments may be combined.

[0230] It is worth noting that the above only describes the components or modules related to the present application, but the present application is not limited thereto. The AI / ML model monitoring device 1700 may also include other components or modules. For the specific content of these components or modules, please refer to the relevant technology.

[0231] In addition, for the sake of simplicity, FIG17 only illustrates the connection relationship or signal direction between various components or modules. However, it should be clear to those skilled in the art that various related technologies such as bus connection can be used. The above-mentioned components or modules can be implemented by hardware facilities such as processors, memories, transmitters, and receivers; the implementation of this application is not limited to this.

[0232] As can be seen from the above embodiment, the terminal device receives the downlink reference signal sent by the network device; the terminal device uses the downlink reference signal to monitor the performance of the AI / ML model. This allows for obtaining appropriate label data at a low cost, thereby supporting the monitoring of the AI / ML model.

[0233] Embodiments of the fifth aspect

[0234] An embodiment of the present application also provides a communication system, and reference may be made to FIG1 . The contents that are the same as those in the first to fourth aspects of the embodiments will not be repeated.

[0235] In some embodiments, the communication system 100 may include at least:

[0236] a terminal device that receives a downlink reference signal sent by a network device; and uses the downlink reference signal to monitor the performance of the AI / ML model; and / or

[0237] A network device that receives an uplink reference signal sent by a terminal device; and uses the uplink reference signal to monitor the performance of the AI / ML model.

[0238] An embodiment of the present application further provides a network device, which may be, for example, a base station, but the present application is not limited thereto and may also be other network devices.

[0239] Figure 18 is a schematic diagram illustrating the structure of a network device according to an embodiment of the present application. As shown in Figure 18 , network device 1800 may include a processor 1810 (e.g., a central processing unit (CPU)) and a memory 1820 ; the memory 1820 is coupled to the processor 1810 . The memory 1820 may store various data and may also store an information processing program 1830 , which is executed under the control of the processor 1810 .

[0240] For example, the processor 1810 may be configured to execute a program to implement the method for monitoring an AI / ML model as described in the embodiment of the first aspect. For example, the processor 1810 may be configured to perform the following control: receiving an uplink reference signal sent by a terminal device; and using the uplink reference signal to perform performance monitoring on the AI / ML model.

[0241] In addition, as shown in FIG18 , network device 1800 may further include: a transceiver 1840 and an antenna 1850, etc.; wherein, the functions of the above components are similar to those in the prior art and are not further described here. It is worth noting that network device 1800 does not necessarily include all the components shown in FIG18 ; in addition, network device 1800 may also include components not shown in FIG18 , and reference may be made to the prior art for such components.

[0242] The embodiment of the present application also provides a terminal device, but the present application is not limited thereto and may also be other devices.

[0243] Figure 19 is a schematic diagram of a terminal device according to an embodiment of the present application. As shown in Figure 19 , terminal device 1900 may include a processor 1910 and a memory 1920. Memory 1920 stores data and programs and is coupled to processor 1910. It should be noted that this diagram is exemplary; other types of structures may be used to supplement or replace this structure to implement telecommunication or other functions.

[0244] For example, the processor 1910 may be configured to execute a program to implement the method for monitoring an AI / ML model as described in the embodiment of the second aspect. For example, the processor 1910 may be configured to perform the following control: receive a downlink reference signal sent by a network device; and use the downlink reference signal to monitor the performance of the AI / ML model.

[0245] As shown in Figure 19 , the terminal device 1900 may further include: a communication module 1930, an input unit 1940, a display 1950, and a power supply 1960. The functions of these components are similar to those in the prior art and are not described here in detail. It is worth noting that the terminal device 1900 does not necessarily include all of the components shown in Figure 19 , and these components are not essential. Furthermore, the terminal device 1900 may also include components not shown in Figure 19 , for which reference may be made to the prior art.

[0246] An embodiment of the present application also provides a computer program, wherein when the program is executed in a terminal device, the program causes the terminal device to execute the AI / ML model monitoring method described in the embodiment of the second aspect.

[0247] An embodiment of the present application also provides a storage medium storing a computer program, wherein the computer program enables a terminal device to execute the AI / ML model monitoring method described in the embodiment of the second aspect.

[0248] An embodiment of the present application also provides a computer program, wherein when the program is executed in a network device, the program causes the network device to execute the AI / ML model monitoring method described in the embodiment of the first aspect.

[0249] An embodiment of the present application also provides a storage medium storing a computer program, wherein the computer program enables a network device to execute the AI / ML model monitoring method described in the embodiment of the first aspect.

[0250] The above devices and methods of the present application can be implemented by hardware or by a combination of hardware and software. The present application relates to such a computer-readable program that, when executed by a logic component, enables the logic component to implement the devices or components described above, or enables the logic component to implement the various methods or steps described above. The present application also relates to a storage medium for storing the above program, such as a hard disk, a magnetic disk, an optical disk, a DVD, a flash memory, etc.

[0251] The method / device described in conjunction with the embodiments of the present application can be directly embodied as hardware, a software module executed by a processor, or a combination of the two. For example, one or more of the functional block diagrams shown in the figure and / or one or more combinations of functional block diagrams can correspond to various software modules of the computer program flow or to various hardware modules. These software modules can respectively correspond to the various steps shown in the figure. These hardware modules can be implemented by solidifying these software modules, for example, using a field programmable gate array (FPGA).

[0252] The software module may be located in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. A storage medium may be coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium; or the storage medium may be an integral part of the processor. The processor and the storage medium may be located in an ASIC. The software module may be stored in the memory of the mobile terminal or in a memory card that can be inserted into the mobile terminal. For example, if the device (such as a mobile terminal) uses a large-capacity MEGA-SIM card or a large-capacity flash memory device, the software module may be stored in the MEGA-SIM card or the large-capacity flash memory device.

[0253] One or more of the functional blocks and / or one or more combinations of functional blocks described in the accompanying drawings may be implemented as a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, or any appropriate combination thereof for performing the functions described in this application. One or more of the functional blocks and / or one or more combinations of functional blocks described in the accompanying drawings may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in communication with a DSP, or any other such configuration.

[0254] The present application has been described above in conjunction with specific embodiments. However, those skilled in the art should understand that these descriptions are merely illustrative and are not intended to limit the scope of protection of the present application. Those skilled in the art may make various modifications and variations to the present application based on the spirit and principles of the present application, and such modifications and variations are also within the scope of the present application.

[0255] Regarding the implementation methods including the above embodiments, the following additional notes are also disclosed:

[0256] 1. A method for monitoring an AI / ML model, wherein a network device monitors the performance of the AI / ML model, the method comprising:

[0257] The network device receives an uplink reference signal sent by the terminal device; and

[0258] The network device uses the uplink reference signal to monitor the performance of the AI / ML model.

[0259] 2. The method according to Supplement 1, wherein the method further comprises:

[0260] The network device sends configuration information and / or indication information to the terminal device to schedule the terminal device to send the uplink reference signal.

[0261] 3. The method according to Note 1 or 2, wherein the method further comprises:

[0262] The network device sends, to the terminal device, first indication information for instructing to turn on or off the AI / ML model based on the performance monitoring result, or sends second indication information for instructing to switch from the AI / ML model to another AI / ML model.

[0263] 4. The method according to any one of Notes 1 to 3, wherein the method further comprises:

[0264] The network device determines a channel change according to a channel state or information reported by the terminal device; and

[0265] The network device selects another AI / ML model and notifies the terminal device to use the other AI / ML model.

[0266] 5. The method according to Supplement 4, wherein the method further comprises:

[0267] The network device sends the model identifier and / or version identifier of the other AIML model to the terminal device;

[0268] receiving response information fed back by the terminal device using the other AI / ML model; and

[0269] The network device deactivates the current AI / ML model and activates the other AI / ML model.

[0270] 6. The method according to any one of Notes 1 to 5, wherein the method further comprises:

[0271] The network device sends the monitoring results and corresponding data of the AI / ML model to the core network.

[0272] 7. The method according to any one of Notes 1 to 6, wherein the method further comprises:

[0273] The network device sends an AI / ML related capability query to the terminal device; and

[0274] The network device receives an AI / ML-related capability response fed back by the terminal device.

[0275] 8. The method according to Note 7, wherein the AI / ML-related capabilities include at least one of the following: signal processing module information, AI / ML support information, AI / ML model identification information, version information, data configuration information, AI / ML supported training capability information, AI / ML upgrade capability information, and performance monitoring capability information.

[0276] 9. The method according to any one of Notes 1 to 8, wherein the AI / ML model is used to generate and report channel state information (CSI); the AI / ML model has a CSI generation unit deployed on the terminal device and a CSI reconstruction unit deployed on the network device.

[0277] 10. The method according to Note 9, wherein a network model for monitoring the AI / ML model is set in the network device, including a first neural network and a second neural network; the first neural network is the same as the CSI generation part of the AI / ML model, and the second neural network is the same as the CSI reconstruction part of the AI / ML model, or the CSI reconstruction part of the AI / ML model is used as the second neural network.

[0278] 11. The method according to Note 10, wherein the uplink reference signal is input into the network model to obtain label data; and the network device compares the label data with the output data of the AI / ML model, and performs performance monitoring on the AI / ML model based on the comparison result.

[0279] 12. The method according to any one of Notes 9 to 11, wherein the CSI generation unit and the CSI reconstruction unit have corresponding model identifiers and / or version identifiers.

[0280] 13. The method according to any one of Notes 9 to 12, wherein the uplink reference signal includes at least one of the following: a sounding reference signal (SRS), a positioning sounding reference signal (Positioning SRS), a demodulation reference signal (DMRS), and a positioning reference signal (PRS).

[0281] 14. The method according to any one of Notes 9 to 13, wherein the method further comprises:

[0282] The network device configures and sends a CSI-RS for measuring and reporting the channel state information (CSI) to the terminal device;

[0283] The bandwidth of the uplink reference signal is the same as or similar to the bandwidth of the CSI-RS, or the bandwidth of the uplink reference signal is greater than the bandwidth of the CSI-RS.

[0284] 15. The method according to Supplement 14, further comprising:

[0285] The network device extracts the uplink reference signal at intervals according to the frequency domain density of the CSI-RS.

[0286] 16. The method according to Supplement 15, wherein:

[0287] The network device performs channel estimation on the uplink reference signal according to the port information of the CSI-RS to obtain a channel coefficient corresponding to the CSI-RS port information, and inputs the channel coefficient into the first neural network for performance monitoring of the AI / ML model.

[0288] 17. The method according to Supplement 16, wherein:

[0289] When the performance monitoring result of the AI / ML model does not meet the requirements, the network device notifies the terminal devices in the cell through cell-specific signaling to stop using the AI / ML model.

[0290] 18. The method according to Note 17, wherein the cell-specific signaling includes a model identifier and / or version identifier of the AI / ML model.

[0291] 19. The method according to Supplement 15, wherein the method further comprises:

[0292] The network device receives the terminal side antenna port information reported by the terminal device; and

[0293] The network device performs channel estimation on the uplink reference signal according to the terminal side antenna port information to obtain a corresponding channel coefficient, and inputs the channel coefficient into the first neural network for performance monitoring of the AI / ML model.

[0294] 20. The method according to Supplementary Note 19, further comprising:

[0295] The network device configures the terminal device to perform SRS antenna switching (SRS antenna swithing) with reference to the terminal side antenna port information.

[0296] 21. The method according to Supplement 19 or 20, wherein the method further comprises:

[0297] When the performance monitoring result of the AI / ML model does not meet the requirements, the network device notifies the terminal device through user-specific (UE-specific) signaling to stop using the AI / ML model, or deactivate the CSI reporting of the terminal device, or activate non-AI CSI reporting.

[0298] 22. The method according to any one of Notes 1 to 21, wherein the method further comprises:

[0299] The network device trains the AI / ML model using the uplink reference signal.

[0300] 23. The method according to Supplementary Note 22, wherein:

[0301] The network device sends a reference signal dedicated configuration to the terminal device to configure the terminal device to send the uplink reference signal according to the reference signal dedicated configuration.

[0302] 24. The method according to Note 23, wherein the uplink reference signal is sent in consecutive symbols and consecutive time slots.

[0303] 25. The method according to any one of Notes 22 to 24, further comprising:

[0304] When the training result of the AI / ML model meets the requirements, the network device sends a model update instruction and / or model update information to the terminal device.

[0305] 26. The method according to Note 25, wherein the model update information includes at least one of the following: a trained information generation unit, generation unit model update configuration information, generation unit model update layer number information, generation unit model update parameter information, model identification or version identification.

[0306] 27. The method according to Note 25 or 26, wherein the model update information is sent via RRC signaling or MAC CE.

[0307] 28. A method for monitoring an AI / ML model, wherein a terminal device monitors the performance of the AI / ML model, the method comprising:

[0308] The terminal device receives a downlink reference signal sent by the network device; and

[0309] The terminal device uses the downlink reference signal to monitor the performance of the AI / ML model.

[0310] 29. The method according to Supplementary Note 28, wherein the method further comprises:

[0311] The terminal device receives configuration information and / or indication information sent by the network device to schedule the terminal device to receive the downlink reference signal.

[0312] 30. The method according to Supplement 28 or 29, wherein the method further comprises:

[0313] The terminal device sends third indication information to the network device for instructing to turn on or off the AI / ML model based on the performance monitoring result, or sends fourth indication information for instructing to switch from the AI / ML model to another AI / ML model.

[0314] 31. The method according to any one of Notes 28 to 30, further comprising:

[0315] The terminal device determines to use another AI / ML model if the performance of the AI / ML model does not meet the requirements; and

[0316] The terminal device sends a request to the network device to use the other AI / ML model; and

[0317] Upon receiving the confirmation information sent by the network device, the terminal device stops the current AI / ML model and starts the other AI / ML model.

[0318] 32. The method according to Supplementary Note 31, further comprising:

[0319] Upon receiving the rejection information sent by the network device, the terminal device stops the current AI / ML model and falls back to a non-AI / ML mode.

[0320] 33. The method according to any one of Notes 28 to 30, wherein the method further comprises:

[0321] The terminal device stops using the AI / ML model and falls back to a non-AI / ML mode if the performance of the AI / ML model does not meet the requirements; and

[0322] The terminal device sends an indication message that the AI / ML model has stopped to the network device.

[0323] 34. The method according to any one of Notes 28 to 30, further comprising:

[0324] When the performance of the AI / ML model does not meet the requirements, the terminal device sends a request to the network device to stop using the AI / ML model; and

[0325] Upon receiving confirmation information sent by the network device, stop using the AI / ML model and fall back to a non-AI / ML mode.

[0326] 35. The method according to any one of Notes 28 to 34, further comprising:

[0327] The terminal device sends the model monitoring information and / or channel state information of the AI / ML model to a server or a network device.

[0328] 36. The method according to Supplementary Note 35, further comprising:

[0329] The terminal device receives the configuration information and / or instruction information sent by the network device to upload the model monitoring information and / or channel status information.

[0330] 37. The method according to any one of Notes 28 to 36, further comprising:

[0331] The terminal device receives an AI / ML-related capability query sent by the network device; and

[0332] The terminal device feeds back an AI / ML-related capability response to the network device.

[0333] 38. The method according to Note 37, wherein the AI / ML-related capabilities include at least one of the following: signal processing module information, AI / ML support information, AI / ML model identification information, version information, data configuration information, AI / ML support training capability information, AI / ML upgrade capability information, and performance monitoring capability information.

[0334] 39. The method according to any one of Notes 28 to 38, wherein the method further comprises:

[0335] The terminal device receives indicator information for the performance monitoring sent by the network device; the indicator information includes a threshold value and / or a counter value.

[0336] 40. The method according to Note 39, wherein the threshold value includes a first threshold value and a second threshold value, and the counter value includes a first counter value and a second counter value; the first threshold value and the first counter value are used to stop the AI / ML model, and the second threshold value and the second counter value are used to start the AI / ML model.

[0337] 41. A method according to any one of Notes 28 to 40, wherein the AI / ML model is used to generate and report channel state information (CSI); the AI / ML model has a CSI generation unit deployed in the terminal device and a CSI reconstruction unit deployed in the network device.

[0338] 42. The method according to Note 41, wherein a network model for monitoring the AI / ML model is set in the terminal device, including a first neural network and a second neural network; the first neural network is the same as the CSI generation part of the AI / ML model or the CSI generation part of the AI / ML model is used as the first neural network, and the second neural network is the same as the CSI reconstruction part of the AI / ML model.

[0339] 43. The method according to Note 42, wherein the downlink reference signal is input into the network model to obtain label data; and the terminal device compares the label data with the output data of the AI / ML model, and performs performance monitoring on the AI / ML model based on the comparison result.

[0340] 44. The method according to any one of Notes 41 to 43, wherein the CSI generation unit and the CSI reconstruction unit have corresponding model identifiers and / or version identifiers.

[0341] 45. The method according to any one of Notes 41 to 44, wherein the downlink reference signal is a first channel state information reference signal (CSI-RS) used for AI / ML model monitoring and / or training.

[0342] 46. ​​The method according to Supplement 45, wherein the method further comprises:

[0343] The terminal device receives a second CSI-RS sent by the network device for the channel state information (CSI) measurement; wherein the bandwidth of the first CSI-RS is the same as the bandwidth of the second CSI-RS.

[0344] 47. The method according to any one of Notes 41 to 46, wherein the method further comprises:

[0345] When the performance monitoring result of the AI / ML model does not meet the requirements, the terminal device sends a request message to the network device for requesting to stop using the AI / ML model or deactivate CSI measurement reporting, or sends a request message for requesting to switch the model, or sends a request message for activating non-AI CSI reporting.

[0346] 48. The method according to Note 47, wherein the request information includes a model identifier and / or version identifier of the AI / ML model.

[0347] 49. The method according to any one of Notes 41 to 48, wherein the method further comprises:

[0348] The terminal device trains the AI / ML model using the downlink reference signal.

[0349] 50. The method according to Supplementary Note 49, wherein the method further comprises:

[0350] The terminal device receives the reference signal dedicated configuration sent by the network device; and

[0351] The terminal device receives the downlink reference signal according to the reference signal dedicated configuration.

[0352] 51. The method according to Note 50, wherein the downlink reference signal is sent in consecutive symbols and consecutive time slots.

[0353] 52. The method according to any one of Notes 49 to 51, further comprising:

[0354] When the training result of the AI / ML model meets the requirements, the terminal device sends a model update request and / or model update information to the network device.

[0355] 53. The method according to Note 52, wherein the model update information includes at least one of the following: a trained information reconstruction unit, reconstruction unit model update configuration information, reconstruction unit model update layer number information, reconstruction unit model update parameter information, model identification or version identification.

[0356] 54. The method according to Note 52 or 53, wherein the model update information is sent via RRC signaling or MAC CE.

[0357] 55. A network device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the computer program to implement the monitoring method of the AI / ML model as described in any one of Notes 1 to 27.

[0358] 56. A terminal device comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the computer program to implement the AI / ML model monitoring method as described in any one of Notes 28 to 54.

Claims

1. A monitoring device for an AI / ML model, comprising: a receiving unit, configured to receive an uplink reference signal sent by a terminal device; as well as A monitoring unit is configured to perform performance monitoring on the AI / ML model using the uplink reference signal.

2. The device according to claim 1, wherein The device further comprises: A sending unit, which sends configuration information and / or indication information to the terminal device to schedule the terminal device to send the uplink reference signal.

3. The device according to claim 2, wherein The sending unit also sends, to the terminal device, first indication information for instructing to turn on or off the AI / ML model based on the performance monitoring result, or sends second indication information for instructing to switch from the AI / ML model to another AI / ML model.

4. The device according to claim 2, wherein The sending unit further sends an AI / ML related capability query to the terminal device; and The receiving unit also receives an AI / ML-related capability response fed back by the terminal device.

5. The device according to claim 4, wherein The AI / ML-related capabilities include at least one of the following: signal processing module information, AI / ML support information, AI / ML model identification information, version information, data configuration information, AI / ML supported training capability information, AI / ML upgrade capability information, and performance monitoring capability information.

6. The device according to claim 1, wherein The AI / ML model is used to generate and report channel state information; the AI / ML model has a channel state information generation unit deployed in the terminal device and a channel state information reconstruction unit deployed in the network device; The channel state information generating unit and the channel state information reconstructing unit have corresponding model identifiers and / or version identifiers.

7. The device according to claim 6, wherein The monitoring unit has a network model for monitoring the AI / ML model, including a first neural network and a second neural network; the first neural network is the same as the channel state information generation unit of the AI / ML model, and the second neural network is the same as the channel state information reconstruction unit of the AI / ML model, or the channel state information reconstruction unit of the AI / ML model is used as the second neural network.

8. The device according to claim 7, wherein The uplink reference signal is input into the network model to obtain label data; and the monitoring unit compares the label data with the output data of the AI / ML model, and performs performance monitoring on the AI / ML model based on the comparison result.

9. The device according to claim 6, wherein The uplink reference signal includes at least one of the following: a sounding reference signal, a positioning sounding reference signal, and a demodulation reference signal; The receiving unit extracts the uplink reference signal at intervals according to the frequency domain density of the channel state information reference signal.

10. The device according to claim 6, wherein The network device configures and sends a channel state information reference signal for measuring and reporting the channel state information to the terminal device; The bandwidth of the uplink reference signal is the same as or similar to the bandwidth of the channel state information reference signal, or the bandwidth of the uplink reference signal is greater than the bandwidth of the channel state information reference signal.

11. The device according to claim 6, wherein The monitoring unit performs channel estimation on the uplink reference signal according to the port information of the channel state information reference signal to obtain a channel coefficient corresponding to the channel state information reference signal port information, and inputs the channel coefficient into a first neural network for performance monitoring of the AI / ML model.

12. The device according to claim 11, wherein If the performance monitoring result of the AI / ML model does not meet the requirements, notify the terminal devices in the cell to stop using the AI / ML model through cell-specific signaling; The cell-specific signaling includes a model identifier and / or a version identifier of the AI / ML model.

13. The device according to claim 6, wherein The monitoring unit performs channel estimation on the uplink reference signal according to the terminal side antenna port information reported by the terminal device to obtain a corresponding channel coefficient, and inputs the channel coefficient into a first neural network for performance monitoring of the AI / ML model.

14. The device according to claim 13, wherein When the performance monitoring result of the AI / ML model does not meet the requirements, the terminal device is notified through user-specific signaling to stop using the AI / ML model, or to deactivate the channel state information reporting of the terminal device, or to activate non-AI / ML channel state information reporting.

15. The device according to claim 1, wherein The device further comprises: A training unit is configured to train the AI / ML model using the uplink reference signal.

16. The device according to claim 15, wherein The training unit further sends a reference signal dedicated configuration to the terminal device to configure the terminal device to send the uplink reference signal according to the reference signal dedicated configuration; The uplink reference signal is sent in consecutive symbols and consecutive time slots.

17. The device according to claim 15, wherein The training unit sends a model update instruction and / or model update information to the terminal device when the training result of the AI / ML model meets the requirements; The model update information includes at least one of the following: a trained information generation unit, generation unit model update configuration information, generation unit model update layer number information, generation unit model update parameter information, model identifier or version identifier; The model update information is sent via radio resource control signaling or medium access control element.

18. A device for monitoring an AI / ML model, comprising: a receiving unit, configured to receive a downlink reference signal sent by a network device; as well as A monitoring unit is configured to perform performance monitoring on the AI / ML model using the downlink reference signal.

19. The device according to claim 18, wherein The device further comprises: A training unit is configured to train the AI / ML model using the downlink reference signal.

20. A communication system comprising: A terminal device receives a downlink reference signal sent by a network device; and using the downlink reference signal to monitor the performance of the AI / ML model; and / or A network device that receives an uplink reference signal sent by a terminal device; and uses the uplink reference signal to monitor the performance of the AI / ML model.

Images