Method and apparatus for monitoring performance of artificial intelligence (AI) model, and communication device, communication system and storage medium

Abstract

Provided in the present disclosure are a method and apparatus for monitoring the performance of an artificial intelligence (AI) model, and a communication device, a communication system and a storage medium. The method comprises: determining reference channel state information (CSI) and predicted CSI, wherein the reference CSI is used as a reference for monitoring the performance of an AI model, the predicted CSI is predicted by means of the AI model and corresponds to a first prediction moment, and both the reference CSI and the predicted CSI are used for determining performance information of the AI model. Thus, the performance of an AI model for predicting CSI can be effectively monitored.

Description

Performance monitoring method and device for artificial intelligence (AI) models, communication equipment, communication system, and storage medium Technical Field

[0001] The present disclosure relates to the field of communication technology, and in particular to a performance monitoring method and device for an artificial intelligence (AI) model, communication equipment, a communication system, and a storage medium. Background Art

[0002] With the development of artificial intelligence (AI) technology, AI has been widely applied to the physical layer of wireless communications. AI model reasoning can be used to predict channel state information (CSI), that is, to predict the channel state information at a future time. The prediction performance of AI models can be superior to that of non-AI models.

[0003] Summary of the Invention

[0004] The embodiments of the present disclosure provide a method, apparatus, device, chip system, storage medium, computer program, and computer program product for monitoring the performance of an artificial intelligence (AI) model, which can be applied in the field of communication technology to solve the technical problem of "not being able to effectively monitor the performance of an AI model used to predict CSI."

[0005] The present disclosure proposes a performance monitoring method and device, communication equipment, communication system, and storage medium for artificial intelligence (AI) models.

[0006] According to a first aspect of an embodiment of the present disclosure, a performance monitoring method for an artificial intelligence (AI) model is proposed, including: determining reference channel state information (CSI) and predicted CSI, wherein the reference CSI is used as a reference for AI model performance monitoring, and the predicted CSI is predicted by the AI ​​model and corresponds to a first prediction moment, and the reference CSI and the predicted CSI are jointly used to determine the performance information of the AI ​​model.

[0007] According to a second aspect of an embodiment of the present disclosure, a performance monitoring method for an artificial intelligence (AI) model is proposed, including: determining performance information of the AI ​​model, wherein the performance information is determined by reference channel state information (CSI) and predicted CSI, the reference CSI is used for AI model performance monitoring reference, and the predicted CSI is predicted by the AI ​​model and corresponds to a first prediction moment.

[0008] According to a third aspect of an embodiment of the present disclosure, a performance monitoring device for an artificial intelligence (AI) model is proposed, including: a processing module for determining reference channel state information (CSI) and predicted CSI, wherein the reference CSI is used as a reference for AI model performance monitoring, and the predicted CSI is predicted by the AI ​​model and corresponds to a first prediction moment, and the reference CSI and the predicted CSI are jointly used to determine the performance information of the AI ​​model.

[0009] According to the fourth aspect of an embodiment of the present disclosure, a performance monitoring device for an artificial intelligence (AI) model is proposed, including: a processing module for determining performance information of the AI ​​model, wherein the performance information is determined by reference channel state information (CSI) and predicted CSI, the reference CSI is used for AI model performance monitoring reference, and the predicted CSI is predicted by the AI ​​model and corresponds to a first prediction moment.

[0010] According to a fifth aspect of an embodiment of the present disclosure, a communication device is proposed, comprising: one or more processors; wherein the processor is used to call instructions so that the communication device executes the performance monitoring method of the artificial intelligence AI model of the first aspect, or executes the performance monitoring method of the artificial intelligence AI model of the second aspect.

[0011] According to the sixth aspect of an embodiment of the present disclosure, a communication system is proposed, characterized in that it includes a network device and a terminal, wherein the terminal is configured to implement the performance monitoring method of the artificial intelligence AI model of the first aspect, and the network device is configured to implement the performance monitoring method of the artificial intelligence AI model of the second aspect.

[0012] According to the seventh aspect of the embodiment of the present disclosure, a storage medium is proposed, which stores instructions, and is characterized in that when the instructions are executed on a communication device, the communication device executes the performance monitoring method of the artificial intelligence AI model as in the first aspect, or executes the performance monitoring method of the artificial intelligence AI model as in the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the background technology, the drawings required for use in the embodiments of the present disclosure or the background technology will be described below.

[0014] FIG1 is a schematic diagram of the architecture of a communication system according to an embodiment of the present disclosure;

[0015] FIG2 is a schematic diagram of an observation window and a prediction window in an embodiment of the present disclosure;

[0016] FIG3a is an interactive schematic diagram of a method for monitoring the performance of an artificial intelligence (AI) model according to an embodiment of the present disclosure;

[0017] FIG3 b is an interactive schematic diagram of a method for monitoring the performance of an artificial intelligence (AI) model according to an embodiment of the present disclosure;

[0018] FIG3 c is an interactive schematic diagram of a method for monitoring the performance of an artificial intelligence (AI) model according to an embodiment of the present disclosure;

[0019] FIG3 d is an interactive schematic diagram of a method for monitoring the performance of an artificial intelligence (AI) model according to an embodiment of the present disclosure;

[0020] FIG4a is an interactive schematic diagram of a method for monitoring the performance of an artificial intelligence (AI) model according to an embodiment of the present disclosure;

[0021] FIG4 b is an interactive schematic diagram of a method for monitoring the performance of an artificial intelligence (AI) model according to an embodiment of the present disclosure;

[0022] FIG4c is an interactive schematic diagram of a method for monitoring the performance of an artificial intelligence (AI) model according to an embodiment of the present disclosure;

[0023] FIG5a is an interactive schematic diagram of a method for monitoring the performance of an artificial intelligence (AI) model according to an embodiment of the present disclosure;

[0024] FIG5 b is an interactive schematic diagram of a method for monitoring the performance of an artificial intelligence (AI) model according to an embodiment of the present disclosure;

[0025] FIG5c is an interactive schematic diagram illustrating a method for monitoring the performance of an artificial intelligence (AI) model according to an embodiment of the present disclosure;

[0026] FIG6 is a schematic diagram of an application of a method for monitoring the performance of an AI model according to an embodiment of the present disclosure;

[0027] FIG7 is a schematic diagram of an application of a performance monitoring method for an AI model according to an embodiment of the present disclosure;

[0028] FIG8a is a schematic diagram of the structure of a performance monitoring device for an artificial intelligence (AI) model proposed in an embodiment of the present disclosure;

[0029] FIG8b is a schematic diagram of the structure of a performance monitoring device for an artificial intelligence (AI) model proposed in an embodiment of the present disclosure;

[0030] FIG9a is a schematic structural diagram of a communication device proposed in an embodiment of the present disclosure;

[0031] FIG9 b is a schematic structural diagram of a chip proposed in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] The embodiments of the present disclosure provide methods and devices for monitoring the performance of artificial intelligence (AI) models, as well as communication equipment, communication systems, and storage media. In some embodiments, the terms "artificial intelligence (AI) model performance monitoring method" and "information processing method" and "communication method" are interchangeable; "information transmission device" and "information processing device" and "communication device" are interchangeable; and "information processing system" and "communication system" are interchangeable.

[0033] The embodiments of the present disclosure are not exhaustive and are merely illustrative of some embodiments, and are not intended to be a specific limitation on the scope of protection of the present disclosure. In the absence of contradiction, each step in a certain embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a certain embodiment can also be implemented as an independent embodiment, and the order of the steps in a certain embodiment can be arbitrarily exchanged. In addition, the optional implementation methods in a certain embodiment can be arbitrarily combined; in addition, the embodiments can be arbitrarily combined. For example, some or all steps of different embodiments can be arbitrarily combined, and a certain embodiment can be arbitrarily combined with the optional implementation methods of other embodiments.

[0034] In each embodiment of the present disclosure, unless otherwise specified or provided for by logic, the terms and / or descriptions between the embodiments are consistent and can be referenced by each other. The technical features in different embodiments can be combined to form a new embodiment based on their inherent logical relationships.

[0035] The terms used in the embodiments of the present disclosure are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure.

[0036] In the embodiments of the present disclosure, unless otherwise specified, elements expressed in the singular, such as "a", "an", "the", "above", "the", "the", etc., may mean "one and only one", or "one or more", "at least one", etc. For example, when articles such as "a", "an", "the" in English are used in translation, the noun following the article may be understood as a singular expression or a plural expression.

[0037] In the embodiments of the present disclosure, “plurality” refers to two or more.

[0038] In some embodiments, the terms "at least one," "one or more," "a plurality of," "multiple," etc. may be used interchangeably.

[0039] In some embodiments, descriptions such as "at least one of A and B," "A and / or B," "A in one case, B in another case," or "in response to one case A, in response to another case B" may include the following technical solutions depending on the situation: in some embodiments, A (A is executed independently of B); in some embodiments, B (B is executed independently of A); in some embodiments, execution is selected from A and B (A and B are selectively executed); and in some embodiments, A and B (both A and B are executed). The above is also applicable when there are more branches such as A, B, and C.

[0040] In some embodiments, "A or B" and other descriptions may include the following technical solutions depending on the situation: in some embodiments, A (A is executed independently of B); in some embodiments, B (B is executed independently of A); in some embodiments, execution is selected from A and B (A and B are selectively executed). The above is also applicable when there are more branches such as A, B, C, etc.

[0041] The prefixes such as "first" and "second" in the embodiments of the present disclosure are only used to distinguish different description objects and do not constitute any restriction on the position, order, priority, quantity or content of the description objects. For the statement of the description object, please refer to the description in the context of the claims or embodiments, and no unnecessary restriction should be constituted due to the use of prefixes. For example, if the description object is a "field", the ordinal number before the "field" in the "first field" and the "second field" does not limit the position or order between the "fields". "First" and "second" do not limit whether the "fields" they modify are in the same message, nor do they limit the order of the "first field" and the "second field". For another example, if the description object is a "level", the ordinal number before the "level" in the "first level" and the "second level" does not limit the priority between the "levels". For another example, the number of description objects is not limited by the ordinal number and can be one or more. Taking "first device" as an example, the number of "devices" can be one or more. In addition, the objects modified by different prefixes can be the same or different. For example, if the description object is "device", then the "first device" and the "second device" can be the same device or different devices, and their types can be the same or different; for another example, if the description object is "information", then the "first information" and the "second information" can be the same information or different information, and their contents can be the same or different.

[0042] In some embodiments, “including A,” “comprising A,” “used to indicate A,” and “carrying A” can be interpreted as directly carrying A or indirectly indicating A.

[0043] In some embodiments, terms such as "in response to...", "in response to determining...", "in the case of...", "at the time of...", "when...", "if...", "if...", etc. can be used interchangeably.

[0044] In some embodiments, terms such as "greater than", "greater than or equal to", "not less than", "more than", "more than or equal to", "not less than", "higher than", "higher than or equal to", "not less than", and "above" can be replaced with each other, and terms such as "less than", "less than or equal to", "not greater than", "less than", "less than or equal to", "not more than", "lower than", "lower than or equal to", "not higher than", and "below" can be replaced with each other.

[0045] In some embodiments, devices, etc. can be interpreted as physical or virtual, and their names are not limited to the names recorded in the embodiments. Terms such as "device", "equipment", "device", "circuit", "network element", "node", "function", "unit", "section", "system", "network", "chip", "chip system", "entity", and "subject" can be used interchangeably.

[0046] In some embodiments, the terms "access network device (AN device)", "radio access network device (RAN device)", "base station (BS)", "radio base station" "fixed station", "node", "access point", "transmission point (TP)", "reception point (RP)", "transmission / reception point (TRP)", "antenna panel", "antenna panel", "antenna array", "cell", "macro cell", "small cell", "femto cell", "pico cell", "sector", "cell group", "carrier", "component carrier", "bandwidth part (BWP)" and the like may be used interchangeably.

[0047] In some embodiments, the terms "terminal", "terminal device", "user equipment (UE)", "user terminal", "mobile station (MS)", "mobile terminal (MT)", subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, etc. can be used interchangeably.

[0048] In some embodiments, obtaining data, information, etc. may comply with the laws and regulations of the country where the data is obtained.

[0049] In some embodiments, data, information, etc. may be obtained with the user's consent.

[0050] In addition, each element, each row, or each column in the table of the embodiment of the present disclosure can be implemented as an independent embodiment, and the combination of any elements, any rows, and any columns can also be implemented as an independent embodiment.

[0051] Figure 1 is a schematic diagram of the architecture of a communication system according to an embodiment of the present disclosure. As shown in Figure 1, the communication system 100 may include a terminal 101 and a network device 102. The network device 102 may include at least one of an access network device and a core network device.

[0052] In some embodiments, the terminal 101 includes, for example, a mobile phone, a wearable device, an Internet of Things device, a car with communication function, a smart car, a tablet computer, a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal device in industrial control, a wireless terminal device in self-driving, a wireless terminal device in remote medical surgery, a wireless terminal device in a smart grid, a wireless terminal device in transportation safety, a wireless terminal device in a smart city, and at least one of a wireless terminal device in a smart home, but is not limited thereto.

[0053] In some embodiments, the access network device is, for example, a node or device that accesses a terminal to a wireless network. The access network device may include an evolved NodeB (eNB) in a 5G communication system, a next generation evolved NodeB (ng-eNB), a next generation NodeB (gNB), a node B (NB), a home node B (HNB), a home evolved nodeB (HeNB), a wireless backhaul device, a radio network controller (RNC), a base station controller (BSC), a base transceiver station (BTS), a base band unit (BBU), a mobile switching center, a base station in a 6G communication system, an open base station (Open RAN), a cloud base station (Cloud RAN), a base station in other communication systems, and at least one of an access node in a WiFi system, but is not limited thereto.

[0054] In some embodiments, the technical solution of the present disclosure can be applied to the Open RAN architecture. In this case, the interfaces between or within the access network devices involved in the embodiments of the present disclosure can be transformed into internal interfaces of the Open RAN, and the processes and information interactions between these internal interfaces can be implemented through software or programs.

[0055] In some embodiments, the access network device can be composed of a centralized unit (CU) and a distributed unit (DU), where the CU can also be called a control unit. The CU-DU structure can be used to split the protocol layer of the access network device, with the functions of some protocol layers centrally controlled by the CU, and the functions of the remaining part or all of the protocol layers distributed in the DU, which is centrally controlled by the CU, but is not limited to this.

[0056] In some embodiments, a core network device may be a single device comprising one or more network elements, or may be a plurality of devices or device groups, each comprising all or part of one or more network elements. A network element may be virtual or physical. The core network may, for example, comprise at least one of an Evolved Packet Core (EPC), a 5G Core Network (5GCN), or a Next Generation Core (NGC).

[0057] It can be understood that the communication system described in the embodiment of the present disclosure is for the purpose of more clearly illustrating the technical solution of the embodiment of the present disclosure, and does not constitute a limitation on the technical solution proposed in the embodiment of the present disclosure. Ordinary technicians in this field can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solution proposed in the embodiment of the present disclosure is also applicable to similar technical problems.

[0058] The following embodiments of the present disclosure may be applied to the communication system 100 shown in Figure 1, or a portion thereof, but are not limited thereto. The entities shown in Figure 1 are illustrative only. The communication system may include all or part of the entities shown in Figure 1, or may include other entities outside of Figure 1. The number and form of the entities may be arbitrary. The connection relationship between the entities is illustrative only. The entities may be connected or disconnected, and the connection may be in any manner, including direct or indirect, wired or wireless.

[0059] The embodiments of the present disclosure can be applied to Long Term Evolution (LTE), LTE-Advanced (LTE-A), LTE-Beyond (LTE-B), SUPER 3G, IMT-Advanced, 4th generation mobile communication system (4G), 5th generation mobile communication system (5G), 5G new radio (NR), future radio access (FRA), new radio access technology (RAT), new radio (NR), new radio access (NX), future generation radio access (FX), Global System for Mobile communications (GSM (registered trademark)), CDMA2000, Ultra Mobile Broadband (UMB), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.20, Ultra-WideBand (UWB), Bluetooth (registered trademark), Public Land Mobile Network (PLMN) networks, Device-to-Device (D2D) systems, Machine-to-Machine (M2M) systems, Internet of Things (IoT) systems, Vehicle-to-Everything (V2X), systems utilizing other communication methods, and next-generation systems based on and extending these methods. Furthermore, multiple systems may be combined (for example, a combination of LTE or LTE-A with 5G).

[0060] Optionally, both the AI ​​model-based CSI prediction and the non-AI model-based CSI prediction utilize CSI at multiple historical moments.

[0061] Optionally, downlink channel information is estimated based on channel state information reference signals (CSI-RS) sent at multiple historical moments, and a time range within which the multiple historical moments are located may be referred to as an observation window.

[0062] Optionally, the CSI at multiple future moments is predicted, and the time range within which the multiple moments are located may be referred to as a prediction window.

[0063] Alternatively, as shown in FIG2 , FIG2 is a schematic diagram of an observation window and a prediction window in an embodiment of the present disclosure. FIG2 shows a schematic diagram of predicting CSI in a future prediction window based on CSI at historical moments in an observation window under different parameter configurations. The parameters are defined as follows:

[0064] N represents the number of times CSI-RS is sent within the observation window, where N = 4, 5, 8, or 10.

[0065] M represents the interval between adjacent CSI-RSs, M = 2.5 slots (time slots), or 4 slots, or 5 slots;

[0066] K represents the CSI predicted at K moments in the prediction window, K = 1, 3, or 4;

[0067] D represents the CSI interval between adjacent moments predicted by the prediction window, D=1 slot, or 2.5 slots, or 4 slots, or 5 slots, or 8 slots.

[0068] The length of the prediction window w d =K·D.

[0069] Optionally, for CSI prediction based on an AI model, performance monitoring of the AI ​​model may be performed.

[0070] Optionally, if it is determined that the performance of the AI ​​model reasoning has deteriorated, the AI ​​model may be deactivated, or switched to a new AI model to predict CSI based on the new AI model, or fall back to predicting CSI based on a non-AI model.

[0071] In the embodiments of the present disclosure, it is possible to effectively monitor the performance of the AI ​​model used to predict CSI and prevent the prediction performance from being degraded.

[0072] In the disclosed embodiment, the performance monitoring of the AI ​​model on the terminal side is achieved by sending a downlink reference signal within the prediction window or predicting CSI based on a non-AI model as a reference.

[0073] Figure 3a is an interactive diagram of a method for monitoring the performance of an artificial intelligence (AI) model according to an embodiment of the present disclosure. As shown in Figure 3a, the embodiment of the present disclosure relates to a method for monitoring the performance of an artificial intelligence (AI) model, which is used in a communication system 100. The method includes:

[0074] Step S3101: The network device sends a downlink reference signal to the terminal at a second predicted time.

[0075] In some embodiments, the terminal receives a downlink reference signal sent by the network device at the second predicted time.

[0076] In some embodiments, the time at which the CSI is to be predicted may be referred to as a prediction time.

[0077] In some embodiments, the corresponding CSI may be predicted based on a downlink reference signal sent by the network device at a predicted time, and the predicted time may be referred to as a second predicted time.

[0078] In some embodiments, the number of the second prediction moments may be one or more.

[0079] In some embodiments, the terminal may receive a downlink reference signal sent by the network device at a second predicted time.

[0080] In some embodiments, the terminal may receive multiple downlink reference signals respectively sent by the network device at multiple second predicted time instants.

[0081] In some embodiments, the downlink reference signal may be M ≥ 1 periodic reference signal resources, or N ≥ 1 reference non-periodic reference signal resources, or a combination of M periodic reference signal resources and N non-periodic reference signal resources, where M and N may be positive integers greater than or equal to 1.

[0082] In some embodiments, the reference signal resource may be a CSI-RS resource or a demodulation reference signal (DMRS) resource, etc.

[0083] In some embodiments, the second predicted time is within the prediction window.

[0084] In some embodiments, when the number of the second prediction moment is one, the second prediction moment is the first moment in the prediction window, or the second moment in the prediction window. moment, or the first moment within the prediction window moment, or the last moment in the prediction window, where w d represents the length of the prediction window, Indicates rounding up. Indicates rounding down.

[0085] In some embodiments, when the number of second prediction moments is two, the second prediction moments are the first moment in the prediction window and the second moment in the prediction window. moment, or the first moment in the prediction window and the moment, or the first moment within the prediction window The first moment and the last moment in the prediction window, or the first moment in the prediction window The first moment and the last moment in the prediction window, or the first moment in the prediction window and the last moment in the prediction window, where w d represents the length of the prediction window, Indicates rounding up. Indicates rounding down.

[0086] In some embodiments, when the number of second prediction moments is greater than two, the second prediction moment is the first prediction moment in the prediction window. moment, or the first moment within the prediction window moments, l=1,…,L, L represents the number of the second prediction moments, where w d represents the length of the prediction window, Indicates rounding up. Indicates rounding down.

[0087] This allows for flexible and accurate determination of one or more second prediction moments, effectively adapting to personalized communication systems. Furthermore, when the CSI used as a reference for AI model performance monitoring is determined based on the downlink reference signal sent by the network device at the second prediction moment, accuracy can be effectively improved.

[0088] Step S3102: The terminal determines a reference CSI according to a downlink reference signal.

[0089] In some embodiments, the reference CSI is CSI used as a reference for monitoring the performance of the AI ​​model.

[0090] In some embodiments, CSI may be estimated based on a downlink reference signal, and the estimated CSI may be referred to as reference CSI.

[0091] In some embodiments, when performing performance monitoring reference on an AI model, the CSI estimated based on a downlink reference signal may be used as a reference CSI.

[0092] In some embodiments, one reference CSI may be estimated based on one downlink reference signal, or multiple reference CSIs may be estimated based on multiple downlink reference signals.

[0093] In some embodiments, CSI can be the full channel information or the eigenvector of the full channel information in the spatial and frequency domains, or the coefficient information in the angle and delay domains, or the full channel information or the eigenvector of the full channel information in the time, spatial and frequency domains, or the PMI calculated based on the Type II codebook.

[0094] Therefore, the terminal can receive the downlink reference signal sent by the network device at the second prediction moment, and determine the reference CSI used as a reference for AI model performance monitoring based on the downlink reference signal, thereby effectively improving the accuracy and convenience of AI model performance monitoring and improving the performance monitoring effect of the AI ​​model.

[0095] In step S3103 , the terminal determines a predicted CSI, where the predicted CSI is predicted by the AI ​​model and corresponds to the first prediction moment.

[0096] In some embodiments, the reference CSI and the predicted CSI are used together to determine the performance information of the AI ​​model, and the first prediction moment and the second prediction moment are the same or different.

[0097] In some embodiments, an AI model can be used to predict the CSI at one or more corresponding prediction moments. The prediction moment can be referred to as a first prediction moment. The first prediction moment can be the same prediction moment as the second prediction moment, or the first prediction moment can be a prediction moment different from the second prediction moment.

[0098] In some embodiments, the CSI predicted based on the AI ​​model may be referred to as predicted CSI.

[0099] In some embodiments, the accuracy of the predicted CSI output by the AI ​​model may be evaluated based on the reference CSI to determine the performance of the AI ​​model.

[0100] In some embodiments, the reference CSI and the predicted CSI may be used jointly to determine the performance information of the AI ​​model.

[0101] In some embodiments, performance information can be used to represent the performance of the AI ​​model.

[0102] In some embodiments, the reference CSI and the predicted CSI may be jointly used to determine a monitoring value of the AI ​​model performance, and the performance information of the AI ​​model may be determined based on the monitoring value.

[0103] In step S3104, the terminal determines a monitoring value representing the performance of the AI ​​model based on the reference CSI and the predicted CSI.

[0104] In some embodiments, the monitoring value may be, for example, a Key Performance Indicator (KPI).

[0105] In some embodiments, the monitored value may be, for example, a KPI between the reference CSI and the predicted CSI.

[0106] In some embodiments, the monitoring value may be, for example, a KPI between a precoding matrix indication (PMI) corresponding to the reference CSI and the PMI corresponding to the predicted CSI.

[0107] In some embodiments, if the reference CSI is estimated based on a downlink reference signal, a first key performance indicator KPI between the reference CSI and the predicted CSI may be calculated, and the first KPI may be used as a monitoring value, where .

[0108] In some embodiments, the first KPI refers to a KPI between a reference CSI estimated based on a downlink reference signal and a predicted CSI predicted based on an AI model.

[0109] In some embodiments, a first KPI between the reference CSI estimated based on the downlink reference signal and the predicted CSI may be calculated, and the first KPI may be used as a monitoring value.

[0110] In some embodiments, a KPI may be calculated based on a reference CSI determined based on a received downlink reference signal and a predicted CSI predicted based on an AI model.

[0111] In some embodiments, if the reference CSI is estimated based on a downlink reference signal, a eigenvector may be calculated based on the reference CSI, and a fourth KPI between the eigenvector and the second PMI may be calculated, and the fourth KPI may be used as a monitoring value.

[0112] In some embodiments, the fourth KPI may refer to a KPI between a feature vector calculated based on the CSI and a second PMI corresponding to the predicted CSI.

[0113] In some embodiments, a KPI may be calculated based on a feature vector calculated based on a reference CSI determined based on a received downlink reference signal and a PMI determined based on a predicted CSI predicted by an AI model.

[0114] In some embodiments, the various KPIs mentioned above may be, for example, intermediate KPIs, such as Square Generalized Cosine Similarity (SGCS) or Normalized Mean Square Error (NMSE).

[0115] In some embodiments, the terminal may further calculate a final KPI based on the various intermediate KPIs mentioned above.

[0116] In some embodiments, the final KPI may include at least one of the following: system throughput, network block error rate (BLER), acknowledgment character ACK, and non-acknowledgment character NACK.

[0117] In this way, the joint reference CSI and predicted CSI can be accurately analyzed to obtain the monitoring value representing the performance of the AI ​​model. When the AI ​​model performance information is determined based on the monitoring value, the accuracy and monitoring effect of the AI ​​model performance monitoring can be effectively improved.

[0118] Step S3105: The terminal sends first indication information, where the first indication information includes: a monitoring value and / or information associated with the monitoring value.

[0119] In some embodiments, the information associated with the monitoring value may be, for example, information on a comparison result between the monitoring value and a threshold value, or may be any other possible information related to the monitoring value.

[0120] In some embodiments, the information associated with the monitoring value may be, for example, a final KPI calculated based on an intermediate KPI, or performance information of an AI model determined based on the monitoring value.

[0121] In some embodiments, the terminal may report the monitoring value and / or information associated with the monitoring value to the network device.

[0122] In some embodiments, the terminal may carry the monitoring value and / or information associated with the monitoring value in the first indication information, and report the monitoring value and / or information associated with the monitoring value by sending the first indication information to the network device.

[0123] In some embodiments, the number of reference CSIs is multiple, the number of predicted CSIs is multiple, different reference CSIs correspond to different first prediction moments or second prediction moments, and different predicted CSIs correspond to different first prediction moments. Accordingly, the number of first KPIs is multiple, and / or the number of second KPIs is multiple, and / or the number of third KPIs is multiple, and the number of fourth KPIs is multiple, so as to effectively apply to personalized scenarios of AI model performance monitoring and effectively improve the accuracy of AI model performance monitoring.

[0124] Therefore, the terminal can send a first indication message to the network device, wherein the first indication message includes: monitoring values ​​and / or information associated with the monitoring values, so as to support the network device to analyze the AI ​​model performance based on the monitoring values ​​and / or information associated with the monitoring values, thereby effectively reducing the computing consumption of AI model performance monitoring on the terminal side.

[0125] In some embodiments, a first indication message is sent, wherein the first indication message includes: a preset number of monitoring values, and / or information associated with each monitoring value.

[0126] In some embodiments, the preset number can be represented by K, and K can be a positive integer greater than or equal to 1.

[0127] In some embodiments, the preset number may be less than or equal to the total number at the second prediction moment (which may be represented by L).

[0128] In some embodiments, the terminal may report K KPIs at a time, where the KPI is any one of the multiple KPIs mentioned above.

[0129] In some embodiments, a first indication message may be sent, where the first indication message carries K KPIs, or carries information associated with each of the K KPIs, such as a comparison result between each KPI and a corresponding threshold value.

[0130] In some embodiments, a preset number of first indication information are sent respectively, wherein each first indication information includes: a monitoring value, and / or information associated with a monitoring value.

[0131] In some embodiments, the terminal may report K KPIs multiple times, where the KPI is any one of the multiple KPIs mentioned above.

[0132] In some embodiments, multiple first indication messages may be sent, each of which carries a KPI or information associated with a KPI, such as a comparison result between a KPI and a corresponding threshold value.

[0133] In this way, it is possible to flexibly send monitoring values ​​representing AI model performance and / or information associated with each monitoring value, which is effectively applicable to personalized communication scenarios.

[0134] In some embodiments, the terminal may receive second indication information sent by the network device, where the second indication information includes a preset number, and determine the preset number indicated by the second indication information.

[0135] In some embodiments, the terminal may determine the preset number based on a protocol agreement.

[0136] In some embodiments, the terminal may also report a preset number to the network device.

[0137] In this way, the reporting quantity of the monitoring value can be accurately determined, and the reporting quantity can also be flexibly configured, thereby effectively applying to personalized communication scenarios.

[0138] In some embodiments, the information associated with the monitoring value includes: a comparison result between the monitoring value and its corresponding monitoring threshold value. Thus, not only the monitoring value can be reported to the network device, but also the comparison result between the monitoring value and its corresponding monitoring threshold value can be reported to the network device. This not only reduces the feedback overhead reported by the UE, but also enables the network device to fully understand the performance of the AI ​​model, thereby supporting the network device to accurately and flexibly determine the performance information of the AI ​​model.

[0139] In some embodiments, the terminal may receive second indication information sent by the network device, and determine the monitoring threshold value included in the second indication information.

[0140] In some embodiments, the terminal may determine the monitoring threshold based on a protocol agreement.

[0141] In some embodiments, the terminal may also report a monitoring threshold to the network device.

[0142] Thus, it is possible to accurately determine the monitoring threshold value corresponding to the monitoring value, and the monitoring threshold value corresponding to the monitoring value can also be flexibly configured, thereby effectively applying to personalized communication scenarios.

[0143] Step S3106: The network device determines performance information based on the monitored value and / or information associated with the monitored value.

[0144] In some embodiments, the network device may receive monitoring values ​​and / or information associated with the monitoring values ​​that are reported by the terminal and represent the performance of the AI ​​model.

[0145] In some embodiments, the network device may receive a first indication message, wherein the first indication message includes: a preset number of monitoring values, and / or information associated with each monitoring value.

[0146] In some embodiments, the network device may receive K KPIs reported by the terminal at one time, where the KPI is any one of the multiple KPIs mentioned above.

[0147] In some embodiments, the network device may respectively receive a preset number (eg, K) of first indication information, wherein each first indication information includes: a monitoring value, and / or information associated with a monitoring value.

[0148] In some embodiments, the network device may receive K KPIs reported by the terminal multiple times, where the KPI is any one of the multiple KPIs mentioned above.

[0149] In some embodiments, the network device may send second indication information and indicate a preset number based on the second indication information.

[0150] In some embodiments, the network device may determine the preset number based on a protocol agreement.

[0151] In some embodiments, the network device may receive a preset number reported by the terminal.

[0152] In some embodiments, the information associated with the monitoring value includes: a comparison result between the monitoring value and its corresponding monitoring threshold value.

[0153] In some embodiments, the network device may send second indication information and indicate a monitoring threshold value to the terminal based on the second indication information.

[0154] In some embodiments, the network device may determine the monitoring threshold based on a protocol agreement.

[0155] In some embodiments, the network device may receive a monitoring threshold value reported by the terminal.

[0156] In some embodiments, the network device can determine the performance information of the AI ​​model based on the monitoring values ​​reported by the terminal device and / or information associated with the monitoring values.

[0157] In some embodiments, the network device may calculate a final KPI based on various intermediate KPIs reported by the terminal.

[0158] In some embodiments, the final KPI may include at least one of the following: system throughput, network block error rate (BLER), acknowledgment character ACK, and non-acknowledgment character NACK.

[0159] The performance monitoring method of the artificial intelligence (AI) model involved in the embodiments of the present disclosure may include at least one of steps S3101 to S3106. For example, step S3101 can be implemented as an independent embodiment, step S3102 can be implemented as an independent embodiment, and so on, but are not limited thereto. Steps S3101+S3102 can be implemented as independent embodiments, and steps S3101+S3102+S3103 can be implemented as independent embodiments, but are not limited thereto.

[0160] In this embodiment or example, unless there is any contradiction, each step can be independent, arbitrarily combined or exchanged in order, the optional methods or optional examples can be arbitrarily combined, and can be arbitrarily combined with any steps of other embodiments or other examples.

[0161] In this embodiment, the terminal may receive a downlink reference signal sent by the network device at the second prediction moment, and determine, based on the downlink reference signal, a reference CSI for reference in AI model performance monitoring, and determine the predicted CSI corresponding to the first prediction moment predicted by the AI ​​model, wherein the reference CSI and the predicted CSI are jointly used to determine the performance information of the AI ​​model, and the first prediction moment and the second prediction moment are the same or different. Based on the reference CSI and the predicted CSI, a monitoring value representing the performance of the AI ​​model is determined. The terminal may send first indication information to the network device, wherein the first indication information includes: a monitoring value and / or information associated with the monitoring value, and the network device determines performance information based on the monitoring value and / or information associated with the monitoring value, so as to support the network device in analyzing the AI ​​model performance based on the monitoring value and / or information associated with the monitoring value, thereby effectively reducing the computing consumption of AI model performance monitoring on the terminal side. It can also effectively improve the accuracy and convenience of AI model performance monitoring and improve the performance monitoring effect of the AI ​​model.

[0162] Figure 3b is an interactive diagram of a method for monitoring the performance of an artificial intelligence (AI) model according to an embodiment of the present disclosure. As shown in Figure 3b, the embodiment of the present disclosure relates to a method for monitoring the performance of an artificial intelligence (AI) model, which is used in a communication system 100. The method includes:

[0163] In step S3201, the terminal determines a reference CSI corresponding to a first prediction moment based on a non-AI model method, where the non-AI model method is a method of predicting CSI without relying on an AI model.

[0164] In some embodiments, the reference CSI is CSI used as a reference for monitoring the performance of the AI ​​model.

[0165] In some embodiments, the reference CSI may also be CSI corresponding to one or more first prediction moments that is predicted based on a non-AI model method.

[0166] In some embodiments, the non-AI model approach is an approach that does not rely on an AI model to predict CSI.

[0167] In some embodiments, the non-AI model approach is, for example, an autoregressive model (AR) algorithm.

[0168] In step S3202, the terminal determines a predicted CSI, where the predicted CSI is predicted by the AI ​​model and corresponds to the first prediction moment.

[0169] In some embodiments, the reference CSI and the predicted CSI are used together to determine the performance information of the AI ​​model.

[0170] In step S3203, the terminal determines a monitoring value representing the performance of the AI ​​model based on the reference CSI and the predicted CSI.

[0171] In some embodiments, if the reference CSI is predicted based on a non-AI model, a second KPI between the reference CSI and the predicted CSI may be calculated, and the second KPI may be used as a monitoring value.

[0172] In some embodiments, the second KPI may refer to a KPI between a reference CSI predicted based on a non-AI model method and a predicted CSI predicted based on an AI model.

[0173] In some embodiments, if the reference CSI is predicted based on a non-AI model method, a first precoding matrix indication PMI corresponding to the reference CSI can be determined, and a second PMI corresponding to the predicted CSI can be determined, a third KPI between the first PMI and the second PMI can be calculated, and the third KPI can be used as a monitoring value.

[0174] In some embodiments, the third KPI may refer to a KPI between the PMI corresponding to the reference CSI and the PMI corresponding to the predicted CSI.

[0175] In some embodiments, another KPI may be determined based on a PMI calculated based on a reference CSI predicted by a non-AI model method and a PMI calculated based on a predicted CSI predicted by an AI model.

[0176] In some embodiments, the various KPIs described above can be used as monitoring values ​​to represent the performance of the AI ​​model.

[0177] In some embodiments of the present disclosure, the reference CSI determined based on the downlink reference signal and the reference CSI predicted based on a non-AI model method can be jointly used to jointly analyze the performance information of the AI ​​model, thereby further improving the flexibility and effectiveness of the performance information determination of the AI ​​model and improving the reliability of the performance information analysis of the AI ​​model.

[0178] In some embodiments of the present disclosure, if the reference CSI determined based on the downlink reference signal and the reference CSI predicted based on the non-AI model method are jointly analyzed to analyze the performance information of the AI ​​model, a part of the reference CSI can be estimated based on the downlink reference signal sent and received by the network device at one or more second prediction moments, and another part of the reference CSI can be predicted based on the non-AI model method, and the performance information of the AI ​​model can be jointly analyzed based on the reference CSI obtained by the above two methods.

[0179] In some embodiments, the one or more second predicted moments may be determined based on the following methods:

[0180] In some embodiments, the second predicted time is within the prediction window.

[0181] In some embodiments, when the number of the second prediction moment is one, the second prediction moment is the first moment in the prediction window, or the second moment in the prediction window. moment, or the first moment in the prediction window moment, or the last moment in the prediction window, where w d represents the length of the prediction window, Indicates rounding up. Indicates rounding down.

[0182] In some embodiments, when the number of second prediction moments is two, the second prediction moments are the first moment in the prediction window and the second moment in the prediction window. moment, or the first moment in the prediction window and the moment, or the first moment in the prediction window The first moment and the last moment in the prediction window, or the first moment in the prediction window The first moment and the last moment in the prediction window, or the first moment in the prediction window and the last moment in the prediction window, where w d represents the length of the prediction window, Indicates rounding up. Indicates rounding down.

[0183] In some embodiments, when the number of second prediction moments is greater than two, the second prediction moment is the first prediction moment in the prediction window. moment, or the first moment within the prediction window moments, l=1,…,L, L represents the number of the second prediction moments, where w d represents the length of the prediction window, Indicates rounding up. Indicates rounding down.

[0184] Step S3204: The terminal sends first indication information, where the first indication information includes: a monitoring value and / or information associated with the monitoring value.

[0185] Step S3205: The network device determines performance information based on the monitored value and / or information associated with the monitored value.

[0186] The performance monitoring method of the artificial intelligence (AI) model involved in the embodiments of the present disclosure may include at least one of steps S3201 to S3205. For example, step S3201 can be implemented as an independent embodiment, step S3202 can be implemented as an independent embodiment, and so on, but are not limited thereto. Steps S3201+S3202 can be implemented as independent embodiments, and steps S3201+S3202+S3203 can be implemented as independent embodiments, but are not limited thereto.

[0187] In this embodiment or example, unless there is any contradiction, each step can be independent, arbitrarily combined or exchanged in order, the optional methods or optional examples can be arbitrarily combined, and can be arbitrarily combined with any steps of other embodiments or other examples.

[0188] In this embodiment, the terminal determines the reference CSI corresponding to the first prediction moment based on a non-AI model method, wherein the non-AI model method is a method that does not rely on the AI ​​model to predict the CSI, the reference CSI is the CSI used as a reference for AI model performance monitoring, and determines the predicted CSI, wherein the predicted CSI is predicted by the AI ​​model and corresponds to the first prediction moment, the reference CSI and the predicted CSI are used together to determine the performance information of the AI ​​model, and according to the reference CSI and the predicted CSI, a monitoring value representing the performance of the AI ​​model is determined, and a first indication information is sent, wherein the first indication information includes: the monitoring value and / or information associated with the monitoring value, and the network device determines the performance information according to the monitoring value and / or information associated with the monitoring value, which can effectively improve the flexibility, accuracy and convenience of AI model performance monitoring and improve the performance monitoring effect of the AI ​​model.

[0189] Figure 3c is an interactive diagram of a method for monitoring the performance of an artificial intelligence (AI) model according to an embodiment of the present disclosure. As shown in Figure 3c, the embodiment of the present disclosure relates to a method for monitoring the performance of an artificial intelligence (AI) model, which is used in a communication system 100. The method includes:

[0190] Step S3301: The terminal determines reference channel state information CSI and predicted CSI.

[0191] In some embodiments, the reference CSI is used as a reference for AI model performance monitoring, the predicted CSI is predicted by the AI ​​model and corresponds to the first prediction moment, and the reference CSI and the predicted CSI are used together to determine the performance information of the AI ​​model.

[0192] In step S3302, the terminal generates a partial model based on the CSI to process the reference CSI to obtain a first processed CSI, and generates a partial model based on the CSI to process the predicted CSI to obtain a second processed CSI, wherein the first processed CSI and the second processed CSI are used together to determine the performance information of the AI ​​model.

[0193] In some embodiments, a terminal device may feed back the Channel Status Information (CSI) to a network device via a CSI feedback model. Specifically, the terminal device compresses the CSI using the CSI generation model, quantizes it into a binary bit stream, and sends it to the network device. The network device then recovers the CSI using the CSI recovery model.

[0194] In some embodiments, the terminal may input the reference CSI into a CSI generation model and obtain the CSI after compression processing by the CSI generation model. The compressed CSI may be referred to as first processed CSI.

[0195] In some embodiments, the terminal may input the predicted CSI into a CSI generation model, and obtain the CSI after compression processing by the CSI generation model. The compressed CSI may be referred to as second processed CSI.

[0196] In some embodiments, performance information of the AI ​​model may be determined based on the first processed CSI and the second processed CSI.

[0197] In some embodiments, there are multiple reference CSIs, and different reference CSIs correspond to different first prediction moments or second prediction moments.

[0198] In some embodiments, the terminal may process multiple reference CSIs respectively based on the CSI generation partial model to obtain multiple first processed CSIs.

[0199] In some embodiments, a reference CSI can be used as input to the CSI generation partial model, and compression processing can be performed on the reference CSI based on the CSI generation partial model to obtain a first processed CSI corresponding to the reference CSI. Each reference CSI is processed accordingly to obtain multiple first processed CSIs.

[0200] In some embodiments, multiple reference CSIs may be inputted into a CSI generation partial model, so as to jointly process the multiple reference CSIs based on the CSI generation partial model to obtain a first processed CSI.

[0201] In this way, the reference CSI can be compressed flexibly and effectively to help improve the reporting timeliness of the first processed CSI obtained by compression processing, so that the network equipment can quickly monitor the performance information of the AI ​​model.

[0202] In some embodiments, there are multiple predicted CSIs, and different predicted CSIs correspond to different first prediction moments.

[0203] In some embodiments, the terminal may process multiple predicted CSIs respectively based on the CSI generation partial model to obtain multiple second processed CSIs.

[0204] In some embodiments, a predicted CSI can be used as the input of the CSI generation partial model, and the predicted CSI can be compressed based on the CSI generation partial model to obtain a second processed CSI corresponding to the predicted CSI. Each predicted CSI is processed accordingly to obtain multiple second processed CSIs.

[0205] In some embodiments, a plurality of predicted CSIs may be inputted into the CSI generation partial model, so as to jointly process the plurality of predicted CSIs based on the CSI generation partial model to obtain a second processed CSI.

[0206] In this way, the predicted CSI can be compressed flexibly and effectively to help improve the reporting timeliness of the second processed CSI obtained by compression processing, so that the network equipment can quickly monitor the performance information of the AI ​​model.

[0207] Step S3303: The terminal sends third indication information, where the third indication information includes: first processing CSI and second processing CSI.

[0208] In some embodiments, the indication information used to indicate the first processing CSI and the second processing CSI may be referred to as third indication information.

[0209] In some embodiments, the terminal may report third indication information to the network device.

[0210] In some embodiments, the terminal may send third indication information to the network device.

[0211] In some embodiments, the first processing CSI and the second processing CSI may be indicated based on the third indication information.

[0212] Therefore, the terminal can generate partial model processing reference CSI based on CSI to obtain first processing CSI, and generate partial model processing predicted CSI based on CSI to obtain second processing CSI, wherein the first processing CSI and the second processing CSI are jointly used to determine the performance information of the AI ​​model and send third indication information, wherein the third indication information includes: first processing CSI and second processing CSI, supporting network equipment to analyze AI model performance based on the first processing CSI and the second processing CSI, thereby effectively reducing the computing consumption of AI model performance monitoring on the terminal side, and since the reference CSI and the predicted CSI are compressed and then sent, the reporting timeliness of the first processing CSI and the second processing CSI obtained by compression processing can be effectively improved, so that the network equipment can quickly monitor the performance information of the AI ​​model.

[0213] In step S3304 , the network device processes the first processed CSI based on the CSI recovery partial model to obtain the reference CSI, and processes the second processed CSI based on the CSI recovery partial model to obtain the predicted CSI.

[0214] In some embodiments, the network device recovers the channel state information through a CSI recovery partial model.

[0215] In some embodiments, the first processed CSI may be used as an input of a CSI recovery partial model, the reference CSI may be recovered based on the CSI recovery partial model, and the reference CSI output by the CSI recovery partial model may be obtained.

[0216] In some embodiments, the second processed CSI may be used as an input of a CSI recovery partial model, and the predicted CSI may be recovered based on the CSI recovery partial model to obtain the predicted CSI output by the CSI recovery partial model.

[0217] In some embodiments, the number of the first processed CSI is one or more.

[0218] In some embodiments, a first processed CSI may be processed based on a CSI recovery partial model to obtain multiple reference CSIs.

[0219] In some embodiments, a joint recovery process may be performed based on a partial CSI recovery model, for example, multiple reference CSIs may be recovered based on a first processed CSI obtained through a joint compression process.

[0220] In some embodiments, a plurality of first processed CSIs are processed separately based on the CSI recovery partial model to obtain a plurality of reference CSIs.

[0221] In some embodiments, a first processed CSI is used as an input of the CSI recovery partial model to obtain a reference CSI obtained by the CSI recovery partial model recovering the first processed CSI, and each first processed CSI is processed accordingly to obtain multiple reference CSIs.

[0222] In some embodiments, the number of the second processed CSI is one or more.

[0223] In some embodiments, a second processed CSI is processed based on the CSI recovery partial model to obtain multiple predicted CSIs.

[0224] In some embodiments, a joint recovery process may be performed based on a partial CSI recovery model, for example, a plurality of predicted CSIs may be recovered based on a second processed CSI obtained through a joint compression process.

[0225] In some embodiments, a plurality of second processed CSIs are processed separately based on the CSI recovery partial model to obtain a plurality of predicted CSIs.

[0226] In some embodiments, a second processed CSI is used as an input of the CSI recovery partial model to obtain a predicted CSI obtained by the CSI recovery partial model recovering the second processed CSI, and each second processed CSI is processed accordingly to obtain multiple predicted CSIs.

[0227] In some embodiments, the network device may also recover a preset number of reference CSIs or predicted CSIs, such as K.

[0228] In step S3305, the network device determines the performance information of the AI ​​model based on the reference CSI and the predicted CSI.

[0229] In some embodiments, performance information of the AI ​​model is determined based on the reference CSI and the predicted CSI.

[0230] In some embodiments, the reference CSI and the predicted CSI may be processed based on the same or corresponding method as that on the terminal side described above to determine performance information of the AI ​​model.

[0231] In some embodiments, an intermediate KPI may be calculated based on the reference CSI and the predicted CSI, and the performance information of the AI ​​model may be analyzed based on the intermediate KPI.

[0232] In some embodiments, intermediate KPIs and final KPIs can also be calculated based on the reference CSI and the predicted CSI to analyze the performance information of the AI ​​model.

[0233] The performance monitoring method of the artificial intelligence (AI) model involved in the embodiments of the present disclosure may include at least one of steps S3301 to S3305. For example, step S3301 can be implemented as an independent embodiment, step S3302 can be implemented as an independent embodiment, and so on, but are not limited thereto. Steps S3301+S3302 can be implemented as independent embodiments, and steps S3301+S3302+S3303 can be implemented as independent embodiments, but are not limited thereto.

[0234] In this embodiment or example, unless there is any contradiction, each step can be independent, arbitrarily combined or exchanged in order, the optional methods or optional examples can be arbitrarily combined, and can be arbitrarily combined with any steps of other embodiments or other examples.

[0235] In this embodiment, the terminal can generate a partial model based on the CSI to process the reference CSI to obtain the first processed CSI, and generate a partial model based on the CSI to process the predicted CSI to obtain the second processed CSI, wherein the first processed CSI and the second processed CSI are used together to determine the performance information of the AI ​​model and send a third indication information, wherein the third indication information includes: the first processed CSI and the second processed CSI, supporting the network device to analyze the performance of the AI ​​model based on the first processed CSI and the second processed CSI, thereby effectively reducing the computing consumption of the terminal side for AI model performance monitoring, and because the reference CSI and the predicted CSI are compressed and then sent, the reporting timeliness of the first processed CSI and the second processed CSI obtained by the compression processing can be effectively improved. The network device can determine the performance information of the AI ​​model based on the recovered reference CSI and the predicted CSI, and accurately analyze and obtain the performance information of the AI ​​model.

[0236] Figure 3d is an interactive diagram of a method for monitoring the performance of an artificial intelligence (AI) model according to an embodiment of the present disclosure. As shown in Figure 3d, the embodiment of the present disclosure relates to a method for monitoring the performance of an artificial intelligence (AI) model, which is used in a communication system 100. The method includes:

[0237] Step S3401: The terminal determines reference channel state information CSI and predicted CSI.

[0238] In some embodiments, the reference CSI is used as a reference for AI model performance monitoring, the predicted CSI is predicted by the AI ​​model and corresponds to the first prediction moment, and the reference CSI and the predicted CSI are used together to determine the performance information of the AI ​​model.

[0239] In step S3402, the terminal determines a first PMI corresponding to the reference CSI and determines a second PMI corresponding to the predicted CSI, wherein the first PMI and the second PMI are used together to determine performance information of the AI ​​model.

[0240] In some embodiments, the indication information used to indicate the first PMI and the second PMI may be referred to as third indication information.

[0241] In some embodiments, the terminal may report third indication information to the network device.

[0242] In some embodiments, the terminal may send third indication information to the network device.

[0243] In some embodiments, the first PMI and the second PMI may be indicated based on the third indication information.

[0244] Step S3403: The terminal sends third indication information, where the third indication information includes: the first PMI and the second PMI.

[0245] In step S3404, the network device determines performance information of the AI ​​model based on the first PMI and the second PMI.

[0246] In some embodiments, the network device may compare the first PMI and the second PMI to determine performance information of the AI ​​model.

[0247] The performance monitoring method of the artificial intelligence (AI) model involved in the embodiments of the present disclosure may include at least one of steps S3401 to S3404. For example, step S3401 can be implemented as an independent embodiment, step S3402 can be implemented as an independent embodiment, and so on, but are not limited thereto. Steps S3401+S3402 can be implemented as independent embodiments, and steps S3401+S3402+S3403 can be implemented as independent embodiments, but are not limited thereto.

[0248] In this embodiment or example, unless there is any contradiction, each step can be independent, arbitrarily combined or exchanged in order, the optional methods or optional examples can be arbitrarily combined, and can be arbitrarily combined with any steps of other embodiments or other examples.

[0249] In this embodiment, the terminal can determine a first PMI corresponding to the reference CSI and determine a second PMI corresponding to the predicted CSI, wherein the first PMI and the second PMI are used together to determine the performance information of the AI ​​model, and send third indication information, wherein the third indication information includes: the first PMI and the second PMI, supporting the network device to determine the performance information of the AI ​​model based on the first PMI and the second PMI, thereby effectively reducing the computing consumption of the AI ​​model performance monitoring on the terminal side, and because the PMIs corresponding to the reference CSI and the predicted CSI are determined respectively, and the PMIs corresponding to the reference CSI and the predicted CSI are indicated to the network device. The network device can determine the performance information of the AI ​​model based on the PMIs corresponding to the reference CSI and the predicted CSI, respectively, to accurately and quickly analyze and obtain the performance information of the AI ​​model.

[0250] Figure 4a is an interactive diagram of a method for monitoring the performance of an artificial intelligence (AI) model according to an embodiment of the present disclosure. As shown in Figure 4a, an embodiment of the present disclosure relates to a method for monitoring the performance of an artificial intelligence (AI) model, which is used in a terminal. The method includes:

[0251] Step S4101: Determine reference channel state information CSI and predicted CSI, where the reference CSI is used as a reference for AI model performance monitoring, and the predicted CSI is predicted by the AI ​​model and corresponds to the first prediction moment. The reference CSI and the predicted CSI are jointly used to determine the performance information of the AI ​​model.

[0252] For a detailed description of step S4101, please refer to the embodiment shown in FIG3 .

[0253] In this embodiment, the terminal can determine the reference channel state information CSI and the predicted CSI, where the reference CSI is used as a reference for AI model performance monitoring, and the predicted CSI is predicted by the AI ​​model and corresponds to the first prediction moment. The reference CSI and the predicted CSI are jointly used to determine the performance information of the AI ​​model, which can effectively monitor the performance of the AI ​​model used to predict CSI.

[0254] Figure 4b is an interactive diagram of a method for monitoring the performance of an artificial intelligence (AI) model according to an embodiment of the present disclosure. As shown in Figure 4b, an embodiment of the present disclosure relates to a method for monitoring the performance of an artificial intelligence (AI) model, which is used in a terminal. The method includes:

[0255] Step S4201: Determine reference channel state information CSI and predicted CSI, where the reference CSI is used as a reference for AI model performance monitoring, and the predicted CSI is predicted by the AI ​​model and corresponds to the first prediction moment. The reference CSI and the predicted CSI are jointly used to determine the performance information of the AI ​​model.

[0256] Step S4202: Determine a monitoring value representing the performance of the AI ​​model based on the reference CSI and the predicted CSI.

[0257] Step S4203: Send first indication information, where the first indication information includes: monitoring values ​​and / or information associated with the monitoring values.

[0258] For a detailed description of steps S4201 - S4203 , please refer to the embodiment shown in FIG3 .

[0259] The performance monitoring method of the artificial intelligence (AI) model involved in the embodiments of the present disclosure may include at least one of steps S4201 to S4203. For example, step S4201 can be implemented as an independent embodiment, step S4202 can be implemented as an independent embodiment, and so on, but are not limited thereto. Steps S4201+S4202 can be implemented as independent embodiments, and steps S4201+S4202+S4203 can be implemented as independent embodiments, but are not limited thereto.

[0260] In this embodiment, the terminal can determine reference channel state information CSI and predicted CSI, wherein the reference CSI is used as a reference for AI model performance monitoring, the predicted CSI is predicted by the AI ​​model and corresponds to the first prediction moment, and the reference CSI and the predicted CSI are used together to determine the performance information of the AI ​​model, which can effectively monitor the performance of the AI ​​model used to predict the CSI. It is also possible to determine a monitoring value representing the performance of the AI ​​model based on the reference CSI and the predicted CSI, and send a first indication message, wherein the first indication message includes: the monitoring value and / or information associated with the monitoring value. The network device can use the monitoring value and / or information associated with the monitoring value to estimate the performance information of the AI ​​model, which can effectively reduce the computing consumption of the terminal side for AI model performance monitoring.

[0261] Figure 4c is an interactive diagram of a method for monitoring the performance of an artificial intelligence (AI) model according to an embodiment of the present disclosure. As shown in Figure 4c, an embodiment of the present disclosure relates to a method for monitoring the performance of an artificial intelligence (AI) model, which is used in a terminal. The method includes:

[0262] Step S4301: Determine reference channel state information CSI and predicted CSI, where the reference CSI is used as a reference for AI model performance monitoring, and the predicted CSI is predicted by the AI ​​model and corresponds to the first prediction moment. The reference CSI and the predicted CSI are jointly used to determine the performance information of the AI ​​model.

[0263] Step S4302: Process the reference CSI based on the CSI generation part of the model to obtain the first processed CSI, and process the predicted CSI based on the CSI generation part of the model to obtain the second processed CSI, wherein the first processed CSI and the second processed CSI are used together to determine the performance information of the AI ​​model.

[0264] Step S4303: Send third indication information, where the third indication information includes: first processing CSI and second processing CSI.

[0265] For a detailed description of steps S4301-S4303, please refer to the embodiment of FIG3 above.

[0266] The performance monitoring method of the artificial intelligence (AI) model involved in the embodiments of the present disclosure may include at least one of steps S4301 to S4303. For example, step S4301 can be implemented as an independent embodiment, step S4302 can be implemented as an independent embodiment, and so on, but is not limited thereto. Steps S4301+S4302 can be implemented as independent embodiments, and steps S4301+S4302+S4303 can be implemented as independent embodiments, but are not limited thereto.

[0267] In this embodiment, a terminal can determine reference channel state information (CSI) and predicted CSI, wherein the reference CSI is used as a reference for AI model performance monitoring, and the predicted CSI is predicted by the AI ​​model and corresponds to a first prediction time. The reference CSI and the predicted CSI are jointly used to determine the performance information of the AI ​​model, thereby effectively monitoring the performance of the AI ​​model used to predict the CSI. The terminal can also generate partial model-processed reference CSI based on the CSI to obtain first processed CSI, and generate partial model-processed predicted CSI based on the CSI to obtain second processed CSI. The first processed CSI and the second processed CSI are jointly used to determine the performance information of the AI ​​model, and third indication information is sent. The third indication information includes the first processed CSI and the second processed CSI. A network device can use the first processed CSI and the second processed CSI to estimate the performance information of the AI ​​model, supporting the network device to analyze the AI ​​model performance based on the first processed CSI and the second processed CSI. This effectively reduces the computational cost of AI model performance monitoring on the terminal side. Furthermore, because the reference CSI and the predicted CSI are compressed before transmission, the reporting timeliness of the compressed first processed CSI and second processed CSI can be effectively improved.

[0268] Figure 5a is an interactive diagram of a method for monitoring the performance of an artificial intelligence (AI) model according to an embodiment of the present disclosure. As shown in Figure 5a, an embodiment of the present disclosure relates to a method for monitoring the performance of an artificial intelligence (AI) model for a network device, the method comprising:

[0269] Step S5101: Determine the performance information of the AI ​​model, where the performance information is determined by reference channel state information CSI and predicted CSI. The reference CSI is used for AI model performance monitoring reference, and the predicted CSI is predicted by the AI ​​model and corresponds to the first prediction moment.

[0270] For a detailed description of step S5101, please refer to the embodiment shown in FIG3 .

[0271] In this embodiment, the network device can determine the performance information of the AI ​​model, wherein the performance information is determined by the reference channel state information CSI and the predicted CSI. The reference CSI is used as a reference for AI model performance monitoring. The predicted CSI is predicted by the AI ​​model and corresponds to the first prediction moment, which can effectively monitor the performance of the AI ​​model used to predict CSI.

[0272] Figure 5b is an interactive diagram of a method for monitoring the performance of an artificial intelligence (AI) model according to an embodiment of the present disclosure. As shown in Figure 5b, an embodiment of the present disclosure relates to a method for monitoring the performance of an artificial intelligence (AI) model for a network device, the method comprising:

[0273] Step S5201: Receive first indication information, where the first indication information includes: a monitoring value representing the performance of the AI ​​model and / or information associated with the monitoring value, where the monitoring value is determined by a reference CSI and a predicted CSI.

[0274] In some embodiments, the monitoring value includes at least one of the following: a first KPI between the reference CSI and the predicted CSI, wherein the reference CSI is estimated based on a downlink reference signal; a second KPI between the reference CSI and the predicted CSI, wherein the reference CSI is predicted based on a non-AI model method; a third KPI between the first precoding matrix indication PMI corresponding to the reference CSI and the second PMI corresponding to the predicted CSI, wherein the reference CSI is predicted based on a non-AI model method; a fourth KPI between the eigenvector calculated based on the reference CSI and the second PMI, wherein the reference CSI is estimated based on a downlink reference signal.

[0275] Step S5202: Determine performance information based on the monitoring value and / or information associated with the monitoring value.

[0276] For a detailed description of steps S5201 - S5202 , please refer to the embodiment shown in FIG3 .

[0277] The performance monitoring method of the artificial intelligence (AI) model involved in the embodiments of the present disclosure may include at least one of steps S5201 and S5202. For example, step S5201 can be implemented as an independent embodiment, and step S5202 can be implemented as an independent embodiment, but are not limited thereto. Steps S5201+S5202 can be implemented as independent embodiments, but are not limited thereto.

[0278] In this embodiment, the network device can use the monitoring values ​​and / or information associated with the monitoring values ​​to estimate the performance information of the AI ​​model, effectively monitoring the performance of the AI ​​model used to predict CSI. This also supports reducing the computing power of AI model performance monitoring on the terminal side.

[0279] Figure 5c is an interactive diagram of a method for monitoring the performance of an artificial intelligence (AI) model according to an embodiment of the present disclosure. As shown in Figure 5c, an embodiment of the present disclosure relates to a method for monitoring the performance of an artificial intelligence (AI) model for a network device, the method comprising:

[0280] Step S5301: Receive third indication information, where the third indication information includes: first processed CSI and second processed CSI, the first processed CSI is obtained by processing reference CSI based on a CSI generation partial model, and the second processed CSI is obtained by processing predicted CSI based on a CSI generation partial model.

[0281] Step S5302 : Process the first processed CSI based on the CSI recovery partial model to obtain the reference CSI, and process the second processed CSI based on the CSI recovery partial model to obtain the predicted CSI.

[0282] Step S5303: Determine the performance information of the AI ​​model based on the reference CSI and the predicted CSI.

[0283] For a detailed description of steps S5301 - S5303 , please refer to the embodiment shown in FIG3 .

[0284] The performance monitoring method of the artificial intelligence (AI) model involved in the embodiments of the present disclosure may include at least one of steps S5301 to S5303. For example, step S5301 can be implemented as an independent embodiment, step S5302 can be implemented as an independent embodiment, and so on, but is not limited thereto. Steps S5301+S5302 can be implemented as independent embodiments, and steps S5301+S5302+S5303 can be implemented as independent embodiments, but are not limited thereto.

[0285] In this embodiment, the network device receives the first processed CSI and the second processed CSI obtained through compression processing, which can effectively improve the timeliness of receiving the first processed CSI and the second processed CSI obtained through compression processing. The network device can use the first processed CSI and the second processed CSI to estimate the performance information of the AI ​​model, which can effectively monitor the performance of the AI ​​model used to predict CSI. This also supports reducing the computing power of AI model performance monitoring on the terminal side.

[0286] Optional example:

[0287] Assume that an AI model for time-domain CSI prediction is deployed on the terminal side. Alternatively, a bilateral AI / machine learning (ML) model is deployed to compress CSI in the space-frequency domain or angle-delay domain, or a bilateral AI / ML model is deployed to compress CSI in the time-space-frequency domain or Doppler-angle-delay domain.

[0288] In some embodiments, the terminal side may monitor the performance of the AI ​​model for time-domain CSI prediction:

[0289] Option 1: The gNB sends a downlink reference signal at L ≥ 1 time points within the prediction window. The UE obtains the CSI at this time point as H1 based on the reference signal. The AI ​​model on the terminal side infers the t-th time point among the T time points in the prediction window. l , l = 1, ..., the CSI at L moments is Compare it with H1, or calculate the KPI of the two, such as NMSE. Then, the terminal reports the calculated KPI to the network.

[0290] The L time instants for sending the downlink reference signal within the prediction window are defined as follows:

[0291] Option 1-1: The L time points for sending the downlink reference signal are the same as the T time points inferred within the prediction window.

[0292] Option 1-2: Based on the length of the prediction window w d It is determined to send a downlink reference signal at L time moments within the prediction window.

[0293] If L=1, the first moment of the predicted CSI in the prediction window is defined, or the first moment or A moment, or the last moment.

[0294] If L=2, define the first and second moments of the predicted CSI within the prediction window. or moment, or or the first moment and the last moment, or the first moment and the last moment.

[0295] If L>2, define the first predicted CSI in the prediction window. or moments, l=1,…,L.

[0296] The downlink reference signal sent above may be a reference signal resource with M≥1 period, or a reference signal resource with N≥1 non-periodic period, or a combination of M periodic reference signal resources and N reference signal resources.

[0297] Optionally, the reference signal resource may be a CSI-RS resource or a DMRS resource, etc.

[0298] Option 2: Predict the CSI at future moments using a non-AI algorithm and an AI-based algorithm respectively.

[0299] The non-AI algorithm is used as a reference. The terminal calculates KPIs, such as NMSE, based on the CSI obtained by the two algorithms. The terminal reports the calculated KPIs to the network. The performance of the current AI model is determined based on the comparison results.

[0300] Opiton3: The terminal uses a non-AI algorithm and an AI-based method to predict the CSI. Then, based on the predicted CSI, the corresponding PMI is calculated using one of the Type II codebook methods. Finally, the KPI of the two PMIs, such as SGCS, can be calculated, or KPI1 and KPI2 corresponding to the non-AI algorithm and AI-based method can be calculated respectively. The terminal reports the calculated KPI to the network.

[0301] Option 4: For the above Options 1 to 3, the terminal reports the calculated KPI to the network side for judging the performance of the model.

[0302] The terminal may report the calculated K ≥ 1 KPIs once or multiple times.

[0303] Alternatively, the network configures a KPI threshold for the terminal, and the terminal reports the result of judging the AI ​​model performance to the network based on the threshold. The network can configure I≥1 thresholds for the terminal, and I>1 thresholds corresponds to I different moments. Optionally,

[0304] The terminal reports K measurement results to the network, or the network configures a threshold for the K measurement results. The terminal can continue using the current model only if at least 70% of the comparison results from the K measurements indicate no degradation in performance. Otherwise, the terminal needs to update the model, switch to another model, or fall back to the existing non-AI algorithm for CSI prediction.

[0305] In some embodiments, the above threshold value and the value of K may be predefined and determined based on negotiation between the network and the terminal, or reported to the network by the terminal.

[0306] In some embodiments, the network side monitors the performance of the AI ​​model for AI-based time-domain CSI prediction on the terminal side based on the CSI reported by the terminal:

[0307] Option 1: The network side determines the AI ​​model performance on the terminal side based on the KPI reported by the terminal.

[0308] Option 2: The terminal compresses the CSI at L moments within the prediction window using non-AI algorithms and AI-based predictions, and then compresses the CSI at L moments using the CSI generation model in the bilateral AI / ML model that compresses the space-frequency domain or angle-delay domain CSI, and then reports it to the network side. The network side then uses the CSI recovery model to recover the predicted CSI at K moments. Optionally, the terminal uses the CSI generation model in the bilateral AI / ML model to compress the CSI at L moments using the time-space-frequency domain CSI or the Doppler-angle-delay domain CSI, and then reports it to the network side. The network side then uses the CSI recovery model to infer all the CSI predicted at L moments.

[0309] Optionally, the terminal compresses and reports the CSI estimated based on the downlink reference signal within the reception prediction window. Then, the network uses the estimated CSI reported by the terminal and the CSI predicted based on AI to compare the two using the above-mentioned terminal side implementation method.

[0310] Option 3: The terminal predicts CSI using both a non-AI algorithm and an AI-based method, and reports the predicted CSI to the network as a PMI. When K = 1, the terminal reports the PMI using the Type II codebook. When K > 1, the terminal reports the PMI using the Rel-18 Type II codebook. The network compares the PMIs reported by the two methods to determine the performance of the terminal-side AI model.

[0311] In some embodiments, Option 1:

[0312] Assume that the gNB configures a periodic CSI-RS resource for the terminal, with a period of M = 4 slots.

[0313] An observation window contains N = 4 moments of CSI-RS transmission. The UE predicts K = 4 future moments of spatial-frequency domain full channel information in a prediction window, and the adjacent moments of the predicted full channel information are separated by D = 4 slots. The length of the prediction window is w d= K·D = 16 slots. At the first time in the prediction window, the gNB also sends an aperiodic CSI-RS to the UE, or the gNB sends a periodic CSI-RS at the fifth time, as indicated by the bidirectional arrows in FIG6 , which is a schematic diagram of an application of the performance monitoring method of the AI ​​model in an embodiment of the present disclosure.

[0314] Assume that the channel bandwidth estimated by CSI-RS includes 52 PRBs. The UE estimates the full channel information H corresponding to the jth PRB at the i-th time based on the CSI-RS sent at four times within the observation window. i，j , i = 1, ..., 4, j = 1, ..., 52, then, the full channel information corresponding to the four moments in the prediction window is inferred by the AI ​​model deployed on the terminal and recorded as H i，j , i=5,…,8. H 5，j That is, H predicted at the first moment in the prediction window 5，j The UE then estimates the full channel information of different PRBs based on the CSI-RS sent by the gNB at the first moment in the prediction window: Then the NMSE is calculated by the following formula (1):

[0315] The terminal can report the NMSE value directly to the gNB.

[0316] Assume that the gNB configures the UE with an NMSE threshold of -10dB. If the calculated NMSE is greater than -10dB, this indicates that the performance of the current AI model has deteriorated. The UE reports an indication indicating the poor AI model, allowing the gNB to reconfigure parameters such as N and M, or instruct the UE to update the AI ​​model or switch to a new one.

[0317] The gNB may send two non-periodic CSI-RSs to the UE at two moments within the prediction window, corresponding to the first and last moments within the prediction window, respectively, as shown in FIG7 , which is a schematic diagram of an application of the performance monitoring method of the AI ​​model in an embodiment of the present disclosure.

[0318] Based on the above, the UE calculates the NMSE at the two times and reports it to the gNB, or reports it by averaging the two NMSEs. Because the CSI prediction accuracy varies at different times, the gNB can configure the UE with thresholds of -10dB and -5dB at the first and second times, respectively. The UE reports the comparison results of the two times to the gNB via an indication message.

[0319] In some embodiments, Option 2:

[0320] The parameters of the observation window and prediction window configured by the gNB for the UE are the same as those in Example 1. The UE estimates the full channel information corresponding to different moments based on the configured parameters, and then uses the AI ​​model inference and the autoregressive AR algorithm (non-AI algorithm) to predict the full channel information corresponding to different moments in the future. As in Example 1, the UE can calculate NMSE1 and NMSE2 corresponding to a certain moment using formula (1). NMSE1 and NMSE2 are calculated based on the full channel information recommended by the AI ​​model and predicted by the AR algorithm, respectively. The UE reports NMSE1 and NMSE2 to the gNB, or the UE reports the difference or ratio between NMSE1 and NMSE2 to the gNB, or the UE directly reports the comparison result of NMSE1 and NMSE2 to the gNB via indication information.

[0321] In some embodiments, the gNB may also configure a corresponding threshold value for the UE, and the UE reports the comparison result to the gNB through indication information based on the threshold value.

[0322] In some embodiments, Option 3:

[0323] In some embodiments, the UE can use AR and AI model reasoning to predict the full channel information at a future time. Assuming rank = 1 (i.e., the data transmission flow is 1), the gNB configures the UE with a Type II codebook and the configured codebook parameter N4 = 1. The UE can calculate the precoding vector v of rank = 1 at a certain time in the future based on the configured codebook parameter information and the calculated full channel information. 1，i and v 2，i , and then the value of SGCS can be calculated according to formula (2):

[0324] N3 is the number of precoding subbands configured by the gNB for the UE.

[0325] Optionally, the gNB also sends CSI-RS to the UE at the time of CSI prediction, and the UE calculates its eigenvector v based on the estimated channel information. i , and then K can be calculated by formula (2) 1，1 and K 1，2 .K 1，1 Indicates v obtained based on non-AI algorithm reasoning 1，i With v i The value of SGCS calculated using formula (2) is K 1，2 Represents v obtained based on AI model reasoning 2，i With v i The value of SGCS is calculated using formula (2).

[0326] UE calculates K1, or K 1，1 and K 1，2Report to gNB, or K 1，1 and K 1，2 Report the ratio or difference of K to gNB, or 1，1 and K 1，2 The comparison result is reported to the gNB through indication information.

[0327] The embodiments of the present disclosure further provide an apparatus for implementing any of the above methods. For example, an apparatus is provided, comprising units or modules for implementing each step performed by a terminal in any of the above methods. For another example, another apparatus is provided, comprising units or modules for implementing each step performed by a network device (e.g., an access network device, a core network function node, a core network device, etc.) in any of the above methods.

[0328] Figure 8a is a schematic diagram of the structure of the performance monitoring device for an artificial intelligence (AI) model proposed in an embodiment of the present disclosure. As shown in Figure 8a, the performance monitoring device for an artificial intelligence (AI) model includes: a processing module for determining reference channel state information (CSI) and predicted CSI, wherein the reference CSI is used as a reference for AI model performance monitoring, the predicted CSI is predicted by the AI ​​model and corresponds to a first prediction moment, and the reference CSI and the predicted CSI are used together to determine the performance information of the AI ​​model.

[0329] Optionally, the performance monitoring device of the artificial intelligence AI model also includes a transceiver module, which is used to execute the relevant steps performed by the terminal in any of the above methods, and will not be repeated here.

[0330] Optionally, the performance monitoring device of the artificial intelligence AI model also includes at least one of a sending module and a receiving module. The sending module is used to execute the steps related to sending performed by the terminal in any of the above methods, and the receiving module is used to execute the steps related to receiving performed by the terminal in any of the above methods, which will not be repeated here.

[0331] Figure 8b is a schematic diagram of the structure of a performance monitoring device for an artificial intelligence (AI) model proposed in an embodiment of the present disclosure. As shown in Figure 8b, the performance monitoring device for an artificial intelligence (AI) model includes: a processing module for determining performance information of the AI ​​model, wherein the performance information is determined by reference channel state information (CSI) and predicted CSI, the reference CSI being used as a reference for AI model performance monitoring, and the predicted CSI being predicted by the AI ​​model and corresponding to a first prediction moment.

[0332] Optionally, the performance monitoring device of the artificial intelligence AI model also includes a transceiver module, which is used to execute the relevant steps performed by the network device in any of the above methods, which will not be repeated here.

[0333] Optionally, the performance monitoring device of the artificial intelligence AI model also includes at least one of a sending module and a receiving module. The sending module is used to execute the steps related to sending performed by the network device in any of the above methods, and the receiving module is used to execute the steps related to receiving performed by the network device in any of the above methods, which will not be repeated here.

[0334] It should be understood that the division of the various units or modules in the above device is merely a division of logical functions. In actual implementation, they may be fully or partially integrated into a physical entity, or they may be physically separated. In addition, the units or modules in the device may be implemented in the form of a processor calling software: for example, the device includes a processor, the processor is connected to a memory, and the memory stores instructions. The processor calls the instructions stored in the memory to implement any of the above methods or implement the functions of the various units or modules of the above device, wherein the processor is, for example, a general-purpose processor, such as a central processing unit (CPU) or a microprocessor, and the memory is a memory within the device or a memory outside the device. Alternatively, the units or modules in the device can be implemented in the form of hardware circuits, and the functions of some or all of the units or modules can be realized by designing the hardware circuits. The above-mentioned hardware circuits can be understood as one or more processors; for example, in one implementation, the above-mentioned hardware circuit is an application-specific integrated circuit (ASIC), which realizes the functions of some or all of the above units or modules by designing the logical relationship of the components in the circuit; for example, in another implementation, the above-mentioned hardware circuit can be realized by a programmable logic device (PLD). Taking a field programmable gate array (FPGA) as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by configuring the configuration file, thereby realizing the functions of some or all of the above units or modules. All units or modules of the above devices can be realized in the form of software called by the processor, or in the form of hardware circuits, or in part by the form of software called by the processor, and the rest by hardware circuits.

[0335] Figure 9a is a schematic diagram of the structure of a communication device proposed in an embodiment of the present disclosure. As shown in Figure 9a, the communication device 9100 includes one or more processors 9101. The processor 9101 can be a general-purpose processor or a dedicated processor, for example, a baseband processor or a central processing unit. The baseband processor can be used to process communication protocols and communication data, and the central processing unit can be used to control communication devices (such as base stations, baseband chips, terminal devices, terminal device chips, DUs or CUs, etc.), execute programs, and process program data. The processor 9101 is used to call instructions to enable the communication device 9100 to execute any of the above methods.

[0336] In some embodiments, the communication device 9100 further includes one or more memories 9102 for storing instructions. Optionally, all or part of the memories 9102 may be located outside the communication device 9100.

[0337] In some embodiments, the communication device 9100 further includes one or more transceivers 9103. When the communication device 9100 includes one or more transceivers 9103, the communication steps such as sending and receiving in the above method are performed by the transceiver 9103, and the other steps are performed by the processor 9101.

[0338] In some embodiments, a transceiver may include a receiver and a transmitter, which may be separate or integrated. Optionally, the terms transceiver, transceiver unit, transceiver, and transceiver circuit may be used interchangeably; the terms transmitter, transmitting unit, transmitter, and transmitting circuit may be used interchangeably; and the terms receiver, receiving unit, receiver, and receiving circuit may be used interchangeably.

[0339] Optionally, the communication device 9100 further includes one or more interface circuits 9104, which are connected to the memory 9102. The interface circuits 9104 can be used to receive signals from the memory 9102 or other devices, and can be used to send signals to the memory 9102 or other devices. For example, the interface circuits 9104 can read instructions stored in the memory 9102 and send the instructions to the processor 9101.

[0340] The communication device 9100 described in the above embodiments may be a network device or a terminal, but the scope of the communication device 9100 described in the present disclosure is not limited thereto, and the structure of the communication device 9100 may not be limited by FIG. 9a. The communication device may be an independent device or may be part of a larger device. For example, the communication device may be: 1) an independent integrated circuit IC, or a chip, or a chip system or subsystem; (2) a collection of one or more ICs, optionally, the above IC collection may also include a storage component for storing data or programs; (3) an ASIC, such as a modem; (4) a module that can be embedded in other devices; (5) a receiver, a terminal device, an intelligent terminal device, a cellular phone, a wireless device, a handheld device, a mobile unit, an in-vehicle device, a network device, a cloud device, an artificial intelligence device, etc.; (6) others, etc.

[0341] FIG9 b is a schematic diagram of the structure of a chip according to an embodiment of the present disclosure. If the communication device 9100 can be a chip or a chip system, reference can be made to the schematic diagram of the structure of the chip 9200 shown in FIG9 b , but the present disclosure is not limited thereto.

[0342] The chip 9200 includes one or more processors 9201, and the processor 9201 is used to call instructions so that the chip 9200 executes any of the above methods.

[0343] In some embodiments, chip 9200 further includes one or more interface circuits 9202, which are connected to memory 9203. Interface circuits 9202 can be used to receive signals from memory 9203 or other devices, and can be used to send signals to memory 9203 or other devices. For example, interface circuit 9202 can read instructions stored in memory 9203 and send the instructions to processor 9201. Optionally, the terms interface circuit, interface, transceiver pin, and transceiver are interchangeable.

[0344] In some embodiments, the chip 9200 further includes one or more memories 9203 for storing instructions. Alternatively, all or part of the memories 9203 may be located outside the chip 9200.

[0345] The present disclosure also proposes a storage medium having instructions stored thereon, which, when executed on the communication device 9100, causes the communication device 9100 to execute any of the above methods. Optionally, the storage medium is an electronic storage medium. Optionally, the storage medium is a computer-readable storage medium, but is not limited thereto and may also be a storage medium readable by other devices. Optionally, the storage medium may be a non-transitory storage medium, but is not limited thereto and may also be a temporary storage medium.

[0346] In the embodiment of the present disclosure, the processor is a circuit with signal processing capability. In one implementation, the processor can be a circuit with instruction reading and execution capability, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), or a digital signal processor (DSP); in another implementation, the processor can implement certain functions through the logical relationship of the hardware circuit, and the logical relationship of the above hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by a processor as an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and implementing the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units or modules. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc.

[0347] The present disclosure also provides a program product, which, when executed by the communication device 9100, enables the communication device 9100 to perform any of the above methods. Optionally, the program product is a computer program product.

[0348] The present disclosure also proposes a computer program, which, when executed on a computer, causes the computer to perform any one of the above methods.

[0349] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer programs. When the computer program is loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present disclosure are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. Available media may be magnetic media (eg, floppy disks, hard disks, tapes), optical media (eg, high-density digital video discs (DVDs)), or semiconductor media (eg, solid state disks (SSDs)).

[0350] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0351] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0352] The above are only specific embodiments of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A performance monitoring method for an artificial intelligence (AI) model, characterized in that: The method comprises: Determine reference channel state information CSI and predicted CSI, wherein the reference CSI is used for reference in AI model performance monitoring, the predicted CSI is predicted by the AI ​​model and corresponds to a first prediction moment, and the reference CSI and the predicted CSI are used together to determine performance information of the AI ​​model.

2. The method according to claim 1, characterized in that The determining of the reference CSI includes at least one of the following: receiving a downlink reference signal sent by a network device at a second predicted time, and determining the reference CSI according to the downlink reference signal, wherein the first predicted time and the second predicted time are the same or different; Based on a non-AI model approach, a reference CSI corresponding to the first prediction moment is determined, wherein the non-AI model approach is a approach that does not rely on the AI ​​model to predict the CSI.

3. The method according to claim 2, characterized in that The second prediction time is within the prediction window, wherein, When the number of the second prediction moments is one, the second prediction moments include: the first moment in the prediction window, or the second moment in the prediction window. moment, or a moment, or the last moment; In the case where the number of the second prediction moments is two, the second prediction moments include: the first moment in the prediction window and the second moment in the prediction window. moment, or the first moment and the moment, or the the first moment and the last moment, or the first a moment and the last moment, or the first moment and the last moment; In the case where the number of the second prediction moments is greater than two, the second prediction moments include: moment, or moments, l=1, ..., L, L represents the number of the second predicted moments; Among them, the w d represents the length of the prediction window, Indicates rounding up. Indicates rounding down.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Determining a monitoring value representing performance of the AI ​​model according to the reference CSI and the predicted CSI; Sending first indication information, wherein the first indication information includes: the monitoring value and / or information associated with the monitoring value.

5. The method according to claim 4, characterized in that The determining, according to the reference CSI and the predicted CSI, a monitoring value representing the performance of the AI ​​model includes at least one of the following: Calculating a first key performance indicator KPI between the reference CSI and the predicted CSI, and using the first KPI as the monitoring value, wherein the reference CSI is estimated based on a downlink reference signal; Calculating a second KPI between the reference CSI and the predicted CSI, and using the second KPI as the monitoring value, wherein the reference CSI is predicted based on a non-AI model; Determine a first precoding matrix indicator PMI corresponding to the reference CSI, determine a second PMI corresponding to the predicted CSI, calculate a third KPI between the first PMI and the second PMI, and use the third KPI as the monitoring value, wherein the reference CSI is predicted based on a non-AI model; A feature vector is calculated based on the reference CSI, a fourth KPI between the feature vector and the second PMI is calculated, and the fourth KPI is used as the monitoring value, wherein the reference CSI is estimated based on a downlink reference signal.

6. The method according to claim 5, characterized in that The number of the reference CSI is multiple, the number of the predicted CSI is multiple, different reference CSIs correspond to different first prediction moments or second prediction moments, and different predicted CSIs correspond to different first prediction moments. Accordingly, the number of the first KPIs is multiple, and / or the number of the second KPIs is multiple, and / or the number of the third KPIs is multiple, and the number of the fourth KPIs is multiple.

7. The method according to any one of claims 4 to 6, characterized in that: The sending of the first indication information includes any one of the following: Sending a first indication message, wherein the first indication message includes: a preset number of the monitoring values, and / or information associated with each of the monitoring values; A preset number of first indication information are sent respectively, wherein each of the first indication information includes: one of the monitoring values, and / or information associated with one of the monitoring values.

8. The method according to claim 7, characterized in that The preset number is determined based on at least one of the following methods: receiving second indication information, wherein the second indication information includes the preset number; Determine the preset quantity based on the agreement; Report the preset quantity to the network device.

9. The method according to any one of claims 4 to 8, characterized in that: The information associated with the monitoring value includes: a comparison result between the monitoring value and a corresponding monitoring threshold value.

10. The method according to claim 9, characterized in that The monitoring threshold is determined based on at least one of the following methods: receiving second indication information, wherein the second indication information includes the monitoring threshold value; Determining the monitoring threshold value based on the protocol agreement; Report the monitoring threshold to the network device.

11. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Processing the reference CSI based on the CSI generation part model to obtain a first processed CSI, and processing the predicted CSI based on the CSI generation part model to obtain a second processed CSI, wherein the first processed CSI and the second processed CSI are used together to determine the performance information of the AI ​​model; Send third indication information, where the third indication information includes: the first processing CSI and the second processing CSI.

12. The method according to claim 11, characterized in that The number of the reference CSIs is multiple, and different reference CSIs correspond to different first prediction moments or second prediction moments; wherein the processing of the reference CSI based on the CSI generation partial model to obtain a first processed CSI includes: Processing the plurality of reference CSIs respectively based on the CSI generation partial model to obtain a plurality of first processed CSIs; or, A plurality of the reference CSIs are jointly processed based on the CSI generation partial model to obtain the first processed CSI.

13. The method according to claim 11, characterized in that The number of the predicted CSIs is multiple, and different predicted CSIs correspond to different first prediction moments; wherein the processing of the predicted CSI based on the CSI generation partial model to obtain a second processed CSI includes: processing the plurality of predicted CSIs respectively based on the CSI generation partial model to obtain a plurality of the second processed CSIs; or, The plurality of predicted CSIs are jointly processed based on the CSI generation partial model to obtain the second processed CSI.

14. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Determine a first PMI corresponding to the reference CSI, and determine a second PMI corresponding to the predicted CSI, wherein the first PMI and the second PMI are used together to determine performance information of the AI ​​model; Send third indication information, where the third indication information includes: the first PMI and the second PMI.

15. A performance monitoring method for an artificial intelligence (AI) model, characterized in that: The method comprises: Determine performance information of the AI ​​model, wherein the performance information is determined by reference channel state information CSI and predicted CSI, the reference CSI is used for reference of the AI ​​model performance monitoring, and the predicted CSI is predicted by the AI ​​model and corresponds to a first prediction moment.

16. The method according to claim 15, characterized in that The reference CSI is determined based on at least one of the following methods: Sending a downlink reference signal to the terminal at a second predicted time, wherein the downlink reference signal is used to determine the reference CSI, wherein the first predicted time and the second predicted time are the same or different; The reference CSI is determined based on a non-AI model method and corresponds to the first prediction moment, wherein the non-AI model method is a method of predicting CSI without relying on the AI ​​model.

17. The method according to claim 16, characterized in that The second prediction time is within the prediction window, wherein, When the number of the second prediction moments is one, the second prediction moments include: the first moment in the prediction window, or the second moment in the prediction window. moment, or a moment, or the last moment; In the case where the number of the second prediction moments is two, the second prediction moments include: the first moment in the prediction window and the second moment in the prediction window. moment, or the first moment and the moment, or the the first moment and the last moment, or the first a moment and the last moment, or the first moment and the last moment; In the case where the number of the second prediction moments is greater than two, the second prediction moments include: moment, or moments, l=1, ..., L, L represents the number of the second predicted moments; Among them, the w d represents the length of the prediction window, Indicates rounding up. Indicates rounding down.

18. The method according to any one of claims 15 to 17, characterized in that: The determining the performance information of the AI ​​model includes: receiving first indication information, wherein the first indication information includes: a monitoring value indicating performance of the AI ​​model and / or information associated with the monitoring value, wherein the monitoring value is determined by the reference CSI and the predicted CSI; The performance information is determined according to the monitoring value and / or information associated with the monitoring value.

19. The method according to claim 18, characterized in that The monitoring value includes at least one of the following: A first KPI between the reference CSI and the predicted CSI, wherein the reference CSI is estimated based on a downlink reference signal; A second KPI between the reference CSI and the predicted CSI, wherein the reference CSI is predicted based on a non-AI model approach; a third KPI between a first precoding matrix indication PMI corresponding to the reference CSI and a second PMI corresponding to the predicted CSI, wherein the reference CSI is predicted based on a non-AI model approach; A fourth KPI between an eigenvector calculated based on the reference CSI and the second PMI, wherein the reference CSI is estimated based on a downlink reference signal.

20. The method of claim 19, wherein: The number of the reference CSI is multiple, the number of the predicted CSI is multiple, different reference CSIs correspond to different first prediction moments or second prediction moments, and different predicted CSIs correspond to different first prediction moments. Accordingly, the number of the first KPIs is multiple, and / or the number of the second KPIs is multiple, and / or the number of the third KPIs is multiple, and the number of the fourth KPIs is multiple.

21. The method according to any one of claims 18 to 20, characterized in that: The receiving first indication information comprises: Receive a first indication information, wherein the first indication information includes: a preset number of the monitoring values, and / or information associated with each of the monitoring values; A preset number of first indication information are received respectively, wherein each of the first indication information includes: one of the monitoring values, and / or information associated with one of the monitoring values.

22. The method according to claim 21, characterized in that The preset number is determined based on at least one of the following methods: Sending second indication information, wherein the second indication information includes the preset number; Determine the preset quantity based on the agreement; The preset number reported by the receiving terminal.

23. The method according to any one of claims 18 to 22, characterized in that The information associated with the monitoring value includes: a comparison result between the monitoring value and a corresponding monitoring threshold value.

24. The method of claim 23, wherein: The monitoring threshold is determined based on at least one of the following methods: Sending second indication information, wherein the second indication information includes the monitoring threshold value; Determining the monitoring threshold value based on the protocol agreement; The monitoring threshold reported by the receiving terminal.

25. The method according to any one of claims 15 to 17, characterized in that The determining the performance information of the AI ​​model includes: receiving third indication information, wherein the third indication information includes: a first processed CSI and a second processed CSI, wherein the first processed CSI is obtained by processing the reference CSI based on a CSI generation partial model, and the second processed CSI is obtained by processing the predicted CSI based on the CSI generation partial model; Processing the first processed CSI based on a CSI recovery partial model to obtain the reference CSI, and processing the second processed CSI based on the CSI recovery partial model to obtain the predicted CSI; Determine performance information of the AI ​​model based on the reference CSI and the predicted CSI.

26. The method of claim 25, wherein: The number of the first processed CSI is one or more; and the processing of the first processed CSI based on the CSI recovery partial model to obtain the reference CSI includes: Processing the first processed CSI based on the CSI recovery partial model to obtain a plurality of reference CSIs; The plurality of first processed CSIs are processed respectively based on the CSI recovery partial model to obtain the plurality of reference CSIs.

27. The method of claim 25, wherein: The number of the second processed CSI is one or more; and processing the second processed CSI based on the CSI recovery partial model to obtain the predicted CSI includes: Processing a second processed CSI based on the CSI recovery partial model to obtain a plurality of predicted CSIs; The plurality of second processed CSIs are processed respectively based on the CSI recovery partial model to obtain a plurality of predicted CSIs.

28. The method according to any one of claims 15 to 17, characterized in that: The determining the performance information of the AI ​​model includes: receiving third indication information, wherein the third indication information includes: a first PMI corresponding to the reference CSI and a second PMI corresponding to the predicted CSI; Determine performance information of the AI ​​model based on the first PMI and the second PMI.

29. A performance monitoring device for an artificial intelligence (AI) model, characterized in that: The device comprises: A processing module is used to determine reference channel state information CSI and predicted CSI, wherein the reference CSI is used for reference in AI model performance monitoring, the predicted CSI is predicted by the AI ​​model and corresponds to a first prediction moment, and the reference CSI and the predicted CSI are used together to determine performance information of the AI ​​model.

30. A performance monitoring device for an artificial intelligence (AI) model, characterized in that: The device comprises: A processing module is used to determine performance information of the AI ​​model, wherein the performance information is determined by reference channel state information CSI and predicted CSI, the reference CSI is used as a reference for performance monitoring of the AI ​​model, and the predicted CSI is predicted by the AI ​​model and corresponds to a first prediction moment.

31. A communication device, characterized in that: include: One or more processors; The processor is used to call instructions so that the communication device executes the method according to any one of claims 1-28.

32. A communication system, characterized in that: The invention comprises a network device and a terminal, wherein the terminal is configured to implement the method according to any one of claims 1 to 14, and the terminal is configured to implement the method according to any one of claims 15 to 28.

33. A storage medium storing instructions, characterized in that: When the instruction is executed on a communication device, the communication device is caused to execute the method according to any one of claims 1 to 28.

Images