Model performance monitoring method and apparatus

Abstract

The embodiments of the present disclosure can be applied to the technical field of communications. Disclosed are a model performance monitoring method and apparatus. The method executed by a terminal device comprises: monitoring a model, and acquiring channel status information (CSI) currently associated with the model, wherein the CSI comprises any one of the following: first information, the current first input of the model and the first information of the model, the first information is the current first output of the model or associated information of the first output, and the model is a CSI feedback model; according to the CSI currently associated with the model, determining current performance information of the model; and sending first indication information, wherein the first indication information includes the current performance information of the model. Thus, the terminal device can accurately determine the current performance information of the model by means of the CSI, thereby improving the accuracy and reliability of model performance monitoring.

Description

Model performance monitoring method and device Technical Field

[0001] The present disclosure relates to the field of communication technology, and in particular to a method and device for monitoring model performance. Background Art

[0002] In related technologies, terminal devices can provide Channel Status Information (CSI) feedback to network devices through a CSI feedback model. Specifically, the terminal device compresses the CSI using the CSI generation model, quantizes it into a binary bit stream, and sends it to the network device. The network device then recovers the CSI using the CSI recovery model. However, poor performance of the CSI feedback model may result in inaccurate signal status information recovered by the network device.

[0003] Summary of the Invention

[0004] The disclosed embodiments provide a method and apparatus for monitoring model performance. In these embodiments, a terminal device monitors a model to obtain the channel state information (CSI) currently associated with the model, then determines the model's current performance information based on the CSI currently associated with the model, and finally sends first indication information containing the model's current performance information to a network device. Thus, the terminal device can accurately determine the model's current performance information through the CSI, improving the accuracy and reliability of model performance monitoring.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for monitoring model performance, which is executed by a terminal device, and the method includes: monitoring the model, obtaining channel state information CSI currently associated with the model, wherein the CSI includes any one of the following: first information, the current first input of the model and the first information; wherein the first information is the current first output of the model or associated information of the first output, and the model is a CSI feedback model; determining the current performance information of the model based on the CSI currently associated with the model; and sending first indication information, wherein the first indication information includes the current performance information of the model.

[0006] In a second aspect, an embodiment of the present disclosure also provides another method for monitoring model performance, which is executed by a network device and includes: receiving first indication information, wherein the first indication information includes current performance information of the channel state information CSI feedback model.

[0007] In a third aspect, an embodiment of the present disclosure provides a terminal device, including:

[0008] a processing module, configured to monitor the model and obtain channel state information (CSI) currently associated with the model, wherein the CSI includes any one of the following: first information, a current first input of the model, and the first information; wherein the first information is a current first output of the model or associated information of the first output, and the model is a CSI feedback model;

[0009] The processing module is further configured to determine current performance information of the model based on the CSI currently associated with the model;

[0010] The transceiver module is further used to send first indication information, wherein the first indication information includes current performance information of the model.

[0011] In a fourth aspect, the embodiments of the present disclosure further provide another network device, including:

[0012] The transceiver module is configured to receive first indication information, wherein the first indication information includes current performance information of a channel state information (CSI) feedback model.

[0013] In a fifth aspect, an embodiment of the present disclosure provides a communication device, which includes a processor. When the processor calls a computer program in a memory, the method described in the first aspect is executed.

[0014] In a sixth aspect, an embodiment of the present disclosure provides a communication device, which includes a processor. When the processor calls a computer program in a memory, the method described in the second aspect is executed.

[0015] In the seventh aspect, an embodiment of the present disclosure provides a communication device, which includes a processor and a memory, in which a computer program is stored; the processor executes the computer program stored in the memory so that the communication device executes the method described in the first aspect above.

[0016] In an eighth aspect, an embodiment of the present disclosure provides a communication device, which includes a processor and a memory, in which a computer program is stored; the processor executes the computer program stored in the memory so that the communication device executes the method described in the second aspect above.

[0017] In a ninth aspect, an embodiment of the present disclosure provides a communication device, which includes a processor and an interface circuit, wherein the interface circuit is used to receive code instructions and transmit them to the processor, and the processor is used to run the code instructions to enable the device to execute the method described in the first aspect above.

[0018] In the tenth aspect, an embodiment of the present disclosure provides a communication device, which includes a processor and an interface circuit, wherein the interface circuit is used to receive code instructions and transmit them to the processor, and the processor is used to run the code instructions to enable the device to execute the method described in the second aspect above.

[0019] In the eleventh aspect, an embodiment of the present disclosure provides a communication system, which includes the terminal device described in the third aspect, the network device described in the fourth aspect, or the system includes the communication device described in the fifth to sixth aspects, or the system includes the communication device described in the seventh to eighth aspects, or the system includes the communication device described in the ninth to tenth aspects.

[0020] In a twelfth aspect, an embodiment of the present invention provides a computer-readable storage medium for storing instructions for the above-mentioned terminal device, and when the instructions are executed, the terminal device executes the method described in the first aspect.

[0021] In a thirteenth aspect, an embodiment of the present invention provides a computer-readable storage medium for storing instructions for the above-mentioned network device, and when the instructions are executed, the network device executes the method described in the second aspect.

[0022] In a fourteenth aspect, the present disclosure further provides a computer program product comprising a computer program, which, when executed on a computer, enables the computer to execute the method described in the first aspect above.

[0023] In a fifteenth aspect, the present disclosure further provides a computer program product comprising a computer program, which, when executed on a computer, enables the computer to execute the method described in the second aspect above.

[0024] In a sixteenth aspect, the present disclosure provides a chip system comprising at least one processor and an interface for supporting a terminal device in implementing the functions described in the first aspect, such as determining or processing at least one of the data and information described in the aforementioned method. In one possible design, the chip system further comprises a memory for storing computer programs and data necessary for the terminal device. The chip system may consist of a chip alone or may include a chip and other discrete components.

[0025] In a seventeenth aspect, the present disclosure provides a chip system comprising at least one processor and an interface for supporting a network device in implementing the functions described in the second aspect, such as determining or processing at least one of the data and information described in the aforementioned method. In one possible design, the chip system further comprises a memory for storing computer programs and data necessary for the network device. The chip system may consist of a single chip or may include a chip and other discrete components.

[0026] In an eighteenth aspect, the present disclosure provides a computer program, which, when executed on a computer, enables the computer to execute the method described in the first aspect above.

[0027] In a nineteenth aspect, the present disclosure provides a computer program, which, when executed on a computer, enables the computer to execute the method described in the second aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the background technology, the drawings required for use in the embodiments of the present disclosure or the background technology will be described below.

[0029] FIG1 is a schematic diagram of the architecture of a communication system provided by the present disclosure;

[0030] FIG2 is a flow chart of a method for monitoring model performance provided by an embodiment of the present disclosure;

[0031] FIG3 is a flow chart of another method for monitoring model performance provided by an embodiment of the present disclosure;

[0032] FIG4 is a flow chart of another method for monitoring model performance provided by an embodiment of the present disclosure;

[0033] FIG5 is a flow chart of another method for monitoring model performance provided by an embodiment of the present disclosure;

[0034] FIG5A is a CSI measurement and reporting timing diagram provided by an embodiment of the present disclosure;

[0035] FIG6 is a flowchart of another method for monitoring model performance provided by an embodiment of the present disclosure;

[0036] FIG7 is a flow chart of a method for monitoring model performance provided by an embodiment of the present disclosure;

[0037] FIG8 is a flowchart of another method for monitoring model performance provided by an embodiment of the present disclosure;

[0038] FIG9 is a flowchart of another method for monitoring model performance provided by an embodiment of the present disclosure;

[0039] FIG10 is a flowchart of another method for monitoring model performance provided by an embodiment of the present disclosure;

[0040] FIG11 is a signaling interaction diagram of a method for monitoring model performance according to an embodiment of the present disclosure;

[0041] FIG12 is a structural diagram of a communication device provided by an embodiment of the present disclosure;

[0042] FIG13 is a structural diagram of another communication device provided in an embodiment of the present disclosure;

[0043] FIG14 is a schematic structural diagram of a chip provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0044] To facilitate understanding of the technical solutions of the present disclosure, some terms involved in the embodiments of the present disclosure are briefly introduced below.

[0045] 1. Reference signal (RS)

[0046] A reference signal, also known as a pilot signal, is provided by the transmitter to the receiver for channel estimation or sounding. It can be used for coherent detection and demodulation in terminal devices, beamforming, coherent detection and monitoring of network equipment, and channel quality measurement.

[0047] 2. Channel State Information Reference Signal (CSI-RS)

[0048] The primary purpose of CSI-RS is to measure downlink signal information. It is a known signal provided by the transmitter to the receiver for channel estimation or channel sounding. It can be used for channel state information measurement, beam management, time-frequency tracking, and mobility management on terminal devices.

[0049] 3. CSI Feedback Model

[0050] The CSI feedback model, also known as the CSI bilateral model, consists of a CSI generation model and a CSI recovery model. The CSI generation model compresses the channel state information (CSI), while the CSI recovery model recovers the compressed CSI. Both models are trained using collected datasets.

[0051] The CSI generation model can be defined as an encoder, and the CSI recovery model can be defined as a decoder. An encoder that can output different CSI compression information lengths or different CSI feedback payloads is called a scalable encoder.

[0052] 4. Radio Resource Control (RRC)

[0053] Radio resource control (RRC), also known as radio resource management or radio resource allocation, refers to the management, control, and scheduling of radio resources through specific strategies and means. It aims to make the best possible use of limited wireless network resources while meeting quality of service requirements, ensuring that the planned coverage area is reached and maximizing service capacity and resource utilization.

[0054] 5. Medium access control (MAC) control element (CE)

[0055] MAC CE is a way to exchange control information between terminal devices and the network, in addition to radio resource control (RRC) messages and non-access stratum (NAS) messages. It exchanges control information about the MAC layer.

[0056] 6. Downlink Control Information (DCI)

[0057] DCI is control information related to the physical uplink and downlink shared channel and transmitted on the physical downlink control channel (PDCCH). The DCI information includes several related contents such as resource block (RB) allocation information and modulation mode.

[0058] In order to better understand the model performance monitoring method and device disclosed in the embodiment of the present disclosure, the communication system to which the embodiment of the present disclosure is applicable is first described below.

[0059] Please refer to Figure 1, which is a schematic diagram of the architecture of a communication system provided in an embodiment of the present disclosure. The communication system may include, but is not limited to, one network device and one terminal device. The number and form of devices shown in Figure 1 are for example purposes only and do not constitute a limitation on the embodiments of the present disclosure. In actual applications, two or more network devices and two or more terminal devices may be included. The communication system shown in Figure 1 includes a network device 101 and a terminal device 102.

[0060] It should be noted that the technical solutions of the embodiments of the present disclosure can be applied to various communication systems, such as long-term evolution (LTE) systems, fifth-generation (5G) mobile communication systems, 5G new radio (NR) systems, or other future new mobile communication systems.

[0061] The network device 101 in the embodiment of the present disclosure is an entity on the network side for transmitting or receiving signals and a device for communicating with a terminal device. The network device 101 can be a base transceiver station (BTS) in a global system for mobile communications (GSM) system or a code division multiple access (CDMA) system, a base station (nodeB, NB) in a wideband code division multiple access (WCDMA) system, an evolved nodeB (eNB or eNodeB) in an LTE system, a wireless controller in a cloud radio access network (CRAN) scenario, a radio network controller (RNC), a base station controller (BSC), a home base station (for example, a home evolved nodeB or home nodeB, HNB), a baseband unit (BBU), or a baseband controller. The network device may be a baseband unit (BBU), or the network device may be a relay station, an access point, an in-vehicle device, a wearable device, a network device in a 5G network, or a network device in a future evolved PLMN network, etc. It may be an access point (AP) in a WLAN, a wireless relay node, a wireless backhaul node, a transmission point (TP), or a transmission and reception point (TRP), etc. It may be a gNB or a transmission point (TRP or TP) in a new radio (NR) system, or one or a group of antenna panels (including multiple antenna panels) of a base station in a 5G system, or it may also be a network node constituting a gNB or a transmission point, such as a baseband unit (BBU) or a distributed unit (DU), etc. The embodiments of the present disclosure do not limit the specific technology and specific device form adopted by the network device.The network device provided by the embodiment of the present disclosure may be composed of a centralized unit (CU) and a distributed unit (DU), wherein the CU may also be called a control unit. The CU-DU structure may be used to split the network device, such as the protocol layer of the network device, and the functions of some protocol layers may be centrally controlled by the CU, while the functions of the remaining part or all of the protocol layers may be distributed in the DU, and the DU may be centrally controlled by the CU.

[0062] The terminal device 102 in the embodiment of the present disclosure may be a device that provides voice / data connectivity to a user, for example, a handheld device or a vehicle-mounted device with a wireless connection function. Some examples of terminal devices include: mobile phones, tablet computers, laptop computers, PDAs, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, cellular phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to wireless modems, vehicle-mounted devices, wearable devices, terminal devices in 5G networks or future evolved public land mobile communication networks (PLMNs). The present invention relates to a terminal device in a wireless communication system (PLMN) and / or any other suitable device for communicating on a wireless communication system, and the embodiments of the present disclosure are not limited thereto.

[0063] Wearable devices, also known as wearable smart devices, are a general term for wearable devices that use wearable technology to intelligently design and develop wearable devices for daily wear, such as glasses, bracelets, watches, clothing, and shoes. Wearable devices are portable devices that are worn directly on the body or integrated into the user's clothing or accessories. Wearable devices are not just hardware devices, but also achieve powerful functions through software support, data interaction, and cloud interaction. In a broad sense, wearable smart devices include those that are fully functional, large in size, and can achieve full or partial functions without relying on smartphones, such as smart watches or smart glasses, as well as those that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various smart bracelets and smart jewelry for vital sign monitoring.

[0064] In addition, in the embodiment of the present disclosure, the terminal device 102 can also be a terminal device in an Internet of Things system. IoT (Internet of Things) is an important part of the future development of information technology. Its main technical feature is to connect objects to the network through communication technology, thereby realizing an intelligent network of human-machine interconnection and object-to-object interconnection.

[0065] In addition, in the embodiment of the present disclosure, the terminal device 102 may also include sensors such as smart printers, train detectors, and gas stations. Its main functions include collecting data (part of the terminal device), receiving control information and downlink data from the network device, and sending electromagnetic waves to transmit uplink data to the network device.

[0066] It should be noted that the technical solutions of the embodiments of the present disclosure can be applied to various communication systems, such as long-term evolution (LTE) systems, fifth-generation (5G) mobile communication systems, 5G new radio (NR) systems, or other future new mobile communication systems.

[0067] Among them, the terminal device provided by the embodiment of the present disclosure can execute the method shown in the embodiments of Figures 2 to 6, and the network device provided by the embodiment of the present disclosure can execute the method shown in the embodiments of Figures 7 to 10.

[0068] It can be understood that the communication system described in the embodiment of the present disclosure is for the purpose of more clearly illustrating the technical solution of the embodiment of the present disclosure, and does not constitute a limitation on the technical solution provided by the embodiment of the present disclosure. Ordinary technicians in this field can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solution provided by the embodiment of the present disclosure is also applicable to similar technical problems.

[0069] In addition, in order to facilitate understanding of the embodiments of the present disclosure, the following points are explained.

[0070] First, the first, second, and various numbers in this disclosure are only used for the convenience of description and are not intended to limit the scope of the embodiments of this disclosure, for example, to distinguish different information, different terminal devices, etc.

[0071] Second, the "protocol" involved in the embodiments of the present disclosure may refer to a standard protocol in the communication field, for example, it may include an LTE protocol, a NR protocol, and related protocols used in future communication systems, and the present disclosure does not limit this.

[0072] Third, the embodiments of the present disclosure list multiple implementation methods to clearly illustrate the technical solutions of the embodiments of the present disclosure. Of course, those skilled in the art will understand that the multiple embodiments provided in the embodiments of the present disclosure can be implemented individually, combined with the methods of other embodiments of the embodiments of the present disclosure, or implemented individually or in combination with some methods in other related technologies; the embodiments of the present disclosure are not limited to this.

[0073] The following is a detailed introduction to a model performance monitoring method and device provided by the present disclosure with reference to the accompanying drawings.

[0074] Please refer to Figure 2, which is a flow chart of a method for monitoring model performance provided by an embodiment of the present disclosure. As shown in Figure 2, the method is applied to a terminal device. The method may include but is not limited to the following steps:

[0075] Step 201: Monitor the model and obtain the channel state information CSI currently associated with the model.

[0076] The model is a CSI feedback model.

[0077] The CSI may include any one of the following: first information, the current first input of the model and the first information.

[0078] The first information is the current first output of the model or related information of the first output.

[0079] The first input may be an input of the CSI generation model, and the first output may be an output of the CSI recovery model.

[0080] In some possible implementations, the first input may be determined based on a received second CSI-RS. The second CSI-RS may be a signal sent by a network device and used for channel measurement and channel estimation.

[0081] Specifically, the terminal device can estimate the downlink channel information H of different frequency domain units based on the received second CSI-RS, and perform singular value decomposition on the downlink channel information H of the i-th frequency domain unit to obtain the eigenvector v i, the feature vector v i as the first input.

[0082] In some possible implementations, when the terminal device side includes a CSI recovery partial model, the output of the CSI recovery partial model is determined as the first output, wherein the CSI recovery partial model on the terminal device side is the same as or different from the CSI recovery partial model on the network device side.

[0083] Specifically, the CSI compressed by the CSI generation model is quantized and sent to the CSI recovery model on the terminal device side, so that the CSI recovery model on the terminal device side outputs vector information of different frequency domain units as e i , which is the first output.

[0084] If the CSI recovery model on the terminal device side is different from the CSI recovery model on the network device side, the CSI recovery model on the terminal device side can be defined as a proxy model or reference model. In some possible implementations, the CSI recovery model on the terminal device side can be generated based on the CSI recovery model on the network device side.

[0085] Alternatively, when the terminal device side does not include a CSI recovery partial model, the first output sent by the receiving network device is received.

[0086] In this disclosure, if the terminal device side does not include a CSI recovery model, it is necessary to obtain the output of the CSI recovery model on the network device side. Specifically, the terminal device quantizes the CSI compressed by the CSI generation model and reports it to the network device. The network device inputs it into the CSI recovery model, and the CSI recovery model recovers the characteristic vector e of each frequency domain unit. i , i.e. the first output. Then, the network device can i Perform scalar quantization or send it to the terminal device in the form of codebook parameters.

[0087] Alternatively, when the terminal device side does not include a CSI recovery partial model, the associated information is determined based on the received first channel state information reference signal CSI-RS, wherein the first CSI-RS is the CSI-RS after beamforming corresponding to the first output.

[0088] In the present disclosure, the network device recovers the characteristic vector e of each frequency domain unit through the CSI recovery model. i Afterwards, the eigenvector e can be iPerform beamforming and send the beamformed first CSI-RS to the terminal device, and the terminal device estimates the association information of the first output based on the received beamformed first CSI-RS. The beam used is the first output, that is, the eigenvector e i .

[0089] In some possible implementations, when the terminal device does not include the CSI recovery partial model, a first information acquisition request is sent to the network device. That is, the terminal device sends the first information acquisition request to the network device to request the network device to send the first output of the CSI recovery partial model or the first CSI-RS to the terminal device.

[0090] In some possible implementations, when the terminal device side does not include a CSI recovery partial model, the network device may also actively send the first output of the CSI recovery partial model or the first CSI-RS to the terminal device.

[0091] Step 202: Determine the current performance information of the model based on the CSI currently associated with the model.

[0092] In some possible implementations, the current performance information of the model may include one or more of the intermediate KPI, the final KPI, the distribution offset between the first input and first output of the model and the input data and output data of the training data, and the performance results determined based on the intermediate KPI, the final KPI, and the distribution offset.

[0093] The intermediate KPI may be the square of the cosine similarity, the normalized minimum mean square error, etc. This disclosure does not limit this.

[0094] The final KPI may also become a transmission parameter, and may include at least one of the following: system throughput, network block error rate (BLER), assumed BLER, acknowledgment character ACK, and non-acknowledgment character NACK.

[0095] System throughput and network block error rate (BER) represent data transmission performance. Data transmission is based on precoding, which is determined based on the first output. ACK and NACK are determined based on the BLER. When the BLER exceeds the corresponding threshold, the message is restored to NACK; otherwise, it is ACK.

[0096] Step 203: Send first indication information, where the first indication information includes current performance information of the model.

[0097] It can be understood that after the terminal device determines the current performance information of the model, it can send the performance information to the network device so that the network device can perform any of the following operations on the CSI feedback model based on the current performance information of the CSI feedback model: activation, deactivation, update, and switching.

[0098] In summary, the terminal device monitors the model, obtains the channel state information (CSI) currently associated with the model, and then determines the model's current performance information based on the model's current CSI. Finally, the terminal device sends first indication information containing the model's current performance information to the network device. This allows the terminal device to accurately determine the model's current performance information based on the CSI, improving the accuracy and reliability of model performance monitoring.

[0099] Please refer to Figure 3, which is a flow chart of a method for monitoring model performance provided by an embodiment of the present disclosure. As shown in Figure 3, the method is applied to a terminal device. The method may include but is not limited to the following steps:

[0100] Step 301: Receive second indication information, wherein the second indication information includes monitoring parameters associated with the model.

[0101] In some possible implementations, the monitoring parameters may include one or more of the following:

[0102] Monitoring start time, number of monitoring results reported, monitoring time information and monitoring cycle.

[0103] The monitoring start time may be the start time of monitoring the model by the terminal device.

[0104] The number of times the detection result is reported may be the total number of times the terminal device reports the monitoring result to the network device during the process of monitoring the model.

[0105] The monitoring time information may be the length of time that the terminal device monitors the model.

[0106] The monitoring period can be the period during which the terminal device monitors the model, for example, how often the model is monitored.

[0107] In some possible implementations, the second indication information sent by the network device may be received by one or more of the following:

[0108] Radio Resource Control (RRC) message, Media Access Control Element (MAC-CE), Downlink Control Information (DCI).

[0109] For example, the terminal device may receive the second indication information through a combination of RRC and MAC-CE, or through a combination of RRC and DCI, which is not specifically limited in the present disclosure.

[0110] In some possible implementations, the terminal device may also determine the monitoring parameters associated with the model based on protocol agreements.

[0111] In some possible implementations, after determining the monitoring parameters, the terminal device may further send the monitoring parameters associated with the model to the network device, so that the network device can send the second CSI-RS to the terminal device based on the monitoring parameters.

[0112] In some possible implementations, the terminal device may send model-associated monitoring parameters to the network device based on a preset event trigger.

[0113] The preset event may be an event pre-set by the terminal device and used to trigger the sending of monitoring parameters associated with the model to the network device.

[0114] Step 302: Monitor the model and obtain the channel state information CSI currently associated with the model.

[0115] Step 303: Determine the current performance information of the model based on the CSI currently associated with the model.

[0116] Step 304: Send first indication information, where the first indication information includes current performance information of the model.

[0117] In the present disclosure, a terminal device first receives second indication information containing monitoring parameters associated with a model, then monitors the model based on the monitoring parameters, obtains the channel state information (CSI) currently associated with the model, and then determines the model's current performance information based on the model's current CSI. Finally, a first indication information containing the model's current performance information is sent to a network device. This allows monitoring of the model to be triggered based on the monitoring parameters, and the model's current performance information can be accurately determined based on the determined CSI, further improving the accuracy and reliability of model performance monitoring.

[0118] Please refer to Figure 4, which is a flow chart of a method for monitoring model performance provided by an embodiment of the present disclosure. As shown in Figure 4, the method is applied to a terminal device. The method may include but is not limited to the following steps:

[0119] Step 401: Monitor the model and obtain the channel state information CSI currently associated with the model.

[0120] Step 402: When the data transmission rank is greater than 1, determine the CSI corresponding to the model in each data transmission layer.

[0121] It can be understood that when the data transmission rank is greater than 1, the network device can send different CSI-RS to the terminal device through each channel (data transmission layer), so that the terminal device can determine the CSI corresponding to each data transmission layer through each received CSI-RS.

[0122] Among them, the specific implementation method of determining the corresponding CSI of each data transmission layer can refer to the detailed description in other embodiments of the present disclosure, and will not be repeated here.

[0123] Step 403: Determine the performance information of the model in each data transmission layer according to the CSI corresponding to each data transmission layer, and / or determine the performance information of the model in all data transmission layers.

[0124] It can be understood that after determining the CSI corresponding to each data transmission layer, the performance information of the model in each data transmission layer can be determined, and / or the performance information of the model in all data transmission layers can be determined.

[0125] It should be noted that whether to determine the performance information of the model at each data transmission layer or to determine the performance information of the model at all data transmission layers is related to the design of the model and is not specifically limited in this disclosure.

[0126] Step 404: Send first indication information, where the first indication information includes current performance information of the model.

[0127] In the present disclosure, the terminal device first monitors the model based on the monitoring parameters, obtains the channel state information CSI currently associated with the model, and when the data transmission rank is greater than 1, determines the CSI corresponding to the model in each data transmission layer, and based on the CSI corresponding to each data transmission layer, determines the performance information of the model in each data transmission layer, and / or determines the performance information of the model in all data transmission layers, and finally sends the first indication information containing the current performance information of the model to the network device. In this way, the performance information of the model in each data transmission layer can be accurately determined based on the CSI corresponding to the model in each data transmission layer, thereby improving the accuracy, reliability and comprehensiveness of model performance monitoring.

[0128] Please refer to Figure 5, which is a flow chart of a method for monitoring model performance provided by an embodiment of the present disclosure. As shown in Figure 5, the method is applied to a terminal device. The method may include but is not limited to the following steps:

[0129] Step 501: Monitor the model and obtain the channel state information CSI currently associated with the model.

[0130] Step 502: Determine the current performance information of the model based on the CSI currently associated with the model.

[0131] In some possible implementations, a current first performance parameter of the model is determined based on the first input and the first information.

[0132] The first performance parameter, which may also be referred to as an intermediate KPI, may be the square of the cosine similarity between the first input and the first information, or the normalized minimum mean square error between the first input and the first information, etc. This disclosure does not limit this.

[0133] The formula for determining the square of the cosine similarity may be:

[0134] Where K1 is the square of the cosine similarity, H is the downlink channel information H of different frequency domain units estimated by the terminal device according to the second CSI-RS, v i is the first input, e i is the first information, and N3 represents the number of frequency domain units.

[0135] In some possible implementations, the downlink channel information used to calculate the KPI is: downlink channel information that is not earlier than the CSI reporting time and the most recent estimated time before the CSI reporting time. If there are multiple CSI reports, the downlink channel information that is not earlier than the first CSI reporting time and the most recent estimated time before the CSI reporting time is used.

[0136] Figure 5A is a CSI measurement and reporting timing diagram provided by an embodiment of the present disclosure. As shown in Figure 5A, there are CSI report1 and CSI report2. The terminal device estimates the downlink channel information at three different times based on CSI-RS1, CSI-RS2, and CSI-RS3. However, the terminal device calculates the first performance parameter using the downlink channel information estimated with CSI-RS3.

[0137] In some possible implementations, a current first performance parameter of the model is determined based on the first input and the first information, and then a comparison result between the current first performance parameter and the first threshold value is determined as the current second performance parameter of the model.

[0138] The comparison result between the first performance parameter and the first threshold value may be a magnitude relationship between the first performance parameter and the first threshold value, or may be a difference between the first performance parameter and the first threshold value.

[0139] The first threshold value may be used to judge the first performance parameter to determine the performance result of the model. For example, if the first performance parameter is less than or equal to the first threshold value, the performance of the model is determined to be poor, and if the first performance parameter is greater than the first threshold value, the performance of the model is determined to be good.

[0140] In the present disclosure, the terminal device may indicate the second performance parameter using one bit. For example, when the first performance parameter is less than or equal to a first threshold, the value of the bit may be 0, indicating poor performance of the model; otherwise, the value may be 1, indicating good performance of the model.

[0141] The first threshold value may be indicated by the network device, may be pre-set, or may be agreed upon in a protocol, which is not limited in the present disclosure.

[0142] In some possible implementations, the current first performance parameter of the model can also be determined based on the first input and the first information, and then the reference parameter can be determined based on the received codebook parameter information and the first input, and then the reference parameter and the first performance parameter, or the comparison result of the reference parameter and the first performance parameter, can be determined as the current third performance parameter of the model.

[0143] The codebook parameter information may include a codebook parameter combination configured by the network device for the terminal device. The codebook parameter information may include one or more codebook parameter combinations. This disclosure does not limit this.

[0144] The codebook parameter combination may include a codebook parameter and a codebook type. The codebook type may include Type I and Type II, wherein the Type II codebook is further divided into a Rel-15 / 16 Type II codebook or a Rel-17 Type II port selection codebook.

[0145] The reference parameter may include the square of the reference cosine similarity, or the reference normalized minimum mean square error, etc.

[0146] The comparison result between the reference parameter and the first performance parameter may be a difference between the reference parameter and the first performance parameter, or a ratio between the first performance parameter and the reference parameter.

[0147] Optionally, the terminal device can determine the compressed first CSI feedback load size corresponding to the model and the second CSI feedback load size determined by the codebook parameter information, and then determine the absolute value of the difference between the second CSI feedback load size and the first CSI feedback load size, and then determine the reference parameter based on the codebook parameter combination with the smallest corresponding absolute value and the first input.

[0148] The CSI feedback payload size refers to the feedback payload for each transmission layer or for all transmission layers. The CSI feedback payload size can be a measure of the channel resources or data capacity required for CSI feedback. It can be expressed as the number of bits, bandwidth, or time slices used to transmit CSI.

[0149] The first CSI feedback payload size may be a feedback payload size corresponding to the CSI compressed by the CSI generation model, and the second CSI feedback payload size may be a CSI feedback payload size determined based on codebook parameter information.

[0150] Specifically, the downlink channel information H of different frequency domain units estimated according to the second CSI-RS is precoded based on the codebook parameter combination with the smallest absolute value to obtain the precoding vector b of the different frequency domain units, and the reference parameter is determined based on the precoding vector b and the first input.

[0151] Among them, the reference parameters may include the square of the reference cosine similarity (i.e., the square of the cosine similarity between the precoding vector b and the first input), or the reference normalized minimum mean square error (i.e., the normalized minimum mean square error between the precoding vector b and the first input), etc.

[0152] For example, the codebook parameter information configures three codebook parameter combinations, namely codebook parameter combination 1, codebook parameter combination 2, and codebook parameter combination 3. The second CSI feedback payload size corresponding to codebook parameter combination 1 is 50, the second CSI feedback payload size corresponding to codebook parameter combination 2 is 100, the second CSI feedback payload size corresponding to codebook parameter combination 3 is 150, and the first CSI feedback payload size is 90. Codebook parameter combination 2 is determined to be the codebook parameter combination with the smallest absolute value, and the reference parameter is determined based on codebook parameter combination 2.

[0153] In some possible implementations, the terminal device may also determine a first distribution offset of the first input relative to the second input in the training data corresponding to the model, or a second distribution offset of the first output relative to the second output in the training data, and then determine the first distribution offset or the second distribution offset as the current fourth performance parameter of the model.

[0154] In some possible implementations, the terminal device may also determine a first distribution offset of the first input relative to the second input in the training data corresponding to the model, and a second distribution offset of the first output relative to the second output in the training data, and then determine the comparison result of the first distribution offset with the second threshold value, or the comparison result of the second distribution offset with the third threshold value, as the current fifth performance parameter of the model.

[0155] The comparison result between the first distribution offset and the second threshold value may be a magnitude relationship between the first distribution offset and the second threshold value, or a difference between the first distribution offset and the second threshold value. Similarly, the comparison result between the second distribution offset and the third threshold value may be a magnitude relationship between the second distribution offset and the third threshold value, or a difference between the second distribution offset and the third threshold value.

[0156] The second threshold value may be used to judge the first distribution offset to determine the performance of the model. For example, if the first distribution offset is less than or equal to the second threshold value, the model performance is determined to be poor, and if the first distribution offset is greater than the second threshold value, the model performance is determined to be good.

[0157] The third threshold value may be used to judge the second distribution offset to determine the performance of the model. For example, if the second distribution offset is less than or equal to the third threshold value, the model performance is determined to be poor, and if the second distribution offset is greater than the third threshold value, the model performance is determined to be good.

[0158] In the present disclosure, the terminal device may indicate the fifth performance parameter using one bit. For example, when the second distribution offset is less than or equal to the third threshold, the value of the bit may be 0, indicating poor performance of the model; otherwise, the value may be 1, indicating good performance of the model.

[0159] The second threshold value and the third threshold value may be indicated by the network device, may be pre-set, or may be agreed upon in a protocol, which is not limited in this disclosure.

[0160] In some possible implementations, the terminal device may also determine a current transmission parameter of the system based on the first information, and then determine a sixth performance parameter corresponding to the model based on the current transmission parameter.

[0161] The current transmission parameter may also become the final KPI, and the transmission parameter includes at least one of the following: system throughput, network block error rate BLER, assumed BLER, acknowledgment character ACK, and non-acknowledgment character NACK.

[0162] In some possible implementations, the terminal device may also determine the current transmission parameter of the system based on the first information, and then determine the seventh performance parameter corresponding to the model based on a comparison result between the current transmission parameter and the fourth threshold value.

[0163] The comparison result between the transmission parameter and the fourth threshold may be a magnitude relationship between the transmission parameter and the fourth threshold, or may be a difference between the transmission parameter and the fourth threshold.

[0164] The fourth threshold value may be used to judge the transmission parameter to determine the performance result of the model. For example, if the transmission parameter is less than or equal to the fourth threshold value, the performance of the model is determined to be poor, and if the transmission parameter is greater than the fourth threshold value, the performance of the model is determined to be good.

[0165] In the present disclosure, the terminal device may indicate the seventh performance parameter using one bit. For example, when the transmission parameter is less than or equal to the fourth threshold, the value of the bit may be 0, indicating poor performance of the model; otherwise, the value may be 1, indicating good performance of the model.

[0166] The fourth threshold value may be indicated by the network device, may be pre-set, or may be agreed upon in a protocol, which is not limited in the present disclosure.

[0167] In some possible implementations, the terminal device may also determine a first comparison result between the current performance information and the fifth threshold value, and then determine an eighth performance parameter corresponding to the model based on K consecutive first comparison results of the model, where k is a natural number.

[0168] The fifth threshold value, K, may be indicated by the network device, may be pre-set, or may be agreed upon in a protocol, which is not limited in this disclosure.

[0169] The eighth performance parameter may include the number of times the performance information is greater than the fifth threshold value in the K first comparison results, or may also include the proportion of the performance information greater than the fifth threshold value in the K first comparison results.

[0170] In some possible implementations, the terminal device can also determine a first comparison result between the current performance information and the fifth threshold value, and then determine a ninth performance parameter corresponding to the model based on K consecutive first comparison results of the model and a second comparison result of the sixth threshold value, where K is a natural number.

[0171] The sixth threshold value may be indicated by the network device, or may be pre-set, or may be agreed upon in a protocol, which is not limited in the present disclosure.

[0172] The sixth threshold value may be a times threshold value or a ratio threshold value.

[0173] Among them, the ninth performance parameter may include the relationship between the number of times the performance information is greater than the fifth threshold value in the K first comparison results and the sixth threshold value (number threshold); or it may also include the relationship between the proportion of the performance information greater than the fifth threshold value in the K first comparison results and the sixth threshold value (proportion threshold).

[0174] Step 503: Send first indication information, where the first indication information includes current performance information of the model.

[0175] The terminal device may report the performance parameters of a single test or the performance parameters of multiple tests through the first indication information, which is not limited in the present disclosure.

[0176] In this disclosure, a terminal device monitors a model, obtains the channel state information (CSI) currently associated with the model, and then determines the model's current performance information based on the CSI. Finally, the terminal device sends first indication information containing the model's current performance information to a network device. This allows the terminal device to accurately determine the model's current performance information based on the CSI, improving the accuracy and reliability of model performance monitoring.

[0177] Please refer to Figure 6, which is a flow chart of a method for monitoring model performance provided by an embodiment of the present disclosure. As shown in Figure 6, the method is applied to a terminal device. The method may include but is not limited to the following steps:

[0178] Step 601: Determine a second CSI feedback payload size determined by a selected codebook parameter combination and a compressed first CSI feedback payload size corresponding to each model in N models.

[0179] The selected codebook parameter combination may be selected by the terminal device from a plurality of codebook parameter combinations configured by the network device, or may be configured by the network device.

[0180] Step 602: Determine the model among the N models whose corresponding first CSI feedback payload size is less than or equal to the second CSI feedback payload size, or the model whose absolute value of the difference between the model and the first CSI feedback payload size is the smallest, as the model to be monitored.

[0181] For example, assume that the terminal device deploys N=3 CSI generation part models, and each CSI generation part model corresponds to a different first CSI feedback payload size. In addition, the second CSI feedback payload size determined by the selected codebook parameter combination is also determined. If the CSI compression feedback payload of CSI generation part model 1 and CSI generation part model 2 are both smaller than the second CSI feedback payload size, the terminal device can select CSI generation part model 1 and CSI generation part model 2 as the models to be monitored.

[0182] In some possible implementations, the terminal device may also monitor at least M models out of N models, where N is the number of models deployed in the terminal device and M is a value less than or equal to N. That is, the terminal device may also randomly select M models to monitor.

[0183] Step 603: Monitor the model and obtain the channel state information CSI currently associated with the model.

[0184] Step 604: Determine the current performance information of the model based on the CSI currently associated with the model.

[0185] Step 605: Determine a sending mode of the first indication information.

[0186] The sending mode includes at least one of the following: sending frequency, sending times, and sending content.

[0187] The sending frequency may be how often the terminal device sends the first indication information.

[0188] The number of times of sending may be the total number of times the terminal device sends the first indication information during the process of monitoring the model.

[0189] The sent content may be the specific performance parameters of the performance information sent by the terminal through the first indication information. That is, the sent content may be one or more of the first performance parameter, the second performance parameter, the third performance parameter, the fourth performance parameter, the fifth performance parameter, the sixth performance parameter, the seventh performance parameter, the eighth performance parameter, and the ninth performance parameter.

[0190] In some possible implementations, the terminal device may determine the sending mode according to an instruction of the network device; or determine the sending mode according to a protocol agreement.

[0191] Alternatively, the sending mode may be determined based on the current performance information of the model. For example, if the performance information indicates that the model's performance is relatively good, the sending frequency or number of times may be reduced; if the performance information indicates that the model's performance is relatively poor, the sending frequency or number of times may be increased.

[0192] Step 606: Send first indication information based on the sending mode, wherein the first indication information includes current performance information of the model.

[0193] In the present disclosure, after the sending mode is determined, the first indication information can be sent based on the sending mode.

[0194] In the present disclosure, a terminal device can select a model to be detected from the N deployed models, monitor the model, obtain the channel state information (CSI) currently associated with the model, determine the model's current performance information based on the CSI currently associated with the model, and finally, based on a determined transmission mode, send first indication information containing the model's current performance information to a network device. Thus, the terminal device can first determine the model to be monitored, and then accurately determine the model's current performance information based on the CSI corresponding to the model to be monitored, thereby improving the accuracy and reliability of model performance monitoring.

[0195] Please refer to Figure 7, which is a flow chart of a method for monitoring model performance provided by an embodiment of the present disclosure. As shown in Figure 7, the method is applied to a network device. The method may include but is not limited to the following steps:

[0196] Step 701: Receive first indication information, where the first indication information includes current performance information of a channel state information (CSI) feedback model.

[0197] The first indication information is sent by the terminal device to the network device. The terminal device can report the performance parameters of a single test or the performance parameters of multiple tests through the first indication information. This disclosure does not make specific restrictions on this.

[0198] In some possible implementations, the performance information may include at least one of the following:

[0199] A first performance parameter, wherein the first performance parameter is determined based on a current first input and first information of the model;

[0200] A comparison result of the first performance parameter and the first threshold value;

[0201] A reference parameter and a first performance parameter, wherein the reference parameter is a parameter determined by the terminal device based on the received codebook parameter information and the first input;

[0202] a comparison result between the reference parameter and the first performance parameter;

[0203] A first distribution offset, wherein the first distribution offset is an offset between the first input and the second input in the training data corresponding to the model;

[0204] A second distribution offset, wherein the second distribution offset is an offset between the first output and the second output in the training data corresponding to the model;

[0205] a fifth performance parameter, wherein the fifth performance parameter is a comparison result of the first distribution offset and the second threshold value, or a comparison result of the second distribution offset and the third threshold value;

[0206] a sixth performance parameter, wherein the sixth performance parameter is determined by the terminal device based on a current transmission parameter of the system, wherein the transmission parameter includes at least one of the following: system throughput, network block error rate (BLER), assumed BLER, acknowledgment character (ACK), and non-acknowledgment character (NACK);

[0207] a comparison result of the sixth performance parameter and the fourth threshold value;

[0208] an eighth performance parameter, wherein the eighth performance parameter is a first comparison result of K consecutive performance information of the model and the fifth threshold value;

[0209] The ninth performance parameter is the second comparison result of the K consecutive first comparison results of the model and the sixth threshold value, wherein the first comparison result is the comparison result of each consecutive performance information of the model and the fifth threshold value.

[0210] The first performance parameter, which may also be referred to as an intermediate KPI, may be the square of the cosine similarity between the first input and the first information, or the normalized minimum mean square error between the first input and the first information, etc. This disclosure does not limit this.

[0211] The first input can be that the terminal device estimates the downlink channel information H of different frequency domain units according to the received second CSI-RS, and then performs singular value decomposition on the downlink channel information H of the i-th frequency domain unit to obtain the eigenvector v i .

[0212] The first information is the current first output of the model or related information of the first output.

[0213] The comparison result between the first performance parameter and the first threshold value may be a magnitude relationship between the first performance parameter and the first threshold value, or a difference between the first performance parameter and the first threshold value. The comparison result between the first performance parameter and the first threshold value is the second performance parameter.

[0214] The codebook parameter information may include a codebook parameter combination configured by the network device for the terminal device. The codebook parameter information may include one or more codebook parameter combinations. This disclosure does not limit this.

[0215] The codebook parameter combination may include a codebook parameter and a codebook type. The codebook type may include Type I and Type II, wherein the Type II codebook is further divided into a Rel-15 / 16 Type II codebook or a Rel-17 Type II port selection codebook.

[0216] The comparison result between the reference parameter and the first performance parameter may be a difference between the reference parameter and the first performance parameter, or a ratio between the first performance parameter and the reference parameter. The comparison result between the reference parameter and the first performance parameter is the third performance parameter.

[0217] The first distribution offset or the second distribution offset may also be referred to as a fourth performance parameter.

[0218] The comparison result between the sixth performance parameter and the fourth threshold may be the magnitude relationship between the sixth performance parameter and the fourth threshold, or may be the difference between the sixth performance parameter and the fourth threshold.

[0219] The comparison result between the sixth performance parameter and the fourth threshold value may also be referred to as the seventh performance parameter.

[0220] The eighth performance parameter may include the number of times the performance information is greater than the fifth threshold value in the K first comparison results, or may also include the proportion of the performance information greater than the fifth threshold value in the K first comparison results.

[0221] The sixth threshold value may be a times threshold value or a ratio threshold value.

[0222] Among them, the ninth performance parameter may include the relationship between the number of times the performance information is greater than the fifth threshold value in the K first comparison results and the sixth threshold value (number threshold); or it may also include the relationship between the proportion of the performance information greater than the fifth threshold value in the K first comparison results and the sixth threshold value (proportion threshold).

[0223] In some possible implementations, the first threshold, the second threshold, the third threshold, the fourth threshold, the fifth threshold, and the sixth threshold may be indicated by a network device, may be pre-set, or may be agreed upon in a protocol. This disclosure does not limit this.

[0224] Therefore, after receiving the first indication information, the network device can determine the current performance information of the CSI feedback model.

[0225] In the present disclosure, the network device can accurately obtain the current performance of the channel state information CSI feedback model by receiving first indication information corresponding to the current performance information of the CSI feedback model.

[0226] Please refer to Figure 8, which is a flow chart of a method for monitoring model performance provided by an embodiment of the present disclosure. As shown in Figure 8, the method is applied to a network device. The method may include but is not limited to the following steps:

[0227] Step 801: Send a first output currently corresponding to the model to the terminal device; or, send a first channel state information reference signal CSI-RS to the terminal device, wherein the first CSI-RS is the CSI-RS after beamforming corresponding to the first output.

[0228] In this disclosure, if the terminal device side does not include a CSI recovery partial model, it is necessary to obtain the first output of the CSI recovery partial model on the network device side. Specifically, the terminal device quantizes the CSI compressed by the CSI generation partial model and reports it to the network device. The network device inputs it into the CSI recovery partial model, and the CSI recovery partial model recovers the characteristic vector e of each frequency domain unit. i , i.e. the first output. Then, the network device can i Perform scalar quantization or send it to the terminal device in the form of codebook parameters.

[0229] In the present disclosure, the network device recovers the characteristic vector e of each frequency domain unit through the CSI recovery model. i Afterwards, the eigenvector e can be i Perform beamforming and send the beamformed first CSI-RS to the terminal device, and the terminal device estimates the association information of the first output based on the received beamformed first CSI-RS. The beam used is the first output, that is, the eigenvector e i .

[0230] In some possible implementations, the network device may receive a first information acquisition request sent by the terminal device, where the first information is the first output of the model or associated information of the first output.

[0231] In some possible implementations, when the terminal device side does not include a CSI recovery partial model, the network device may also actively send the first output of the CSI recovery partial model or the first CSI-RS to the terminal device.

[0232] Step 802: Receive first indication information, where the first indication information includes current performance information of a channel state information (CSI) feedback model.

[0233] In some possible implementations, when the data transmission rank is greater than 1, the performance information corresponding to the model in each data transmission layer is determined, and / or the performance information of the model in all transmission layers is determined.

[0234] It can be understood that when the data transmission rank is greater than 1, the network device can send different CSI-RS to the terminal device through each channel (data transmission layer), so that the terminal device can determine the CSI corresponding to each data transmission layer through each received CSI-RS. Then, based on the CSI corresponding to each data transmission layer, the performance information of the model at each data transmission layer is determined, and / or the performance information of the model at all data transmission layers is determined.

[0235] In the present disclosure, a network device first sends a first output or first CSI-RS corresponding to the current model to a terminal, and then receives first indication information corresponding to the current performance information of the channel state information (CSI) feedback model. This allows the terminal device to accurately determine the current performance information of the CSI feedback model by combining the first output or first CSI-RS, and thus accurately obtain the current performance of the CSI feedback model.

[0236] Please refer to Figure 9, which is a flow chart of a method for monitoring model performance provided by an embodiment of the present disclosure. As shown in Figure 9, the method is applied to a network device. The method may include but is not limited to the following steps:

[0237] Step 901: Send second indication information, wherein the second indication information includes monitoring parameters associated with the model.

[0238] In some possible implementations, the monitoring parameters include one or more of the following:

[0239] Monitoring start time, number of monitoring results reported, monitoring time information and monitoring cycle.

[0240] The monitoring start time may be the start time of monitoring the model by the terminal device.

[0241] The number of times the detection result is reported may be the total number of times the terminal device reports the monitoring result to the network device during the process of monitoring the model.

[0242] The monitoring time information may be the length of time that the terminal device monitors the model.

[0243] The monitoring period can be the period during which the terminal device monitors the model, for example, how often the model is monitored.

[0244] In some possible implementations, the second indication information may be sent through one or more of the following:

[0245] Radio Resource Control (RRC) message, Media Access Control Element (MAC-CE), Downlink Control Information (DCI).

[0246] For example, the second indication information may be sent through a combination of RRC and MAC-CE, or through a combination of RRC and DCI, which is not specifically limited in the present disclosure.

[0247] In some possible implementations, the network device may determine the monitoring parameters associated with the model based on protocol agreements.

[0248] In some possible implementations, the terminal device may also determine model-associated monitoring parameters based on protocol agreements, and after determining the monitoring parameters, send the model-associated monitoring parameters to the network device. The network device can then receive the model-associated monitoring parameters sent by the terminal device and send the second CSI-RS to the terminal device based on the monitoring parameters.

[0249] Step 902: Receive first indication information, where the first indication information includes current performance information of a channel state information (CSI) feedback model.

[0250] The sending frequency may be how often the terminal device sends the first indication information.

[0251] The number of times of sending may be the total number of times the terminal device sends the first indication information during the process of monitoring the model.

[0252] The sent content may be the specific performance parameters of the performance information sent by the terminal through the first indication information. That is, the sent content may be one or more of the first performance parameter, the second performance parameter, the third performance parameter, the fourth performance parameter, the fifth performance parameter, the sixth performance parameter, the seventh performance parameter, the eighth performance parameter, and the ninth performance parameter.

[0253] In the present disclosure, a network device may first send model-associated monitoring parameters to a terminal, and then receive indication information from the terminal containing current performance information of the channel state information (CSI) feedback model. This allows the terminal to trigger model monitoring based on the monitoring parameters, and then receive the current performance information of the CSI feedback model, thereby accurately obtaining the current performance of the CSI feedback model.

[0254] Please refer to Figure 10, which is a flow chart of a method for monitoring model performance provided by an embodiment of the present disclosure. As shown in Figure 10, the method is applied to a network device. The method may include but is not limited to the following steps:

[0255] Step 1001: Determine a sending mode of first indication information.

[0256] The sending mode includes at least one of the following: sending frequency, sending times, and sending content;

[0257] The sending frequency may be how often the terminal device sends the first indication information.

[0258] The number of times of sending may be the total number of times the terminal device sends the first indication information during the process of monitoring the model.

[0259] The sent content may be the specific performance parameters of the performance information sent by the terminal through the first indication information. That is, the sent content may be one or more of the first performance parameter, the second performance parameter, the third performance parameter, the fourth performance parameter, the fifth performance parameter, the sixth performance parameter, the seventh performance parameter, the eighth performance parameter, and the ninth performance parameter.

[0260] In some possible implementations, the network device may determine the sending mode according to a protocol agreement.

[0261] Alternatively, after determining the sending mode, the terminal device indicates the sending mode to the terminal device, so that the terminal device can send the first indication information based on the sending mode.

[0262] Alternatively, the network device may also receive the transmission mode sent by the terminal device.

[0263] Step 1002: Receive first indication information based on the sending mode.

[0264] Step 1003: Perform any one of the following operations on the CSI feedback model according to the current performance information of the CSI feedback model: activation, deactivation, update, and switching.

[0265] In some possible implementations, the CSI feedback method may be switched from implementing CSI feedback based on a CSI feedback model to implementing CSI feedback based on Type I and Type II codebooks.

[0266] In the present disclosure, after determining the current performance information of the CSI feedback model, the CSI feedback model can be activated, deactivated, updated, or switched to improve the performance of the CSI feedback model, or the CSI feedback method can be switched to improve the accuracy of the acquired CSI feedback information.

[0267] In the present disclosure, a network device may first determine a transmission mode for the first indication information and then, based on the transmission mode, receive indication information corresponding to the current performance information of the channel state information (CSI) feedback model sent by the terminal. Thus, the current performance information of the CSI feedback model sent by the terminal device can be accurately obtained based on the transmission mode.

[0268] Please refer to Figure 11, which is a signaling interaction diagram of another model performance monitoring method provided by an embodiment of the present disclosure. As shown in Figure 11, the method may include but is not limited to the following steps:

[0269] Step 1101: The terminal device monitors the model and obtains the channel state information CSI currently associated with the model.

[0270] Step 1102: The terminal device determines the current performance information of the model based on the CSI currently associated with the model.

[0271] Step 1103: The terminal device sends first indication information to the network device, where the first indication information includes current performance information of the model.

[0272] Step 1104: Upon receiving the first indication information, the network device performs any one of the following operations on the CSI feedback model according to the current performance information of the CSI feedback model: activation, deactivation, update, and switching.

[0273] Thus, the terminal device monitors the model, obtains the channel state information (CSI) currently associated with the model, and then determines the model's current performance information based on the model's current CSI. Finally, the terminal device sends first indication information containing the model's current performance information to the network device. This allows the terminal device to accurately determine the model's current performance information based on the CSI, improving the accuracy and reliability of model performance monitoring.

[0274] Please refer to Figure 12, which is a schematic diagram of the structure of a communication device provided in an embodiment of the present disclosure. The communication device 1200 shown in Figure 12 may include a transceiver module 1201 and a processing module 1202. The transceiver module may include a sending module and / or a receiving module. The sending module is used to implement a sending function, and the receiving module is used to implement a receiving function. The transceiver module can implement the sending function and / or the receiving function.

[0275] The communication device 1200 may be a terminal device, a device in a terminal device, or a device that can be used in conjunction with the terminal device.

[0276] The communication device 1200 is on the terminal device side, wherein:

[0277] Processing module 1202 is configured to monitor the model and obtain channel state information (CSI) currently associated with the model, where the CSI includes any of the following: first information, a current first input of the model, and first information; wherein the first information is a current first output of the model or information associated with the first output, and the model is a CSI feedback model;

[0278] The processing module 1202 is further configured to determine current performance information of the model based on the CSI currently associated with the model;

[0279] The transceiver module 1201 is further configured to send first indication information, wherein the first indication information includes current performance information of the model.

[0280] In some possible implementations, the processing module 1202 is further configured to:

[0281] In a case where the terminal device side includes a CSI recovery partial model, determining an output of the CSI recovery partial model as a first output, wherein the CSI recovery partial model on the terminal device side is the same as or different from the CSI recovery partial model on the network device side; or

[0282] In the case where the terminal device side does not include the CSI recovery partial model, receiving the first output sent by the network device; or,

[0283] When the terminal device side does not include a CSI recovery partial model, the associated information is determined based on the received first channel state information reference signal CSI-RS, wherein the first CSI-RS is the CSI-RS after beamforming corresponding to the first output.

[0284] In some possible implementations, the transceiver module 1201 is further configured to:

[0285] When the terminal device side does not include the CSI recovery partial model, a first information acquisition request is sent.

[0286] In some possible implementations, the processing module 1202 is further configured to:

[0287] A first input is determined according to the received second CSI-RS.

[0288] In some possible implementations,

[0289] The transceiver module 1201 is further configured to receive second indication information, wherein the second indication information includes monitoring parameters associated with the model; or

[0290] The processing module 1202 is further configured to determine monitoring parameters associated with the model based on the protocol agreement; or

[0291] The transceiver module 1201 is further configured to send monitoring parameters associated with the model to the network device.

[0292] In some possible implementations, the transceiver module 1201 is further configured to:

[0293] Based on preset event triggers, the monitoring parameters associated with the model are sent to the network device.

[0294] In some possible implementations, the monitoring parameters include one or more of the following:

[0295] Monitoring start time, number of monitoring results reported, monitoring time information and monitoring cycle.

[0296] In some possible implementations, the second indication information is received through one or more of the following: a radio resource control RRC message, a media access control element MAC-CE, and downlink control information DCI.

[0297] In some possible implementations, the processing module 1202 is further configured to:

[0298] When the data transmission rank is greater than 1, determine the CSI corresponding to the model in each data transmission layer;

[0299] According to the CSI corresponding to each transmission layer, performance information of the model at each transmission layer is determined, and / or performance information of the model at all transmission layers is determined.

[0300] In some possible implementations, the processing module 1202 is further configured to:

[0301] A current first performance parameter of the model is determined according to the first input and the first information.

[0302] In some possible implementations, the processing module 1202 is further configured to:

[0303] Determining a current first performance parameter of the model based on the first input and the first information;

[0304] A comparison result between the current first performance parameter and the first threshold value is determined as the current second performance parameter of the model.

[0305] In some possible implementations, the processing module 1202 is further configured to:

[0306] Determining a current first performance parameter of the model based on the first input and the first information;

[0307] Determining a reference parameter based on the received codebook parameter information and the first input;

[0308] The reference parameter and the first performance parameter, or a comparison result between the reference parameter and the first performance parameter, is determined as the current third performance parameter of the model.

[0309] In some possible implementations, the processing module 1202 is further configured to:

[0310] Determine a compressed first CSI feedback payload size corresponding to the model and a second CSI feedback payload size determined by codebook parameter information;

[0311] determining an absolute value of a difference between a second CSI feedback payload size and a first CSI feedback payload size;

[0312] A reference parameter is determined based on the codebook parameter combination corresponding to the smallest absolute value and the first input.

[0313] In some possible implementations, the processing module 1202 is further configured to:

[0314] determining a first distributional shift of the first input relative to a second input in the training data corresponding to the model, or

[0315] a second distribution shift of the first output relative to the second output in the training data;

[0316] The first distribution offset or the second distribution offset is determined as the fourth current performance parameter of the model.

[0317] In some possible implementations, the processing module 1202 is further configured to:

[0318] Determine the first distribution offset of the first input relative to the second input in the training data corresponding to the model, and

[0319] a second distribution offset of the first output relative to the second output in the training data;

[0320] Compare the first distribution offset with the second threshold value, or compare the second distribution offset with the third threshold value.

[0321] The comparison result of the limit values ​​is determined as the current fifth performance parameter of the model.

[0322] In some possible implementations, the processing module 1202 is further configured to:

[0323] Determine a current transmission parameter of the system based on the first information, where the transmission parameter includes at least one of the following:

[0324] System throughput, network block error rate (BLER), assumed BLER, acknowledgment character (ACK), non-acknowledgment character (NACK);

[0325] A sixth performance parameter corresponding to the model is determined according to the current transmission parameter.

[0326] In some possible implementations, the processing module 1202 is further configured to:

[0327] Determine a current transmission parameter of the system according to the first information, wherein the transmission parameter includes at least one of the following: system throughput, network block error rate (BLER), assumed BLER, acknowledgment character (ACK), and non-acknowledgment character (NACK);

[0328] A seventh performance parameter corresponding to the model is determined based on a comparison result between the current transmission parameter and the fourth threshold value.

[0329] In some possible implementations, the processing module 1202 is further configured to:

[0330] determining a first comparison result between the current performance information and a fifth threshold;

[0331] An eighth performance parameter corresponding to the model is determined according to K consecutive first comparison results of the model, where K is a natural number.

[0332] In some possible implementations, the processing module 1202 is further configured to:

[0333] determining a first comparison result between the current performance information and a fifth threshold;

[0334] A ninth performance parameter corresponding to the model is determined according to K consecutive first comparison results of the model and a second comparison result of the sixth threshold value, where K is a natural number.

[0335] In some possible implementations, the processing module 1202 is further configured to:

[0336] At least M models out of N models are monitored, where N is the number of models deployed in the terminal device and M is a value less than or equal to N.

[0337] In some possible implementations, the processing module 1202 is further configured to:

[0338] Determine a second CSI feedback payload size determined by the selected codebook parameter combination and a compressed first CSI feedback payload size corresponding to each model in the N models;

[0339] The model whose corresponding first CSI feedback payload size is less than or equal to the second CSI feedback payload size among the N models, or the model whose absolute value of the difference between the model and the first CSI feedback payload size is the smallest, is determined as the model to be monitored.

[0340] In some possible implementations, the transceiver module 1201 is further configured to:

[0341] Determine a sending mode of the first indication information, wherein the sending mode includes at least one of the following: sending frequency, sending times, and sending content;

[0342] Based on the sending mode, first indication information is sent.

[0343] In some possible implementations, the processing module 1202 is further configured to:

[0344] Determine the sending mode according to the instructions of the network device; or

[0345] Determine the sending mode according to the agreement; or,

[0346] Determine the sending mode based on the current performance information of the model.

[0347] In summary, the terminal device monitors the model, obtains the channel state information (CSI) currently associated with the model, and then determines the model's current performance information based on the model's current CSI. Finally, the terminal device sends first indication information containing the model's current performance information to the network device. This allows the terminal device to accurately determine the model's current performance information based on the CSI, improving the accuracy and reliability of model performance monitoring.

[0348] The communication device 1200 may also be a network device, a device in a network device, or a device that can be used in conjunction with a network device.

[0349] The communication device 1200 is on the network device side, wherein:

[0350] The transceiver module 1201 is configured to receive first indication information, where the first indication information includes current performance information of a channel state information (CSI) feedback model.

[0351] In some possible implementations, the transceiver module 1201 is further configured to:

[0352] Sending the first output currently corresponding to the model to the terminal device; or,

[0353] A first channel state information reference signal CSI-RS is sent to the terminal device, where the first CSI-RS is the CSI-RS after beamforming corresponding to the first output.

[0354] In some possible implementations, the transceiver module 1201 is further configured to:

[0355] A first information acquisition request sent by a terminal device is received, wherein the first information is a first output of the model or associated information of the first output.

[0356] In some possible implementations,

[0357] The transceiver module 1201 is further configured to send a second indication message, wherein the second indication message includes monitoring parameters associated with the model; or

[0358] The processing module 1202 is further configured to determine monitoring parameters associated with the model based on the protocol agreement; or

[0359] The transceiver module 1201 is further configured to receive model-associated monitoring parameters sent by a terminal device.

[0360] In some possible implementations, the second indication information may be sent via one or more of the following: a radio resource control RRC message, a media access control element MAC-CE, or downlink control information DCI.

[0361] In some possible implementations, the monitoring parameters include one or more of the following:

[0362] Monitoring start time, number of monitoring results reported, monitoring time information and monitoring cycle.

[0363] In some possible implementations, the processing module 1202 is further configured to:

[0364] When the data transmission rank is greater than 1, the performance information corresponding to the model in each data transmission layer is determined, and / or the performance information of the model in all transmission layers is determined.

[0365] In some possible implementations, the performance information includes at least one of the following:

[0366] a first performance parameter, wherein the first performance parameter is determined by the terminal device according to the first input and the first information of the current model;

[0367] A comparison result of the first performance parameter and the first threshold value;

[0368] A reference parameter and a first performance parameter, wherein the reference parameter is a parameter determined by the terminal device based on the received codebook parameter information and the first input;

[0369] a comparison result between the reference parameter and the first performance parameter;

[0370] A first distribution offset, wherein the first distribution offset is an offset between the first input and the second input in the training data corresponding to the model;

[0371] A second distribution offset, wherein the second distribution offset is an offset between the first output and the second output in the training data corresponding to the model;

[0372] a fifth performance parameter, wherein the fifth performance parameter is a comparison result of the first distribution offset and the second threshold value, or a comparison result of the second distribution offset and the third threshold value;

[0373] a sixth performance parameter, wherein the sixth performance parameter is determined by the terminal device based on a current transmission parameter of the system, wherein the transmission parameter includes at least one of the following: system throughput, network block error rate (BLER), assumed BLER, acknowledgment character (ACK), and non-acknowledgment character (NACK);

[0374] a comparison result of the sixth performance parameter and the fourth threshold value;

[0375] an eighth performance parameter, wherein the eighth performance parameter is a first comparison result of K consecutive performance information of the model and the fifth threshold value;

[0376] The ninth performance parameter is the second comparison result of the K consecutive first comparison results of the model and the sixth threshold value, wherein the first comparison result is the comparison result of each consecutive performance information of the model and the fifth threshold value.

[0377] In some possible implementations, the processing module 1202 is further configured to determine a sending mode of the first indication information, where the sending mode includes at least one of the following: sending frequency, sending times, and sending content;

[0378] The transceiver module 1201 is further configured to receive first indication information based on the sending mode.

[0379] In some possible implementations,

[0380] The processing module 1202 is further configured to determine a sending mode according to the protocol; or

[0381] The transceiver module 1201 is further configured to indicate a sending mode to the terminal device; or

[0382] Receive the sending mode sent by the terminal device.

[0383] In some possible implementations, the processing module 1202 is further configured to

[0384] According to the current performance information of the CSI feedback model, any one of the following operations is performed on the CSI feedback model: activation, deactivation, update, or switching.

[0385] In summary, the network device can accurately obtain the current performance of the CSI feedback model by receiving the first indication information corresponding to the current performance information of the channel state information CSI feedback model.

[0386] Please refer to Figure 13, which is a schematic diagram of the structure of another communication device 1300 provided in an embodiment of the present disclosure. Communication device 1300 can be a terminal device or a network device, or a chip, chip system, or processor that supports the terminal device to implement the above method, or a chip, chip system, or processor that supports the network device to implement the above method. Communication device 1300 can be used to implement the method described in the above method embodiment. For details, please refer to the description of the above method embodiment.

[0387] The communication device 1300 may include one or more processors 1301. The processor 1301 may be a general-purpose processor or a dedicated processor. For example, it may be a baseband processor or a central processing unit (CPU). The baseband processor may be used to process communication protocols and communication data, while the CPU may be used to control a communication device (e.g., a network device, a baseband chip, a terminal device, a terminal device chip, a DU or CU, etc.), execute computer programs, and process computer program data.

[0388] In some possible implementations, the communication device 1300 may further include one or more memories 1302, on which a computer program 1304 may be stored. The memories 1302 execute the computer program 1304, so that the communication device 1300 performs the methods described in the above method embodiments. In some possible implementations, the memories 1302 may also store data. The communication device 1300 and the memories 1302 may be provided separately or integrated together.

[0389] In some possible implementations, the communication device 1300 may further include a transceiver 1305 and an antenna 1306. The transceiver 1305 may be referred to as a transceiver unit, a transceiver, or a transceiver circuit, and is configured to implement transceiver functions. The transceiver 1305 may include a receiver and a transmitter. The receiver may be referred to as a receiver or a receiving circuit, and is configured to implement a receiving function; the transmitter may be referred to as a transmitter or a transmitting circuit, and is configured to implement a transmitting function.

[0390] In some possible implementations, the communication device 1300 may further include one or more interface circuits 1307. The interface circuit 1307 is configured to receive code instructions and transmit them to the processor 1301. The processor 1301 executes the code instructions to enable the communication device 1300 to perform the method described in the above method embodiment.

[0391] In one implementation, processor 1301 may include a transceiver for implementing receiving and transmitting functions. For example, the transceiver may be a transceiver circuit, an interface, or an interface circuit. The transceiver circuit, interface, or interface circuit for implementing the receiving and transmitting functions may be separate or integrated. The transceiver circuit, interface, or interface circuit may be used for reading and writing code / data, or may be used for transmitting or delivering signals.

[0392] In one implementation, processor 1301 may store a computer program 1303. Computer program 1303, when executed on processor 1301, enables communication device 1300 to perform the method described in the above method embodiment. Computer program 1303 may be embedded in processor 1301, in which case processor 1301 may be implemented by hardware.

[0393] In one implementation, the communication device 800 may include a circuit that can implement the functions of sending, receiving, or communicating in the aforementioned method embodiments. The processor and transceiver described in the present disclosure can be implemented on an integrated circuit (IC), an analog IC, a radio frequency integrated circuit RFIC, a mixed signal IC, an application specific integrated circuit (ASIC), a printed circuit board (PCB), an electronic device, etc. The processor and transceiver can also be manufactured using various IC process technologies, such as complementary metal oxide semiconductor (CMOS), N-type metal oxide semiconductor (nMetal-oxide-semiconductor, NMOS), P-type metal oxide semiconductor (positive channel metal oxide semiconductor, PMOS), bipolar junction transistor (bipolar junction transistor, BJT), bipolar CMOS (BiCMOS), silicon germanium (SiGe), gallium arsenide (GaAs), etc.

[0394] The communication device described in the above embodiments may be a terminal device, but the scope of the communication device described in this disclosure is not limited thereto, and the structure of the communication device may not be limited to FIG13. The communication device may be an independent device or may be part of a larger device. For example, the communication device may be:

[0395] (1) An independent integrated circuit (IC), or chip, or chip system or subsystem;

[0396] (2) having a set of one or more ICs, which, in some possible implementations, may also include a storage component for storing data and computer programs;

[0397] (3) ASIC, such as modem;

[0398] (4) Modules that can be embedded in other devices;

[0399] (5) Receivers, terminal devices, intelligent terminal devices, cellular phones, wireless devices, handheld devices, mobile units, vehicle-mounted devices, network devices, cloud devices, artificial intelligence devices, etc.;

[0400] (6)Others, etc.

[0401] For the case where the communication device may be a chip or a chip system, please refer to FIG14 , which is a structural diagram of a chip provided in an embodiment of the present disclosure.

[0402] The chip 1400 includes a processor 1401 and an interface 1403. The number of the processor 1401 can be one or more, and the number of the interface 1403 can be multiple.

[0403] For the case where the chip is used to implement the functions of a terminal device, a network device, a BSF entity, or a UDM entity in the embodiments of the present disclosure:

[0404] Interface 1403 is used to receive code instructions and transmit them to the processor.

[0405] The processor 1401 is configured to execute code instructions to execute the model performance monitoring method as described in some of the above embodiments.

[0406] In some possible implementations, the chip 1400 further includes a memory 1402 , which is used to store necessary computer programs and data.

[0407] Those skilled in the art will also appreciate that the various illustrative logical blocks and steps listed in the embodiments of the present disclosure may be implemented by electronic hardware, computer software, or a combination of both. Whether such functionality is implemented by hardware or software depends on the specific application and the design requirements of the entire system. Those skilled in the art may use various methods to implement functionality for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of the present disclosure.

[0408] An embodiment of the present disclosure also provides a communication system, which includes the communication device in the embodiment of Figure 13 as a terminal device and the communication device in the embodiment of network equipment, or the system includes the communication device in the embodiment of Figure 14 as a terminal device and the communication device in the embodiment of network equipment.

[0409] The present disclosure also provides a readable storage medium having instructions stored thereon, which implement the functions of any of the above method embodiments when executed by a computer.

[0410] The present disclosure also provides a computer program product, which implements the functions of any of the above method embodiments when executed by a computer.

[0411] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer programs. When the computer program is loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present disclosure are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. Available media may be magnetic media (eg, floppy disks, hard disks, tapes), optical media (eg, high-density digital video discs (DVDs)), or semiconductor media (eg, solid state disks (SSDs)).

[0412] Those skilled in the art will understand that the various numerical numbers such as first and second involved in the present disclosure are only for the convenience of description and are not used to limit the scope of the embodiments of the present disclosure, and also indicate the order of precedence.

[0413] At least one in the present disclosure can also be described as one or more, and multiple can be two, three, four or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b and c can represent: a, or b, or c, or a and b, or a and c, or b and c, or a, b and c. Wherein a, b and c can be single or multiple, respectively. This disclosure is not limited. In the embodiments of the present disclosure, for a technical feature, the technical features in the technical feature are distinguished by "first", "second", "third", "A", "B", "C" and "D", and there is no order of precedence or size between the technical features described by "first", "second", "third", "A", "B", "C" and "D".

[0414] The correspondences shown in the tables of the present disclosure can be configured or predefined. The values ​​of the information in each table are merely examples and can be configured to other values, which are not limited by the present disclosure. When configuring the correspondences between information and parameters, it is not necessarily required to configure all the correspondences shown in each table. For example, in the tables of the present disclosure, the correspondences shown in certain rows may not be configured. For another example, appropriate deformation adjustments can be made based on the above tables, such as splitting, merging, etc. The names of the parameters shown in the titles of the above tables may also adopt other names that can be understood by the communication device, and the values ​​or representations of the parameters may also adopt other values ​​or representations that can be understood by the communication device. When implementing the above tables, other data structures may also be used, such as arrays, queues, containers, stacks, linear lists, pointers, linked lists, trees, graphs, structures, classes, heaps, hash tables or hash tables, etc.

[0415] The predefined in the present disclosure may be understood as defined, predefined, stored, pre-stored, pre-negotiated, pre-configured, solidified, or pre-burned.

[0416] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0417] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0418] The above are only specific embodiments of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. A method for monitoring model performance, characterized in that: Executed by a terminal device, the method includes: Monitor the model to obtain channel state information CSI currently associated with the model, wherein the CSI includes any one of the following: first information, a current first input of the model and the first information; wherein the first information is a current first output of the model or associated information of the first output, and the model is a CSI feedback model; Determining current performance information of the model according to the CSI currently associated with the model; Sending first indication information, wherein the first indication information includes current performance information of the model.

2. The method according to claim 1, characterized in that The obtaining of channel state information CSI currently associated with the model includes: In a case where the terminal device side includes a CSI recovery partial model, determining an output of the CSI recovery partial model as the first output, wherein the CSI recovery partial model on the terminal device side is the same as or different from the CSI recovery partial model on the network device side; or In a case where the terminal device side does not include a CSI recovery partial model, receiving the first output sent by the network device; or, In a case where the terminal device side does not include a CSI recovery partial model, the associated information is determined based on a received first channel state information reference signal CSI-RS, wherein the first CSI-RS is a CSI-RS after beamforming corresponding to the first output.

3. The method according to claim 2, characterized in that Also includes: When the terminal device side does not include the CSI recovery partial model, a first information acquisition request is sent.

4. The method according to claim 2, characterized in that Also includes: The first input is determined according to the received second CSI-RS.

5. The method according to claim 1, characterized in that Also includes: receiving second indication information, wherein the second indication information includes monitoring parameters associated with the model; or Determine monitoring parameters associated with the model based on protocol agreement; or, The monitoring parameters associated with the model are sent to the network device.

6. The method according to claim 5, characterized in that The sending the monitoring parameters associated with the model to the network device includes: Based on a preset event trigger, the monitoring parameters associated with the model are sent to the network device.

7. The method according to claim 5 or 6, characterized in that The monitoring parameters include one or more of the following: Monitoring start time, number of monitoring results reported, monitoring time information and monitoring cycle.

8. The method according to any one of claims 5 or 7, characterized in that The receiving the second indication information includes: The second indication information is received through one or more of the following: a radio resource control RRC message, a media access control control element MAC-CE, and downlink control information DCI.

9. The method according to claim 1, characterized in that The determining, according to the CSI currently associated with the model, current performance information of the model includes: When the data transmission rank is greater than 1, determining the CSI corresponding to the model in each data transmission layer; According to the CSI corresponding to each transmission layer, the performance information of the model in each transmission layer is determined, and / or the performance information of the model in all transmission layers is determined.

10. The method according to any one of claims 1 to 9, characterized in that: The determining, according to the CSI currently associated with the model, current performance information of the model includes: A current first performance parameter of the model is determined according to the first input and the first information.

11. The method according to any one of claims 1 to 9, characterized in that: The determining, according to the CSI currently associated with the model, current performance information of the model includes: Determining a current first performance parameter of the model according to the first input and the first information; A comparison result between the current first performance parameter and the first threshold value is determined as the current second performance parameter of the model.

12. The method according to any one of claims 1 to 9, characterized in that: The determining, according to the CSI currently associated with the model, current performance information of the model includes: Determining a current first performance parameter of the model according to the first input and the first information; Determining a reference parameter based on the received codebook parameter information and the first input; The reference parameter and the first performance parameter, or a comparison result between the reference parameter and the first performance parameter, is determined as a current third performance parameter of the model.

13. The method according to claim 12, characterized in that The determining a reference parameter based on the received codebook parameter information and the first input includes: Determine a compressed first CSI feedback payload size corresponding to the model and a second CSI feedback payload size determined by the codebook parameter information; determining an absolute value of a difference between the second CSI feedback payload size and the first CSI feedback payload size; The reference parameter is determined based on a codebook parameter combination corresponding to a minimum absolute value and the first input.

14. The method according to any one of claims 1 to 9, characterized in that: The determining, according to the CSI currently associated with the model, current performance information of the model includes: determining a first distribution offset of the first input relative to a second input in training data corresponding to the model, or a second distribution offset of the first output relative to a second output in the training data; The first distribution offset or the second distribution offset is determined as a current fourth performance parameter of the model.

15. The method according to any one of claims 1 to 9, characterized in that: The determining, according to the CSI currently associated with the model, current performance information of the model includes: Determine a first distribution offset of the first input relative to a second input in the training data corresponding to the model, and a second distribution offset of the first output relative to the second output in the training data; The comparison result of the first distribution offset and the second threshold value, or the comparison result of the second distribution offset and the third threshold value is The comparison result of the limit values ​​is determined as the current fifth performance parameter of the model.

16. The method according to any one of claims 1 to 9, characterized in that: The determining, according to the CSI currently associated with the model, current performance information of the model includes: Determine a current transmission parameter of the system according to the first information, wherein the transmission parameter includes at least one of the following: System throughput, network block error rate BLER, assumed BLER, acknowledgment character ACK, non-acknowledgment character NACK; A sixth performance parameter corresponding to the model is determined according to the current transmission parameter.

17. The method according to any one of claims 1 to 9, characterized in that: The determining, according to the CSI currently associated with the model, current performance information of the model includes: Determine the current transmission parameters of the system according to the first information, wherein the transmission parameters include at least one of the following: system throughput, network block error rate BLER, assumed BLER, confirmation character ACK, non-confirmation character NACK; A seventh performance parameter corresponding to the model is determined according to a comparison result between the current transmission parameter and the fourth threshold value.

18. The method according to any one of claims 1 to 9, characterized in that: The determining, according to the CSI currently associated with the model, current performance information of the model includes: Determine a first comparison result between the current performance information and a fifth threshold value; An eighth performance parameter corresponding to the model is determined according to K consecutive first comparison results of the model, where K is a natural number.

19. The method according to any one of claims 1 to 9, characterized in that: The determining, according to the CSI currently associated with the model, current performance information of the model includes: Determine a first comparison result between the current performance information and a fifth threshold value; Determine the ninth threshold value corresponding to the model according to the K consecutive first comparison results of the model and the second comparison result of the sixth threshold value. Performance parameters, where K is a natural number.

20. The method according to any one of claims 1 to 19, characterized in that: The monitoring of the model comprises: At least M models out of N models are monitored, where N is the number of models deployed in the terminal device and M is a value less than or equal to N.

21. The method according to any one of claims 1 to 19, characterized in that: Also includes: Determine a second CSI feedback payload size determined by the selected codebook parameter combination and a compressed first CSI feedback payload size corresponding to each model in the N models; A model whose corresponding first CSI feedback load size among the N models is less than or equal to the second CSI feedback load size, or a model whose absolute value of the difference between the model and the first CSI feedback load size is the smallest, is determined as the model to be monitored.

22. The method according to any one of claims 1 to 21, characterized in that: The sending of the first indication information includes: Determine a sending mode of the first indication information, wherein the sending mode includes at least one of the following: sending frequency, sending times, and sending content; Based on the sending mode, the first indication information is sent.

23. The method of claim 22, wherein: The determining a sending mode of the first indication information includes: determining the sending mode according to an instruction of the network device; or, Determine the sending mode according to the agreement; or, The sending mode is determined according to the current performance information of the model.

24. A method for monitoring model performance, characterized in that: The method is performed by a network device, and the method includes: First indication information is received, wherein the first indication information includes current performance information of a channel state information (CSI) feedback model.

25. The method of claim 24, wherein: Also includes: Sending a first output currently corresponding to the model to a terminal device; or, A first channel state information reference signal CSI-RS is sent to the terminal device, wherein the first CSI-RS is a CSI-RS after beamforming corresponding to the first output.

26. The method of claim 24, wherein: Also includes: A first information acquisition request sent by the terminal device is received, wherein the first information is a first output of the model or associated information of the first output.

27. The method of claim 24, wherein: Also includes: Sending second indication information, wherein the second indication information includes monitoring parameters associated with the model; or, Determine monitoring parameters associated with the model based on protocol agreement; or, Receive monitoring parameters associated with the model sent by the terminal device.

28. The method of claim 27, wherein: The sending of the second indication information includes: The second indication information is sent through one or more of the following: a radio resource control RRC message, a media access control control element MAC-CE, and downlink control information DCI.

29. The method according to claim 27 or 28, characterized in that The monitoring parameters include one or more of the following: Monitoring start time, number of monitoring results reported, monitoring time information and monitoring cycle.

30. The method according to any one of claims 24 to 29, characterized in that: Also includes: When the data transmission rank is greater than 1, the performance information corresponding to the model in each data transmission layer is determined, and / or the performance information of the model in all transmission layers is determined.

31. The method according to any one of claims 24 to 30, characterized in that: The performance information includes at least one of the following: A first performance parameter, wherein the first performance parameter is determined by the terminal device according to a current first input and first information of the model; A comparison result of the first performance parameter and a first threshold value; A reference parameter and the first performance parameter, wherein the reference parameter is a parameter determined by the terminal device based on the received codebook parameter information and the first input; a comparison result between the reference parameter and the first performance parameter; A first distribution offset, wherein the first distribution offset is an offset between the first input and a second input in the training data corresponding to the model; A second distribution offset, wherein the second distribution offset is an offset between the first output and a second output in the training data corresponding to the model; a fifth performance parameter, wherein the fifth performance parameter is a comparison result of the first distribution offset and a second threshold value, or a comparison result of the second distribution offset and a third threshold value; A sixth performance parameter, wherein the sixth performance parameter is determined by the terminal device based on a current transmission parameter of the system, wherein the transmission parameter includes at least one of the following: system throughput, network block error rate BLER, assumed BLER, confirmation character ACK, non-confirmation character NACK; A comparison result of the sixth performance parameter and the fourth threshold value; an eighth performance parameter, wherein the eighth performance parameter is a first comparison result of K consecutive performance information of the model and the fifth threshold value respectively; A ninth performance parameter, wherein the ninth performance parameter is a second comparison result of K consecutive first comparison results of the model with a sixth threshold value, wherein the first comparison result is a comparison result of each consecutive performance information of the model with a fifth threshold value.

32. The method according to any one of claims 24 to 31, characterized in that: The receiving the first indication information includes: Determine a sending mode of the first indication information, wherein the sending mode includes at least one of the following: sending frequency, sending times, and sending content; Based on the sending mode, the first indication information is received.

33. The method of claim 32, wherein: The determining a sending mode of the first indication information includes: Determine the sending mode according to the agreement; or, indicating the sending mode to the terminal device; or, Receive the sending mode sent by the terminal device.

34. The method according to any one of claims 24 to 33, characterized in that: Also includes: According to the current performance information of the CSI feedback model, any one of the following operations is performed on the CSI feedback model: activation, deactivation, updating, and switching.

35. A terminal device, characterized in that: include: A processing module, configured to monitor the model and obtain channel state information CSI currently associated with the model, wherein the CSI includes any one of the following: first information, a current first input of the model and the first information; wherein the first information is a current first output of the model or associated information of the first output, and the model is a CSI feedback model; The processing module is further configured to determine current performance information of the model according to the CSI currently associated with the model; The transceiver module is also used to send first indication information, wherein the first indication information includes current performance information of the model.

36. A network device, characterized in that: include: The transceiver module is used to receive first indication information, wherein the first indication information includes current performance information of a channel state information CSI feedback model.

37. A communication device, characterized in that: The device includes a processor and a memory, wherein a computer program is stored in the memory, and the processor executes the computer program stored in the memory so that the device performs the method as described in any one of claims 1 to 23, or the processor executes the computer program stored in the memory so that the device performs the method as described in any one of claims 24 to 34.

38. A communication system, characterized in that: Including terminal equipment and network equipment; The terminal device is used to execute the method as claimed in any one of claims 1 to 23, and the network device is used to execute the method as claimed in any one of claims 24 to 34.

39. A computer-readable storage medium storing instructions, which, when executed, enable the method according to any one of claims 1 to 23 to be implemented, or, when executed, enable the method according to any one of claims 24 to 34 to be implemented.

Images