Model performance monitoring method, device, system, and storage medium

Abstract

The present disclosure relates to a model performance monitoring method, a device, a system, and a storage medium. The method can be executed by a network device. The method comprises: inputting second information into a first AI / ML model and / or a second AI / ML model to obtain third information, the second information comprising at least one piece of CSI quantization information, and the third information comprising CSIs recovered from the at least one piece of CSI quantization information; and on the basis of the third information, determining whether the first AI / ML model satisfies performance requirements, and / or determining a reason why the performance of the first AI / ML model changes. The present invention can effectively ensure that a model deployed on a network side in a bilateral model has good performance, and ensure the communication reliability.

Description

Model performance monitoring method, device, system and storage medium TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of communication, and particularly relates to a model performance monitoring method, device, system and storage medium. BACKGROUND

[0002] With the development of Artificial Intelligence (AI) and Machine Learning (ML) technologies, it is found that AI technologies can be used to reduce feedback overhead of terminals or improve CSI feedback accuracy. For example, a bilateral AI / ML model that generates a terminal-side channel state information (CSI) partial model and a network-side CSI recovery partial model can be used to respectively implement compressed feedback and recovery of CSI.

[0003] SUMMARY

[0004] The present disclosure provides a model performance monitoring method, device, system and storage medium.

[0005] According to a first aspect of the present disclosure, a model performance monitoring method is provided, which is performed by a network device, and includes:

[0006] inputting second information into a first Artificial Intelligence (AI) / Machine Learning (ML) model and / or a second AI / ML model to obtain third information, wherein the second information includes at least one Channel State Information (CSI) quantization information, and the third information includes CSI recovered from the at least one CSI quantization information;

[0007] determining whether the first AI / ML model meets a performance requirement according to the third information, and / or determining a cause of performance change of the first AI / ML model;

[0008] wherein model parameters of the first AI / ML model are same as model parameters of the second AI / ML model, or the first AI / ML model is obtained by updating the second AI / ML model and / or a third AI / ML model by the network device;

[0009] The second AI / ML model and the third AI / ML model are AI / ML models trained by a terminal, the third AI / ML model is a first part of a bilateral model, and the first AI / ML model and the second AI / ML model are a second part of the bilateral model. According to a second aspect of the present disclosure, a model performance monitoring method is provided, which is performed by a terminal, and includes:

[0010] transmit first information, the first information being used to indicate that a network device is to input second information into a first artificial intelligence (AI) / machine learning (ML) model and / or a second AI / ML model to obtain third information, the second information including at least one channel state information (CSI) quantization information, and the third information including CSI recovered from the at least one CSI quantization information;

[0011] the third information is used by the network device to determine whether the first AI / ML model meets a performance requirement, and / or to determine a cause of a performance change of the first AI / ML model;

[0012] wherein model parameters of the first AI / ML model are same as model parameters of the second AI / ML model, or the first AI / ML model is obtained by updating the second AI / ML model and / or a third AI / ML model by the network device;

[0013] the second AI / ML model and the third AI / ML model are AI / ML models trained by the terminal, the third AI / ML model is a first part of a bilateral model, and the first AI / ML model and the second AI / ML model are a second part of the bilateral model.

[0014] According to a third aspect of embodiments of the present disclosure, a communication device is provided, including:

[0015] a processing module configured to input second information into a first AI / ML model and / or a second AI / ML model to obtain third information, the second information including at least one CSI quantization information, and the third information including CSI recovered from the at least one CSI quantization information;

[0016] the processing module is configured to determine, according to the third information, whether the first AI / ML model meets a performance requirement, and / or to determine a cause of a performance change of the first AI / ML model;

[0017] wherein model parameters of the first AI / ML model are same as model parameters of the second AI / ML model, or the first AI / ML model is obtained by updating the second AI / ML model and / or a third AI / ML model by the network device;

[0018] the second AI / ML model and the third AI / ML model are AI / ML models trained by the terminal, the third AI / ML model is a first part of a bilateral model, and the first AI / ML model and the second AI / ML model are a second part of the bilateral model.

[0019] According to a third aspect of embodiments of the present disclosure, a communication device is provided, comprising:

[0020] a transceiver configured to transmit first information, the first information being used to instruct a network device to input second information into a first artificial intelligence (AI) / machine learning (ML) model and / or a second AI / ML model to obtain third information, the second information comprising at least one channel state information (CSI) quantization information, and the third information comprising CSI recovered from the at least one CSI quantization information;

[0021] the third information being used by the network device to determine whether the first AI / ML model meets a performance requirement, and / or to determine a cause of a performance change of the first AI / ML model;

[0022] wherein model parameters of the first AI / ML model are same as model parameters of the second AI / ML model, or the first AI / ML model is obtained by updating the second AI / ML model and / or a third AI / ML model by the network device;

[0023] the second AI / ML model and the third AI / ML model are AI / ML models trained by a terminal, the third AI / ML model is a first part of a bilateral model, and the first AI / ML model and the second AI / ML model are a second part of the bilateral model.

[0024] According to a fifth aspect of embodiments of the present disclosure, a communication device is provided, comprising:

[0025] one or more processors;

[0026] wherein the communication device is configured to perform the model performance monitoring method of the first aspect or the second aspect.

[0027] According to a sixth aspect of embodiments of the present disclosure, a communication system is provided, comprising a network device and a terminal, the network device being configured to input second information into a first artificial intelligence (AI) / machine learning (ML) model and / or a second AI / ML model to obtain third information, the second information comprising at least one channel state information (CSI) quantization information, and the third information comprising CSI recovered from the at least one CSI quantization information;

[0028] the network device being configured to determine, according to the third information, whether the first AI / ML model meets a performance requirement, and / or to determine a cause of a performance change of the first AI / ML model;

[0029] The model parameters of the first AL / ML model are the same as the model parameters of the second AL / ML model, or the first AL / ML model is obtained by updating the second AL / ML model and / or a third AL / ML model by the network device.

[0030] The second AI / ML model and the third AI / ML model are AI / ML models trained by the terminal, the third AL / ML model is a first part of a bilateral model, and the first AI / ML model and the second AL / ML model are a second part of the bilateral model.

[0031] According to a seventh aspect of the embodiments of the present disclosure, a storage medium is provided, which stores instructions, when the instructions are executed on a communication device, the communication device executes the model performance monitoring method in the first aspect or the second aspect.

[0032] According to an eighth aspect of the embodiments of the present disclosure, a computer program product is provided, which includes a computer program and / or instructions, and when the computer program and / or the instructions are executed by a communication device, the model performance monitoring method in the first aspect or the second aspect is implemented.

[0033] In the above embodiments, the network device can generate corresponding third information by using the first AI / ML model and / or the second AI / ML model based on the second information, and then detect the performance of the first AI / ML model based on the generated third information, and determine the reason for the performance change of the first AI / ML model, which can effectively ensure that the model deployed on the network side in the bilateral model has better performance, and ensure the reliability of the communication process. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following describes the drawings required for the embodiments, and the following drawings are only some embodiments of the present disclosure, and do not specifically limit the protection scope of the present disclosure.

[0035] FIG. 1A is one exemplary schematic diagram of an architecture of a communication system according to an embodiment of the present disclosure.

[0036] FIG. 1B is a schematic diagram illustrating implementation of CSI compression feedback and recovery based on a bilateral AI / ML model according to an embodiment of the present disclosure.

[0037] FIG. 2 is one exemplary interactive schematic diagram of a model performance monitoring method according to an embodiment of the present disclosure.

[0038] FIG. 3A is one exemplary flow schematic diagram of a model performance monitoring method according to an embodiment of the present disclosure.

[0039] FIG. 3B is an exemplary flow diagram of a model performance monitoring method according to an embodiment of the present disclosure.

[0040] FIG. 3C is an exemplary flow diagram of a model performance monitoring method according to an embodiment of the present disclosure.

[0041] FIG. 3D is an exemplary flow diagram of a model performance monitoring method according to an embodiment of the present disclosure.

[0042] FIG. 3E is an exemplary flow diagram of a model performance monitoring method according to an embodiment of the present disclosure.

[0043] FIG. 4A is an exemplary flow diagram of a model performance monitoring method according to an embodiment of the present disclosure.

[0044] FIG. 4B is an exemplary flow diagram of a model performance monitoring method according to an embodiment of the present disclosure.

[0045] FIG. 4C is an exemplary flow diagram of a model performance monitoring method according to an embodiment of the present disclosure.

[0046] FIG. 4D is an exemplary flow diagram of a model performance monitoring method according to an embodiment of the present disclosure.

[0047] FIG. 4E is an exemplary flow diagram of a model performance monitoring method according to an embodiment of the present disclosure.

[0048] FIG. 5 is an exemplary flow diagram of a model performance monitoring method according to an embodiment of the present disclosure.

[0049] FIG. 6 is an exemplary flow diagram of a model performance monitoring method according to an embodiment of the present disclosure.

[0050] FIG. 7A is an exemplary structural diagram of a terminal according to an embodiment of the present disclosure.

[0051] FIG. 7B is an exemplary structural diagram of a network device according to an embodiment of the present disclosure.

[0052] FIG. 8A is an exemplary structural diagram of a communication device according to an embodiment of the present disclosure.

[0053] FIG. 8B is an exemplary structural diagram of a communication device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0054] The present disclosure provides a model performance monitoring method, device, system and storage medium.

[0055] In a first aspect, the embodiments of the present disclosure provide a model performance monitoring method, executed by a network device, comprising:

[0056] determining that the first information sent by the terminal is received, inputting second information into the first AI / ML model and / or the second AI / ML model to obtain third information, the second information comprising at least one CSI quantization information, and the third information comprising CSI recovered from the at least one CSI quantization information;

[0057] determining, according to the third information, whether the first AI / ML model meets the performance requirement, and / or the cause of the change in the performance of the first AI / ML model;

[0058] wherein the model parameters of the first AL / ML model are the same as the model parameters of the second AL / ML model, or the first AL / ML model is obtained by updating the second AL / ML model and / or a third AL / ML model by the network device;

[0059] the second AI / ML model and the third AI / ML model are AI / ML models trained by the terminal, the third AL / ML model is a first part of a bilateral model, and the first AI / ML model and the second AL / ML model are a second part of the bilateral model.

[0060] In the above embodiments, the network device can generate corresponding third information by using the first AI / ML model and / or the second AI / ML model based on the second information, and then detect the performance of the first AI / ML model based on the third information generated by it, and determine the cause of the change in the performance of the first AI / ML model, which can effectively ensure that the model deployed on the network side in the bilateral model has better performance and ensures the reliability of the communication process.

[0061] In combination with some embodiments of the first aspect, in some embodiments, the second information comprises one or more of the following:

[0062] first quantization information, the first quantization information being obtained by inputting first CSI into the third AI / ML model by the terminal or the network device, the first CSI being a predefined CSI;

[0063] second quantization information, the second quantization information being obtained by inputting second CSI into the third AI / ML model by the terminal, the second CSI being measured by the terminal;

[0064] third quantization information, the third quantization information being quantization information corresponding to third CSI, the third CSI being a CSI used by the first AI / ML model obtained by updating by the network device.

[0065] In the above embodiments, the network device can recover the corresponding CSI based on one or more of the above-mentioned quantization information, and then reliably monitor the performance of the first AI / ML model.

[0066] In some embodiments of the first aspect, the method comprises:

[0067] receiving the first information sent by the terminal;

[0068] determining the second information based on the first information.

[0069] In some embodiments of the first aspect, the first information comprises one or more of:

[0070] the first CSI;

[0071] the first quantization information;

[0072] the second CSI;

[0073] the second quantization information.

[0074] In the above embodiments, the terminal can indicate the network device to determine the second information input into the first AI / ML model and / or the second AI / ML model through the first information, and can also carry part of the second information through the first information, or carry information related to generating the second information, effectively ensuring the flexibility of the network device monitoring the first AI / ML model.

[0075] In some embodiments of the first aspect, the method comprises:

[0076] sending a first signal to the terminal, the first signal being used by the terminal to measure the second CSI.

[0077] In the above embodiments, by sending the first signal to the terminal by the network device, the accuracy of the second CSI measured by the terminal can be effectively ensured.

[0078] In some embodiments of the first aspect, the inputting of the second information into the first AI / ML model and / or the second AI / ML model to obtain the third information comprises one or more of:

[0079] inputting the first quantization information into the first AI / ML model to obtain the fourth CSI output by the first AI / ML model;

[0080] inputting the second quantization information into the first AI / ML model to obtain the fifth CSI output by the first AI / ML model;

[0081] inputting the third quantization information into the first AI / ML model to obtain sixth CSI output by the first AI / ML model;

[0082] inputting the first quantization information into the second AI / ML model to obtain seventh CSI output by the second AI / ML model;

[0083] inputting the second quantization information into the second AI / ML model to obtain eighth CSI output by the second AI / ML model;

[0084] inputting the third quantization information into the second AI / ML model to obtain ninth CSI output by the second AI / ML model.

[0085] In the above embodiments, the network device can input different information into different AI / ML models, and then obtain different CSI, and then perform multi-dimensional performance monitoring on the first AI / ML model.

[0086] In some embodiments of the first aspect, the determining whether the first AI / ML model meets the performance requirement according to the third information comprises one or more of the following:

[0087] determining whether the first AI / ML model meets a first performance requirement according to the first CSI and the fourth CSI, or the second CSI and the fifth CSI, or the third CSI and the sixth CSI;

[0088] determining whether the first AI / ML model meets a second performance requirement according to the first CSI, the second CSI, the fourth CSI, and the fifth CSI;

[0089] determining whether the first AI / ML model meets a third performance requirement according to the first CSI, the fourth CSI, and the seventh CSI;

[0090] determining whether the first AI / ML model meets a third performance requirement according to the second CSI, the fifth CSI, and the eighth CSI;

[0091] determining whether the first AI / ML model meets a third performance requirement according to the third CSI, the sixth CSI, and the ninth CSI;

[0092] determining whether the first AI / ML model meets a fourth performance requirement according to the second CSI, the third CSI, the fifth CSI, and the sixth CSI.

[0093] In the above embodiments, whether the first AI / ML model meets the performance requirements can be determined based on the different CSIs, thereby effectively ensuring the reliability of the determination.

[0094] In some embodiments of the first aspect, in some embodiments, the determining, according to the third information, the cause of the performance change of the first AI / ML model comprises one or more of the following:

[0095] The first AL / ML model does not meet the first performance requirement, and it is determined that the performance change of the first AL / ML model is caused by model deployment;

[0096] The first AL / ML model does not meet the second performance requirement or the fourth performance requirement, and it is determined that the performance change of the first AL / ML model is caused by actual channel information change;

[0097] The second AL / ML model does not meet the third performance requirement, and it is determined that the performance change of the second AL / ML model is caused by model update.

[0098] In the above embodiments, the cause of the performance change of the first AI / ML model can be accurately determined according to whether the first AI / ML model meets the performance requirements.

[0099] In some embodiments of the first aspect, in some embodiments, the method comprises:

[0100] sending fourth information to the terminal, the fourth information comprising the third information and / or the third CSI, the fourth information being used by the terminal to determine whether the first AI / ML model meets the performance requirements, and / or the cause of the performance change of the first AI / ML model.

[0101] In the above embodiments, the network device can send the CSI recovered and / or the third CSI to the terminal by sending the fourth information, so that the terminal can monitor the performance of the first AI / ML model based on the recovered CSI and / or the third CSI.

[0102] In some embodiments of the first aspect, in some embodiments, the method comprises:

[0103] receiving fifth information sent by the terminal, the fifth information comprising one or more of the second AI / ML model or model parameters of the one or more second AI / ML models, and / or one or more of the third AL / ML model or model parameters of the one or more third AI / ML models.

[0104] In the above embodiments, the terminal can indicate the second AI / ML model and / or the third AI / ML model trained by the terminal to the network device by sending the fifth information, so that the network device can use the model trained by the terminal to infer the CSI-related information.

[0105] In combination with some embodiments of the first aspect, in some embodiments, the network device is deployed with a plurality of first AI / ML models, and the method comprises:

[0106] sending sixth information to the terminal, the sixth information being used to indicate whether the model parameters of each of the plurality of first AL / ML models are obtained by updating.

[0107] In the above embodiments, the network device can send the sixth information to enable the terminal to know which of the plurality of models deployed by the terminal are obtained by updating and which are not obtained by updating, so that the performance of each model can be monitored more flexibly.

[0108] In a second aspect, the embodiments of the present disclosure provide a model performance monitoring method, which is characterized by being performed by a terminal, and the method comprises:

[0109] sending first information, the first information being used to indicate that a network device inputs second information into a first artificial intelligence (AI) / machine learning (ML) model and / or a second AI / ML model to obtain third information, the second information comprising at least one channel state information (CSI) quantization information, and the third information comprising CSI recovered from the at least one CSI quantization information;

[0110] the third information being used by the network device to determine whether the first AI / ML model meets a performance requirement and / or to determine a cause of a performance change of the first AI / ML model;

[0111] wherein the model parameters of the first AL / ML model are the same as the model parameters of the second AL / ML model, or the first AL / ML model is obtained by the network device updating the second AL / ML model and / or a third AL / ML model;

[0112] the second AI / ML model and the third AI / ML model being AI / ML models trained by the terminal, the third AL / ML model being a first part of a bilateral model, and the first AI / ML model and the second AL / ML model being a second part of the bilateral model.

[0113] In combination with some embodiments of the second aspect, in some embodiments, the second information comprises one or more of the following:

[0114] a first quantization information, the first quantization information being obtained by the terminal or the network device by inputting a first CSI into the third AI / ML model, the first CSI being a predefined CSI;

[0115] a second quantization information, the second quantization information being obtained by the terminal by inputting a second CSI into the third AI / ML model, the second CSI being measured by the terminal;

[0116] a third quantization information, the third quantization information being quantization information corresponding to a third CSI, the third CSI being a CSI used by the network device to update the first AI / ML model.

[0117] With some embodiments of the second aspect, in some embodiments, the second information is determined by the network device based on the first information.

[0118] With some embodiments of the second aspect, in some embodiments, the first information comprises one or more of:

[0119] the first CSI;

[0120] the first quantization information;

[0121] the second CSI;

[0122] the second quantization information.

[0123] With some embodiments of the second aspect, in some embodiments, the method comprises:

[0124] receiving a first signal sent by the network device;

[0125] measuring the first signal to obtain the second CSI.

[0126] With some embodiments of the second aspect, in some embodiments, the third information comprises one or more of:

[0127] a fourth CSI, the fourth CSI being obtained by the network device by inputting the first quantization information into the first AI / ML model;

[0128] a fifth CSI, the fifth CSI being obtained by the network device by inputting the second quantization information into the first AI / ML model;

[0129] a sixth CSI, the sixth CSI being obtained by the network device by inputting the third quantization information into the first AI / ML model;

[0130] a seventh CSI, the seventh CSI being obtained by the network device by inputting the first quantized information into the second AI / ML model;

[0131] an eighth CSI, the eighth CSI being obtained by the network device by inputting the second quantized information into the second AI / ML model;

[0132] a ninth CSI, the ninth CSI being obtained by the network device by inputting the third quantized information into the second AI / ML model.

[0133] In some embodiments in combination with the second aspect, the method comprises:

[0134] receiving fourth information sent by the network device, the fourth information comprising the third information and / or the third CSI;

[0135] determining, according to the fourth information, whether the first AI / ML model meets the performance requirement, and / or the reason for the change in performance of the first AI / ML model.

[0136] In some embodiments in combination with the second aspect, determining, according to the fourth information, whether the first AI / ML model meets the performance requirement comprises one or more of:

[0137] determining, according to the first CSI and the fourth CSI, or the second CSI and the fifth CSI, or the third CSI and the sixth CSI, whether the first AI / ML model meets a first performance requirement;

[0138] determining, according to the first CSI, the second CSI, the fourth CSI and the fifth CSI, whether the first AI / ML model meets a second performance requirement;

[0139] determining, according to the first CSI, the fourth CSI and the seventh CSI, whether the first AI / ML model meets a third performance requirement;

[0140] determining, according to the second CSI, the fifth CSI and the eighth CSI, whether the first AI / ML model meets a third performance requirement;

[0141] determining, according to the third CSI, the sixth CSI and the ninth CSI, whether the first AI / ML model meets a third performance requirement;

[0142] determining, according to the second CSI, the third CSI, the fifth CSI and the sixth CSI, whether the first AI / ML model meets a fourth performance requirement.

[0143] In some embodiments of the second aspect, in some embodiments, the determining, according to the fourth information, the cause of the performance change of the first AI / ML model comprises one or more of the following:

[0144] determining that the first AI / ML model does not meet the first performance requirement, and determining that the performance change of the first AI / ML model is caused by model deployment;

[0145] determining that the first AI / ML model does not meet the second performance requirement or the fourth performance requirement, and determining that the performance change of the first AI / ML model is caused by actual channel information change;

[0146] determining that the second AI / ML model does not meet the third performance requirement, and determining that the performance change of the second AI / ML model is caused by model update.

[0147] In some embodiments of the second aspect, in some embodiments, the method comprises:

[0148] sending fifth information to the network device, the fifth information comprising one or more of the second AI / ML model or model parameters of the one or more second AI / ML model, and / or one or more of the third AI / ML model or model parameters of the one or more third AI / ML model.

[0149] In some embodiments of the second aspect, in some embodiments, the network device is deployed with a plurality of the first AI / ML model, and the method comprises:

[0150] receiving sixth information sent by the network device, the sixth information being used to indicate whether model parameters of each of the plurality of the first AI / ML model are obtained by update.

[0151] In a third aspect, the embodiments of the present disclosure provide a communication device, comprising:

[0152] a processing module configured to input second information into the first AI / ML model and / or the second AI / ML model to obtain third information, the second information comprising at least one CSI quantization information, and the third information comprising CSI recovered from the at least one CSI quantization information;

[0153] the processing module is configured to determine whether the first AI / ML model meets a performance requirement according to the third information, and / or determine a cause of a performance change of the first AI / ML model;

[0154] The model parameters of the first AL / ML model are the same as the model parameters of the second AL / ML model, or the first AL / ML model is obtained by updating the second AL / ML model and / or a third AL / ML model by the network device.

[0155] The second AI / ML model and the third AI / ML model are AI / ML models trained by a terminal, the third AL / ML model is a first part of a bilateral model, and the first AI / ML model and the second AL / ML model are a second part of the bilateral model.

[0156] In a fourth aspect, an embodiment of the present disclosure provides a communication device, characterized in that comprising:

[0157] The transceiver is configured to send first information, the first information being used to instruct the network device to input second information into a first AI / ML model and / or a second AI / ML model to obtain third information, the second information including at least one CSI quantization information, and the third information including CSI recovered from the at least one CSI quantization information;

[0158] The third information is used for the network device to determine whether the first AI / ML model meets a performance requirement and / or to determine a reason for performance change of the first AI / ML model;

[0159] The model parameters of the first AL / ML model are the same as the model parameters of the second AL / ML model, or the first AL / ML model is obtained by updating the second AL / ML model and / or a third AL / ML model by the network device.

[0160] The second AI / ML model and the third AI / ML model are AI / ML models trained by a terminal, the third AL / ML model is a first part of a bilateral model, and the first AI / ML model and the second AL / ML model are a second part of the bilateral model.

[0161] In a fifth aspect, an embodiment of the present disclosure provides a communication device, characterized in that comprising:

[0162] One or more processors;

[0163] The communication device is configured to perform the communication method of the first aspect or the second aspect.

[0164] In a sixth aspect, the embodiments of the present disclosure provide a communication system, which comprises a terminal and a network device. The terminal is configured to perform the method described in the optional implementation manner of the second aspect. The network device is configured to perform the method described in the optional implementation manner of the first aspect.

[0165] In some embodiments, the network device is configured to input second information into a first artificial intelligence (AI) / machine learning (ML) model and / or a second AI / ML model to obtain third information, the second information comprising at least one channel state information (CSI) quantization information, and the third information comprising CSI recovered from the at least one CSI quantization information.

[0166] The network device is configured to determine whether the first AI / ML model meets a performance requirement and / or determine a cause of a performance change of the first AI / ML model according to the third information.

[0167] The model parameters of the first AL / ML model are the same as the model parameters of the second AL / ML model, or the first AL / ML model is obtained by updating the second AL / ML model and / or a third AL / ML model by the network device.

[0168] The second AI / ML model and the third AI / ML model are AI / ML models trained by the terminal. The third AL / ML model is a first part of a bilateral model, and the first AI / ML model and the second AL / ML model are a second part of the bilateral model.

[0169] In a seventh aspect, the embodiments of the present disclosure provide a storage medium, which stores instructions. When the instructions are run on a communication device, the communication device performs the method described in the optional implementation manner of the first aspect and the second aspect.

[0170] In an eighth aspect, the embodiments of the present disclosure provide a computer program product, which comprises a computer program and / or instructions. When the computer program and / or instructions are executed by a communication device, the communication device performs the method described in the optional implementation manner of the first aspect and the second aspect.

[0171] In a ninth aspect, the embodiments of the present disclosure provide a computer program, which, when run on a computer, causes the computer to perform the method described in the optional implementation manner of the first aspect and the second aspect.

[0172] In a tenth aspect, the embodiments of the present disclosure provide a chip or a chip system. The chip or the chip system comprises processing circuitry configured to perform the method described in the optional implementation manner of the first aspect and the second aspect.

[0173] It can be understood that the terminal, network device, communication system, storage medium, program product, computer program, chip or chip system described above are used to execute the method proposed in the embodiments of the present disclosure. Therefore, the beneficial effects achieved thereby can refer to the beneficial effects in the corresponding method, which will not be described here.

[0174] The embodiments of the present disclosure propose a model performance monitoring method, a communication device, a communication system and a storage medium. In some embodiments, the terms of the communication method, the information processing method and the model performance monitoring method can be replaced with each other, the terms of the communication device, the information processing device and the model performance monitoring device can be replaced with each other, and the terms of the information processing system and the communication system can be replaced with each other.

[0175] The embodiments of the present disclosure are not exhaustive, but only illustrate some embodiments, and are not specific limitations on the protection scope of the present disclosure. In the case of no contradiction, each step in an embodiment can be implemented as an independent embodiment, and the steps can be combined arbitrarily, for example, the scheme after removing some steps in an embodiment can also be implemented as an independent embodiment, and the order of the steps in an embodiment can be exchanged arbitrarily, in addition, the optional implementation manners in an embodiment can be combined arbitrarily; in addition, the embodiments can be combined arbitrarily, for example, the steps of different embodiments or part or all of the steps of different embodiments can be combined arbitrarily, an embodiment can be combined with the optional implementation manners of other embodiments.

[0176] In each embodiment of the present disclosure, the terms and / or descriptions between the embodiments are consistent if there is no special description and logical conflict, and can be referred to each other, and the technical features in different embodiments can be combined to form a new embodiment according to their inherent logical relationship.

[0177] The terms used in the embodiments of the present disclosure are only for the purpose of describing specific embodiments, and not as a limitation on the present disclosure.

[0178] In the embodiments of the present disclosure, unless otherwise specified, the elements expressed in singular form, such as "one", "a", "the", "above", "said", "preceding", "this" and the like, can represent "one and only one", or "one or more", "at least one" and the like. For example, in the case of using articles such as "a", "an", "the" and the like in English, the noun after the article can be understood as singular expression, or can be understood as plural expression.

[0179] In the embodiments of the present disclosure, "a plurality of" means two or more.

[0180] In some embodiments, the terms "at least one of," "one or more of," "a plurality of," "multiple," and the like can be used interchangeably.

[0181] In some embodiments, the recitations "at least one of A, B," "A and / or B," "in one case A, in another case B," "in response to a case A, in response to a case B," and the like can include the following technical solutions according to the case: in some embodiments A (A is executed regardless of B); in some embodiments B (B is executed regardless of A); in some embodiments, A and B are selectively executed (A and B are selectively executed); in some embodiments, A and B (A and B are executed). When there are more branches such as A, B, C, and the like, the above is similar.

[0182] In some embodiments, the recitations "A or B" and the like can include the following technical solutions according to the case: in some embodiments A (A is executed regardless of B); in some embodiments B (B is executed regardless of A); in some embodiments, A and B are selectively executed (A and B are selectively executed). When there are more branches such as A, B, C, and the like, the above is similar.

[0183] In the embodiments of the present disclosure, the prefix words "first", "second", and the like are only used to distinguish different description objects, and do not constitute a limitation on the position, order, priority, quantity, or content of the description objects. The description of the description objects should refer to the description in the context of the claims or embodiments, and should not constitute an additional limitation because of the use of the prefix words. For example, the description object is "field", and the ordinal words before "field" in "first field" and "second field" do not limit the position or order between "fields", and "first" and "second" do not limit whether the "fields" modified thereby are in the same message or not, nor do they limit the order of "first field" and "second field". For another example, the description object is "level", and the ordinal words before "level" in "first level" and "second level" do not limit the priority between "levels". For another example, the quantity of the description object is not limited by the ordinal words, and can be one or more. For example, "first device", wherein the quantity of "device" can be one or more. In addition, the objects modified by different prefix words can be the same or different, for example, the description object is "device", and "first device" and "second device" can be the same device or different devices, and their types can be the same or different; for another example, the description object is "information", and "first information" and "second information" can be the same information or different information, and their contents can be the same or different.

[0184] In some embodiments, “comprising A”, “including A”, “for indicating A”, “carrying A” can be interpreted as directly carrying A, or can be interpreted as indirectly indicating A.

[0185] In some embodiments, the terms “time / frequency”, “time / frequency domain” and the like refer to time domain and / or frequency domain.

[0186] In some embodiments, the terms “in response to”, “in response to determining”, “in the case of”, “when”, “when”, “if”, “if” and the like can be replaced with each other.

[0187] In some embodiments, the terms “greater than”, “greater than or equal to”, “not less than”, “more than”, “more than or equal to”, “not less than”, “higher than”, “higher than or equal to”, “not lower than”, “above” and the like can be replaced with each other, and the terms “less than”, “less than or equal to”, “not greater than”, “less than”, “less than or equal to”, “not more than”, “lower than”, “lower than or equal to”, “not higher than”, “below” and the like can be replaced with each other.

[0188] In some embodiments, the apparatus and the like can be interpreted as physical or virtual, and the name thereof is not limited to the name recorded in the embodiments. The terms “apparatus”, “equipment”, “device”, “circuit”, “network element”, “node”, “function”, “unit”, “section”, “system”, “network”, “chip”, “chip system”, “entity”, “subject” and the like can be replaced with each other.

[0189] In some embodiments, “network” can be interpreted as an apparatus (for example, access network device, core network device and the like) contained in the network.

[0190] In some embodiments, the terms “access network device (AN device),” “radio access network device (RAN device),” “base station (BS),” “radio base station,” “fixed station,” “node,” “access point,” “transmission point (TP),” “reception point (RP),” “transmission / reception point (TRP),” “panel,” “antenna panel,” “antenna array,” “cell,” “macro cell,” “small cell,” “femto cell,” “pico cell,” “sector,” “cell group,” “serving cell,” “carrier,” “component carrier,” “bandwidth part (BWP),” and the like can be used interchangeably.

[0191] In some embodiments, the terms "terminal," "terminal device," "user equipment (UE)," "user terminal," "mobile station (MS)," "mobile terminal (MT)," "subscriber station," "mobile unit," "subscriber unit," "wireless unit," "remote unit," "mobile device," "wireless device," "wireless communication device," "remote device," "mobile subscriber station," "access terminal," "mobile terminal," "wireless terminal," "remote terminal," "handset," "user agent," "mobile client," "client," and so on can be replaced with each other.

[0192] In some embodiments, the access network device, the core network device, or the network device can be replaced with a terminal. For example, the embodiments of the present disclosure can also be applied to a structure in which communication between the access network device, the core network device, or the network device and the terminal is replaced with communication between a plurality of terminals (e.g., device-to-device (D2D), vehicle-to-everything (V2X), etc.). In this case, the terminal can also be configured to have all or part of the functions of the access network device. In addition, the terms "uplink," "downlink," and the like can also be replaced with terms corresponding to the inter-terminal communication (e.g., "side"). For example, the uplink channel, the downlink channel, and the like can be replaced with the side channel, and the uplink, the downlink, and the like can be replaced with the sidelink.

[0193] In some embodiments, the terminal can be replaced with the access network device, the core network device, or the network device. In this case, the access network device, the core network device, or the network device can also be configured to have all or part of the functions of the terminal.

[0194] In some embodiments, the data, information, etc. can be obtained in compliance with the laws and regulations of the country where the location is situated.

[0195] In some embodiments, the data, information, etc. can be obtained after obtaining the consent of the user.

[0196] In addition, each element, each row, or each column in the table of the embodiments of the present disclosure can be implemented as an independent embodiment, and any combination of any element, any row, or any column can also be implemented as an independent embodiment.

[0197] FIG. 1A is a schematic diagram of an architecture of a communication system according to an embodiment of the present disclosure. As shown in FIG. 1A, the communication system 100 includes a terminal 101 and a network device 102. In some embodiments, the network device 102 includes at least one of an access network device and a core network device.

[0198] In some embodiments, the terminal 101 includes at least one of a mobile phone, a wearable device, an Internet of Things device, a car with communication function, a smart car, a Pad, a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal device in industrial control, a wireless terminal device in self-driving, a wireless terminal device in remote medical surgery, a wireless terminal device in smart grid, a wireless terminal device in transportation safety, a wireless terminal device in smart city, a wireless terminal device in smart home, etc., but is not limited thereto.

[0199] In some embodiments, the access network device is, for example, a node or device that accesses a terminal to a wireless network, and the access network device can include at least one of an evolved NodeB (eNB) in a 5G communication system, a next generation eNB (ng-eNB), a next generation NodeB (gNB), a node B (NB), a home node B (HNB), a home evolved node B (HeNB), a wireless backhaul device, a radio network controller (RNC), a base station controller (BSC), a base transceiver station (BTS), a base band unit (BBU), a mobile switching center, a base station in a 6G communication system, an open base station (Open RAN), a cloud base station (Cloud RAN), a base station in other communication systems, an access node in a Wi-Fi system, but is not limited thereto.

[0200] In some embodiments, the technical solutions of the present disclosure can be applied to an Open RAN architecture, at this time, the interfaces between or within the access network devices involved in the embodiments of the present disclosure can become internal interfaces of the Open RAN, and the processes and information interactions between these internal interfaces can be realized by software or programs.

[0201] In some embodiments, the access network device can be composed of a central unit (CU) and a distributed unit (DU), wherein the CU can also be referred to as a control unit (control unit). The CU-DU structure can split the protocol layers of the access network device, and part of the functions of the protocol layers are controlled by the CU, and the remaining part or all of the functions of the protocol layers are distributed in the DU and controlled by the CU, but the present disclosure is not limited thereto.

[0202] In some embodiments, the core network device can be one device including a first network element, a second network element, etc., or can be multiple devices or device groups, respectively including all or part of the first network element, the second network element, etc. The network element can be virtual or physical. The core network includes, for example, at least one of an evolved packet core (EPC), a 5G core network (5GCN), and a next generation core (NGC).

[0203] It can be understood that the communication system described in the embodiments of the present disclosure is for more clearly illustrating the technical solutions of the embodiments of the present disclosure, and does not constitute a limitation on the technical solutions proposed by the embodiments of the present disclosure. Those skilled in the art can know that, with the evolution of system architecture and the appearance of new business scenarios, the technical solutions proposed by the embodiments of the present disclosure are also applicable to similar technical problems.

[0204] The following embodiments of the present disclosure can be applied to the communication system 100 shown in FIG. 1A or part of the subject, but are not limited thereto. The subjects shown in FIG. 1A are exemplary, and the communication system can include all or part of the subjects in FIG. 1A, or other subjects other than FIG. 1A. The number and form of each subject is arbitrary, each subject can be physical or virtual, the connection relationship between each subject is exemplary, each subject can not be connected or can be connected, the connection can be in any way, can be direct connection or indirect connection, can be wired connection or wireless connection.

[0205] Embodiments of the present disclosure can be applied to Long Term Evolution (LTE), LTE-Advanced (LTE-A), LTE-Beyond (LTE-B), SUPER 3G, IMT-Advanced, 4th generation mobile communication system (4G), 5th generation mobile communication system (5G), 5G new radio (NR), Future Radio Access (FRA), New-Radio Access Technology (RAT), New Radio (NR), New radio access (NX), Future generation radio access (FX), Global System for Mobile communications (GSM (registered trademark)), CDMA2000, Ultra Mobile Broadband (UMB), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.20, Ultra-WideBand (UWB), Bluetooth (Bluetooth (registered trademark)), Public Land Mobile Network (PLMN) network, Device-to-Device (D2D) system, Machine to Machine (M2M) system, Internet of Things (IoT) system, Vehicle-to-Everything (V2X), system using other communication methods, next-generation system expanded based thereon, and the like. Further, a plurality of systems can be applied in combination (for example, combination of LTE or LTE-A and 5G, and the like).

[0206] FIG. IB is a schematic diagram illustrating implementation of CSI compression feedback and recovery based on a bilateral AI / ML model according to an embodiment of the present disclosure. As shown in FIG. IB, the UE side compresses the downlink channel information H through a CSI generation partial model (for example, referred to as an encoder) and sends it to the device gNB of the network (NW) side in the form of a binary bit stream after quantization, and the gNB side recovers the recovered downlink channel H' similar to the original downlink information through a CSI recovery partial model (defined as a decoder).

[0207] In some embodiments, the CSI generation partial model can be sent to the UE by the gNB, or can be further trained by the UE based on the model sent by the gNB. Optionally, the second AI / ML model involved in some embodiments described below can be the CSI generation partial model, which can be the first part of the bilateral model. Optionally, the third AI / ML model involved in the embodiments described below can be the CSI recovery partial model, which can be the second part of the bilateral model.

[0208] In some implementations, since the encoder and the decoder described above are respectively deployed on the UE side and the NW side, the training of the model needs to be completed by bilateral cooperation, or the model trained in a single end sends part of the model to the opposite end for model inference, or re-trains a new model based on the received transfer model. If the UE and each NW vendor independently perform bilateral model training, the complexity of bilateral model training will be greatly increased.

[0209] In order to alleviate the complexity of bilateral model training cooperation for CSI compression feedback, the following optional implementations for bilateral AI / ML training cooperation are proposed to obtain a bilateral AI / ML model:

[0210] Option 1, standardize the model structure and model parameters of the reference model.

[0211] Option 2, standardize the data set.

[0212] Option 3, standardize the model structure, and exchange the model parameters through the NW side and the UE side.

[0213] Option 4, standardize the format of the data, and exchange the data set through the NW side and the UE side.

[0214] Option 5, standardize the model format, and exchange the reference model through the NW side and the UE side.

[0215] In some embodiments, there are the following optional schemes for different uses and / or transferred contents:

[0216] Optionally, for the above optional solution three and optional solution five, the UE side can retrain at least one new encoder for the received encoder parameters or models. Alternatively, the UE side retrain a new encoder for the received decoder parameters or models. Alternatively, the UE side retrain a new encoder for the received encoder and decoder parameters or models.

[0217] Optionally, for the above optional solution three, the UE can directly use the encoder parameters received by the NW through the air interface for model inference.

[0218] Optionally, for the above optional solution five, the UE can directly use the encoder parameters received by the NW through the air interface for model inference.

[0219] Optionally, for the above optional solution four, the NW side sends the target CSI and CSI feedback information to the UE side. Alternatively, the NW side sends the recovered target CSI and CSI feedback information to the UE side. Optionally, the NW side sends the target CSI, CSI feedback information and recovered target CSI to the UE side.

[0220] Optionally, for the above optional solution one, the UE side and the NW side can respectively train the updated encoder of the UE side and the decoder model of the NW side according to the collected data set (different from the data set used for training the standardized model).

[0221] In some embodiments, for the above optional solutions, because of the transmission error of the model or parameters, the retrained decoder model is deteriorated, and other factors, it is possible that the decoder model deployed in the NW cannot meet the performance requirements, so it is necessary to study the corresponding performance monitoring mechanism to identify the cause of performance deterioration, so as to ensure that the deployed bilateral model obtains better performance. Due to the change of the channel environment where the UE is located or the difference in network parameter configuration, the channel data of the UE side and the current decoder model do not match, which will also cause the model performance to deteriorate. If only the aforementioned NW side or UE side monitoring method is used, the monitoring result may not be reliable. How to reduce the signaling transmission overhead or ensure the reliability of the bilateral model monitoring result is a problem to be solved.

[0222] In some embodiments, for the model performance monitoring of the NW side, it can be implemented based on the target CSI reported by the UE side, which can be indicated by the eType II codebook or a higher-precision eType II codebook.

[0223] In some embodiments, the decoder deployed on the NW side can be the same as or different from the decoder deployed on the UE side, or be a reference model provided by the UE or a proxy model developed on the UE side.

[0224] However, the above-mentioned embodiments for monitoring the performance of the NW side or UE monitoring model may not obtain accurate monitoring results, because this method cannot verify whether the performance degradation is caused by changes in channel data or by the degradation of the updated partial model. In addition, the transmission of target CSI or recovered CSI by the UE and the NW side will consume uplink or downlink transmission resources.

[0225] To this end, in some embodiments, the NW can input the pre-defined compressed CSI, or the compressed CSI sent by the UE, or the compressed CSI corresponding to the target CSI for training a new model into the decoder to obtain information for monitoring the performance of the model deployed by the NW.

[0226] FIG. 2 is an interaction diagram of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG. 2, the present disclosure relates to a model performance monitoring method, and the above-mentioned method comprises:

[0227] In step S2101, the terminal sends fifth information to the network device.

[0228] In some embodiments, the fifth information includes the second AI / ML model or the model parameters of the second AI / ML model, and / or the third AI / ML model or the model parameters of the third AI / ML model.

[0229] In the present disclosure, the second AI / ML model and the third AI / ML model can be models trained by the terminal or models pre-configured in the terminal. That is, for the terminal, at least the model structure and the model parameters of the second AI / ML model and the third AI / ML model are known.

[0230] Optionally, the second AI / ML model can be the first part of the bilateral model. Optionally, the second AI / ML model can be the CSI recovery partial model of the bilateral model, i.e., the decoder part. Optionally, the second AI / ML model can be based on an AI algorithm to recover the quantized compressed CSI quantization information to obtain the corresponding CSI.

[0231] Optionally, the third AI / ML model can be the second part of the bilateral model. Optionally, the third AI / ML model can be the CSI compression partial model of the bilateral model, i.e., the encoder part. Optionally, the third AI / ML model can be based on an AI algorithm to compress and quantize the CSI to obtain the corresponding CSI quantization information.

[0232] It can be understood that the terminal can convert the first part and / or the second part of the bilateral model into a standardized model format and send it to the terminal through the fifth information, or standardize the model structure and send the model parameters to the network device through the fifth information. For example, the network device can pre-configure the structure of the model, and the network device can determine the model structure corresponding to the model parameters sent by the terminal, and then determine the corresponding model.

[0233] In some embodiments, after receiving the fifth information, the network device can directly use the second AI / ML model as a decoder for inference, or update the second AI / ML model and / or the third AI / ML model to obtain an updated decoder for inference. For example, the network device can use the second AI / ML model as the first AI / ML model used by it, or train the first AI / ML model according to the second AI / ML model and / or the third AI / ML model.

[0234] In some embodiments, the third AI / ML model can have the ability to quantize data, or the third AI / ML model can use a pre-configured quantization module to quantize the data output by it.

[0235] It is worth noting that the first AI / ML model involved in the present disclosure can refer to the model used by the network device. The second AI / ML model can be the model sent by the terminal to the network device, i.e. the model used by the terminal or known by the terminal. That is, the terminal can indicate the decoder or the model parameters of the decoder used by it to the network device.

[0236] In addition, updating the second AI / ML model and / or the third AI / ML model can refer to the process of training an updated AI / ML model based on the second AI / ML model and / or the third AI / ML model. For example, the network device can train an updated decoder for CSI recovery based on the second AI / ML model indicated by the fifth information, or train an updated decoder based on the third AI / ML model indicated by the fifth information, or train an updated decoder by combining the second AI / ML model and the third AI / ML model indicated by the fifth information.

[0237] In some embodiments, the fifth information comprises one or more second AI / ML models or model parameters of the one or more second AI / ML models, and / or one or more third AI / ML models or model parameters of the one or more third AI / ML models. Optionally, the network device can be deployed with multiple AI / ML models for CSI related inference, i.e., the network device can be deployed with multiple first AI / ML models. Different second AI / ML models or third AI / ML models can correspond to different models.

[0238] It should be understood that the model structure of each bilateral model can be the same, and can include a CSI generation part model (encoder) and a CSI recovery part model (decoder). Optionally, the model parameters of different bilateral models can be different, and different bilateral models can correspond to different use scenarios.

[0239] Optionally, the number of models (i.e., first AI / ML models) deployed by the network device can be greater than or equal to one. For example, the network device can use different models for inference according to different scenarios, or can use multiple models for inference at the same time. Optionally, different first AI / ML models can be obtained based on different second AI / ML models. For example, the fifth information can include 5 reference models or model parameters of the reference models (such as model parameters of the second AI / ML models and / or the third AI / ML models), and the network device can determine 5 different first AI / ML models according to the 5 reference models, and use different first AI / ML models for inference in different scenarios, or use multiple different first AI / ML models for inference at the same time.

[0240] In some embodiments, after receiving the fifth information, the network device can directly use all or part of the multiple second AI / ML models as its deployed models, or can update all or part of the multiple second AI / ML models to obtain its deployed models. For example, the fifth information can include model parameters of 4 second AI / ML models, and the network device can use the parameters of the first and third second AI / ML models as model parameters of two first AI / ML models deployed by it, and update the second and fourth second AI / ML models to obtain model parameters of another two first AI / ML models deployed by it.

[0241] In some embodiments, the fifth information comprises model parameters of the plurality of second AI / ML models, and / or model parameters of the plurality of third AI / ML models; and the fifth information further indicates the second AI / ML model to which the model parameter of the plurality of second AI / ML models corresponds respectively, and / or the AI / ML model to which the model parameter of the plurality of third AI / ML models corresponds respectively.

[0242] For example, the fifth information can comprise model parameter 1, model parameter 2, model parameter 3 and model parameter 4, which are all second parts of a bilateral model (i.e., second AI / ML models), and the network device can use four different models (i.e., first AI / ML models) to infer the CSI quantization information in four scenarios respectively, and the fifth information can further indicate that the model parameter 1 corresponds to the model used by the network device in scenario 1, the model parameter 2 corresponds to the model used by the network device in scenario 2, the model parameter 3 corresponds to the model used by the network device in scenario 3, and the model parameter 2 corresponds to the model used by the network device in scenario 3. Further, the network device can determine the model parameters of the model used in different scenarios according to different model parameters.

[0243] In some embodiments, the fifth information can be referred to as “model indication information”, “model parameter information”, “training indication information”, etc., and the name thereof is not limited in the embodiments of the present disclosure.

[0244] In some embodiments, the network device receives the fifth information. Optionally, the network device determines the first AI / ML model according to the fifth information. Optionally, the network device determines one or more first AI / ML models according to the fifth information. Optionally, after determining the one or more first AI / ML models according to the fifth information, the network device performs step S2102.

[0245] In step S2102, the network device sends the sixth information to the terminal.

[0246] In some embodiments, the network device is deployed with a plurality of first AI / ML models, and the sixth information is used to indicate whether the model parameters of each first AI / ML model in the plurality of first AI / ML models are obtained by updating.

[0247] For example, the fifth information sent by the terminal in step S2101 can include four second AI / ML models (or model parameters), and the network device can determine corresponding sixth information after receiving the fifth information and determining the four first AI / ML models used in the four different scenarios. The sixth information can be, for example, a bitmap, which can include four bits. The first bit is used to indicate whether the model used in scenario 1 is obtained by updating the second AI / ML model, the second bit is used to indicate whether the model used in scenario 2 is obtained by updating the second AI / ML model, the third bit is used to indicate whether the model used in scenario 3 is obtained by updating the second AI / ML model, and the fourth bit is used to indicate whether the model used in scenario 4 is obtained by updating the second AI / ML model.

[0248] In some embodiments, step S2102 is optional. For example, whether the network device updates the model parameters of each second AI / ML model (and / or third AI / ML model) can be default or predefined, or can be indicated by the fifth information. In this case, the network device can not need to send the sixth information to the terminal.

[0249] In some embodiments, the sixth information can be referred to as "update indication information", "update bitmap", etc., and the name of the embodiment of the present disclosure is not limited.

[0250] In step S2103, the network device sends a first signal to the terminal.

[0251] In some embodiments, the first signal can be a downlink pilot signal. Optionally, the first signal can be a channel state information reference signal (Channel State Information-Reference Signal, CSI-RS).

[0252] In some embodiments, the first signal is used by the terminal to determine the second CSI.

[0253] In some embodiments, the terminal receives the first signal sent by the network device. Optionally, after receiving the first signal, the terminal performs step S2104.

[0254] In step S2104, the terminal determines the second CSI according to the first signal.

[0255] In some embodiments, the terminal measures the first signal to obtain the second CSI.

[0256] In some embodiments, step S2103 and step S2104 can be performed before step S2101.

[0257] In some embodiments, the step S2103 and the step S2104 can be performed in a case that the first AI / ML model is determined to meet the first performance requirement. For example, in the step S2105, the terminal can first send the first information not including the second CSI and the second quantization information to the network device, and then determine the second CSI and / or the second quantization information and send the first information including the second CSI and / or the second quantization information to the network device after determining that the first AI / ML model meets the first performance requirement.

[0258] In the step S2105, the terminal sends the first information to the network device.

[0259] In some embodiments, the first information is used to instruct the network device to generate the third information. Optionally, the first information is used to instruct the network device to input the second information into the first AI / ML model and / or the second AI / ML model to generate the third information. Optionally, the first information is used for the network device to determine the second information input into the first AI / ML model and / or the second AI / ML model.

[0260] In some embodiments, the first information can include one or more of the following: the first CSI, the first CSI being a predefined CSI; the first quantization information, the first quantization information being obtained by the terminal inputting the first CSI into the third AI / ML model; the second CSI, the second CSI being a CSI measured by the terminal; and the second quantization information, the second quantization information being obtained by the terminal inputting the second CSI into the third AI / ML model.

[0261] In some embodiments, the second CSI can be obtained based on the step S2103 and the step S2104.

[0262] In some embodiments, the optional implementation of inputting the second information into the first AI / ML model and / or the second AI / ML model by the network device to generate the third information will be described in detail in the step S2106 below, and will not be repeated here.

[0263] In some embodiments, when the fourth CSI and the seventh CSI are not required to be included in the third information generated by the network device, the first information can not include the first CSI and the first quantization information described above. Optionally, when the first CSI and / or the first quantization information are known by the network device, the first information described above can also not include the first CSI and the first quantization information described above.

[0264] In some embodiments, when the fifth CSI and the seventh CSI are not required to be included in the third information generated by the network device, the first information can not include the second CSI and the second quantization information.

[0265] Optionally, when the network device is aware of the third AI / ML model, and the fifth information includes model parameters of the third AI / ML model, the first information can only include the first CSI and / or the second CSI without the first quantization information and the second quantization information, and the network device can generate the first quantization information and / or the second quantization information based on the third AI / ML model.

[0266] In some embodiments, the terminal can send different first information based on the monitoring requirement of the first AI / ML model. Optionally, the different first information is used to instruct the network device to generate different third information.

[0267] For example, if the terminal needs to monitor whether the first AI / ML model meets the first performance requirement, the terminal can send the first information including the first CSI and the first quantization information to the network device, which can be used to instruct the network device to generate the fourth CSI; or send the first information to the network device to instruct the network device to generate the sixth CSI and the ninth CSI. If the terminal needs to monitor whether the first AI / ML model meets the second performance requirement, the terminal can send the first information to the network device to instruct the network device to generate the fourth CSI and the fifth CSI. And so on.

[0268] In some embodiments, the terminal can send different first information for different first AI / ML models. For example, the network device deploys multiple first AI / ML models, and the terminal can monitor whether different first AI / ML models meet different performance requirements. For example, the terminal can send the first information 1 for the first AI / ML model used by the network device in scenario 1, and send the first information 2 for the first AI / ML model used by the network device in scenario 2, wherein the first information 1 can be used to instruct the network device to generate the fourth CSI, and the first information 2 can be used to instruct the network device to generate the fourth CSI and the seventh CSI. And so on.

[0269] It can be understood that when the model parameters of the first AI / ML model and the second AI / ML model are the same, the fourth CSI and the seventh CSI, the fifth CSI and the eighth CSI, and the sixth CSI and the ninth CSI are respectively the same CSI, and at this time the first information can be used to instruct the network device to generate one or more of the fourth CSI, the fifth CSI, and the sixth CSI, without generating the seventh CSI, the eighth CSI, and the ninth CSI.

[0270] In step S2106, the network device inputs the second information into the first AI / ML model and / or the second AI / ML model to obtain the third information.

[0271] In some embodiments, the second information includes at least one CSI quantization information.

[0272] In some embodiments, the quantization information can also be referred to as "compressed CSI", "CSI feedback information", "CSI compressed quantization information", etc., and the name thereof is not limited in the embodiments of the present disclosure.

[0273] In some embodiments, the second information comprises one or more of the following:

[0274] The first quantization information, the first quantization information being obtained by inputting the first CSI into the third AI / ML model by the terminal or the network device, the first CSI being a predefined CSI;

[0275] The second quantization information, the second quantization information being obtained by inputting the second CSI into the third AI / ML model by the terminal or the network device, the second CSI being measured by the terminal;

[0276] The third quantization information, the third quantization information being quantization information corresponding to the third CSI, the third CSI being a CSI used for updating the first AI / ML model by the network device.

[0277] It can be understood that the network device can take the third CSI as a training set, and train the first AI / ML model based on the second AI / ML model and / or the third AI / ML model sent by the terminal.

[0278] Optionally, the third CSI can comprise a plurality of CSIs. Optionally, the third CSI is any one of the plurality of CSIs used for updating the first AI / ML model. Optionally, the third CSI is an average value of the plurality of CSIs.

[0279] In some embodiments, the first quantization information and the second quantization information can be sent by the terminal to the network device, for example, sent to the network device through the first information.

[0280] In some embodiments, the first quantization information can be obtained by inputting the first CSI into the third AI / ML model by the network device. Optionally, the first CSI can be sent by the terminal to the network device, for example, sent through the first information.

[0281] In some embodiments, the second quantization information can be sent by the terminal to the network device, for example, sent to the network device through the first information. Optionally, the second quantization information can be obtained by inputting the second CSI into the third AI / ML model by the network device. Optionally, the second CSI can be sent by the terminal to the network device.

[0282] In some embodiments, the second information can be determined based on the first information. Optionally, the second information can be determined based on an indication of the first information, and / or content comprised in the first information. Optionally, the network device receives the first information transmitted by the terminal, and determines the second information based on the first information. Optionally, the network device receives the first information, and determines the second information based on an indication of the first information, and / or content comprised in the first information.

[0283] That is, which information is input into the first AI / ML model by the network device in step S2106 can be determined by the network device based on the first information, for example, the network device can determine the second information based on an indication of the first information, and / or content comprised in the first information.

[0284] For example, the network device can determine that the second information can comprise first quantization information, second quantization information and third quantization information according to the indication of the first information, wherein the first quantization information can be indicated by the terminal through the first information or can be determined by the network device based on the first CSI in advance, the second quantization information can be determined by the network device according to the second CSI, and the third quantization information can be determined by the network device based on the third CSI in advance. Alternatively, the first information comprises the first quantization information and the second CSI, and the network device determines that the second information comprises the first quantization information and the second quantization information according to the content comprised in the first information, and inputs the second CSI into the third AI / ML model to obtain the second quantization information.

[0285] In some embodiments, the network device inputs the second information into the first AI / ML model and / or the second AI / ML model to obtain the third information, comprising one or more of:

[0286] inputting the first quantization information into the first AI / ML model to obtain the fourth CSI output by the first AI / ML model;

[0287] inputting the second quantization information into the first AI / ML model to obtain the fifth CSI output by the first AI / ML model;

[0288] inputting the third quantization information into the first AI / ML model to obtain the sixth CSI output by the first AI / ML model;

[0289] inputting the first quantization information into the second AI / ML model to obtain the seventh CSI output by the second AI / ML model;

[0290] inputting the second quantization information into the second AI / ML model to obtain the eighth CSI output by the second AI / ML model;

[0291] inputting the third quantization information into the second AI / ML model to obtain the ninth CSI output by the second AI / ML model.

[0292] In some embodiments, the third information comprises one or more of:

[0293] a fourth CSI, the fourth CSI being obtained by the network device by inputting the first quantized information into the first AI / ML model;

[0294] a fifth CSI, the fifth CSI being obtained by the network device by inputting the second quantized information into the first AI / ML model;

[0295] a sixth CSI, the sixth CSI being obtained by the network device by inputting the third quantized information into the first AI / ML model;

[0296] a seventh CSI, the seventh CSI being obtained by the network device by inputting the first quantized information into the second AI / ML model;

[0297] an eighth CSI, the eighth CSI being obtained by the network device by inputting the second quantized information into the second AI / ML model;

[0298] a ninth CSI, the ninth CSI being obtained by the network device by inputting the third quantized information into the second AI / ML model.

[0299] In some embodiments, when the model parameters of the first AI / ML model and the second AI / ML model are the same, the fourth CSI and the seventh CSI, the fifth CSI and the eighth CSI, and the sixth CSI and the ninth CSI are respectively the same CSI, and accordingly, the network device can not need to determine the seventh CSI, the eighth CSI, and the ninth CSI. That is, the network device only when the model parameters of the first AI / ML model and the second AI / ML model are different, i.e., the first AI / ML model is the updated model, inputs the corresponding second information into the second AI / ML model to obtain one or more of the seventh CSI, the eighth CSI, and the ninth CSI based on the indication of the first information.

[0300] In some embodiments, the network device can generate what third information can be determined based on the first information.

[0301] It can be understood that the network device can generate and send the third information including different contents to the terminal when receiving different first information. When the contents included in the third information are different, the third information can be used to determine whether the first AI / ML model meets different performance requirements, or to determine different performance change reasons. For example, when monitoring whether the first AI / ML model meets the performance change caused by model deployment, the terminal can send corresponding first information to indicate that the third information includes the fourth CSI, and at this time, the network device determines that the second information at least includes the first quantization information; when monitoring whether the first AI / ML model meets the performance change caused by channel information change, the terminal can send corresponding first information to indicate that the third information includes the fifth CSI, and at this time, the network device determines that the second information at least includes the second quantization information, and the like.

[0302] In step S2107, the network device sends fourth information to the terminal.

[0303] In some embodiments, the fourth information includes the third information and / or the third CSI. Wherein the third information can be obtained based on step S2106

[0304] Optionally, when the third information generated by the network device includes the sixth CSI and / or the ninth CSI, the fourth information can include the third CSI. Optionally, when the third information does not include the sixth CSI and the seventh CSI, the fourth information can not include the third CSI, that is, the network device can directly send the third information to the terminal.

[0305] In some embodiments, the network device sending the fourth information to the terminal can be sent through one or more signals / signaling, for example, the network device can pre-send the third CSI to the terminal, and then send the third information to the terminal after determining the third information.

[0306] In some embodiments, the third CSI can also be carried by other information. For example, when the network device sends the sixth information to the terminal, the sixth information can not only be used to indicate whether each first AI / ML model is obtained by updating, but also be used to indicate the third CSI corresponding to each updated first AI / ML model.

[0307] It can be understood that the network device transmits the third CSI through the fourth information is only one possible implementation, when the terminal needs to monitor the first AI / ML model by using the third CSI, the network device can transmit the third CSI to the terminal, and the present embodiment does not limit the specific transmission mode of the third CSI.

[0308] In some embodiments, the fourth information is used by the terminal to determine whether the first AI / ML model meets the performance requirement, and / or determine the reason for the change in performance of the first AI / ML model.

[0309] In some embodiments, the terminal receives the fourth information (or the third information) sent by the network device. Optionally, the terminal determines whether the first AI / ML model meets the performance requirement, and / or determines the reason for the change in performance of the first AI / ML model, according to the fourth information (or the third information).

[0310] In some embodiments, the optional implementation of the terminal determining whether the first AI / ML model meets the performance requirement, and / or determining the reason for the change in performance of the first AI / ML model, is substantially the same as the optional implementation of the network device determining whether the first AI / ML model meets the performance requirement, and / or determining the reason for the change in performance of the first AI / ML model, according to the third information in step S2108, and the embodiments of the present disclosure will not be described here.

[0311] In some embodiments, step S2107 is optional. For example, when the terminal does not need to determine whether the first AI / ML model meets the performance requirement, and / or determine the reason for the change in performance of the first AI / ML model, the network device can not need to send the fourth information to the terminal.

[0312] In step S2108, the network device determines whether the first AI / ML model meets the performance requirement, and / or determines the reason for the change in performance of the first AI / ML model, according to the third information.

[0313] In some embodiments, the network device determines whether the first AI / ML model meets the performance requirement according to the third information, or the terminal determines whether the first AI / ML model meets the performance requirement according to the fourth information, including one or more of the following:

[0314] determining whether the first AI / ML model meets the first performance requirement according to the first CSI and the fourth CSI, or the second CSI and the fifth CSI, or the third CSI and the sixth CSI;

[0315] determining whether the first AI / ML model meets the second performance requirement according to the first CSI, the second CSI, the fourth CSI, and the fifth CSI;

[0316] determining whether the first AI / ML model meets the third performance requirement according to the first CSI, the fourth CSI, and the seventh CSI;

[0317] determining whether the first AI / ML model meets the third performance requirement according to the second CSI, the fifth CSI, and the eighth CSI;

[0318] determine whether the first AI / ML model meets the third performance requirement according to the third CSI, the sixth CSI, and the ninth CSI;

[0319] determine whether the first AI / ML model meets the fourth performance requirement according to the second CSI, the third CSI, the fifth CSI, and the sixth CSI.

[0320] In some embodiments, the network device or the terminal determines the cause of the performance change of the first AI / ML model, including one or more of the following:

[0321] the first AL / ML model does not meet the first performance requirement, and it is determined that the performance change of the first AL / ML model is caused by model deployment;

[0322] the first AL / ML model does not meet the second performance requirement or the fourth performance requirement, and it is determined that the performance change of the first AL / ML model is caused by actual channel information change;

[0323] the second AL / ML model does not meet the third performance requirement, and it is determined that the performance change of the second AL / ML model is caused by model update.

[0324] In some embodiments, when the first AI / ML model is untrained, the network device and the terminal can not need to determine whether the third performance requirement is met.

[0325] In some embodiments, after determining that the first AI / ML model meets or does not meet a certain performance requirement, the network device and the terminal can further determine whether the first AI / ML model meets other performance requirements to obtain a more detailed determination of the cause of the performance change. For example, the terminal and the network device can repeatedly perform steps S2105 to S2108, or steps S2103 to S2108 to monitor the first AI / ML model in multiple dimensions.

[0326] For example, if the first AI / ML model is not updated, i.e., the parameters of the first AI / ML model are the same as those of the second AI / ML model, when the network device determines that the first AI / ML model meets the first performance requirement based on the first CSI and the fourth CSI, the network device can expect to receive the updated first information sent by the terminal, and generate the fifth CSI based on the second CSI indicated by the first information. Then, the network device determines whether the first AI / ML model meets the second performance requirement based on the first CSI, the second CSI, the fourth CSI, and the fifth CSI.

[0327] Alternatively, if the first AI / ML model is updated (or trained), the network device can expect to receive updated first information sent by the terminal, and based on an indication of the first information, generate a sixth CSI further based on the third CSI, and then determine whether the first AI / ML model meets the fourth performance requirement based on the second CSI, the third CSI, the fifth CSI, and the sixth CSI. If the terminal meets the second performance requirement but not the fourth performance requirement, it can be determined that the first AI / ML model has changed in performance, and the change is caused by channel information update. If the network device determines that the first AI / ML model meets the fourth performance requirement, the network device can also expect to receive another first information sent by the terminal, and generate a seventh CSI based on an indication of the first information, and then determine whether the first AI / ML model meets the third performance requirement based on the first CSI, the fourth CSI, and the seventh CSI.

[0328] In some embodiments, the network device can send the fourth CSI to the terminal after determining the fourth CSI, send the third CSI and the sixth CSI to the terminal after determining the sixth CSI, and send the seventh CSI to the terminal after determining the seventh CSI.

[0329] In some embodiments, determining whether the first AI / ML model meets the first performance requirement can be achieved by calculating the similarity between two CSIs, such as Square Generalized Cosine Similarity (SGCS). For example, if the similarity between the first CSI and the fourth CSI, or the similarity between the second CSI and the fifth CSI, or the similarity between the third CSI and the sixth CSI is less than or equal to a preset threshold, it is determined that the first AI / ML does not meet the first performance requirement.

[0330] In some embodiments, determining whether the first AI / ML model meets the third performance requirement can be achieved by calculating the similarity of each of the two CSIs with another CSI, and then comparing the values of the two similarities. For example, the SGCS of the fourth CSI and the seventh CSI with respect to the first CSI can be calculated, and if the difference between the two SGCSs is greater than a preset threshold, it can be determined that the first AI / ML model does not meet the third performance requirement.

[0331] In some embodiments, determining whether the first AI / ML model meets the fourth performance requirement can be achieved by calculating the similarity between the second CSI and the fifth CSI, and the similarity between the third CSI and the sixth CSI, and then comparing the values of the two similarities.

[0332] In some embodiments, when the fourth information includes multiple CSIs, the multiple CSIs can be predefined or indicated by an indication information. The fourth information can include the indication information or be carried by other information or signaling. For example, when the fourth information includes a third CSI and a sixth CSI, the first CSI in the fourth information can be the third CSI and the second CSI can be the sixth CSI by default.

[0333] In some embodiments, the CSI involved in the embodiments of the present disclosure can be a mean value of multiple target CSIs or determined by multiple target CSIs. The target CSI can be full channel information or a feature vector of full channel information, or channel information in the angle and / or delay domain after DFT inverse transformation.

[0334] In some embodiments, the names of information and the like are not limited to the names described in the embodiments. The terms of "information", "message", "signal", "signaling", "report", "configuration", "indication", "instruction", "command", "channel", "parameter", "domain", "field", "symbol", "codebook", "codeword", "codepoint", "bit", "data", "program", "chip", and the like can be replaced with each other.

[0335] In some embodiments, the terms of "acquire", "obtain", "get", "receive", "transmit", "bidirectional transmission", "send and / or receive" can be replaced with each other, which can be interpreted as receiving from other subjects, acquiring from protocols, acquiring from higher layers, obtaining by self-processing, and implementing autonomously.

[0336] In some embodiments, the terms of "send", "transmit", "report", "issue", "transmit", "bidirectional transmission", "send and / or receive" can be replaced with each other.

[0337] In some embodiments, the terms "certain", "preset", "pre-set", "set", "indicated", "a certain", "any", "first", and the like can be replaced with each other, and "certain A", "preset A", "pre-set A", "set A", "indicated A", "a certain A", "any A", "first A" can be interpreted as A predetermined in a protocol or the like, or A obtained by setting, configuration, or indication, or a certain A, a certain A, any A, or first A, but are not limited thereto.

[0338] In some embodiments, the determination or judgment can be made by a value represented by 1 bit (0 or 1), or by a true or false value (Boolean value) represented by true or false, or by comparison of numerical values (for example, comparison with a predetermined value), but is not limited thereto.

[0339] In some embodiments, "not expected to receive" can be interpreted as not receiving on the time domain resource and / or the frequency domain resource, or as not performing subsequent processing on the data or the like after receiving the data or the like; "not expected to send" can be interpreted as not sending, or as sending but not expecting the receiving party to respond to the content of the sending.

[0340] The communication method related to the embodiments of the present disclosure can include at least one of steps S2101 to S2108. For example, step S2102 can be implemented as an independent embodiment, step S2105 can be implemented as an independent embodiment, step S2106 can be implemented as an independent embodiment, step S2108 can be implemented as an independent embodiment, step S2106+step S2107 can be implemented as an independent embodiment, step S2101+step S2102 can be implemented as an independent embodiment, S2103 to step S2108 can be implemented as an independent embodiment, but are not limited thereto.

[0341] In some embodiments, the order of steps S2101 and S2105 can be exchanged or performed simultaneously, and the order of steps S2102 and S2103 can be exchanged or performed simultaneously.

[0342] In some embodiments, steps S2101 to S2105 and steps S2107 to S2108 are optional, and one or more of these steps can be omitted or replaced in different embodiments.

[0343] In some embodiments, steps S2101 to S2107 are optional, and one or more of these steps can be omitted or replaced in different embodiments.

[0344] In the embodiments of the present disclosure, part or all of the steps, the optional implementation manners thereof, can be combined with part or all of the steps in other embodiments, or can be combined with the optional implementation manners of other embodiments.

[0345] In some embodiments, other optional implementation manners can be described before or after the corresponding description of FIG. 2.

[0346] FIG. 3A is a flow diagram of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG. 3A, the embodiment of the present disclosure relates to a model performance monitoring method (network device side), and the above method comprises the following steps:

[0347] Step S3101: acquiring fifth information.

[0348] The optional implementation manner of step S3101 can refer to the optional implementation manner of step S2101 in FIG. 2 and other associated parts in the embodiments involved in FIG. 2, which will not be repeated here.

[0349] In some embodiments, the network device receives the fifth information sent by the terminal, but is not limited thereto, and can also receive the fifth information sent by other subjects.

[0350] In some embodiments, the network device acquires the fifth information specified by a protocol.

[0351] In some embodiments, the network device acquires the fifth information from an upper layer.

[0352] In some embodiments, the network device processes to obtain the fifth information.

[0353] In some embodiments, step S3101 is omitted, and the network device autonomously implements the function indicated by the fifth information, or the above function is default or default.

[0354] Step S3102: sending sixth information.

[0355] The optional implementation manner of step S3101 can refer to the optional implementation manner of step S2102 in FIG. 2 and other associated parts in the embodiments involved in FIG. 2, which will not be repeated here.

[0356] In some embodiments, the network device sends the sixth information to the terminal, but is not limited thereto, and can also send the sixth information to other subjects.

[0357] Step S3103: sending a first signal.

[0358] The optional implementation of step S3101 can refer to the optional implementation of step S2103 in FIG. 2 and other associated parts in the embodiments involved in FIG. 2, which will not be repeated here.

[0359] In some embodiments, the network device sends the first signal to the terminal, but is not limited thereto, and can send the first signal to other subjects.

[0360] Step S3104: obtaining the first information.

[0361] The optional implementation of step S3101 can refer to the optional implementation of step S2105 in FIG. 2 and other associated parts in the embodiments involved in FIG. 2, which will not be repeated here.

[0362] In some embodiments, the network device receives the first information sent by the terminal, but is not limited thereto, and can receive the first information sent by other subjects.

[0363] In some embodiments, the network device obtains the first information specified by a protocol.

[0364] In some embodiments, the network device obtains the first information from upper layer(s).

[0365] In some embodiments, the network device processes to obtain the first information.

[0366] In some embodiments, step S3104 is omitted, and the network device autonomously implements the function indicated by the first information, or the above function is default or default.

[0367] Step S3105: inputting the second information into the first AI / ML model and / or the second AI / ML model to obtain the third information.

[0368] The optional implementation of step S3101 can refer to the optional implementation of step S2106 in FIG. 2 and other associated parts in the embodiments involved in FIG. 2, which will not be repeated here.

[0369] Step S3106: sending the fourth information.

[0370] The optional implementation of step S3101 can refer to the optional implementation of step S2107 in FIG. 2 and other associated parts in the embodiments involved in FIG. 2, which will not be repeated here.

[0371] In some embodiments, the network device sends the fourth information to the terminal, but is not limited thereto, and can send the fourth information to other subjects.

[0372] Step S3107. Determine, according to the third information, whether the first AI / ML model meets the performance requirement, and / or determine the cause of the performance change of the first AI / ML model.

[0373] The optional implementation of step S3101 can refer to the optional implementation of step S2108 in FIG. 2 and other associated parts in the embodiments related to FIG. 2, which will not be repeated here.

[0374] The communication method related to the embodiments of the present disclosure can include at least one of steps S3101-S3107. For example, step S3102 can be implemented as an independent embodiment, step S3104 can be implemented as an independent embodiment, step S3105 can be implemented as an independent embodiment, step S3107 can be implemented as an independent embodiment, step S3105+step S3107 can be implemented as an independent embodiment, step S3105+step S3106 can be implemented as an independent embodiment, S3103 to step S3107 can be implemented as an independent embodiment, but not limited thereto.

[0375] In some embodiments, steps S3101-S3104 and steps S3106-S3107 are optional, and one or more of these steps can be omitted or replaced in different embodiments.

[0376] In some embodiments, steps S3101-S3106 are optional, and one or more of these steps can be omitted or replaced in different embodiments.

[0377] In the embodiments of the present disclosure, part or all of the steps and their optional implementations can be combined with part or all of the steps in other embodiments, or combined with the optional implementations of other embodiments.

[0378] FIG. 3B is a flow diagram of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG. 3B, the embodiments of the present disclosure relate to a model performance monitoring method (network device side), and the above method includes:

[0379] Step S3201. Obtain fifth information.

[0380] The optional implementation of step S3201 can refer to the optional implementation of step S2101 in FIG. 2, step S3101 in FIG. 3A, and other associated parts in the embodiments related to FIG. 2 and FIG. 3A, which will not be repeated here.

[0381] Step S3202. Obtain first information.

[0382] The optional implementation of step S3202 can refer to the optional implementation of step S2105 in FIG. 2, step S3104 in FIG. 3A, and other associated parts in the embodiments related to FIG. 2 and FIG. 3A, which will not be repeated here.

[0383] In step S3203, the second information is input into the first AI / ML model and / or the second AI / ML model to obtain third information.

[0384] The optional implementation of step S3203 can refer to the optional implementation of step S2106 in FIG. 2, step S3105 in FIG. 3A, and other associated parts in the embodiments related to FIG. 2 and FIG. 3A, which will not be repeated here.

[0385] In step S3204, it is determined whether the first AI / ML model meets the performance requirement according to the third information, and / or the reason for the change in the performance of the first AI / ML model is determined.

[0386] The optional implementation of step S3204 can refer to the optional implementation of step S2108 in FIG. 2, step S3107 in FIG. 3A, and other associated parts in the embodiments related to FIG. 2 and FIG. 3A, which will not be repeated here.

[0387] The communication method related to the embodiments of the present disclosure can include at least one of steps S3201 to S3204. For example, step S3201 can be implemented as an independent embodiment, step S3203 can be implemented as an independent embodiment, step S3204 can be implemented as an independent embodiment, step S3203+step S3204 can be implemented as an independent embodiment, step S3201+step S3202 can be implemented as an independent embodiment, step S3202+step S3203 can be implemented as an independent embodiment, but is not limited thereto.

[0388] In some embodiments, steps S3201 to S3202 and step S3204 are optional, and one or more of these steps can be omitted or replaced in different embodiments.

[0389] In some embodiments, steps S3201 to S3203 are optional, and one or more of these steps can be omitted or replaced in different embodiments.

[0390] In the embodiments of the present disclosure, part or all of the steps and their optional implementations can be combined with part or all of the steps in other embodiments, or combined with the optional implementations of other embodiments.

[0391] FIG. 3C is a flow diagram illustrating a method for monitoring model performance, according to an embodiment of the present disclosure. As shown in FIG. 3C, the embodiments of the present disclosure relate to a method for monitoring model performance (network device side), which comprises the following steps:

[0392] In step S3301, the sixth information is transmitted.

[0393] The optional implementation of step S3301 can refer to the optional implementation of step S2102 in FIG. 2, step S3102 in FIG. 3A, and other associated parts in the embodiments related to FIG. 2, FIG. 3A, and FIG. 3B, which will not be repeated here.

[0394] In step S3302, the first information is obtained.

[0395] The optional implementation of step S3302 can refer to the optional implementation of step S2105 in FIG. 2, step S3104 in FIG. 3A, step S3202 in FIG. 3B, and other associated parts in the embodiments related to FIG. 2, FIG. 3A, and FIG. 3B, which will not be repeated here.

[0396] In step S3303, the second information is input into the first AI / ML model and / or the second AI / ML model to obtain the third information.

[0397] The optional implementation of step S3303 can refer to the optional implementation of step S2106 in FIG. 2, step S3105 in FIG. 3A, step S3203 in FIG. 3B, and other associated parts in the embodiments related to FIG. 2, FIG. 3A, and FIG. 3B, which will not be repeated here.

[0398] In step S3304, it is determined whether the first AI / ML model meets the performance requirement according to the third information, and / or the cause of the change in the performance of the first AI / ML model is determined.

[0399] The optional implementation of step S3304 can refer to the optional implementation of step S2105 in FIG. 2, step S3106 in FIG. 3A, step S3204 in FIG. 3B, and other associated parts in the embodiments related to FIG. 2, FIG. 3A, and FIG. 3B, which will not be repeated here.

[0400] The communication method related to the embodiments of the present disclosure can comprise at least one of steps S3301 to S3304. For example, step S3301 can be implemented as an independent embodiment, step S3303 can be implemented as an independent embodiment, step S3304 can be implemented as an independent embodiment, step S3303+step S3304 can be implemented as an independent embodiment, step S3301+step S3302 can be implemented as an independent embodiment, step S3302+step S3303 can be implemented as an independent embodiment, but is not limited thereto.

[0401] In some embodiments, steps S3301-S3302 and S3304 are optional, and one or more of these steps can be omitted or replaced in different embodiments.

[0402] In some embodiments, steps S3301-S3303 are optional, and one or more of these steps can be omitted or replaced in different embodiments.

[0403] In the embodiments of the present disclosure, part or all of the steps, and their optional implementations, can be combined with part or all of the steps in other embodiments, or combined with optional implementations of other embodiments.

[0404] In some embodiments, step S3301 can be combined with one or more of steps S3201-S3204 in FIG. 3B.

[0405] FIG. 3D is a flow diagram of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG. 3D, the embodiments of the present disclosure relate to a model performance monitoring method (network device side), which includes the following steps:

[0406] In step S3401, first information is obtained.

[0407] Optional implementations of step S3401 can be found in the optional implementations of steps S2105 in FIG. 2, S3104 in FIG. 3A, S3202 in FIG. 3B, S3302 in FIG. 3C, and other associated parts in the embodiments related to FIG. 2, FIG. 3A, FIG. 3B, and FIG. 3C, which will not be repeated here.

[0408] In step S3402, second information is input into the first AI / ML model and / or the second AI / ML model to obtain third information.

[0409] Optional implementations of step S3402 can be found in the optional implementations of steps S2106 in FIG. 2, S3105 in FIG. 3A, S3203 in FIG. 3B, S3303 in FIG. 3C, and other associated parts in the embodiments related to FIG. 2, FIG. 3A, FIG. 3B, and FIG. 3C, which will not be repeated here.

[0410] In step S3403, fourth information is sent.

[0411] Optional implementations of step S3403 can be found in the optional implementations of steps S2107 in FIG. 2, S3106 in FIG. 3A, and other associated parts in the embodiments related to FIG. 2, FIG. 3A, FIG. 3B, and FIG. 3C, which will not be repeated here.

[0412] At step S3404, it is determined, according to the third information, whether the first AI / ML model meets the performance requirement, and / or the reason for the change in the performance of the first AI / ML model is determined.

[0413] The optional implementation of step S3404 can refer to the optional implementation of step S2105 in FIG. 2, step S3106 in FIG. 3A, step S3204 in FIG. 3B, step S3304 in FIG. 3C, and other related parts in the embodiments related to FIG. 2, FIG. 3A, FIG. 3B, and FIG. 3C, which are not described herein again.

[0414] The communication method related to the embodiments of the present disclosure can include at least one of steps S3401-S3404. For example, step S3401 can be implemented as an independent embodiment, step S3403 can be implemented as an independent embodiment, step S3404 can be implemented as an independent embodiment, step S3403+step S3404 can be implemented as an independent embodiment, step S3401+step S3402 can be implemented as an independent embodiment, step S3402+step S3403 can be implemented as an independent embodiment, but is not limited thereto.

[0415] In some embodiments, steps S3401 and steps S3403-S3404 are optional, and one or more of these steps can be omitted or replaced in different embodiments.

[0416] In some embodiments, steps S3401-S3403 are optional, and one or more of these steps can be omitted or replaced in different embodiments.

[0417] In the embodiments of the present disclosure, part or all of the steps and their optional implementations can be combined with part or all of the steps in other embodiments, or can be combined with the optional implementations of other embodiments.

[0418] In some embodiments, step S3403 can be combined with one or more of steps S3201-S3204 in FIG. 3B and / or steps S3301-S3304 in FIG. 3C.

[0419] FIG. 3E is a flow diagram of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG. 3E, the embodiments of the present disclosure relate to a model performance monitoring method (network device side), and the above method includes:

[0420] At step S3501, the second information is input into the first AI / ML model and / or the second AI / ML model to obtain third information.

[0421] The optional implementation of step S3501 can refer to the optional implementation of step S2106 in FIG. 2, step S3105 in FIG. 3A, step S3203 in FIG. 3B, step S3303 in FIG. 3C, step S3402 in FIG. 3D, and other associated parts in the embodiments related to FIG. 2, FIG. 3A, FIG. 3B, FIG. 3C, and FIG. 3D, which are not described here again.

[0422] In step S3502, it is determined whether the first AI / ML model meets the performance requirement according to the third information, and / or the reason for the change in the performance of the first AI / ML model is determined.

[0423] The optional implementation of step S3502 can refer to the optional implementation of step S2105 in FIG. 2, step S3106 in FIG. 3A, step S3204 in FIG. 3B, step S3304 in FIG. 3C, step S3404 in FIG. 3D, and other associated parts in the embodiments related to FIG. 2, FIG. 3A, FIG. 3B, FIG. 3C, and FIG. 3D, which are not described here again.

[0424] In some embodiments, the second information is input into the first AI / ML model and / or the second AI / ML model to obtain the third information, the second information includes at least one channel state information (CSI) quantization information, and the third information includes CSI recovered from the at least one CSI quantization information.

[0425] It is determined whether the first AI / ML model meets the performance requirement according to the third information, and / or the reason for the change in the performance of the first AI / ML model is determined.

[0426] The model parameters of the first AL / ML model are the same as the model parameters of the second AL / ML model, or the first AL / ML model is obtained by updating the second AL / ML model and / or the third AL / ML model by the network device.

[0427] The second AI / ML model and the third AI / ML model are AI / ML models trained by the terminal, the third AL / ML model is the first part of the bilateral model, and the first AI / ML model and the second AL / ML model are the second part of the bilateral model.

[0428] In some embodiments, the second information includes one or more of the following:

[0429] The first quantization information is obtained by inputting first CSI into the third AI / ML model by the terminal or the network device, and the first CSI is a predefined CSI;

[0430] The second quantization information is obtained by inputting second CSI into the third AI / ML model by the terminal or the network device, and the second CSI is measured by the terminal.

[0431] third quantization information, the third quantization information being quantization information corresponding to a third CSI, the third CSI being a CSI used by the network device to update the first AI / ML model.

[0432] In some embodiments, the method comprises:

[0433] receiving first information sent by the terminal;

[0434] determining second information based on the first information.

[0435] In some embodiments, the first information comprises one or more of:

[0436] a first CSI;

[0437] first quantization information;

[0438] a second CSI;

[0439] second quantization information.

[0440] In some embodiments, the method comprises:

[0441] sending a first signal to the terminal, the first signal being used by the terminal to measure a second CSI.

[0442] In some embodiments, inputting the second information into the first AI / ML model and / or the second AI / ML model to obtain third information, comprises one or more of:

[0443] inputting the first quantization information into the first AI / ML model to obtain a fourth CSI output by the first AI / ML model;

[0444] inputting the second quantization information into the first AI / ML model to obtain a fifth CSI output by the first AI / ML model;

[0445] inputting the third quantization information into the first AI / ML model to obtain a sixth CSI output by the first AI / ML model;

[0446] inputting the first quantization information into the second AI / ML model to obtain a seventh CSI output by the second AI / ML model;

[0447] inputting the second quantization information into the second AI / ML model to obtain an eighth CSI output by the second AI / ML model;

[0448] inputting the third quantization information into the second AI / ML model to obtain a ninth CSI output by the second AI / ML model.

[0449] In some embodiments, determining, according to the third information, whether the first AI / ML model meets the performance requirement comprises one or more of:

[0450] determining, according to the first CSI and the fourth CSI, or the second CSI and the fifth CSI, or the third CSI and the sixth CSI, whether the first AI / ML model meets the first performance requirement;

[0451] determining, according to the first CSI, the second CSI, the fourth CSI and the fifth CSI, whether the first AI / ML model meets the second performance requirement;

[0452] determining, according to the first CSI, the fourth CSI and the seventh CSI, whether the first AI / ML model meets the third performance requirement;

[0453] determining, according to the second CSI, the fifth CSI and the eighth CSI, whether the first AI / ML model meets the third performance requirement;

[0454] determining, according to the third CSI, the sixth CSI and the ninth CSI, whether the first AI / ML model meets the third performance requirement;

[0455] determining, according to the second CSI, the third CSI, the fifth CSI and the sixth CSI, whether the first AI / ML model meets the fourth performance requirement.

[0456] In some embodiments, determining, according to the third information, the cause of the performance change of the first AI / ML model comprises one or more of:

[0457] determining, according to the first AL / ML model not meeting the first performance requirement, that the performance change of the first AL / ML model is caused by model deployment;

[0458] determining, according to the first AL / ML model not meeting the second performance requirement or the fourth performance requirement, that the performance change of the first AL / ML model is caused by actual channel information change;

[0459] determining, according to the second AL / ML model not meeting the third performance requirement, that the performance change of the second AL / ML model is caused by model update.

[0460] In some embodiments, the method comprises:

[0461] sending, to the terminal, fourth information, the fourth information comprising the third information and / or the third CSI, the fourth information being used by the terminal to determine whether the first AI / ML model meets the performance requirement and / or to determine the cause of the performance change of the first AI / ML model.

[0462] In some embodiments, the method comprises:

[0463] receive the fifth information sent by the terminal, the fifth information comprising one or more second AI / ML models or model parameters of the one or more second AI / ML models, and / or one or more third AI / ML models or model parameters of the one or more third AI / ML models.

[0464] In some embodiments, the fifth information is further used to indicate the second AI / ML model to which the model parameter of each of the plurality of second AI models corresponds, and / or the third AI / ML model to which the model parameter of each of the plurality of third AI / ML models corresponds.

[0465] In some embodiments, the network device is deployed with the plurality of first AI / ML models, and the method comprises:

[0466] send, to the terminal, sixth information, the sixth information being used to indicate whether the model parameter of each of the plurality of first AI / ML models is obtained by updating.

[0467] FIG. 4A is a flow diagram of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG. 4A, the present disclosure relates to a model performance monitoring method (network device side), which comprises:

[0468] Step S4101: sending the fifth information.

[0469] The optional implementation of step S4101 can refer to the optional implementation of step S2101 in FIG. 2 and other associated parts in the embodiments related to FIG. 2, which will not be described here again.

[0470] In some embodiments, the terminal sends the fifth information to the network device, but is not limited thereto, and can send the fifth information to other subjects.

[0471] Step S4102: obtaining the sixth information.

[0472] The optional implementation of step S4102 can refer to the optional implementation of step S2102 in FIG. 2 and other associated parts in the embodiments related to FIG. 2, which will not be described here again.

[0473] In some embodiments, the terminal receives the sixth information sent by the network device, but is not limited thereto, and can receive the sixth information sent by other subjects.

[0474] In some embodiments, the terminal obtains the sixth information specified by a protocol.

[0475] In some embodiments, the terminal obtains the sixth information from upper layer(s).

[0476] In some embodiments, the terminal processes to obtain the sixth information.

[0477] In some embodiments, step S4102 is omitted, and the terminal autonomously implements the function indicated by the sixth information, or the above function is default or default.

[0478] Step S4103, obtaining the first signal.

[0479] The optional implementation of step S4103 can refer to the optional implementation of step S2103 in FIG. 2 and other associated parts in the embodiments involved in FIG. 2, which will not be repeated here.

[0480] In some embodiments, the terminal receives the first signal sent by the network device, but is not limited thereto, and can also receive the first signal sent by other subjects.

[0481] In some embodiments, the terminal obtains the first signal specified by the protocol.

[0482] In some embodiments, the terminal obtains the first signal from the upper layer(s).

[0483] In some embodiments, the terminal processes to obtain the first signal.

[0484] In some embodiments, step S4102 is omitted, and the terminal autonomously implements the function indicated by the first signal, or the above function is default or default.

[0485] Step S4104, determining the second CSI.

[0486] The optional implementation of step S4104 can refer to the optional implementation of step S2104 in FIG. 2 and other associated parts in the embodiments involved in FIG. 2, which will not be repeated here.

[0487] Step S4105, sending the first information.

[0488] The optional implementation of step S4105 can refer to the optional implementation of step S2105 in FIG. 2 and other associated parts in the embodiments involved in FIG. 2, which will not be repeated here.

[0489] In some embodiments, the terminal sends the first information to the network device, but is not limited thereto, and can also send the first information to other subjects.

[0490] Step S4106, obtaining the fourth information.

[0491] The optional implementation of step S4106 can refer to the optional implementation of step S2107 in FIG. 2 and other associated parts in the embodiments involved in FIG. 2, which will not be repeated here.

[0492] In some embodiments, the terminal receives the fourth information sent by the network device, but is not limited thereto, and can also receive the fourth information sent by other subjects.

[0493] In some embodiments, the terminal acquires the fourth information specified by a protocol.

[0494] In some embodiments, the terminal acquires the fourth information from upper layer(s).

[0495] In some embodiments, the terminal performs processing to obtain the fourth information.

[0496] In some embodiments, step S4102 is omitted, and the terminal autonomously implements the function indicated by the fourth information, or the above function is default or default.

[0497] Step S4107: According to the fourth information, it is determined whether the first AI / ML model meets the performance requirement, and / or the reason for the performance change of the first AI / ML model is determined.

[0498] The optional implementation of step S4107 can refer to the optional implementation of step S2108 in FIG. 2 and other associated parts in the embodiments involved in FIG. 2, which will not be repeated here.

[0499] The communication method involved in the embodiments of the present disclosure can include at least one of steps S4101 to S4107. For example, step S4102 can be implemented as an independent embodiment, step S4105 can be implemented as an independent embodiment, step S4106 can be implemented as an independent embodiment, step S4107 can be implemented as an independent embodiment, step S4106+step S4107 can be implemented as an independent embodiment, step S4101+step S4102 can be implemented as an independent embodiment, S4103 to step S4107 can be implemented as an independent embodiment, but not limited thereto.

[0500] In some embodiments, the order of steps S4102 and S4104 can be exchanged or executed simultaneously, and the order of steps S4101 and S4105 can be exchanged or executed simultaneously.

[0501] In some embodiments, steps S4101 to S4104 and steps S4106 to S4107 are optional, and one or more of these steps can be omitted or replaced in different embodiments.

[0502] In some embodiments, steps S4101 to S4105 and step S4107 are optional, and one or more of these steps can be omitted or replaced in different embodiments.

[0503] In some embodiments, steps S4101 to S4106 are optional, and one or more of these steps can be omitted or replaced in different embodiments.

[0504] In the embodiments of the present disclosure, part or all of the steps and their optional implementation manners can be combined with part or all of the steps in other embodiments, or combined with the optional implementation manners of other embodiments.

[0505] FIG. 4B is a flow diagram of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG. 4B, the present embodiment relates to a model performance monitoring method (network device side), which comprises the following steps:

[0506] Step S4201: transmitting fifth information.

[0507] The optional implementation manner of step S4201 can be referred to the optional implementation manner of step S2101 in FIG. 2, step S4101 in FIG. 4A, and other associated parts in the embodiments related to FIG. 2 and FIG. 4A, which will not be described here.

[0508] Step S4202: transmitting first information

[0509] The optional implementation manner of step S4202 can be referred to the optional implementation manner of step S2105 in FIG. 2, step S4105 in FIG. 4A, and other associated parts in the embodiments related to FIG. 2 and FIG. 4A, which will not be described here.

[0510] Step S4203: obtaining fourth information.

[0511] The optional implementation manner of step S4203 can be referred to the optional implementation manner of step S2107 in FIG. 2, step S4106 in FIG. 4A, and other associated parts in the embodiments related to FIG. 2 and FIG. 4A, which will not be described here.

[0512] Step S4204: determining whether the first AI / ML model meets the performance requirement according to the fourth information, and / or determining the reason for the performance change of the first AI / ML model.

[0513] The optional implementation manner of step S4204 can be referred to the optional implementation manner of step S2108 in FIG. 2, step S4107 in FIG. 4A, and other associated parts in the embodiments related to FIG. 2 and FIG. 4A, which will not be described here.

[0514] The optional implementation manner of step S3101 can be referred to the optional implementation manner of step S2108 in FIG. 2, and other associated parts in the embodiments related to FIG. 2, which will not be described here.

[0515] The communication method related to the embodiments of the present disclosure can include at least one of steps S4201-S4204. For example, step S4201 can be implemented as an independent embodiment, step S4203 can be implemented as an independent embodiment, step S4204 can be implemented as an independent embodiment, step S4203+step S4204 can be implemented as an independent embodiment, step S4201+step S4202 can be implemented as an independent embodiment, step S4202+step S4203 can be implemented as an independent embodiment, but the present disclosure is not limited thereto.

[0516] In some embodiments, steps S4201-S4202 and step S4204 are optional, and one or more of these steps can be omitted or replaced in different embodiments.

[0517] In some embodiments, steps S4201-S4203 are optional, and one or more of these steps can be omitted or replaced in different embodiments.

[0518] In the embodiments of the present disclosure, part or all of the steps and their optional implementation manners can be combined with part or all of the steps in other embodiments, or combined with the optional implementation manners of other embodiments.

[0519] FIG. 4C is a flow diagram of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG. 4C, the embodiments of the present disclosure relate to a model performance monitoring method (network device side), and the above method includes:

[0520] Step S4301, obtaining sixth information.

[0521] The optional implementation manner of step S4301 can be referred to the optional implementation manners of step S2102 in FIG. 2, step S4102 in FIG. 4A, and other associated parts in the embodiments related to FIG. 2, FIG. 4A, and FIG. 4B, which are not described here.

[0522] Step S4302, sending first information

[0523] The optional implementation manner of step S4302 can be referred to the optional implementation manners of step S2105 in FIG. 2, step S4105 in FIG. 4A, step S4202 in FIG. 4B, and other associated parts in the embodiments related to FIG. 2, FIG. 4A, and FIG. 4B, which are not described here.

[0524] Step S4303, obtaining fourth information.

[0525] The optional implementation of step S4303 can refer to the optional implementation of step S2107 in FIG. 2, step S4106 in FIG. 4A, step S4203 in FIG. 4B, and other associated parts in the embodiments related to FIG. 2, FIG. 4A and FIG. 4B, which are not described here again.

[0526] In step S4304, it is determined whether the first AI / ML model meets the performance requirement according to the fourth information, and / or the reason for the performance change of the first AI / ML model is determined.

[0527] The optional implementation of step S4304 can refer to the optional implementation of step S2108 in FIG. 2, step S4107 in FIG. 4A, step S4204 in FIG. 4B, and other associated parts in the embodiments related to FIG. 2, FIG. 4A and FIG. 4B, which are not described here again.

[0528] The communication method related to the embodiments of the present disclosure can include at least one of steps S4301 to S4304. For example, step S4301 can be implemented as an independent embodiment, step S4303 can be implemented as an independent embodiment, step S4304 can be implemented as an independent embodiment, step S4303+step S4304 can be implemented as an independent embodiment, step S4301+step S4302 can be implemented as an independent embodiment, step S4302+step S4303 can be implemented as an independent embodiment, but is not limited thereto.

[0529] In some embodiments, steps S4301 to S4302 and step S4304 are optional, and one or more of these steps can be omitted or replaced in different embodiments.

[0530] In some embodiments, steps S4301 to S4303 are optional, and one or more of these steps can be omitted or replaced in different embodiments.

[0531] In the embodiments of the present disclosure, part or all of the steps and their optional implementations can be combined with part or all of the steps in other embodiments, or combined with the optional implementations of other embodiments.

[0532] In some embodiments, step S4301 can be combined with one or more of steps S4201 to S4204 in FIG. 4B.

[0533] FIG. 4D is a flow diagram of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG. 4D, the present disclosure relates to a model performance monitoring method (network device side), and the above method includes:

[0534] In step S4401, the first information is sent.

[0535] The optional implementation of step S4401 can refer to the optional implementation of step S2105 in FIG. 2, step S4105 in FIG. 4A, step S4202 in FIG. 4B, step S4302 in FIG. 4C, and other associated parts in the embodiments related to FIG. 2, FIG. 4A, FIG. 4B, and FIG. 4C, which are not described here again.

[0536] In step S4402, fourth information is acquired.

[0537] The optional implementation of step S4402 can refer to step S2107 in FIG. 2, step S4106 in FIG. 4A, step S4203 in FIG. 4B, step S4303 in FIG. 4C, and other associated parts in the embodiments related to FIG. 2, FIG. 4A, FIG. 4B, and FIG. 4C, which are not described here again.

[0538] In step S4404, it is determined whether the first AI / ML model meets the performance requirement according to the fourth information, and / or the reason for the performance change of the first AI / ML model is determined.

[0539] The optional implementation of step S4404 can refer to step S2108 in FIG. 2, step S4107 in FIG. 4A, step S4204 in FIG. 4B, step S4304 in FIG. 4C, and other associated parts in the embodiments related to FIG. 2, FIG. 4A, FIG. 4B, and FIG. 4C, which are not described here again.

[0540] The communication method related to the embodiments of the present disclosure can include at least one of steps S4401 to S4404. For example, step S4401 can be implemented as an independent embodiment, step S4403 can be implemented as an independent embodiment, step S4404 can be implemented as an independent embodiment, step S4403+step S4404 can be implemented as an independent embodiment, step S4401+step S4402 can be implemented as an independent embodiment, step S4402+step S4403 can be implemented as an independent embodiment, but is not limited thereto.

[0541] In some embodiments, steps S4401 and steps S4403 to S4404 are optional, and one or more of these steps can be omitted or replaced in different embodiments.

[0542] In some embodiments, steps S4401 to S4403 are optional, and one or more of these steps can be omitted or replaced in different embodiments.

[0543] In the embodiments of the present disclosure, part or all of the steps, and optional implementation manners thereof, can be combined with part or all of the steps in other embodiments, or combined with optional implementation manners of other embodiments.

[0544] In some embodiments, step S4403 can be combined with one or more of steps S4201 to S4204 in FIG. 4B, and / or steps S4301 to S4304 in FIG. 4C.

[0545] FIG. 4E is a flow diagram of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG. 4E, the present embodiment relates to a model performance monitoring method (network device side), which comprises the following steps:

[0546] Step S4501: transmitting first information.

[0547] Optional implementation manners of step S4501 can refer to the optional implementation manners of step S2105 in FIG. 2, step S4105 in FIG. 4A, step S4202 in FIG. 4B, step S4302 in FIG. 4C, step S4404 in FIG. 4D, and other related parts in the embodiments related to FIG. 2, FIG. 4A, FIG. 4B, FIG. 4C, and FIG. 4D, which will not be repeated here.

[0548] In some embodiments, the first information is used to instruct the network device to input second information into the first AI / ML model and / or the second AI / ML model to obtain third information, the second information comprises at least one CSI quantization information, and the third information comprises CSI recovered from the at least one CSI quantization information;

[0549] The third information is used for the network device to determine whether the first AI / ML model meets the performance requirement, and / or to determine the cause of the performance change of the first AI / ML model;

[0550] The model parameters of the first AL / ML model are the same as the model parameters of the second AL / ML model, or the first AL / ML model is obtained by updating the second AL / ML model and / or the third AL / ML model by the network device;

[0551] The second AI / ML model and the third AI / ML model are AI / ML models trained by the terminal, the third AL / ML model is the first part of the bilateral model, and the first AI / ML model and the second AL / ML model are the second part of the bilateral model.

[0552] In some embodiments, the second information comprises one or more of the following:

[0553] The first quantization information is obtained by inputting the first CSI into the third AI / ML model by the terminal or the network device, and the first CSI is a predefined CSI.

[0554] The second quantization information is obtained by inputting the second CSI into the third AI / ML model by the terminal or the network device, and the second CSI is measured by the terminal.

[0555] The third quantization information is quantization information corresponding to the third CSI, and the third CSI is the CSI used by the network device to update the first AI / ML model.

[0556] In some embodiments, the second information is determined by the network device based on the first information.

[0557] In some embodiments, the first information includes one or more of the following:

[0558] The first CSI;

[0559] The first quantization information;

[0560] The second CSI;

[0561] The second quantization information.

[0562] In some embodiments, the method comprises:

[0563] Receiving the first signal sent by the network device;

[0564] Measuring the first signal to obtain the second CSI.

[0565] In some embodiments, the third information includes one or more of the following:

[0566] The fourth CSI is obtained by inputting the first quantization information into the first AI / ML model by the network device;

[0567] The fifth CSI is obtained by inputting the second quantization information into the first AI / ML model by the network device;

[0568] The sixth CSI is obtained by inputting the third quantization information into the first AI / ML model by the network device;

[0569] The seventh CSI is obtained by inputting the first quantization information into the second AI / ML model by the network device;

[0570] The eighth CSI is obtained by inputting the second quantization information into the second AI / ML model by the network device;

[0571] The ninth CSI is obtained by inputting the third quantization information into the second AI / ML model by the network device.

[0572] In some embodiments, the method comprises:

[0573] receiving fourth information sent by the network device, the fourth information comprising the third information and / or the third CSI;

[0574] determining whether the first AI / ML model meets the performance requirement according to the fourth information, and / or determining the cause of the performance change of the first AI / ML model.

[0575] In some embodiments, determining whether the first AI / ML model meets the performance requirement according to the third information comprises one or more of:

[0576] determining whether the first AI / ML model meets the first performance requirement according to the first CSI and the fourth CSI, or the second CSI and the fifth CSI, or the third CSI and the sixth CSI;

[0577] determining whether the first AI / ML model meets the second performance requirement according to the first CSI, the second CSI, the fourth CSI and the fifth CSI;

[0578] determining whether the first AI / ML model meets the third performance requirement according to the first CSI, the fourth CSI and the seventh CSI;

[0579] determining whether the first AI / ML model meets the third performance requirement according to the second CSI, the fifth CSI and the eighth CSI;

[0580] determining whether the first AI / ML model meets the third performance requirement according to the third CSI, the sixth CSI and the ninth CSI;

[0581] determining whether the first AI / ML model meets the fourth performance requirement according to the second CSI, the third CSI, the fifth CSI and the sixth CSI.

[0582] In some embodiments, determining the cause of the performance change of the first AI / ML model according to the third information comprises one or more of:

[0583] determining that the first AL / ML model causes the performance change due to model deployment when the first AL / ML model does not meet the first performance requirement;

[0584] determining that the first AL / ML model causes the performance change due to actual channel information change when the first AL / ML model does not meet the second performance requirement or the fourth performance requirement;

[0585] The second AL / ML model does not meet the third performance requirement, and it is determined that the performance change of the second AL / ML model is caused by model updating.

[0586] In some embodiments, the method comprises:

[0587] sending fifth information to the network device, the fifth information comprising one or more second AI / ML models or model parameters of the one or more second AI / ML models, and / or one or more third AI / ML models or model parameters of the one or more third AI / ML models.

[0588] In some embodiments, the fifth information comprises model parameters of a plurality of second AI / ML models, and / or model parameters of a plurality of third AI / ML models.

[0589] The fifth information is further used to indicate the second AI / ML models to which the model parameters of the plurality of second AI / ML models correspond respectively, and / or the third AI / ML models to which the model parameters of the plurality of third AI / ML models correspond respectively.

[0590] In some embodiments, the network device is deployed with a plurality of first AI / ML models, and the method comprises:

[0591] receiving sixth information sent by the network device, the sixth information being used to indicate whether the model parameters of each first AL / ML model in the plurality of first AL / ML models are obtained by updating.

[0592] FIG. 5 is a flow diagram of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG. 5, the present disclosure relates to a model performance monitoring method, and the above method comprises:

[0593] In step S5101, the network device inputs the second information into the first AI / ML model and / or the second AI / ML model to obtain third information.

[0594] The optional implementation of step S5101 can refer to the optional implementation of step S2106 in FIG. 2, step S3105 in FIG. 3A, step S3203 in FIG. 3B, step S3303 in FIG. 3C, step S3402 in FIG. 3D, step S3501 in FIG. 3E, and other related parts in the embodiments related to FIG. 2, FIG. 3A, FIG. 3B, FIG. 3C, FIG. 3D, FIG. 3E, FIG. 4A, FIG. 4B, FIG. 4C, FIG. 4D, and FIG. 4E, which will not be described here.

[0595] In step S5102, the network device determines whether the first AI / ML model meets the performance requirement according to the third information, and / or determines the cause of the performance change of the first AI / ML model.

[0596] The optional implementation of step S5103 can refer to the optional implementation of step S2105 in FIG. 2, step S3106 in FIG. 3A, step S3204 in FIG. 3B, step S3304 in FIG. 3C, step S3404 in FIG. 3D, step S3502 in FIG. 3E, and other related parts in the embodiments related to FIG. 2, FIG. 3A, FIG. 3B, FIG. 3C, FIG. 3D, FIG. 3E, FIG. 4A, FIG. 4B, FIG. 4C, FIG. 4D, and FIG. 4E, which are not described herein.

[0597] In some embodiments, the above method can include the method described in the above embodiments related to the network device side, the terminal side, and the like, which are not described herein.

[0598] FIG. 6 is a flowchart of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG. 6, the present disclosure relates to a model performance monitoring method, and the above method includes:

[0599] In step S6101, the NW inputs the pre-defined compressed CSI, or the compressed CSI sent by the UE, or the compressed CSI corresponding to the target CSI for training a new model as the decoder input, to obtain information for monitoring the performance of the model deployed by the NW.

[0600] In some embodiments, the model deployed by the NW can be the first AI / ML model described in the above embodiments. Alternatively, the first AI / ML model can be an updated model, or a model without updating. For example, the NW can directly use the received model parameters or model (such as the second AI / ML model) for inference. Alternatively, the NW can retrain a new model for inference based on the received model parameters or model (such as the second AI / ML model and / or the third AI / ML).

[0601] For the scenario where the NW directly uses the received model parameters or model for inference, the following embodiments can be used:

[0602] In some embodiments, the UE sends the recovered target CSI information (such as the first CSI in some embodiments) output by the decoder on the UE side, and / or the CSI feedback information (such as the first quantization information in some embodiments) to the NW. The CSI feedback information can be the decoder input information in the process of obtaining the recovered target CSI information on the UE side. Both the recovered target CSI information and the CSI feedback information need to be measured and reported by the UE.

[0603] Optionally, the NW side takes the received CSI feedback information as input information of the NW side decoder, and sends the recovered target CSI information (e.g., the fourth CSI in some embodiments) output by the decoder to the UE after quantization.

[0604] Optionally, the NW side determines whether the model deployed at the decoder side changes the performance due to the deployment of the model according to the CSI feedback information reported by the UE (if the CSI feedback information is predefined, the UE can not need to report the information again) and / or the recovered target CSI information.

[0605] Optionally, the UE side determines whether the model deployed at the NW side changes the performance due to the deployment of the model according to the recovered target CSI information sent by the NW.

[0606] In some embodiments, the UE instructs the NW to take the predefined compressed CSI (e.g., the first quantization information in some embodiments) as input of the NW side decoder.

[0607] Optionally, the NW side determines whether the model deployed at the UE side changes the performance due to the deployment of the model according to the target CSI corresponding to the predefined compressed CSI and the recovered CSI output by the decoder.

[0608] In some embodiments, the UE side instructs the NW to take the predefined compressed CSI and the CSI feedback information (e.g., the second quantization information in some embodiments) measured by the UE in the actual scene as input of the NW side decoder. The CSI feedback information can be obtained by inputting the measured target CSI (e.g., the second CSI in some embodiments) to the encoder and reported to the NW by the UE, and the measured target CSI is also sent to the NW side by the UE.

[0609] Optionally, the NW can send the two recovered target CSIs (e.g., the fourth CSI and the fifth CSI in some embodiments) output by the decoder to the UE side.

[0610] Optionally, the NW side determines whether the model at the UE side meets the performance requirement according to the CSI feedback quantization information (e.g., the second quantization information in some embodiments), the target CSI (e.g., the second CSI in some embodiments), the predefined compressed CSI (e.g., the first quantization information in some embodiments), and the predefined target CSI (e.g., the first CSI in some embodiments) corresponding to the predefined compressed CSI, to verify whether the performance changes due to the change of the channel information measured in the field.

[0611] Optionally, the UE determines whether the model performance changes due to the change of the on-site measured channel information according to the two recovered target CSIs, the predefined target CSI and the measured target CSI.

[0612] In some embodiments, if multiple decoders are deployed on the NW side, when the UE only delivers the parameters of part of the decoders, the UE also sends an indication information to the NW to indicate which parameters of the decoders are delivered by the UE.

[0613] For the scenario that the UE re-trains a new model on the basis of the received model parameters or model for inference, the following embodiments can be used:

[0614] In some embodiments, the NW indicates which models are updated by an indication information, which can be A bits of information to indicate whether the model is updated, or a new model ID or function ID, or an ID corresponding to the training data set associated with the updated model to implicitly indicate whether the model is updated.

[0615] In some embodiments, the UE side indicates to the NW to respectively use the predefined compressed CSI (such as the first quantization information involved in some embodiments) or the CSI feedback information (such as the first quantization information or the second quantization information involved in some embodiments) sent by the UE to the NW as the input of the updated decoder on the NW side. Optionally, the UE also reports the target CSI (such as the first CSI or the second CSI involved in some embodiments) corresponding to the CSI feedback information.

[0616] Optionally, the NW side determines whether the model on the NW side causes performance changes due to the update of the model according to the predefined compressed CSI feedback information or the received target CSI.

[0617] In some embodiments, the UE side indicates to the NW to respectively use the predefined compressed CSI, and the NW side uses the predefined compressed CSI as the input of the updated decoder to obtain the recovered target CSI and send it to the UE.

[0618] Optionally, the UE side determines whether the model on the NW side causes performance changes due to the update of the model according to the two recovered target CSIs.

[0619] In some embodiments, the UE-side indication causes the NW to use the compressed CSI trained by the new model and the CSI feedback information reported by the UE as inputs of the decoder at the NW side, and two different recovered target CSIs are obtained. Optionally, the UE also sends the target CSI obtained by on-site measurement to the NW.

[0620] Optionally, the NW side determines whether the performance changes due to the change of the on-site measured channel information based on the two recovered target CSIs and the on-site measured target CSI.

[0621] In the embodiments of the present disclosure, the performance of the AI / ML model can be monitored to determine the cause of the performance change in one or more combinations of the above manners.

[0622] Based on some of the above embodiments, the present disclosure also provides the following examples:

[0623] In an example, assuming that the UE side has trained the encoder and decoder models based on the collected training data set, if the structure of the encoder model is standardized, the UE needs to pass the parameters of the decoder to the NW based on the aforementioned Option 3, and the NW can directly use the received decoder parameters for inference.

[0624] For the NW side to verify the performance of the decoder deployed at the NW side: the UE sends the recovered target CSI T0 output by the UE-side decoder to the NW, wherein the recovered target CSI is obtained by the decoder based on the predefined CSI feedback information as the input of the UE-side decoder. Then, the NW obtains the recovered target CSI T1 by performing inference on the predefined compressed CSI as the input of the NW-side decoder. The NW determines whether the model at the NW side is deteriorating or whether the decoder deployed at the NW side can meet the performance requirements by comparing the relationship between T1 and T2.

[0625] For the UE side to verify the performance of the decoder deployed at the NW side: the NW side sends the recovered target CSI T1 to the UE. Then the UE determines whether the model at the NW side is deteriorating or whether the decoder deployed at the NW side can meet the performance requirements by comparing the relationship between T0 and T1.

[0626] wherein the aforementioned T0 or T1 is exemplary, and T0 or T1 can be a value determined for multiple target CSIs or an average value of multiple times. The target CSI can be full channel information or feature vector information of the full channel, or channel information in the angle and / or delay domain after DFT inverse transformation, etc.

[0627] For further verifying whether the performance of the deployed decoder at the NW side changes due to the change of channel information: the NW sends CSI-RS to the UE for measuring the target CSI on site, and instructs the UE to report the measured target CSI and the compressed CSI output by the encoder to the NW. The NW inputs the received compressed CSI into the decoder to obtain the recovered target CSI, and then determines whether the decoder meets the performance requirement by calculating the NMMSE or SGCS between the recovered target CSI and the target CSI delivered by the UE.

[0628] In another example, the NW can be pre-configured with the model structures of 4 decoders, assuming that the UE delivers the model parameters of 4 decoders to the NW, the NW can determine the 4 decoders according to the model parameters. Among them, in some embodiments, the UE can also transmit indication information to the NW to indicate the model structure corresponding to the parameters of the 4 decoders respectively. For example, when the UE delivers the model parameters of the decoder to the NW, it also indicates the model structure identification corresponding to each model parameter, and the NW can determine the model corresponding to each model parameter according to the model structure identification corresponding to each model parameter.

[0629] Further, the NW can send bitmap indication information A=4 bits to the UE to indicate which decoder model the NW has retrained and updated the model for, as shown in Table 1.

[0630] Table 1

[0631] Referring to Table 1, the UE can then know that the first decoder and the third decoder have been updated by the NW, and the following takes the first decoder as an example.

[0632] For the NW side to verify the performance of the decoder: the UE instructs the NW to use the pre-defined compressed CSI as the input of the decoder before and after the update at the NW side. The NW recovers two recovered target CSIs T3 and T4 through the decoder before and after the update. Assuming that the pre-defined compressed CSI corresponds to the target CSI T2. The NW can calculate the SGCS value S1 according to T3 and T2, and calculate the SGCS S2 according to T4 and T2, and the NW determines whether the updated decoder meets the performance requirement by comparing the values of S1 and S2. Further, the NW can send T3 and T4 to the UE, and the UE can determine whether the updated decoder meets the performance requirement by performing corresponding calculation.

[0633] For further verification of whether the performance of the decoder deployed on the NW side changes due to the change of channel information: the UE side instructs the NW to respectively use the compressed CSI trained by the new model and the compressed CSI reported by the UE (such as the first CSI in some embodiments described above) as the input of the decoder on the NW side to update the decoder, and the NW outputs two recovered target CSIs through the decoder inference, denoted as T5 and T6. Assuming that the target CSI trained by the new model is T7 and the target CSI reported by the UE is T8. The NW can calculate the SGCS S2 according to T5 and T7, calculate the SGCS S3 according to T6 and T8, and judge whether the performance changes due to the change of channel information by comparing the values of S2 and S3.

[0634] In some embodiments, when the NW sends two target CSIs to the UE, the method of distinguishing the two target CSIs can include predefining or sending a display information indication. For example, it can be predefined that the first one is the recovered target CSI corresponding to the pre-defined compressed CSI, and the other one is the recovered target CSI corresponding to the compressed CSI reported by the UE. Or, the corresponding relationship of the two target CSIs can be indicated through a display information.

[0635] In the embodiments of the present disclosure, part or all of the steps, and optional implementation manners thereof, can be combined with part or all of the steps in other embodiments, or can be combined with optional implementation manners of other embodiments.

[0636] The embodiments of the present disclosure also propose an apparatus for implementing any of the above methods, for example, an apparatus including units or modules for implementing each step performed by a terminal in any of the above methods. For another example, another apparatus is also proposed, including units or modules for implementing each step performed by a network device (such as an access network device, a core network function node, a core network device, etc.) in any of the above methods.

[0637] It should be understood that the division of each unit or module in the above apparatus is only a logical function division, and all or part of them can be integrated into a physical entity or physically separated in actual implementation. In addition, the units or modules in the apparatus can be implemented in the form of processor calling software: for example, the apparatus includes a processor, the processor is connected with a memory, the memory stores instructions, and the processor calls the instructions stored in the memory to realize any of the above methods or realize the functions of each unit or module of the above apparatus, wherein the processor is a general processor such as a central processing unit (CPU) or a microprocessor, and the memory is a memory in the apparatus or a memory outside the apparatus. Alternatively, the units or modules in the apparatus can be implemented in the form of hardware circuit, and the functions of part or all of the units or modules can be realized by the design of hardware circuit. The above hardware circuit can be understood as one or more processors; for example, in one implementation, the above hardware circuit is an application-specific integrated circuit (ASIC), and the functions of part or all of the units or modules are realized by the design of the logical relationship of elements in the circuit; for another example, in another implementation, the above hardware circuit is a programmable logic device (PLD), and a field programmable gate array (FPGA) is taken as an example, which can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by a configuration file, so as to realize the functions of part or all of the above units or modules. All units or modules of the above apparatus can be all implemented in the form of processor calling software, or all implemented in the form of hardware circuit, or part implemented in the form of processor calling software and the remaining part implemented in the form of hardware circuit.

[0638] In embodiments of the present disclosure, the processor is a circuit with signal processing capability. In one implementation, the processor can be a circuit with instruction reading and running capability, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), a digital signal processor (DSP), or the like. In another implementation, the processor can implement certain functions through a logical relationship of hardware circuit. The logical relationship of the hardware circuit is fixed or reconfigurable. For example, the processor is a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as an FPGA. In the reconfigurable hardware circuit, the processor loads a configuration document to implement the configuration of the hardware circuit. It can be understood that the processor loads instructions to implement the functions of part or all of the units or modules described above. In addition, the hardware circuit can also be designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), or the like.

[0639] FIG. 7A is a structural schematic diagram of a terminal according to an embodiment of the present disclosure. As shown in FIG. 7A, the terminal 7100 can include at least one of a transceiver module 7101, a processing module 7102, and the like. Optionally, the transceiver module 7101 is configured to perform at least one of the communication steps, such as sending and / or receiving, performed by the terminal in any of the methods described above. Details are not described herein again. Optionally, the processing module 7102 is configured to perform at least one of the other steps performed by the terminal in any of the methods described above. Details are not described herein again.

[0640] FIG. 7B is a structural schematic diagram of a network device according to an embodiment of the present disclosure. As shown in FIG. 7B, the network device 7200 can include at least one of a transceiver module 7201, a processing module 7202, and the like. Optionally, the transceiver module 7201 is configured to perform at least one of the communication steps, such as sending and / or receiving, performed by the network device in any of the methods described above. Details are not described herein again. Optionally, the processing module 7202 is configured to perform at least one of the other steps performed by the network device in any of the methods described above. Details are not described herein again.

[0641] In some embodiments, the transceiving module can include a transmitting module and / or a receiving module, which can be separate or integrated together. Alternatively, the transceiving module can be mutually replaced with a transceiver.

[0642] In some embodiments, the processing module can be one module or include multiple sub-modules. Alternatively, the multiple sub-modules perform all or part of the steps required to be performed by the processing module, respectively. Alternatively, the processing module can be mutually replaced with a processor.

[0643] FIG. 8A is a structural schematic diagram of a communication device 8100 according to the embodiments of the present disclosure. The communication device 8100 can be a network device (such as an access network device, a core network device, etc.), a terminal (such as a user equipment, etc.), a chip, a chip system, or a processor supporting the network device to implement any of the above methods, or a chip, a chip system, or a processor supporting the terminal to implement any of the above methods. The communication device 8100 can be used to implement the methods described in the above method embodiments, and details can be referred to the descriptions in the above method embodiments.

[0644] As shown in FIG. 8A, the communication device 8100 includes one or more processors 8101. The processor 8101 can be a general-purpose processor or a special-purpose processor, for example, a baseband processor or a central processing unit. The baseband processor can be used to process communication protocols and communication data, and the central processing unit can be used to control the communication device (such as a base station, a baseband chip, a terminal device, a terminal device chip, a DU or a CU, etc.), execute programs, and process data of the programs. Alternatively, the communication device 8100 is configured to perform any of the above methods. Alternatively, the one or more processors 8101 are configured to invoke instructions to cause the communication device 8100 to perform any of the above methods.

[0645] In some embodiments, the communication device 8100 further includes one or more transceivers 8102. When the communication device 8100 includes the one or more transceivers 8102, the transceiver 8102 performs at least one of the communication steps such as transmitting and / or receiving in the above methods, and the processor 8101 performs at least one of the other steps. In alternative embodiments, the transceiver can include a receiver and / or a transmitter, which can be separate or integrated together. Alternatively, the terms of transceiver, transceiving unit, transceiver, transceiving circuit, interface circuit, interface, etc. can be mutually replaced, and the terms of transmitter, transmitting unit, transmitter, transmitting circuit, etc. can be mutually replaced, and the terms of receiver, receiving unit, receiver, receiving circuit, etc. can be mutually replaced.

[0646] In some embodiments, the communication device 8100 further includes one or more memories 8103 for storing data. Alternatively, all or part of the memories 8103 can be external to the communication device 8100. In optional embodiments, the communication device 8100 can include one or more interface circuits 8104. Optionally, the interface circuit 8104 is connected to the memory 8103, and the interface circuit 8104 can be used to receive data from the memory 8103 or other devices, and can be used to send data to the memory 8103 or other devices. For example, the interface circuit 8104 can read data stored in the memory 8103 and send the data to the processor 8101.

[0647] The communication device 8100 described in the above embodiments can be a network device or a terminal, but the scope of the communication device 8100 described in the present disclosure is not limited thereto, and the structure of the communication device 8100 can not be limited by Figure 8A. The communication device can be a standalone device or can be part of a larger device. For example, the communication device can be: 1) a standalone integrated circuit (IC), or a chip, or a chip system or subsystem; (2) a set of one or more ICs, which can optionally include a storage component for storing data, programs; (3) an ASIC, such as a modem; (4) a module that can be embedded in other devices; (5) a receiver, a terminal device, a smart terminal device, a cellular phone, a wireless device, a handset, a mobile unit, a vehicle-mounted device, a network device, a cloud device, an artificial intelligence device, etc.; (6) other devices, etc.

[0648] Figure 8B is a structural schematic diagram of a chip 8200 according to an embodiment of the present disclosure. For the case where the communication device 8100 is a chip or a chip system, the structural schematic diagram of the chip 8200 shown in Figure 8B can be referred to, but is not limited thereto.

[0649] The chip 8200 includes one or more processors 8201. The chip 8200 is configured to execute any of the above methods.

[0650] In some embodiments, the chip 8200 further includes one or more interface circuits 8202. Optionally, the terms interface circuit, interface, transceiver pin, etc. can be replaced by each other. In some embodiments, the chip 8200 further includes one or more memories 8203 for storing data. Optionally, all or part of the memories 8203 can be external to the chip 8200. Optionally, the interface circuit 8202 is connected to the memory 8203, and the interface circuit 8202 can be used to receive data from the memory 8203 or other devices, and the interface circuit 8202 can be used to send data to the memory 8203 or other devices. For example, the interface circuit 8202 can read data stored in the memory 8203 and send the data to the processor 8201.

[0651] In some embodiments, the interface circuit 8202 performs at least one of the communication steps such as transmitting and / or receiving in the above-described methods. The interface circuit 8202 performing the communication steps such as transmitting and / or receiving in the above-described methods refers to, for example, the interface circuit 8202 performing data interaction between the processor 8201, the chip 8200, the memory 8203, or a transceiver device. In some embodiments, the processor 8201 performs at least one of the other steps.

[0652] The modules and / or devices described in each of the embodiments of the virtual device, the physical device, the chip, etc. can be combined or separated according to actual conditions. Optionally, part or all of the steps can also be performed by a plurality of modules and / or devices in cooperation, which is not limited here.

[0653] The disclosure further proposes a storage medium having instructions stored thereon, which, when executed on the communication device 8100, causes the communication device 8100 to perform any of the above methods. Optionally, the storage medium is an electronic storage medium. Optionally, the storage medium is a computer-readable storage medium, but is not limited to this, and it can also be a storage medium readable by other devices. Optionally, the storage medium can be a non-transitory storage medium, but is not limited to this, and it can also be a transitory storage medium.

[0654] The disclosure further proposes a program product, which, when executed by the communication device 8100, causes the communication device 8100 to perform any of the above methods. Optionally, the program product is a computer program product.

[0655] The disclosure further proposes a computer program, which, when executed on a computer, causes the computer to perform any of the above methods.

Claims

1. A method for monitoring model performance, characterized in that, Performed by a network device, the method includes: The second information is input into the first AI / ML model and / or the second AI / ML model to obtain the third information. The second information includes at least one Channel State Information (CSI) quantization information, and the third information includes the CSI recovered from at least one of the CSI quantization information. Based on the third information, determine whether the first AI / ML model meets the performance requirements, and / or determine the reason for the performance change of the first AI / ML model; Wherein, the model parameters of the first AL / ML model are the same as those of the second AL / ML model, or the first AL / ML model is obtained by the network device updating the second AL / ML model and / or the third AL / ML model; The second AI / ML model and the third AI / ML model are AI / ML models trained by the terminal. The third AI / ML model is the first part of the bilateral model, and the first AI / ML model and the second AI / ML model are the second part of the bilateral model.

2. The method according to claim 1, characterized in that, The second information includes one or more of the following: The first quantization information is obtained by the terminal or the network device by inputting the first CSI into the third AI / ML model, and the first CSI is a predefined CSI. The second quantification information is obtained by the terminal or the network device inputting the second CSI into the third AI / ML model, and the second CSI is measured by the terminal. The third quantization information is the quantization information corresponding to the third CSI, which is the CSI used by the network device to update the first AI / ML model.

3. The method according to claim 2, characterized in that, The method includes: Receive the first information sent by the terminal; Based on the first information, the second information is determined.

4. The method according to claim 3, characterized in that, The first information includes one or more of the following; The first CSI; The first quantification information; The second CSI; The second quantification information.

5. The method according to claim 4, characterized in that, The method includes: A first signal is sent to the terminal, the first signal being used by the terminal to measure the second CSI.

6. The method according to any one of claims 2-5, characterized in that, The step of inputting the second information into the first AI / ML model and / or the second AI / ML model to obtain the third information includes one or more of the following: The first quantization information is input into the first AI / ML model to obtain the fourth CSI output by the first AI / ML model; The second quantization information is input into the first AI / ML model to obtain the fifth CSI output by the first AI / ML model; The third quantization information is input into the first AI / ML model to obtain the sixth CSI output by the first AI / ML model; The first quantization information is input into the second AI / ML model to obtain the seventh CSI output by the second AI / ML model; The second quantization information is input into the second AI / ML model to obtain the eighth CSI output by the second AI / ML model; The third quantization information is input into the second AI / ML model to obtain the ninth CSI output by the second AI / ML model.

7. The method according to claim 6, characterized in that, The step of determining whether the first AI / ML model meets the performance requirements based on the third information includes one or more of the following: Based on the first CSI and the fourth CSI, or the second CSI and the fifth CSI, or the third CSI and the sixth CSI, determine whether the first AI / ML model meets the first performance requirement; Based on the first CSI, the second CSI, the fourth CSI, and the fifth CSI, determine whether the first AI / ML model meets the second performance requirement; Based on the first CSI, the fourth CSI, and the seventh CSI, determine whether the first AI / ML model meets the third performance requirement; Based on the second CSI, the fifth CSI, and the eighth CSI, determine whether the first AI / ML model meets the third performance requirement; Based on the third CSI, the sixth CSI, and the ninth CSI, determine whether the first AI / ML model meets the third performance requirement; Based on the second CSI, the third CSI, the fifth CSI, and the sixth CSI, determine whether the first AI / ML model meets the fourth performance requirement.

8. The method according to claim 7, characterized in that, The determination of the reason for the performance change of the first AI / ML model based on the third information includes one or more of the following: The first AL / ML model does not meet the first performance requirement, and it is determined that the performance of the first AL / ML model has changed due to model deployment. If the first AL / ML model does not meet the second performance requirement or the fourth performance requirement, it is determined that the performance of the first AL / ML model has changed due to changes in actual channel information. The second AL / ML model does not meet the third performance requirement, and it is determined that the performance of the second AL / ML model has changed due to the model update.

9. The method according to any one of claims 1-8, characterized in that, The method includes: Send a fourth message to the terminal, the fourth message including the third message and / or the third CSI, the fourth message being used by the terminal to determine whether the first AI / ML model meets the performance requirements, and / or the reason for the change in the performance of the first AI / ML model.

10. The method according to any one of claims 1-9, characterized in that, The method includes: The terminal sends a fifth message, which includes model parameters of one or more second AI / ML models or one or more second AI / ML models, and / or model parameters of one or more third AI / ML models or one or more third AI / ML models.

11. The method according to any one of claims 1-10, characterized in that, The network device is deployed with multiple of the first AI / ML models, and the method includes: A sixth message is sent to the terminal, the sixth message indicating whether the model parameters of each of the multiple first AL / ML models have been updated.

12. A method for monitoring model performance, characterized in that, The method, executed by a terminal, includes: Send first information, which instructs the network device to input second information into a first artificial intelligence (AI) / machine learning (ML) model and / or a second AI / ML model to obtain third information. The second information includes at least one channel state information (CSI) quantization information, and the third information includes CSI recovered from at least one of the CSI quantization information. The third information is used by the network device to determine whether the first AI / ML model meets the performance requirements, and / or to determine the reason for the performance change of the first AI / ML model; Wherein, the model parameters of the first AL / ML model are the same as those of the second AL / ML model, or the first AL / ML model is obtained by the network device updating the second AL / ML model and / or the third AL / ML model; The second AI / ML model and the third AI / ML model are AI / ML models trained by the terminal. The third AI / ML model is the first part of the bilateral model, and the first AI / ML model and the second AI / ML model are the second part of the bilateral model.

13. The method according to claim 12, characterized in that, The second information includes one or more of the following: The first quantization information is obtained by the terminal or the network device by inputting the first CSI into the third AI / ML model, and the first CSI is a predefined CSI. The second quantification information is obtained by the terminal or the network device inputting the second CSI into the third AI / ML model, and the second CSI is measured by the terminal. The third quantization information is the quantization information corresponding to the third CSI, which is the CSI used by the network device to update the first AI / ML model.

14. The method according to claim 13, characterized in that, The second information is determined by the network device based on the first information.

15. The method according to claim 14, characterized in that, The first information includes one or more of the following; The first CSI; The first quantification information; The second CSI; The second quantification information.

16. The method according to claim 15, characterized in that, The method includes: Receive the first signal sent by the network device; The second CSI is obtained by measuring the first signal.

17. The method according to any one of claims 13-16, characterized in that, The third information includes one or more of the following: The fourth CSI is obtained by the network device by inputting the first quantization information into the first AI / ML model; The fifth CSI is obtained by the network device by inputting the second quantization information into the first AI / ML model; The sixth CSI is obtained by the network device by inputting the third quantization information into the first AI / ML model; The seventh CSI is obtained by the network device by inputting the first quantization information into the second AI / ML model; The eighth CSI is obtained by the network device by inputting the second quantization information into the second AI / ML model; The ninth CSI is obtained by the network device by inputting the third quantization information into the second AI / ML model.

18. The method according to claim 17, characterized in that, The method includes: Receive fourth information sent by the network device, the fourth information including the third information and / or the third CSI; Based on the fourth information, determine whether the first AI / ML model meets the performance requirements, and / or determine the reason for the performance change of the first AI / ML model.

19. The method according to claim 18, characterized in that, The step of determining whether the first AI / ML model meets the performance requirements based on the fourth information includes one or more of the following: Based on the first CSI and the fourth CSI, or the second CSI and the fifth CSI, or the third CSI and the sixth CSI, determine whether the first AI / ML model meets the first performance requirement; Based on the first CSI, the second CSI, the fourth CSI, and the fifth CSI, determine whether the first AI / ML model meets the second performance requirement; Based on the first CSI, the fourth CSI, and the seventh CSI, determine whether the first AI / ML model meets the third performance requirement; Based on the second CSI, the fifth CSI, and the eighth CSI, determine whether the first AI / ML model meets the third performance requirement; Based on the third CSI, the sixth CSI, and the ninth CSI, determine whether the first AI / ML model meets the third performance requirement; Based on the second CSI, the third CSI, the fifth CSI, and the sixth CSI, determine whether the first AI / ML model meets the fourth performance requirement.

20. The method according to claim 19, characterized in that, The determination of the cause of the performance change of the first AI / ML model based on the fourth information includes one or more of the following: The first AL / ML model does not meet the first performance requirement, and it is determined that the performance of the first AL / ML model has changed due to model deployment. If the first AL / ML model does not meet the second performance requirement or the fourth performance requirement, it is determined that the performance of the first AL / ML model has changed due to changes in actual channel information. The second AL / ML model does not meet the third performance requirement, and it is determined that the performance of the second AL / ML model has changed due to the model update.

21. The method according to any one of claims 12-20, characterized in that, The method includes: Send a fifth message to the network device, the fifth message including one or more of the second AI / ML models or one or more The model parameters of the second AI / ML model, and / or, the model parameters of one or more of the third AI / ML models.

22. The method according to any one of claims 12-21, characterized in that, The network device is deployed with multiple of the first AI / ML models, and the method includes: The network device receives a sixth message, which indicates whether the model parameters of each of the multiple first AL / ML models have been updated.

23. A communication device, characterized in that, include: The processing module is configured to input second information into a first AI / ML model and / or a second AI / ML model to obtain third information, wherein the second information includes at least one CSI quantization information, and the third information includes CSI recovered from at least one of the CSI quantization information; The processing module is configured to determine, based on the third information, whether the first AI / ML model meets the performance requirements, and / or to determine the reason for the performance change of the first AI / ML model; Wherein, the model parameters of the first AL / ML model are the same as those of the second AL / ML model, or the first AL / ML model is obtained by the network device updating the second AL / ML model and / or the third AL / ML model; The second AI / ML model and the third AI / ML model are AI / ML models trained by the terminal. The third AI / ML model is the first part of the bilateral model, and the first AI / ML model and the second AI / ML model are the second part of the bilateral model.

24. A communication device, characterized in that, include: The transceiver module is configured to send first information, which instructs the network device to input second information into a first artificial intelligence (AI) / machine learning (ML) model and / or a second AI / ML model to obtain third information. The second information includes at least one channel state information (CSI) quantization information, and the third information includes CSI recovered from at least one of the CSI quantization information. The third information is used by the network device to determine whether the first AI / ML model meets the performance requirements, and / or to determine the reason for the performance change of the first AI / ML model; Wherein, the model parameters of the first AL / ML model are the same as those of the second AL / ML model, or the first AL / ML model is obtained by the network device updating the second AL / ML model and / or the third AL / ML model; The second AI / ML model and the third AI / ML model are AI / ML models trained by the terminal. The third AI / ML model is the first part of the bilateral model, and the first AI / ML model and the second AI / ML model are the second part of the bilateral model.

25. A communication device, characterized in that, include: One or more processors; The communication device is used to execute the model performance monitoring method according to any one of claims 1-11 or any one of claims 12-22.

26. A communication system, characterized in that, The network device includes network equipment and terminals, wherein the network equipment is configured to input second information into a first artificial intelligence (AI) / machine learning (ML) model and / or a second AI / ML model to obtain third information, wherein the second information includes at least one channel state information (CSI) quantization information, and the third information includes CSI recovered from at least one of the CSI quantization information; The network device is configured to determine, based on the third information, whether the first AI / ML model meets the performance requirements, and / or to determine the reason for the performance change of the first AI / ML model; Wherein, the model parameters of the first AL / ML model are the same as those of the second AL / ML model, or the first AL / ML model is obtained by the network device updating the second AL / ML model and / or the third AL / ML model; The second AI / ML model and the third AI / ML model are AI / ML models trained by the terminal. The third AI / ML model is the first part of the bilateral model, and the first AI / ML model and the second AI / ML model are the second part of the bilateral model.

27. A storage medium storing instructions, characterized in that, When the instruction is executed on the communication device, the communication device performs the model performance monitoring method as claimed in any one of claims 1-11 or any one of claims 12-22.

28. A computer program product comprising a computer program and / or instructions, characterized in that, When the computer program and / or the instructions are executed by the communication device, they implement the model performance monitoring method as described in any one of claims 1-11 or any one of claims 12-22.

Images