Model monitoring method, terminal, network equipment and storage medium

Abstract

The invention relates to a model monitoring method, a terminal, network equipment and a storage medium. The model monitoring method comprises the steps that first information is sent when a first condition is met, the first information comprises information used for monitoring the performance of an artificial intelligence AI model, and the first condition is related to the performance of the AI model. Under the condition that the first condition is met, the terminal is triggered to report the measurement result. The terminal is prevented from reporting a large number of measurement results, the signaling overhead is saved, and the monitoring time is shortened.

Description

Model monitoring method, terminal, network device and storage medium Technical Field

[0001] The present disclosure relates to the field of communication technology, and in particular to a model monitoring method, a terminal, a network device, and a storage medium. Background Art

[0002] In related technologies, for new radio networks (NR), especially when the communication frequency band is in frequency range 2, beam-based transmission and reception are required to ensure coverage due to the rapid attenuation of high-frequency channels.

[0003] In some beam management processes, in order to reduce the number of beam pairs measured by the terminal, an artificial intelligence (AI) model can be used to perform beam prediction.

[0004] Summary of the Invention

[0005] For current AI model performance monitoring, terminals need to report a large number of measurement results, resulting in high signaling overhead and long monitoring time.

[0006] The embodiments of the present disclosure provide a model monitoring method, a terminal, a network device, and a storage medium.

[0007] According to a first aspect of an embodiment of the present disclosure, a model monitoring method is proposed, the method comprising: sending first information, the first information comprising information for performance monitoring of an artificial intelligence (AI) model, and the first condition being related to the performance of the AI ​​model.

[0008] According to a second aspect of an embodiment of the present disclosure, a model monitoring method is proposed, the method comprising: receiving first information sent by a terminal, the first information comprising information for artificial intelligence (AI) model performance monitoring.

[0009] According to a third aspect of an embodiment of the present disclosure, a model monitoring method is proposed, which includes: a terminal sending first information to a network device, where the first information includes information for artificial intelligence (AI) model performance monitoring; and the network device receiving the first information sent by the terminal.

[0010] According to a fourth aspect of an embodiment of the present disclosure, a terminal is proposed, including: a transceiver module; the transceiver module is used to send first information, and the first information includes information for artificial intelligence AI model performance monitoring.

[0011] According to a fifth aspect of an embodiment of the present disclosure, a network device is proposed, including: a transceiver module; the transceiver module is used to receive first information sent by a terminal, and the first information includes information for artificial intelligence AI model performance monitoring.

[0012] According to a sixth aspect of an embodiment of the present disclosure, a terminal is proposed, comprising: one or more processors; wherein the terminal is used to execute the first aspect and any one of the model monitoring methods in the first aspect.

[0013] According to a seventh aspect of an embodiment of the present disclosure, a network device is proposed, comprising: one or more processors; wherein the network device is used to execute the second aspect and any one of the model monitoring methods in the second aspect.

[0014] According to the eighth aspect of an embodiment of the present disclosure, a communication system is proposed, comprising a terminal and a network device, wherein the terminal is configured to implement the first aspect and any one of the model monitoring methods in the first aspect, and the network device is configured to implement the first aspect and any one of the model monitoring methods in the first aspect.

[0015] According to the ninth aspect of the embodiment of the present disclosure, a storage medium is proposed, which stores instructions. When the instructions are executed on a communication device, the communication device executes a model monitoring method such as the first aspect and any one of the first aspects or the first aspect and any one of the first aspects.

[0016] The present disclosure triggers the terminal to report the measurement results so that the network device can determine the performance of the AI ​​model and / or AI function based on the measurement results, and then make corresponding adjustments to the AI ​​model and / or AI function. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following drawings required for describing the embodiments are introduced. The following drawings are merely some embodiments of the present disclosure and do not impose specific limitations on the protection scope of the present disclosure.

[0018] FIG1 is a schematic diagram showing a communication system architecture according to an embodiment of the present disclosure.

[0019] FIG2 is an interactive diagram of a model monitoring method according to an embodiment of the present disclosure.

[0020] FIG3 a is a flow chart of a model monitoring method according to an exemplary embodiment.

[0021] FIG3 b is a flow chart of another model monitoring method according to an exemplary embodiment.

[0022] FIG3 c is a flow chart of another model monitoring method according to an exemplary embodiment.

[0023] FIG3 d is a flow chart of yet another model monitoring method according to an exemplary embodiment.

[0024] FIG3 e is a flow chart of another model monitoring method according to an exemplary embodiment.

[0025] FIG3 f is a flow chart of another model monitoring method according to an exemplary embodiment.

[0026] FIG3 g is a flow chart of yet another model monitoring method according to an exemplary embodiment.

[0027] FIG4 a is a flow chart showing another model monitoring method according to an exemplary embodiment.

[0028] FIG4 b is a flow chart showing another model monitoring method according to an exemplary embodiment.

[0029] FIG4 c is a flow chart of yet another model monitoring method according to an exemplary embodiment.

[0030] FIG4 d is a flow chart of another model monitoring method according to an exemplary embodiment.

[0031] FIG4e is a flow chart of another model monitoring method according to an exemplary embodiment.

[0032] FIG4 f is a flow chart of yet another model monitoring method according to an exemplary embodiment.

[0033] FIG5 is a flow chart showing another model monitoring method according to an exemplary embodiment.

[0034] Fig. 6a is a schematic diagram of a model monitoring device according to an exemplary embodiment.

[0035] FIG6 b is a schematic diagram of another model monitoring device according to an exemplary embodiment.

[0036] Fig. 7a is a schematic diagram showing a communication device according to an exemplary embodiment.

[0037] FIG7 b is a schematic diagram of a chip according to an exemplary embodiment. DETAILED DESCRIPTION

[0038] The embodiments of the present disclosure provide a model monitoring method, apparatus, device, and storage medium.

[0039] In a first aspect, an embodiment of the present disclosure proposes a model monitoring method, which includes: sending first information, where the first information includes information for monitoring the performance of an artificial intelligence (AI) model.

[0040] In the above embodiment, the present disclosure triggers the terminal to report the measurement results so that the network device can determine the performance of the AI ​​model and / or AI function based on the measurement results and then make corresponding adjustments to the AI ​​model and / or AI function.

[0041] In combination with some embodiments of the first aspect, the AI ​​model is deployed on a network device; the first information includes at least one of the following: second information corresponding to the beams and / or beam pairs in the first set, the second information is used to represent the first parameters of the beams and / or beam pairs in the first set, the beams and / or beam pairs in the first set correspond to the input of the AI ​​model, and the first parameters include measured identifiers and / or measured values; third information corresponding to the beams and / or beam pairs in the second set, the third information is used to represent the first parameters of the beams and / or beam pairs in the second set, the second set includes the first set, or the first set and the second set are different, the output of the AI ​​model is used to obtain the second parameters of the beams and / or beam pairs in the second set, and the second parameters include predicted identifiers and / or predicted values.

[0042] In the above embodiments, when an AI model is deployed on a network device, the measurement results reported by the terminal include various information. This allows the network device to determine the performance of the AI ​​model and / or AI function based on the first information, perform more reasonable AI operations, and improve model operation efficiency.

[0043] In combination with some embodiments of the first aspect, the AI ​​model is deployed on the terminal; the first information includes at least one of the following: fourth information, the fourth information is used to indicate the AI ​​operation on the AI ​​model; third information and fifth information corresponding to the beams and / or beam pairs in the second set, wherein the third information is used to represent the first parameters of the beams and / or beam pairs in the second set, and the fifth information is used to represent the second parameters of the beams and / or beam pairs in the second set, and the fifth information is determined based on the output of the AI ​​model, the first parameter includes the measured identifier and / or measured value, and the second parameter includes the predicted identifier and / or predicted value; the first indicator, the first indicator is used to represent the prediction accuracy of the second number of beams and / or beam pairs.

[0044] In the above embodiments, when an AI model is deployed on a terminal, various information is provided in the measurement results reported by the terminal. This allows the network device to determine the performance of the AI ​​model and / or AI function based on the first information, perform more reasonable AI operations, and improve model operation efficiency.

[0045] In combination with some embodiments of the first aspect, sending the first information includes at least one of the following: when the second parameter obtained by the AI ​​model output N times meets the first condition, the first information is sent, wherein the first condition is used to indicate that the AI ​​model prediction is accurate; when the second parameter obtained by the AI ​​model output M times meets the second condition, the first information is sent, wherein the second condition is used to indicate that the AI ​​model prediction is inaccurate.

[0046] In the above embodiment, multiple situations are provided for triggering the terminal to send the first information, thereby preventing the terminal from frequently reporting a large number of measurement results, thereby reducing signaling overhead and shortening detection time.

[0047] In combination with some embodiments of the first aspect, sending the first information also includes at least one of the following: within the first time, the second parameter obtained by the AI ​​model output N times meets the first condition, and the first information is sent; within the first time, the second parameter obtained by the AI ​​model output M times meets the second condition, and the first information is sent.

[0048] In the above embodiment, multiple situations are provided for triggering the terminal to send the first information, thereby preventing the terminal from frequently reporting a large number of measurement results, thereby reducing signaling overhead and shortening detection time.

[0049] In combination with some embodiments of the first aspect, the first condition includes at least one of the following: the K target beams and / or beam pairs obtained by the AI ​​model output include the L target beams and / or beam pairs measured in the second set, and the L target beams and / or beam pairs are the first L beams and / or beam pairs in the second set sorted from high to low according to the measurement value; the 1 target beam and / or beam pair obtained by the AI ​​model output is among the L target beams and / or beam pairs measured in the second set; the measurement value corresponding to the predicted identifier of the 1 target beam and / or beam pair obtained by the AI ​​model output is different from the measurement value of the 1 target beam and / or beam pair measured in the second set. is less than or equal to the first threshold; the second difference between the predicted value corresponding to the predicted identifier of one target beam and / or beam pair obtained by the AI ​​model output and the measured value corresponding to the predicted identifier is less than or equal to the second threshold; the throughput difference is greater than or equal to the third threshold, wherein the throughput difference represents the difference between the throughput corresponding to one beam and / or beam pair among the K target beams and / or beam pairs obtained based on the AI ​​model output and the throughput corresponding to one beam and / or beam pair among the L target beams and / or beam pairs in the second set; the reference signal resource overhead sent by the network device is less than or equal to the fourth threshold; the overhead of the second information is less than or equal to the fifth threshold.

[0050] In the above embodiment, multiple situations are provided for triggering the terminal to send the first information, thereby preventing the terminal from frequently reporting a large number of measurement results, thereby reducing signaling overhead and shortening detection time.

[0051] In combination with some embodiments of the first aspect, the second condition includes at least one of the following: the K target beams and / or beam pairs obtained by the AI ​​model output do not include the L target beams and / or beam pairs measured in the second set, and the L target beams and / or beam pairs are the first L beams and / or beam pairs in the second set sorted from high to low according to the measurement value; the 1 target beam and / or beam pair obtained by the AI ​​model output is not among the L target beams and / or beam pairs measured in the second set; the measurement value corresponding to the predicted identifier of the 1 target beam and / or beam pair obtained by the AI ​​model output is the same as the 1 target beam and / or beam pair measured in the second set. The first difference between the measured values ​​of the beam pairs is greater than the first threshold; the second difference between the predicted value corresponding to the predicted identifier of a target beam and / or beam pair obtained by the AI ​​model output and the measured value corresponding to the predicted identifier is greater than the second threshold; the throughput difference is less than the third threshold, wherein the throughput difference represents the difference between the throughput corresponding to one beam and / or beam pair among the K target beams and / or beam pairs obtained based on the AI ​​model output and the throughput corresponding to one beam and / or beam pair among the L target beams and / or beam pairs in the second set; the reference signal resource overhead sent by the network device is greater than the fourth threshold; the overhead of the second information is greater than the fifth threshold.

[0052] In the above embodiment, multiple situations are provided for triggering the terminal to send the first information, thereby preventing the terminal from frequently reporting a large number of measurement results, thereby reducing signaling overhead and shortening detection time.

[0053] In combination with some embodiments of the first aspect, the AI ​​model is deployed on a network device; sending the first information includes: periodically sending the first information.

[0054] In the above embodiment, the first information may also be sent periodically to avoid the terminal from frequently reporting a large number of measurement results, thereby reducing signaling overhead and shortening detection time.

[0055] In combination with some embodiments of the first aspect, the first information is carried by at least one of radio resource control RRC signaling, uplink media access control element UL MAC CE, and uplink control information UCI.

[0056] In the above embodiments, multiple carrying modes of the first information are provided to be applicable to various scenarios and improve universality.

[0057] In combination with some embodiments of the first aspect, the method further includes: sending sixth information, where the sixth information is used to request monitoring of the AI ​​model.

[0058] In the above embodiment, the terminal can request the network device to monitor the AI ​​model, and can more flexibly configure when the terminal performs AI model monitoring.

[0059] In combination with some embodiments of the first aspect, the method also includes at least one of the following: determining the first information at a second time after sending the sixth information; determining the first information at a third time after receiving the seventh information, wherein the seventh information is feedback from the network device based on the sixth information; determining the first information after the network device sends a reference signal for AI model monitoring.

[0060] In the above embodiment, multiple ways of determining the first information are provided, and then the first information is sent based on different ways, which can avoid the terminal from frequently reporting a large number of measurement results, thereby reducing signaling overhead and shortening detection time.

[0061] In combination with some embodiments of the first aspect, the sixth information is carried in at least one of the following ways: a scheduling request SR; an uplink media access control UL MAC; or a random access procedure RA.

[0062] In the above embodiment, multiple carrying modes of the sixth information are provided to be applicable to various scenarios and improve universality.

[0063] In combination with some embodiments of the first aspect, the method further includes: receiving sixth information, where the sixth information is used to request monitoring of the AI ​​model.

[0064] In the above embodiment, the network device can request the terminal to monitor the AI ​​model, and can more flexibly configure when the terminal performs AI model monitoring.

[0065] In combination with some embodiments of the first aspect, the method further includes: determining the first information at a fourth time after receiving the sixth information.

[0066] In the above embodiment, a method for determining the first information is provided, and the first information is sent based on the method, which can avoid the terminal from frequently reporting a large number of measurement results, thereby reducing signaling overhead and shortening detection time.

[0067] In combination with some embodiments of the first aspect, the sixth information is carried in at least one of the following ways: RRC; downlink media access control element DL MAC CE; paging; downlink control information DCI.

[0068] In the above embodiment, multiple carrying modes of the sixth information are provided to be applicable to various scenarios and improve universality.

[0069] In combination with some embodiments of the first aspect, one AI model corresponds to one or more AI functions, and / or one AI function corresponds to one or more AI models.

[0070] In the above embodiments, it can be applied to monitoring in AI models and / or AI function scenarios to improve universality.

[0071] In combination with some embodiments of the first aspect, the AI ​​model is deployed on the terminal; the method also includes: receiving eighth information, where the eighth information is used to instruct the terminal to perform AI operations on the AI ​​model.

[0072] In the above embodiment, the network device can determine the appropriate AI operation to be performed on the AI ​​model based on the measurement results reported by the terminal, thereby improving the operating efficiency of the AI ​​model.

[0073] In combination with some embodiments of the first aspect, the AI ​​operation includes at least one of the following: activating the AI ​​model and / or AI function; deactivating the AI ​​model and / or AI function; switching the AI ​​model and / or AI function; updating the AI ​​model and / or AI function; switching to non-AI mode and / or non-AI function.

[0074] In the above embodiments, a variety of possible AI operations are provided to accommodate various adjustments to AI models and / or AI functions. By adjusting and adopting a more appropriate AI model, the operating efficiency of the AI ​​model can be improved.

[0075] In combination with some embodiments of the first aspect, the method further includes: receiving ninth information, where the ninth information is used to configure reference signal resources, and the reference signal resources are used to perform beam measurement.

[0076] In the above embodiment, the network device can instruct the terminal which reference signal resources to measure to achieve monitoring of the AI ​​model.

[0077] In combination with some embodiments of the first aspect, the measured value and / or predicted value includes at least one of the following: layer 1 reference signal received power L1-RSRP; layer 1 signal to interference and noise ratio L1-SINR.

[0078] In the above embodiments, a variety of beam measurement parameter conditions are provided to adapt to AI model monitoring in different scenarios and improve universality.

[0079] According to a second aspect of an embodiment of the present disclosure, a model monitoring method is provided, the method comprising: receiving first information sent by a terminal, the first information comprising information for artificial intelligence (AI) model performance monitoring.

[0080] In the above embodiment, the present disclosure triggers the terminal to report the measurement results so that the network device can determine the performance of the AI ​​model and / or AI function based on the measurement results, and then make corresponding adjustments to the AI ​​model and / or AI function.

[0081] In combination with some embodiments of the second aspect, the AI ​​model is deployed on a network device; the first information includes at least one of the following: second information corresponding to the beams and / or beam pairs in the first set, the second information is used to represent the first parameters of the beams and / or beam pairs in the first set, the beams and / or beam pairs in the first set correspond to the input of the AI ​​model, and the first parameters include measured identifiers and / or measured values; third information corresponding to the beams and / or beam pairs in the second set, the third information is used to represent the first parameters of the beams and / or beam pairs in the second set, the second set includes the first set, or the first set and the second set are different, the output of the AI ​​model is used to obtain the second parameters of the beams and / or beam pairs in the second set, and the second parameters include predicted identifiers and / or predicted values.

[0082] In combination with some embodiments of the second aspect, the AI ​​model is deployed on the terminal; the first information includes at least one of the following: fourth information, the fourth information is used to indicate the AI ​​operation on the AI ​​model; third information and fifth information corresponding to the beams and / or beam pairs in the second set, wherein the third information is used to represent the first parameters of the beams and / or beam pairs in the second set, and the fifth information is used to represent the second parameters of the beams and / or beam pairs in the second set, and the fifth information is determined based on the output of the AI ​​model, the first parameter includes the measured identifier and / or measured value, and the second parameter includes the predicted identifier and / or predicted value; the first indicator, the first indicator is used to represent the prediction accuracy of the second number of beams and / or beam pairs.

[0083] In combination with some embodiments of the second aspect, the first information sent by the receiving terminal includes at least one of the following: the second parameter obtained by the AI ​​model output N times meets the first condition, and the first information is sent by the receiving terminal, wherein the first condition is used to indicate that the AI ​​model prediction is accurate; the second parameter obtained by the AI ​​model output M times meets the second condition, and the first information is sent by the receiving terminal, wherein the second condition is used to indicate that the AI ​​model prediction is inaccurate.

[0084] In combination with some embodiments of the second aspect, the first information sent by the receiving terminal also includes at least one of the following: within the first time, the second parameter obtained by the AI ​​model output N times meets the first condition, and the first information is sent by the receiving terminal; within the first time, the second parameter obtained by the AI ​​model output M times meets the second condition, and the first information is sent by the receiving terminal.

[0085] In combination with some embodiments of the second aspect, the first condition includes at least one of the following: the K target beams and / or beam pairs obtained by the AI ​​model output include the L target beams and / or beam pairs measured in the second set, and the L target beams and / or beam pairs are the first L beams and / or beam pairs in the second set sorted from high to low according to the measurement value; the 1 target beam and / or beam pair obtained by the AI ​​model output is among the L target beams and / or beam pairs measured in the second set; the measurement value corresponding to the predicted identifier of the 1 target beam and / or beam pair obtained by the AI ​​model output is different from the measurement value of the 1 target beam and / or beam pair measured in the second set. is less than or equal to the first threshold; the second difference between the predicted value corresponding to the predicted identifier of one target beam and / or beam pair obtained by the AI ​​model output and the measured value corresponding to the predicted identifier is less than or equal to the second threshold; the throughput difference is greater than or equal to the third threshold, wherein the throughput difference represents the difference between the throughput corresponding to one beam and / or beam pair among the K target beams and / or beam pairs obtained based on the AI ​​model output and the throughput corresponding to one beam and / or beam pair among the L target beams and / or beam pairs in the second set; the reference signal resource overhead sent by the network device is less than or equal to the fourth threshold; the overhead of the second information is less than or equal to the fifth threshold.

[0086] In combination with some embodiments of the second aspect, the second condition includes at least one of the following: the K target beams and / or beam pairs obtained by the AI ​​model output do not include the L target beams and / or beam pairs measured in the second set, and the L target beams and / or beam pairs are the first L beams and / or beam pairs in the second set sorted from high to low according to the measurement value; the 1 target beam and / or beam pair obtained by the AI ​​model output is not among the L target beams and / or beam pairs measured in the second set; the measurement value corresponding to the predicted identifier of the 1 target beam and / or beam pair obtained by the AI ​​model output is the same as the 1 target beam and / or beam pair measured in the second set. The first difference between the measured values ​​of the beam pairs is greater than the first threshold; the second difference between the predicted value corresponding to the predicted identifier of a target beam and / or beam pair obtained by the AI ​​model output and the measured value corresponding to the predicted identifier is greater than the second threshold; the throughput difference is less than the third threshold, wherein the throughput difference represents the difference between the throughput corresponding to one beam and / or beam pair among the K target beams and / or beam pairs obtained based on the AI ​​model output and the throughput corresponding to one beam and / or beam pair among the L target beams and / or beam pairs in the second set; the reference signal resource overhead sent by the network device is greater than the fourth threshold; the overhead of the second information is greater than the fifth threshold.

[0087] In combination with some embodiments of the second aspect, the AI ​​model is deployed on a network device; receiving the first information sent by the terminal when the first condition is met includes: periodically receiving the first information sent by the terminal.

[0088] In combination with some embodiments of the second aspect, the first information is carried by at least one of radio resource control RRC signaling, uplink media access control element UL MAC CE, and uplink control information UCI.

[0089] In combination with some embodiments of the second aspect, the method further includes: receiving sixth information, where the sixth information is used to request monitoring of the AI ​​model.

[0090] In combination with some embodiments of the second aspect, the first information sent by the receiving terminal under the first condition includes at least one of the following: receiving the first information after the second time after receiving the sixth information; receiving the first information after the third time after sending the seventh information, wherein the seventh information is feedback from the network device based on the sixth information; receiving the first information after sending a reference signal for AI model monitoring.

[0091] In combination with some embodiments of the second aspect, the sixth information is carried in at least one of the following ways: scheduling request SR; uplink media access control UL MAC; random access process RA.

[0092] In combination with some embodiments of the second aspect, the method further includes: sending sixth information, where the sixth information is used to request monitoring of the AI ​​model.

[0093] In combination with some embodiments of the second aspect, receiving the first information sent by the terminal when the first condition is met includes: receiving the first information after a fourth time after sending the sixth information.

[0094] In combination with some embodiments of the second aspect, the sixth information is carried in at least one of the following ways: RRC; downlink media access control element DL MAC CE; paging; downlink control information DCI.

[0095] In combination with some embodiments of the second aspect, the method also includes: obtaining a second indicator based on the first information, the second indicator is used to describe the performance of the AI ​​model and / or AI function; the second indicator includes at least one of the following: prediction accuracy of the AI ​​model and / or AI function; layer 1 reference signal received power L1-RSRP average difference; L1-RSRP difference; throughput difference; sent reference signal overhead; received UCI overhead.

[0096] In the above embodiment, the network device can determine the model performance index based on the terminal measurement results, so as to determine the corresponding AI operation to be performed on the AI ​​model and / or AI function based on the index, thereby improving the accuracy of model monitoring and the efficiency of model operation.

[0097] In combination with some embodiments of the second aspect, one AI model corresponds to one or more AI functions, and / or one AI function corresponds to one or more AI models.

[0098] In combination with some embodiments of the second aspect, the AI ​​model is deployed on the terminal; the method also includes: sending eighth information, where the eighth information is used to instruct the terminal to perform AI operations on the AI ​​model.

[0099] In combination with some embodiments of the second aspect, the AI ​​operation includes at least one of the following: activating the AI ​​model and / or AI function; deactivating the AI ​​model and / or AI function; switching the AI ​​model and / or AI function; updating the AI ​​model and / or AI function; switching to non-AI mode and / or non-AI function.

[0100] In combination with some embodiments of the second aspect, the method further includes: sending ninth information, where the ninth information is used to configure reference signal resources, and the reference signal resources are used to perform beam measurement.

[0101] In combination with some embodiments of the second aspect, the measured value and / or predicted value includes at least one of the following: L1-RSRP; L1-SINR.

[0102] According to a third aspect of an embodiment of the present disclosure, a model monitoring method is provided, the method comprising: a terminal sending first information to a network device, the first information comprising information for artificial intelligence (AI) model performance monitoring; and the network device receiving the first information sent by the terminal.

[0103] In the above embodiment, the present disclosure triggers the terminal to report the measurement results so that the network device can determine the performance of the AI ​​model and / or AI function based on the measurement results, and then make corresponding adjustments to the AI ​​model and / or AI function.

[0104] According to a fourth aspect of an embodiment of the present disclosure, a terminal is provided, including: a transceiver module; the transceiver module is used to send first information, and the first information includes information for artificial intelligence AI model performance monitoring.

[0105] In the above embodiment, the present disclosure triggers the terminal to report the measurement results so that the network device can determine the performance of the AI ​​model and / or AI function based on the measurement results and then make corresponding adjustments to the AI ​​model and / or AI function.

[0106] According to a fifth aspect of an embodiment of the present disclosure, a network device is provided, including: a transceiver module; the transceiver module is used to receive first information sent by a terminal, and the first information includes information for artificial intelligence AI model performance monitoring.

[0107] In the above embodiment, the present disclosure triggers the terminal to report the measurement results so that the network device can determine the performance of the AI ​​model and / or AI function based on the measurement results, and then make corresponding adjustments to the AI ​​model and / or AI function.

[0108] According to a sixth aspect of an embodiment of the present disclosure, a terminal is provided, comprising: one or more processors; wherein the terminal is configured to execute the first aspect and any one of the model monitoring methods in the first aspect.

[0109] In the above embodiment, the present disclosure triggers the terminal to report the measurement results so that the network device can determine the performance of the AI ​​model and / or AI function based on the measurement results, and then make corresponding adjustments to the AI ​​model and / or AI function.

[0110] According to a seventh aspect of an embodiment of the present disclosure, a network device is provided, comprising: one or more processors; wherein the network device is used to execute the second aspect and any one of the model monitoring methods in the second aspect.

[0111] In the above embodiment, the present disclosure triggers the terminal to report the measurement results so that the network device can determine the performance of the AI ​​model and / or AI function based on the measurement results, and then make corresponding adjustments to the AI ​​model and / or AI function.

[0112] According to the eighth aspect of an embodiment of the present disclosure, a communication system is provided, comprising a terminal and a network device, wherein the terminal is configured to implement the first aspect and any one of the model monitoring methods in the first aspect, and the network device is configured to implement the first aspect and any one of the model monitoring methods in the first aspect.

[0113] In the above embodiment, the present disclosure triggers the terminal to report the measurement results so that the network device can determine the performance of the AI ​​model and / or AI function based on the measurement results, and then make corresponding adjustments to the AI ​​model and / or AI function.

[0114] According to the ninth aspect of an embodiment of the present disclosure, a storage medium is provided, which stores instructions. When the instructions are executed on a communication device, the communication device executes a model monitoring method such as the first aspect and any one of the first aspects or the first aspect and any one of the first aspects.

[0115] In the above embodiment, the present disclosure triggers the terminal to report the measurement results so that the network device can determine the performance of the AI ​​model and / or AI function based on the measurement results, and then make corresponding adjustments to the AI ​​model and / or AI function.

[0116] According to a tenth aspect of an embodiment of the present disclosure, an embodiment of the present disclosure proposes a program product. When the program product is executed by a communication device, the communication device executes the method described in the optional implementation manner of the first aspect or the second aspect.

[0117] According to an eleventh aspect of the embodiments of the present disclosure, the embodiments of the present disclosure propose a computer program, which, when executed on a computer, enables the computer to execute the method described in the optional implementation of the first aspect or the second aspect.

[0118] According to a twelfth aspect of the embodiments of the present disclosure, the embodiments of the present disclosure provide a chip or a chip system, wherein the chip or chip system includes a processing circuit configured to execute the method described in the optional implementation of the first or second aspect.

[0119] It is understandable that the terminal, access network device, first network element, other network elements, core network device, communication system, storage medium, program product, computer program, chip, or chip system involved in each embodiment of the present disclosure are all used to perform the method proposed in the embodiment of the present disclosure. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding method and will not be repeated here.

[0120] The present disclosure provides a model monitoring method, apparatus, device, and storage medium. In some embodiments, the terms "model monitoring method" and "information processing method" and "communication method" are interchangeable; the terms "model monitoring apparatus" and "information processing apparatus" and "communication apparatus" are interchangeable; and the terms "information processing system" and "communication system" are interchangeable.

[0121] The embodiments of the present disclosure are not exhaustive and are merely illustrative of some embodiments, and are not intended to be a specific limitation on the scope of protection of the present disclosure. In the absence of contradiction, each step in a certain embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a certain embodiment can also be implemented as an independent embodiment, and the order of the steps in a certain embodiment can be arbitrarily exchanged. In addition, the optional implementation methods in a certain embodiment can be arbitrarily combined; in addition, the embodiments can be arbitrarily combined. For example, some or all steps of different embodiments can be arbitrarily combined, and a certain embodiment can be arbitrarily combined with the optional implementation methods of other embodiments.

[0122] In each embodiment of the present disclosure, unless otherwise specified or provided for by logic, the terms and / or descriptions between the embodiments are consistent and can be referenced by each other. The technical features in different embodiments can be combined to form a new embodiment based on their inherent logical relationships.

[0123] The terms used in the embodiments of the present disclosure are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure.

[0124] In the embodiments of the present disclosure, unless otherwise specified, elements expressed in the singular, such as "a", "an", "the", "above", "said", "the", "the", etc., may mean "one and only one", or "one or more", "at least one", etc. For example, when using articles such as "a", "an", "the" in English in translation, the noun following the article may be understood as a singular expression or a plural expression.

[0125] In the embodiments of the present disclosure, “plurality” refers to two or more.

[0126] In some embodiments, the terms "at least one," "one or more," "a plurality of," "multiple," etc. may be used interchangeably.

[0127] In some embodiments, descriptions such as "at least one of A and B," "A and / or B," "A in one case, B in another case," or "in response to one case A, in response to another case B" may include the following technical solutions depending on the situation: in some embodiments, A (A is executed independently of B); in some embodiments, B (B is executed independently of A); in some embodiments, execution is selected from A and B (A and B are selectively executed); and in some embodiments, A and B (both A and B are executed). The above is also applicable when there are more branches such as A, B, and C.

[0128] In some embodiments, "A or B" and other descriptions may include the following technical solutions depending on the situation: in some embodiments, A (A is executed independently of B); in some embodiments, B (B is executed independently of A); in some embodiments, execution is selected from A and B (A and B are selectively executed). The above is similar when there are more branches such as A, B, C, etc.

[0129] The prefixes such as "first" and "second" in the embodiments of the present disclosure are only used to distinguish different description objects and do not constitute any restriction on the position, order, priority, quantity or content of the description objects. For the statement of the description object, please refer to the description in the context of the claims or embodiments, and no unnecessary restriction should be constituted due to the use of prefixes. For example, if the description object is a "field", the ordinal number before the "field" in the "first field" and the "second field" does not limit the position or order between the "fields". "First" and "second" do not limit whether the "fields" they modify are in the same message, nor do they limit the order of the "first field" and the "second field". For another example, if the description object is a "level", the ordinal number before the "level" in the "first level" and the "second level" does not limit the priority between the "levels". For another example, the number of description objects is not limited by the ordinal number and can be one or more. Taking "first device" as an example, the number of "devices" can be one or more. In addition, the objects modified by different prefixes can be the same or different. For example, if the description object is "device", then the "first device" and the "second device" can be the same device or different devices, and their types can be the same or different; for another example, if the description object is "information", then the "first information" and the "second information" can be the same information or different information, and their contents can be the same or different.

[0130] In some embodiments, “including A,” “comprising A,” “used to indicate A,” and “carrying A” can be interpreted as directly carrying A or indirectly indicating A.

[0131] In some embodiments, terms such as "in response to...", "in response to determining...", "in the case of...", "at the time of...", "when...", "if...", "if...", etc. can be used interchangeably.

[0132] In some embodiments, terms such as "greater than", "greater than or equal to", "not less than", "more than", "more than or equal to", "not less than", "higher than", "higher than or equal to", "not less than", and "above" can be replaced with each other, and terms such as "less than", "less than or equal to", "not greater than", "less than", "less than or equal to", "not more than", "lower than", "lower than or equal to", "not higher than", and "below" can be replaced with each other.

[0133] In some embodiments, devices and equipment can be interpreted as physical or virtual, and their names are not limited to the names recorded in the embodiments. In some cases, they can also be understood as "equipment", "device", "circuit", "network element", "node", "function", "unit", "section", "system", "network", "chip", "chip system", "entity", "subject", etc.

[0134] In some embodiments, "network" can be interpreted as devices included in the network, such as access network equipment, core network equipment, etc.

[0135] In some embodiments, "access network device (AN device)" may also be referred to as "radio access network device (RAN device)", "base station (BS)", "radio base station", "fixed station", and in some embodiments may also be understood as "node", "access point", "transmission point (TP)", "reception point (RP)", "transmission and / or reception point (TRP)" "panel", "antenna panel", "antenna array", "cell", "macro cell", "small cell", "femto cell", "pico cell", "sector", "cell group", "serving cell", "carrier", "component carrier", "bandwidth part (BWP)", etc.

[0136] In some embodiments, "terminal" or "terminal device" may be referred to as "user equipment (UE)", "user terminal" "mobile station (MS)", "mobile terminal (MT)", subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, etc.

[0137] In some embodiments, obtaining data, information, etc. may comply with the laws and regulations of the country where the data is obtained.

[0138] In some embodiments, data, information, etc. may be obtained with the user's consent.

[0139] In addition, each element, each row, or each column in the table of the embodiment of the present disclosure can be implemented as an independent embodiment, and the combination of any elements, any rows, and any columns can also be implemented as an independent embodiment.

[0140] FIG1 is a schematic diagram showing a communication system architecture according to an embodiment of the present disclosure.

[0141] As shown in FIG1 , a communication system 100 includes a terminal 101 and a network device 102 .

[0142] In some embodiments, the terminal 101 includes, for example, a mobile phone, a wearable device, an Internet of Things device, a car with communication function, a smart car, a tablet computer, a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal device in industrial control, a wireless terminal device in self-driving, a wireless terminal device in remote medical surgery, a wireless terminal device in a smart grid, a wireless terminal device in transportation safety, a wireless terminal device in a smart city, and at least one of a wireless terminal device in a smart home, but is not limited thereto.

[0143] In some embodiments, the network device 102 may include at least one of an access network device and a core network device.

[0144] In some embodiments, the access network device is, for example, a node or device that accesses a terminal to a wireless network. The access network device may include an evolved NodeB (eNB), a next generation evolved NodeB (ng-eNB), a next generation NodeB (gNB), a node B (NB), a home node B (HNB), a home evolved nodeB (HeNB), a wireless backhaul device, a radio network controller (RNC), a base station controller (BSC), a base transceiver station (BTS), a base band unit (BBU), a mobile switching center, a base station in a 6G communication system, an open base station (Open RAN), a cloud base station (Cloud RAN), a base station in other communication systems, and at least one of an access node in a Wi-Fi system, but is not limited thereto.

[0145] In some embodiments, the technical solution of the present disclosure can be applied to the Open RAN architecture. In this case, the interfaces between or within the access network devices involved in the embodiments of the present disclosure can be transformed into internal interfaces of the Open RAN, and the processes and information interactions between these internal interfaces can be implemented through software or programs.

[0146] In some embodiments, the access network device can be composed of a centralized unit (CU) and a distributed unit (DU), where the CU can also be called a control unit. The CU-DU structure can be used to split the protocol layer of the access network device, with the functions of some protocol layers centrally controlled by the CU, and the functions of the remaining part or all of the protocol layers distributed in the DU, which is centrally controlled by the CU, but is not limited to this.

[0147] In some embodiments, a core network device may be a device including one or more network elements, or may be multiple devices or device groups, each including all or part of the one or more network elements. The network element may be virtual or physical. The core network may include, for example, at least one of an Evolved Packet Core (EPC), a 5G Core Network (5GCN), and a Next Generation Core (NGC).

[0148] It can be understood that the communication system described in the embodiment of the present disclosure is for the purpose of more clearly illustrating the technical solution of the embodiment of the present disclosure, and does not constitute a limitation on the technical solution proposed in the embodiment of the present disclosure. Ordinary technicians in this field can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solution proposed in the embodiment of the present disclosure is also applicable to similar technical problems.

[0149] The following embodiments of the present disclosure may be applied to the communication system 100 shown in FIG1 , or a portion thereof, but are not limited thereto. The entities shown in FIG1 are illustrative only. The communication system may include all or part of the entities shown in FIG1 , or may include other entities outside of FIG1 . The number and form of the entities are arbitrary, and the entities may be physical or virtual. The connection relationships between the entities are illustrative only. The entities may be connected or disconnected, and the connection may be in any manner, including direct or indirect, wired or wireless.

[0150] The embodiments of the present disclosure can be applied to Long Term Evolution (LTE), LTE-Advanced (LTE-A), LTE-Beyond (LTE-B), SUPER 3G, IMT-Advanced, 4th generation mobile communication system (4G), 5th generation mobile communication system (5G), 5G new radio (NR), future radio access (FRA), new radio access technology (RAT), new radio (NR), new radio access (NX), future generation radio access (FX), Global System for Mobile communications (GSM (registered trademark)), CDMA2000, Ultra Mobile Broadband (UMB), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.20, Ultra-WideBand (UWB), Bluetooth (registered trademark), Public Land Mobile Network (PLMN) networks, Device-to-Device (D2D) systems, Machine-to-Machine (M2M) systems, Internet of Things (IoT) systems, Vehicle-to-Everything (V2X), systems utilizing other communication methods, and next-generation systems based on and extending these methods. Furthermore, multiple systems may be combined (for example, a combination of LTE or LTE-A with 5G).

[0151] In the disclosed embodiments, in NR, especially when the communication frequency band is in frequency range 2, since the high-frequency channel attenuates rapidly, beam-based transmission and reception are required to ensure coverage.

[0152] In some traditional beam management processes, network equipment configures a set of reference signal resources for beam measurement. The terminal measures the reference signal resources in this set and reports the IDs of some of the stronger reference signal resources, along with the corresponding layer 1 reference signal received power (L1-RSRP) and / or layer 1 signal to interference plus noise ratio (L1-SINR), to the network equipment.

[0153] In related art, it is assumed that the reference signal resource set configured by a network device includes X reference signals, each corresponding to a different transmit beam of the network device. For each reference signal, the terminal needs to use all receive beams to measure that reference signal. This results in the beam measurement quality corresponding to all receive beams. In some cases, one or more best beam measurement qualities and / or the beam identifier corresponding to the best beam measurement quality can be determined. Therefore, the number of beam pairs that the terminal needs to measure is G*H. Here, G represents the number of transmit beams of the network device, and H represents the number of receive beams of the terminal.

[0154] In some embodiments, an AI model and / or AI function is provided to implement beam prediction, wherein an AI function can be considered as one or more AI models that implement the same function and purpose.

[0155] In some embodiments, the AI ​​model used for beam prediction can be referred to as a beam prediction model. Of course, it can also be referred to as a beam prediction AI model, a prediction AI model, a predicted beam model, etc. This disclosure does not limit the names of such AI models.

[0156] In some implementations, when the beam prediction model uses spatial prediction, the terminal measures the L1-RSRP of set B and inputs the measured value into the beam prediction model. The beam prediction model can predict the L1-RSRP of the best beam and / or beam pair in set A, and / or the identifier of the best beam and / or beam pair in set A.

[0157] The relationship between set B and set A includes the following two types:

[0158] The first relationship is that set B is a subset of set A. For example, if set A contains 32 reference signals (each reference signal corresponds to a beam direction), then set B contains some of them, for example, 8 of the 32 reference signals.

[0159] The second relationship is that set B is a wide beam and set A is a narrow beam. For example, set A contains 32 reference signals (each reference signal corresponds to a beam direction, and the 32 reference signals cover a 120-degree direction). Set B contains another Y reference signals, for example, Y = 8. These Y reference signals also cover a 120-degree direction, that is, the beam direction of each reference signal in set B covers the beam directions of multiple reference signals in set A. It can be understood that the 32 / Y reference signals in set A and the same reference signal in set B are in a QCL Type D relationship.

[0160] It should be understood that the examples of the first and second relationships described above only describe the transmit beam case. When considering beam pairs consisting of transmit and receive beams, the terminal's receive beams must also be considered. For example, if there are 32 transmit beams and 4 receive beams, then set A consists of 32*4 beam pairs, and set B can consist of 32 beam pairs, 16 beam pairs, and so on.

[0161] In some embodiments, if the performance of the beam prediction model does not need to be monitored, assuming the AI ​​model has been trained in advance, the network device only needs to periodically send the reference signals of set B (for example, the first period). The terminal then measures the L1-RSRP of the reference signals in set B and inputs it into the beam prediction model. This outputs the L1-RSRP of set A or the strongest reference signal ID or IDs of one or more of the 32 reference signals in set A.

[0162] In some embodiments, if the performance of the beam prediction model needs to be monitored, the network device is required to periodically send the reference signal of set A (for example, the second period). Then the terminal needs to measure the L1-RSRP of the reference signal of set B. For the terminal side model, the terminal inputs the measured L1-RSRP of the reference signal of set B into the beam prediction model, obtains the predicted beam information, and reports it to the network device; for the network side model, the terminal reports the measured L1-RSRP and / or identifier of the reference signal of set B to the network device, and the network device inputs the L1-RSRP and / or identifier of the reference signal of set B into the beam prediction model to obtain the predicted beam information. At the same time, the terminal also measures the L1-RSRP of all reference signals in set A, and reports the measured L1-RSRP and / or identifier of all reference signals in set A, and / or the L1-RSRP and / or identifier of the best L beams / beam pairs as the beam information obtained by the traditional method to the base station.

[0163] In various embodiments of the present disclosure, the terms "terminal-side model" and "AI model deployed on the terminal" can be used interchangeably. The terms "network device-side model" and "AI model deployed on the network device" can be used interchangeably.

[0164] If set B is a subset of set A, the terminal may only report the beam information of all beams or beam pairs of set A.

[0165] In some embodiments, the second period is greater than the first period, or the second period is a multiple of the first period, or how much greater the second period is than the first period is not limited in this disclosure. Of course, the second period can also be less than or equal to the first period, which is not limited in this disclosure.

[0166] In some implementations, when the beam prediction model uses time-domain prediction, the terminal measures the L1-RSRP at a historical time point, set B, and inputs this into the beam prediction model to predict the L1-RSRP at a future time point, set A. In addition to the two aforementioned relationships between set B and set A, there is another relationship in which set B and set A are identical.

[0167] When beam measurement is performed using methods described in related technologies, the base station also needs to transmit reference signals at future times. The terminal measures the reference signals at future times and reports the measured beam information to the network device. Specifically, the terminal needs to measure and report the beam information for all beams and / or beam pairs in set A to the network device.

[0168] In some related technologies, in order to reduce the number of beam pairs measured by the terminal, an AI model is used for beam prediction. For example, the terminal originally needs to measure a total of G*H beam pairs. When using the AI ​​model for beam prediction, for spatial domain beam prediction, the terminal only needs to measure a part of the G*H beam pairs. For example, measure 1 / 8, 1 / 4 of G*H beam pairs, etc. Then, the beam measurement quality of these beam pairs measured is input into the AI ​​model, and the AI ​​model can output the beam information of G*H beam pairs. For time domain beam prediction, the terminal can measure the beam measurement quality of beam pairs at historical times to predict the beam information of beam pairs at future times.

[0169] Of course, for the input and output of some AI models, it is also possible to not consider the beam quality or beam identification of the beam pair, but only consider the beam quality or beam identification of the downlink transmit beam. For example, an AI model based on a downlink beam, rather than an AI model based on a beam pair, is not limited in this disclosure.

[0170] It's clear that AI models have a specific lifecycle or scope of application. For example, some AI models are suitable for specific environments, such as suburban areas, urban areas, and indoors. Another example is that some AI models are suitable for rush hour, while others are suitable for specific time periods, such as during the commuting hours when there are fewer people. Another example is that some AI models may be suitable for specific times and environments, such as during the commuting hours when there are fewer people. Therefore, it's necessary to monitor the performance of AI models in real time. If the performance of an AI model falls short of the requirements, it's necessary to promptly update or switch the model.

[0171] In some embodiments, when monitoring the performance of AI models and / or AI functions deployed on the network device side, the terminal can perform periodic reporting. However, the reporting frequency may be relatively frequent, resulting in high signaling overhead.

[0172] In some embodiments, when the AI ​​model and / or AI function is deployed on the network device side and monitored, the content reported by the terminal may include two parts of data. One part of the data is used as the input of the AI ​​model and / or AI function on the network side. For example, the L1-RSRP of the beam and / or beam pair in set B used for model input, or the identifier and L1-RSRP of the beam and / or beam pair in set B. So that the AI ​​model and / or AI function on the network side can obtain the predicted best K beams and / or identifiers of beam pairs and / or L1-RSRP of set A based on the input of set B. The other part of the data may include the identifiers and / or L1-RSRP of the best K beams and / or beam pairs of set A actually measured by the terminal.

[0173] The network device can obtain a difference value based on the predicted identifiers and / or L1-RSRP of the best K beams and / or beam pairs of set A, and the identifiers and / or L1-RSRP of the best K beams and / or beam pairs of set A actually measured and reported by the terminal. The difference value can also be called a metric. The network device can make a judgment based on the metric to determine whether it is necessary to activate, deactivate, switch, update, or roll back the AI ​​model and / or AI function.

[0174] In some embodiments, the AI ​​model and / or AI function deployed on the terminal side can be monitored by the terminal itself. For example, the terminal compares the beam information of set A obtained based on set B and the AI ​​model with the beam information of set A obtained by actual measurement to obtain a metric. The terminal can compare the metric with the corresponding threshold value. Of course, in some cases, the AI ​​model and / or AI function can also be monitored by the network device, and the terminal sends the beam information of set A obtained based on the AI ​​model and the actually measured beam information to the network device; or the terminal obtains the metric based on the beam information of set A obtained based on the AI ​​model and the actually measured beam information, and sends the metric to the network device.

[0175] In some embodiments, when the terminal reports beam information or metrics, it is based on a prediction accuracy ratio as a condition, such as a beam prediction accuracy ratio.

[0176] For example, each determination of whether the prediction is accurate may include: the predicted strongest K reference signal identifiers include the strongest reference signal identifier in the actual measurement. Wherein, K is an integer greater than or equal to 1. The reference signal identifier may be a synchronization signal and a physical broadcast channel block (SSB) identifier, or a channel state information reference signal (CIS-RS) identifier, or a sounding reference signal (SRS) identifier.

[0177] Among them, the strongest can be considered to be the strongest L1-RSRP or the strongest L1-SINR.

[0178] Of course, the above ratio may be a ratio determined based on multiple predictions and according to the accuracy of the predictions.

[0179] For example, the ratio is calculated by comparing an L1-RSRP difference with a corresponding threshold; or the ratio is calculated by taking an average of multiple predicted L1-RSRP differences and comparing the average with the corresponding threshold.

[0180] In some embodiments, the ratio may be a probability that the multiple L1-RSRP differences are lower than or equal to the threshold value; or, the ratio may be a probability that the multiple L1-RSRP differences are higher than or equal to the threshold value.

[0181] Of course, in the case of an average value, it can be based on the average of multiple L1-RSRP differences. The L1-RSRP difference may refer to the L1-RSRP difference between the actual L1-RSRP corresponding to the predicted best beam identifier and the actual L1-RSRP corresponding to the actual best beam identifier. The L1-RSRP difference may also be the L1-RSRP difference between the predicted L1-RSRP corresponding to the predicted best beam identifier and the actually measured L1-RSRP corresponding to the beam identifier.

[0182] It is understood that in each embodiment of the present disclosure, the best beam, the strongest beam, and the optimal beam can be used interchangeably, the best reference signal, the strongest reference signal, and the optimal reference signal can be used interchangeably, and the best reference signal identifier, the strongest reference signal identifier, and the optimal reference signal identifier can be used interchangeably. Among them, the beam can be indicated by a reference signal. For example, when indicating a certain optimal reference signal identifier, it can indicate that the reference signal corresponding to the reference signal identifier is sent and / or received, and it can also indicate that the beam used for sending and / or receiving the reference signal corresponding to the reference signal identifier is the optimal beam. Therefore, in some cases, the best beam, the strongest beam, the optimal beam, the best reference signal, the strongest reference signal, the optimal reference signal, the best reference signal identifier, the strongest reference signal identifier, and the optimal reference signal identifier can be used interchangeably, and the present disclosure does not limit them.

[0183] In some embodiments, the accuracy of the prediction can also be determined by the difference in average UE throughput. For example, based on the strongest reference signal predicted by the AI ​​model and the actual strongest reference signal, the SINR corresponding to each reference signal is determined. The capacity corresponding to each reference signal is then calculated based on the Shannon capacity. The difference in the capacity corresponding to the two reference signals can be used as the average UE throughput difference.

[0184] Of course, the average UE throughput difference may also be based on a certain prediction result, or an average value corresponding to multiple prediction results may be taken, which is not limited in the present disclosure.

[0185] In some embodiments, the accuracy of the prediction may also be determined by reference signal overhead, uplink control information overhead, and the like.

[0186] However, in the above scheme, too many samples are required for ratio, which leads to excessively long monitoring time and reduces the detection efficiency.

[0187] FIG2 is an interactive diagram of a model monitoring method according to an embodiment of the present disclosure. As shown in FIG2 , the present disclosure embodiment relates to a model monitoring method for a communication system 100, the method comprising:

[0188] Step S2101 , the network device 102 sends ninth information to the terminal 101 .

[0189] In some embodiments, the network device 102 sends ninth information to the terminal 101 .

[0190] In some embodiments, the terminal 101 receives ninth information sent by the network device 102 .

[0191] In some embodiments, "obtain", "get", "get", "receive", "transmit", "bidirectional transmission", "send and / or receive" can be interchangeable, and can be interpreted as receiving from other entities, obtaining from protocols, obtaining from higher layers, obtaining by self-processing, autonomous implementation, etc.

[0192] In some embodiments, terms such as "send", "transmit", "report", "download", "transmit", "bidirectional transmission", "send and / or receive" can be used interchangeably.

[0193] In some embodiments, the ninth information is used to configure a reference signal resource for beam measurement.

[0194] In some embodiments, the ninth information is used to configure a reference signal resource corresponding to at least one of set B and set A.

[0195] For example, after receiving the ninth information, the terminal 101 may determine to measure the reference signal resources configured in the ninth information. For example, if the reference signal resources corresponding to set B are configured in the ninth information, the terminal 101 may determine to measure the reference signal resources corresponding to set B. This allows the measurement results corresponding to set B to be subsequently reported to the network device 102. For another example, if the reference signal resources corresponding to set A are configured in the ninth information, the terminal 101 may determine to measure the reference signal resources corresponding to set A. This allows the measurement results corresponding to set A to be subsequently reported to the network device 102. Of course, for the ninth information in which both set B and set A are configured, the terminal 101 may determine to measure the reference signal resources corresponding to set B and to measure the reference signal resources corresponding to set A, respectively.

[0196] In some embodiments, the ninth information may be referred to as "configuration information", "reference signal resource configuration information", "beam configuration information", etc. The present disclosure does not limit the name of the ninth information.

[0197] In some embodiments, the terms "precoding", "precoder", "weight", "precoding weight", "quasi-co-location (QCL)", "transmission configuration indication (TCI) state", "spatial relation", "spatial domain filter", "transmission power", "phase rotation", "antenna port", "antenna port group", "layer", "the number of layers", "rank", "resource", "resource set", "resource group", "beam", "beam width", "beam angular degree", "antenna", "antenna element", "panel" and the like can be used interchangeably.

[0198] In some embodiments, the TCI state can be used to indicate which receive beams the terminal uses when receiving the physical downlink control channel (PDCCH) and / or the demodulation reference signal (DMRS) of the PDCCH, and / or the physical downlink shared channel (PDSCH) and / or the DMRS of the PDSCH. For example, the receive beams are the same as the synchronization signal and physical broadcast channel block (SSB) or channel state information reference signal (CSI-RS) sent by the network device. The TCI state can also be used to indicate which transmit beams the terminal uses when transmitting the physical uplink control channel (PUCCH) and / or the DMRS of the PUCCH, and / or the physical uplink shared channel (PUSCH) and / or the DMRS of the PUSCH. For example, the transmit beam corresponding to the receive beam of the SSB or CSI-RS sent by the receiving network device, or the same transmit beam as the SRS sent by the terminal. The beam refers to quasi-colocation (QCL) type D.

[0199] In some embodiments, the beam may also be replaced by a DL TCI state, a UL TCI state, a SL TCI state, a joint TCI state, or spatial relation information.

[0200] Step S2102 , the terminal 101 sends sixth information to the network device 102 .

[0201] In some embodiments, the terminal 101 receives the sixth information sent by the network device 102 .

[0202] In some embodiments, the network device 102 sends sixth information to the terminal 101.

[0203] In some embodiments, the names of information, etc. are not limited to the names described in the embodiments, and terms such as "information", "message", "signal", "signaling", "report", "configuration", "indication", "instruction", "command", "channel", "parameter", "domain", "field", "symbol", "symbol", "codeword", "codebook", "codeword", "codepoint", "bit", "data", "program", and "chip" can be used interchangeably.

[0204] In some embodiments, the terminal 101 sends information to the network device 102, which can be considered as uplink communication.

[0205] In some embodiments, the terms "uplink", "uplink", "physical uplink", etc. can be used interchangeably.

[0206] In some embodiments, the terms "physical uplink shared channel (PUSCH)" and "UL data" may be used interchangeably.

[0207] In some embodiments, the sixth information is used to request monitoring of the AI ​​model and / or AI function.

[0208] In some embodiments, the sixth information may also be referred to as "AI model monitoring request", "AI function monitoring request", "AI monitoring request", etc. This disclosure does not limit the name of the sixth information.

[0209] In some embodiments, when the AI ​​model is deployed on the terminal 101, the terminal 101 may send sixth information to the network device 102 to request the terminal 101 to monitor the AI ​​model and / or AI function deployed on the terminal 101.

[0210] In some embodiments, when the AI ​​model is deployed on the network device 102, the terminal 101 may also send sixth information to the network device 102 to request the terminal 101 to monitor the AI ​​model and / or AI function deployed on the network device 102.

[0211] In some embodiments, an AI model may correspond to one or more AI functionalities.

[0212] In some embodiments, the AI ​​function may correspond to one or more AI models.

[0213] An AI function can be understood as one or more AI models that implement a specific function. For example, an AI function can be a beam prediction function or a positioning function. Different AI models can be used to implement the same function. For example, AI model 1 and AI model 2 are both AI models for positioning. However, AI model 1 and AI model 2 may be suitable for positioning in different scenarios. For example, AI model 1 is used for positioning in area A, while AI model 2 is used for positioning in area B.

[0214] Of course, one AI model can also correspond to multiple AI functions. For example, AI model 1 can be used for both macrocell beam prediction and indoor beam prediction. Of course, the above-mentioned AI functions are only an exemplary description, and the specific AI functions can be replaced according to actual conditions, which is not limited in this disclosure.

[0215] For example, when managing AI models based on AI function identifiers, for example, terminal 101 reports that supported AI function identifier 1 is the spatial beam prediction function. Terminal 101 then has multiple AI models for the spatial beam prediction function. However, network device 102 does not need to know about operations such as switching between these multiple AI models. Terminal 101 can implement this on its own. Therefore, terminal 101 does not need to inform network device 102 of the specific multiple AI models.

[0216] For another example, if model management needs to be performed based on AI model identification, the terminal 101 needs to inform the network device 102 of each AI model, and the switching and activation between any two AI models need to be informed to the network device 102.

[0217] In some embodiments, the terminal 101 may send sixth information to the network device 102 based on a scheduling request (SR).

[0218] In some embodiments, the terminal 101 may send the sixth information to the network device 102 based on an uplink (UL) medium access control element (MAC CE).

[0219] In some embodiments, the terminal 101 may send the sixth information to the network device 102 based on a random access procedure (RA).

[0220] Step S2103 , the network device 102 sends seventh information to the terminal 101 .

[0221] In some embodiments, the network device 102 sends seventh information to the terminal 101 .

[0222] In some embodiments, the terminal 101 receives the seventh information sent by the network device 102 .

[0223] In some embodiments, the network device 102 sends information to the terminal 101, which can be considered as downlink communication.

[0224] In some embodiments, the terms "downlink", "physical downlink", etc. can be used interchangeably.

[0225] In some embodiments, the seventh information is used to indicate that the network device 102 has received feedback corresponding to the sixth information.

[0226] In some embodiments, the seventh information is used to indicate that the network device 102 has received a response corresponding to the sixth information.

[0227] It is understandable that when network device 102 receives the sixth information sent by terminal 101, it can determine that terminal 101 requests to monitor the AI ​​model and / or AI function. Network device 102 can respond to the request and send seventh information to terminal 101. The seventh information is used to inform terminal 101 that network device 102 allows terminal 101 to monitor the AI ​​model and / or AI function.

[0228] In some embodiments, the seventh information may also be called a "response message", "feedback message", "monitoring permission indication", etc. The present disclosure does not limit the name of the seventh information.

[0229] Step S2104 , the network device 102 sends sixth information to the terminal 101 .

[0230] In some embodiments, the network device 102 sends sixth information to the terminal 101 .

[0231] In some embodiments, the terminal 101 receives sixth information sent by the network device 102 .

[0232] In some embodiments, when the AI ​​model is deployed on the terminal 101, the network device 102 may send sixth information to the terminal 101 to instruct the terminal 101 to monitor the AI ​​model and / or AI function deployed on the terminal 101.

[0233] In some embodiments, when the AI ​​model is deployed on the network device 102, the network device 102 may also send sixth information to the terminal 101 to instruct the terminal 101 to monitor the AI ​​model and / or AI function deployed on the network device 102. For example, the terminal 101 is instructed to perform beam measurement to monitor the AI ​​model and / or AI function using the measurement results.

[0234] In some embodiments, the network device 102 may send the sixth information to the terminal 101 based on radio resource control (RRC).

[0235] In some embodiments, the network device 102 may send sixth information to the terminal 101 based on a downlink (DL) MAC CE.

[0236] In some embodiments, the network device 102 may send the sixth information to the terminal 101 based on paging.

[0237] In some embodiments, the network device 102 may send the sixth information to the terminal 101 based on downlink control information (DCI), wherein the DCI may be DCI for scheduling a physical downlink shared channel (PDSCH).

[0238] In some embodiments, the terms "DCI", "downlink (DL) assignment", "DL DCI", "uplink (UL) grant", "UL DCI" and the like may be used interchangeably.

[0239] In some embodiments, the terms "PDSCH", "DL data", etc. can be used interchangeably.

[0240] In some embodiments, step S2104 may be performed alternatively to step S2102 or step S2103. That is, step S2102 and step S2103 may be omitted, and only step S2104 may be performed; or, step S2104 may be omitted, and only steps S2102 and S2103 may be performed. In other words, the departure terminal 101 may initiate a request to the AI ​​model and / or AI function from the terminal 101 to the network device 102, or the network device 102 may send a request to the terminal 101, and this disclosure does not limit this.

[0241] In step S2105, terminal 101 performs beam measurement.

[0242] In some embodiments, the terminal 101 may perform beam measurement to determine the first information while sending the sixth information.

[0243] In some embodiments, the terminal 101 determines the first information at a second time after sending the sixth information.

[0244] For example, after terminal 101 completes step S2102, a second time period elapses. Terminal 101 may determine to begin monitoring the AI ​​model and / or AI function. Terminal 101 measures the corresponding reference signal based on the reference signal resource configured in the ninth information, and generates first information based on the measurement result.

[0245] In some embodiments, the second time may be a default value.

[0246] In some embodiments, the second time may be configured by the network device 102 .

[0247] In some embodiments, upon receiving the seventh information, the terminal 101 may perform beam measurement to determine the first information.

[0248] In some embodiments, the terminal 101 determines the first information a third time after receiving the seventh information.

[0249] For example, after terminal 101 completes step S2103, a third time period elapses. Terminal 101 may determine to begin monitoring the AI ​​model and / or AI function. Terminal 101 measures the corresponding reference signal based on the reference signal resource configured in the ninth information, and generates first information based on the measurement result.

[0250] In some embodiments, the third time may be a default value.

[0251] In some embodiments, the third time may be configured by the network device 102 .

[0252] In some embodiments, the terminal 101 may perform beam measurement to determine the first information after the network device 102 sends a reference signal for AI model monitoring.

[0253] In some embodiments, after network device 102 transmits a reference signal corresponding to the reference signal resource configured in the ninth information, terminal 101 may determine to begin monitoring the AI ​​model and / or AI function. Terminal 101 measures the corresponding reference signal based on the reference signal resource configured in the ninth information, and generates first information based on the measurement result.

[0254] For example, when predicting the spatial domain, the AI ​​model needs to be monitored. Then, the network device 102 needs to send reference signals for beam measurement based on all transmit beams.

[0255] For another example, for time domain prediction, the network device 102 needs to send a reference signal for beam measurement in each time period.

[0256] In some embodiments, the terminal 101 may perform beam measurement to determine the first information upon receiving the sixth information.

[0257] In some embodiments, the terminal 101 may determine the first information at a fourth time after receiving the sixth information.

[0258] For example, after the fourth time after network device 102 sends the sixth information, terminal 101 may determine to monitor the AI ​​model and / or AI function. Terminal 101 measures the reference signal in response based on the reference signal resource configured in the ninth information, and generates the first information based on the measurement result.

[0259] In some embodiments, the fourth time may be a default value.

[0260] In some embodiments, the fourth time may be configured by the network device 102 .

[0261] In some embodiments, terms such as "moment", "time point", "time", and "time position" can be replaced with each other, and terms such as "duration", "period", "time window", "window", and "time" can be replaced with each other.

[0262] Step S2106 , the terminal 101 sends the first information to the network device 102 .

[0263] In some embodiments, the terminal 101 sends first information to the network device 102 .

[0264] In some embodiments, the network device 102 receives the first information sent by the terminal 101 .

[0265] In some embodiments, when the first condition or the second condition is met, the terminal 101 sends the first information to the network device 102.

[0266] In some embodiments, when the first condition or the second condition is met, the network device 102 receives the first information sent by the terminal 101.

[0267] In some embodiments, the first information is used to represent a measurement result of the beam measurement.

[0268] In some embodiments, the first information can be called beam measurement information, beam measurement results, beam information, etc. The present disclosure does not limit the name of the first information.

[0269] In some embodiments, the first information may include information for monitoring the performance of the AI ​​model. For example, the first information includes a measurement result obtained by terminal 101 measuring the beam.

[0270] In some embodiments, the first information is carried via RRC.

[0271] For example, the first information is sent based on a physical uplink shared channel (PUSCH). For example, the PUSCH may be a dynamically scheduled PUSCH. For another example, the PUSCH may be a configured grant PUSCH.

[0272] In some embodiments, the first information is carried by a UL MAC CE.

[0273] In some embodiments, the first information is carried via uplink control information (UCI).

[0274] In some embodiments, the first information is carried by at least one of RRC, UL MAC CE, and UCI.

[0275] In some embodiments, for the case where the AI ​​model is deployed on the network device 102, the first information may include second information corresponding to the beams and / or beam pairs in the first set.

[0276] In some embodiments, the second information is used to represent first parameters of the beams and / or beam pairs in the first set, wherein the beams and / or beam pairs in the first set correspond to inputs of the AI ​​model.

[0277] For example, the second information may be first parameters of all beams and / or beam pairs in the first set.

[0278] For another example, the second information may be first parameters of some of the beams and / or beam pairs in the first set. The some of the beams and / or beam pairs in the first set may be a first number of beams and / or beam pairs in the first set. The first number is greater than or equal to 1 and less than or equal to the total number of beams and / or beam pairs in the first set.

[0279] In some embodiments, the first parameter includes an identifier and / or a measurement value obtained by the terminal 101 measuring the reference signal corresponding to the reference signal resource.

[0280] For example, the identifier may be a reference signal identifier, and the measurement value may be a measurement value corresponding to the reference signal indicated by the reference signal identifier.

[0281] In some embodiments, the first set may be set B.

[0282] In some embodiments, the second information includes L1-RSRPs of a first number of beams and / or beam pairs measured in set B.

[0283] In some embodiments, the second information includes L1-RSRPs of the first number of beams and / or beam pairs measured in set B, and identities of the first number of beams and / or beam pairs measured in set B.

[0284] In some embodiments, the second information includes L1-SINRs of a first number of beams and / or beam pairs measured in set B.

[0285] In some embodiments, the second information includes L1-SINRs of the first number of beams and / or beam pairs measured in set B, and identities of the first number of beams and / or beam pairs measured in set B.

[0286] In some embodiments, for the case where the AI ​​model is deployed on the network device 102, the first information may include third information corresponding to the beams and / or beam pairs in the second set.

[0287] In some embodiments, the third information is used to represent first parameters of beams and / or beam pairs in a second set, wherein the second set includes the first set, or the second set is different from the first set.

[0288] For example, the third information may be first parameters of all beams and / or beam pairs in the second set.

[0289] For another example, the third information may be first parameters of some of the beams and / or beam pairs in the second set. The some of the beams and / or beam pairs in the second set may be a second number of beams and / or beam pairs in the second set. The second number is greater than or equal to 1 and less than or equal to the total number of beams and / or beam pairs in the second set.

[0290] In some embodiments, the second set may be set A.

[0291] In some embodiments, the third information includes L1-RSRPs of a second number of beams and / or beam pairs measured in set A.

[0292] In some embodiments, the third information includes L1-RSRPs of the second number of beams and / or beam pairs measured in set A, and identifiers of the second number of beams and / or beam pairs measured in set A.

[0293] In some embodiments, the third information includes L1-SINRs of a second number of beams and / or beam pairs measured in set A.

[0294] In some embodiments, the third information includes L1-SINRs of the second number of beams and / or beam pairs measured in set A, and identifiers of the second number of beams and / or beam pairs measured in set A.

[0295] In some embodiments, set B is a subset of set A. For example, set B is included in set A. For another example, set B is the same as set A.

[0296] In some embodiments, set B is different from set A. For example, set B corresponds to a wide beam and set A corresponds to a narrow beam.

[0297] In some embodiments, the output of the AI ​​model is used to obtain second parameters of the beams and / or beam pairs in the second set.

[0298] In some embodiments, the output of the AI ​​model is used to obtain second parameters of all beams and / or beam pairs in the second set.

[0299] In some embodiments, the output of the AI ​​model is used to obtain second parameters of the K beams and / or beam pairs in the second set.

[0300] For example, the output of the AI ​​model may be directly the second parameters of K beams and / or beam pairs, where the K beams and / or beam pairs may be considered to be the best K reference signals.

[0301] For example, the output of the AI ​​model may be the second parameters of all beams and / or beam pairs in the second set, and the network device 102 selects the K best reference signals from all beams and / or beam pairs in the second set.

[0302] In some embodiments, the second parameter includes an identification and / or a predicted value obtained by the network device 102 through the AI ​​model prediction.

[0303] In some embodiments, the predicted value includes L1-RSRP of K beams and / or beam pairs predicted by the AI ​​model.

[0304] In some embodiments, the predicted value includes the L1-RSRPs of the K beams and / or beam pairs predicted by the AI ​​model, and the identifiers of the K beams and / or beam pairs predicted by the AI ​​model.

[0305] In some embodiments, the predicted value includes L1-SINRs of K beams and / or beam pairs predicted by the AI ​​model.

[0306] In some embodiments, the predicted value includes the L1-SINRs of the K beams and / or beam pairs predicted by the AI ​​model, and the identifiers of the K beams and / or beam pairs predicted by the AI ​​model.

[0307] In some embodiments, for the case where the AI ​​model is deployed on the network device 102, the first information may include second information corresponding to the beams and / or beam pairs within the first set, and the first information may include third information corresponding to the beams and / or beam pairs within the second set.

[0308] Of course, if set B is a subset of set A, the first information may only include the third information corresponding to the beams and / or beam pairs in the second set. This is because the third information corresponding to the beams and / or beam pairs in the second set will include the second information corresponding to the beams and / or beam pairs in the first set.

[0309] In some embodiments, when the AI ​​model is deployed on the terminal 101, the first information may include the fourth information.

[0310] In some embodiments, the fourth information is used to instruct to perform an AI operation on the AI ​​model.

[0311] In some embodiments, when the AI ​​model is deployed on terminal 101, terminal 101 may independently measure the first parameters of the beams and / or beam pairs in set B and use them as input to the AI ​​model to obtain the second parameters of K beams and / or beam pairs, or obtain the second parameters of all beams and / or beam pairs in set A. Terminal 101 may also perform beam measurement on all beams and / or beam pairs in set A to obtain the first parameters of all beams and / or beam pairs in set A, and may further determine the first parameters of the K beams and / or beam pairs with the best reference signals. Terminal 101 may determine the performance of the AI ​​model based on the actual beam measurement and the prediction output by the AI ​​model. Based on the determined performance of the AI ​​model, terminal 101 may decide which AI operations to perform on the AI ​​model. Terminal 101 may send fourth information indicating the decision on which AI operations to perform on the AI ​​model to network device 102.

[0312] It can be understood that the terminal 101 sending the fourth information can be applicable to the scenario where the AI ​​model is deployed on the terminal 101 and the model is monitored by the terminal 101.

[0313] In some embodiments, for the case where the AI ​​model is deployed on terminal 101, the first information may include third information and fifth information corresponding to the beams and / or beam pairs in the second set.

[0314] The third information is as described in the above embodiment, and will not be repeated here in this disclosure.

[0315] In some embodiments, the fifth information is used to represent second parameters of the beams and / or beam pairs in the second set.

[0316] For example, the fifth information may be second parameters of all beams and / or beam pairs in the second set.

[0317] For another example, the fifth information may be a second parameter of some of the beams and / or beam pairs in the second set. The some of the beams and / or beam pairs in the second set may be a second number of beams and / or beam pairs in the second set. The second number is greater than or equal to 1 and less than or equal to the total number of beams and / or beam pairs in the second set.

[0318] In some embodiments, the fifth information includes L1-RSRPs of K beams and / or beam pairs predicted by the AI ​​model in set A.

[0319] In some embodiments, the fifth information includes the L1-RSRP of the K beams and / or beam pairs predicted by the AI ​​model in set A, and the identifiers of the K beams and / or beam pairs predicted by the AI ​​model in set A.

[0320] In some embodiments, the fifth information includes the L1-SINRs of the K beams and / or beam pairs predicted by the AI ​​model in set A.

[0321] In some embodiments, the fifth information includes the L1-SINRs of the K beams and / or beam pairs predicted by the AI ​​model in set A, and the identifiers of the K beams and / or beam pairs predicted by the AI ​​model in set A.

[0322] In some embodiments, when the AI ​​model is deployed on the terminal 101, the first information may include a first indicator.

[0323] In some embodiments, the first indicator may be used to identify a prediction accuracy of the second number of beams and / or beam pairs.

[0324] For example, terminal 101 may calculate a metric based on the third information and the fifth information corresponding to the beams and / or beam pairs in the second set. The metric is the first indication. Terminal 101 may report the metric to network device 102.

[0325] In some embodiments, the first indicator may directly or indirectly reflect the prediction accuracy of at least one of the identification of the beam and / or beam pair, the L1-RSRP of the beam and / or beam pair, and the L1-SINR of the beam and / or beam pair.

[0326] In some embodiments, if the second parameter obtained by the AI ​​model output N times satisfies the first condition, the terminal sends the first information to the network device. The N times can be continuous or non-continuous.

[0327] In some embodiments, if the second parameter obtained by the AI ​​model output M times satisfies the second condition, the terminal sends the first information to the network device. The M times can be continuous or non-continuous.

[0328] In the above embodiment, the terms "continuous N times" and "continuous M times" need to be multiple times. For consecutive N times, if the Nith time does not meet the requirement, the calculation needs to be recalculated. The same applies to the Mth time, which will not be repeated here.

[0329] In some embodiments, the first condition may include: within a first time period, if the second parameter obtained by the AI ​​model output satisfies the first condition N times, the terminal sends the first information to the network device. The N times may be continuous or non-continuous.

[0330] In some embodiments, the first condition may include: within a first time period, if the second parameter obtained by the AI ​​model output satisfies the second condition M times, the terminal sends the first information to the network device. The M times may be continuous or discontinuous.

[0331] In some embodiments, the first condition may include: the K target beams and / or beam pairs obtained by the AI ​​model output include the L target beams and / or beam pairs measured in the second set.

[0332] The K target beams and / or beams represent the K best beams and / or beam pairs output by the AI ​​model, and the L target beams and / or beam pairs represent the L best beams and / or beam pairs actually predicted.

[0333] For example, the AI ​​model can directly output the K target beams and / or beam pairs obtained. Alternatively, the AI ​​model can output the predicted values ​​for all beams and / or beam pairs in set A. These predicted values ​​are sorted in descending or descending order. The top K beams and / or beam pairs can be selected as the optimal beams and / or beam pairs.

[0334] For example, terminal 101 may measure all beams and / or beam pairs in set A and sort them in descending order or from high to low according to the magnitude of the measured values. The first L beams and / or beam pairs may be selected as the best beams and / or beam pairs.

[0335] That is, when the K best beams and / or beam pairs output by the AI ​​model include the L best beams and / or beam pairs actually measured, the first condition can be considered satisfied, and the AI ​​model prediction is relatively accurate, or the AI ​​model performance is considered good.

[0336] In some embodiments, the first condition may include: 1 target beam and / or beam pair obtained by the AI ​​model output is among the L target beams and / or beam pairs measured in the second set.

[0337] Among them, 1 target beam and / or beam represents 1 optimal beam and / or beam pair obtained by the AI ​​model output.

[0338] For example, the AI ​​model can directly output the obtained target beam and / or beam pair. Alternatively, the AI ​​model can output the predicted values ​​for all beams and / or beam pairs in set A. The predicted values ​​are sorted in descending order or from high to low. The beam and / or beam pair with the largest or highest prediction value can be selected as the optimal beam and / or beam pair.

[0339] That is, when the best beam and / or beam pair output by the AI ​​model is among the L best beams and / or beam pairs actually measured, the first condition is considered to be met, and the AI ​​model prediction is relatively accurate, or the AI ​​model performance is considered to be good.

[0340] In some embodiments, when the AI ​​model is deployed on the network device 102, the K or 1 target beams and / or beam pairs output by the AI ​​model may be indicated by the network device 102. For example, the network device 102 sends information indicating these beams, such as a beam indication or a TCI status indication. The TCI status may include at least one of an uplink TCI status, a downlink TCI status, and a joint TCI status.

[0341] In some embodiments, the values ​​of K and L mentioned above can be any positive integers. K and L can be the same or different, and this disclosure does not limit this.

[0342] In some embodiments, when the network sends a beam indication, it is worth noting that the beam indication from network device 102 can be an AI model output obtained by network device 102 based on the measurement results of set B measured by terminal 101 at time A as input to the AI ​​model. When terminal 101 determines the optimal beam and / or beam pair actually measured, the measurement results of set A measured at time A are also required to ensure the accuracy of AI model monitoring.

[0343] In some embodiments, the first condition may include: the first difference between the measurement value corresponding to the predicted identification of one target beam and / or beam pair obtained by the AI ​​model output and the measurement value of one target beam and / or beam pair measured in the second set is less than or equal to the first threshold.

[0344] For example, the AI ​​model outputs an optimal beam and / or beam pair, determines a measured value of the predicted optimal beam and / or beam pair, and an actually measured measured value of the optimal beam and / or beam pair, and calculates a first difference. If the first difference is less than or equal to a first threshold, it can be considered that the first condition is met, the AI ​​model prediction is relatively accurate, or the AI ​​model performance is considered good.

[0345] In some embodiments, the first condition may include: a second difference between a predicted value corresponding to a predicted identifier of a target beam and / or beam pair obtained by the AI ​​model output and a measured value corresponding to the predicted identifier is less than or equal to a second threshold.

[0346] For example, the AI ​​model outputs an optimal beam and / or beam pair, determines a predicted value of the predicted optimal beam and / or beam pair, and a measured value of the optimal beam and / or beam pair obtained by actual measurement, and calculates a second difference. If the second difference is less than or equal to a second threshold, it can be considered that the first condition is met, the AI ​​model prediction is relatively accurate, or the AI ​​model performance is considered good.

[0347] In some embodiments, the measurement value and the predicted value involved in the above embodiments may be, for example, L1-RSRP or L1-SINR.

[0348] In some embodiments, the first condition may include: a throughput difference being greater than or equal to a third threshold. The throughput difference represents a difference between a throughput corresponding to one of the K target beams and / or beam pairs obtained based on the AI ​​model output and a throughput corresponding to one of the L target beams and / or beam pairs in the second set.

[0349] For example, the throughput difference can be the average UE throughput difference mentioned above. The throughput difference can be the difference between the throughput corresponding to beam 1 and / or beam pair 1 and the throughput corresponding to beam 2 and / or beam pair 2. Beam 1 and / or beam pair 1 is the best beam and / or beam pair among the K best beams and / or beam pairs obtained by the AI ​​model output. Beam 2 and / or beam pair 2 is the best beam and / or beam pair among the L best beams and / or beam pairs actually measured in set A.

[0350] In some embodiments, the first condition may include: the resource overhead of the reference signal sent by the network device 102 is less than or equal to a fourth threshold.

[0351] For example, the overhead of reference signal resources used by terminal 101 for beam measurement is less than or equal to a fourth threshold.

[0352] In some embodiments, the first condition may include: the overhead of the second information is less than or equal to a fifth threshold.

[0353] For example, the UCI overhead used to report the beam measurement result is less than or equal to the fifth threshold. For example, the beam measurement result includes the measurement results of the beams and / or beam pairs in set B.

[0354] In some embodiments, the second condition may include: the K target beams and / or beam pairs obtained by the AI ​​model output do not include the L target beams and / or beam pairs measured in the second set.

[0355] That is, if the K best beams and / or beam pairs output by the AI ​​model do not include the L best beams and / or beam pairs actually measured, the second condition can be considered met, indicating that the AI ​​model prediction is inaccurate or that the AI ​​model performance is poor.

[0356] In some embodiments, the second condition may include: one target beam and / or beam pair obtained by the AI ​​model output is not among the L target beams and / or beam pairs measured in the second set.

[0357] In other words, if the optimal beam and / or beam pair output by the AI ​​model is not among the L optimal beams and / or beam pairs actually measured, the second condition can be considered to be met, indicating that the AI ​​model prediction is inaccurate or that the AI ​​model performance is poor.

[0358] In some embodiments, the second condition may include: a first difference between a measurement value corresponding to a predicted identifier of a target beam and / or beam pair obtained by the AI ​​model output and a measurement value of a target beam and / or beam pair measured in the second set is greater than a first threshold.

[0359] For example, the AI ​​model outputs an optimal beam and / or beam pair, determines the measured value of the predicted optimal beam and / or beam pair, and the measured value of the optimal beam and / or beam pair actually obtained, and calculates a first difference. If the first difference is greater than a first threshold, it can be considered that the second condition is met, the AI ​​model prediction is inaccurate, or the AI ​​model performance is considered poor.

[0360] In some embodiments, the second condition may include: a second difference between a predicted value corresponding to a predicted identifier of a target beam and / or beam pair obtained by the AI ​​model output and a measured value corresponding to the predicted identifier is less than or equal to a second threshold.

[0361] For example, the AI ​​model outputs an optimal beam and / or beam pair, determines a predicted value of the predicted optimal beam and / or beam pair, and a measured value of the optimal beam and / or beam pair obtained by actual measurement, and calculates a second difference. If the second difference is greater than a second threshold, it can be considered that the second condition is met, the AI ​​model prediction is inaccurate, or the AI ​​model performance is considered poor.

[0362] In some embodiments, the second condition may include: the throughput difference is less than a third threshold.

[0363] In some embodiments, the second condition may include: the resource overhead of the reference signal sent by the network device 102 is greater than a fourth threshold.

[0364] For example, the overhead of reference signal resources used by terminal 101 for beam measurement is greater than a fourth threshold.

[0365] In some embodiments, the second condition may include: the overhead of the second information is greater than a fifth threshold.

[0366] For example, the UCI overhead used to report the beam measurement result is greater than the fifth threshold.

[0367] It can be understood that the definition and determination method of each parameter in the second condition can refer to the embodiment corresponding to the first condition, and the present disclosure will not go into details here.

[0368] In some embodiments, when the AI ​​model is deployed on the network device 102 , the terminal 101 may periodically send the first information to the network device 102 .

[0369] In some embodiments, when the AI ​​model is deployed on the network device 102, the terminal 101 may also trigger the sending of the first information to the network device 102 based on the above-mentioned first condition and / or second condition.

[0370] In some embodiments, when the AI ​​model is deployed on the terminal 101 , the terminal 101 may periodically send the first information to the network device 102 .

[0371] In some embodiments, when the AI ​​model is deployed on the terminal 101, the terminal 101 may also trigger the sending of the first information to the network device 102 based on the first condition and / or the second condition.

[0372] For example, the terminal 101 triggers the sending of the measurement results corresponding to set A to the network device 102 based on the above-mentioned first condition and / or second condition. For example, the network device 102 sends a beam indication to the terminal 101 to indicate K' best beams. The terminal 101 determines the K" best beams based on actual measurements. The terminal 101 can determine whether to trigger the reporting of the measurement results corresponding to set A based on the relationship between the K' best beams and the K" best beams. Of course, the terminal 101 can also determine whether to trigger the reporting of the measurement results corresponding to set A based on UCI overhead, reference signal resource overhead, etc., which is not limited in the present disclosure.

[0373] In some embodiments, if the AI ​​model is deployed on terminal 101 and terminal 101 monitors the AI ​​model on its own, terminal 101 may determine which AI operations to perform based on the first condition and / or the second condition included in the first condition. N and M may be configured by network device 102.

[0374] In some embodiments, if the AI ​​model is deployed on terminal 101 and terminal 101 and network device 102 monitor the AI ​​model together, terminal 101 may determine which AI operations to perform based on the first condition and / or the second condition included in the first condition. N and M may be configured by network device 102.

[0375] In this case, the first information sent by the terminal 101 may be a metric, and the network device 102 determines which AI operations to perform on the AI ​​model based on the metric.

[0376] In some embodiments, if the AI ​​model is deployed on terminal 101 and network device 102 monitors the AI ​​model, terminal 101 may determine which AI operations to perform based on the first condition and / or the second condition included in the first condition. N and M may be configured by network device 102.

[0377] In this case, the first information sent by terminal 101 can be the identifier of the best beam and / or beam pair output by multiple predictions, the identifier of the best beam and / or beam pair obtained by actual measurement, and the measurement results corresponding to each best beam and / or beam pair.

[0378] In some embodiments, after receiving the first information, the network device 102 may determine the performance of the AI ​​model based on the first information.

[0379] For example, network device 102 may determine a second indication based on the measurement results reported by terminal 101. The second indication may be used to indicate the performance of the AI ​​model and / or AI function. For example, the second indication may be referred to as a model performance indication. Of course, the second indication may also be referred to as a performance indication, an AI function performance indication, etc., and this disclosure does not limit the name of the second indication.

[0380] In some embodiments, the second indication may be, for example, prediction accuracy.

[0381] For example, the prediction accuracy indicates the probability or proportion of accurate prediction. The prediction accuracy may mean that the predicted strongest K reference signal identifiers include the strongest reference signal identifier actually measured. The reference signal identifier may include an SSB ID, a CSI-RS ID, or an SRS ID.

[0382] In some embodiments, terms such as "synchronization signal (SS)", "synchronization signal block (SSB)", "reference signal (RS)", "pilot", and "pilot signal" can be used interchangeably.

[0383] For example, the strongest, optimal, best, highest, etc. mentioned above may refer to the strongest L1-RSRP or the strongest L1-SINR.

[0384] Of course, the prediction accuracy can be based on multiple predictions to see the accuracy rate of the prediction.

[0385] In some embodiments, the second indication may be, for example, an average L1-RSRP difference, or may also be referred to as an average L1-RSRP difference.

[0386] For example, the average L1-RSRP difference may be a difference between the actual L1-RSRP corresponding to the predicted strongest reference signal ID and the actual L1-RSRP corresponding to the actual strongest reference signal ID.

[0387] For example, the average L1-RSRP difference may be a difference between the predicted L1-RSRP corresponding to the predicted strongest reference signal ID and the actual L1-RSRP corresponding to the predicted strongest reference signal ID.

[0388] For example, the average L1-RSRP difference may be a difference between the actual L1-RSRP corresponding to the strongest predicted reference signal ID and the predicted L1-RSRP corresponding to the strongest predicted reference signal ID.

[0389] For example, the average L1-RSRP difference may be a difference between the actual L1-RSRP corresponding to the actual strongest reference signal ID and the predicted L1-RSRP corresponding to the actual strongest reference signal ID.

[0390] Of course, any of the above-mentioned L1-RSRP average differences can be viewed as a single prediction result or an average of multiple prediction results.

[0391] In some embodiments, the second indication may be, for example, an L1-RSRP difference.

[0392] For example, the L1-RSRP difference can be determined by a point in a L1-RSRP difference cumulative distribution function (CDF) curve to determine the L1-RSRP difference corresponding to the point. Of course, the L1-RSRP difference CDF curve can be obtained by fitting the L1-RSRP differences obtained from multiple prediction results.

[0393] For another example, the CDF may also be obtained based on the difference between the predicted L1-RSRP corresponding to the predicted strongest reference signal ID and the actual L1-RSRP corresponding to the actual strongest reference signal ID.

[0394] In some embodiments, the second indication may be, for example, a throughput difference.

[0395] For example, the SINRs of the two reference signals can be obtained based on the predicted strongest reference signal and the actual strongest reference signal. The capacities corresponding to the SINRs are then calculated based on the Shannon capacity. The difference between the capacities is the throughput difference.

[0396] For another example, the throughput difference may be viewed as a single prediction result or an average of multiple prediction results.

[0397] In some embodiments, the second indication may be, for example, a reference signal resource overhead.

[0398] In some embodiments, the second indication may be, for example, UCI overhead.

[0399] Of course, in the case where the second indication is reference signal resource overhead and / or UCI overhead, the second indication may not need to be determined based on the first information.

[0400] In some embodiments, the terms "target", "certain", "preset", "preset", "setting", "indicated", "a certain", "any", "first", etc. can be used interchangeably, and "specific A", "preset A", "preset A", "setting A", "indicated A", "a certain A", "any A", "first A" can be interpreted as A pre-specified in a protocol, etc., or as A obtained through setting, configuration, or indication, etc., or as specific A, a certain A, any A, or first A, etc., but not limited to this.

[0401] Step S2107 , the network device 102 sends the eighth information to the terminal 101 .

[0402] In some embodiments, the network device 102 sends eighth information to the terminal 101 .

[0403] In some embodiments, the terminal 101 receives the eighth information sent by the network device 102 .

[0404] In some embodiments, the eighth information is used to instruct the terminal 101 to perform AI operations on the AI ​​model.

[0405] For example, if an AI model is deployed on network device 102, network device 102 determines whether an AI operation needs to be performed on the AI ​​model and / or AI function, and which AI operation to perform on the AI ​​model and / or AI function based on the first information reported by terminal 101. Network device 102 may send eighth information indicating the relevant AI operation to terminal 101. Terminal 101 can then determine to perform the corresponding AI operation on the AI ​​model based on the received eighth information.

[0406] Of course, in this case, the network device may not send this indication information to the terminal, and this disclosure does not limit this.

[0407] For another example, an AI model is deployed on terminal 101 but is monitored by network device 102, or is monitored jointly by terminal 101 and network device 102. Network device 102 can determine whether an AI operation needs to be performed on the AI ​​model and / or AI function, and which AI operation to perform on the AI ​​model and / or AI function based on the first information reported by terminal 101. Network device 102 can send eighth information indicating the relevant AI operation to terminal 101. Terminal 101 can then determine to perform the corresponding AI operation on the AI ​​model based on the received eighth information.

[0408] In some embodiments, AI operations may include: activating an AI model and / or an AI function;

[0409] In some embodiments, AI operations may include deactivating an AI model and / or an AI function.

[0410] In some embodiments, AI operations may include switching AI models and / or AI functions.

[0411] In some embodiments, AI operations may include updating AI models and / or AI functions.

[0412] For example, updating the AI ​​model and / or AI function may be fine-tuning the AI ​​model and / or AI function.

[0413] In some embodiments, AI operation may include switching to a non-AI mode and / or non-AI functionality.

[0414] In some embodiments, the eighth information may be carried by RRC.

[0415] In some embodiments, the eighth information may be carried by a MAC CE.

[0416] In some embodiments, the eighth information may be carried by DCI.

[0417] In some embodiments, the eighth information may be carried by at least one of RRC, MAC CE, and DCI.

[0418] For example, RRC signaling provides a correspondence between multiple identifiers and specific parameters, while MAC CE or DCI can directly indicate the index.

[0419] For another example, RRC signaling provides a correspondence between multiple identifiers and specific parameters. MAC CE activates a portion of indexes, and DCI further indicates one of the portion of indexes activated by MAC CE.

[0420] In various embodiments of the present disclosure, the identifier may be an identity (ID) or an index.

[0421] In some embodiments, the eighth information may also include at least one of an identifier for indicating an updated AI model, configuration parameters, a model identifier, a model version identifier, etc.

[0422] For example, the model identifiers may correspond to different AI function models. AI functions may include CSI compression, beam prediction, positioning, etc.

[0423] For another example, the model identifier can be used to indicate different versions of the same AI model, or to indicate different AI models with different parameter configurations.

[0424] For another example, when different AI models with the same AI function have the same model identifier, the model version identifier can be used to distinguish AI models with different versions or different parameter configurations.

[0425] The perception communication method involved in the embodiments of the present disclosure may include at least one of steps S2101 to S2107. For example, steps S2106 + S2107 can be implemented as independent embodiments, steps S2105 + S2106 + S2107 can be implemented as independent embodiments, steps S2101 + S2105 + S2106 + S2107 can be implemented as independent embodiments, steps S2101 + S2102 + S2103 + S2105 + S2106 + S2107 can be implemented as independent embodiments, and steps S2101 + S2104 + S2105 + S2106 + S2107 can be implemented as independent embodiments, but are not limited thereto.

[0426] In some embodiments, steps S2101, S2102 and S2101 may be executed in an interchangeable order or simultaneously.

[0427] In some embodiments, steps S2104 and S2101 may be executed in an interchangeable order or simultaneously.

[0428] In some embodiments, step S2101 is optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0429] In some embodiments, steps S2102 and S2103 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0430] In some embodiments, step S2104 is optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0431] In some embodiments, step S2107 is optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0432] In some embodiments, reference may be made to other optional implementations described before or after the description corresponding to FIG. 2 .

[0433] Figure 3a is a flow chart of a model monitoring method according to an exemplary embodiment. As shown in Figure 3a, the embodiment of the present disclosure relates to a model monitoring method that can be executed on a terminal. The method includes:

[0434] Step S3101, obtain ninth information.

[0435] The optional implementation of step S3101 can refer to the optional implementation of step S2101 in Figure 2 and other related parts in the embodiment involved in Figure 2, which will not be repeated here.

[0436] In some embodiments, the terminal obtains ninth information specified by the protocol.

[0437] In some embodiments, the terminal obtains the ninth information from a higher layer.

[0438] In some embodiments, the terminal performs processing to obtain the ninth information.

[0439] In some embodiments, step S3101 is omitted, and the terminal autonomously implements the function indicated by the ninth information, or the above function is default or by default.

[0440] Step S3102, sending the sixth information.

[0441] The optional implementation of step S3102 can refer to the optional implementation of step S2102 in Figure 2 and other related parts in the embodiment involved in Figure 2, which will not be repeated here.

[0442] Step S3103, obtain the seventh information.

[0443] The optional implementation of step S3103 can refer to the optional implementation of step S2103 in Figure 2 and other related parts in the embodiment involved in Figure 2, which will not be repeated here.

[0444] In some embodiments, the terminal obtains seventh information specified by the protocol.

[0445] In some embodiments, the terminal obtains the seventh information from a higher layer.

[0446] In some embodiments, the terminal performs processing to obtain the seventh information.

[0447] In some embodiments, step S3103 is omitted, and the terminal autonomously implements the function indicated by the seventh information, or the above function is default or by default.

[0448] Step S3104, obtaining sixth information.

[0449] The optional implementation of step S3104 can refer to the optional implementation of step S2104 in Figure 2 and other related parts in the embodiment involved in Figure 2, which will not be repeated here.

[0450] In some embodiments, the terminal obtains sixth information specified by the protocol.

[0451] In some embodiments, the terminal obtains the sixth information from a higher layer.

[0452] In some embodiments, the terminal performs processing to obtain the sixth information.

[0453] In some embodiments, step S3104 is omitted, and the terminal autonomously implements the function indicated by the sixth information, or the above function is default or by default.

[0454] Of course, step S3102, step S3103, and step S3104 may be performed alternatively. For example, step S3102 and step S3103 may be performed, but step S3104 may not be performed; or step S3104 may be performed, but step S3102 and step S3103 may not be performed.

[0455] Step S3105: perform beam prediction.

[0456] The optional implementation of step S3105 can refer to the optional implementation of step S2105 in Figure 2 and other related parts in the embodiment involved in Figure 2, which will not be repeated here.

[0457] Step S3106, sending the first information.

[0458] The optional implementation of step S3106 can refer to the optional implementation of step S2106 in Figure 2 and other related parts in the embodiment involved in Figure 2, which will not be repeated here.

[0459] Step S3107, obtain the eighth information.

[0460] The optional implementation of step S3107 can refer to the optional implementation of step S2107 in Figure 2 and other related parts in the embodiment involved in Figure 2, which will not be repeated here.

[0461] In some embodiments, the terminal obtains eighth information specified by the protocol.

[0462] In some embodiments, the terminal obtains the eighth information from a higher layer.

[0463] In some embodiments, the terminal performs processing to obtain the eighth information.

[0464] In some embodiments, step S3107 is omitted, and the terminal autonomously implements the function indicated by the eighth information, or the above function is default or by default.

[0465] FIG3b is a flow chart of another model monitoring method according to an exemplary embodiment. As shown in FIG3b, the embodiment of the present disclosure relates to a model monitoring method that can be executed on a terminal. The method includes:

[0466] Step S3201, obtain ninth information.

[0467] The optional implementation of step S3201 can refer to the optional implementation of step S2101 in Figure 2, the optional implementation of step S3101 in Figure 3a, and other related parts in the embodiment involved in Figure 2 and other related parts in the embodiment involved in Figure 3a, which will not be repeated here.

[0468] In some embodiments, step S3201 is omitted, and the terminal autonomously implements the function indicated by the ninth information, or the above function is default or by default.

[0469] Step S3202, sending the sixth information.

[0470] The optional implementation of step S3202 can refer to the optional implementation of step S2102 in Figure 2, the optional implementation of step S3102 in Figure 3a, and other related parts in the embodiment involved in Figure 2 and other related parts in the embodiment involved in Figure 3a, which will not be repeated here.

[0471] Step S3203, obtain the seventh information.

[0472] The optional implementation of step S3203 can refer to the optional implementation of step S2103 in Figure 2, the optional implementation of step S3103 in Figure 3a, and other related parts in the embodiment involved in Figure 2 and other related parts in the embodiment involved in Figure 3a, which will not be repeated here.

[0473] In some embodiments, step S3203 is omitted, and the terminal autonomously implements the function indicated by the seventh information, or the above function is default or by default.

[0474] Step S3204: perform beam prediction.

[0475] The optional implementation of step S3204 can refer to the optional implementation of step S2105 in Figure 2, the optional implementation of step S3105 in Figure 3a, and other related parts in the embodiment involved in Figure 2 and other related parts in the embodiment involved in Figure 3a, which will not be repeated here.

[0476] Step S3205, sending the first information.

[0477] The optional implementation of step S3205 can refer to the optional implementation of step S2106 in Figure 2, the optional implementation of step S3106 in Figure 3a, and other related parts in the embodiment involved in Figure 2 and other related parts in the embodiment involved in Figure 3a, which will not be repeated here.

[0478] Step S3206, sending the eighth information.

[0479] The optional implementation of step S3206 can refer to the optional implementation of step S2107 in Figure 2, the optional implementation of step S3107 in Figure 3a, and other related parts in the embodiment involved in Figure 2 and other related parts in the embodiment involved in Figure 3a, which will not be repeated here.

[0480] Figure 3c is a flow chart of another model monitoring method according to an exemplary embodiment. As shown in Figure 3c, the embodiment of the present disclosure relates to a model monitoring method that can be executed on a terminal. The method includes:

[0481] Step S3301, obtain ninth information.

[0482] The optional implementation method of step S3301 can be found in the optional implementation method of step S2101 in Figure 2, the optional implementation method of step S3101 in Figure 3a, the optional implementation method of step S3201 in Figure 3b, and other related parts in the embodiment involved in Figure 2, other related parts in the embodiment involved in Figure 3a, and other related parts in the embodiment involved in Figure 3b, which will not be repeated here.

[0483] In some embodiments, step S3201 is omitted, and the terminal autonomously implements the function indicated by the ninth information, or the above function is default or by default.

[0484] Step S3302, obtain sixth information.

[0485] The optional implementation of step S3302 can refer to the optional implementation of step S2102 in Figure 2, the optional implementation of step S3104 in Figure 3a, and other related parts in the embodiment involved in Figure 2 and other related parts in the embodiment involved in Figure 3a, which will not be repeated here.

[0486] Step S3303: perform beam prediction.

[0487] The optional implementation method of step S3303 can be found in the optional implementation method of step S2105 in Figure 2, the optional implementation method of step S3105 in Figure 3a, the optional implementation method of step S3204 in Figure 3b, and other related parts in the embodiment involved in Figure 2, other related parts in the embodiment involved in Figure 3a, and other related parts in the embodiment involved in Figure 3b, which will not be repeated here.

[0488] Step S3304, sending the first information.

[0489] The optional implementation method of step S3304 can be found in the optional implementation method of step S2106 in Figure 2, the optional implementation method of step S3106 in Figure 3a, the optional implementation method of step S3205 in Figure 3b, and other related parts in the embodiment involved in Figure 2, other related parts in the embodiment involved in Figure 3a, and other related parts in the embodiment involved in Figure 3b, which will not be repeated here.

[0490] Step S3305, obtain the eighth information.

[0491] The optional implementation method of step S3305 can be found in the optional implementation method of step S2107 in Figure 2, the optional implementation method of step S3107 in Figure 3a, the optional implementation method of step S3206 in Figure 3b, and other related parts in the embodiment involved in Figure 2, other related parts in the embodiment involved in Figure 3a, and other related parts in the embodiment involved in Figure 3b, which will not be repeated here.

[0492] Figure 3d is a flow chart of another model monitoring method according to an exemplary embodiment. As shown in Figure 3d, the embodiment of the present disclosure relates to a model monitoring method that can be executed on a terminal. The method includes:

[0493] Step S3401, obtain ninth information.

[0494] The optional implementation method of step S3401 can refer to the optional implementation method of step S2101 in Figure 2, the optional implementation method of step S3101 in Figure 3a, the optional implementation method of step S3201 in Figure 3b, the optional implementation method of step S3301 in Figure 3c, and other related parts in the embodiment involved in Figure 2, other related parts in the embodiment involved in Figure 3a, other related parts in the embodiment involved in Figure 3b, and other related parts in the embodiment involved in Figure 3c, which will not be repeated here.

[0495] In some embodiments, step S3201 is omitted, and the terminal autonomously implements the function indicated by the ninth information, or the above function is default or by default.

[0496] Step S3402: perform beam prediction.

[0497] The optional implementation method of step S3402 can refer to the optional implementation method of step S2105 in Figure 2, the optional implementation method of step S3105 in Figure 3a, the optional implementation method of step S3204 in Figure 3b, the optional implementation method of step S3303 in Figure 3c, and other related parts in the embodiment involved in Figure 2, other related parts in the embodiment involved in Figure 3a, other related parts in the embodiment involved in Figure 3b, and other related parts in the embodiment involved in Figure 3c, which will not be repeated here.

[0498] Step S3403, sending the first information.

[0499] The optional implementation method of step S3403 can refer to the optional implementation method of step S2106 in Figure 2, the optional implementation method of step S3106 in Figure 3a, the optional implementation method of step S3205 in Figure 3b, the optional implementation method of step S3304 in Figure 3c, and other related parts in the embodiment involved in Figure 2, other related parts in the embodiment involved in Figure 3a, other related parts in the embodiment involved in Figure 3b, and other related parts in the embodiment involved in Figure 3c, which will not be repeated here.

[0500] Step S3404, sending the eighth information.

[0501] The optional implementation method of step S3404 can refer to the optional implementation method of step S2107 in Figure 2, the optional implementation method of step S3107 in Figure 3a, the optional implementation method of step S3206 in Figure 3b, the optional implementation method of step S3305 in Figure 3c, and other related parts in the embodiment involved in Figure 2, other related parts in the embodiment involved in Figure 3a, other related parts in the embodiment involved in Figure 3b, and other related parts in the embodiment involved in Figure 3c, which will not be repeated here.

[0502] Figure 3e is a flow chart of another model monitoring method according to an exemplary embodiment. As shown in Figure 3e, the embodiment of the present disclosure relates to a model monitoring method that can be executed on a terminal. The method includes:

[0503] Step S3501: perform beam prediction.

[0504] For the optional implementation of step S3501, please refer to the optional implementation of step S2105 in Figure 2, the optional implementation of step S3105 in Figure 3a, the optional implementation of step S3204 in Figure 3b, the optional implementation of step S3303 in Figure 3c, the optional implementation of step S3402 in Figure 3d, and other related parts in the embodiment involved in Figure 2, other related parts in the embodiment involved in Figure 3a, other related parts in the embodiment involved in Figure 3b, other related parts in the embodiment involved in Figure 3c, and other related parts in the embodiment involved in Figure 3d, which will not be repeated here.

[0505] Step S3502, sending the first information.

[0506] For the optional implementation of step S3502, please refer to the optional implementation of step S2106 in Figure 2, the optional implementation of step S3106 in Figure 3a, the optional implementation of step S3205 in Figure 3b, the optional implementation of step S3304 in Figure 3c, the optional implementation of step S3403 in Figure 3d, and other related parts in the embodiment involved in Figure 2, other related parts in the embodiment involved in Figure 3a, other related parts in the embodiment involved in Figure 3b, other related parts in the embodiment involved in Figure 3c, and other related parts in the embodiment involved in Figure 3d, which will not be repeated here.

[0507] Step S3503, sending the eighth information.

[0508] For the optional implementation of step S3503, please refer to the optional implementation of step S2107 in Figure 2, the optional implementation of step S3107 in Figure 3a, the optional implementation of step S3206 in Figure 3b, the optional implementation of step S3305 in Figure 3c, the optional implementation of step S3404 in Figure 3d, and other related parts in the embodiment involved in Figure 2, other related parts in the embodiment involved in Figure 3a, other related parts in the embodiment involved in Figure 3b, other related parts in the embodiment involved in Figure 3c, and other related parts in the embodiment involved in Figure 3d, which will not be repeated here.

[0509] Figure 3f is a flow chart of another model monitoring method according to an exemplary embodiment. As shown in Figure 3f, the embodiment of the present disclosure relates to a model monitoring method that can be executed on a terminal. The method includes:

[0510] Step S3601, sending the first information.

[0511] For the optional implementation of step S3601, please refer to the optional implementation of step S2106 in Figure 2, the optional implementation of step S3106 in Figure 3a, the optional implementation of step S3205 in Figure 3b, the optional implementation of step S3304 in Figure 3c, the optional implementation of step S3403 in Figure 3d, the optional implementation of step S3502 in Figure 3e, and other related parts in the embodiment involved in Figure 2, other related parts in the embodiment involved in Figure 3a, other related parts in the embodiment involved in Figure 3b, other related parts in the embodiment involved in Figure 3c, other related parts in the embodiment involved in Figure 3d, and other related parts in the embodiment involved in Figure 3e, which will not be repeated here.

[0512] Step S3602, obtain the eighth information.

[0513] For the optional implementation of step S3602, please refer to the optional implementation of step S2107 in Figure 2, the optional implementation of step S3107 in Figure 3a, the optional implementation of step S3206 in Figure 3b, the optional implementation of step S3305 in Figure 3c, the optional implementation of step S3404 in Figure 3d, the optional implementation of step S3503 in Figure 3e, and other related parts in the embodiment involved in Figure 2, other related parts in the embodiment involved in Figure 3a, other related parts in the embodiment involved in Figure 3b, other related parts in the embodiment involved in Figure 3c, other related parts in the embodiment involved in Figure 3d, and other related parts in the embodiment involved in Figure 3e, which will not be repeated here.

[0514] FIG3g is a flow chart of another model monitoring method according to an exemplary embodiment. As shown in FIG3g, the embodiment of the present disclosure relates to a model monitoring method, which can be executed on a terminal. The method includes:

[0515] Step S3701, sending the first information.

[0516] For the optional implementation of step S3101, please refer to the optional implementation of step S2106 in Figure 2, the optional implementation of step S3106 in Figure 3a, the optional implementation of step S3205 in Figure 3b, the optional implementation of step S3304 in Figure 3c, the optional implementation of step S3403 in Figure 3d, the optional implementation of step S3502 in Figure 3e, the optional implementation of step S3601 in Figure 3f, and other related parts in the embodiment involved in Figure 2, other related parts in the embodiment involved in Figure 3a, other related parts in the embodiment involved in Figure 3b, other related parts in the embodiment involved in Figure 3c, other related parts in the embodiment involved in Figure 3d, other related parts in the embodiment involved in Figure 3e, and other related parts in the embodiment involved in Figure 3f, which will not be repeated here.

[0517] In some embodiments, the AI ​​model is deployed on a network device; the first information includes at least one of the following: second information corresponding to the beams and / or beam pairs in the first set, the second information is used to represent first parameters of the beams and / or beam pairs in the first set, the beams and / or beam pairs in the first set correspond to inputs of the AI ​​model, and the first parameters include measured identifiers and / or measured values; third information corresponding to the beams and / or beam pairs in the second set, the third information is used to represent first parameters of the beams and / or beam pairs in the second set, the second set includes the first set, or the first set and the second set are different, the output of the AI ​​model is used to obtain second parameters of the beams and / or beam pairs in the second set, and the second parameters include predicted identifiers and / or predicted values;

[0518] In some embodiments, the AI ​​model is deployed on the terminal; the first information includes at least one of the following: fourth information, the fourth information is used to indicate the AI ​​operation on the AI ​​model; third information and fifth information corresponding to the beams and / or beam pairs in the second set, wherein the third information is used to represent the first parameters of the beams and / or beam pairs in the second set, and the fifth information is used to represent the second parameters of the beams and / or beam pairs in the second set, and the fifth information is determined based on the output of the AI ​​model, the first parameter includes the measured identifier and / or measured value, and the second parameter includes the predicted identifier and / or predicted value; the first indicator, the first indicator is used to represent the prediction accuracy of the second number of beams and / or beam pairs.

[0519] In some embodiments, sending the first information includes at least one of the following: the second parameter obtained by the AI ​​model output N times meets the first condition, and the first information is sent, wherein the first condition is used to indicate that the AI ​​model prediction is accurate; the second parameter obtained by the AI ​​model output M times meets the second condition, and the first information is sent, wherein the second condition is used to indicate that the AI ​​model prediction is inaccurate.

[0520] In some embodiments, sending the first information also includes at least one of the following: within the first time, the second parameter obtained by the AI ​​model output N times meets the first condition, and the first information is sent; within the first time, the second parameter obtained by the AI ​​model output M times meets the second condition, and the first information is sent.

[0521] In some embodiments, the first condition includes at least one of the following: the K target beams and / or beam pairs obtained by the AI ​​model output include the L target beams and / or beam pairs measured in the second set, and the L target beams and / or beam pairs are the first L beams and / or beam pairs in the second set sorted from high to low according to the measurement value; the 1 target beam and / or beam pair obtained by the AI ​​model output is among the L target beams and / or beam pairs measured in the second set; the measurement value corresponding to the predicted identifier of the 1 target beam and / or beam pair obtained by the AI ​​model output is the first one between the measurement value of the 1 target beam and / or beam pair measured in the second set. A difference is less than or equal to a first threshold; a second difference between a predicted value corresponding to a predicted identifier of a target beam and / or beam pair obtained by the AI ​​model output and a measured value corresponding to the predicted identifier is less than or equal to a second threshold; a throughput difference is greater than or equal to a third threshold, wherein the throughput difference represents a difference between the throughput corresponding to one beam and / or beam pair among the K target beams and / or beam pairs obtained based on the AI ​​model output and the throughput corresponding to one beam and / or beam pair among the L target beams and / or beam pairs in the second set; the reference signal resource overhead sent by the network device is less than or equal to a fourth threshold; the overhead of the second information is less than or equal to a fifth threshold.

[0522] In some embodiments, the second condition includes at least one of the following: the K target beams and / or beam pairs obtained by the AI ​​model output do not include the L target beams and / or beam pairs measured in the second set, and the L target beams and / or beam pairs are the first L beams and / or beam pairs in the second set sorted from high to low according to the measurement value; the 1 target beam and / or beam pair obtained by the AI ​​model output is not among the L target beams and / or beam pairs measured in the second set; the measurement value corresponding to the predicted identifier of the 1 target beam and / or beam pair obtained by the AI ​​model output is the same as the 1 target beam and / or beam pair measured in the second set. is greater than a first threshold; a second difference between a predicted value corresponding to a predicted identifier of a target beam and / or beam pair obtained by the AI ​​model output and a measured value corresponding to the predicted identifier is greater than a second threshold; a throughput difference is less than a third threshold, wherein the throughput difference represents a difference between a throughput corresponding to one beam and / or beam pair among the K target beams and / or beam pairs obtained based on the AI ​​model output and a throughput corresponding to one beam and / or beam pair among the L target beams and / or beam pairs in the second set; the reference signal resource overhead sent by the network device is greater than a fourth threshold; the overhead of the second information is greater than a fifth threshold.

[0523] In some embodiments, the AI ​​model is deployed on a network device; sending the first information includes: periodically sending the first information.

[0524] In some embodiments, the first information is carried by at least one of radio resource control RRC signaling, uplink media access control element UL MAC CE, and uplink control information UCI.

[0525] In some embodiments, the method further includes: sending sixth information, where the sixth information is used to request monitoring of the AI ​​model.

[0526] In some embodiments, the method also includes at least one of the following: determining the first information at a second time after sending the sixth information; determining the first information at a third time after receiving the seventh information, wherein the seventh information is feedback from the network device based on the sixth information; determining the first information after the network device sends a reference signal for AI model monitoring.

[0527] In some embodiments, the sixth information is carried in at least one of the following ways: a scheduling request SR; an uplink medium access control UL MAC; or a random access procedure RA.

[0528] In some embodiments, the method further includes: receiving sixth information, where the sixth information is used to request monitoring of the AI ​​model.

[0529] In some embodiments, the method further includes: determining the first information a fourth time after receiving the sixth information.

[0530] In some embodiments, the sixth information is carried in at least one of the following ways: RRC; downlink media access control element DL MAC CE; paging; downlink control information DCI.

[0531] In some embodiments, one AI model corresponds to one or more AI functions, and / or one AI function corresponds to one or more AI models.

[0532] In some embodiments, the AI ​​model is deployed on the terminal; the method also includes: receiving eighth information, where the eighth information is used to instruct the terminal to perform AI operations on the AI ​​model.

[0533] In some embodiments, the AI ​​operation includes at least one of: activating an AI model and / or an AI function; deactivating an AI model and / or an AI function; switching an AI model and / or an AI function; updating an AI model and / or an AI function; switching to a non-AI mode and / or a non-AI function.

[0534] In some embodiments, the method further includes: receiving ninth information, where the ninth information is used to configure reference signal resources, and the reference signal resources are used to perform beam measurement.

[0535] In some embodiments, the measured value and / or predicted value includes at least one of the following: layer 1 reference signal received power L1-RSRP; layer 1 signal to interference and noise ratio L1-SINR.

[0536] FIG4a is a flow chart of another model monitoring method according to an exemplary embodiment. As shown in FIG4a, the embodiment of the present disclosure relates to a model monitoring method that can be executed on a network device. The method includes:

[0537] Step S4101, sending the ninth message.

[0538] The optional implementation of step S4101 can refer to the optional implementation of step S2101 in Figure 2 and other related parts in the embodiment involved in Figure 2, which will not be repeated here.

[0539] Step S4102, obtaining sixth information.

[0540] The optional implementation of step S4102 can refer to the optional implementation of step S2102 in Figure 2 and other related parts in the embodiment involved in Figure 2, which will not be repeated here.

[0541] In some embodiments, the network device obtains sixth information specified by the protocol.

[0542] In some embodiments, the network device obtains the sixth information from a higher layer.

[0543] In some embodiments, the network device performs processing to obtain the sixth information.

[0544] In some embodiments, step S4102 is omitted, and the network device autonomously implements the function indicated by the sixth information, or the above function is default or by default.

[0545] Step S4103, sending the seventh information.

[0546] The optional implementation of step S4103 can refer to the optional implementation of step S2103 in Figure 2 and other related parts in the embodiment involved in Figure 2, which will not be repeated here.

[0547] Step S4104, sending the sixth information.

[0548] The optional implementation of step S4104 can refer to the optional implementation of step S2104 in Figure 2 and other related parts in the embodiment involved in Figure 2, which will not be repeated here.

[0549] Of course, step S4102, step S4103, and step S4104 may be performed alternatively. For example, step S4102 and step S4103 may be performed, but step S4104 may not be performed; or step S4104 may be performed, but step S4102 and step S4103 may not be performed.

[0550] Step S4105, obtaining first information.

[0551] The optional implementation of step S4105 can refer to the optional implementation of step S2106 in Figure 2 and other related parts in the embodiment involved in Figure 2, which will not be repeated here.

[0552] In some embodiments, the network device obtains first information specified by a protocol.

[0553] In some embodiments, the network device obtains the first information from a higher layer.

[0554] In some embodiments, the network device performs processing to obtain the first information.

[0555] In some embodiments, step S4105 is omitted, and the network device autonomously implements the function indicated by the first information, or the above function is default or by default.

[0556] Step S4106, sending the eighth information.

[0557] The optional implementation of step S4106 can refer to the optional implementation of step S2107 in Figure 2 and other related parts in the embodiment involved in Figure 2, which will not be repeated here.

[0558] FIG4b is a flow chart of another model monitoring method according to an exemplary embodiment. As shown in FIG4b, the embodiment of the present disclosure relates to a model monitoring method that can be executed on a network device. The method includes:

[0559] Step S4201, sending the ninth message.

[0560] The optional implementation of step S4201 can refer to the optional implementation of step S2101 in Figure 2, the optional implementation of step S4101 in Figure 4a, and other related parts in the embodiment involved in Figure 2 and other related parts in the embodiment involved in Figure 4a, which will not be repeated here.

[0561] Step S4202, obtain sixth information.

[0562] The optional implementation of step S4202 can refer to the optional implementation of step S2102 in Figure 2, the optional implementation of step S4102 in Figure 4a, and other related parts in the embodiment involved in Figure 2 and other related parts in the embodiment involved in Figure 4a, which will not be repeated here.

[0563] In some embodiments, step S4102 is omitted, and the network device autonomously implements the function indicated by the sixth information, or the above function is default or by default.

[0564] Step S4203, sending the seventh information.

[0565] The optional implementation of step S4203 can refer to the optional implementation of step S2103 in Figure 2, the optional implementation of step S4103 in Figure 4a, and other related parts in the embodiment involved in Figure 2 and other related parts in the embodiment involved in Figure 4a, which will not be repeated here.

[0566] Step S4204, obtaining first information.

[0567] The optional implementation of step S4204 can refer to the optional implementation of step S2106 in Figure 2, the optional implementation of step S4105 in Figure 4a, and other related parts in the embodiment involved in Figure 2 and other related parts in the embodiment involved in Figure 4a, which will not be repeated here.

[0568] In some embodiments, step S4204 is omitted, and the network device autonomously implements the function indicated by the first information, or the above function is default or by default.

[0569] Step S4205, sending the eighth information.

[0570] The optional implementation of step S4205 can refer to the optional implementation of step S2107 in Figure 2, the optional implementation of step S4106 in Figure 4a, and other related parts in the embodiment involved in Figure 2 and other related parts in the embodiment involved in Figure 4a, which will not be repeated here.

[0571] FIG4c is a flow chart of another model monitoring method according to an exemplary embodiment. As shown in FIG4c, the embodiment of the present disclosure relates to a model monitoring method that can be executed on a network device. The method includes:

[0572] Step S4301, sending the ninth message.

[0573] The optional implementation method of step S4301 can refer to the optional implementation method of step S2101 in Figure 2, the optional implementation method of step S4101 in Figure 4a, the optional implementation method of step S4201 in Figure 4b, and other related parts in the embodiment involved in Figure 2, other related parts in the embodiment involved in Figure 4a, and other related parts in the embodiment involved in Figure 4b, which will not be repeated here.

[0574] Step S4302, sending the sixth information.

[0575] The optional implementation of step S4302 can refer to the optional implementation of step S2102 in Figure 2, the optional implementation of step S4102 in Figure 4a, and other related parts in the embodiment involved in Figure 2 and other related parts in the embodiment involved in Figure 4a, which will not be repeated here.

[0576] Step S4303, obtain first information.

[0577] The optional implementation method of step S4303 can be found in the optional implementation method of step S2106 in Figure 2, the optional implementation method of step S4105 in Figure 4a, the optional implementation method of step S4204 in Figure 4b, and other related parts in the embodiment involved in Figure 2, other related parts in the embodiment involved in Figure 4a, and other related parts in the embodiment involved in Figure 4b, which will not be repeated here.

[0578] In some embodiments, step S4303 is omitted, and the network device autonomously implements the function indicated by the first information, or the above function is default or by default.

[0579] Step S4304, sending the eighth information.

[0580] The optional implementation method of step S4304 can refer to the optional implementation method of step S2107 in Figure 2, the optional implementation method of step S4106 in Figure 4a, the optional implementation method of step S4205 in Figure 4b, and other related parts in the embodiment involved in Figure 2, other related parts in the embodiment involved in Figure 4a, and other related parts in the embodiment involved in Figure 4b, which will not be repeated here.

[0581] FIG4d is a flow chart of another model monitoring method according to an exemplary embodiment. As shown in FIG4d , the embodiment of the present disclosure relates to a model monitoring method that can be executed on a network device. The method includes:

[0582] Step S4401, sending the ninth message.

[0583] The optional implementation method of step S4401 can refer to the optional implementation method of step S2101 in Figure 2, the optional implementation method of step S4101 in Figure 4a, the optional implementation method of step S4201 in Figure 4b, the optional implementation method of step S4301 in Figure 4c, and other related parts in the embodiment involved in Figure 2, other related parts in the embodiment involved in Figure 4a, other related parts in the embodiment involved in Figure 4b, and other related parts in the embodiment involved in Figure 4c, which will not be repeated here.

[0584] Step S4402, obtaining first information.

[0585] The optional implementation method of step S4402 can refer to the optional implementation method of step S2106 in Figure 2, the optional implementation method of step S4105 in Figure 4a, the optional implementation method of step S4204 in Figure 4b, the optional implementation method of step S4303 in Figure 4c, and other related parts in the embodiment involved in Figure 2, other related parts in the embodiment involved in Figure 4a, other related parts in the embodiment involved in Figure 4b, and other related parts in the embodiment involved in Figure 4c, which will not be repeated here.

[0586] In some embodiments, step S4402 is omitted, and the network device autonomously implements the function indicated by the first information, or the above function is default or by default.

[0587] Step S4403, sending the eighth information.

[0588] The optional implementation method of step S4403 can refer to the optional implementation method of step S2107 in Figure 2, the optional implementation method of step S4106 in Figure 4a, the optional implementation method of step S4205 in Figure 4b, the optional implementation method of step S4304 in Figure 4c, and other related parts in the embodiment involved in Figure 2, other related parts in the embodiment involved in Figure 4a, other related parts in the embodiment involved in Figure 4b, and other related parts in the embodiment involved in Figure 4c, which will not be repeated here.

[0589] Figure 4e is a flow chart of another model monitoring method according to an exemplary embodiment. As shown in Figure 4e, the embodiment of the present disclosure relates to a model monitoring method that can be executed on a network device. The method includes:

[0590] Step S4501, obtain first information.

[0591] For the optional implementation of step S4501, please refer to the optional implementation of step S2106 in Figure 2, the optional implementation of step S4105 in Figure 4a, the optional implementation of step S4204 in Figure 4b, the optional implementation of step S4303 in Figure 4c, the optional implementation of step S4402 in Figure 4d, and other related parts in the embodiment involved in Figure 2, other related parts in the embodiment involved in Figure 4a, other related parts in the embodiment involved in Figure 4b, other related parts in the embodiment involved in Figure 4c, and other related parts in the embodiment involved in Figure 4d, which will not be repeated here.

[0592] In some embodiments, step S4501 is omitted, and the network device autonomously implements the function indicated by the first information, or the above function is default or by default.

[0593] Step S4502, sending the eighth information.

[0594] The optional implementation method of step S4502 can refer to the optional implementation method of step S2107 in Figure 2, the optional implementation method of step S4106 in Figure 4a, the optional implementation method of step S4205 in Figure 4b, the optional implementation method of step S4304 in Figure 4c, the optional implementation method of step S4403 in Figure 4d, and other related parts in the embodiment involved in Figure 2, other related parts in the embodiment involved in Figure 4a, other related parts in the embodiment involved in Figure 4b, other related parts in the embodiment involved in Figure 4c, and other related parts in the embodiment involved in Figure 4d, which will not be repeated here.

[0595] FIG4f is a flow chart of another model monitoring method according to an exemplary embodiment. As shown in FIG4f , the embodiment of the present disclosure relates to a model monitoring method that can be executed on a network device. The method includes:

[0596] Step S4601, obtain first information.

[0597] For the optional implementation of step S4601, please refer to the optional implementation of step S2106 in Figure 2, the optional implementation of step S4105 in Figure 4a, the optional implementation of step S4204 in Figure 4b, the optional implementation of step S4303 in Figure 4c, the optional implementation of step S4402 in Figure 4d, the optional implementation of step S4501 in Figure 4e, and other related parts in the embodiment involved in Figure 2, other related parts in the embodiment involved in Figure 4a, other related parts in the embodiment involved in Figure 4b, other related parts in the embodiment involved in Figure 4c, other related parts in the embodiment involved in Figure 4d, and other related parts in the embodiment involved in Figure 4e, which will not be repeated here.

[0598] In some embodiments, the AI ​​model is deployed on a network device; the first information includes at least one of the following: second information corresponding to the beams and / or beam pairs in the first set, the second information is used to represent the first parameters of the beams and / or beam pairs in the first set, the beams and / or beam pairs in the first set correspond to the input of the AI ​​model, and the first parameters include measured identifiers and / or measured values; third information corresponding to the beams and / or beam pairs in the second set, the third information is used to represent the first parameters of the beams and / or beam pairs in the second set, the second set includes the first set, or the first set and the second set are different, the output of the AI ​​model is used to obtain the second parameters of the beams and / or beam pairs in the second set, and the second parameters include predicted identifiers and / or predicted values.

[0599] In some embodiments, the AI ​​model is deployed on the terminal; the first information includes at least one of the following: fourth information, the fourth information is used to indicate the AI ​​operation on the AI ​​model; third information and fifth information corresponding to the beams and / or beam pairs in the second set, wherein the third information is used to represent the first parameters of the beams and / or beam pairs in the second set, and the fifth information is used to represent the second parameters of the beams and / or beam pairs in the second set, and the fifth information is determined based on the output of the AI ​​model, the first parameter includes the measured identifier and / or measured value, and the second parameter includes the predicted identifier and / or predicted value; the first indicator, the first indicator is used to represent the prediction accuracy of the second number of beams and / or beam pairs.

[0600] In some embodiments, the first information sent by the receiving terminal includes at least one of the following: the second parameter obtained by the AI ​​model output N times satisfies the first condition, and the first information is sent by the receiving terminal, wherein the first condition is used to indicate that the AI ​​model prediction is accurate; the second parameter obtained by the AI ​​model output M times satisfies the second condition, and the first information is sent by the receiving terminal, wherein the second condition is used to indicate that the AI ​​model prediction is inaccurate.

[0601] In some embodiments, the first information sent by the receiving terminal also includes at least one of the following: within the first time, the second parameter obtained by the AI ​​model output N times meets the first condition, and the first information is sent by the receiving terminal; within the first time, the second parameter obtained by the AI ​​model output M times meets the second condition, and the first information is sent by the receiving terminal.

[0602] In some embodiments, the first condition includes at least one of the following: the K target beams and / or beam pairs obtained by the AI ​​model output include the L target beams and / or beam pairs measured in the second set, and the L target beams and / or beam pairs are the first L beams and / or beam pairs in the second set sorted from high to low according to the measurement value; the 1 target beam and / or beam pair obtained by the AI ​​model output is among the L target beams and / or beam pairs measured in the second set; the measurement value corresponding to the predicted identifier of the 1 target beam and / or beam pair obtained by the AI ​​model output is the first one between the measurement value of the 1 target beam and / or beam pair measured in the second set. A difference is less than or equal to a first threshold; a second difference between a predicted value corresponding to a predicted identifier of a target beam and / or beam pair obtained by the AI ​​model output and a measured value corresponding to the predicted identifier is less than or equal to a second threshold; a throughput difference is greater than or equal to a third threshold, wherein the throughput difference represents a difference between the throughput corresponding to one beam and / or beam pair among the K target beams and / or beam pairs obtained based on the AI ​​model output and the throughput corresponding to one beam and / or beam pair among the L target beams and / or beam pairs in the second set; the reference signal resource overhead sent by the network device is less than or equal to a fourth threshold; the overhead of the second information is less than or equal to a fifth threshold.

[0603] In some embodiments, the second condition includes at least one of the following: the K target beams and / or beam pairs obtained by the AI ​​model output do not include the L target beams and / or beam pairs measured in the second set, and the L target beams and / or beam pairs are the first L beams and / or beam pairs in the second set sorted from high to low according to the measurement value; the 1 target beam and / or beam pair obtained by the AI ​​model output is not among the L target beams and / or beam pairs measured in the second set; the measurement value corresponding to the predicted identifier of the 1 target beam and / or beam pair obtained by the AI ​​model output is the same as the 1 target beam and / or beam pair measured in the second set. is greater than a first threshold; a second difference between a predicted value corresponding to a predicted identifier of a target beam and / or beam pair obtained by the AI ​​model output and a measured value corresponding to the predicted identifier is greater than a second threshold; a throughput difference is less than a third threshold, wherein the throughput difference represents a difference between a throughput corresponding to one beam and / or beam pair among the K target beams and / or beam pairs obtained based on the AI ​​model output and a throughput corresponding to one beam and / or beam pair among the L target beams and / or beam pairs in the second set; the reference signal resource overhead sent by the network device is greater than a fourth threshold; the overhead of the second information is greater than a fifth threshold.

[0604] In some embodiments, the AI ​​model is deployed on a network device; receiving the first information sent by the terminal when the first condition is met includes: periodically receiving the first information sent by the terminal.

[0605] In some embodiments, the first information is carried by at least one of radio resource control (RRC) signaling, uplink media access control element (UL MAC CE), and uplink control information (UCI).

[0606] In some embodiments, the method further includes: receiving sixth information, where the sixth information is used to request monitoring of the AI ​​model.

[0607] In some embodiments, the first information sent by the receiving terminal under the first condition includes at least one of the following: receiving the first information after the second time after receiving the sixth information; receiving the first information after the third time after sending the seventh information, wherein the seventh information is feedback from the network device based on the sixth information; receiving the first information after sending a reference signal for AI model monitoring.

[0608] In some embodiments, the sixth information is carried in at least one of the following ways: a scheduling request SR; an uplink medium access control UL MAC; or a random access procedure RA.

[0609] In some embodiments, the method further includes: sending sixth information, where the sixth information is used to request monitoring of the AI ​​model.

[0610] In some embodiments, receiving the first information sent by the terminal when the first condition is met includes: receiving the first information after a fourth time after sending the sixth information.

[0611] In some embodiments, the sixth information is carried in at least one of the following ways: RRC; downlink media access control element DL MAC CE; paging; downlink control information DCI.

[0612] In some embodiments, the method also includes: obtaining a second indicator based on the first information, the second indicator being used to describe the performance of the AI ​​model and / or AI function; the second indicator including at least one of the following: prediction accuracy of the AI ​​model and / or AI function; average difference in layer 1 reference signal received power L1-RSRP; L1-RSRP difference; throughput difference; transmitted reference signal overhead; received UCI overhead.

[0613] In some embodiments, one AI model corresponds to one or more AI functions, and / or one AI function corresponds to one or more AI models.

[0614] In some embodiments, the AI ​​model is deployed on the terminal; the method also includes: sending eighth information, where the eighth information is used to instruct the terminal to perform AI operations on the AI ​​model.

[0615] In some embodiments, the AI ​​operation includes at least one of: activating an AI model and / or an AI function; deactivating an AI model and / or an AI function; switching an AI model and / or an AI function; updating an AI model and / or an AI function; switching to a non-AI mode and / or a non-AI function.

[0616] In some embodiments, the method further includes: sending ninth information, where the ninth information is used to configure reference signal resources, and the reference signal resources are used to perform beam measurement.

[0617] In some embodiments, the measured value and / or predicted value includes at least one of the following: L1-RSRP; L1-SINR.

[0618] FIG5 is a flow chart of another model monitoring method according to an exemplary embodiment. As shown in FIG5 , the embodiment of the present disclosure relates to a model monitoring method, which includes:

[0619] Step S5101: The terminal meets the first condition and sends the first information to the network device.

[0620] For optional implementations of step S5101, see the optional implementation of step S2106 in Figure 2, the optional implementation of step S3106 in Figure 3a, the optional implementation of step S3205 in Figure 3b, the optional implementation of step S3304 in Figure 3c, the optional implementation of step S3403 in Figure 3d, the optional implementation of step S3502 in Figure 3e, the optional implementation of step S3601 in Figure 3f, the optional implementation of step S3701 in Figure 3g, the optional implementation of step S4105 in Figure 4a, the optional implementation of step S4204 in Figure 4b, the optional implementation of step S4303 in Figure 4c, the optional implementation of step S4402 in Figure 4d, and the optional implementation of step S4501 in Figure 4e. 2 , other related parts in the embodiment involved in FIG3a , other related parts in the embodiment involved in FIG3b , other related parts in the embodiment involved in FIG3c , other related parts in the embodiment involved in FIG3d , other related parts in the embodiment involved in FIG3e , other related parts in the embodiment involved in FIG3f , other related parts in the embodiment involved in FIG3g , other related parts in the embodiment involved in FIG4a , other related parts in the embodiment involved in FIG4b , other related parts in the embodiment involved in FIG4c , other related parts in the embodiment involved in FIG4d , other related parts in the embodiment involved in FIG4e , and other related parts in the embodiment involved in FIG4f will not be repeated here.

[0621] Next, the present disclosure will describe the above solution with more detailed embodiments.

[0622] In some embodiments, the terminal receives configuration information, where the configuration information includes reference signal resource configuration information of set B and / or set A. When a trigger condition is met, the terminal sends model performance monitoring data to the network device, where the model performance monitoring data includes at least one of the following:

[0623] 1. For the network side model:

[0624] The beam information of setB measured by the terminal is used as input for the AI ​​model on the network side.

[0625] In some embodiments, the L1-RSRP of the beam (pair) of set B is included, or the L1-RSRP of the beam (pair) of set B and the beam (pair) ID.

[0626] In some embodiments, a beam refers to a beam; a beam measurement refers to measuring the L1-RSRP and / or L1-SINR of reference signals including SSBs, CSI-RSs, and SRSs; a beam indication refers to an indication of a transmission configuration indicator (TCI) state, where the TCI state includes at least one QCL type, including QCL Type A, B, C, and D, with Type D representing receive parameter information, commonly known as a beam. Types A, B, and C include at least one of parameters related to Doppler shift, Doppler spread, average delay, and delay spread. For an uplink beam, this refers to spatial relation information, spatial filter parameters, or an uplink TCI state.

[0627] In some embodiments, the terminal obtains beam information based on traditional methods.

[0628] In some embodiments, the N best beam (pair) IDs and / or corresponding L1-RSRPs in set A, or the L1-RSRPs of all beams (pairs) are included.

[0629] It should be noted that if set B is a subset of set A, the terminal only reports the L1-RSRP of all beams (pairs) in set A, which includes two parts of data.

[0630] 2. For the terminal side model:

[0631] In some embodiments, if model monitoring is performed on the terminal side, the terminal obtains the predicted set A information based on the measurement results of set B obtained by itself and the AI ​​model, and the AI ​​obtains the measured set A information based on the measurement, and compares the predicted set A information with the measured set A information to obtain a comparison result. If the comparison result meets the trigger condition, the terminal makes a decision, such as activating the model, deactivating the model, switching to a certain model, or falling back to the non-AI mode, or updating the model, etc., and then the terminal sends the decision made to the base station. That is to say, the model performance monitoring data includes the decision made by the terminal.

[0632] In some embodiments, if the network side performs model monitoring, the UE reports the beam information of set A obtained based on the AI ​​model and the beam information of set A actually measured, or the UE calculates the metric based on the beam information of set A obtained based on the AI ​​model and the beam information of set A actually measured, and reports the metric to the network.

[0633] In some embodiments, the trigger condition is: the AI ​​model has good performance for N consecutive times or the AI ​​model has poor performance for M consecutive times.

[0634] In some embodiments, if the AI ​​model performs well for N consecutive times, it is necessary to activate the AI ​​model or switch to the AI ​​model. After the trigger, the UE can activate the UE-side model itself or report relevant data to the network.

[0635] In some embodiments, if an AI model performs poorly for M consecutive times, it is necessary to deactivate the AI ​​model, switch to another AI model, revert to a non-AI model, or perform a model update. Once triggered, the UE can perform UE-side model operations or report relevant data to the network.

[0636] In some embodiments, it can also be combined with other methods, such as at least one of the following: within the first time; plus the ratio of various indicators.

[0637] In some embodiments, N consecutive times or M consecutive times must be continuous. If Ni is counted, it is no longer satisfied and the number is counted again.

[0638] In some embodiments, good AI model performance refers to at least one of the following: accurate beam prediction; L1-RSRP difference is lower than a threshold value; average UE throughput is higher than a threshold value; reference signaling overhead is lower than a threshold value; UCI overhead is lower than a threshold value.

[0639] In some embodiments, the beam prediction is accurate, that is, the actual best beam ID is within the predicted best X beam IDs, where X is a positive integer.

[0640] In some embodiments, for the network side model, the terminal may receive a beam indication sent by a network device and find that the indicated beam is a beam other than the best M beams obtained by the terminal by measuring set A. The value of M is greater than or equal to 1, and the maximum value of M can be determined according to the situation, for example, the maximum value of M is 4 or 5.

[0641] In some embodiments, this time needs to correspond, that is, the beam indication on the network side is one of the best M beams in set A obtained by the network side based on the measurement result of set B measured by the terminal at the first time as input, and the best M beams obtained by the terminal measuring set A are also obtained by the terminal's measurement result of set A measured at the first time. In other words, this beam indication is actually indicated after the first time, because it takes time for the terminal to measure set B and report it at the first time, and for the base station to obtain set A based on the AI ​​model and set B as input.

[0642] In some embodiments, poor AI model performance refers to at least one of the following: inaccurate beam prediction, i.e., the actual best beam ID is not within the predicted best X beam IDs, where X is a positive integer; L1-RSRP difference is higher than a threshold; average UE throughput is lower than a threshold; reference signaling overhead is higher than a threshold; and UCI overhead is higher than a threshold.

[0643] In some embodiments, related to signaling:

[0644] In some embodiments, for the AI ​​model on the network device side, the UE reports periodically or based on event triggering. The protocol defines an event as N consecutive times of good model performance or M consecutive times of poor AI model performance, and configures the N or M value to the UE; it can also be that the network side itself makes a judgment based on the AI ​​output and the actual measurement set A information reported by the terminal, but it does not involve UE-related behavioral changes and has no standardized impact.

[0645] The event may be, for example, a situation triggered by the first condition mentioned above.

[0646] In some embodiments, for the UE-side model (whether it is accurate is determined based on the model output and the best beam ID actually measured).

[0647] For example, the terminal performs monitoring: defines an event and configures the N or M value for the UE. The UE activates or deactivates the device based on the measurement and event.

[0648] For example, the terminal and network equipment monitor together: define the event, and configure the N or M value for the UE, and the UE reports the event when the event is met.

[0649] For example, network equipment monitors: define an event and configure N or M values ​​for the UE. When the event is met, the UE reports the N predicted outputs and the actual best beam ID.

[0650] In some embodiments, the terminal reports the model performance monitoring data through RRC signaling, that is, based on PUSCH, including dynamically scheduled or configure-granted PUSCH.

[0651] In some embodiments, before sending the model performance monitoring data, the terminal may initiate a model performance monitoring request to the network device based on SR, UL MAC or RA.

[0652] In some embodiments, the model performance monitoring data includes t time after the terminal device request transmission time, where t is a default value or a terminal indication or a network device indication.

[0653] In some embodiments, the model performance monitoring data includes a time t after the time when the network device sends feedback on the terminal request, where t is a default value or an indication from the network device.

[0654] In some embodiments, the model performance monitoring data includes starting from the network device starting to send a reference signal for model performance monitoring.

[0655] In some embodiments, such as spatial prediction, model monitoring is required, and reference signals for beam measurement need to be sent based on all Tx beams.

[0656] In some embodiments, such as time domain prediction, a reference signal for beam measurement needs to be sent in each time period.

[0657] In some embodiments, before sending the model performance monitoring data, the terminal: receives a model performance monitoring request initiated by a network device based on RRC, DL MAC, paging, or DCI scheduling PUSCH.

[0658] In some embodiments, the model performance monitoring data includes t time after the network device request transmission time, where t is a default value or an indication from the network device.

[0659] In some embodiments, before the terminal sends the model performance monitoring data: the network device obtains the model performance indicator based on the model performance monitoring data reported by the terminal.

[0660] In some embodiments, the performance metric includes prediction accuracy (probability). Prediction accuracy means that the predicted N strongest reference signal IDs include the actual strongest reference signal ID. The reference signal ID can be an SSB ID, CSI-RS ID, or SRS ID, where N is a positive integer and is 1 or greater. The strongest refers to the strongest L1-RSRP or L1-SINR. This metric is a ratio that requires multiple predictions to measure accuracy.

[0661] In some embodiments, the performance indicator includes an average L1-RSRP difference (dB difference). The difference is the actual L1-RSRP corresponding to the strongest reference signal ID and the actual L1-RSRP corresponding to the strongest reference signal ID. Or, the difference is the actual L1-RSRP corresponding to the strongest reference signal ID and the actual L1-RSRP corresponding to the strongest reference signal ID. Or, the difference is the actual L1-RSRP corresponding to the strongest reference signal ID and the predicted L1-RSRP corresponding to the strongest reference signal ID. Or, the difference is the actual L1-RSRP corresponding to the strongest reference signal ID and the predicted L1-RSRP corresponding to the strongest reference signal ID.

[0662] In some embodiments, any of the above performance indicators can be viewed as a single prediction result, or an average of multiple prediction results.

[0663] In some embodiments, the performance metric includes the L1-RSRP difference (dB difference) corresponding to a certain percentage point on the L1-RSRP difference CDF curve. This metric can be viewed based on a CDF curve derived from multiple predictions of the two metrics. Alternatively, the L1-RSRP difference in this metric can be understood as a CDF curve derived from the difference between the predicted L1-RSRP for each reference signal ID, derived from one or more predictions, and the actual L1-RSRP corresponding to each reference signal ID.

[0664] In some embodiments, the performance metric includes an average UE throughput difference. This metric is calculated by calculating the capacity based on the Shannon capacity, using the SINRs of the predicted strongest reference signal and the actual strongest reference signal. This metric can be calculated as a single prediction result or the average of multiple prediction results.

[0665] In some embodiments, the performance indicators include other indicators such as reference signal overhead, uplink control information overhead, etc., which can be obtained without being based on the performance monitoring data of the model in the second embodiment.

[0666] In some embodiments, the method involved in any of the above embodiments is applicable to model-based performance monitoring and functionality-based performance monitoring.

[0667] For example, the principle of managing models based on AI function identifiers: For example, if the terminal reports that it supports AI function identifier #1, which is the spatial beam prediction function, then the terminal may have multiple models for this function. However, the base station does not need to know about the switching between these multiple models, as the terminal can implement it on its own. Therefore, the terminal does not need to inform the base station of these specific model-related information. Model management based on model identifiers, on the other hand, requires the terminal to inform the base station of each of its models, and any switching between two models or the activation / deactivation of a model requires notifying the base station.

[0668] In some embodiments, the terminal receives instructions such as model activation / deactivation / switching / fine-tuning sent by the network device.

[0669] In some embodiments, the network device determines whether to perform model activation / deactivation / switching / fine-tuning based on the model performance monitoring data or model performance indicators sent by the terminal, and sends an instruction to the terminal.

[0670] In some embodiments, the MAC address is indicated by at least one of DCI, MAC CE, and RRC.

[0671] In some embodiments, for example, RRC signaling provides a correspondence between multiple indices and specific parameters, and the MAC CE or DCI may indicate the index.

[0672] In some embodiments, or RRC signaling provides a correspondence between multiple indices and specific parameters, MAC CE activates a part of the indices, and DCI indicates one of the part of the indices activated by MAC CE.

[0673] In some embodiments, it is used to indicate the updated model parameter configuration index (or specific parameters), or the model index, or the model version index.

[0674] In some embodiments, the model index includes different indices corresponding to models with different functions. For example, some models are used for CSI compression, some models are used for beam prediction, and some models are used for positioning prediction.

[0675] In some embodiments, the model index may also include different versions of AI models with the same function or different models corresponding to different parameter configurations.

[0676] In some embodiments, the model version index means that if the same functional AI model corresponds to the same model index, then models with the same function but different versions or different parameter configurations can use the model version index.

[0677] In some embodiments, it is used to indicate switching to a non-AI mechanism.

[0678] In the embodiments of the present disclosure, each step can be implemented as an independent embodiment. Part or all of the steps and their optional implementations can be arbitrarily combined with part or all of the steps in other embodiments, or with optional implementations of other embodiments.

[0679] The embodiments of the present disclosure also provide an apparatus for implementing any of the above methods. For example, a random access apparatus is provided, which includes units or modules for implementing each step performed by a first device (e.g., a terminal, an access network device, a core network function node, a core network device, etc.) in any of the above methods. For another example, another random access apparatus is provided, which includes units or modules for implementing each step performed by a second device (e.g., a terminal, an access network device, a core network function node, a core network device, etc.) in any of the above methods.

[0680] It should be understood that the division of the various units or modules in the above device is merely a division of logical functions. In actual implementation, they may be fully or partially integrated into a physical entity, or they may be physically separated. In addition, the units or modules in the device may be implemented in the form of a processor calling software: for example, the device includes a processor, the processor is connected to a memory, and the memory stores instructions. The processor calls the instructions stored in the memory to implement any of the above methods or implement the functions of the various units or modules of the above device, wherein the processor is, for example, a general-purpose processor, such as a central processing unit (CPU) or a microprocessor, and the memory is a memory within the device or a memory outside the device. Alternatively, the units or modules in the device can be implemented in the form of hardware circuits, and the functions of some or all of the units or modules can be realized by designing the hardware circuits. The above-mentioned hardware circuits can be understood as one or more processors; for example, in one implementation, the above-mentioned hardware circuit is an application-specific integrated circuit (ASIC), which realizes the functions of some or all of the above units or modules by designing the logical relationship of the components in the circuit; for example, in another implementation, the above-mentioned hardware circuit can be realized by a programmable logic device (PLD). Taking a field programmable gate array (FPGA) as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by configuring the configuration file, thereby realizing the functions of some or all of the above units or modules. All units or modules of the above devices can be realized in the form of software called by the processor, or in the form of hardware circuits, or in part by the form of software called by the processor, and the rest by hardware circuits.

[0681] In the embodiments of the present disclosure, the processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), or a digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationship of the hardware circuit. The logical relationship of the above-mentioned hardware circuit is fixed or reconfigurable. For example, the processor is a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and implementing the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units or modules. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc.

[0682] Figure 6a is a schematic diagram of a model monitoring device according to an exemplary embodiment. As shown in Figure 6a, the model monitoring device 6100 can be, for example, the terminal mentioned above, and the device 6100 includes: a transceiver module 6101. Of course, the device 6100 can also include any possible modules such as a processing module, which is not limited in this disclosure. In some embodiments, the transceiver module 6101 is used to meet the first condition and send the first information. Optionally, the transceiver module 6101 is used to execute at least one of the communication steps S2101, S2102, S2103, S2104, S2106, and S2107 such as sending and / or receiving executed by the terminal in any of the above methods, but is not limited to this and will not be repeated here. Optionally, the processing module is used to execute other steps S2105 executed by the terminal in any of the above methods, but is not limited to this and will not be repeated here.

[0683] Figure 6b is a schematic diagram of another model monitoring device according to an exemplary embodiment. As shown in Figure 6b, the model monitoring device 6200 can be, for example, the network device mentioned above, and the device 6200 includes: a transceiver module 6201. Of course, the device 6200 can also include any possible modules such as a processing module, which is not limited in this disclosure. In some embodiments, the above-mentioned transceiver module 6201 is used to receive the first information. Optionally, the above-mentioned transceiver module 6201 is used to execute at least one of the communication steps S2101, step S2102, step S2103, step S2104, step S2106, and step S2107 such as sending and / or receiving performed by the second device in any of the above methods, but is not limited to this and will not be repeated here.

[0684] Figure 7a is a schematic diagram of the structure of a communication device 7100 proposed in an embodiment of the present disclosure. Communication device 7100 can be a network device (e.g., an access network device, a core network device, etc.), a terminal (e.g., a user equipment, etc.), a chip, a chip system, or a processor that supports a network device to implement any of the above methods, or a chip, a chip system, or a processor that supports a terminal to implement any of the above methods. Communication device 7100 can be used to implement the methods described in the above method embodiments. For details, please refer to the description of the above method embodiments.

[0685] As shown in Figure 7a, the communication device 7100 includes one or more processors 7101. The processor 7101 can be a general-purpose processor or a dedicated processor, for example, a baseband processor or a central processing unit. The baseband processor can be used to process communication protocols and communication data, and the central processing unit can be used to control the communication device (such as a base station, baseband chip, terminal device, terminal device chip, DU or CU, etc.), execute programs, and process program data. The communication device 7100 is used to perform any of the above methods.

[0686] In some embodiments, the communication device 7100 further includes one or more memories 7102 for storing instructions. Optionally, all or part of the memories 7102 may be located outside the communication device 7100.

[0687] In some embodiments, the communication device 7100 further includes one or more transceivers 7103. When the communication device 7100 includes one or more transceivers 7103, the transceiver 7103 performs at least one of the steps S2101, S2102, S2103, S2104, S2106, and S2107 of the above method for transmitting and / or receiving communications, but is not limited thereto. The processor 7101 performs the other step S2105, but is not limited thereto.

[0688] In some embodiments, a transceiver may include a receiver and / or a transmitter. The receiver and transmitter may be separate or integrated. Optionally, the terms transceiver, transceiver unit, transceiver, and transceiver circuit may be used interchangeably; the terms transmitter, transmitting unit, transmitter, and transmitting circuit may be used interchangeably; and the terms receiver, receiving unit, receiver, and receiving circuit may be used interchangeably.

[0689] In some embodiments, the communication device 7100 may include one or more interface circuits 7104. Optionally, the interface circuit 7104 is connected to the memory 7102. The interface circuit 7104 may be configured to receive signals from the memory 7102 or other devices, and may be configured to send signals to the memory 7102 or other devices. For example, the interface circuit 7104 may read instructions stored in the memory 7102 and send the instructions to the processor 7101.

[0690] The communication device 7100 described in the above embodiment may be a network device or a terminal, but the scope of the communication device 7100 described in the present disclosure is not limited thereto, and the structure of the communication device 7100 may not be limited by FIG. 7a. The communication device may be an independent device or may be part of a larger device. For example, the communication device may be: 1) an independent integrated circuit IC, or a chip, or a chip system or subsystem; (2) a collection of one or more ICs, optionally, the above IC collection may also include a storage component for storing data or programs; (3) an ASIC, such as a modem; (4) a module that can be embedded in other devices; (5) a receiver, a terminal device, an intelligent terminal device, a cellular phone, a wireless device, a handheld device, a mobile unit, an in-vehicle device, a network device, a cloud device, an artificial intelligence device, etc.; (6) others, etc.

[0691] FIG7 b is a schematic diagram of the structure of a chip 7200 according to an embodiment of the present disclosure. If the communication device 7100 can be a chip or a chip system, reference can be made to the schematic diagram of the structure of the chip 7200 shown in FIG7 b , but the present disclosure is not limited thereto.

[0692] The chip 7200 includes one or more processors 7201 , and the chip 7200 is configured to execute any of the above methods.

[0693] In some embodiments, the chip 7200 further includes one or more interface circuits 7202. Optionally, the interface circuit 7202 is connected to the memory 7203. The interface circuit 7202 can be used to receive signals from the memory 7203 or other devices, and can be used to send signals to the memory 7203 or other devices. For example, the interface circuit 7202 can read instructions stored in the memory 7203 and send the instructions to the processor 7201.

[0694] In some embodiments, the interface circuit 7202 executes at least one of the steps S2101, S2102, S2103, S2104, S2106, and S2107 of the above method for transmitting and / or receiving communications, but is not limited thereto. The processor 7201 executes the other step S2105, but is not limited thereto.

[0695] In some embodiments, terms such as interface circuit, interface, transceiver pin, and transceiver may be used interchangeably.

[0696] In some embodiments, the chip 7200 further includes one or more memories 7203 for storing instructions. Alternatively, all or part of the memories 7203 may be located outside the chip 7200.

[0697] The present disclosure also proposes a storage medium having instructions stored thereon. When the instructions are executed on the communication device 7100, the communication device 7100 executes any of the above methods. Optionally, the storage medium is an electronic storage medium. Optionally, the storage medium is a computer-readable storage medium, but is not limited thereto and may also be a storage medium readable by other devices. Optionally, the storage medium may be a non-transitory storage medium, but is not limited thereto and may also be a temporary storage medium.

[0698] The present disclosure also provides a program product, which, when executed by the communication device 7100, enables the communication device 7100 to perform any of the above methods. Optionally, the program product is a computer program product.

[0699] The present disclosure also proposes a computer program, which, when executed on a computer, causes the computer to perform any one of the above methods.

[0700] This disclosure proposes a method for triggering monitoring data reporting or model management based on the performance of a certain number of consecutive samples when monitoring AI model performance, thereby reducing the latency of model performance monitoring.

Claims

1. A model monitoring method, characterized in that: The method comprises: Sending first information, wherein the first information includes information for artificial intelligence (AI) model performance monitoring.

2. The method according to claim 1, characterized in that The AI ​​model is deployed on a network device; the first information includes at least one of the following: second information corresponding to the beams and / or beam pairs in the first set, the second information being used to represent first parameters of the beams and / or beam pairs in the first set, the beams and / or beam pairs in the first set corresponding to inputs of the AI ​​model, the first parameters comprising measured identifiers and / or measured values; The third information corresponding to the beams and / or beam pairs in the second set, the third information is used to represent the first parameters of the beams and / or beam pairs in the second set, the second set includes the first set, or the first set and the second set are different, the output of the AI ​​model is used to obtain the second parameters of the beams and / or beam pairs in the second set, the second parameters include the predicted identifier and / or predicted value.

3. The method according to claim 1, characterized in that: The AI ​​model is deployed on the terminal; the first information includes at least one of the following: fourth information, where the fourth information is used to instruct to perform an AI operation on the AI ​​model; third information and fifth information corresponding to the beams and / or beam pairs in the second set, wherein the third information is used to represent first parameters of the beams and / or beam pairs in the second set, the fifth information is used to represent second parameters of the beams and / or beam pairs in the second set, the fifth information is determined based on the output of the AI ​​model, the first parameter includes a measured identifier and / or a measured value, and the second parameter includes a predicted identifier and / or a predicted value; A first indicator is used to indicate a prediction accuracy of a second number of beams and / or beam pairs.

4. The method according to any one of claims 1 to 3, characterized in that: The sending of the first information includes at least one of the following: The second parameter obtained by the AI ​​model output N times satisfies the first condition, and the first information is sent, wherein the first condition is used to indicate that the AI ​​model prediction is accurate; The second parameter obtained by the M-times AI model output satisfies the second condition, and the first information is sent, wherein the second condition is used to indicate that the AI ​​model prediction is inaccurate.

5. The method according to claim 4, characterized in that The first condition includes at least one of the following: The K target beams and / or beam pairs obtained by the AI ​​model output include L target beams and / or beam pairs measured in the second set, and the L target beams and / or beam pairs are the first L beams and / or beam pairs in the second set sorted from high to low according to the measured values; One target beam and / or beam pair obtained by the AI ​​model output is among the L target beams and / or beam pairs measured in the second set; A first difference between a measurement value corresponding to a prediction identifier of one target beam and / or a beam pair obtained by the AI ​​model output and a measurement value of one target beam and / or a beam pair measured in the second set is less than or equal to a first threshold; A second difference between a predicted value corresponding to a predicted identifier of a target beam and / or beam pair obtained by the AI ​​model output and a measured value corresponding to the predicted identifier is less than or equal to a second threshold; The throughput difference is greater than or equal to a third threshold, wherein the throughput difference represents a difference between a throughput corresponding to one beam and / or beam pair among the K target beams and / or beam pairs obtained based on the AI ​​model output and a throughput corresponding to one beam and / or beam pair among the L target beams and / or beam pairs in the second set; The reference signal resource overhead sent by the network device is less than or equal to a fourth threshold; The overhead of the second information is less than or equal to a fifth threshold.

6. The method according to claim 4, characterized in that The second condition includes at least one of the following: The K target beams and / or beam pairs obtained by the AI ​​model output do not include the L target beams and / or beam pairs measured in the second set, and the L target beams and / or beam pairs are the first L beams and / or beam pairs in the second set sorted from high to low according to the measured values; The one target beam and / or beam pair obtained by the AI ​​model output is not among the L target beams and / or beam pairs measured in the second set; A first difference between a measurement value corresponding to a prediction identifier of one target beam and / or a beam pair obtained by the AI ​​model output and a measurement value of one target beam and / or a beam pair measured in the second set is greater than a first threshold; A second difference between a predicted value corresponding to a predicted identifier of a target beam and / or beam pair obtained by the AI ​​model output and a measured value corresponding to the predicted identifier is greater than a second threshold; The throughput difference is less than a third threshold, wherein the throughput difference represents a difference between a throughput corresponding to one beam and / or beam pair among the K target beams and / or beam pairs obtained based on the AI ​​model output and a throughput corresponding to one beam and / or beam pair among the L target beams and / or beam pairs in the second set; The reference signal resource overhead sent by the network device is greater than a fourth threshold; The overhead of the second information is greater than a fifth threshold.

7. The method according to any one of claims 1 to 6, characterized in that: The first information is carried by at least one of radio resource control RRC signaling, uplink media access control element UL MAC CE and uplink control information UCI.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: The sixth information is sent, where the sixth information is used to request monitoring of the AI ​​model.

9. The method according to claim 8, characterized in that The method further comprises at least one of the following: Determining the first information at a second time after sending the sixth information; determining the first information at a third time after receiving the seventh information, wherein the seventh information is feedback from the network device based on the sixth information; After the network device sends a reference signal for AI model monitoring, the first information is determined.

10. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Receive sixth information, where the sixth information is used to request monitoring of the AI ​​model.

11. The method according to claim 10, characterized in that The method further comprises: A fourth time after receiving the sixth information, the first information is determined.

12. The method according to any one of claims 1 to 11, characterized in that: The AI ​​model is deployed on a terminal; the method further includes: Receive eighth information, where the eighth information is used to instruct the terminal to perform an AI operation on the AI ​​model.

13. The method according to claim 3 or 12, characterized in that: The AI ​​operation includes at least one of the following: Activate AI models and / or AI functions; Deactivate AI models and / or AI functions; Switching AI models and / or AI functions; Update AI models and / or AI capabilities; Switch to non-AI mode and / or non-AI functionality.

14. The method according to claim 2 or 3, characterized in that: The measured value and / or the predicted value comprises at least one of the following: Layer 1 reference signal received power L1-RSRP; Layer 1 signal to interference and noise ratio L1-SINR.

15. A model monitoring method, characterized in that: The method comprises: The first information sent by the receiving terminal includes information for artificial intelligence (AI) model performance monitoring.

16. The method according to claim 15, characterized in that The AI ​​model is deployed on a network device; the first information includes at least one of the following: second information corresponding to the beams and / or beam pairs in the first set, the second information being used to represent first parameters of the beams and / or beam pairs in the first set, the beams and / or beam pairs in the first set corresponding to inputs of the AI ​​model, the first parameters comprising measured identifiers and / or measured values; The third information corresponding to the beams and / or beam pairs in the second set, the third information is used to represent the first parameters of the beams and / or beam pairs in the second set, the second set includes the first set, or the first set and the second set are different, the output of the AI ​​model is used to obtain the second parameters of the beams and / or beam pairs in the second set, the second parameters include the predicted identifier and / or predicted value.

17. The method according to claim 15, characterized in that The AI ​​model is deployed on the terminal; the first information includes at least one of the following: fourth information, the fourth information being used to instruct to perform an AI operation on the AI ​​model; third information and fifth information corresponding to the beams and / or beam pairs in the second set, wherein the third information is used to represent first parameters of the beams and / or beam pairs in the second set, the fifth information is used to represent second parameters of the beams and / or beam pairs in the second set, the fifth information is determined based on the output of the AI ​​model, the first parameter includes a measured identifier and / or a measured value, and the second parameter includes a predicted identifier and / or a predicted value; A first indicator is used to indicate a prediction accuracy of a second number of beams and / or beam pairs.

18. The method according to any one of claims 15 to 17, characterized in that: The first information sent by the receiving terminal includes at least one of the following: The second parameter obtained by the AI ​​model output N times satisfies the first condition, and the first information sent by the receiving terminal is received, wherein the first condition is used to indicate that the AI ​​model prediction is accurate; The second parameter obtained by the M-times AI model output satisfies the second condition, and the first information sent by the receiving terminal is received, wherein the second condition is used to indicate that the AI ​​model prediction is inaccurate.

19. The method according to claim 18, characterized in that The first condition includes at least one of the following: The K target beams and / or beam pairs obtained by the AI ​​model output include L target beams and / or beam pairs measured in the second set, and the L target beams and / or beam pairs are the first L beams and / or beam pairs in the second set sorted from high to low according to the measured values; One target beam and / or beam pair obtained by the AI ​​model output is among the L target beams and / or beam pairs measured in the second set; A first difference between a measurement value corresponding to a prediction identifier of one target beam and / or a beam pair obtained by the AI ​​model output and a measurement value of one target beam and / or a beam pair measured in the second set is less than or equal to a first threshold; A second difference between a predicted value corresponding to a predicted identifier of a target beam and / or beam pair obtained by the AI ​​model output and a measured value corresponding to the predicted identifier is less than or equal to a second threshold; The throughput difference is greater than or equal to a third threshold, wherein the throughput difference represents a difference between a throughput corresponding to one beam and / or beam pair among the K target beams and / or beam pairs obtained based on the AI ​​model output and a throughput corresponding to one beam and / or beam pair among the L target beams and / or beam pairs in the second set; The reference signal resource overhead sent by the network device is less than or equal to a fourth threshold; The overhead of the second information is less than or equal to a fifth threshold.

20. The method according to claim 18, characterized in that The second condition includes at least one of the following: The K target beams and / or beam pairs obtained by the AI ​​model output do not include the L target beams and / or beam pairs measured in the second set, and the L target beams and / or beam pairs are the first L beams and / or beam pairs in the second set sorted from high to low according to the measured values; The one target beam and / or beam pair obtained by the AI ​​model output is not among the L target beams and / or beam pairs measured in the second set; A first difference between a measurement value corresponding to a prediction identifier of one target beam and / or a beam pair obtained by the AI ​​model output and a measurement value of one target beam and / or a beam pair measured in the second set is greater than a first threshold; A second difference between a predicted value corresponding to a predicted identifier of a target beam and / or beam pair obtained by the AI ​​model output and a measured value corresponding to the predicted identifier is greater than a second threshold; The throughput difference is less than a third threshold, wherein the throughput difference represents a difference between a throughput corresponding to one beam and / or beam pair among the K target beams and / or beam pairs obtained based on the AI ​​model output and a throughput corresponding to one beam and / or beam pair among the L target beams and / or beam pairs in the second set; The reference signal resource overhead sent by the network device is greater than a fourth threshold; The overhead of the second information is greater than a fifth threshold.

21. The method according to any one of claims 15 to 20, characterized in that: The first information is carried by at least one of radio resource control RRC signaling, uplink media access control element UL MAC CE and uplink control information UCI.

22. The method according to any one of claims 15 to 21, characterized in that: The method further comprises: Receive sixth information, where the sixth information is used to request monitoring of the AI ​​model.

23. The method according to claim 22, characterized in that The first information sent by the receiving terminal when the first condition is met includes at least one of the following: receiving the first information after a second time after receiving the sixth information; receiving the first information after a third time after sending the seventh information, wherein the seventh information is feedback from the network device based on the sixth information; After sending a reference signal for AI model monitoring, the first information is received.

24. The method according to any one of claims 15 to 21, characterized in that: The method further comprises: The sixth information is sent, where the sixth information is used to request monitoring of the AI ​​model.

25. The method according to claim 24, characterized in that The first information sent by the receiving terminal when the first condition is met includes: After a fourth time after sending the sixth information, the first information is received.

26. The method according to claim 15, characterized in that The method further comprises: Obtaining a second indicator based on the first information, where the second indicator is used to describe the performance of the AI ​​model and / or the AI ​​function; The second indicator includes at least one of the following: the predictive accuracy of AI models and / or AI functions; Layer 1 reference signal received power L1-RSRP average difference; L1-RSRP difference; Throughput difference; The reference signal overhead sent; Received UCI overhead.

27. The method according to any one of claims 15 to 26, characterized in that: An AI model corresponds to one or more AI functions, and / or an AI function corresponds to one or more AI models.

28. The method according to any one of claims 15 to 27, characterized in that: The AI ​​model is deployed on a terminal; the method further includes: Send eighth information, where the eighth information is used to instruct the terminal to perform an AI operation on the AI ​​model.

29. The method according to claim 17 or 28, characterized in that The AI ​​operation includes at least one of the following: Activate AI models and / or AI functions; Deactivate AI models and / or AI functions; Switching AI models and / or AI functions; Update AI models and / or AI capabilities; Switch to non-AI mode and / or non-AI functionality.

30. The method according to claim 16 or 17, characterized in that The measured value and / or the predicted value includes at least one of the following: L1-RSRP; L1-SINR.

31. A model monitoring method, characterized in that: The method comprises: The terminal sends first information to the network device, where the first information includes information for artificial intelligence AI model performance monitoring; The network device receives first information sent by the terminal.

32. A terminal, characterized in that: include: Transceiver module; The transceiver module is used to send first information, and the first information includes information used for artificial intelligence AI model performance monitoring.

33. A network device, characterized in that: include: Transceiver module; The transceiver module is used to receive first information sent by a terminal, where the first information includes information used for artificial intelligence AI model performance monitoring.

34. A terminal, characterized in that: include: one or more processors; Wherein, the terminal is used to execute the model monitoring method described in any one of claims 1-14.

35. A network device, characterized in that: include: one or more processors; Wherein, the network device is used to execute the model monitoring method described in any one of claims 15-30.

36. A communication system, characterized in that: It comprises a terminal and a network device, wherein the terminal is configured to implement the model monitoring method according to any one of claims 1 to 14, and the network device is configured to implement the model monitoring method according to any one of claims 15 to 30.

37. A storage medium storing instructions, characterized in that: When the instructions are executed on a communication device, the communication device is caused to execute the model monitoring method according to any one of claims 1-14 or 15-30.

Images