Model performance monitoring method, device, and storage medium

Abstract

The present invention relates to a model performance monitoring method, a device, and a storage medium. The method comprises: receiving first information sent by a network device, the first information being used for determining to perform performance monitoring on a first model. In this way, the performance of the first model can be monitored before the first model is activated, so that a more accurate prediction result can be achieved in the use process of the first model, thereby improving the performance of a network.

Description

Model performance monitoring method, device and storage medium Technical Field

[0001] The present disclosure relates to the field of communication technology, and in particular to a model performance monitoring method, device, and storage medium. Background Art

[0002] With advances in communications technology, predictive models based on artificial intelligence (AI) have been introduced into communication systems. These predictive models can generate forecast data for certain scenarios, thereby improving network performance. However, different predictive models may be applicable to different scenarios. For example, some models are suitable for suburban areas, others for urban areas, and some for peak hours, while others are suitable for off-peak hours. Therefore, how to use predictive models has become a pressing issue.

[0003] Summary of the Invention

[0004] The embodiments of the present disclosure provide a model performance monitoring method, device, and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, a model performance monitoring method is proposed, the method comprising:

[0006] First information sent by a network device is received, where the first information is used to determine performance monitoring of a first model.

[0007] According to a second aspect of an embodiment of the present disclosure, a model performance monitoring method is proposed, the method comprising:

[0008] First information is sent to a terminal device, where the first information is used to determine performance monitoring of a first model.

[0009] According to a third aspect of an embodiment of the present disclosure, a model performance monitoring method is proposed, the method comprising:

[0010] The terminal device sends second information to the network device, where the second information is used to request performance monitoring of the first model;

[0011] The network device sends first information to the terminal device, where the first information is used to determine performance monitoring of the first model.

[0012] According to a fourth aspect of an embodiment of the present disclosure, a terminal device is provided, including:

[0013] The transceiver module is configured to receive first information sent by the network device, where the first information is used to determine performance monitoring of the first model.

[0014] According to a fifth aspect of an embodiment of the present disclosure, a network device is provided, including:

[0015] The transceiver module is configured to send first information to the terminal device, where the first information is used to determine performance monitoring of the first model.

[0016] According to a sixth aspect of an embodiment of the present disclosure, a terminal device is proposed, comprising: one or more processors; wherein the terminal device can be used to execute an optional implementation of the first aspect.

[0017] According to a seventh aspect of an embodiment of the present disclosure, a network device is proposed, comprising: one or more processors; wherein the network device can be used to execute an optional implementation of the second aspect.

[0018] According to the eighth aspect of an embodiment of the present disclosure, a communication system is proposed, which may include: a terminal device and a network device; wherein the terminal device is configured to execute the method described in the optional implementation manner of the first aspect, and the network device is configured to execute the method described in the optional implementation manner of the second aspect.

[0019] According to a ninth aspect of an embodiment of the present disclosure, a storage medium is proposed, which stores instructions. When the instructions are executed on a communication device, the communication device executes the method described in the optional implementation of the first aspect or the second aspect.

[0020] The technical solution provided by the embodiment of the present disclosure may include the following beneficial effects: the terminal device receives the first information sent by the network device, and determines to monitor the performance of the first model. In this way, the performance of the first model can be monitored before activating the first model, and the first model can be activated when it is determined that the performance of the first model meets the activation conditions, so that more accurate prediction results can be achieved during the use of the first model, thereby improving the performance of the network.

[0021] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following drawings required for describing the embodiments are introduced. The following drawings are merely some embodiments of the present disclosure and do not impose specific limitations on the protection scope of the present disclosure.

[0023] FIG1 is a schematic diagram showing the architecture of a communication system according to an embodiment of the present disclosure.

[0024] FIG2A is a flow chart illustrating a method for monitoring model performance according to an embodiment of the present disclosure.

[0025] FIG2B is a schematic diagram showing a time relationship according to an embodiment of the present disclosure.

[0026] FIG2C is a schematic diagram showing a time relationship according to an embodiment of the present disclosure.

[0027] FIG2D is a flow chart illustrating a method for monitoring model performance according to an embodiment of the present disclosure.

[0028] FIG2E is a flow chart of a model performance monitoring method according to an embodiment of the present disclosure.

[0029] FIG2F is a flow chart of a model performance monitoring method according to an embodiment of the present disclosure.

[0030] FIG2G is a flow chart illustrating a method for monitoring model performance according to an embodiment of the present disclosure.

[0031] FIG3A is a flow chart illustrating a method for monitoring model performance according to an embodiment of the present disclosure.

[0032] FIG3B is a flow chart illustrating a method for monitoring model performance according to an embodiment of the present disclosure.

[0033] FIG3C is a flow chart illustrating a method for monitoring model performance according to an embodiment of the present disclosure.

[0034] FIG3D is a flow chart illustrating a method for monitoring model performance according to an embodiment of the present disclosure.

[0035] FIG4A is a flow chart illustrating a method for monitoring model performance according to an embodiment of the present disclosure.

[0036] FIG4B is a flow chart illustrating a method for monitoring model performance according to an embodiment of the present disclosure.

[0037] FIG4C is a flow chart illustrating a method for monitoring model performance according to an embodiment of the present disclosure.

[0038] FIG5 is an interactive schematic diagram illustrating a model performance monitoring method according to an embodiment of the present disclosure.

[0039] FIG6A is a schematic structural diagram of a terminal device proposed in an embodiment of the present disclosure.

[0040] FIG6B is a schematic structural diagram of a network device proposed in an embodiment of the present disclosure.

[0041] FIG7A is a schematic structural diagram of a communication device proposed in an embodiment of the present disclosure.

[0042] FIG7B is a schematic diagram of the structure of the chip proposed in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0043] The embodiments of the present disclosure provide a model performance monitoring method, device, and storage medium.

[0044] In a first aspect, an embodiment of the present disclosure provides a model performance monitoring method, the method comprising:

[0045] First information sent by a network device is received, where the first information is used to determine performance monitoring of a first model.

[0046] In the above embodiment, a request may be made to monitor the performance of the first model before activating the first model, thereby achieving more accurate prediction results and improving network performance during use of the first model.

[0047] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes:

[0048] Second information is sent to the network device, where the second information is used to request performance monitoring of the first model.

[0049] In the above embodiment, the terminal device can send the second information to the network device to request performance monitoring of the first model, thereby improving the flexibility of model performance monitoring.

[0050] In conjunction with some embodiments of the first aspect, in some embodiments, the second information includes at least one of the following:

[0051] Scheduling request SR for requesting model performance monitoring;

[0052] The first message Msg 1 or message A Msg A of random access RA for requesting model performance monitoring;

[0053] a model identifier of the first model;

[0054] a function identifier corresponding to the first model;

[0055] First configuration information, wherein the first configuration information includes resource configuration for performance monitoring of the first model.

[0056] In the above embodiment, the terminal device may request the network device for resource configuration for performance monitoring of the first model, and the network device may perform resource configuration based on the request of the terminal device, thereby improving the flexibility of resource configuration.

[0057] In conjunction with some embodiments of the first aspect, in some embodiments, the first model is a model for performing beam prediction, and the first configuration information includes at least one of the following:

[0058] a first reference signal resource set including a beam to be measured;

[0059] a second reference signal resource set, wherein the second reference signal resource set includes a beam to be predicted;

[0060] a relationship between the first reference signal resource set and the second reference signal resource set;

[0061] a first performance monitoring duration, where the first performance monitoring duration is the duration during which the terminal device requests performance monitoring of the first model;

[0062] The first sample number is the number of samples requested by the terminal device for performance monitoring of the first model, and the sample number is the number of times the network device sends reference signal resources.

[0063] In the above embodiment, resource configuration information of the beam prediction model is provided so that the network device can accurately configure resources, thereby improving the accuracy of performance monitoring.

[0064] In conjunction with some embodiments of the first aspect, in some embodiments, the first model is a model for performing spatial beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following:

[0065] The first reference signal resource set is a subset of the second reference signal resource set;

[0066] The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same.

[0067] In the above embodiment, a relationship between the measurement beam and the prediction beam corresponding to the spatial beam prediction model is provided so that the network device can accurately configure resources, thereby improving the accuracy of performance monitoring.

[0068] In conjunction with some embodiments of the first aspect, in some embodiments, the first model is a model for performing time-domain beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following:

[0069] The first reference signal resource set is a subset of the second reference signal resource set;

[0070] The first reference signal resource set is the same as the second reference signal resource set;

[0071] The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same;

[0072] The relationship between a first time and a second time, wherein the first time is a time for performing measurement on the first reference signal resource set, and the second time is a time for performing prediction or measurement on the second reference signal resource set.

[0073] In the above embodiment, a relationship between the measurement beam and the prediction beam corresponding to the time domain beam prediction model is provided so that the network device can accurately configure resources, thereby improving the accuracy of performance monitoring.

[0074] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes:

[0075] receiving third information sent by the network device, where the third information is used to request performance monitoring of the second model;

[0076] Fourth information is sent to the network device according to the third information, where the fourth information is used to indicate that the request for the third information is approved.

[0077] In the above embodiment, the network device may send the third information to the terminal device to request performance monitoring of the first model, thereby improving the flexibility of model performance monitoring.

[0078] In conjunction with some embodiments of the first aspect, in some embodiments, the first information includes at least one of the following:

[0079] a model identifier of the second model;

[0080] a function identifier corresponding to the second model;

[0081] a third reference signal resource set, wherein the third reference signal resource set includes a beam to be measured;

[0082] a fourth reference signal resource set, wherein the fourth reference signal resource set includes a beam to be predicted;

[0083] a relationship between the third reference signal resource set and the fourth reference signal resource set;

[0084] a relationship between a third time and a fourth time, the third time being a time for performing measurement on the third reference signal resource set, and the fourth time being a time for performing prediction or measurement on the fourth reference signal resource set;

[0085] a second performance monitoring duration, where the second performance monitoring duration is the duration during which the network device requests performance monitoring of the second model;

[0086] a second sample number, where the second sample number is the sample number requested by the network device for performance monitoring of the second model, and the sample number is the number of times the network device sends a reference signal resource;

[0087] Performance indicators corresponding to performance monitoring;

[0088] Reporting type of performance monitoring report;

[0089] The number of samples included in a performance monitoring report;

[0090] a model management threshold value, wherein the model management threshold value is used to determine whether to activate the second model;

[0091] Physical uplink control channel PUCCH / downlink physical shared channel PUSCH resources corresponding to the performance monitoring report;

[0092] a first event, the first event including a performance indicator for determining that a performance value of the second model satisfies a preset performance condition;

[0093] a first threshold value corresponding to the first event, the first threshold value being used to determine whether the first event is triggered;

[0094] A first offset value corresponding to the first event, where the first offset value is used to determine whether to trigger the first event.

[0095] In the above embodiment, a reference signal resource is provided for the network device to configure the terminal device. The terminal device can perform measurement, reporting and other operations according to the reference signal resource to cooperate with the network device to complete the performance monitoring of the model, thereby improving the flexibility of the model performance monitoring.

[0096] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes:

[0097] Sending a performance monitoring report to the network device; the performance monitoring report includes at least one of the following:

[0098] Measurement data;

[0099] Forecast data;

[0100] a performance value of the second model, the performance value determined based on the measured data and the predicted data;

[0101] The first event is triggered based on a comparison result between the performance value of the second model and the first threshold value or the first offset value;

[0102] First operation information, where the first operation information is used to instruct a management operation to be performed on the second model, where the management operation includes whether to activate the second model.

[0103] In the above embodiment, the terminal device can report the model performance monitoring report to the network device so that the network device can manage the model according to the performance monitoring report and activate the model in time when the model can meet the performance requirements to improve network performance.

[0104] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes:

[0105] The beam in the fourth reference signal resource set is measured according to the first information to obtain the measurement data.

[0106] In the above embodiment, the terminal device measures the beam to be measured according to the first information sent by the network device to obtain input data for performance monitoring of the model.

[0107] In conjunction with some embodiments of the first aspect, in some embodiments, the measurement data includes at least one of the following:

[0108] The measured layer 1 reference signal received power L1-RSRP of the best K beams in the fourth reference signal resource;

[0109] The measured L1-RSRP of each beam in the fourth reference signal resource set;

[0110] the radio link quality of each of the best K beams measured within a fifth time period;

[0111] The lowest value of the radio link quality of each beam in the best K beams measured within the fifth time period;

[0112] a block error rate (BLER) of each of the best K beams measured within the fifth time period;

[0113] a highest value of the BLER of each beam in the best K beams measured within the fifth time period;

[0114] A probability that the radio link quality of each of the best K beams measured within the fifth time is less than or equal to a preset link quality threshold.

[0115] In the above embodiment, the specific content of the measurement data obtained after the terminal device performs measurement on the beam to be measured is provided, so that the first model can accurately make predictions based on the measurement data, thereby improving the accuracy of performance monitoring.

[0116] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes:

[0117] The beam in the third reference signal resource set is measured according to the first information to obtain the measurement data.

[0118] In the above embodiment, the terminal device measures the beam to be predicted according to the first information sent by the network device to obtain comparative data for performance monitoring of the model.

[0119] In conjunction with some embodiments of the first aspect, in some embodiments, the measurement data includes at least one of the following:

[0120] The measured L1-RSRPs of the best K beams in the third reference signal resource set;

[0121] The measured L1-RSRP of each beam in the third reference signal resource set.

[0122] In the above embodiment, the specific content of the measurement data obtained after the terminal device performs measurement on the beam to be predicted is provided, so that the first model can accurately determine the performance value of the model based on the measurement data, thereby improving the accuracy of performance monitoring.

[0123] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes:

[0124] The predicted data is obtained according to the measurement data corresponding to the third reference signal resource.

[0125] In the above embodiment, the terminal device may perform prediction using the first model based on the measurement result of the beam to be predicted to obtain prediction data.

[0126] In conjunction with some embodiments of the first aspect, in some embodiments, the prediction data includes at least one of the following:

[0127] The predicted L1-RSRPs of the best K beams in the fourth reference signal resource set;

[0128] The predicted L1-RSRP of each beam in the fourth reference signal resource set;

[0129] predicted wireless link qualities of the best K beams within a fifth time period;

[0130] The predicted lowest value of the radio link qualities of the best K beams within the fifth time;

[0131] The predicted BLERs of the best K beams within the fifth time period;

[0132] The predicted highest BLER value of the best K beams within the fifth time period;

[0133] The predicted probability that the wireless link qualities of the best K beams within the fifth time are less than or equal to a preset link quality threshold.

[0134] In the above embodiment, the specific content of the prediction data obtained after the model performs the prediction is provided, so that a more accurate performance value can be determined based on the comparison result of the prediction data and the measurement data, thereby improving the accuracy of performance monitoring.

[0135] In conjunction with some embodiments of the first aspect, in some embodiments, the performance indicator includes at least one of the following:

[0136] Prediction accuracy of beams or beam pairs;

[0137] prediction accuracy of the beam or beam pair for which an L1-RSRP difference is within a first threshold, wherein the L1-RSRP difference is the difference between the measured L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the best measured beam or beam pair;

[0138] L1-RSRP difference, where the L1-RSRP difference is the difference between the measured L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the measured best beam or beam pair, wherein the L1-RSRP difference includes at least one of the following: an average value of the L1-RSRP difference, a cumulative distribution function of the L1-RSRP difference, a proportion of the L1-RSRP difference less than or equal to a second threshold, and a proportion of the L1-RSRP difference greater than a third threshold;

[0139] a predicted L1-RSRP difference, where the predicted L1-RSRP difference is the difference between the predicted L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the best predicted beam or beam pair, wherein the predicted L1-RSRP difference includes at least one of the following: an average value of the predicted L1-RSRP difference, a cumulative distribution function of the predicted L1-RSRP difference, a proportion of the predicted L1-RSRP difference less than or equal to the second threshold, and a proportion of the predicted L1-RSRP difference greater than the third threshold, wherein the predicted L1-RSRP difference is the difference between the predicted L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the best predicted beam or beam pair;

[0140] Terminal device throughput;

[0141] Reference signal overhead;

[0142] Uplink control information overhead;

[0143] Predicted latency;

[0144] Beam failure probability prediction accuracy;

[0145] Wireless link quality prediction variability;

[0146] Block error rate prediction difference;

[0147] Beam failure time prediction accuracy;

[0148] New beam prediction accuracy.

[0149] In the above embodiment, performance indicators for determining model performance values ​​are provided to enable a more comprehensive and accurate performance evaluation of the model, thereby further improving the accuracy of model performance monitoring.

[0150] In conjunction with some embodiments of the first aspect, in some embodiments, the performance value includes at least one of the following:

[0151] the accuracy of the predicted beam or beam pair;

[0152] prediction accuracy of the beam or beam pair for which the L1-RSRP difference is within the first threshold;

[0153] a first L1-RSRP difference, where the first L1-RSRP difference is a difference between a measured L1-RSRP of the best predicted beam or beam pair and a measured L1-RSRP of the measured best beam or beam pair;

[0154] a second L1-RSRP difference, where the second L1-RSRP difference is a difference between a predicted L1-RSRP of the best predicted beam or beam pair and a measured L1-RSRP of the best predicted beam or beam pair;

[0155] The throughput of the terminal device for determining the performance value;

[0156] a reference signal overhead for determining the performance value;

[0157] Uplink control information overhead for determining the performance value;

[0158] used to obtain the predicted delay of the predicted data;

[0159] Beam failure probability prediction accuracy;

[0160] A radio link quality prediction difference, where the radio link quality prediction difference is a difference between the predicted radio link quality and the measured radio link quality;

[0161] A block error rate prediction difference, where the block error rate prediction difference is the difference between the predicted block error rate and the measured block error rate;

[0162] Beam failure time prediction accuracy;

[0163] New beam prediction accuracy.

[0164] In the above embodiment, multiple performance values ​​of the model are provided to enable a more comprehensive and accurate performance evaluation of the model, thereby further improving the accuracy of model performance monitoring.

[0165] In conjunction with some embodiments of the first aspect, in some embodiments, the first threshold value includes at least one of the following:

[0166] a first accuracy threshold;

[0167] a second accuracy threshold;

[0168] a first difference threshold;

[0169] a second difference threshold;

[0170] a first throughput threshold;

[0171] a first overhead threshold;

[0172] a second spending threshold;

[0173] First delay threshold;

[0174] third accuracy threshold;

[0175] The third difference threshold;

[0176] a fourth difference threshold;

[0177] a fourth accuracy threshold;

[0178] Fifth accuracy threshold.

[0179] In the above embodiment, multiple threshold values ​​are provided for determining the performance value of the model, thereby making the determined performance value more accurate.

[0180] In conjunction with some embodiments of the first aspect, in some embodiments, the first event includes at least one of the following:

[0181] The accuracy of the predicted beam or beam pair is greater than the first accuracy threshold;

[0182] The prediction accuracy of the beam or beam pair for which the L1-RSRP difference is within the first threshold is greater than the second accuracy threshold;

[0183] The first L1-RSRP difference is less than the first difference threshold;

[0184] The second L1-RSRP difference is less than the second difference threshold;

[0185] The throughput is greater than the first throughput threshold;

[0186] The reference signal overhead is less than the first overhead threshold;

[0187] The uplink control information overhead is less than the second overhead threshold;

[0188] The predicted delay is less than the first delay threshold;

[0189] The beam failure probability prediction accuracy is greater than the third accuracy threshold;

[0190] The predicted difference of the radio link quality is less than the third difference threshold;

[0191] The block error rate prediction difference is less than the fourth difference threshold;

[0192] The beam failure time prediction accuracy is greater than the fourth accuracy threshold;

[0193] The new beam prediction accuracy is greater than the fifth accuracy threshold.

[0194] In the above embodiment, the terminal device may trigger an event to the network device when determining that the model performance is good, so that the network device can determine the performance status of the model more accurately and quickly.

[0195] In conjunction with some embodiments of the first aspect, in some embodiments, the second model is an inactive model, and the method further includes:

[0196] receiving an activation instruction sent by the network device;

[0197] activating the second model according to the activation instruction.

[0198] In the above embodiment, when the network device determines that the performance status of the model meets the activation condition, it can instruct the terminal device to activate the model, thereby improving network performance.

[0199] In combination with some embodiments of the first aspect, in some embodiments, the first model and / or the second model are deployed on the terminal device side, or the first model and / or the second model are deployed on the network device side.

[0200] In the above embodiments, different modes of model deployment are provided, thereby improving the flexibility of model deployment.

[0201] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes:

[0202] Send capability indication information to the network device, where the capability indication information is used to indicate at least one of the following:

[0203] whether the terminal device supports the first model and / or the second model;

[0204] Whether the terminal device supports performance monitoring of an inactive model deployed on the terminal device side;

[0205] Whether the terminal device supports performance monitoring of the inactivated model deployed on the network device side.

[0206] In the above embodiment, the terminal device can report capability indication information to the network device so that the network device can configure resources according to the capabilities of the terminal device and avoid sending erroneous information, thereby improving the efficiency of model performance monitoring.

[0207] In a second aspect, an embodiment of the present disclosure provides a model performance monitoring method, the method comprising:

[0208] First information is sent to a terminal device, where the first information is used to determine performance monitoring of a first model.

[0209] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes:

[0210] Second information sent by the terminal device is received, where the second information is used to request performance monitoring of the first model.

[0211] In conjunction with some embodiments of the first aspect, in some embodiments, the second information includes at least one of the following:

[0212] Scheduling request SR for requesting model performance monitoring;

[0213] The first message Msg 1 or message A Msg A of random access RA for requesting model performance monitoring;

[0214] a model identifier of the first model;

[0215] a function identifier corresponding to the first model;

[0216] First configuration information, wherein the first configuration information includes resource configuration for performance monitoring of the first model.

[0217] In conjunction with some embodiments of the second aspect, in some embodiments, the first model is a model for performing beam prediction, and the first configuration information includes at least one of the following:

[0218] a first reference signal resource set including a beam to be measured;

[0219] a second reference signal resource set, wherein the second reference signal resource set includes a beam to be predicted;

[0220] a relationship between the first reference signal resource set and the second reference signal resource set;

[0221] a first performance monitoring duration, where the first performance monitoring duration is the duration during which the terminal device requests performance monitoring of the first model;

[0222] The first sample number is the number of samples requested by the terminal device for performance monitoring of the first model, and the sample number is the number of times the network device sends reference signal resources.

[0223] In conjunction with some embodiments of the second aspect, in some embodiments, the first model is a model for performing spatial beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following:

[0224] The first reference signal resource set is a subset of the second reference signal resource set;

[0225] The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same.

[0226] In conjunction with some embodiments of the second aspect, in some embodiments, the first model is a model for performing time-domain beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following:

[0227] The first reference signal resource set is a subset of the second reference signal resource set;

[0228] The first reference signal resource set is the same as the second reference signal resource set;

[0229] The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same;

[0230] The relationship between a first time and a second time, wherein the first time is a time for performing measurement on the first reference signal resource set, and the second time is a time for performing prediction or measurement on the second reference signal resource set.

[0231] In conjunction with some embodiments of the second aspect, in some embodiments, the method further includes:

[0232] Sending three messages to the terminal device, wherein the third message is used to request performance monitoring of the second model;

[0233] Receive fourth information sent by the terminal device based on the third information, where the fourth information is used to indicate approval of the request for the third information.

[0234] In conjunction with some embodiments of the second aspect, in some embodiments, the first information includes at least one of the following:

[0235] a model identifier of the second model;

[0236] a function identifier corresponding to the second model;

[0237] a third reference signal resource set, wherein the third reference signal resource set includes a beam to be measured;

[0238] a fourth reference signal resource set, wherein the fourth reference signal resource set includes a beam to be predicted;

[0239] a relationship between the third reference signal resource set and the fourth reference signal resource set;

[0240] a relationship between a third time and a fourth time, the third time being a time for performing measurement on the third reference signal resource set, and the fourth time being a time for performing prediction or measurement on the fourth reference signal resource set;

[0241] a second performance monitoring duration, where the second performance monitoring duration is the duration during which the network device requests performance monitoring of the second model;

[0242] a second sample number, where the second sample number is the sample number requested by the network device for performance monitoring of the second model, and the sample number is the number of times the network device sends a reference signal resource;

[0243] Performance indicators corresponding to performance monitoring;

[0244] Reporting type of performance monitoring report;

[0245] The number of samples included in a performance monitoring report;

[0246] a model management threshold value, wherein the model management threshold value is used to determine whether to activate the second model;

[0247] Physical uplink control channel PUCCH / downlink physical shared channel PUSCH resources corresponding to the performance monitoring report;

[0248] a first event, the first event including a performance indicator for determining that a performance value of the second model satisfies a preset performance condition;

[0249] a first threshold value corresponding to the first event, the first threshold value being used to determine whether the first event is triggered;

[0250] A first offset value corresponding to the first event, where the first offset value is used to determine whether to trigger the first event.

[0251] In conjunction with some embodiments of the second aspect, in some embodiments, the method further includes:

[0252] Receive a performance monitoring report sent by the terminal device, wherein the performance monitoring report includes at least one of the following:

[0253] Measurement data;

[0254] Forecast data;

[0255] a performance value of the second model, the performance value determined based on the measured data and the predicted data;

[0256] The first event is triggered based on a comparison result between the performance value of the second model and the first threshold value or the first offset value;

[0257] First operation information, where the first operation information is used to instruct a management operation to be performed on the second model, where the management operation includes whether to activate the second model.

[0258] In conjunction with some embodiments of the second aspect, in some embodiments, the measurement data is obtained by the terminal device measuring the beam in the fourth reference signal resource set according to the first information; and the measurement data includes at least one of the following:

[0259] The measured layer 1 reference signal received power L1-RSRP of the best K beams in the fourth reference signal resource;

[0260] The measured L1-RSRP of each beam in the fourth reference signal resource set;

[0261] the radio link quality of each of the best K beams measured within a fifth time period;

[0262] The lowest value of the radio link quality of each beam in the best K beams measured within the fifth time period;

[0263] a block error rate (BLER) of each of the best K beams measured within the fifth time period;

[0264] a highest value of the BLER of each beam in the best K beams measured within the fifth time period;

[0265] A probability that the radio link quality of each of the best K beams measured within the fifth time is less than or equal to a preset link quality threshold.

[0266] In conjunction with some embodiments of the second aspect, in some embodiments, the measurement data is obtained by the terminal device measuring the beam in the third reference signal resource set according to the first information; and the measurement data includes at least one of the following:

[0267] The measured L1-RSRPs of the best K beams in the third reference signal resource set;

[0268] The measured L1-RSRP of each beam in the third reference signal resource set.

[0269] In conjunction with some embodiments of the second aspect, in some embodiments, the prediction data is obtained by the terminal device based on measurement data corresponding to the third reference signal resource; the prediction data includes at least one of the following:

[0270] The predicted L1-RSRPs of the best K beams in the fourth reference signal resource set;

[0271] The predicted L1-RSRP of each beam in the fourth reference signal resource set;

[0272] predicted wireless link qualities of the best K beams within a fifth time period;

[0273] The predicted lowest value of the radio link qualities of the best K beams within the fifth time;

[0274] The predicted BLERs of the best K beams within the fifth time period;

[0275] The predicted highest BLER value of the best K beams within the fifth time period;

[0276] The predicted probability that the wireless link qualities of the best K beams within the fifth time are less than or equal to a preset link quality threshold.

[0277] In conjunction with some embodiments of the second aspect, in some embodiments, the performance indicator includes at least one of the following:

[0278] Prediction accuracy of beams or beam pairs;

[0279] prediction accuracy of the beam or beam pair for which an L1-RSRP difference is within a first threshold, wherein the L1-RSRP difference is the difference between the measured L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the best measured beam or beam pair;

[0280] L1-RSRP difference, where the L1-RSRP difference is the difference between the measured L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the measured best beam or beam pair, wherein the L1-RSRP difference includes at least one of the following: an average value of the L1-RSRP difference, a cumulative distribution function of the L1-RSRP difference, a proportion of the L1-RSRP difference less than or equal to a second threshold, and a proportion of the L1-RSRP difference greater than a third threshold;

[0281] a predicted L1-RSRP difference, where the predicted L1-RSRP difference is the difference between the predicted L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the best predicted beam or beam pair, wherein the predicted L1-RSRP difference includes at least one of the following: an average value of the predicted L1-RSRP difference, a cumulative distribution function of the predicted L1-RSRP difference, a proportion of the predicted L1-RSRP difference less than or equal to the second threshold, and a proportion of the predicted L1-RSRP difference greater than the third threshold, wherein the predicted L1-RSRP difference is the difference between the predicted L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the best predicted beam or beam pair;

[0282] Terminal device throughput;

[0283] Reference signal overhead;

[0284] Uplink control information overhead;

[0285] Predicted latency;

[0286] Beam failure probability prediction accuracy;

[0287] Wireless link quality prediction variability;

[0288] Block error rate predicts difference rate;

[0289] Beam failure time prediction accuracy;

[0290] New beam prediction accuracy.

[0291] In conjunction with some embodiments of the second aspect, in some embodiments, the performance value includes at least one of the following:

[0292] the accuracy of the predicted beam or beam pair;

[0293] prediction accuracy of the beam or beam pair for which the L1-RSRP difference is within the first threshold;

[0294] a first L1-RSRP difference, where the first L1-RSRP difference is a difference between a measured L1-RSRP of the best predicted beam or beam pair and a measured L1-RSRP of the measured best beam or beam pair;

[0295] a second L1-RSRP difference, where the second L1-RSRP difference is a difference between a predicted L1-RSRP of the best predicted beam or beam pair and a measured L1-RSRP of the best predicted beam or beam pair;

[0296] The throughput of the terminal device for determining the performance value;

[0297] a reference signal overhead for determining the performance value;

[0298] Uplink control information overhead for determining the performance value;

[0299] used to obtain the predicted delay of the predicted data;

[0300] Beam failure probability prediction accuracy;

[0301] A radio link quality prediction difference, where the radio link quality prediction difference is a difference between the predicted radio link quality and the measured radio link quality;

[0302] A block error rate prediction difference, where the block error rate prediction difference is the difference between the predicted block error rate and the measured block error rate;

[0303] Beam failure time prediction accuracy;

[0304] New beam prediction accuracy.

[0305] In conjunction with some embodiments of the second aspect, in some embodiments, the first threshold value includes at least one of the following:

[0306] a first accuracy threshold;

[0307] a second accuracy threshold;

[0308] a first difference threshold;

[0309] a second difference threshold;

[0310] a first throughput threshold;

[0311] a first overhead threshold;

[0312] a second spending threshold;

[0313] First delay threshold;

[0314] third accuracy threshold;

[0315] The third difference threshold;

[0316] a fourth difference threshold;

[0317] a fourth accuracy threshold;

[0318] Fifth accuracy threshold.

[0319] In conjunction with some embodiments of the second aspect, in some embodiments, the first event includes at least one of the following:

[0320] The accuracy of the predicted beam or beam pair is greater than the first accuracy threshold;

[0321] The prediction accuracy of the beam or beam pair for which the L1-RSRP difference is within the first threshold is greater than the second accuracy threshold;

[0322] The first L1-RSRP difference is less than the first difference threshold;

[0323] The second L1-RSRP difference is less than the second difference threshold;

[0324] The throughput is greater than the first throughput threshold;

[0325] The reference signal overhead is less than the first overhead threshold;

[0326] The uplink control information overhead is less than the second overhead threshold;

[0327] The predicted delay is less than the first delay threshold;

[0328] The beam failure probability prediction accuracy is greater than the third accuracy threshold;

[0329] The predicted difference of the radio link quality is less than the third difference threshold;

[0330] The block error rate prediction difference is less than the fourth difference threshold;

[0331] The beam failure time prediction accuracy is greater than the fourth accuracy threshold;

[0332] The new beam prediction accuracy is greater than the fifth accuracy threshold.

[0333] In conjunction with some embodiments of the second aspect, in some embodiments, the method further includes:

[0334] determining to activate the second model according to the performance monitoring report;

[0335] Activate the second model, or send an activation instruction to the terminal device, where the activation instruction is used by the terminal device to activate the second model.

[0336] In combination with some embodiments of the second aspect, in some embodiments, the first model and / or the second model are deployed on the terminal device side, or the first model and / or the second model are deployed on the network device side.

[0337] In conjunction with some embodiments of the second aspect, in some embodiments, the method further includes:

[0338] Receive capability indication information sent by the terminal device, where the capability indication information is used to indicate at least one of the following:

[0339] whether the terminal device supports the first model and / or the second model;

[0340] Whether the terminal device supports performance monitoring of an inactive model deployed on the terminal device side;

[0341] Whether the terminal device supports performance monitoring of the inactivated model deployed on the network device side.

[0342] In a third aspect, an embodiment of the present disclosure provides a model performance monitoring method, the method comprising:

[0343] The terminal device sends second information to the network device, where the second information is used to request performance monitoring of the first model;

[0344] The network device sends first information to the terminal device, where the first information is used to determine performance monitoring of the first model.

[0345] In a fourth aspect, an embodiment of the present disclosure proposes a terminal device, which may include at least one of a transceiver module and a processing module; wherein the terminal device can be used to execute the optional implementation method of the first aspect.

[0346] In a fifth aspect, an embodiment of the present disclosure proposes a network device, which may include at least one of a transceiver module and a processing module; wherein the network device can be used to execute the optional implementation method of the second aspect.

[0347] In a sixth aspect, an embodiment of the present disclosure proposes a terminal device, which may include: one or more processors; wherein the terminal device can be used to execute the optional implementation method of the first aspect.

[0348] In a seventh aspect, an embodiment of the present disclosure proposes a network device, which terminal device may include: one or more processors; wherein, the network device can be used to execute the optional implementation method of the second aspect.

[0349] In an eighth aspect, an embodiment of the present disclosure proposes a communication system, which may include: a terminal device and a network device; wherein, the terminal device is configured to execute the method described in the optional implementation manner of the first aspect, and the network device is configured to execute the method described in the optional implementation manner of the second aspect.

[0350] In a ninth aspect, an embodiment of the present disclosure proposes a storage medium storing instructions, which, when executed on a communication device, enables the communication device to execute the method described in the optional implementation of the first or second aspect.

[0351] In a tenth aspect, an embodiment of the present disclosure proposes a program product, which, when executed by a communication device, enables the communication device to execute the method described in the optional implementation manner of the first aspect or the second aspect.

[0352] In an eleventh aspect, an embodiment of the present disclosure proposes a computer program, which, when executed on a computer, enables the computer to execute the method described in the optional implementation of the first or second aspect.

[0353] In a twelfth aspect, an embodiment of the present disclosure provides a chip or a chip system, wherein the chip or chip system includes a processing circuit configured to execute the method described in the optional implementation of the first aspect or the second aspect.

[0354] It is understandable that the aforementioned network devices, terminal devices, communication devices, communication systems, storage media, program products, computer programs, chips, or chip systems can all be used to execute the methods proposed in the embodiments of the present disclosure. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding methods and will not be repeated here.

[0355] The present disclosure provides a model performance monitoring method, device, and storage medium. In some embodiments, the terms "model performance monitoring method" and "monitoring method," "information processing method," and "communication method" are interchangeable; the terms "model performance monitoring device" and "monitoring device," "information processing device," "communication device," and "communication equipment" are interchangeable; and the terms "model performance monitoring system" and "communication system" are interchangeable.

[0356] The embodiments of the present disclosure are not exhaustive and are merely illustrative of some embodiments, and are not intended to be a specific limitation on the scope of protection of the present disclosure. In the absence of contradiction, each step in a certain embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a certain embodiment can also be implemented as an independent embodiment, and the order of the steps in a certain embodiment can be arbitrarily exchanged. In addition, the optional implementation methods in a certain embodiment can be arbitrarily combined; in addition, the embodiments can be arbitrarily combined. For example, some or all steps of different embodiments can be arbitrarily combined, and a certain embodiment can be arbitrarily combined with the optional implementation methods of other embodiments.

[0357] In each embodiment of the present disclosure, unless otherwise specified or provided for by logic, the terms and / or descriptions between the embodiments are consistent and can be referenced by each other. The technical features in different embodiments can be combined to form a new embodiment based on their inherent logical relationships.

[0358] The terms used in the embodiments of the present disclosure are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure.

[0359] In the embodiments of the present disclosure, unless otherwise specified, elements expressed in the singular, such as "a", "an", "the", "above", "said", "the", "the", etc., may mean "one and only one", or "one or more", "at least one", etc. For example, when using articles such as "a", "an", "the" in English in translation, the noun following the article may be understood as a singular expression or a plural expression.

[0360] In some embodiments, "plurality" may refer to two or more.

[0361] In some embodiments, the terms "at least one," "one or more," "a plurality of," "multiple," etc. may be used interchangeably.

[0362] In some embodiments, descriptions such as "at least one of A and B," "A and / or B," "A in one case, B in another case," or "in response to one case A, in response to another case B" may include the following technical solutions depending on the situation: in some embodiments, A (A is executed independently of B); in some embodiments, B (B is executed independently of A); in some embodiments, execution is selected from A and B (A and B are selectively executed); and in some embodiments, A and B (both A and B are executed). The above is also applicable when there are more branches such as A, B, and C.

[0363] In some embodiments, "A or B" and other descriptions may include the following technical solutions depending on the situation: in some embodiments, A (A is executed independently of B); in some embodiments, B (B is executed independently of A); in some embodiments, execution is selected from A and B (A and B are selectively executed). The above is also applicable when there are more branches such as A, B, C, etc.

[0364] The prefixes such as "first" and "second" in the embodiments of the present disclosure are only used to distinguish different description objects and do not constitute any restriction on the position, order, priority, quantity or content of the description objects. For the statement of the description object, please refer to the description in the context of the claims or embodiments, and no unnecessary restriction should be constituted due to the use of prefixes. For example, if the description object is a "field", the ordinal number before the "field" in the "first field" and the "second field" does not limit the position or order between the "fields". "First" and "second" do not limit whether the "fields" they modify are in the same message, nor do they limit the order of the "first field" and the "second field". For another example, if the description object is a "level", the ordinal number before the "level" in the "first level" and the "second level" does not limit the priority between the "levels". For another example, the number of description objects is not limited by the ordinal number and can be one or more. Taking "first device" as an example, the number of "devices" can be one or more. In addition, the objects modified by different prefixes can be the same or different. For example, if the description object is "device", then the "first device" and the "second device" can be the same device or different devices, and their types can be the same or different; for another example, if the description object is "information", then the "first information" and the "second information" can be the same information or different information, and their contents can be the same or different.

[0365] In some embodiments, “including A,” “comprising A,” “used to indicate A,” and “carrying A” can be interpreted as directly carrying A or indirectly indicating A.

[0366] In some embodiments, terms such as "in response to...", "in response to determining...", "in the case of...", "at the time of...", "when...", "if...", "if...", etc. can be used interchangeably.

[0367] In some embodiments, terms such as "greater than", "greater than or equal to", "not less than", "more than", "more than or equal to", "not less than", "higher than", "higher than or equal to", "not less than", and "above" can be replaced with each other, and terms such as "less than", "less than or equal to", "not greater than", "less than", "less than or equal to", "not more than", "lower than", "lower than or equal to", "not higher than", and "below" can be replaced with each other.

[0368] In some embodiments, devices and the like can be interpreted as physical or virtual, and their names are not limited to those described in the embodiments. Terms such as "device," "equipment," "device," "circuit," "network element," "node," "function," "unit," "section," "system," "network," "chip," "chip system," "entity," and "subject" can be used interchangeably.

[0369] In some embodiments, "network" can be interpreted as devices included in the network (eg, access network equipment, core network equipment, etc.).

[0370] In some embodiments, the terms "Access Network Device (AN Device)", "Radio Access Network Device (RAN Device)", "Base Station (BS)", "Radio Base Station (Radio Base Station)", "Fixed Station (Fixed Station)", "Node (Node)", "Access Point (Access Point)", "Transmission Point (TP)", "Reception Point (RP)", "Transmission and / or Reception Point (TRP))", "Panel (Panel)", "Antenna Panel (Antenna Panel)", "Antenna Array (Antenna Array)" "Cell (Cell)", "Macro Cell (Macro Cell)", "Small Cell (Small Cell)", "Femto Cell (Femto Cell)", "Pico Cell (Pico Cell)" "Sector (Sector)", "Cell Group (Cell Group)", "Serving Cell", "Carrier (Carrier)", "Component Carrier (Component Carrier)", "Bandwidth Part (BWP)" and the like can be used interchangeably.

[0371] In some embodiments, the terms "terminal", "terminal device", "user equipment (UE)", "user terminal" "mobile station (MS)", "mobile terminal (MT)", subscriber station (Subscriber Station), mobile unit (Mobile Unit), subscriber unit (Subscriber Unit), wireless unit (Wireless Unit), remote unit (Remote Unit), mobile device (Mobile Device), wireless device (Wireless Device), wireless communication device (Wireless Communication Device), remote device (Remote Device), mobile subscriber station (Mobile Subscriber Station), access terminal (Access Terminal), mobile terminal (Mobile Terminal), wireless terminal (Wireless Terminal), remote terminal (Remote Terminal), handset (Handset), user agent (User Agent), mobile client (Mobile Client), client (Client) and the like can be used interchangeably.

[0372] In some embodiments, the access network device, the core network device, or the network device can be replaced by a terminal. For example, the various embodiments of the present disclosure can also be applied to a structure in which the communication between the access network device, the core network device, or the network device and the terminal is replaced by communication between multiple terminals (for example, device-to-device (D2D), vehicle-to-everything (V2X), etc.). In this case, it is also possible to set the structure in which the terminal has all or part of the functions of the access network device. In addition, terms such as "uplink" and "downlink" can also be replaced by terms corresponding to communication between terminals (for example, "side"). For example, uplink channels, downlink channels, etc. can be replaced by side channels or direct channels, and uplinks, downlinks, etc. can be replaced by side links or direct links.

[0373] In some embodiments, the terminal may be replaced by an access network device, a core network device, or a network device. In this case, the access network device, the core network device, or the network device may have a structure that has all or part of the functions of the terminal.

[0374] In some embodiments, obtaining data, information, etc. may comply with the laws and regulations of the country where the data is obtained.

[0375] In some embodiments, data, information, etc. may be obtained with the user's consent.

[0376] In addition, each element, each row, or each column in the table of the embodiment of the present disclosure can be implemented as an independent embodiment, and the combination of any elements, any rows, and any columns can also be implemented as an independent embodiment.

[0377] FIG1 is a schematic diagram illustrating an architecture of a communication system according to an embodiment of the present disclosure. As shown in FIG1 , the communication system 100 may include a terminal device 101 and a network device 102 .

[0378] In some embodiments, the terminal device 101 may include at least one of a mobile phone, a wearable device, an Internet of Things device, a car with communication capabilities, a smart car, a tablet computer, a computer with wireless transceiver capabilities, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal device in industrial control, a wireless terminal device in self-driving, a wireless terminal device in remote medical surgery, a wireless terminal device in smart grid, a wireless terminal device in transportation safety, a wireless terminal device in smart city, and a wireless terminal device in smart home, but is not limited thereto.

[0379] In some embodiments, the network device 102 may include at least one of an access network device and a core network device.

[0380] In some embodiments, the access network device may be a node or device that accesses the terminal device to the wireless network. The access network device may include an evolved NodeB (eNB), a next generation evolved NodeB (ng-eNB), a next generation NodeB (gNB), a node B (NB), a home node B (HNB), a home evolved nodeB (HeNB), a wireless backhaul device, a radio network controller (RNC), a base station controller (BSC), a base transceiver station (BTS), a base band unit (BBU), a mobile switching center, a base station in a 6G communication system, an open base station (Open RAN), a cloud base station (Cloud RAN), a base station in other communication systems, and at least one of an access node in a Wi-Fi system, but is not limited thereto.

[0381] In some embodiments, the technical solution of the present disclosure can be applied to the Open RAN architecture. In this case, the interfaces between or within the access network devices involved in the embodiments of the present disclosure can be transformed into internal interfaces of the Open RAN, and the processes and information interactions between these internal interfaces can be implemented through software or programs.

[0382] In some embodiments, the access network device can be composed of a centralized unit (CU) and a distributed unit (DU), where the CU can also be called a control unit (Control Unit). The CU-DU structure can be used to split the protocol layer of the access network device, with the functions of some protocol layers centrally controlled by the CU, and the functions of the remaining part or all of the protocol layers distributed in the DU, which is centrally controlled by the CU, but is not limited to this.

[0383] In some embodiments, the core network device may be a single device, or may be multiple devices or a group of devices. The core network may include at least one of an Evolved Packet Core (EPC), a 5G Core Network (5GCN), and a Next Generation Core (NGC).

[0384] It can be understood that the communication system described in the embodiment of the present disclosure is for the purpose of more clearly illustrating the technical solution of the embodiment of the present disclosure, and does not constitute a limitation on the technical solution proposed in the embodiment of the present disclosure. Ordinary technicians in this field can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solution proposed in the embodiment of the present disclosure is also applicable to similar technical problems.

[0385] The following embodiments of the present disclosure may be applied to the communication system 100 shown in FIG1 , or a portion thereof, but are not limited thereto. The entities shown in FIG1 are examples. The communication system may include all or part of the entities shown in FIG1 , or may include other entities outside of FIG1 . The number and form of the entities are arbitrary, and the entities may be physical or virtual. The connection relationship between the entities is an example. The entities may be connected or disconnected, and the connection may be in any manner, including direct or indirect, wired or wireless.

[0386] The embodiments of the present disclosure can be applied to Long Term Evolution (LTE), LTE-Advanced (LTE-A), LTE-Beyond (LTE-B), SUPER 3G, IMT-Advanced, 4th generation mobile communication system (4G), 5th generation mobile communication system (5G), 5G New Radio (NR), Future Radio Access (FRA), New-Radio Access Technology (RAT), New Radio (NR), New Radio Access (NX), Future generation radio access (FX), Global System for Mobile communications (GSM (registered trademark)), CDMA2000, Ultra Mobile Broadband (UMB), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.20, Ultra-WideBand (UWB), Bluetooth (registered trademark), Public Land Mobile Network (PLMN) networks, Device-to-Device (D2D) systems, Machine-to-Machine (M2M) systems, Internet of Things (IoT) systems, Vehicle-to-Everything (V2X), systems utilizing other communication methods, and next-generation systems based on and extending these methods. Furthermore, multiple systems may be combined (for example, a combination of LTE or LTE-A with 5G).

[0387] In some embodiments of the present disclosure, the communication system may introduce a first model, which may be an AI model or another model used for prediction. The first model may be one or more models, each including one or more functions. The first model may be deployed on a terminal device or on a network device.

[0388] In NR communication, for the FR2 communication band, since the high-frequency channel attenuates relatively quickly, beam-based transmission and reception can be used to ensure coverage.

[0389] In some embodiments, the network device may configure a reference signal resource set for beam measurement, and the terminal device may measure the reference signal resources in the reference signal resource set and report the IDs of X reference signal resources with relatively strong signal quality in the measurement results, as well as the Layer 1 Reference Signal Receiving Power (L1-RSRP) and / or Layer 1 Signal to Interference plus Noise Ratio (L1-SINR) of each of the X reference signal resources. The reference signal resource set configured by the network device includes X reference signal resources, each of which corresponds to a different transmit beam of the network device. For each reference signal resource, the terminal device needs to measure the reference signal resource through all receive beams, determine the beam measurement quality corresponding to each receive beam, and determine the strongest beam measurement quality from multiple beam measurement qualities. In the above measurement process, if the number of transmit beams of the network device is M and the number of receive beams of the terminal device is N, the number of beam pairs that the terminal device needs to measure is M*N. In the following embodiments of the present invention, L1-RSRP is used as an example for description, wherein L1-RSRP may also be partially or completely replaced by L1-SINR.

[0390] In some embodiments, the beam measurement quality is predicted by an AI model. For example, for spatial beam prediction, the terminal device may measure only a portion of the beam pairs. For example, the beam pairs measured by the terminal device may be 1 / 8, 1 / 4, etc. of the M*N beam pairs. The beam measurement quality of the measured portion of the beam pairs is input into the AI ​​model, and the beam quality of the M*N beam pairs is predicted by the AI ​​model. For example, for spatial beam prediction, the terminal device may measure only a portion of the beams. For example, the beam pairs measured by the terminal device may be 1 / 8, 1 / 4, etc. of the M transmit beams. The beam measurement quality of the measured portion of the beams is input into the AI ​​model, and the beam quality of the M transmit beams is predicted by the AI ​​model. For time-domain beam prediction, the terminal device may measure the beam quality of the beam pairs at historical times to obtain the beam historical measurement quality. Based on the beam historical measurement quality, the AI ​​model is used to predict the beam quality of the beam pairs at future times. Similarly, for time-domain beam prediction, the beam pairs can also be replaced with transmit beams.

[0391] In some embodiments, for spatial domain beam prediction, the measurement results of the beams in beam set setA can be predicted based on the measurement results of the beams in beam set setB; for time domain beam prediction, the measurement results of the beams in setA at future times can be predicted based on the measurement results of the beams in setB at historical times.

[0392] In some embodiments, for spatial beam prediction, the terminal device can measure the L1-RSRP of each beam in setB, input the measured multiple L1-RSRPs into the AI ​​model, and obtain the LA-RSRP of each beam in setA.

[0393] The relationship between setB and setA may include at least one of the following:

[0394] setB may be a subset of setA. For example, setA includes 32 reference signals (each reference signal corresponds to a beam direction), and setB includes N reference signals, where N<32, for example, N=8.

[0395] The beam corresponding to setB is a wide beam, and the beam corresponding to setA is a narrow beam. For example, setA includes 32 reference signals, each reference signal corresponds to a beam direction, and the range covered by the 32 reference signals is 120 degrees. setB includes N reference signals, for example, N=8, and the range covered by N reference signals is also 120 degrees. That is to say, the beam directions of multiple reference signals in setB cover the beam directions of multiple reference signals in setA. It can also be understood that the 32 / N reference signals in setA and the same reference signal in setB are in a quasi co-location (QCL) Type D relationship.

[0396] In some embodiments, for time domain beam prediction, the terminal device can measure the L1-RSRP of each beam in setB at historical time, input the measured multiple L1-RSRPs into the AI ​​model, and predict the L1-RSRP of each beam in setA at future time.

[0397] The relationship between setB and setA may include at least one of the following:

[0398] setB can be a subset of setA;

[0399] setB is the same as setA;

[0400] The beam corresponding to setB is a wide beam, and the beam corresponding to setA is a narrow beam.

[0401] In some embodiments, for time domain beam prediction, the network device may not send a reference signal to the terminal device in the future. After the terminal device obtains the measurement result through AI model prediction, it can report the measurement result to the network device.

[0402] In some embodiments, the AI ​​model can also predict beam failure information corresponding to the best K beams, for example, the probability of beam failure of the best K beams within a specified time.

[0403] In some embodiments, the AI ​​model may also predict the radio link quality values ​​of the best K beams at a specified time.

[0404] In some embodiments, AI models have corresponding lifecycles. For example, AI models have specific scopes of application and adaptable environments. Some AI models are suitable for suburban areas, some for urban areas, some for indoor use, some for peak hours, and some for off-peak hours. Before activating an AI model, a terminal device must determine its performance so that it can be activated when it can achieve good prediction results. This is a pressing issue.

[0405] FIG2A is a flow chart of a model performance monitoring method according to an embodiment of the present disclosure. The method can be executed by the above-mentioned communication system. As shown in FIG2A , the method may include:

[0406] Step S2101: The terminal device sends second information to the network device.

[0407] In some embodiments, the network device may receive the second message. For example, the network device may receive the second message sent by the terminal device. For another example, the network device may also receive the second message sent by another entity.

[0408] In some embodiments, the second information may be used to request performance monitoring of the first model.

[0409] In some embodiments, the second information may be used to instruct the network device to perform performance monitoring on the first model.

[0410] In some embodiments, the second information may be used to trigger the network device to perform performance monitoring on the first model.

[0411] In some embodiments, the second information may be used to request the network device to send the first information.

[0412] In some embodiments, the second information may be used to instruct the network device to send the first information.

[0413] In some embodiments, the second information may be used to trigger the network device to send the first information.

[0414] In some embodiments, the name of the second information is not limited, for example, it can be "model performance monitoring request information", "model performance monitoring indication information", "functional performance monitoring request information", "functional performance monitoring indication information", etc.

[0415] In some embodiments, the first model may be an inactive model.

[0416] In some embodiments, the first model can be deployed on the terminal device side.

[0417] In some embodiments, the first model may be one or more models.

[0418] In some embodiments, the first model may correspond to one or more functions.

[0419] In some embodiments, the terminal device may send capability indication information to the network device.

[0420] In some embodiments, the capability indication information may include at least one of the following:

[0421] Whether the terminal device supports performance monitoring of an inactive model deployed on the terminal device side;

[0422] Whether the terminal device supports performance monitoring of the inactivated model deployed on the network device side.

[0423] In some embodiments, the first model can be used for channel state information (CSI) compression, CSI prediction, spatial domain beam prediction, time domain beam prediction, beam failure information prediction, positioning, etc.

[0424] In some embodiments, before activating the first model, the terminal device may send the second information to the network device to request performance monitoring of the first model, and activate the first model if it is determined that the performance status of the first model is good based on the results of the performance monitoring.

[0425] In some embodiments, the second information may include at least one of the following:

[0426] Scheduling request (SR) for requesting model performance monitoring;

[0427] The first message Msg 1 or message A Msg A of random access (RA) for requesting model performance monitoring;

[0428] a model identifier of the first model;

[0429] a function identifier corresponding to the first model;

[0430] First configuration information, wherein the first configuration information includes resource configuration for performance monitoring of the first model.

[0431] Among them, the SR can be an SR dedicated to requesting model performance monitoring, or a reused existing SR for requesting allocation of uplink resources, or an SR dedicated to beam failure recovery (Beam failure recovery) and a listen before talk (LBT) SR.

[0432] RA can be a contention free RA and can be configured with a dedicated preamble. When the network device receives Msg 1 or Msg A (including the preamble of Msg 1 and the physical uplink shared channel (PUSCH) of Msg 3) sent by the terminal device, it can determine that the terminal device requests model performance monitoring.

[0433] The RA may also be a contention-based RA, and the network device may determine that the terminal device requests model performance monitoring based on Msg 3 or Msg 3 in Msg A sent by the terminal device.

[0434] If the first model has been identified, the network device may determine that the terminal device requests performance monitoring of the first model after receiving the model identifier of the first model sent by the terminal device. For example, the terminal device may place the model identifier of the first model in the PUSCH of Msg 3 or Msg A or the PUSCH obtained through the SR request and send it to the network device.

[0435] Here, "having identified the first model" can be understood as the terminal device sending model information of the first model to the network device. This model information may include model architecture, model parameters, model performance, model identification, and the environment (condition) in which the model is applicable. The network device has identified the first model based on this model information. After the network device identifies the first model, it can reach a unified understanding of the model information of the first model with the terminal device. Alternatively, if the first model is sent by the network device to the terminal device, the network device and the terminal device can also reach a unified understanding of the model information of the first model.

[0436] If the network device has identified the function corresponding to the first model, then after receiving the function identifier corresponding to the first model sent by the terminal device, the network device can determine that the terminal device requests performance monitoring of the first model. For example, the terminal device can place the function identifier corresponding to the first model in the PUSCH of Msg 3 or Msg A or the PUSCH obtained through the SR request and send it to the network device.

[0437] Here, "having identified the function corresponding to the first model" can be understood as the terminal device sending functional information of the first model to the network device. This functional information may include functional parameters, functional performance, functional identification, the environment (condition) in which the function is applicable, etc., and the network device has identified the first model based on this functional information. After the network device identifies the function corresponding to the first model, it can reach a unified understanding with the terminal device of the performance and architectural parameters of the function corresponding to the first model. Alternatively, if the function corresponding to the first model is sent by the network device to the terminal device, the network device and the terminal device can also reach a unified understanding of the functional information corresponding to the first model.

[0438] In some embodiments, the first model may be a model for performing beam prediction, and the first configuration information may include at least one of the following:

[0439] a first reference signal resource set including a beam to be measured;

[0440] a second reference signal resource set, wherein the second reference signal resource set includes a beam to be predicted;

[0441] a relationship between the first reference signal resource set and the second reference signal resource set;

[0442] a first performance monitoring duration, where the first performance monitoring duration is the duration during which the terminal device requests performance monitoring of the first model;

[0443] The first sample number is the number of samples requested by the terminal device for performance monitoring of the first model, and the sample number is the number of times the network device sends reference signal resources.

[0444] The terms "beam to be measured" and "beam to be predicted" are defined when the model is actually used. The "beam to be measured" can be understood as the beam that needs to be measured as input to the first model, and the "beam to be predicted" can be understood as the beam that can be predicted as output from the first model. A beam here can be understood as a transmit beam or a transmit / receive beam pair.

[0445] For example, the actual measurement result of each beam in the first reference signal resource set may be used as the input of the first model, and the prediction result of each beam in the second reference signal resource set may be obtained by prediction through the first model.

[0446] In some embodiments, a beam may correspond to a reference signal resource, which may correspond to a reference signal, and each reference signal may correspond to a beam direction. For each beam, a network device may configure a reference signal resource corresponding to the reference signal, and a terminal device may measure the beam based on the reference signal.

[0447] In some embodiments, the first performance monitoring duration can be understood as the duration during which the network device needs to send a reference signal.

[0448] In some embodiments, the first configuration information corresponding to different models may be different.

[0449] In some embodiments, the first model is a model for performing spatial beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following:

[0450] The first reference signal resource set is a subset of the second reference signal resource set;

[0451] The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same.

[0452] The first reference signal resource set is a subset of the second reference signal resource set, indicating that the beams to be measured in the first reference signal resource set are a subset of the beams to be predicted in the second reference signal resource set. For example, if the second reference signal resource set includes 32 beams and the first reference signal resource set may include 4 beams of the 32 beams in the second reference signal resource set, then the first reference signal resource set is a subset of the second reference signal resource set.

[0453] The first reference signal resource set is a subset of the second reference signal resource set, and can be represented by which reference signal resources in the second reference signal resource set the first reference signal resource set corresponds to. For example, the second reference signal resource set is setA, setA includes 32 beams (for convenience of explanation, the 32 beams can be recorded as beam 1, beam 2, beam 3, beam 4, beam 5, ..., beam 30, beam 31, beam 32), the first reference signal resource set is setB, setB includes 4 beams (for convenience of explanation, the 4 beams can be recorded as beam 1, beam 2, beam 3, beam 4), beam 1 in setB corresponds to beam 8 in setA, beam 2 in setB corresponds to beam 16 in setA, beam 3 in setB corresponds to beam 24 in setA, and beam 4 in setB corresponds to beam 32 in setA. From the correspondence between the beams in setB and the beams in setA, it can be seen that the 4 beams in setB are a subset of the 32 beams in setA.

[0454] It should also be noted that the number of beams in the first reference signal resource set and the number of beams in the second reference signal resource set are for exemplary purposes only, and the correspondence between the beams in the first reference signal resource set and the beams in the second reference signal resource set is also for exemplary purposes only, and the embodiments of the present disclosure do not limit this.

[0455] In one implementation, the beam coverage corresponding to the first reference signal resource set and the second reference signal resource set is the same, which can be expressed as multiple narrow beams in the second reference signal resource set that can be covered by each wide beam in the first reference signal resource set. For example, if the first reference signal resource set is setB, setB includes 8 wide beams (for convenience of explanation, the 8 beams can be recorded as beam 1, beam 2, beam 3, beam 4, ..., beam 8), and the second reference signal resource set is setA, setA includes 32 narrow beams (for convenience of explanation, the 32 beams can be recorded as beam 1, beam 2, beam 3, beam 4, beam 5, ..., beam 30, beam 31, beam 32), then the first reference signal resource set has a coverage range of 1.5, ... The correspondence between each wide beam in a reference signal resource set and the narrow beam in the second reference signal resource set covered by the wide beam can be expressed as beam 1 in setB covers beam 1, beam 2, beam 3 to beam 4 in setA, beam 2 in setB covers beam 5, beam 6, beam 7 to beam 8 in setA, and so on, beam 8 in setB covers beam 29, beam 20, beam 31 to beam 32 in setA.

[0456] In some embodiments, the first model is a model for performing time-domain beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following:

[0457] The first reference signal resource set is a subset of the second reference signal resource set;

[0458] The first reference signal resource set is the same as the second reference signal resource set;

[0459] The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same;

[0460] The relationship between a first time and a second time, wherein the first time is a time for performing measurement on the first reference signal resource set, and the second time is a time for performing prediction or measurement on the second reference signal resource set.

[0461] The first reference signal resource set and the second reference signal resource set are identical, indicating that the beam to be measured in the first reference signal resource set is identical to the beam to be predicted in the second reference signal resource set. For the time-domain beam prediction model, the measurement results for each beam in the first reference signal resource set measured at the first time can be used to predict the measurement results for each beam in the second reference signal resource set at the second time. This allows the terminal device to avoid performing any measurements at the second time.

[0462] In some embodiments, the relationship between the first time and the second time can be that the second time is between two adjacent first times, in a sliding window mode. Figure 2B is a schematic diagram illustrating a time relationship according to an embodiment of the present disclosure. As shown in Figure 2B, the first time includes N long periods, and the second time is a short period in the (N+1)th long period.

[0463] In some embodiments, the relationship between the first time and the second time may also be intermittent. Figure 2C is a schematic diagram illustrating a time relationship according to an embodiment of the present disclosure. As shown in Figure 2C, the first time includes N cycles, and the second time includes 2 cycles. For example, after measuring the first reference signal resource set during the first to Nth cycles included in the first time, prediction is performed on the second reference signal resource set during the N+1th and N+2th cycles included in the second time.

[0464] It should be noted that in FIG2C , after the prediction is performed at the second time, measurement can be started at the N+3th cycle. For example, taking N=4 as an example, measurement can be performed at the 1st to 4th cycles, prediction can be performed at the 5th to 6th cycles, measurement can be performed at the 7th to 10th cycles, and prediction can be performed at the 11th to 12th cycles.

[0465] In some embodiments, the relationship between the first time and the second time may further include the number of cycles included in the first time, the length of the cycles included in the first time, the duration corresponding to each second time, etc. For example, taking FIG2C as an example, the second time includes the N+1th cycle and the N+2th cycle, so the duration corresponding to the second time is 1 cycle length (corresponding to the N+1th cycle) and 2 cycle lengths (corresponding to the N+2th cycle).

[0466] In some embodiments, if different model identifiers correspond to different reference signal resources for model performance monitoring, that is, the reference signal resources corresponding to each model identifier can be uniquely determined, then the second information may include the model identifier of the first model, but not the first configuration information. For example, the terminal device may trigger the network device to configure the reference signal resources for model performance monitoring by reporting the model identifier of the first model.

[0467] In some embodiments, if different function identifiers correspond to different reference signal resources for model performance monitoring, that is, the reference signal resources corresponding to each function identifier can be uniquely determined, then the second information may include the function identifier of the first model but not the first configuration information. For example, the terminal device may trigger the network device to configure the reference signal resources for model performance monitoring by reporting the function identifier of the first model.

[0468] Step S2102: The network device sends first information to the terminal device.

[0469] The first information may be used to determine whether to perform performance monitoring on the first model.

[0470] In some embodiments, the terminal device may receive the first information. For example, the terminal device may receive the first information sent by the network device. For another example, the terminal device may also receive the first information sent by another entity.

[0471] In some embodiments, the first information may include information for monitoring the performance of the second model.

[0472] In some embodiments, the first information may include a first signal for performance monitoring of the second model. The first signal may include at least one of a channel state information reference signal (CSI-RS), a synchronization signal block (SSB), a positioning reference signal (PRS), or other signals sent by the network device to the terminal device.

[0473] In some embodiments, the name of the first information may not be limited, for example, it may be "performance monitoring information", "monitoring information", "model monitoring information", etc.

[0474] In some embodiments, the network device may send a first message, which may include the first information. Optionally, the terminal device may receive the first message. The first message may include at least one of a Radio Resource Control (RRC) message, a Medium Access Control Control Element (MAC CE), downlink control information (DCI), or other messages sent by the network device to the terminal device.

[0475] In some embodiments, the network device may send the first message to the terminal device. Optionally, the terminal device may receive the first message sent by the network device.

[0476] In some embodiments, the network device may determine to perform performance monitoring on the first model based on the second information, and send the first information to the terminal device.

[0477] In some embodiments, the first information may include at least one of the following:

[0478] a model identifier of the second model;

[0479] a function identifier corresponding to the second model;

[0480] a third reference signal resource set, wherein the third reference signal resource set includes a beam to be measured;

[0481] a fourth reference signal resource set, wherein the fourth reference signal resource set includes a beam to be predicted;

[0482] a relationship between the third reference signal resource set and the fourth reference signal resource set;

[0483] a relationship between a third time and a fourth time, the third time being a time for performing measurement on the third reference signal resource set, and the fourth time being a time for performing prediction or measurement on the fourth reference signal resource set;

[0484] a second performance monitoring duration, where the second performance monitoring duration is the duration during which the network device requests performance monitoring of the second model;

[0485] a second sample number, where the second sample number is the sample number requested by the network device for performance monitoring of the second model, and the sample number is the number of times the network device sends a reference signal resource;

[0486] Performance indicators corresponding to performance monitoring;

[0487] Reporting type of performance monitoring report;

[0488] The number of samples included in a performance monitoring report;

[0489] a model management threshold value, wherein the model management threshold value is used to determine whether to activate the second model;

[0490] Physical uplink control channel PUCCH / downlink physical shared channel PUSCH resources corresponding to the performance monitoring report;

[0491] a first event, the first event including a performance indicator for determining that a performance value of the second model satisfies a preset performance condition;

[0492] a first threshold value corresponding to the first event, the first threshold value being used to determine whether the first event is triggered;

[0493] A first offset value corresponding to the first event, where the first offset value is used to determine whether to trigger the first event.

[0494] In some embodiments, the second model may be the same as the first model, or the second model may be different from the first model.

[0495] In some embodiments, the second model may be a model that functions identically to the first model.

[0496] In some embodiments, the second model may also be any model different from the first model.

[0497] In some embodiments, the preset performance condition may include a performance value greater than a performance threshold value. The performance threshold value may be specified by a protocol or an empirical value, which is not limited in the embodiments of the present disclosure.

[0498] For example, after the network device receives the second information sent by the terminal device, if it is determined to perform performance monitoring on the first model, the model identifier of the first model can be sent to the terminal device; if it is determined to perform performance monitoring on one or more functions of the first model, the function identifier corresponding to the first model can be sent to the terminal device; if it is determined not to perform performance monitoring on the first model, a second model with the same function as the first model can be selected, or one or more functions of the second model can be selected for performance monitoring, and the model identifier of the second model or the function identifier corresponding to the second model can be sent to the terminal device; the network device may also not respond to the second information sent by the terminal device, select any second model that needs to be performance monitored, and send the model identifier of the second model or the function identifier corresponding to the second model to the terminal device. The second model can be a model deployed on the terminal device side or a model deployed on the network device side.

[0499] In some embodiments, the third reference signal resource set may be the same as the first reference signal resource set, and the fourth reference signal resource set may be the same as the second reference signal resource set.

[0500] For example, if the network device determines to perform performance monitoring on the first model, it may send a third reference signal resource set that is the same as the first reference signal resource and a fourth reference signal resource set that is the same as the second reference signal resource to the terminal device.

[0501] In some embodiments, the third reference signal resource set may be different from the first reference signal resource set, and the fourth reference signal resource set may be different from the second reference signal resource set.

[0502] In one implementation, the third reference signal resource set may be determined based on the first reference signal resource set, and the fourth reference signal resource set may be determined based on the second reference signal resource set.

[0503] For example, if the network device determines to perform performance monitoring on the first model, it can adjust the first reference signal resource set and the second reference signal resource set sent by the terminal device, and send the adjusted third reference signal resource set and the fourth reference signal resource set to the terminal device.

[0504] In one implementation, the third reference signal resource set and the fourth reference signal resource set may also be reference signal resource sets re-determined by the network device based on the first model. "Re-determination" may be understood as the network device determining the third reference signal resource set and the fourth reference signal resource set without referring to the first reference signal resource set and the second reference signal resource set.

[0505] In one implementation, the third reference signal resource set and the fourth reference signal resource set may also be reference signal resource sets configured by the network device for performance monitoring of a second model, where the second model is different from the first model.

[0506] It should be noted that the relationship between the third reference signal resource set and the fourth reference signal resource set can refer to the relationship between the first reference signal resource set and the second reference signal resource set, and will not be repeated here. The relationship between the third time and the fourth time can refer to the relationship between the first time and the second time, and will not be repeated here.

[0507] In some embodiments, the performance indicators corresponding to the performance monitoring may include at least one of the following:

[0508] Prediction accuracy of beams or beam pairs;

[0509] prediction accuracy of the beam or beam pair for which an L1-RSRP difference is within a first threshold, wherein the L1-RSRP difference is the difference between the measured L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the best measured beam or beam pair;

[0510] L1-RSRP difference, where the L1-RSRP difference is the difference between the measured L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the measured best beam or beam pair, wherein the L1-RSRP difference includes at least one of the following: an average value of the L1-RSRP difference, a cumulative distribution function of the L1-RSRP difference, a proportion of the L1-RSRP difference less than or equal to a second threshold, and a proportion of the L1-RSRP difference greater than a third threshold;

[0511] a predicted L1-RSRP difference, where the predicted L1-RSRP difference is the difference between the predicted L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the best predicted beam or beam pair, wherein the predicted L1-RSRP difference includes at least one of the following: an average value of the predicted L1-RSRP difference, a cumulative distribution function of the predicted L1-RSRP difference, a proportion of the predicted L1-RSRP difference less than or equal to the second threshold, and a proportion of the predicted L1-RSRP difference greater than the third threshold, wherein the predicted L1-RSRP difference is the difference between the predicted L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the best predicted beam or beam pair;

[0512] Terminal device throughput;

[0513] Reference signal overhead;

[0514] Uplink control information overhead;

[0515] Predicted latency;

[0516] Beam failure probability prediction accuracy;

[0517] Wireless link quality prediction variability;

[0518] Block error rate prediction accuracy;

[0519] Beam failure time prediction accuracy;

[0520] New beam prediction accuracy.

[0521] In some embodiments, the beam prediction accurately indicates that the identity of the predicted best beam includes the identity of the measured best beam, or that the identity of the predicted best beam is included in the identity of the measured best beam.

[0522] In some embodiments, the beam pair prediction accurately indicates that the identification of the predicted best beam pair includes the identification of the measured best beam pair, or the identification of the predicted best beam pair is included in the identification of the measured best beam pair.

[0523] In some embodiments, the identifier of a downlink transmit beam may be equivalent to the identifier of a reference signal resource. For example, the identifier of a downlink transmit beam may be an SSB identifier (ID), a CSI-RS ID, or a sounding reference signal (SRS) ID. The identifier of a downlink receive beam may be a receive beam identifier (Rx beam ID) of a terminal device. The identifier of a beam pair may be an identifier corresponding to a combination of a downlink transmit beam and a downlink receive beam.

[0524] In some embodiments, the best beam may be the beam with the strongest L1-RSRP or L1-SINR, and the best beam pair may be the beam pair with the strongest L1-RSRP or L1-SINR.

[0525] In some embodiments, the best predicted beam is a predicted best beam, the best predicted beam pair is a predicted best beam pair, the measured best beam is a measured best beam, and the measured best beam pair is a measured best beam pair. The first threshold can be determined based on the results of multiple model derivations. For example, the first threshold can be 1 dB.

[0526] In some embodiments, the L1-RSRP difference may be the difference between the measured L1-RSRP of the best predicted beam and the measured L1-RSRP of the best beam, or the difference between the measured L1-RSRP of the best predicted beam pair and the measured L1-RSRP of the best beam pair.

[0527] For example, the prediction accuracy of the beam whose L1-RSRP difference is within the first threshold can be the accuracy of the difference between the measured L1-RSRP of the best predicted beam and the measured L1-RSRP of the best beam within 1dB; the prediction accuracy of the beam pair whose L1-RSRP difference is within the first threshold can be the accuracy of the difference between the measured L1-RSRP of the best predicted beam pair and the measured L1-RSRP of the best beam pair within 1dB.

[0528] In some embodiments, the L1-RSRP difference can be the difference between the measured L1-RSRP of the best predicted beam and the measured L1-RSRP of the best beam, or the difference between the measured L1-RSRP of the best predicted beam pair and the measured L1-RSRP of the best beam pair.

[0529] For example, the L1-RSRP difference may be the average of the difference between the measured L1-RSRP of the best predicted beam and the measured L1-RSRP of the best beam, or the average of the difference between the measured L1-RSRP of the best predicted beam pair and the measured L1-RSRP of the best beam pair. The L1-RSRP difference may be the cumulative distribution function of the difference between the measured L1-RSRP of the best predicted beam and the measured L1-RSRP of the best beam, or the cumulative distribution function of the difference between the measured L1-RSRP of the best predicted beam pair and the measured L1-RSRP of the best beam pair. The L1-RSRP difference may be the ratio of the difference between the measured L1-RSRP of the best predicted beam and the measured L1-RSRP of the best beam being less than or equal to the second threshold, or the ratio of the difference between the measured L1-RSRP of the best predicted beam pair and the measured L1-RSRP of the best beam pair being less than or equal to the second threshold. The L1-RSRP difference can be the ratio of the difference between the measured L1-RSRP of the best predicted beam and the measured L1-RSRP of the best beam greater than the third threshold, or it can be the ratio of the measured L1-RSRP of the best predicted beam pair and the measured L1-RSRP of the best beam pair greater than the third threshold.

[0530] In some embodiments, the second threshold and the third threshold can be determined based on the results of multiple derivations of the model.

[0531] In some embodiments, the predicted L1-RSRP difference can be the difference between the predicted L1-RSRP of the best predicted beam and the measured L1-RSRP of the best predicted beam, or the difference between the predicted L1-RSRP of the best predicted beam pair and the measured L1-RSRP of the best predicted beam pair.

[0532] For example, the predicted L1-RSRP difference may be the average of the predicted L1-RSRP differences between the predicted L1-RSRP of the best predicted beam and the measured L1-RSRP of the best predicted beam, or the average of the predicted L1-RSRP differences between the predicted L1-RSRP of the best predicted beam pair and the measured L1-RSRP of the best predicted beam pair. The predicted L1-RSRP difference may be the cumulative distribution function of the predicted L1-RSRP differences between the predicted L1-RSRP of the best predicted beam and the measured L1-RSRP of the best predicted beam, or the cumulative distribution function of the predicted L1-RSRP differences between the predicted L1-RSRP of the best predicted beam pair and the measured L1-RSRP of the best predicted beam pair. The predicted L1-RSRP difference may be a ratio in which the predicted L1-RSRP difference between the predicted L1-RSRP of the best predicted beam and the measured L1-RSRP of the best predicted beam is less than or equal to the second threshold, or a ratio in which the predicted L1-RSRP difference between the predicted L1-RSRP of the best predicted beam pair and the measured L1-RSRP of the best predicted beam pair is less than or equal to the second threshold. The predicted L1-RSRP difference may be a ratio in which the predicted L1-RSRP difference between the predicted L1-RSRP of the best predicted beam and the measured L1-RSRP of the best predicted beam is greater than the third threshold, or a ratio in which the predicted L1-RSRP difference between the predicted L1-RSRP of the best predicted beam pair and the measured L1-RSRP of the best predicted beam pair is greater than the third threshold.

[0533] In some embodiments, the SINRs corresponding to the two beams or beam pairs can be determined based on the best predicted beam or beam pair and the measured best beam or beam pair, and the capacity value, i.e., the terminal device throughput, can be obtained through Shannon capacity calculation.

[0534] In some embodiments, the uplink control information may include the signaling overhead of information that the terminal device needs to report to the terminal device during the model performance monitoring process, for example, when the second model is deployed on the terminal device side, the performance value sent by the terminal device to the network device; when the second model is deployed on the network device side, the terminal device sends the test results of the first reference signal resource to the network device.

[0535] In some embodiments, the prediction delay may include the time required to predict the measurement result using the second model. For example, the prediction delay may include the time period from inputting the measurement data into the second model to outputting the prediction data from the second model.

[0536] In some embodiments, the beam failure probability prediction accuracy may be determined by the predicted probability of beam failure.

[0537] In some embodiments, the predicted beam failure probability may be a hard probability. For example, if the predicted beam fails and the measured beam also fails, the prediction is accurate; if the predicted beam fails and the measured beam does not fail, the prediction is inaccurate.

[0538] In some embodiments, the predicted probability of beam failure can be a soft probability. For example, if the predicted probability of beam failure is greater than a first probability threshold and the measured probability of beam failure is also greater than the first probability threshold, the prediction is accurate. If the predicted probability of beam failure is greater than the first probability threshold and the measured probability of beam failure is less than or equal to the first probability threshold, the prediction is inaccurate. The first probability threshold can be determined based on the results of multiple model derivations. For example, the first probability threshold can be 50%.

[0539] In some embodiments, the radio link quality prediction difference may be a difference between the predicted radio link quality at a specified time and the measured radio link quality at the specified time.

[0540] For example, if the prediction is for a predicted value of the wireless link quality at a specified time, the measured value of the wireless link quality measured at the specified time can be determined, and the difference between the predicted value and the measured value is the predicted difference of the wireless link quality. If the prediction is for a predicted minimum value of the wireless link quality within the specified time, the measured minimum value of the wireless link quality measured within the specified time can be determined, and the difference between the predicted minimum value and the measured minimum value is the predicted difference of the wireless link quality.

[0541] In some embodiments, the block error rate prediction difference may be a difference between a predicted block error rate and a measured block error rate.

[0542] For example, if the predicted BLER value is a Block Error Rate (BLER) value for a specified time, the measured BLER value of the BLER measured at the specified time can be determined, and the difference between the predicted BLER value and the measured BLER value is the BLER prediction difference. If the predicted BLER maximum value is a BLER value for a specified time, the measured BLER maximum value of the BLER measured during the specified time can be determined, and the difference between the predicted BLER maximum value and the measured BLER maximum value is the BLER difference.

[0543] In some embodiments, the beam failure time prediction accuracy may be determined based on the predicted beam failure time and the measured beam failure time.

[0544] For example, if the difference between the predicted time of the predicted beam failure and the measured time of the beam failure is less than the threshold value, it means that the beam failure time prediction is accurate; if the difference between the predicted time of the predicted beam failure and the measured time of the beam failure is greater than or equal to the threshold value, it means that the beam failure time prediction is inaccurate.

[0545] In some embodiments, the new beam prediction accuracy may be determined based on the predicted new beam and the measured new beam.

[0546] For example, if a beam failure is predicted to occur at a specified time, and a beam failure is also measured at the specified time, it can be determined whether the predicted new beam includes the measured new beam. If it is determined that the predicted new beam includes the measured new beam, the new beam prediction is accurate; if it is determined that the predicted new beam does not include the measured new beam, the new beam prediction is inaccurate.

[0547] In some embodiments, the L1-RSRP difference may also be the L1-SINR difference, the L1-RSRP difference may also be the L1-SINR difference, and the predicted L1-RSRP difference may also be the predicted L1-SINR difference. It should be noted that the determination method of the L1-SINR difference, the L1-SINR difference, and the predicted L1-SINR difference can refer to the above-mentioned determination method of the L1-RSRP difference, the L1-RSRP difference, and the predicted L1-SINR difference, and will not be repeated here.

[0548] In some embodiments, the reporting type of the performance monitoring report may include periodic reporting or aperiodic reporting.

[0549] In some embodiments, the number of samples included in the one-time monitoring report can be understood as the number of samples reported once. For example, if the number of samples included in the one-time monitoring report is 5, then the performance monitoring report is reported when the number of samples reaches 5. Alternatively, it can be understood that the period for sending the report is 5 times the period for sending the reference signal.

[0550] In some embodiments, the model management threshold value may be a threshold value for managing a model or function. For example, when the performance value is higher than the threshold value, the model or function may be activated.

[0551] In some embodiments, the first event may be triggered based on a performance value, such as a comparison between a performance value of performance monitoring and a first threshold. For example, the performance indicator is beam prediction accuracy, and the first event is triggered if the beam prediction accuracy corresponding to the performance value is greater than the first threshold.

[0552] In some embodiments, the first threshold value may include at least one of the following: a first accuracy threshold; a second accuracy threshold; a first difference threshold; a second difference threshold; a first throughput threshold; a first overhead threshold; a second overhead threshold; a first latency threshold; a third accuracy threshold; a third difference threshold; a fourth difference threshold; a fourth accuracy threshold; and a fifth accuracy threshold.

[0553] In some embodiments, the first threshold value may be determined based on multiple derivation results of the model, which is not limited in the embodiments of the present disclosure.

[0554] In some embodiments, the setting method of the first offset value can refer to the setting method of the first threshold value described above, which will not be repeated here.

[0555] Step S2103: The terminal device measures the beam in the fourth reference signal resource set according to the first information to obtain measurement data.

[0556] In some embodiments, the terminal device may measure each beam in the fourth reference signal resource set according to the first information to obtain measurement data.

[0557] In some embodiments, the terminal device may measure the reference signal corresponding to each beam in the fourth reference signal resource set to obtain measurement data.

[0558] In some embodiments, the name of the measurement data is not limited, and may be, for example, “model input data”, “actual measurement data”, “actual measurement results”, etc.

[0559] In some embodiments, the measurement data may include at least one of the following:

[0560] The measured L1-RSRPs of the best K beams in the fourth reference signal resource;

[0561] The measured L1-RSRP of each beam in the fourth reference signal resource set;

[0562] the radio link quality of each of the best K beams measured within a fifth time period;

[0563] The lowest value of the radio link quality of each beam in the best K beams measured within the fifth time period;

[0564] a block error rate (BLER) of each of the best K beams measured within the fifth time period;

[0565] a highest value of the BLER of each beam in the best K beams measured within the fifth time period;

[0566] A probability that the radio link quality of each of the best K beams measured within the fifth time is less than or equal to a preset link quality threshold.

[0567] In some embodiments, the preset link quality threshold may be specified by a protocol, which is not limited in the embodiments of the present disclosure.

[0568] In some embodiments, the best K beams may be the first K beams of the plurality of beams in the fourth reference signal resource set arranged in descending order of beam quality.

[0569] In some embodiments, the measurement data may also include identifiers of the best K beams in the measured fourth reference signal resource, L1-RSRP corresponding to the identifiers of the best K beams in the measured fourth reference signal resource, and L1-RSRP corresponding to the identifier of each beam in the measured fourth reference signal resource set.

[0570] In some embodiments, the measurement data may further include beam identifiers of the best K beams in the measured fourth reference signal resource.

[0571] In some embodiments, the above L1-RSRP may also be L1-SINR.

[0572] In some embodiments, the fifth time may be any specified time, which is not limited in the embodiments of the present disclosure.

[0573] In some embodiments, the wireless link quality may include signal-to-noise ratio (SNR), SINR, etc.

[0574] In some embodiments, the BLER corresponding to the wireless link quality may be obtained. If the BLER is higher than a preset link quality threshold, it indicates a beam failure.

[0575] In some embodiments, the probability that the radio link quality of each of the best K beams measured within the fifth time is less than or equal to a preset link quality threshold may include a hard probability.

[0576] For example, if it is determined that the BLER is higher than the preset link quality threshold, the probability is 1; if it is determined that the BLER is lower than or equal to the preset link quality threshold, the probability is 0.

[0577] In some embodiments, the probability that the radio link quality of each of the best K beams measured within the fifth time is less than or equal to a preset link quality threshold may include a soft probability.

[0578] For example, if it is determined that the BLER is higher than the preset link quality threshold, the difference between the BLER and the preset link quality threshold is greater than the threshold value, then the probability is 1; otherwise, the probability is 80%; if it is determined that the BLER is lower than the preset link quality threshold, the difference between the BLER and the preset link quality threshold is greater than the threshold value, then the probability is 0; otherwise, the probability is 30%.

[0579] In some embodiments, if the highest BLER value is greater than a threshold, it indicates that beam failure may occur.

[0580] Step S2104: The terminal device measures the beam in the third reference signal resource set according to the first information to obtain measurement data.

[0581] In some embodiments, the terminal device may measure each beam in the third reference signal resource set according to the first information to obtain measurement data.

[0582] In some embodiments, the terminal device may measure the reference signal corresponding to each beam in the third reference signal resource set to obtain measurement data.

[0583] In some embodiments, the measurement data may include at least one of the following:

[0584] The measured L1-RSRPs of the best K beams in the third reference signal resource set;

[0585] The measured L1-RSRP of each beam in the third reference signal resource set.

[0586] In some embodiments, the best K beams may be the first K beams of the plurality of beams in the third reference signal resource set arranged in descending order of beam quality.

[0587] In some embodiments, the measurement data may also include identifiers of the best K beams in the measured third reference signal resource, L1-RSRP corresponding to the identifiers of the best K beams in the measured third reference signal resource, and L1-RSRP corresponding to the identifier of each beam in the measured third reference signal resource set.

[0588] In some embodiments, the above L1-RSRP may also be L1-SINR.

[0589] Step S2105: The terminal device obtains predicted data based on the measurement data corresponding to the third reference signal resource.

[0590] In some embodiments, the terminal device may input the measurement data corresponding to the third reference signal resource into the second model to obtain the predicted data output by the second model.

[0591] For example, the terminal device may perform prediction based on the configuration information. For example, the terminal device may perform prediction using the second model based on the configuration information to obtain a prediction result. The configuration information may be information predefined by a protocol, or may be information configured by the terminal device, or may be information received by the terminal device from a network device, for example, configuration information determined by the network device and sent to the terminal device.

[0592] In some embodiments, the prediction data may include at least one of the following:

[0593] The predicted L1-RSRPs of the best K beams in the fourth reference signal resource set;

[0594] The predicted L1-RSRP of each beam in the fourth reference signal resource set;

[0595] predicted wireless link qualities of the best K beams within a fifth time period;

[0596] The predicted lowest value of the radio link qualities of the best K beams within the fifth time;

[0597] The predicted BLERs of the best K beams within the fifth time period;

[0598] The predicted highest BLER value of the best K beams within the fifth time period;

[0599] The predicted probability that the wireless link qualities of the best K beams within the fifth time are less than or equal to a preset link quality threshold.

[0600] It should be noted that the description of the above-mentioned prediction data can refer to the description of the measurement data in the above-mentioned step S2103, and will not be repeated here.

[0601] In some embodiments, the name of the prediction data is not limited, and may be, for example, "model output data", "model output result", "prediction result", etc.

[0602] Step S2106: The terminal device sends a performance monitoring report to the network device.

[0603] In some embodiments, the network device may receive a performance monitoring report. For example, the network device may receive a performance monitoring report sent by a terminal device. For another example, the network device may also receive a performance monitoring report sent by another entity.

[0604] In some embodiments, the name of the performance monitoring report is not limited, and may be, for example, "performance report", "monitoring results", "monitoring report", "model monitoring report", "model performance monitoring report", etc.

[0605] In some embodiments, the performance monitoring report may include at least one of the following:

[0606] Measurement data;

[0607] Forecast data;

[0608] a performance value of the second model, the performance value determined based on the measured data and the predicted data;

[0609] The first event is triggered based on a comparison result between the performance value of the second model and the first threshold value or the first offset value;

[0610] First operation information, where the first operation information is used to instruct a management operation to be performed on the second model, where the management operation includes whether to activate the second model.

[0611] In some embodiments, the performance value may include at least one of the following:

[0612] the accuracy of the predicted beam or beam pair;

[0613] prediction accuracy of the beam or beam pair for which the L1-RSRP difference is within the first threshold;

[0614] a first L1-RSRP difference, where the first L1-RSRP difference is a difference between a measured L1-RSRP of the best predicted beam or beam pair and a measured L1-RSRP of the measured best beam or beam pair;

[0615] a second L1-RSRP difference, where the second L1-RSRP difference is a difference between a predicted L1-RSRP of the best predicted beam or beam pair and a measured L1-RSRP of the best predicted beam or beam pair;

[0616] The throughput of the terminal device for determining the performance value;

[0617] a reference signal overhead for determining the performance value;

[0618] Uplink control information overhead for determining the performance value;

[0619] used to obtain the predicted delay of the predicted data;

[0620] Beam failure probability prediction accuracy;

[0621] A radio link quality prediction difference, where the radio link quality prediction difference is a difference between the predicted radio link quality and the measured radio link quality;

[0622] A block error rate prediction difference, where the block error rate prediction difference is the difference between the predicted block error rate and the measured block error rate;

[0623] Beam failure time prediction accuracy;

[0624] New beam prediction accuracy.

[0625] In some embodiments, the first event includes at least one of the following:

[0626] The accuracy of the predicted beam or beam pair is greater than the first accuracy threshold;

[0627] The prediction accuracy of the beam or beam pair for which the L1-RSRP difference is within the first threshold is greater than the second accuracy threshold;

[0628] The first L1-RSRP difference is less than the first difference threshold;

[0629] The second L1-RSRP difference is less than the second difference threshold;

[0630] The throughput is greater than the first throughput threshold;

[0631] The reference signal overhead is less than the first overhead threshold;

[0632] The uplink control information overhead is less than the second overhead threshold;

[0633] The predicted delay is less than the first delay threshold;

[0634] The beam failure probability prediction accuracy is greater than the third accuracy threshold;

[0635] The predicted difference of the radio link quality is less than the third difference threshold;

[0636] The block error rate prediction difference is less than the fourth difference threshold;

[0637] The beam failure time prediction accuracy is greater than the fourth accuracy threshold;

[0638] The new beam prediction accuracy is greater than the fifth accuracy threshold.

[0639] In some embodiments, the first event may be triggered by a comparison result of the performance value of the second model with a first threshold value or a first offset value.

[0640] Illustratively, the performance value of the second model includes the accuracy of the predicted beam, and if it is determined that the accuracy of the predicted beam is greater than a first accuracy threshold, the first event is triggered.

[0641] In some embodiments, the first operation information may be determined based on a comparison result of the performance value of the second model and a threshold value for operation management. For example, if the terminal device determines to activate the second model based on the performance value of the second model, the first operation information indicates activation of the second model; if the terminal device determines not to activate the second model based on the performance value of the second model, the first operation information indicates deactivation of the second model, or the first operation information is not reported.

[0642] In some embodiments, if it is determined that the model performance meets the first condition based on the comparison result of the performance value of the second model and the threshold value of the operation management, the second model is activated.

[0643] In some embodiments, the first condition may be that the second model can be used or meets performance requirements. For example, if the first condition is met, the second model can be used by the terminal device; if the first condition is not met, the second model cannot be used by the terminal device.

[0644] For example, the first condition may be any condition that can trigger the first event.

[0645] In some embodiments, after obtaining the measurement data and the prediction data, the terminal device may send a performance monitoring report including the measurement data and the prediction data to the network device.

[0646] In some embodiments, after obtaining the measurement data and the prediction data, the terminal device can determine the performance value of the second model based on the measurement data and the prediction data, and send a performance monitoring report including the performance value of the second model to the network device.

[0647] In some embodiments, the terminal device may determine to trigger a first event based on the performance value of the second model and the first threshold value, and send a performance monitoring report including the first event to the network device.

[0648] In some embodiments, the terminal device may determine first operation information based on the performance value of the second model, and send a performance monitoring report including the first operation information to the network device.

[0649] It should be noted that the above-mentioned method of sending a performance monitoring report to a network device is an exemplary description, and a performance monitoring report containing multiple pieces of information may also be sent at one time, which is not limited in the embodiments of the present disclosure.

[0650] Step S2107: The network device determines to activate the second model according to the performance monitoring report.

[0651] In some embodiments, after receiving the performance monitoring report sent by the terminal, the network device may determine whether to activate the second model according to at least one item in the performance monitoring report.

[0652] In some embodiments, the network device may determine whether to activate the second model based on the first operation information in the performance monitoring report.

[0653] For example, if the first operation information indicates to activate the second model, the network device may determine to activate the second model; if the first operation information indicates not to activate the second model, the network device may determine not to activate the second model.

[0654] In some embodiments, the network device may determine whether to activate the second model based on the first event in the performance monitoring report.

[0655] For example, if the performance monitoring report includes any first event, the network device may determine to activate the second model; if the performance monitoring report does not include the first event, the network device may determine not to activate the second model.

[0656] In some embodiments, the network device may determine whether to activate the second model based on the performance value in the performance monitoring report.

[0657] It should be noted that the above-mentioned determination by the network device whether to activate the second model is an exemplary description and is not limited to this in the embodiments of the present disclosure.

[0658] Step S2108: The network device sends an activation instruction to the terminal device.

[0659] In some embodiments, the terminal device may receive an activation instruction. For example, the terminal device may receive an activation instruction sent by a network device. For another example, the terminal device may also receive an activation instruction sent by another entity.

[0660] In some embodiments, the activation instruction can be used by the terminal device to activate the second model.

[0661] In some embodiments, the activation instruction may be used to indicate whether the second model can be used by the terminal device.

[0662] In some embodiments, the activation instruction can be used to indicate whether the terminal device can use the second model.

[0663] In some embodiments, the network device determines to activate the second model and may send the activation instruction.

[0664] In some embodiments, the network device determines not to activate the second model and may not send the activation instruction.

[0665] In some embodiments, the activation instruction may be a model identifier or a function identifier of the second model.

[0666] Step S2109: The terminal device activates the second model according to the activation instruction.

[0667] In some embodiments, the terminal device may activate the second model according to the activation instruction.

[0668] In some embodiments, the terminal device may activate the second model according to the model identifier or function identifier included in the activation instruction.

[0669] Using the above method, the terminal device can request to monitor the performance of the model before activating the model deployed on the terminal device side, and activate the model when it is determined that the performance of the model meets the activation conditions, thereby achieving more accurate prediction results during the use of the model and improving network performance.

[0670] The method according to the embodiments of the present disclosure may include at least one of steps S2101 to S2108. For example, step S2101 may be implemented as an independent embodiment, step S2102 may be implemented as an independent embodiment, steps S2101+S2102 may be implemented as an independent embodiment, steps S2102+S2103+S2104 may be implemented as an independent embodiment, steps S2102+S2103+S2104+S2105 may be implemented as an independent embodiment, steps S2102+S2103+S2104+S2106 may be implemented as an independent embodiment, and steps S2102+S2103+S2104+S2107+S2108 may be implemented as independent embodiments, but the present invention is not limited thereto.

[0671] In some embodiments, the above steps S2101 to S2108 can be executed in a swapped order or simultaneously. For example, steps S2103 and S2104 can be executed in a swapped order or simultaneously.

[0672] In some embodiments, steps S2101 to S2108 are all optional. For example, steps S2101, S2102, S2104, S2105, S2107, and S2108 are optional, and one or more of these steps may be omitted or replaced in different embodiments. For another example, steps S2101, S2105, S2106, and S2107 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0673] In some embodiments, reference may be made to other optional implementations described before or after the description corresponding to FIG. 2A .

[0674] In some embodiments, the names of information, etc. are not limited to the names described in the embodiments, and terms such as "information", "message", "signal", "signaling", "report", "configuration", "indication", "instruction", "command", "channel", "parameter", "domain", "field", "symbol", "symbol", "codeword", "codebook", "codeword", "codepoint", "bit", "data", "program", and "chip" can be used interchangeably.

[0675] In some embodiments, "obtain", "get", "get", "receive", "transmit", "bidirectional transmission", "send and / or receive" can be interchangeable, and can be interpreted as receiving from other entities, obtaining from protocols, obtaining from higher layers, obtaining by self-processing, autonomous implementation, etc.

[0676] In some embodiments, terms such as "send", "transmit", "report", "download", "transmit", "bidirectional transmission", "send and / or receive" can be used interchangeably.

[0677] In some embodiments, terms such as "certain", "preset", "preset", "setting", "indicated", "a certain", "any", and "first" can be interchangeable. "Specific A", "preset A", "preset A", "setting A", "indicated A", "a certain A", "any A", and "first A" can be interpreted as A pre-specified in a protocol, etc., or as A obtained through setting, configuration, or indication, etc., or as specific A, a certain A, any A, or first A, etc., but not limited to this.

[0678] FIG2D is a flow chart of a model performance monitoring method according to an embodiment of the present disclosure. The method can be executed by the above-mentioned communication system. As shown in FIG2D , the method may include:

[0679] Step S2401: The network device sends third information to the terminal device.

[0680] In some embodiments, the terminal device may receive the third information. For example, the terminal device may receive the third information sent by the network device. For another example, the terminal device may also receive the third information sent by another entity.

[0681] In some embodiments, the third information may be used to request performance monitoring of the second model.

[0682] In some embodiments, the third information may be used to instruct the network device to perform performance monitoring on the second model.

[0683] In some embodiments, the third information may be used to trigger the network device to perform performance monitoring on the second model.

[0684] In some embodiments, the third information may be used to request the network device to send the first information.

[0685] In some embodiments, the third information may be used to instruct the network device to send the first information.

[0686] In some embodiments, the third information may be used to trigger the network device to send the first information.

[0687] In some embodiments, the name of the third information is not limited, for example, it can be "model performance monitoring request information", "model performance monitoring indication information", "functional performance monitoring request information", "functional performance monitoring indication information", etc.

[0688] In some embodiments, the second model may be an inactive model.

[0689] In some embodiments, the second model can be deployed on the terminal device side.

[0690] In some embodiments, the second model may be one or more models.

[0691] In some embodiments, the second model may correspond to one or more functions.

[0692] In some embodiments, the second model can be used for channel state information (CSI) compression, CSI prediction, spatial domain beam prediction, time domain beam prediction, beam failure information prediction, positioning, etc.

[0693] In some embodiments, the network device may send the third information to the terminal device before instructing the terminal device to activate the second model, requesting performance monitoring of the second model, and instruct the terminal device to activate the second model if it is determined that the performance status of the second model is good based on the results of the performance monitoring.

[0694] Step S2402: The terminal device sends fourth information to the network device.

[0695] In some embodiments, the network device may receive the fourth information. For example, the network device may receive the fourth information sent by the terminal device. For another example, the network device may also receive the fourth information sent by another entity.

[0696] In some embodiments, the fourth information may be used to indicate a request for the third information.

[0697] In some embodiments, the terminal device determines that it agrees to perform performance monitoring on the second model based on the third information, and can send the fourth information to the network device.

[0698] In some embodiments, the terminal device determines that it does not agree to perform performance monitoring on the second model based on the third information, and may not send the fourth information to the network device.

[0699] In some embodiments, the fourth information may be the same as the second information in the above step S2101, or may be different from the second information, which is not limited in the embodiments of the present disclosure.

[0700] Step S2403: The network device sends first information to the terminal device.

[0701] The optional implementation of step S2403 can refer to the optional implementation of step S2102 in FIG. 2A and other related parts in the embodiment involved in FIG. 2A , which will not be described in detail here.

[0702] In some embodiments, the terminal device may receive the first information. For example, the terminal device may receive the first information sent by the network device. For another example, the terminal device may also receive the first information sent by another entity.

[0703] In some embodiments, the network device may send the first information to the terminal device based on the fourth information.

[0704] In some embodiments, the network device may send the first information to the terminal device according to pre-configured resource configuration information for performance monitoring of the second model.

[0705] Step S2404: The terminal device measures the beam in the fourth reference signal resource set according to the first information to obtain measurement data.

[0706] The optional implementation of step S2404 can refer to the optional implementation of step S2103 in FIG. 2A and other related parts in the embodiment involved in FIG. 2A , which will not be described in detail here.

[0707] Step S2405: The terminal device measures the beam in the third reference signal resource set according to the first information to obtain measurement data.

[0708] The optional implementation of step S2405 can refer to the optional implementation of step S2104 in FIG. 2A and other related parts in the embodiment involved in FIG. 2A , which will not be described in detail here.

[0709] Step S2406: The terminal device obtains predicted data based on the measurement data corresponding to the third reference signal resource.

[0710] The optional implementation of step S2406 can refer to the optional implementation of step S2105 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0711] Step S2407: The terminal device sends a performance monitoring report to the network device.

[0712] The optional implementation of step S2407 can refer to the optional implementation of step S2106 in FIG2A and other related parts in the embodiment involved in FIG2A , which will not be described in detail here.

[0713] Step S2408: The network device determines to activate the second model according to the performance monitoring report.

[0714] The optional implementation of step S2408 can refer to the optional implementation of step S2107 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0715] Step S2409: The network device sends an activation instruction to the terminal device.

[0716] The optional implementation of step S2409 can refer to the optional implementation of step S2108 in FIG2A and other related parts in the embodiment involved in FIG2A , which will not be described in detail here.

[0717] Step S2410: The terminal device activates the second model according to the activation instruction.

[0718] The optional implementation of step S2410 can refer to the optional implementation of step S2109 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0719] In some embodiments, the network device may request to monitor the performance of the model before instructing to activate the model deployed on the terminal device side, and activate the model when it is determined that the performance of the model meets the activation conditions, thereby achieving more accurate prediction results during the use of the model and improving the performance of the network.

[0720] The method involved in the embodiments of the present disclosure may include at least one of the above steps S2401 to S2410. For example, step S2401 can be implemented as an independent embodiment, step S2403 can be implemented as an independent embodiment, steps S2401+S2403+S2404+S2405 can be implemented as an independent embodiment, steps S2401+S2403+S2404+S2405 can be implemented as an independent embodiment, steps S2401+S2403+S2404+S2405+S2406 can be implemented as an independent embodiment, steps S2401+S2403+S2404+S2405+S2407 can be implemented as an independent embodiment, and steps S2401+S2403+S2404+S2405+S2407+S2408+S2409 can be implemented as an independent embodiment, but are not limited thereto.

[0721] In some embodiments, the above steps S2401 to S2410 can be executed in a swapped order or simultaneously. For example, steps S2404 and S2405 can be executed in a swapped order or simultaneously.

[0722] In some embodiments, steps S2401 to S2410 are all optional. For example, steps S2402, S2405, S2407, S2408, and S2410 are optional, and one or more of these steps may be omitted or replaced in different embodiments. For another example, steps S2402, S2407, S2408, S2409, and S2410 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0723] FIG2E is a flow chart of a model performance monitoring method according to an embodiment of the present disclosure. The method can be executed by the above-mentioned communication system. As shown in FIG2E , the method may include:

[0724] Step S2501: The network device sends first information to the terminal device.

[0725] The optional implementation of step S2501 can refer to the optional implementation of step S2102 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0726] In some embodiments, the terminal device may receive the first information. For example, the terminal device may receive the first information sent by the network device. For another example, the terminal device may also receive the first information sent by another entity.

[0727] In some embodiments, the first information may be used to determine performance monitoring of the first model.

[0728] In some embodiments, the first model can be deployed on the terminal device side.

[0729] In some embodiments, the network device may actively send the first information to the terminal device.

[0730] In some embodiments, “the network device actively sends the first information to the terminal device” can be interpreted as the network device sending the first information to the terminal device without receiving the second information sent by the terminal device.

[0731] In some embodiments, “the network device actively sends the first information to the terminal device” can also be interpreted as the network device sending the first information to the terminal device without sending the third information to the terminal device.

[0732] In some embodiments, “the network device actively sends the first information to the terminal device” can also be interpreted as the network device sending the first information to the terminal device without receiving any trigger information.

[0733] Step S2502: The terminal device measures the beam in the fourth reference signal resource set according to the first information to obtain measurement data.

[0734] The optional implementation of step S2502 can refer to the optional implementation of step S2103 in FIG2A and other related parts in the embodiment involved in FIG2A , which will not be described in detail here.

[0735] Step S2503: The terminal device measures the beam in the third reference signal resource set according to the first information to obtain measurement data.

[0736] The optional implementation of step S2503 can refer to the optional implementation of step S2104 in FIG. 2A and other related parts in the embodiment involved in FIG. 2A , which will not be described in detail here.

[0737] Step S2504: The terminal device obtains predicted data based on the measurement data corresponding to the third reference signal resource.

[0738] The optional implementation of step S2504 can refer to the optional implementation of step S2105 in FIG2A and other related parts in the embodiment involved in FIG2A , which will not be described in detail here.

[0739] Step S2505: The terminal device sends a performance monitoring report to the network device.

[0740] The optional implementation of step S2505 can refer to the optional implementation of step S2106 in FIG2A and other related parts in the embodiment involved in FIG2A , which will not be described in detail here.

[0741] Step S2506: The network device determines to activate the second model according to the performance monitoring report.

[0742] The optional implementation of step S2506 can refer to the optional implementation of step S2107 in FIG. 2A and other related parts in the embodiment involved in FIG. 2A , which will not be described in detail here.

[0743] Step S2507: Send an activation instruction to the terminal device.

[0744] The optional implementation of step S2507 can refer to the optional implementation of step S2108 in FIG. 2A and other related parts in the embodiment involved in FIG. 2A , which will not be described in detail here.

[0745] Step S2508: The terminal device activates the second model according to the activation instruction.

[0746] The optional implementation of step S2508 can refer to the optional implementation of step S2109 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0747] In some embodiments, the network device may instruct the monitoring of the performance of the model before instructing the activation of the model deployed on the terminal device side, and activate the model when it is determined that the performance of the model meets the activation conditions, thereby achieving more accurate prediction results during the use of the model and improving the performance of the network.

[0748] The method according to the embodiments of the present disclosure may include at least one of steps S2501 to S2508. For example, step S2501 may be implemented as an independent embodiment, step S2507 may be implemented as an independent embodiment, steps S2501+S2502+S2503+S2504 may be implemented as an independent embodiment, steps S2501+S2503+S2504+S2505 may be implemented as an independent embodiment, steps S2501+S2502+S2503+S2505 may be implemented as an independent embodiment, and steps S2501+S2502+S2503+S2504+S2505 may be implemented as an independent embodiment, but the present invention is not limited thereto.

[0749] In some embodiments, the above steps S2501 to S2508 can be executed in a swapped order or simultaneously. For example, steps S2502 and S2503 can be executed in a swapped order or simultaneously.

[0750] In some embodiments, steps S2501 to S2508 are all optional. For example, steps S2503, S2504, S2505, S2506, and S2507 are optional, and one or more of these steps may be omitted or replaced in different embodiments. For another example, steps S2503, S2504, and S2508 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0751] FIG2F is a flow chart of a model performance monitoring method according to an embodiment of the present disclosure. The method can be executed by the above-mentioned communication system. As shown in FIG2F , the method may include:

[0752] Step S2601: The network device sends first information to the terminal device.

[0753] The optional implementation of step S2601 can refer to the optional implementation of step S2102 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0754] In some embodiments, the first information may be used to determine performance monitoring of the first model.

[0755] In some embodiments, the first model can be deployed on the network device side.

[0756] In some embodiments, the network device determines that performance monitoring of the second model is required, and sends the first information to the terminal device.

[0757] Step S2602: The terminal device measures the beam in the fourth reference signal resource set according to the first information to obtain measurement data.

[0758] The optional implementation of step S2602 can refer to the optional implementation of step S2103 in FIG2A and other related parts in the embodiment involved in FIG2A , which will not be described in detail here.

[0759] Step S2603: The terminal device measures the beam in the third reference signal resource set according to the first information to obtain measurement data.

[0760] The optional implementation of step S2603 can refer to the optional implementation of step S2104 in FIG. 2A and other related parts in the embodiment involved in FIG. 2A , which will not be described in detail here.

[0761] Step S2604: The terminal device sends a performance monitoring report to the network device.

[0762] In some embodiments, the performance monitoring report may include measurement data.

[0763] In some embodiments, after receiving the performance monitoring report, the network device may or may not send a response message to the terminal device, wherein the response message is used to inform the terminal device whether the network device activates the first model.

[0764] The optional implementation of step S2604 can refer to the optional implementation of step S2106 in FIG2A and other related parts in the embodiment involved in FIG2A , which will not be described in detail here.

[0765] In some embodiments, the network device may request to monitor the performance of the model before activating the model deployed on the network device side, and activate the model when it is determined that the performance of the model meets the activation conditions, thereby achieving more accurate prediction results during the use of the model and improving the performance of the network.

[0766] The method according to the embodiments of the present disclosure may include at least one of steps S2601 to S2604. For example, step S2601 may be implemented as an independent embodiment, step S2604 may be implemented as an independent embodiment, steps S2601+S2602+S2604 may be implemented as an independent embodiment, and steps S2601+S2603+S2604 may be implemented as an independent embodiment, but the present invention is not limited thereto.

[0767] In some embodiments, the above steps S2601 to S2604 can be executed in a swapped order or simultaneously. For example, steps S2602 and S2603 can be executed in a swapped order or simultaneously.

[0768] FIG2G is a flow chart of a model performance monitoring method according to an embodiment of the present disclosure. The method can be executed by the above-mentioned communication system. As shown in FIG2G , the method may include:

[0769] Step S2701: The network device sends third information to the terminal device.

[0770] The optional implementation of step S2701 can refer to the optional implementation of step S2102 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0771] In some embodiments, the third information may be used to request performance monitoring of the second model.

[0772] In some embodiments, the second model can be deployed on the network device side.

[0773] Step S2702: The terminal device sends fourth information to the network device.

[0774] The optional implementation of step S2702 can refer to the optional implementation of step S2402 in Figure 2D and other related parts in the embodiment involved in Figure 2D, which will not be repeated here.

[0775] Step S2703: The network device sends first information to the terminal device.

[0776] The optional implementation of step S2703 can refer to the optional implementation of step S2102 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0777] In some embodiments, the terminal device may receive the first information. For example, the terminal device may receive the first information sent by the network device. For another example, the terminal device may also receive the first information sent by another entity.

[0778] In some embodiments, the network device may send the first information to the terminal device based on the fourth information.

[0779] In some embodiments, the network device may send the first information to the terminal device according to pre-configured resource configuration information for performance monitoring of the second model.

[0780] In some embodiments, the first information may be used to determine performance monitoring of the second model.

[0781] In some embodiments, the second model is deployed on the network device side, and the first information sent by the network device to the terminal device may not include the model identifier or function identifier of the second model.

[0782] Step S2704: The terminal device measures the beam in the fourth reference signal resource set according to the first information to obtain measurement data.

[0783] The optional implementation of step S2704 can refer to the optional implementation of step S2103 in FIG2A and other related parts in the embodiment involved in FIG2A , which will not be described in detail here.

[0784] Step S2705: The terminal device measures the beam in the third reference signal resource set according to the first information to obtain measurement data.

[0785] The optional implementation of step S2705 can refer to the optional implementation of step S2104 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0786] Step S2706: The terminal device sends a performance monitoring report to the network device.

[0787] In some embodiments, the performance monitoring report may include measurement data.

[0788] The optional implementation of step S2706 can refer to the optional implementation of step S2106 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0789] In some embodiments, the input data of the second model deployed on the network device side may include at least one of the following:

[0790] If the second model is a beam information prediction model, the input data may include an identifier of a beam of the third reference signal resource set or an identifier of a beam pair, and an L1-RSRP corresponding to the beam or beam pair of the third reference signal resource set. For time domain beam prediction, the input data may further include measurement data of multiple historical measurement times.

[0791] If the second model is a beam failure information prediction model, and the beam failure information prediction model and the beam information prediction model are the same model, the input data may be a fifth time;

[0792] If the second model is a beam failure information prediction model, and the beam failure information prediction model and the beam information prediction model are different models, the input data may include at least one of the following:

[0793] at least one of the predicted L1-RSRP, L1-SINR, beam or beam pair identifier (reference signal resource identifier and / or receive beam identifier) ​​of the best K beams, or at least one of the measured L1-RSRP, L1-SINR, beam or beam pair identifier (reference signal resource identifier and / or receive beam identifier) ​​of the best K beams;

[0794] the measured L1-RSRP or L1-SINR of each beam in the fourth reference signal resource set, or the predicted L1-RSRP or L1-SINR of each beam in the fourth reference signal resource set;

[0795] the L1-RSRP or L1-SINR of the designated beam in the fourth reference signal resource set measured within the fifth time, or the L1-RSRP or L1-SINR of the designated beam in the fourth reference signal resource set predicted within the fifth time;

[0796] at least one of the measured L1-RSRP, L1-SINR, beam or beam pair identifier (reference signal resource identifier and / or receive beam identifier) ​​of a randomly selected beam in the fourth reference signal resource set, or at least one of the predicted L1-RSRP, L1-SINR, beam or beam pair identifier (reference signal resource identifier and / or receive beam identifier) ​​of a randomly selected beam in the fourth reference signal resource set;

[0797] Fifth time.

[0798] It should be noted that each beam may correspond to a reference signal resource, each beam identifier may correspond one-to-one to a reference signal resource identifier, and the beam identifier may also be a reference signal resource identifier.

[0799] In some embodiments, after receiving the performance monitoring report, the network device may or may not send a response message to the terminal device, wherein the response message is used to inform the terminal device whether the network device activates the first model.

[0800] In some embodiments, the network device can monitor the performance of the model before activating the model deployed on the network device side, and activate the model when it is determined that the performance of the model meets the activation conditions, thereby achieving more accurate prediction results during the use of the model and improving the performance of the network.

[0801] The method according to the embodiments of the present disclosure may include at least one of steps S2701 to S2704. For example, step S2701 may be implemented as an independent embodiment, step S2704 may be implemented as an independent embodiment, steps S2701+S2702+S2704 may be implemented as an independent embodiment, and steps S2701+S2703+S2704 may be implemented as an independent embodiment, but the present invention is not limited thereto.

[0802] In some embodiments, the above steps S2701 to S2704 can be executed in a swapped order or simultaneously. For example, steps S2702 and S2703 can be executed in a swapped order or simultaneously.

[0803] FIG3A is a flow chart of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG3A , the embodiment of the present disclosure relates to a model performance method, which can be executed by a terminal device. The method may include:

[0804] Step S3101: Send the second information.

[0805] The optional implementation of step S3101 can refer to the optional implementation of step S2101 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0806] In some embodiments, the terminal device may send the second information to the network device, but is not limited thereto. The terminal device may also send the second information to other entities.

[0807] In some embodiments, the second information may be used to request performance monitoring of the first model.

[0808] Step S3102: Obtain first information.

[0809] The optional implementation of step S3102 can refer to the optional implementation of step S2102 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0810] In some embodiments, the first information may be used to determine performance monitoring of the first model.

[0811] In some embodiments, the terminal device may receive the first information sent by the network device, but is not limited thereto. The terminal device may also receive the first information sent by other entities.

[0812] In some embodiments, the terminal device may obtain first information specified by the protocol.

[0813] Step S3103: Measure the beam in the fourth reference signal resource set according to the first information to obtain measurement data.

[0814] The optional implementation of step S3103 can be found in the optional implementation of step S2103 in FIG2A and other related parts in the embodiment involved in FIG2A , which will not be described in detail here.

[0815] Step S3104: Measure the beam in the third reference signal resource set according to the first information to obtain measurement data.

[0816] The optional implementation of step S3104 can refer to the optional implementation of step S2104 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0817] Step S3105: Obtain predicted data according to the measurement data corresponding to the third reference signal resource.

[0818] The optional implementation of step S3105 can refer to the optional implementation of step S2105 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0819] Step S3106: Send a performance monitoring report.

[0820] The optional implementation of step S3106 can refer to the optional implementation of step S2106 in FIG2A and other related parts in the embodiment involved in FIG2A , which will not be described in detail here.

[0821] Step S3107: Get activation instruction.

[0822] The optional implementation of step S3107 can refer to the optional implementation of step S2108 in FIG2A and other related parts in the embodiment involved in FIG2A , which will not be described in detail here.

[0823] In some embodiments, the activation instruction may be used to activate the second model.

[0824] In some embodiments, the terminal device may receive an activation instruction sent by a network device, but is not limited thereto. The terminal device may also receive an activation instruction sent by other entities.

[0825] In some embodiments, step S3102 may be omitted, and the network device may autonomously implement the function indicated by the first information, or the above function may be default or by default.

[0826] Step S3108: Activate the second model.

[0827] The optional implementation of step S3108 can refer to the optional implementation of step S2109 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0828] The method according to the embodiments of the present disclosure may include at least one of steps S3101 to S3108. For example, step S3101 may be implemented as an independent embodiment, steps S3101+S3102 may be implemented as an independent embodiment, steps S3102+S3103+S3104+S3106 may be implemented as an independent embodiment, and steps S3102+S3103+S3104+S3105+S3106 may be implemented as an independent embodiment, but the present invention is not limited thereto.

[0829] In some embodiments, the above steps S3101 to S3108 can be executed in a swapped order or simultaneously. For example, steps S3103 and S3104 can be executed in a swapped order or simultaneously.

[0830] In some embodiments, steps S3101 to S3108 are all optional steps. For example, steps S3101, S3105, S3107, and S3108 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0831] In some embodiments, reference may be made to other optional implementations described before or after the description corresponding to FIG. 3A .

[0832] FIG3B is a flow chart of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG3B , the embodiment of the present disclosure relates to a model performance monitoring method, which can be executed by a terminal device. The method may include:

[0833] Step S3201: Obtain first information.

[0834] The optional implementation of step S3201 can be found in step S2102 of FIG. 2A , the optional implementation of step S3102 of FIG. 3A , and other related parts in the embodiments involved in FIG. 2A and FIG. 3A , which will not be repeated here.

[0835] Step S3202: Measure the beam in the fourth reference signal resource set according to the first information to obtain measurement data.

[0836] The optional implementation of step S3202 can be found in step S2103 of FIG. 2A , the optional implementation of step S3103 of FIG. 3A , and other related parts in the embodiments involved in FIG. 2A and FIG. 3A , which will not be described in detail here.

[0837] Step S3203: Measure the beam in the third reference signal resource set according to the first information to obtain measurement data.

[0838] Optional implementations of step S3203 may be found in step S2104 of FIG. 2A , optional implementations of step S3104 of FIG. 3A , and other related parts in the embodiments involved in FIG. 2A and FIG. 3A , which will not be described in detail here.

[0839] Step S3204: Obtain predicted data according to the measurement data corresponding to the third reference signal resource.

[0840] The optional implementation of step S3204 can be found in step S2105 of FIG. 2A , the optional implementation of step S3105 of FIG. 3A , and other related parts in the embodiments involved in FIG. 2A and FIG. 3A , which will not be described in detail here.

[0841] Step S3205: Send a performance monitoring report.

[0842] The optional implementation of step S3205 can be found in step S2106 of FIG. 2A , the optional implementation of step S3106 of FIG. 3A , and other related parts in the embodiments involved in FIG. 2A and FIG. 3A , which will not be described in detail here.

[0843] Step S3206: Get activation instruction.

[0844] The optional implementation of step S3206 can be found in step S2108 of FIG. 2A , the optional implementation of step S3107 of FIG. 3A , and other related parts in the embodiments involved in FIG. 2A and FIG. 3A , which will not be described in detail here.

[0845] In some embodiments, the activation instruction may be used to activate the second model.

[0846] In some embodiments, the terminal device may receive an activation instruction sent by a network device, but is not limited thereto. The terminal device may also receive an activation instruction sent by other entities.

[0847] In some embodiments, step S3202 may be omitted, and the network device may autonomously implement the function indicated by the first information, or the above function may be default or by default.

[0848] Step S3207: Activate the second model.

[0849] The optional implementation of step S3207 can be found in the optional implementation of step S2109 in FIG. 2A , step S3108 in FIG. 3A , and other related parts in the embodiments involved in FIG. 2A and FIG. 3A , which will not be described in detail here.

[0850] The method according to the embodiments of the present disclosure may include at least one of steps S3201 to S3207. For example, step S3202 may be implemented as an independent embodiment, steps S3202+S3203+S3204+S3205 may be implemented as an independent embodiment, and steps S3202+S3203+S3204+S3205+S3206 may be implemented as an independent embodiment, but the present invention is not limited thereto.

[0851] In some embodiments, the above steps S3201 to S3208 can be executed in a swapped order or simultaneously. For example, steps S3203 and S3204 can be executed in a swapped order or simultaneously.

[0852] In some embodiments, steps S3201 to S3207 are all optional steps. For example, steps S3201, S3204, S3206, and S3207 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0853] In the embodiment of the present disclosure, step S3202 may be combined with step S3101 of FIG. 3A .

[0854] FIG3C is a flow chart of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG3C , the embodiment of the present disclosure relates to a model performance monitoring method, which can be executed by a terminal device. The method may include:

[0855] Step S3301: Obtain first information.

[0856] The optional implementation of step S3301 can be found in step S2102 of FIG. 2A , the optional implementation of step S3102 of FIG. 3A , and other related parts in the embodiments involved in FIG. 2A and FIG. 3A , which will not be repeated here.

[0857] Step S3302: Measure the beam in the fourth reference signal resource set according to the first information to obtain measurement data.

[0858] The optional implementation of step S3302 can be found in step S2103 of FIG. 2A , the optional implementation of step S3103 of FIG. 3A , and other related parts in the embodiments involved in FIG. 2A and FIG. 3A , which will not be repeated here.

[0859] Step S3303: Measure the beam in the third reference signal resource set according to the first information to obtain measurement data.

[0860] The optional implementation of step S3303 can be found in step S2104 of FIG. 2A , the optional implementation of step S3104 of FIG. 3A , and other related parts in the embodiments involved in FIG. 2A and FIG. 3A , which will not be repeated here.

[0861] Step S3304: Send a performance monitoring report.

[0862] The optional implementation of step S3304 can be found in step S2106 of FIG. 2A , the optional implementation of step S3106 of FIG. 3A , and other related parts in the embodiments involved in FIG. 2A and FIG. 3A , which will not be repeated here.

[0863] In some embodiments, the above steps are all optional steps.

[0864] In some embodiments, the embodiment shown in FIG. 3C may also be combined with step S3101 in the embodiment shown in FIG. 3A as a new embodiment.

[0865] In some embodiments, the embodiment shown in FIG. 3C may also be combined with step S3105 in the embodiment shown in FIG. 3A as a new embodiment.

[0866] In some embodiments, the embodiment shown in FIG. 3C may also be combined with step S3107 and step S3108 in the embodiment shown in FIG. 3A to form a new embodiment.

[0867] FIG3D is a flow chart of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG3D , the embodiment of the present disclosure relates to a model performance monitoring method, which can be executed by a terminal device. The method may include:

[0868] Step S3401: Obtain first information.

[0869] The optional implementation of step S3301 can be found in step S2102 of FIG. 2A , the optional implementation of step S3102 of FIG. 3A , and other related parts in the embodiments involved in FIG. 2A and FIG. 3A , which will not be repeated here.

[0870] In some embodiments, the method further comprises:

[0871] Second information is sent to the network device, where the second information is used to request performance monitoring of the first model.

[0872] In some embodiments, the second information includes at least one of the following:

[0873] Scheduling request SR for requesting model performance monitoring;

[0874] The first message Msg 1 or message A Msg A of random access RA for requesting model performance monitoring;

[0875] a model identifier of the first model;

[0876] a function identifier corresponding to the first model;

[0877] First configuration information, wherein the first configuration information includes resource configuration for performance monitoring of the first model.

[0878] In some embodiments, the first model is a model for performing beam prediction, and the first configuration information includes at least one of the following:

[0879] a first reference signal resource set including a beam to be measured;

[0880] a second reference signal resource set, wherein the second reference signal resource set includes a beam to be predicted;

[0881] a relationship between the first reference signal resource set and the second reference signal resource set;

[0882] a first performance monitoring duration, where the first performance monitoring duration is the duration during which the terminal device requests performance monitoring of the first model;

[0883] The first sample number is the number of samples requested by the terminal device for performance monitoring of the first model, and the sample number is the number of times the network device sends reference signal resources.

[0884] In some embodiments, the first model is a model for performing spatial beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following:

[0885] The first reference signal resource set is a subset of the second reference signal resource set;

[0886] The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same.

[0887] In some embodiments, the first model is a model for performing time-domain beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following:

[0888] The first reference signal resource set is a subset of the second reference signal resource set;

[0889] The first reference signal resource set is the same as the second reference signal resource set;

[0890] The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same;

[0891] The relationship between a first time and a second time, wherein the first time is a time for performing measurement on the first reference signal resource set, and the second time is a time for performing prediction or measurement on the second reference signal resource set.

[0892] In some embodiments, the method further comprises:

[0893] receiving third information sent by the network device, where the third information is used to request performance monitoring of the second model;

[0894] Fourth information is sent to the network device according to the third information, where the fourth information is used to indicate that the request for the third information is approved.

[0895] In some embodiments, the first information includes at least one of the following:

[0896] a model identifier of the second model;

[0897] a function identifier corresponding to the second model;

[0898] a third reference signal resource set, wherein the third reference signal resource set includes a beam to be measured;

[0899] a fourth reference signal resource set, wherein the fourth reference signal resource set includes a beam to be predicted;

[0900] a relationship between the third reference signal resource set and the fourth reference signal resource set;

[0901] a relationship between a third time and a fourth time, the third time being a time for performing measurement on the third reference signal resource set, and the fourth time being a time for performing prediction or measurement on the fourth reference signal resource set;

[0902] a second performance monitoring duration, where the second performance monitoring duration is the duration during which the network device requests performance monitoring of the second model;

[0903] a second sample number, where the second sample number is the sample number requested by the network device for performance monitoring of the second model, and the sample number is the number of times the network device sends a reference signal resource;

[0904] Performance indicators corresponding to performance monitoring;

[0905] Reporting type of performance monitoring report;

[0906] The number of samples included in a performance monitoring report;

[0907] a model management threshold value, wherein the model management threshold value is used to determine whether to activate the second model;

[0908] Physical uplink control channel PUCCH / downlink physical shared channel PUSCH resources corresponding to the performance monitoring report;

[0909] a first event, the first event including a performance indicator for determining that a performance value of the second model satisfies a preset performance condition;

[0910] a first threshold value corresponding to the first event, the first threshold value being used to determine whether the first event is triggered;

[0911] A first offset value corresponding to the first event, where the first offset value is used to determine whether to trigger the first event.

[0912] In some embodiments, the method further comprises:

[0913] Sending a performance monitoring report to the network device; the performance monitoring report includes at least one of the following:

[0914] Measurement data;

[0915] Forecast data;

[0916] a performance value of the second model, the performance value determined based on the measured data and the predicted data;

[0917] The first event is triggered based on a comparison result between the performance value of the second model and the first threshold value or the first offset value;

[0918] First operation information, where the first operation information is used to instruct a management operation to be performed on the second model, where the management operation includes whether to activate the second model.

[0919] In some embodiments, the method further comprises:

[0920] The beam in the fourth reference signal resource set is measured according to the first information to obtain the measurement data.

[0921] In some embodiments, the measurement data includes at least one of the following:

[0922] The measured layer 1 reference signal received power L1-RSRP of the best K beams in the fourth reference signal resource;

[0923] The measured L1-RSRP of each beam in the fourth reference signal resource set;

[0924] the radio link quality of each of the best K beams measured within a fifth time period;

[0925] The lowest value of the radio link quality of each beam in the best K beams measured within the fifth time period;

[0926] a block error rate (BLER) of each of the best K beams measured within the fifth time period;

[0927] a highest value of the BLER of each beam in the best K beams measured within the fifth time period;

[0928] A probability that the radio link quality of each of the best K beams measured within the fifth time is less than or equal to a preset link quality threshold.

[0929] In some embodiments, the method further comprises:

[0930] The beam in the third reference signal resource set is measured according to the first information to obtain the measurement data.

[0931] In some embodiments, the measurement data includes at least one of the following:

[0932] The measured L1-RSRPs of the best K beams in the third reference signal resource set;

[0933] The measured L1-RSRP of each beam in the third reference signal resource set.

[0934] In some embodiments, the method further comprises:

[0935] The predicted data is obtained according to the measurement data corresponding to the third reference signal resource.

[0936] In some embodiments, the prediction data includes at least one of the following:

[0937] The predicted L1-RSRPs of the best K beams in the fourth reference signal resource set;

[0938] The predicted L1-RSRP of each beam in the fourth reference signal resource set;

[0939] predicted wireless link qualities of the best K beams within a fifth time period;

[0940] The predicted lowest value of the radio link qualities of the best K beams within the fifth time;

[0941] The predicted BLERs of the best K beams within the fifth time period;

[0942] The predicted highest BLER value of the best K beams within the fifth time period;

[0943] The predicted probability that the wireless link qualities of the best K beams within the fifth time are less than or equal to a preset link quality threshold.

[0944] In some embodiments, the performance indicator includes at least one of the following:

[0945] Prediction accuracy of beams or beam pairs;

[0946] prediction accuracy of the beam or beam pair for which an L1-RSRP difference is within a first threshold, wherein the L1-RSRP difference is the difference between the measured L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the best measured beam or beam pair;

[0947] L1-RSRP difference, where the L1-RSRP difference is the difference between the measured L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the measured best beam or beam pair, wherein the L1-RSRP difference includes at least one of the following: an average value of the L1-RSRP difference, a cumulative distribution function of the L1-RSRP difference, a proportion of the L1-RSRP difference less than or equal to a second threshold, and a proportion of the L1-RSRP difference greater than a third threshold;

[0948] a predicted L1-RSRP difference, where the predicted L1-RSRP difference is the difference between the predicted L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the best predicted beam or beam pair, wherein the predicted L1-RSRP difference includes at least one of the following: an average value of the predicted L1-RSRP difference, a cumulative distribution function of the predicted L1-RSRP difference, a proportion of the predicted L1-RSRP difference less than or equal to the second threshold, and a proportion of the predicted L1-RSRP difference greater than the third threshold, wherein the predicted L1-RSRP difference is the difference between the predicted L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the best predicted beam or beam pair;

[0949] Terminal device throughput;

[0950] Reference signal overhead;

[0951] Uplink control information overhead;

[0952] Predicted latency;

[0953] Beam failure probability prediction accuracy;

[0954] Wireless link quality prediction variability;

[0955] Block error rate prediction difference;

[0956] Beam failure time prediction accuracy;

[0957] New beam prediction accuracy.

[0958] In some embodiments, the performance value includes at least one of the following:

[0959] the accuracy of the predicted beam or beam pair;

[0960] prediction accuracy of the beam or beam pair for which the L1-RSRP difference is within the first threshold;

[0961] a first L1-RSRP difference, where the first L1-RSRP difference is a difference between a measured L1-RSRP of the best predicted beam or beam pair and a measured L1-RSRP of the measured best beam or beam pair;

[0962] a second L1-RSRP difference, where the second L1-RSRP difference is a difference between a predicted L1-RSRP of the best predicted beam or beam pair and a measured L1-RSRP of the best predicted beam or beam pair;

[0963] The throughput of the terminal device for determining the performance value;

[0964] a reference signal overhead for determining the performance value;

[0965] Uplink control information overhead for determining the performance value;

[0966] used to obtain the predicted delay of the predicted data;

[0967] Beam failure probability prediction accuracy;

[0968] A radio link quality prediction difference, where the radio link quality prediction difference is a difference between the predicted radio link quality and the measured radio link quality;

[0969] A block error rate prediction difference, where the block error rate prediction difference is the difference between the predicted block error rate and the measured block error rate;

[0970] Beam failure time prediction accuracy;

[0971] New beam prediction accuracy.

[0972] In some embodiments, the first threshold value includes at least one of the following:

[0973] a first accuracy threshold;

[0974] a second accuracy threshold;

[0975] a first difference threshold;

[0976] a second difference threshold;

[0977] a first throughput threshold;

[0978] a first overhead threshold;

[0979] a second spending threshold;

[0980] First delay threshold;

[0981] third accuracy threshold;

[0982] The third difference threshold;

[0983] a fourth difference threshold;

[0984] a fourth accuracy threshold;

[0985] Fifth accuracy threshold.

[0986] In some embodiments, the first event includes at least one of the following:

[0987] The accuracy of the predicted beam or beam pair is greater than the first accuracy threshold;

[0988] The prediction accuracy of the beam or beam pair for which the L1-RSRP difference is within the first threshold is greater than the second accuracy threshold;

[0989] The first L1-RSRP difference is less than the first difference threshold;

[0990] The second L1-RSRP difference is less than the second difference threshold;

[0991] The throughput is greater than the first throughput threshold;

[0992] The reference signal overhead is less than the first overhead threshold;

[0993] The uplink control information overhead is less than the second overhead threshold;

[0994] The predicted delay is less than the first delay threshold;

[0995] The beam failure probability prediction accuracy is greater than the third accuracy threshold;

[0996] The predicted difference of the radio link quality is less than the third difference threshold;

[0997] The block error rate prediction difference is less than the fourth difference threshold;

[0998] The beam failure time prediction accuracy is greater than the fourth accuracy threshold;

[0999] The new beam prediction accuracy is greater than the fifth accuracy threshold.

[1000] In some embodiments, the second model is an inactive model, and the method further includes:

[1001] receiving an activation instruction sent by the network device;

[1002] activating the second model according to the activation instruction.

[1003] In some embodiments, the first model and / or the second model are deployed on the terminal device side, or the first model and / or the second model are deployed on the network device side.

[1004] In some embodiments, the method further comprises:

[1005] Send capability indication information to the network device, where the capability indication information is used to indicate at least one of the following:

[1006] whether the terminal device supports the first model and / or the second model;

[1007] Whether the terminal device supports performance monitoring of an inactive model deployed on the terminal device side;

[1008] Whether the terminal device supports performance monitoring of the inactivated model deployed on the network device side.

[1009] FIG4A is a flow chart of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG4A , the embodiment of the present disclosure relates to a model performance monitoring method, which can be executed by a network device. The method may include:

[1010] Step S4101: Obtain second information.

[1011] The optional implementation of step S4101 can refer to the optional implementation of step S2101 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[1012] In some embodiments, the network device may receive the second information sent by the terminal device, but is not limited thereto. The network device may also receive the second information sent by other entities.

[1013] In some embodiments, the network device may obtain second information specified by the protocol.

[1014] In some embodiments, the terminal device may perform processing to obtain the second information.

[1015] In some embodiments, the second information may be used to request performance monitoring of the first model.

[1016] Step S4102: Send the first information.

[1017] The optional implementation of step S4102 can refer to the optional implementation of step S2102 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[1018] In some embodiments, the network device may send the first information to the terminal device, but is not limited thereto. The network device may also send the first information to other entities.

[1019] In some embodiments, the first information may be used to determine performance monitoring of the second model.

[1020] In some embodiments, the second model may be the same as the first model or different from the first model.

[1021] Step S4103: Obtain a performance monitoring report.

[1022] The optional implementation of step S4103 can refer to the optional implementation of step S2106 in FIG2A and other related parts in the embodiment involved in FIG2A , which will not be described in detail here.

[1023] In some embodiments, if the first model is deployed on the network device side, the terminal device may not send information other than the measurement data to the network device.

[1024] Step S4104: Determine whether to activate the second model according to the performance monitoring report.

[1025] The optional implementation of step S4104 can refer to the optional implementation of step S2107 in FIG2A and other related parts in the embodiment involved in FIG2A , which will not be described in detail here.

[1026] Step S4105: Send activation instruction.

[1027] The optional implementation of step S4105 can refer to the optional implementation of step S2108 in FIG2A and other related parts in the embodiment involved in FIG2A , which will not be described in detail here.

[1028] In some embodiments, the activation instruction may be used to activate the second model.

[1029] In some embodiments, the network device may send the activation instruction to the terminal device, but is not limited thereto. The network device may also send the activation instruction to other entities.

[1030] In some embodiments, step S4105 may be omitted, or the above functions may be default or by default.

[1031] The method according to the embodiments of the present disclosure may include at least one of steps S4101 to S4105. For example, step S4102 may be implemented as an independent embodiment, steps S4101+S4102 may be implemented as an independent embodiment, steps S4102+S4103 may be implemented as an independent embodiment, and steps S4102+S4103+S4104 may be implemented as an independent embodiment, but the present invention is not limited thereto.

[1032] In some embodiments, steps S4101 to S4105 are all optional steps. For example, steps S4101, S4103, and S4105 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[1033] In some embodiments, reference may be made to other optional implementations described before or after the description corresponding to FIG. 4A .

[1034] FIG4B is a flow chart of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG4B , the embodiment of the present disclosure relates to a model performance monitoring method, which can be executed by a network device. The method may include:

[1035] Step S4201: Send the first message.

[1036] The optional implementation of step S4201 can be found in step S2102 of FIG. 2A , the optional implementation of step S4102 of FIG. 4A , and other related parts in the embodiments involved in FIG. 2A and FIG. 4A , which will not be repeated here.

[1037] Step S4202: Obtain a performance monitoring report.

[1038] The optional implementation of step S4202 can be found in step S2106 of FIG. 2A , the optional implementation of step S4103 of FIG. 4A , and other related parts in the embodiments involved in FIG. 2A and FIG. 4A , which will not be described in detail here.

[1039] Step S4203: Determine whether to activate the second model according to the performance monitoring report.

[1040] The optional implementation of step S4203 can be found in step S2107 of FIG. 2A , the optional implementation of step S4104 of FIG. 4A , and other related parts in the embodiments involved in FIG. 2A and FIG. 4A , which will not be described in detail here.

[1041] Step S4204: Send activation instruction.

[1042] The optional implementation of step S4204 can be found in step S2107 of FIG. 2A , the optional implementation of step S4105 of FIG. 4A , and other related parts in the embodiments involved in FIG. 2A and FIG. 4A , which will not be described in detail here.

[1043] In some embodiments, the above steps are all optional steps.

[1044] In some embodiments, the embodiment shown in FIG. 4B may also be combined with step S4101 in the embodiment shown in FIG. 4A as a new embodiment.

[1045] FIG4C is a flow chart of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG4C , the embodiment of the present disclosure relates to a model performance monitoring method, which can be executed by a network device. The method may include:

[1046] Step S4301: Send the first message.

[1047] The optional implementation of step S4301 can be found in step S2102 of FIG. 2A , the optional implementation of step S4102 of FIG. 4A , and other related parts in the embodiments involved in FIG. 2A and FIG. 4A , which will not be repeated here.

[1048] In some embodiments, the above steps are all optional steps.

[1049] In some embodiments, the embodiment shown in FIG. 4C may also be combined with at least one of steps S4101 , S4103 , and S4104 in the embodiment shown in FIG. 4A as a new embodiment.

[1050] In some embodiments, the method further comprises:

[1051] Second information sent by the terminal device is received, where the second information is used to request performance monitoring of the first model.

[1052] In some embodiments, the second information includes at least one of the following:

[1053] Scheduling request SR for requesting model performance monitoring;

[1054] The first message Msg 1 or message A Msg A of random access RA for requesting model performance monitoring;

[1055] a model identifier of the first model;

[1056] a function identifier corresponding to the first model;

[1057] First configuration information, wherein the first configuration information includes resource configuration for performance monitoring of the first model.

[1058] In some embodiments, the first model is a model for performing beam prediction, and the first configuration information includes at least one of the following:

[1059] a first reference signal resource set including a beam to be measured;

[1060] a second reference signal resource set, wherein the second reference signal resource set includes a beam to be predicted;

[1061] a relationship between the first reference signal resource set and the second reference signal resource set;

[1062] a first performance monitoring duration, where the first performance monitoring duration is the duration during which the terminal device requests performance monitoring of the first model;

[1063] The first sample number is the number of samples requested by the terminal device for performance monitoring of the first model, and the sample number is the number of times the network device sends reference signal resources.

[1064] In some embodiments, the first model is a model for performing spatial beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following:

[1065] The first reference signal resource set is a subset of the second reference signal resource set;

[1066] The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same.

[1067] In some embodiments, the first model is a model for performing time-domain beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following:

[1068] The first reference signal resource set is a subset of the second reference signal resource set;

[1069] The first reference signal resource set is the same as the second reference signal resource set;

[1070] The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same;

[1071] The relationship between a first time and a second time, wherein the first time is a time for performing measurement on the first reference signal resource set, and the second time is a time for performing prediction or measurement on the second reference signal resource set.

[1072] In some embodiments, the method further comprises:

[1073] Sending three messages to the terminal device, wherein the third message is used to request performance monitoring of the second model;

[1074] Receive fourth information sent by the terminal device based on the third information, where the fourth information is used to indicate approval of the request for the third information.

[1075] In some embodiments, the first information includes at least one of the following:

[1076] a model identifier of the second model;

[1077] a function identifier corresponding to the second model;

[1078] a third reference signal resource set, wherein the third reference signal resource set includes a beam to be measured;

[1079] a fourth reference signal resource set, wherein the fourth reference signal resource set includes a beam to be predicted;

[1080] a relationship between the third reference signal resource set and the fourth reference signal resource set;

[1081] a relationship between a third time and a fourth time, the third time being a time for performing measurement on the third reference signal resource set, and the fourth time being a time for performing prediction or measurement on the fourth reference signal resource set;

[1082] a second performance monitoring duration, where the second performance monitoring duration is the duration during which the network device requests performance monitoring of the second model;

[1083] a second sample number, where the second sample number is the sample number requested by the network device for performance monitoring of the second model, and the sample number is the number of times the network device sends a reference signal resource;

[1084] Performance indicators corresponding to performance monitoring;

[1085] Reporting type of performance monitoring report;

[1086] The number of samples included in a performance monitoring report;

[1087] a model management threshold value, wherein the model management threshold value is used to determine whether to activate the second model;

[1088] Physical uplink control channel PUCCH / downlink physical shared channel PUSCH resources corresponding to the performance monitoring report;

[1089] a first event, the first event including a performance indicator for determining that a performance value of the second model satisfies a preset performance condition;

[1090] a first threshold value corresponding to the first event, the first threshold value being used to determine whether the first event is triggered;

[1091] A first offset value corresponding to the first event, where the first offset value is used to determine whether to trigger the first event.

[1092] In some embodiments, the method further comprises:

[1093] Receive a performance monitoring report sent by the terminal device, wherein the performance monitoring report includes at least one of the following:

[1094] Measurement data;

[1095] Forecast data;

[1096] a performance value of the second model, the performance value determined based on the measured data and the predicted data;

[1097] The first event is triggered based on a comparison result between the performance value of the second model and the first threshold value or the first offset value;

[1098] First operation information, where the first operation information is used to instruct a management operation to be performed on the second model, where the management operation includes whether to activate the second model.

[1099] In some embodiments, the measurement data is obtained by the terminal device measuring the beam in the fourth reference signal resource set according to the first information; the measurement data includes at least one of the following:

[1100] The measured layer 1 reference signal received power L1-RSRP of the best K beams in the fourth reference signal resource;

[1101] The measured L1-RSRP of each beam in the fourth reference signal resource set;

[1102] the radio link quality of each of the best K beams measured within a fifth time period;

[1103] The lowest value of the radio link quality of each beam in the best K beams measured within the fifth time period;

[1104] a block error rate (BLER) of each of the best K beams measured within the fifth time period;

[1105] a highest value of the BLER of each beam in the best K beams measured within the fifth time period;

[1106] A probability that the radio link quality of each of the best K beams measured within the fifth time is less than or equal to a preset link quality threshold.

[1107] In some embodiments, the measurement data is obtained by the terminal device measuring the beam in the third reference signal resource set according to the first information; the measurement data includes at least one of the following:

[1108] The measured L1-RSRPs of the best K beams in the third reference signal resource set;

[1109] The measured L1-RSRP of each beam in the third reference signal resource set.

[1110] In some embodiments, the prediction data is obtained by the terminal device according to measurement data corresponding to the third reference signal resource; the prediction data includes at least one of the following:

[1111] The predicted L1-RSRPs of the best K beams in the fourth reference signal resource set;

[1112] The predicted L1-RSRP of each beam in the fourth reference signal resource set;

[1113] predicted wireless link qualities of the best K beams within a fifth time period;

[1114] The predicted lowest value of the radio link qualities of the best K beams within the fifth time;

[1115] The predicted BLERs of the best K beams within the fifth time period;

[1116] The predicted highest BLER value of the best K beams within the fifth time period;

[1117] The predicted probability that the wireless link qualities of the best K beams within the fifth time are less than or equal to a preset link quality threshold.

[1118] In some embodiments, the performance indicator includes at least one of the following:

[1119] Prediction accuracy of beams or beam pairs;

[1120] prediction accuracy of the beam or beam pair for which an L1-RSRP difference is within a first threshold, wherein the L1-RSRP difference is the difference between the measured L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the best measured beam or beam pair;

[1121] L1-RSRP difference, where the L1-RSRP difference is the difference between the measured L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the measured best beam or beam pair, wherein the L1-RSRP difference includes at least one of the following: an average value of the L1-RSRP difference, a cumulative distribution function of the L1-RSRP difference, a proportion of the L1-RSRP difference less than or equal to a second threshold, and a proportion of the L1-RSRP difference greater than a third threshold;

[1122] a predicted L1-RSRP difference, where the predicted L1-RSRP difference is the difference between the predicted L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the best predicted beam or beam pair, wherein the predicted L1-RSRP difference includes at least one of the following: an average value of the predicted L1-RSRP difference, a cumulative distribution function of the predicted L1-RSRP difference, a proportion of the predicted L1-RSRP difference less than or equal to the second threshold, and a proportion of the predicted L1-RSRP difference greater than the third threshold, wherein the predicted L1-RSRP difference is the difference between the predicted L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the best predicted beam or beam pair;

[1123] Terminal device throughput;

[1124] Reference signal overhead;

[1125] Uplink control information overhead;

[1126] Predicted latency;

[1127] Beam failure probability prediction accuracy;

[1128] Wireless link quality prediction variability;

[1129] Block error rate predicts difference rate;

[1130] Beam failure time prediction accuracy;

[1131] New beam prediction accuracy.

[1132] In some embodiments, the performance value includes at least one of the following:

[1133] the accuracy of the predicted beam or beam pair;

[1134] prediction accuracy of the beam or beam pair for which the L1-RSRP difference is within the first threshold;

[1135] a first L1-RSRP difference, where the first L1-RSRP difference is a difference between a measured L1-RSRP of the best predicted beam or beam pair and a measured L1-RSRP of the measured best beam or beam pair;

[1136] a second L1-RSRP difference, where the second L1-RSRP difference is a difference between a predicted L1-RSRP of the best predicted beam or beam pair and a measured L1-RSRP of the best predicted beam or beam pair;

[1137] The throughput of the terminal device for determining the performance value;

[1138] a reference signal overhead for determining the performance value;

[1139] Uplink control information overhead for determining the performance value;

[1140] used to obtain the predicted delay of the predicted data;

[1141] Beam failure probability prediction accuracy;

[1142] A radio link quality prediction difference, where the radio link quality prediction difference is a difference between the predicted radio link quality and the measured radio link quality;

[1143] A block error rate prediction difference, where the block error rate prediction difference is the difference between the predicted block error rate and the measured block error rate;

[1144] Beam failure time prediction accuracy;

[1145] New beam prediction accuracy.

[1146] In some embodiments, the first threshold value includes at least one of the following:

[1147] a first accuracy threshold;

[1148] a second accuracy threshold;

[1149] a first difference threshold;

[1150] a second difference threshold;

[1151] a first throughput threshold;

[1152] a first overhead threshold;

[1153] a second spending threshold;

[1154] First delay threshold;

[1155] third accuracy threshold;

[1156] The third difference threshold;

[1157] a fourth difference threshold;

[1158] a fourth accuracy threshold;

[1159] Fifth accuracy threshold.

[1160] In some embodiments, the first event includes at least one of the following:

[1161] The accuracy of the predicted beam or beam pair is greater than the first accuracy threshold;

[1162] The prediction accuracy of the beam or beam pair for which the L1-RSRP difference is within the first threshold is greater than the second accuracy threshold;

[1163] The first L1-RSRP difference is less than the first difference threshold;

[1164] The second L1-RSRP difference is less than the second difference threshold;

[1165] The throughput is greater than the first throughput threshold;

[1166] The reference signal overhead is less than the first overhead threshold;

[1167] The uplink control information overhead is less than the second overhead threshold;

[1168] The predicted delay is less than the first delay threshold;

[1169] The beam failure probability prediction accuracy is greater than the third accuracy threshold;

[1170] The predicted difference of the radio link quality is less than the third difference threshold;

[1171] The block error rate prediction difference is less than the fourth difference threshold;

[1172] The beam failure time prediction accuracy is greater than the fourth accuracy threshold;

[1173] The new beam prediction accuracy is greater than the fifth accuracy threshold.

[1174] In some embodiments, the method further comprises:

[1175] determining to activate the second model according to the performance monitoring report;

[1176] Activate the second model, or send an activation instruction to the terminal device, where the activation instruction is used by the terminal device to activate the second model.

[1177] In some embodiments, the first model and / or the second model are deployed on the terminal device side, or the first model and / or the second model are deployed on the network device side.

[1178] In some embodiments, the method further comprises:

[1179] Receive capability indication information sent by the terminal device, where the capability indication information is used to indicate at least one of the following:

[1180] whether the terminal device supports the first model and / or the second model;

[1181] Whether the terminal device supports performance monitoring of an inactive model deployed on the terminal device side;

[1182] Whether the terminal device supports performance monitoring of the inactivated model deployed on the network device side.

[1183] FIG5 is an interactive diagram illustrating a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG5 , an embodiment of the present disclosure relates to a model performance monitoring method, which can be executed by a communication system. The method may include:

[1184] Step S5101: The terminal device sends second information to the network device.

[1185] The second information is used to request performance monitoring of the first model.

[1186] The optional implementation of step S5101 can be found in step S2101 of FIG. 2A , the optional implementation of step S4101 of FIG. 4A , and other related parts in the embodiments involved in FIG. 2 and FIG. 4A , which will not be described in detail here.

[1187] Step S5102: The network device sends first information to the terminal device.

[1188] The first information is used to determine performance monitoring of the first model.

[1189] For the optional implementation of step S5102, please refer to step S2102 of Figure 2A, step S3102 of Figure 3A, step S3201 of Figure 3B, step S3301 of Figure 3C, step S3401 of Figure 3D, step S4102 of Figure 4A, step S4201 of Figure 4B, and step S4301 of Figure 4C, as well as other related parts in the embodiments involved in Figures 2, 3A, 3B, 3C, 3D, 4A, 4B, and 4C, which will not be repeated here.

[1190] In some embodiments, the above method may include the method described in the embodiments of the above communication system, terminal equipment, network equipment, etc., which will not be repeated here.

[1191] The above embodiment is described below in three specific ways.

[1192] Method 1: The first model (inactive AI model) can be deployed on the terminal device side, and the terminal device initiates a request to monitor the performance of the first model.

[1193] In some embodiments, the terminal device may initiate a request based on SR or RA to request performance monitoring of an inactive AI model.

[1194] Among them, SR can be an SR dedicated to requesting model performance monitoring, or reuse the existing initial SR used when uplink data needs to be transmitted, or an SR dedicated to Beam failure recovery, or an SR for LBT (listen before talk). RA can be contention-free RA, that is, a dedicated preamble is configured. As long as Msg 1 or Msg A (including the preamble of Msg 1 and the PUSCH of Msg 3) is sent, the base station will know that the terminal device is requesting model performance monitoring. RA can also be contention-based, but the network device needs to know that it is requesting model performance monitoring based on Msg 3 or Msg 3 in Msg A.

[1195] In some embodiments, if the first model is a recognized model, the request initiated by the terminal device may include a model identifier or a function identifier.

[1196] In some embodiments, the request initiated by the terminal device may also include reference signal resource configuration.

[1197] It should be noted that if the reference signal resource configuration for performance monitoring corresponding to the model ID or functionality ID can be uniquely determined, then when the model ID or functionality ID is reported on the terminal device, the reference signal resource configuration may not be included in the request initiated by the terminal device.

[1198] In some embodiments, if the first model is a spatial beam prediction model, the reference signal resource configuration may include: a first reference signal resource set set B, a second reference signal resource set set A, and a mapping relationship between set B and the beams in set A. The mapping relationship may include: set B is a subset of set A. In this case, it is also necessary to specify which reference signal resources in set A set B represents; the beam corresponding to set B is a wide beam, and the beam corresponding to set A is a narrow beam. It is necessary to specify which narrow beams of set A each wide beam of set B can cover.

[1199] In some embodiments, if the first model is a time-domain beam prediction model, the reference signal resource configuration includes, in addition to the information included in the reference signal configuration resource corresponding to the spatial-domain beam prediction model, the relationship between historical measurement times and future prediction times. For example, the future prediction time is between two adjacent measurement times, which is a sliding window mode: that is, the input measurement times all correspond to long-period measurement results, and the predicted beam information of the short period between the two long periods is predicted. The number of input measurement times, the time corresponding to each predicted time, and the period of the measurement time must also be specified. The relationship between historical measurement times and future prediction times can also be a measurement discontinuity mode. For example, the 1st to 4th periods are measurement times, and the 5th to 6th periods are prediction times; the next 7th to 10th periods are measurement times, and the 11th to 12th periods are prediction times. The number of input measurement times, the time corresponding to each predicted time, and the period of the measurement time must also be specified. In addition, the mapping relationship between the beams in set B and set A also includes the case where set B is the same as set A.

[1200] In some embodiments, the request initiated by the terminal device may also include monitoring time and number of monitoring samples. The monitoring time is the time used for model performance monitoring, that is, it tells the network device how long the reference signal resource needs to be sent. The number of monitoring samples is the number of samples used for model performance monitoring, that is, it tells the network device how many times the reference signal resource needs to be sent to meet the required number of samples.

[1201] Method 2: The first model (inactive AI model) can be deployed on the terminal device side, and the network device initiates a request to monitor the performance of the first model.

[1202] In some embodiments, the network device may initiate the request based on at least one of RRC, MAC CE, and DCI.

[1203] In some embodiments, the request initiated by the network device may include at least one of the following:

[1204] Model identification or function identification;

[1205] Reference signal resource configuration information, which may include a reference signal resource identifier and time domain configuration information. The time domain configuration information may include a transmission time, a number of transmissions, a start transmission time, and an end transmission time. The transmission time may include a period and an offset.

[1206] Performance monitoring reporting information.

[1207] It should be noted that, when the first model has been identified, the request initiated by the network device includes a model identifier or a function identifier.

[1208] In some embodiments, the performance monitoring report information may include at least one of the following:

[1209] 1. Periodic reporting or non-periodic reporting;

[1210] 2. How many samples should be reported every time?

[1211] 3. Corresponding PUCCH / PUSCH resource information;

[1212] 4. Performance metric for performance monitoring;

[1213] 5. Used to determine the threshold value for managing a model or function. For example, if the metric is higher than the threshold value, the model or function can be activated.

[1214] 6. If reporting is based on an event or if you decide to associate a model or function, you also need to configure the event and the corresponding threshold or offset value;

[1215] 7. Data content used to calculate performance metrics: The network device can configure the terminal device to report data used to calculate the metric, such as the measured beam information of the best beam in set A actually measured by the terminal device and the predicted beam information of the best beam in set A output by the first model. The network device calculates the metric based on the predicted measured beam information and the predicted beam information. The measured beam information and the predicted beam information each include at least one of the following:

[1216] (1) Best K beam IDs;

[1217] (2) L1-RSRP corresponding to the best K beam IDs;

[1218] (3) L1-RSRP of all beams in set A;

[1219] (4) The probability that the radio link quality of each of the best K beams within a specified time is lower than Qout, and the BLER corresponding to the radio link quality is obtained. If the BLER is higher than Qout, it indicates a beam failure, where the radio link quality includes SNR and SINR. The probability value can be a hard probability, that is, if the BLER is higher than Qout, the probability is 1, otherwise the probability is 0; the probability value can also be a soft probability, for example, if the BLER is higher than Qout and the difference above Qout is greater than a threshold, the probability is 1, otherwise the probability is 80%; if the BLER is lower than Qout and the difference below Qout is greater than a threshold, the probability is 0, otherwise the probability is 30%;

[1220] (4) The radio link quality value or BLER value of each of the best K beams at the specified time, that is, the radio link quality value or BLER value at the time point corresponding to the specified time is predicted;

[1221] (5) The lowest radio link quality or the highest BLER of each beam among the best K beams within the specified time. This means the lowest radio link quality or the highest BLER before and after the time point corresponding to the specified time. If the corresponding BLER is greater than Qout, it indicates that a beam failure has occurred.

[1222] In some embodiments, the network device can initiate a request in two steps. The first step is to request model performance monitoring, which may include the functionality ID or model ID of the corresponding model (if any). If the terminal receives the performance monitoring, the network device then configures reference signal resource configuration information and reporting information in the second step. When the terminal receives the performance monitoring request, it can also inform the terminal of the ID corresponding to the functionality or model or the corresponding condition (including information on set B and set A, as well as applicable environment, etc.) for the performance monitoring.

[1223] In some embodiments, if the request for performance monitoring is initiated by the terminal device, the network device will also send corresponding configuration information. The configuration information may be consistent with the configuration information corresponding to the network device initiating the request (if a two-step process is used, it may be the same as the configuration information sent by the network device in the second step). That is, after receiving the request or configuration information sent by the network device, the terminal measures the corresponding reference signal resources, obtains a report, and sends the report.

[1224] In some embodiments, the report content reported by the terminal includes at least one of the following:

[1225] Data used to calculate performance metrics;

[1226] Calculated performance metric;

[1227] Event ID that triggers the report;

[1228] Model management decisions made by the terminal: for example, the functionality ID or model ID recommended for activation.

[1229] In some embodiments, after reporting, the terminal receives activation signaling sent by the base station, which is used to indicate the activated functionality ID or model ID.

[1230] Among them, the first model (first function) can be used for CSI compression, CSI prediction, or spatial domain beam prediction, or time domain beam prediction, or beam failure information prediction, or positioning. Beam refers to beam, and beam measurement refers to measuring the L1-RSRP and / or L1-SINR of the reference signal including SSB, CSI-RS, and SRS; beam indication refers to the indication of the transmission configuration indication TCI state, and the TCI state includes at least one QCL (quasi Co-location) type, QCL Type A, B, C, D, where type D is the receiving parameter information, commonly known as beam. Type A, B, and C include at least one of the parameters related to Doppler frequency shift, Doppler spread, average delay, and delay spread. For the uplink beam, it refers to spatial relation information or spatial filter parameter or uplink TCI state.

[1231] In some embodiments, the performance metric may include at least one of the following:

[1232] 1. Beam prediction accuracy: beam prediction accuracy of top 1 / K beam(pair),

[1233] Among them, correct beam prediction means: the predicted strongest beam (pair) ID contains the actual strongest beam (pair) ID or the predicted strongest beam (pair) ID is included in the actual strongest N beam (pair) IDs. Among them, the downlink transmit beam ID can be equivalent to the reference signal resource ID, such as the SSB ID or CSI-RS ID or SRS ID. The downlink receive beam ID is the Rx beam ID of the terminal. The beam pair ID is the ID corresponding to the combination of the downlink transmit beam and the downlink receive beam. The strongest means that the L1-RSRP or L1-SINR is the strongest. The accuracy indicator can be based on the accuracy of the output beam information derived multiple times by the model, such as a ratio.

[1234] 2. Beam prediction accuracy within 1 dB: This refers to the accuracy when the difference between the actual L1-RSRP of the predicted optimal beam (pair) and the actual L1-RSRP of the optimal beam is within 1 dB. This accuracy can be measured by the accuracy of the output beam information from multiple model derivations, for example, a ratio.

[1235] 3. L1-RSRP Difference: The difference between the actual L1-RSRP of the predicted best beam (pair) and the actual L1-RSRP of the best beam. For example, the average value is X dB, or the value at the 5% percentile of the cumulative distribution function is Y dB. It can also be the percentage of L1-RSRP differences below or above a threshold.

[1236] 4. Predicted L1-RSRP Difference: The difference between the actual L1-RSRP of the predicted best beam (pair) and the predicted L1-RSRP of the best beam of the predicted best beam (pair). For example, the average value is X dB, or the value at the 5% percentile of the cumulative distribution function is Y dB. It can also be the percentage of Predicted L1-RSRP differences below or above a threshold.

[1237] 5. Average UE throughput, or the UE throughput at the 5% cumulative distribution function: Based on the predicted strongest beam (pair) and the actual strongest beam (pair), the SINR corresponding to the two beams (pairs) is obtained. The capacity is calculated based on the Shannon capacity. The difference in capacity is the metric.

[1238] 6. Reference signal overhead: How much reference signal resources does the model require? The main influencing factors include the size of set B corresponding to the model input and the number of historical measurement times during time domain prediction.

[1239] 7. Uplink control information overhead: If the model is deployed on the network equipment side (NW-side), the measurement results of set B need to be reported to the network, which is the signaling overhead of this report.

[1240] 8. Predict latency.

[1241] 9. Beam failure probability prediction accuracy: If the prediction is a hard probability, then if the beam is predicted to fail and it actually does fail, the prediction is accurate; otherwise, it is inaccurate. If the prediction is a soft probability, then if the probability of the predicted value being higher than Qout is greater than 50% and it actually is higher than Qout, the prediction is accurate; otherwise, it is inaccurate.

[1242] 10. Radio link quality difference: If the predicted radio link quality value is for a specified time, the measured radio link quality value at the specified time is determined, and the difference is the difference between the predicted and measured values. If the predicted minimum radio link quality value within the specified time is for a specified time, the minimum measured radio link quality value within the specified time is determined, and the difference is the difference between the predicted and measured minimum values.

[1243] 11. BLER value difference: If the predicted BLER value at a specified time is used, the measured BLER value at the specified time is determined, and the difference between the predicted value and the measured value is the difference. If the predicted minimum BLER value within the specified time is used, the minimum measured BLER value within the specified time is determined, and the difference between the predicted minimum value and the measured minimum value is the difference.

[1244] 12. Beam failure time difference or beam failure time prediction accuracy: If beam failure is predicted to occur within a specified time, then the difference between the predicted beam failure time and the measured beam failure time is the beam failure time difference; or if the difference between the predicted beam failure time and the measured beam failure time is below a threshold, the prediction is accurate; otherwise, the prediction is inaccurate.

[1245] 13. New Beam Prediction Accuracy: If a beam failure is predicted within the specified time, then the measured beam failure will also occur. If the predicted new beam contains the measured new beam, the prediction is accurate; otherwise, it is inaccurate.

[1246] In some embodiments, an Event may include at least one of the following:

[1247] The beam prediction accuracy is higher than the threshold;

[1248] L1-RSRP difference is lower than the threshold;

[1249] UE throughput is higher than the threshold;

[1250] UCI signaling is below the threshold;

[1251] The RS overhead is lower than the threshold.

[1252] It should be noted that Event can be a condition for each metric to reflect the better performance of the AI ​​function or model.

[1253] Method 3: The first model (inactive AI model) can be deployed on the network device side, and the network device initiates a request to monitor the performance of the first model.

[1254] In some embodiments, the request initiated by the network device may not include the functionality ID or model ID of the first model. For other contents in the request, reference may be made to the contents in the request in the second method, which will not be repeated here.

[1255] In some embodiments, the request initiated by the network device may also include content reported by the terminal, which may include at least one of the following:

[1256] 1. Input of the network-side model

[1257] (1) If the first model is beam information prediction, the input includes the beam (pair) ID and L1-RSRP of set B, or the L1-RSRP corresponding to the beam (pair) of set B. For beam prediction in the time domain, the input also includes measurement beam information of multiple historical measurement times.

[1258] (2) If the first model is a beam failure information prediction model, and the model for beam failure information prediction is the same model as the beam information prediction model, the input may include a specified time; if the model for beam failure information prediction is different from the model for beam information prediction, or if the first model is used only to predict beam failure information, the input may include at least one of the following:

[1259] Including at least one of the L1-RSRP, L1-SINR, and beam (pair) ID (reference signal resource ID and / or Rx beam ID) of the best K beams actually measured or predicted;

[1260] The measured or predicted L1-RSRP or L1-SINR of all beams in set A;

[1261] The measured or predicted L1-RSRP or L1-SINR of the fixed multiple beams in set A;

[1262] At least one of the measured or predicted L1-RSRP, L1-SINR, and beam (pair) ID (reference signal resource ID and / or Rx beam ID) of a plurality of randomly selected beams within set A;

[1263] Specify a time.

[1264] 2. Measurement beam information actually measured by the terminal: This measurement beam information is the same as that in method 2 and will not be repeated here.

[1265] In some embodiments, the network device may inform the terminal device whether to activate the first model (or first function) or may not inform the terminal device.

[1266] In some embodiments, the terminal device can report capability information: reporting support for performance monitoring of non-activated AI models on the terminal device side (or network device side), which can be reported separately (i.e., only one can be supported) or jointly (i.e., both must be supported).

[1267] In some embodiments of the present disclosure, a communication system is provided, which may include a network device and a terminal device, wherein the network device can execute the model performance monitoring method executed by the terminal device in the aforementioned embodiment of the present disclosure; the terminal device can execute the model performance monitoring method executed by the network device in the aforementioned embodiment of the present disclosure.

[1268] The embodiments of the present disclosure also propose an apparatus for implementing any of the above methods. For example, a device is proposed, wherein the device includes a unit or module for implementing each step performed by a terminal in any of the above methods. For another example, another device is proposed, wherein the device includes a unit or module for implementing each step performed by a first network device (e.g., an access network device, a core network function node, a core network device, etc.) in any of the above methods. For another example, another device is proposed, wherein the device includes a unit or module for implementing each step performed by a second network device (e.g., an access network device, a core network function node, a core network device, etc.) in any of the above methods.

[1269] It should be understood that the division of the various units or modules in the above device is merely a division of logical functions. In actual implementation, they may be fully or partially integrated into a physical entity, or they may be physically separated. In addition, the units or modules in the device may be implemented in the form of a processor calling software: for example, the device includes a processor, the processor is connected to a memory, and the memory stores instructions. The processor calls the instructions stored in the memory to implement any of the above methods or implement the functions of the various units or modules of the above device, wherein the processor is, for example, a general-purpose processor, such as a central processing unit (CPU) or a microprocessor, and the memory is a memory within the device or a memory outside the device. Alternatively, the units or modules in the device can be implemented in the form of hardware circuits, and the functions of some or all of the units or modules can be realized by designing the hardware circuits. The above-mentioned hardware circuits can be understood as one or more processors; for example, in one implementation, the above-mentioned hardware circuit is an application-specific integrated circuit (ASIC), and the functions of some or all of the above units or modules are realized by designing the logical relationship of the components in the circuit; for example, in another implementation, the above-mentioned hardware circuit can be realized by a programmable logic device (PLD). Taking a field programmable gate array (FPGA) as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by configuring the configuration file, thereby realizing the functions of some or all of the above units or modules. All units or modules of the above devices can be realized in the form of software called by the processor, or in the form of hardware circuits, or in part by the form of software called by the processor, and the rest by hardware circuits.

[1270] In the embodiments of the present disclosure, the processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), or a digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationship of a hardware circuit. The logical relationship of the above-mentioned hardware circuit is fixed or reconfigurable. For example, the processor is a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units or modules. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc.

[1271] Figure 6A is a schematic diagram of the structure of a terminal device proposed in an embodiment of the present disclosure. As shown in Figure 6A, the terminal device 101 may include a transceiver module 6101. In some embodiments, the transceiver module 6101 is configured to receive first information sent by a network device, and the first information is used to determine performance monitoring of a first model. Optionally, the transceiver module 6101 can be used to perform at least one of the communication steps such as sending and / or receiving (for example, S2101, but not limited to this) performed by the terminal device 101 in any of the above methods, which will not be repeated here.

[1272] In some embodiments, the transceiver module may include a transmitting module and / or a receiving module, and the transmitting module and the receiving module may be separate or integrated. Optionally, the transceiver module may be interchangeable with the transceiver.

[1273] Figure 6B is a schematic diagram of the structure of a network device proposed in an embodiment of the present disclosure. As shown in Figure 6B, the network device 102 may include: at least one of a transceiver module 6201, a processing module 6202, etc. In some embodiments, the transceiver module 6201 is configured to send first information to the terminal device, and the first information is used to determine the performance monitoring of the first model. Optionally, the transceiver module 6201 can be used to perform at least one of the communication steps such as sending and / or receiving (for example, step S4101, but not limited to this) performed by the network device 102 in any of the above methods, which will not be repeated here.

[1274] In some embodiments, the transceiver module may include a transmitting module and / or a receiving module, and the transmitting module and the receiving module may be separate or integrated. Optionally, the transceiver module may be interchangeable with the transceiver.

[1275] In some embodiments, the processing module can be a single module or can include multiple submodules. Optionally, the multiple submodules respectively execute all or part of the steps required to be executed by the processing module. Optionally, the processing module can be interchangeable with the processor.

[1276] Figure 7A is a schematic diagram of the structure of a communication device 7100 proposed in an embodiment of the present disclosure. Communication device 7100 can be a network device (e.g., an access network device, a core network device, etc.), a terminal (e.g., a user device, etc.), a chip, a chip system, or a processor that supports a first device to implement any of the above methods, or a chip, a chip system, or a processor that supports a terminal to implement any of the above methods. Communication device 7100 can be used to implement the methods described in the above method embodiments. For details, please refer to the description of the above method embodiments.

[1277] As shown in FIG7A , the communication device 7100 includes one or more processors 7101. The processor 7101 may be a general-purpose processor or a dedicated processor, for example, a baseband processor or a central processing unit. The baseband processor may be used to process communication protocols and communication data, and the central processing unit may be used to control a communication device (e.g., a base station, a baseband chip, an IoT device, an IoT device chip, a DU or CU, etc.), execute programs, and process program data. The communication device 7100 is used to perform any of the above methods.

[1278] In some embodiments, the communication device 7100 further includes one or more memories 7102 for storing instructions. Optionally, all or part of the memories 7102 may be located outside the communication device 7100.

[1279] In some embodiments, the communication device 7100 further includes one or more transceivers 7103. When the communication device 7100 includes one or more transceivers 7103, the transceiver 7103 performs at least one of the communication steps such as sending and / or receiving in the above method (for example, step S2101 and step S4101, but not limited thereto), and the processor 7101 performs at least one of the other steps (for example, step S2103, but not limited thereto).

[1280] In some embodiments, a transceiver may include a receiver and / or a transmitter. The receiver and transmitter may be separate or integrated. Optionally, the terms transceiver, transceiver unit, transceiver, and transceiver circuit may be used interchangeably; the terms transmitter, transmitting unit, transmitter, and transmitting circuit may be used interchangeably; and the terms receiver, receiving unit, receiver, and receiving circuit may be used interchangeably.

[1281] In some embodiments, the communication device 7100 may include one or more interface circuits. Optionally, the interface circuits are connected to the memory 7102 and may be used to receive signals from the memory 7102 or other devices, or to send signals to the memory 7102 or other devices. For example, the interface circuits may read instructions stored in the memory 7102 and send the instructions to the processor 7101.

[1282] The communication device 7100 described in the above embodiments may be a first device or an IoT device, but the scope of the communication device 7100 described in the present disclosure is not limited thereto, and the structure of the communication device 7100 may not be limited by FIG. 7A . The communication device may be an independent device or may be part of a larger device. For example, the communication device may be: 1) an independent integrated circuit IC, or a chip, or a chip system or subsystem; (2) a collection of one or more ICs, optionally, the above IC collection may also include a storage component for storing data or programs; (3) an ASIC, such as a modem; (4) a module that can be embedded in other devices; (5) a receiver, an IoT device, an intelligent IoT device, a cellular phone, a wireless device, a handheld device, a mobile unit, an in-vehicle device, a first device, a cloud device, an artificial intelligence device, etc.; (6) others, etc.

[1283] 7B is a schematic diagram of the structure of a chip 7200 proposed in an embodiment of the present disclosure. If the communication device 7100 can be a chip or a chip system, please refer to the schematic diagram of the structure of the chip 7200 shown in FIG7B , but the present disclosure is not limited thereto.

[1284] The chip 7200 includes one or more processors 7201 , and the chip 7200 is configured to execute any of the above methods.

[1285] In some embodiments, the chip 7200 further includes one or more interface circuits 7203. Optionally, the interface circuit 7203 is connected to the memory 7202. The interface circuit 7203 can be used to receive signals from the memory 7202 or other devices, and can be used to send signals to the memory 7202 or other devices. For example, the interface circuit 7203 can read instructions stored in the memory 7202 and send the instructions to the processor 7201.

[1286] In some embodiments, the interface circuit 7203 performs at least one of the communication steps such as sending and / or receiving in the above method (for example, step S2101, step S4101, but not limited to this), and the processor 7201 performs at least one of the other steps (for example, step S2103, but not limited to this).

[1287] In some embodiments, terms such as interface circuit, interface, transceiver pin, and transceiver may be used interchangeably.

[1288] In some embodiments, the chip 7200 further includes one or more memories 7202 for storing instructions. Alternatively, all or part of the memory 7202 may be located outside the chip 7200.

[1289] The embodiments of the present disclosure further provide a storage medium having instructions stored thereon. When the instructions are executed on the communication device 7100, the communication device 7100 executes any of the above methods. Optionally, the storage medium is an electronic storage medium. Optionally, the storage medium is a computer-readable storage medium, but is not limited thereto, and may also be a storage medium readable by other devices. Optionally, the storage medium may be a non-transitory storage medium, but is not limited thereto, and may also be a temporary storage medium.

[1290] The present disclosure also provides a program product, which, when executed by the communication device 7100, enables the communication device 7100 to perform any of the above methods. Optionally, the program product may be a computer program product.

[1291] The embodiments of the present disclosure also provide a computer program, which, when executed on a computer, enables the computer to execute any one of the above methods.

Claims

1. A model performance monitoring method, characterized in that: The method comprises: First information sent by a network device is received, where the first information is used to determine performance monitoring of a first model.

2. The method according to claim 1, characterized in that The method further comprises: Sending second information to the network device, where the second information is used to request performance monitoring of the first model.

3. The method according to claim 2, characterized in that The second information includes at least one of the following: Scheduling request SR for requesting model performance monitoring; The first message Msg 1 or message A Msg A of random access RA for requesting model performance monitoring; a model identifier of the first model; A function identifier corresponding to the first model; First configuration information, wherein the first configuration information includes resource configuration for performance monitoring of the first model.

4. The method according to claim 3, characterized in that: The first model is a model for performing beam prediction, and the first configuration information includes at least one of the following: A first reference signal resource set, wherein the first reference signal resource set includes a beam to be measured; a second reference signal resource set, wherein the second reference signal resource set includes a beam to be predicted; A relationship between the first reference signal resource set and the second reference signal resource set; A first performance monitoring duration, where the first performance monitoring duration is the duration during which the terminal device requests to perform performance monitoring on the first model; A first sample number, where the first sample number is the sample number requested by the terminal device for performance monitoring of the first model, and the sample number is the number of times the network device sends reference signal resources.

5. The method according to claim 4, characterized in that The first model is a model for performing spatial beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following: The first reference signal resource set is a subset of the second reference signal resource set; The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same.

6. The method according to claim 4, characterized in that The first model is a model for performing time domain beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following: The first reference signal resource set is a subset of the second reference signal resource set; The first reference signal resource set is the same as the second reference signal resource set; The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same; The relationship between a first time and a second time, the first time is a time for performing measurement on the first reference signal resource set, and the second time is a time for performing prediction or measurement on the second reference signal resource set.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: receiving third information sent by the network device, wherein the third information is used to request performance monitoring of the second model; Fourth information is sent to the network device according to the third information, where the fourth information is used to indicate approval of the request for the third information.

8. The method according to any one of claims 1 to 7, characterized in that: The first information includes at least one of the following: a model identifier of the second model; A function identifier corresponding to the second model; a third reference signal resource set, wherein the third reference signal resource set includes a beam to be measured; a fourth reference signal resource set, wherein the fourth reference signal resource set includes a beam to be predicted; A relationship between the third reference signal resource set and the fourth reference signal resource set; a relationship between a third time and a fourth time, the third time being a time for performing measurement on the third reference signal resource set, and the fourth time being a time for performing prediction or measurement on the fourth reference signal resource set; A second performance monitoring duration, where the second performance monitoring duration is the duration during which the network device requests to perform performance monitoring on the second model; a second sample number, where the second sample number is the sample number requested by the network device for performance monitoring of the second model, and the sample number is the number of times the network device sends a reference signal resource; Performance indicators corresponding to performance monitoring; The reporting type of performance monitoring report; The number of samples included in a performance monitoring report; a model management threshold value, wherein the model management threshold value is used to determine whether to activate the second model; Physical uplink control channel PUCCH / downlink physical shared channel PUSCH resources corresponding to the performance monitoring report; a first event, the first event comprising a performance indicator for determining that a performance value of the second model satisfies a preset performance condition; a first threshold value corresponding to the first event, the first threshold value being used to determine whether to trigger the first event; A first offset value corresponding to the first event, where the first offset value is used to determine whether to trigger the first event.

9. The method according to claim 8, characterized in that The method further comprises: Sending a performance monitoring report to the network device; the performance monitoring report includes at least one of the following: Measurement data; Predicted data; a performance value of the second model, the performance value being determined based on the measured data and the predicted data; The first event is triggered based on a comparison result between the performance value of the second model and the first threshold value or the first offset value; The first operation information is used to indicate a management operation to be performed on the second model, where the management operation includes whether to activate the second model.

10. The method according to claim 9, characterized in that The method further comprises: The beam in the fourth reference signal resource set is measured according to the first information to obtain the measurement data.

11. The method according to claim 9 or 10, characterized in that: The measurement data includes at least one of the following: The measured layer 1 reference signal received power L1-RSRP of the best K beams in the fourth reference signal resource; The measured L1-RSRP of each beam in the fourth reference signal resource set; The radio link quality of each of the best K beams measured within a fifth time; The lowest value of the radio link quality of each beam in the best K beams measured within the fifth time; A block error rate BLER of each beam in the best K beams measured within the fifth time; the highest value of the BLER of each beam in the best K beams measured within the fifth time; The probability that the radio link quality of each of the best K beams measured within the fifth time is less than or equal to a preset link quality threshold.

12. The method according to claim 9, characterized in that The method further comprises: The beam in the third reference signal resource set is measured according to the first information to obtain the measurement data.

13. The method according to claim 9 or 12, characterized in that: The measurement data includes at least one of the following: The measured L1-RSRPs of the best K beams in the third reference signal resource set; The measured L1-RSRP of each beam in the third reference signal resource set.

14. The method according to claim 12 or 13, characterized in that The method further comprises: The predicted data is obtained according to the measurement data corresponding to the third reference signal resource.

15. The method according to any one of claims 9 to 14, characterized in that: The prediction data includes at least one of the following: The predicted L1-RSRPs of the best K beams in the fourth reference signal resource set; The predicted L1-RSRP of each beam in the fourth reference signal resource set; Predicted wireless link qualities of the best K beams within a fifth time period; the predicted minimum value of the radio link quality of the best K beams within the fifth time; The predicted BLER of the best K beams within the fifth time; The predicted highest value of the BLER of the best K beams within the fifth time; The predicted probability that the wireless link quality of the best K beams within the fifth time is less than or equal to a preset link quality threshold.

16. The method according to any one of claims 9 to 15, characterized in that: The performance indicators include at least one of the following: The prediction accuracy of a beam or beam pair; prediction accuracy of the beam or beam pair for which the L1-RSRP difference is within a first threshold, wherein the L1-RSRP difference is the difference between the measured L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the best beam or beam pair; L1-RSRP difference, the L1-RSRP difference is the difference between the measured L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the measured best beam or beam pair, wherein the L1-RSRP difference includes at least one of the following: an average value of the L1-RSRP difference, a cumulative distribution function of the L1-RSRP difference, a proportion of the L1-RSRP difference less than or equal to a second threshold, and a proportion of the L1-RSRP difference greater than a third threshold; A predicted L1-RSRP difference, wherein the predicted L1-RSRP difference is the difference between the predicted L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the best predicted beam or beam pair, wherein the predicted L1-RSRP difference includes at least one of the following: an average value of predicted L1-RSRP differences, a cumulative distribution function of the predicted L1-RSRP differences, a proportion of the predicted L1-RSRP differences less than or equal to the second threshold, and a proportion of the predicted L1-RSRP differences greater than the third threshold, wherein the predicted L1-RSRP difference is the difference between the predicted L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the best predicted beam or beam pair; Terminal equipment throughput; Reference signal overhead; Uplink control information overhead; Predict latency; Beam failure probability prediction accuracy; Wireless link quality prediction difference; Block error rate prediction difference; Beam failure time prediction accuracy; New beam prediction accuracy.

17. The method according to claim 16, characterized in that The performance value includes at least one of the following: the accuracy of the predicted beam or beam pair; prediction accuracy of the beam or beam pair for which the L1-RSRP difference is within the first threshold; a first L1-RSRP difference, where the first L1-RSRP difference is a difference between a measured L1-RSRP of the best predicted beam or beam pair and a measured L1-RSRP of the measured best beam or beam pair; a second L1-RSRP difference, where the second L1-RSRP difference is a difference between a predicted L1-RSRP of the best predicted beam or beam pair and a measured L1-RSRP of the best predicted beam or beam pair; The throughput of the terminal device for determining the performance value; a reference signal overhead for determining the performance value; Uplink control information overhead for determining the performance value; Used to obtain the predicted delay of the predicted data; Beam failure probability prediction accuracy; A wireless link quality prediction difference, where the wireless link quality prediction difference is a difference between a predicted wireless link quality and a measured wireless link quality; A block error rate prediction difference, wherein the block error rate prediction difference is a difference between a predicted block error rate and a measured block error rate; Beam failure time prediction accuracy; New beam prediction accuracy.

18. The method according to claim 16 or 17, characterized in that The first threshold value includes at least one of the following: A first accuracy threshold; a second accuracy threshold; A first difference threshold; A second difference threshold; a first throughput threshold; a first overhead threshold; a second spending threshold; The first delay threshold; The third accuracy threshold; The third difference threshold; a fourth difference threshold; fourth accuracy threshold; Fifth accuracy threshold.

19. The method according to claim 18, characterized in that The first event includes at least one of the following: The accuracy of the predicted beam or beam pair is greater than the first accuracy threshold; The prediction accuracy of the beam or beam pair for which the L1-RSRP difference is within the first threshold is greater than the second accuracy threshold; The first L1-RSRP difference is less than the first difference threshold; The second L1-RSRP difference is less than the second difference threshold; The throughput is greater than the first throughput threshold; The reference signal overhead is less than the first overhead threshold; The uplink control information overhead is less than the second overhead threshold; The predicted delay is less than the first delay threshold; The beam failure probability prediction accuracy is greater than the third accuracy threshold; The predicted difference of the wireless link quality is less than the third difference threshold; The block error rate prediction difference is less than the fourth difference threshold; The beam failure time prediction accuracy is greater than the fourth accuracy threshold; The new beam prediction accuracy is greater than the fifth accuracy threshold.

20. The method according to any one of claims 8 to 19, characterized in that: The second model is a non-activated model, and the method further includes: Receiving an activation instruction sent by the network device; The second model is activated according to the activation instruction.

21. The method according to any one of claims 8 to 20, characterized in that: The first model and / or the second model are deployed on the terminal device side, or the first model and / or the second model are deployed on the network device side.

22. The method according to any one of claims 8 to 21, characterized in that: The method further comprises: Send capability indication information to the network device, where the capability indication information is used to indicate at least one of the following: Whether the terminal device supports the first model and / or the second model; Whether the terminal device supports performance monitoring of an inactive model deployed on the terminal device side; Whether the terminal device supports performance monitoring of the inactivated model deployed on the network device side.

23. A model performance monitoring method, characterized in that: The method comprises: First information is sent to a terminal device, where the first information is used to determine performance monitoring of a first model.

24. The method according to claim 23, characterized in that The method further comprises: Second information sent by the terminal device is received, where the second information is used to request performance monitoring of the first model.

25. The method according to claim 24, characterized in that The second information includes at least one of the following: Scheduling request SR for requesting model performance monitoring; The first message Msg 1 or message A Msg A of random access RA for requesting model performance monitoring; a model identifier of the first model; A function identifier corresponding to the first model; First configuration information, wherein the first configuration information includes resource configuration for performance monitoring of the first model.

26. The method according to claim 25, characterized in that The first model is a model for performing beam prediction, and the first configuration information includes at least one of the following: A first reference signal resource set, wherein the first reference signal resource set includes a beam to be measured; a second reference signal resource set, wherein the second reference signal resource set includes a beam to be predicted; A relationship between the first reference signal resource set and the second reference signal resource set; a first performance monitoring duration, where the first performance monitoring duration is the duration during which the terminal device requests to perform performance monitoring on the first model; The first sample number is the number of samples requested by the terminal device for performance monitoring of the first model, and the sample number is the number of times the network device sends reference signal resources.

27. The method according to claim 26, characterized in that The first model is a model for performing spatial beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following: The first reference signal resource set is a subset of the second reference signal resource set; The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same.

28. The method according to claim 26, characterized in that The first model is a model for performing time domain beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following: The first reference signal resource set is a subset of the second reference signal resource set; The first reference signal resource set is the same as the second reference signal resource set; The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same; The relationship between a first time and a second time, the first time is a time for performing measurement on the first reference signal resource set, and the second time is a time for performing prediction or measurement on the second reference signal resource set.

29. The method according to any one of claims 23 to 28, characterized in that: The method further comprises: Sending third information to the terminal device, wherein the third information is used to request performance monitoring of the second model; Receive fourth information sent by the terminal device based on the third information, where the fourth information is used to indicate approval of the request for the third information.

30. The method according to any one of claims 23 to 29, characterized in that: The first information includes at least one of the following: a model identifier of the second model; A function identifier corresponding to the second model; a third reference signal resource set, wherein the third reference signal resource set includes a beam to be measured; a fourth reference signal resource set, wherein the fourth reference signal resource set includes a beam to be predicted; A relationship between the third reference signal resource set and the fourth reference signal resource set; a relationship between a third time and a fourth time, the third time being a time for performing measurement on the third reference signal resource set, and the fourth time being a time for performing prediction or measurement on the fourth reference signal resource set; A second performance monitoring duration, where the second performance monitoring duration is the duration during which the network device requests to perform performance monitoring on the second model; a second sample number, where the second sample number is the sample number requested by the network device for performance monitoring of the second model, and the sample number is the number of times the network device sends a reference signal resource; Performance indicators corresponding to performance monitoring; The reporting type of performance monitoring report; The number of samples included in a performance monitoring report; a model management threshold value, wherein the model management threshold value is used to determine whether to activate the second model; Physical uplink control channel PUCCH / downlink physical shared channel PUSCH resources corresponding to the performance monitoring report; a first event, the first event comprising a performance indicator for determining that a performance value of the second model satisfies a preset performance condition; a first threshold value corresponding to the first event, the first threshold value being used to determine whether to trigger the first event; A first offset value corresponding to the first event, where the first offset value is used to determine whether to trigger the first event.

31. The method according to claim 30, characterized in that The method further comprises: Receive a performance monitoring report sent by the terminal device; the performance monitoring report includes at least one of the following: Measurement data; Predicted data; a performance value of the second model, the performance value being determined based on the measured data and the predicted data; The first event is triggered based on a comparison result between the performance value of the second model and the first threshold value or the first offset value; The first operation information is used to indicate whether to perform a management operation on the second model. Live the second model.

32. The method according to claim 31, characterized in that The measurement data is obtained by the terminal device measuring the beam in the fourth reference signal resource set according to the first information; the measurement data includes at least one of the following: The measured layer 1 reference signal received power L1-RSRP of the best K beams in the fourth reference signal resource; The measured L1-RSRP of each beam in the fourth reference signal resource set; The radio link quality of each of the best K beams measured within a fifth time; The lowest value of the radio link quality of each beam in the best K beams measured within the fifth time; A block error rate BLER of each beam in the best K beams measured within the fifth time; the highest value of the BLER of each beam in the best K beams measured within the fifth time; The probability that the radio link quality of each of the best K beams measured within the fifth time is less than or equal to a preset link quality threshold.

33. The method according to claim 31, characterized in that The measurement data is obtained by the terminal device measuring the beam in the third reference signal resource set according to the first information; the measurement data includes at least one of the following: The measured L1-RSRPs of the best K beams in the third reference signal resource set; The measured L1-RSRP of each beam in the third reference signal resource set.

34. The method according to any one of claims 31 to 33, characterized in that: The prediction data is obtained by the terminal device according to the measurement data corresponding to the third reference signal resource; the prediction data includes at least one of the following: The predicted L1-RSRPs of the best K beams in the fourth reference signal resource set; The predicted L1-RSRP of each beam in the fourth reference signal resource set; Predicted wireless link qualities of the best K beams within a fifth time period; the predicted minimum value of the radio link quality of the best K beams within the fifth time; The predicted BLER of the best K beams within the fifth time; The predicted highest value of the BLER of the best K beams within the fifth time; The predicted probability that the wireless link quality of the best K beams within the fifth time is less than or equal to a preset link quality threshold.

35. The method according to any one of claims 31 to 34, characterized in that The performance indicators include at least one of the following: The prediction accuracy of a beam or beam pair; prediction accuracy of the beam or beam pair for which the L1-RSRP difference is within a first threshold, wherein the L1-RSRP difference is the difference between the measured L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the best beam or beam pair; L1-RSRP difference, the L1-RSRP difference is the difference between the measured L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the measured best beam or beam pair, wherein the L1-RSRP difference includes at least one of the following: an average value of the L1-RSRP difference, a cumulative distribution function of the L1-RSRP difference, a proportion of the L1-RSRP difference less than or equal to a second threshold, and a proportion of the L1-RSRP difference greater than a third threshold; A predicted L1-RSRP difference, wherein the predicted L1-RSRP difference is the difference between the predicted L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the best predicted beam or beam pair, wherein the predicted L1-RSRP difference includes at least one of the following: an average value of predicted L1-RSRP differences, a cumulative distribution function of the predicted L1-RSRP differences, a proportion of the predicted L1-RSRP differences less than or equal to the second threshold, and a proportion of the predicted L1-RSRP differences greater than the third threshold, wherein the predicted L1-RSRP difference is the difference between the predicted L1-RSRP of the best predicted beam or beam pair and the measured L1-RSRP of the best predicted beam or beam pair; Terminal equipment throughput; Reference signal overhead; Uplink control information overhead; Predict latency; Beam failure probability prediction accuracy; Wireless link quality prediction difference; Block error rate predicts difference rate; Beam failure time prediction accuracy; New beam prediction accuracy.

36. The method according to claim 35, characterized in that The performance value includes at least one of the following: the accuracy of the predicted beam or beam pair; prediction accuracy of the beam or beam pair for which the L1-RSRP difference is within the first threshold; a first L1-RSRP difference, where the first L1-RSRP difference is a difference between a measured L1-RSRP of the best predicted beam or beam pair and a measured L1-RSRP of the measured best beam or beam pair; a second L1-RSRP difference, where the second L1-RSRP difference is a difference between a predicted L1-RSRP of the best predicted beam or beam pair and a measured L1-RSRP of the best predicted beam or beam pair; The throughput of the terminal device for determining the performance value; a reference signal overhead for determining the performance value; Uplink control information overhead for determining the performance value; A predicted delay for obtaining the predicted data; Beam failure probability prediction accuracy; A wireless link quality prediction difference, where the wireless link quality prediction difference is a difference between a predicted wireless link quality and a measured wireless link quality; A block error rate prediction difference, wherein the block error rate prediction difference is a difference between a predicted block error rate and a measured block error rate; Beam failure time prediction accuracy; New beam prediction accuracy.

37. The method according to claim 35 or 36, characterized in that The first threshold value includes at least one of the following: A first accuracy threshold; a second accuracy threshold; A first difference threshold; A second difference threshold; a first throughput threshold; a first overhead threshold; a second spending threshold; The first delay threshold; The third accuracy threshold; The third difference threshold; a fourth difference threshold; fourth accuracy threshold; Fifth accuracy threshold.

38. The method according to claim 37, characterized in that The first event includes at least one of the following: The accuracy of the predicted beam or beam pair is greater than the first accuracy threshold; The prediction accuracy of the beam or beam pair for which the L1-RSRP difference is within the first threshold is greater than the second accuracy threshold; The first L1-RSRP difference is less than the first difference threshold; The second L1-RSRP difference is less than the second difference threshold; The throughput is greater than the first throughput threshold; The reference signal overhead is less than the first overhead threshold; The uplink control information overhead is less than the second overhead threshold; The predicted delay is less than the first delay threshold; The beam failure probability prediction accuracy is greater than the third accuracy threshold; The predicted difference of the wireless link quality is less than the third difference threshold; The block error rate prediction difference is less than the fourth difference threshold; The beam failure time prediction accuracy is greater than the fourth accuracy threshold; The new beam prediction accuracy is greater than the fifth accuracy threshold.

39. The method according to claim 38, characterized in that The method further comprises: Determining to activate the second model according to the performance monitoring report; Activate the second model, or send an activation instruction to the terminal device, where the activation instruction is used by the terminal device to activate the second model.

40. The method according to any one of claims 30 to 39, characterized in that: The first model and / or the second model are deployed on the terminal device side, or the first model and / or the second model are deployed on the network device side.

41. The method according to any one of claims 30 to 40, characterized in that: The method further comprises: Receive capability indication information sent by the terminal device, where the capability indication information is used to indicate at least one of the following: Whether the terminal device supports the first model and / or the second model; Whether the terminal device supports performance monitoring of an inactive model deployed on the terminal device side; Whether the terminal device supports performance monitoring of the inactivated model deployed on the network device side.

42. A terminal device, characterized in that: include: The transceiver module is configured to receive first information sent by a network device, where the first information is used to determine performance monitoring of a first model.

43. A network device, characterized in that: include: The transceiver module is configured to send first information to the terminal device, where the first information is used to determine performance monitoring of the first model.

44. A terminal device, characterized in that: include: one or more processors; Wherein, the terminal device is used to execute the model performance monitoring method described in any one of claims 1 to 22.

45. A network device, characterized in that: include: one or more processors; Wherein, the network device is used to execute the model performance monitoring method described in any one of claims 23 to 41.

46. ​​A storage medium storing instructions, characterized in that: When the instruction is executed on a terminal device, the terminal device executes the model performance monitoring method as described in any one of claims 1 to 22, or when the instruction is executed on a network device, the network device executes the model performance monitoring method as described in any one of claims 23 to 41.

Images