Performance monitoring method and communication device

Abstract

The present disclosure provides a performance monitoring method and a communication device. The performance monitoring method comprises: a terminal determines whether a reporting condition for monitoring-related information of a first model is satisfied; and when the reporting condition is satisfied, the terminal reports the monitoring-related information of the first model to a network device.

Description

Performance monitoring method and communication device

[0001] This disclosure claims priority to the Chinese patent application filed with the China Patent Office on August 25, 2023, with application number 202311079134.2 and application name “Model Monitoring Method and Communication Device,” the entire contents of which are incorporated by reference into this disclosure. Technical Field

[0002] The present disclosure relates to the field of communication technology, and in particular to a performance monitoring method and communication equipment. Background Art

[0003] To reduce the measurement overhead of communication systems, solutions using artificial intelligence (AI) or machine learning (ML) have been proposed. These solutions use AI / ML models to make predictions based on a small number of measurement results or historical measurement results. To ensure the performance of AI / ML applications in communication systems, model monitoring of the AI / ML models is required. During model monitoring, terminals are required to report measurement and / or prediction results to network devices. Currently, there is no clear solution for reporting model monitoring information during the model monitoring process.

[0004] Summary of the Invention

[0005] The embodiments of the present disclosure provide a performance monitoring method and a communication device to solve the problem of how to report performance monitoring related information during the performance monitoring process.

[0006] The present disclosure provides a performance monitoring method, including:

[0007] The terminal determines whether a reporting condition for monitoring-related information of the first model is met;

[0008] When the reporting condition is met, the terminal reports the monitoring-related information of the first model to the network device.

[0009] In some embodiments, the method further comprises:

[0010] The terminal determines, based on first information sent by the network device, a reporting condition for monitoring-related information of the first model, where the first information is used to indicate at least one of the following:

[0011] The monitoring and reporting period of the first model;

[0012] The monitoring start time of the first model;

[0013] The prediction result of the first model needs to meet the first preset condition.

[0014] In some embodiments, the method further comprises:

[0015] The terminal determines a reporting condition for monitoring-related information of the first model according to predefined performance monitoring-related parameters, where the performance monitoring-related parameters include at least one of the following:

[0016] The monitoring and reporting period of the first model;

[0017] The monitoring start time of the first model;

[0018] The prediction result of the first model needs to meet the first preset condition.

[0019] In some embodiments, the reporting condition includes at least one of the following:

[0020] The reporting time of the monitoring reporting cycle of the first model arrives;

[0021] The terminal receives reporting trigger information from the network device, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model;

[0022] The terminal detects that the prediction result of the first model meets a first preset condition.

[0023] In some embodiments, the first preset condition includes at least one of the following:

[0024] The number of times the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal;

[0025] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, where the actual optimal K beams are determined based on a measurement result of the terminal;

[0026] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

[0027] In some embodiments, the monitoring-related information reported by the terminal includes at least one of the following:

[0028] measurement results of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1;

[0029] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0030] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0031] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0032] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0033] the difference, the absolute value of the difference, or the average of the absolute values ​​of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the actual layer 1 reference signal received power values ​​of the optimal K beams;

[0034] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the actual layer 1 reference signal received power value of the optimal beam, or an absolute value of the difference, or an average of the absolute values ​​of the difference;

[0035] Identification information of the first model.

[0036] In some embodiments, the N measurements are all measurements of the terminal; or, the N measurements selected by the terminal from all measurements; or, the N measurements corresponding to the N predictions selected by the terminal from all predictions of the first model.

[0037] In some embodiments, the terminal indicates the measurement order of the N measurements in all measurements through a bitmap, or indicates measurement identification information of the N measurements.

[0038] In some embodiments, the N measurements screened by the terminal from all measurements include at least one of the following:

[0039] The first measurement within the monitoring and reporting period of the first model;

[0040] The first measurement from the monitoring start time of the first model;

[0041] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0042] a measurement in which the difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold;

[0043] a measurement in which the difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than a fifth threshold;

[0044] a measurement in which the difference between the average value of the measurement results and the average value of the measurement results of the previous measurement is greater than a sixth threshold;

[0045] a measurement in which the difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than a seventh threshold;

[0046] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0047] In some embodiments, the N predictions selected by the terminal from all predictions of the first model include at least one of the following:

[0048] The first forecast within the monitoring and reporting period;

[0049] The first prediction from the monitoring start time;

[0050] A forecast whose predicted result is different from the predicted result of the previous forecast;

[0051] The difference between the maximum value of the predicted result and the maximum value of the predicted result of the previous prediction is greater than the ninth threshold;

[0052] The difference between the minimum value of the predicted result and the minimum value of the predicted result of the previous prediction is greater than the tenth threshold;

[0053] The difference between the average value of the predicted results and the average value of the predicted results of the previous prediction is greater than the eleventh threshold;

[0054] The difference between the standard deviation of the predicted result and the standard deviation of the predicted result of the previous prediction is greater than the twelfth threshold;

[0055] The difference between the prediction result variance and the prediction result variance of the previous prediction is greater than the prediction of the thirteenth threshold.

[0056] In some embodiments, the terminal reports the monitoring-related information of the first model to the network device, including:

[0057] The terminal reports the monitoring-related information of the first model to the network device through the first resource;

[0058] The first resource includes any one of the following:

[0059] Resources configured by the network device for reporting monitoring-related information of the first model;

[0060] Physical uplink shared channel PUSCH resources already available to the terminal;

[0061] The terminal requests PUSCH resources through a scheduling request SR.

[0062] In some embodiments, the method further comprises:

[0063] When the terminal stores the measurement result of the Lth measurement, if the required storage space exceeds the remaining storage space of the terminal, the terminal discards the earliest stored measurement result and stores the measurement result of the Lth measurement.

[0064] The present disclosure provides a performance monitoring method, including:

[0065] The network device receives monitoring related information of the first model reported by the terminal;

[0066] The network device performs performance monitoring of the first model based on the monitoring-related information.

[0067] In some embodiments, the method further comprises:

[0068] The network device sends first information to the terminal, where the first information is used to indicate at least one of the following:

[0069] The monitoring and reporting period of the first model;

[0070] The monitoring start time of the first model;

[0071] The prediction result of the first model needs to meet the first preset condition.

[0072] In some embodiments, the reporting condition is determined by predefined performance monitoring related parameters, and the performance monitoring related parameters include at least one of the following:

[0073] The monitoring and reporting period of the first model;

[0074] The monitoring start time of the first model;

[0075] The prediction result of the first model needs to meet the first preset condition.

[0076] In some embodiments, the method further comprises:

[0077] The network device sends reporting trigger information to the terminal, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model.

[0078] In some embodiments, the network device sending the reporting trigger information to the terminal includes:

[0079] When the network device monitors that the system performance indicator meets the second preset condition, the network device sends reporting trigger information to the terminal.

[0080] In some embodiments, the first preset condition includes at least one of the following:

[0081] The number of times the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal;

[0082] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, where the actual optimal K beams are determined based on a measurement result of the terminal;

[0083] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

[0084] In some embodiments, the network device performs performance monitoring of the first model based on the monitoring-related information, including:

[0085] The network device calculates the monitoring index of the first model based on the monitoring-related information.

[0086] In some embodiments, when the first model is deployed on the network device, the monitoring-related information includes measurement results of the terminal performing N measurements on the reference signal, where N is an integer greater than 1;

[0087] The network device calculates the monitoring index of the first model based on the monitoring-related information, including:

[0088] The network device selects N predictions corresponding to the N measurements from all predictions of the first model;

[0089] The network device calculates the monitoring indicator of the first model according to the prediction results of the N predictions and the measurement results of the N measurements;

[0090] or,

[0091] The network device calculates the monitoring index of the first model based on the monitoring-related information, including:

[0092] The network device selects M measurements from the N measurements, where M is an integer greater than 1;

[0093] The network device selects M predictions corresponding to the M measurements from all predictions of the first model;

[0094] The network device calculates the monitoring indicator of the first model according to the prediction results of the M predictions and the measurement results of the M measurements.

[0095] In some embodiments, the M measurements filtered out by the network device from the N measurements include at least one of the following:

[0096] The first measurement within the monitoring and reporting period;

[0097] The first measurement from the monitoring start time;

[0098] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0099] a measurement in which the difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold;

[0100] a measurement in which the difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than a fifth threshold;

[0101] a measurement in which the difference between the average value of the measurement results and the average value of the measurement results of the previous measurement is greater than a sixth threshold;

[0102] a measurement in which the difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than a seventh threshold;

[0103] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0104] In some embodiments, when the first model is deployed on the terminal, the monitoring-related information includes at least one of the following:

[0105] measurement results of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1;

[0106] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0107] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0108] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0109] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0110] the difference, the absolute value of the difference, or the average of the absolute values ​​of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the actual layer 1 reference signal received power values ​​of the optimal K beams;

[0111] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the actual layer 1 reference signal received power value of the optimal beam, or an absolute value of the difference, or an average of the absolute values ​​of the difference;

[0112] Identification information of the first model.

[0113] In some embodiments, the network device receives the monitoring-related information of the first model reported by the terminal, including:

[0114] The network device receives, through a first resource, monitoring-related information of the first model reported by the terminal;

[0115] The first resource includes any one of the following:

[0116] Resources configured by the network device for reporting monitoring-related information of the first model;

[0117] Physical uplink shared channel PUSCH resources already available to the terminal;

[0118] The terminal requests PUSCH resources through a scheduling request SR.

[0119] In some embodiments, the method further comprises:

[0120] The network device performs a first process according to the monitoring indicator of the first model, where the first process includes any one of the following:

[0121] activating the first model;

[0122] Deactivate the first model.

[0123] The present disclosure also provides a terminal, including: a memory, a transceiver, and a processor, wherein:

[0124] A memory for storing a computer program; a transceiver for transmitting and receiving data under the control of the processor; and a processor for reading the computer program in the memory and performing the following operations:

[0125] Determining whether a reporting condition for monitoring-related information of the first model is met;

[0126] When the reporting condition is met, the monitoring-related information of the first model is reported to the network device.

[0127] In some embodiments, the processor is configured to read the computer program in the memory and perform the following operations:

[0128] Determine, based on first information sent by the network device, a reporting condition for monitoring-related information of the first model, where the first information is used to indicate at least one of the following:

[0129] The monitoring and reporting period of the first model;

[0130] The monitoring start time of the first model;

[0131] The prediction result of the first model needs to meet the first preset condition.

[0132] In some embodiments, the processor is configured to read the computer program in the memory and perform the following operations:

[0133] Determine a reporting condition for monitoring-related information of the first model based on predefined performance monitoring-related parameters, where the performance monitoring-related parameters include at least one of the following:

[0134] The monitoring and reporting period of the first model;

[0135] The monitoring start time of the first model;

[0136] The prediction result of the first model needs to meet the first preset condition.

[0137] In some embodiments, the reporting condition includes at least one of the following:

[0138] The reporting time of the monitoring reporting cycle of the first model arrives;

[0139] The terminal receives reporting trigger information from the network device, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model;

[0140] The terminal detects that the prediction result of the first model meets a first preset condition.

[0141] In some embodiments, the first preset condition includes at least one of the following:

[0142] The number of times the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal;

[0143] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, where the actual optimal K beams are determined based on a measurement result of the terminal;

[0144] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

[0145] In some embodiments, the monitoring-related information reported by the terminal includes at least one of the following:

[0146] measurement results of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1;

[0147] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0148] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0149] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0150] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0151] the difference, the absolute value of the difference, or the average of the absolute values ​​of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the actual layer 1 reference signal received power values ​​of the optimal K beams;

[0152] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the actual layer 1 reference signal received power value of the optimal beam, or an absolute value of the difference, or an average of the absolute values ​​of the difference;

[0153] Identification information of the first model.

[0154] In some embodiments, the N measurements are all measurements; or, N measurements selected from all measurements; or, N measurements corresponding to N predictions selected from all predictions of the first model.

[0155] In some embodiments, the processor is configured to read the computer program in the memory and perform the following operations:

[0156] The measurement order of the N measurements in all measurements is indicated by a bitmap, or the measurement identification information of the N measurements is indicated.

[0157] In some embodiments, the N measurements selected from all measurements include at least one of the following:

[0158] The first measurement within the monitoring and reporting period of the first model;

[0159] The first measurement from the monitoring start time of the first model;

[0160] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0161] a measurement in which the difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold;

[0162] a measurement in which the difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than a fifth threshold;

[0163] a measurement in which the difference between the average value of the measurement results and the average value of the measurement results of the previous measurement is greater than a sixth threshold;

[0164] a measurement in which the difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than a seventh threshold;

[0165] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0166] In some embodiments, the N predictions selected from all predictions of the first model include at least one of the following:

[0167] The first forecast within the monitoring and reporting period;

[0168] The first prediction from the monitoring start time;

[0169] A forecast whose predicted result is different from the predicted result of the previous forecast;

[0170] The difference between the maximum value of the predicted result and the maximum value of the predicted result of the previous prediction is greater than the ninth threshold;

[0171] The difference between the minimum value of the predicted result and the minimum value of the predicted result of the previous prediction is greater than the tenth threshold;

[0172] The difference between the average value of the predicted results and the average value of the predicted results of the previous prediction is greater than the eleventh threshold;

[0173] The difference between the standard deviation of the predicted result and the standard deviation of the predicted result of the previous prediction is greater than the twelfth threshold;

[0174] The difference between the prediction result variance and the prediction result variance of the previous prediction is greater than the prediction of the thirteenth threshold.

[0175] In some embodiments, the processor is configured to read the computer program in the memory and perform the following operations:

[0176] reporting the monitoring-related information of the first model to the network device through the first resource;

[0177] The first resource includes any one of the following:

[0178] Resources configured by the network device for reporting monitoring-related information of the first model;

[0179] Physical uplink shared channel PUSCH resources already available to the terminal;

[0180] The terminal requests PUSCH resources through a scheduling request SR.

[0181] In some embodiments, the processor is configured to read the computer program in the memory and perform the following operations:

[0182] When storing the measurement result of the Lth measurement, if the required storage space exceeds the remaining storage space of the terminal, the earliest stored measurement result is discarded and the measurement result of the Lth measurement is stored.

[0183] The present disclosure also provides a network device, including a memory, a transceiver, and a processor, wherein:

[0184] A memory for storing a computer program; a transceiver for transmitting and receiving data under the control of the processor; and a processor for reading the computer program in the memory and performing the following operations:

[0185] receiving monitoring-related information of the first model reported by the terminal;

[0186] Based on the monitoring-related information, performance monitoring of the first model is performed.

[0187] In some embodiments, the processor is configured to read the computer program in the memory and perform the following operations:

[0188] Sending first information to the terminal, where the first information is used to indicate at least one of the following:

[0189] The monitoring and reporting period of the first model;

[0190] The monitoring start time of the first model;

[0191] The prediction result of the first model needs to meet the first preset condition.

[0192] In some embodiments, the reporting condition is determined by predefined performance monitoring related parameters, and the performance monitoring related parameters include at least one of the following:

[0193] The monitoring and reporting period of the first model;

[0194] The monitoring start time of the first model;

[0195] The prediction result of the first model needs to meet the first preset condition.

[0196] In some embodiments, the method further comprises:

[0197] The network device sends reporting trigger information to the terminal, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model.

[0198] In some embodiments, the processor is configured to read the computer program in the memory and perform the following operations:

[0199] When it is monitored that the system performance indicator meets the second preset condition, reporting trigger information is sent to the terminal.

[0200] In some embodiments, the first preset condition includes at least one of the following:

[0201] The number of times the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal;

[0202] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, where the actual optimal K beams are determined based on a measurement result of the terminal;

[0203] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

[0204] In some embodiments, the processor is configured to read the computer program in the memory and perform the following operations:

[0205] Based on the monitoring-related information, a monitoring indicator of the first model is calculated.

[0206] In some embodiments, when the first model is deployed on the network device, the monitoring-related information includes measurement results of the terminal performing N measurements on the reference signal, where N is an integer greater than 1;

[0207] A processor is configured to read the computer program in the memory and perform the following operations:

[0208] Filtering N predictions corresponding to the N measurements from all predictions of the first model;

[0209] Calculating a monitoring indicator of the first model according to the prediction results of the N predictions and the measurement results of the N measurements;

[0210] or,

[0211] Filtering M measurements from the N measurements, where M is an integer greater than 1;

[0212] Filtering M predictions corresponding to the M measurements from all predictions of the first model;

[0213] The monitoring index of the first model is calculated according to the prediction results of the M predictions and the measurement results of the M measurements.

[0214] In some embodiments, the M measurements selected from the N measurements include at least one of the following:

[0215] The first measurement within the monitoring and reporting period;

[0216] The first measurement from the monitoring start time;

[0217] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0218] a measurement in which the difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold;

[0219] a measurement in which the difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than a fifth threshold;

[0220] a measurement in which the difference between the average value of the measurement results and the average value of the measurement results of the previous measurement is greater than a sixth threshold;

[0221] a measurement in which the difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than a seventh threshold;

[0222] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0223] In some embodiments, when the first model is deployed on the terminal, the monitoring-related information includes at least one of the following:

[0224] measurement results of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1;

[0225] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0226] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0227] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0228] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0229] the difference, the absolute value of the difference, or the average of the absolute values ​​of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the actual layer 1 reference signal received power values ​​of the optimal K beams;

[0230] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the actual layer 1 reference signal received power value of the optimal beam, or an absolute value of the difference, or an average of the absolute values ​​of the difference;

[0231] Identification information of the first model.

[0232] In some embodiments, the processor is configured to read the computer program in the memory and perform the following operations:

[0233] receiving, through a first resource, monitoring-related information of the first model reported by the terminal;

[0234] The first resource includes any one of the following:

[0235] Resources configured by the network device for reporting monitoring-related information of the first model;

[0236] Physical uplink shared channel PUSCH resources already available to the terminal;

[0237] The terminal requests PUSCH resources through a scheduling request SR.

[0238] In some embodiments, the processor is configured to read the computer program in the memory and perform the following operations:

[0239] Performing a first process according to the monitoring indicator of the first model, where the first process includes any one of the following:

[0240] activating the first model;

[0241] Deactivate the first model.

[0242] The present disclosure also provides a terminal, including:

[0243] A first determining module, configured to determine whether a reporting condition for monitoring-related information of a first model is met;

[0244] The reporting module is used to report the monitoring related information of the first model to the network device when the reporting conditions are met.

[0245] In some embodiments, the terminal further includes:

[0246] A second determining module is configured to determine a reporting condition for monitoring-related information of the first model based on first information sent by the network device, where the first information is used to indicate at least one of the following:

[0247] The monitoring and reporting period of the first model;

[0248] The monitoring start time of the first model;

[0249] The prediction result of the first model needs to meet the first preset condition.

[0250] In some embodiments, the terminal further includes:

[0251] A third determining module is configured to determine a reporting condition for monitoring-related information of the first model based on predefined performance monitoring-related parameters, where the performance monitoring-related parameters include at least one of the following:

[0252] The monitoring and reporting period of the first model;

[0253] The monitoring start time of the first model;

[0254] The prediction result of the first model needs to meet the first preset condition.

[0255] In some embodiments, the reporting condition includes at least one of the following:

[0256] The reporting time of the monitoring reporting cycle of the first model arrives;

[0257] The terminal receives reporting trigger information from the network device, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model;

[0258] The terminal detects that the prediction result of the first model meets a first preset condition.

[0259] In some embodiments, the first preset condition includes at least one of the following:

[0260] The number of times the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal;

[0261] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, where the actual optimal K beams are determined based on a measurement result of the terminal;

[0262] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

[0263] In some embodiments, the monitoring-related information reported by the terminal includes at least one of the following:

[0264] measurement results of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1;

[0265] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0266] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0267] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0268] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0269] the difference, the absolute value of the difference, or the average of the absolute values ​​of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the actual layer 1 reference signal received power values ​​of the optimal K beams;

[0270] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the actual layer 1 reference signal received power value of the optimal beam, or an absolute value of the difference, or an average of the absolute values ​​of the difference;

[0271] Identification information of the first model.

[0272] In some embodiments, the N measurements are all measurements; or, N measurements selected from all measurements; or, N measurements corresponding to N predictions selected from all predictions of the first model.

[0273] In some embodiments, the terminal indicates the measurement order of the N measurements in all measurements through a bitmap, or indicates measurement identification information of the N measurements.

[0274] In some embodiments, the N measurements selected from all measurements include at least one of the following:

[0275] The first measurement within the monitoring and reporting period of the first model;

[0276] The first measurement from the monitoring start time of the first model;

[0277] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0278] a measurement in which the difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold;

[0279] a measurement in which the difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than a fifth threshold;

[0280] a measurement in which the difference between the average value of the measurement results and the average value of the measurement results of the previous measurement is greater than a sixth threshold;

[0281] a measurement in which the difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than a seventh threshold;

[0282] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0283] In some embodiments, the N predictions selected from all predictions of the first model include at least one of the following:

[0284] The first forecast within the monitoring and reporting period;

[0285] The first prediction from the monitoring start time;

[0286] A forecast whose predicted result is different from the predicted result of the previous forecast;

[0287] The difference between the maximum value of the predicted result and the maximum value of the predicted result of the previous prediction is greater than the ninth threshold;

[0288] The difference between the minimum value of the predicted result and the minimum value of the predicted result of the previous prediction is greater than the tenth threshold;

[0289] The difference between the average value of the predicted results and the average value of the predicted results of the previous prediction is greater than the eleventh threshold;

[0290] The difference between the standard deviation of the predicted result and the standard deviation of the predicted result of the previous prediction is greater than the twelfth threshold;

[0291] The difference between the prediction result variance and the prediction result variance of the previous prediction is greater than the prediction of the thirteenth threshold.

[0292] In some embodiments, the reporting module is specifically configured to:

[0293] reporting the monitoring-related information of the first model to the network device through the first resource;

[0294] The first resource includes any one of the following:

[0295] Resources configured by the network device for reporting monitoring-related information of the first model;

[0296] Physical uplink shared channel PUSCH resources already available to the terminal;

[0297] The terminal requests PUSCH resources through a scheduling request SR.

[0298] In some embodiments, the terminal further includes:

[0299] The processing module is configured to discard the earliest stored measurement result and store the measurement result of the Lth measurement if the required storage space exceeds the remaining storage space of the terminal when storing the measurement result of the Lth measurement.

[0300] The present disclosure also provides a network device, including:

[0301] A receiving module, configured to receive monitoring-related information of the first model reported by the terminal;

[0302] A monitoring module is used to perform performance monitoring of the first model based on the monitoring-related information.

[0303] In some embodiments, the network device further comprises:

[0304] A first sending module is configured to send first information to the terminal, where the first information is used to indicate at least one of the following:

[0305] The monitoring and reporting period of the first model;

[0306] The monitoring start time of the first model;

[0307] The prediction result of the first model needs to meet the first preset condition.

[0308] In some embodiments, the reporting condition is determined by predefined performance monitoring related parameters, and the performance monitoring related parameters include at least one of the following:

[0309] The monitoring and reporting period of the first model;

[0310] The monitoring start time of the first model;

[0311] The prediction result of the first model needs to meet the first preset condition.

[0312] In some embodiments, the network device further comprises:

[0313] The second sending module is used to send reporting trigger information to the terminal, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model.

[0314] In some embodiments, the network device further comprises:

[0315] The third sending module is used to send reporting trigger information to the terminal when it is monitored that the system performance indicator meets the second preset condition.

[0316] In some embodiments, the first preset condition includes at least one of the following:

[0317] The number of times the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal;

[0318] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, where the actual optimal K beams are determined based on a measurement result of the terminal;

[0319] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

[0320] In some embodiments, the monitoring module includes:

[0321] A calculation unit is used to calculate the monitoring index of the first model based on the monitoring related information.

[0322] In some embodiments, when the first model is deployed on the network device, the monitoring-related information includes measurement results of the terminal performing N measurements on the reference signal, where N is an integer greater than 1;

[0323] The calculation unit includes:

[0324] a first screening subunit, configured to screen out N predictions corresponding to the N measurements from all predictions of the first model;

[0325] A first calculation subunit, configured to calculate a monitoring indicator of the first model based on the prediction results of the N predictions and the measurement results of the N measurements;

[0326] or,

[0327] The calculation unit includes:

[0328] A second screening subunit, configured to screen out M measurements from the N measurements, where M is an integer greater than 1;

[0329] a third screening subunit, configured to screen out M predictions corresponding to the M measurements from all predictions of the first model;

[0330] The second calculation subunit is used to calculate the monitoring index of the first model according to the prediction results of the M predictions and the measurement results of the M measurements.

[0331] In some embodiments, the M measurements selected from the N measurements include at least one of the following:

[0332] The first measurement within the monitoring and reporting period;

[0333] The first measurement from the monitoring start time;

[0334] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0335] a measurement in which the difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold;

[0336] a measurement in which the difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than a fifth threshold;

[0337] a measurement in which the difference between the average value of the measurement results and the average value of the measurement results of the previous measurement is greater than a sixth threshold;

[0338] a measurement in which the difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than a seventh threshold;

[0339] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0340] In some embodiments, when the first model is deployed on the terminal, the monitoring-related information includes at least one of the following:

[0341] measurement results of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1;

[0342] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0343] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0344] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0345] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0346] the difference, the absolute value of the difference, or the average of the absolute values ​​of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the actual layer 1 reference signal received power values ​​of the optimal K beams;

[0347] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the actual layer 1 reference signal received power value of the optimal beam, or an absolute value of the difference, or an average of the absolute values ​​of the difference;

[0348] Identification information of the first model.

[0349] In some embodiments, the receiving module is specifically configured to:

[0350] receiving, through a first resource, monitoring-related information of the first model reported by the terminal;

[0351] The first resource includes any one of the following:

[0352] Resources configured by the network device for reporting monitoring-related information of the first model;

[0353] Physical uplink shared channel PUSCH resources already available to the terminal;

[0354] The terminal requests PUSCH resources through a scheduling request SR.

[0355] In some embodiments, the network device further includes:

[0356] A processing module is configured to perform a first process according to the monitoring indicator of the first model, where the first process includes any one of the following:

[0357] activating the first model;

[0358] Deactivate the first model.

[0359] An embodiment of the present disclosure also provides a processor-readable storage medium, which stores a computer program, and the computer program is used to enable the processor to execute any one of the terminal-side performance monitoring methods provided by the embodiment of the present disclosure, or the computer program is used to enable the processor to execute any one of the network device-side performance monitoring methods provided by the embodiment of the present disclosure.

[0360] In the disclosed embodiment, the terminal determines whether a reporting condition for monitoring-related information of a first model is satisfied; if the reporting condition is satisfied, the terminal reports the monitoring-related information of the first model to the network device. Through the above process, if the reporting condition for monitoring-related information of the first model is satisfied, the terminal reports the monitoring-related information of the first model to the network device. In this way, the terminal can report multiple performance monitoring-related information to the network device at the same time, which can reduce the reporting overhead of the terminal during the performance monitoring process. BRIEF DESCRIPTION OF THE DRAWINGS

[0361] FIG1 is a schematic diagram of a network architecture applicable to embodiments of the present disclosure;

[0362] Figure 2 is a performance monitoring flow chart corresponding to the base station side deployment model;

[0363] Figure 3 is a performance monitoring flow chart corresponding to the terminal side deployment model;

[0364] FIG4 is a flow chart of a performance monitoring method provided by an embodiment of the present disclosure;

[0365] FIG5 is a schematic diagram of a base station sending a reference signal and a UE reporting a measurement value during a performance monitoring process provided by an embodiment of the present disclosure;

[0366] FIG6 is a flow chart of another performance monitoring method provided by an embodiment of the present disclosure;

[0367] FIG7 is a structural diagram of a terminal provided by an embodiment of the present disclosure;

[0368] FIG8 is a structural diagram of a network device provided by an embodiment of the present disclosure;

[0369] FIG9 is a structural diagram of another terminal provided in an embodiment of the present disclosure;

[0370] FIG10 is a structural diagram of another network device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0371] In order to make the technical problems, technical solutions and advantages to be solved by the present disclosure clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0372] In the embodiments of the present disclosure, the term "and / or" describes the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0373] In the embodiments of the present disclosure, the term "plurality" refers to two or more than two, and other quantifiers are similar thereto.

[0374] The following will be combined with the accompanying drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the embodiments described are only part of the embodiments of the present disclosure and not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0375] The embodiments of the present disclosure provide a performance monitoring method, a performance monitoring method, a terminal, a network device, and a storage medium, which are used to solve the problem of how to report performance monitoring related information during the performance monitoring process.

[0376] Among them, the method and the device are based on the same application concept. Since the principles of solving problems by the method and the device are similar, the implementation of the device and the method can refer to each other, and the repeated parts will not be repeated.

[0377] The technical solution provided by the embodiments of the present disclosure can be applicable to a variety of systems, especially 5G systems and 6G systems. For example, applicable systems can be global system of mobile communication (GSM) systems, code division multiple access (CDMA) systems, wideband code division multiple access (WCDMA) general packet radio service (GPRS) systems, long term evolution (LTE) systems, LTE frequency division duplex (FDD) systems, LTE time division duplex (TDD) systems, advanced long term evolution (LTE-A) systems, universal mobile telecommunication systems (UMTS), worldwide interoperability for microwave access (WiMAX) systems, 5G New Radio (5G NR) systems, 6G systems, etc. These various systems all include terminals and network equipment. The system may also include core network parts, such as the Evolved Packet System (EPS), 5G System (5GS), etc.

[0378] Please refer to FIG1 , which is a schematic diagram of a network architecture applicable to the present disclosure. As shown in FIG1 , the network architecture includes a terminal 11 and a network device 12, wherein:

[0379] The terminal 11 involved in the embodiments of the present disclosure may refer to a device that provides voice and / or data connectivity to a user, a handheld device with wireless connection capabilities, or other processing devices connected to a wireless modem. In different systems, the name of the terminal may also be different. For example, in a 5G system, the terminal may be called User Equipment (UE). A wireless terminal can communicate with one or more core networks (CN) via a radio access network (RAN). A wireless terminal can be a mobile terminal, such as a mobile phone (or "cellular" phone) and a computer with a mobile terminal. For example, it can be a portable, pocket-sized, handheld, computer-built-in, or vehicle-mounted mobile device that exchanges voice and / or data with a radio access network. For example, a Personal Communication Service (PCS) phone, a cordless phone, a Session Initiated Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA), a Reduced Capability (Redcap) terminal, and other devices. A wireless terminal may also be referred to as a system, a subscriber unit, a subscriber station, a mobile station, a mobile station, a remote station, an access point, a remote terminal, an access terminal, a user terminal, a user agent, or a user device, but is not limited in the embodiments of the present disclosure.

[0380] The network device 12 involved in the embodiments of the present disclosure may be a base station, which may include multiple cells providing services to the terminal. Depending on the specific application scenario, the base station may also be called an access point, or may be a device in the access network that communicates with the wireless terminal through one or more sectors on the air interface, or may be called another name. The network device may be used to interchange received air frames with Internet Protocol (IP) packets, acting as a router between the wireless terminal and the rest of the access network, wherein the rest of the access network may include an Internet Protocol (IP) communication network. The network device may also coordinate the attribute management of the air interface. For example, the network device involved in the embodiments of the present disclosure may be a base transceiver station (BTS) in the Global System for Mobile communications (GSM) or code division multiple access (CDMA), a network device (NodeB) in wide-band code division multiple access (WCDMA), an evolutionary Node B (eNB or e-NodeB) in the long term evolution (LTE) system, a 5G base station (gNB) in the 5G network architecture (next generation system), a base station in 6G, a home evolved Node B (HeNB), a relay node, a femto, a pico, etc., and is not limited in the embodiments of the present disclosure. In some network structures, the network device may include a centralized unit (CU) and a distributed unit (DU), and the centralized unit and the distributed unit may also be geographically separated.

[0381] In the disclosed embodiments, a network device and a terminal may each use one or more antennas for multiple-input multiple-output (MIMO) transmission. MIMO transmission may be single-user MIMO (SU-MIMO) or multi-user MIMO (MU-MIMO). Depending on the form and number of antenna combinations, MIMO transmission may be two-dimensional MIMO (2D-MIMO), three-dimensional MIMO (3D-MIMO), full-dimensional MIMO (FD-MIMO), or massive MIMO. It may also be diversity transmission, precoded transmission, or beamforming transmission.

[0382] Before describing the embodiments of the present disclosure, the following briefly introduces the relevant technologies:

[0383] In the NR system, to combat path loss in high-frequency scenarios, the transmitter and receiver obtain matching beam pairs through beam management (BM) to improve beamforming gain. In the downlink beam management process of the related art, the base station selects an appropriate transmit beam (Tx beam) for subsequent communication based on the beam measurement information reported by the UE, and indicates the selected beam information to the UE. The main drawback of the related art is that in order to obtain the optimal beam pair, the base station needs to send the Channel State Information Reference Signal (CSI-RS) / Synchronization Signal Block (SSB) on all Tx beams, which consumes a lot of reference signal resources. At the same time, the UE needs to use all receive beams (Rx beams) to measure the CSI-RS / SSB sent on all Tx beams separately, which results in a large measurement overhead. To address these problems, it is proposed to use AI / ML technology to predict the optimal beam (pair) based on the measurement results of some beams (pairs) or historical beam (pairs), thereby saving reference signal resource overhead and UE measurement overhead, and reducing UE measurement latency.

[0384] For example, the 3rd Generation Partnership Project (3GPP) Release 18 (Rel-18) Air Interface AI topic identifies the following two sub-use cases as sub-use cases for AI beam management:

[0385] BM-case 1: Spatial beam prediction, which predicts the best K (also known as Top-K) beams (pairs) in Set A (Set A) based on Set B (Set B) measured at a certain moment;

[0386] BM-case2: Time-domain beam prediction, that is, predicting the top-K beams (pairs) of SetA at the next N' moments based on SetB measured at the past N moments;

[0387] SetA can be understood as the full set of beams, which may include 32 transmit beams. SetB can be understood as the subset of beams, i.e., a subset of SetA, which may include 8 transmit beams.

[0388] The output of the AI / ML model can be the ID of the predicted Top-K beams (pairs) of SetA or the predicted Reference Signal Received Power (RSRP) of the beams (pairs) of SetA. The latter can select the Top-K beam pairs based on the size of the predicted RSRP.

[0389] As data-driven algorithms, AI / ML algorithms suffer from generalization issues and lack theoretical guarantees for their performance. To ensure the performance of AI / ML in communication systems, model monitoring of AI / ML models is necessary.

[0390] Different AI / ML models can use different metrics for model monitoring. Currently, 3GPP supports research on model monitoring metrics such as the beam prediction metric or the difference between the measured and predicted RSRP. The beam prediction metric applies to models that output the Top-K beam (pair) ID or the RSRP of SetA; the difference between the measured and predicted RSRP metric applies to models that output the RSRP of SetA.

[0391] Beam prediction metrics may include the prediction accuracy of the top K / best (also known as Top-K / 1) beams, the prediction accuracy of the top-1 / K beams, and the prediction accuracy of the best beam within an X dB range, which are defined as follows:

[0392] Top-K / 1 beam prediction accuracy: The probability that the predicted Top-K beams include the actual optimal (also known as Top-1) beam;

[0393] Top-1 / K beam prediction accuracy: the probability that the predicted Top-1 beam belongs to the actual Top-K beam;

[0394] X dB range optimal beam prediction accuracy: The probability that the difference between the predicted Layer 1 RSRP (L1-RSRP) measurement value of the top-1 beam and the actual L1-RSRP measurement value of the top-1 beam is within X dB.

[0395] The above-mentioned "actual Top-1 / Top-K beam" is obtained by the UE measuring a certain reference signal set. For example, the UE measures the SetA reference signal set and obtains the beam corresponding to the Top-1 or Top-K L1-RSRP.

[0396] The difference between the measured and predicted RSRP refers to the difference between the measured and predicted RSRP for a subset of beams (e.g., Set B beams or the model-predicted Top-K beams). This metric has the advantage of only transmitting and measuring the subset of beams, rather than all beams in Set A. This saves reference signal resource overhead and UE measurement overhead.

[0397] AI models can be deployed on the base station side or on the UE side. If the model is deployed on the base station side, the base station generally performs model monitoring. That is, the base station compares the model prediction results (also known as inference results) with the benchmark results to determine whether the current model is applicable. If the model is deployed on the UE side, the UE can report the model inference results and measurement results to the base station, which will compare the model inference results with the benchmark results. Alternatively, the UE can directly report the model monitoring results to the base station, which will determine whether the current model is applicable.

[0398] Taking the deployment of an AI / ML model on the base station side as an example, if the output of the AI / ML model is the predicted L1-RSRP of the SetA beam, the difference between the measured L1-RSRP value and the predicted L1-RSRP value of the Top-K beams is used as the model monitoring indicator. The model monitoring process is shown in Figure 2. If the AI / ML model is deployed on the UE side, the model monitoring process is shown in Figure 3.

[0399] It can be seen that no matter whether the AI / ML model is deployed on the base station side or on the UE side, when the model is monitored by the base station side, the UE needs to report the measurement results and / or inference results to the base station. For example, steps six and seven in Figures 2 and 3 both involve the UE measuring the Top-K beams predicted by the model and feeding back the relevant results to the base station. In the reporting of layer 1 beam quality (such as L1-RSRP, L1-SINR) in the related technology, the base station configures or indicates the reporting resources and configures the measurement reference signal associated with the reporting. The UE feeds back the measurement results on the Physical Uplink Control Channel (PUCCH) or the Physical Uplink Shared Channel (PUSCH) through layer 1 signaling. Each time, the feedback is a single measurement result of one or some beams or the average value of the measurement results at multiple times.

[0400] In the model monitoring process, in order to ensure the robustness of model monitoring, the performance of the AI / ML model will be judged based on multiple monitoring results of the AI / ML model over a period of time. According to the existing layer 1 beam reporting process, each time the UE measures the beam set used for model monitoring (such as the Top-K beams inferred by the model), it will feedback to the base station through layer 1 signaling. However, model monitoring judges the performance of the model based on the monitoring results over a period of time, and the latency requirements are lower than other layer 1 measurements. On the other hand, multiple measurement results of the UE may be similar, and the results of model monitoring are also similar. If all these similar results are reported to the base station, it may cause information redundancy and waste reporting overhead.

[0401] It should be noted that since model monitoring is to judge the performance of the model based on the monitoring results over a period of time, model monitoring can be understood or replaced by performance monitoring.

[0402] In light of this, embodiments of the present disclosure propose a performance monitoring method. During the performance monitoring process, a UE reports multiple performance monitoring measurement results simultaneously. Furthermore, the UE can filter these measurement results and report only those that are relevant to performance monitoring. This method reduces unnecessary reporting overhead while ensuring the performance monitoring function is completed.

[0403] The following describes the performance monitoring method, performance monitoring method, terminal, and network device provided by the embodiments of the present disclosure in conjunction with the accompanying drawings.

[0404] Please refer to Figure 4, which is a flow chart of a performance monitoring method provided by an embodiment of the present disclosure. As shown in Figure 4, the performance monitoring method includes the following steps:

[0405] Step 401: The terminal determines whether a reporting condition for monitoring-related information of a first model is met;

[0406] Step 402: When the reporting condition is met, the terminal reports the monitoring-related information of the first model to the network device.

[0407] In the embodiment of the present disclosure, the first model can be deployed on the terminal side or on the network device side. Regardless of which side the first model is deployed on, the network device needs to monitor the first model based on the information reported by the terminal.

[0408] The monitoring-related information of the first model (referred to as model monitoring-related information or performance monitoring-related information) can be understood as information required to monitor the first model, such as the terminal's measurement results, the model's prediction results, or a comparison result of the two. The content of the performance monitoring-related information reported by the UE to the network device may also vary depending on the deployment location of the first model and the monitoring indicators of the first model.

[0409] In the disclosed embodiment, the terminal determines whether a reporting condition for monitoring-related information of a first model is satisfied; if the reporting condition is satisfied, the terminal reports the monitoring-related information of the first model to the network device. Through the above process, if the reporting condition for monitoring-related information of the first model is satisfied, the terminal reports performance monitoring-related information to the network device. In this way, the terminal can report multiple performance monitoring-related information to the network device at the same time, which can reduce the reporting overhead of the terminal during the performance monitoring process.

[0410] In the embodiment of the present disclosure, before the reporting condition is met, the terminal can store the performance monitoring related information (such as measurement results) that needs to be reported. When the reporting condition is met, the terminal reports the stored performance monitoring related information to the network device.

[0411] In some embodiments, the method further comprises:

[0412] When the terminal stores the measurement result of the Lth measurement, if the required storage space exceeds the remaining storage space of the terminal, the terminal discards the earliest stored measurement result and stores the measurement result of the Lth measurement.

[0413] In the embodiment of the present disclosure, satisfying the reporting condition can be understood as the UE being triggered to report performance monitoring related information. The triggering methods may include the following three:

[0414] Method 1: Periodic or semi-continuous reporting of network device configuration;

[0415] Method 2: Event trigger reporting;

[0416] Method 3: Aperiodic reporting triggered by network devices.

[0417] Mode 2 is when a certain event occurs (or after it occurs), and the UE reports performance monitoring related information. Mode 3 is applicable when the network equipment detects that the system performance indicators (such as throughput and beam failure probability) have deteriorated, and the network equipment triggers the UE to report performance monitoring related information aperiodically.

[0418] The reporting mechanisms under the above-mentioned methods reduce unnecessary reporting by the terminal and save reporting overhead.

[0419] After the UE reports the performance monitoring related information through the above-mentioned methods 1 to 3, the network device can perform performance monitoring of the first model. For example, the network device can calculate the monitoring indicators of the first model based on the performance monitoring related information reported by the UE. The network device can determine the performance of the model based on the calculation result and perform corresponding processing based on the determination result, such as activating the model, deactivating the model, and falling back to the traditional beam management method.

[0420] In some embodiments, the method further comprises:

[0421] The terminal determines, based on first information sent by the network device, a reporting condition for monitoring-related information of the first model, where the first information is used to indicate at least one of the following:

[0422] The monitoring and reporting period of the first model;

[0423] The monitoring start time of the first model;

[0424] The prediction result of the first model needs to meet the first preset condition.

[0425] In this implementation, the reporting condition of the performance monitoring related information may be indicated to the UE via the first information sent by the network device.

[0426] In some embodiments, the method further comprises:

[0427] The terminal determines a reporting condition for monitoring-related information of the first model according to predefined performance monitoring-related parameters, where the performance monitoring-related parameters include at least one of the following:

[0428] The monitoring and reporting period of the first model;

[0429] The monitoring start time of the first model;

[0430] The prediction result of the first model needs to meet the first preset condition.

[0431] In this implementation, when performance monitoring related parameters are predefined (eg, as agreed upon in a protocol), the terminal may also determine a reporting condition for performance monitoring related information based on the predefined performance monitoring related parameters.

[0432] In some embodiments, the reporting condition includes at least one of the following:

[0433] The reporting time of the monitoring reporting cycle of the first model arrives; (corresponding to the triggering method of method 1)

[0434] The terminal receives reporting trigger information from the network device, where the reporting trigger information is used to instruct reporting of monitoring-related information of the first model; (corresponding to the triggering method of method 3)

[0435] The terminal detects that the prediction result of the first model meets the first preset condition. (Corresponding to the triggering method of method 2)

[0436] In the event-triggered reporting method (i.e., Method 2), the event triggering the reporting may be, for example, model performance falling below a predetermined condition (e.g., the model's prediction result meets a first preset condition). That is, only when the model performance falls below the predetermined condition will the UE report performance monitoring-related information to the network device.

[0437] In some embodiments, the first preset condition includes at least one of the following:

[0438] The number of times the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal;

[0439] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, where the actual optimal K beams are determined based on a measurement result of the terminal;

[0440] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

[0441] It should be noted that the first preset condition can be configured or indicated by the network device (such as indicated by the first information) or predefined (such as agreed in a protocol).

[0442] In some embodiments, the monitoring-related information reported by the terminal includes at least one of the following:

[0443] measurement results of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1;

[0444] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0445] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0446] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0447] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0448] the difference, the absolute value of the difference, or the average of the absolute values ​​of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the actual layer 1 reference signal received power values ​​of the optimal K beams;

[0449] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the actual layer 1 reference signal received power value of the optimal beam, or an absolute value of the difference, or an average of the absolute values ​​of the difference;

[0450] Identification information of the first model.

[0451] As previously mentioned, the monitoring-related information reported by the terminal can vary depending on the model deployment location or performance monitoring indicators. For example, when the model is deployed on the terminal side, the monitoring-related information reported by the terminal may include the terminal's measurement results and the model's prediction results, etc.; or, the terminal's measurement results, the model's prediction results, and a comparison of the two, etc.; or, the model's identification information. When the model is deployed on the network device side, the monitoring-related information reported by the terminal may include the terminal's measurement results.

[0452] The measurement result of the terminal can be obtained by measuring the reference signal sent by the network device. For example, the network device sends the reference signal on the entire set of beams, or the network device sends the reference signal on the optimal K beams predicted by the model.

[0453] In some embodiments, the N measurements are all measurements of the terminal; or, the N measurements selected by the terminal from all measurements; or, the N measurements corresponding to the N predictions selected by the terminal from all predictions of the first model.

[0454] The terminal can report all measurement results to the network device, or it can filter the measurement results before reporting them to the network device, or it can filter the model prediction results before reporting them to the network device. In other words, both the measurement results and the prediction results used for performance monitoring can be filtered results. One method is for the UE to report all measurement results (and / or model prediction results), and the network device performs the filtering. Another method is for the UE to perform the filtering and only report the filtered measurement results (and / or model prediction results).

[0455] In some embodiments, the N measurements screened by the terminal from all measurements include at least one of the following:

[0456] The first measurement within the monitoring and reporting period of the first model;

[0457] The first measurement from the monitoring start time of the first model;

[0458] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0459] a measurement in which the difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold;

[0460] a measurement in which the difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than a fifth threshold;

[0461] a measurement in which the difference between the average value of the measurement results and the average value of the measurement results of the previous measurement is greater than a sixth threshold;

[0462] a measurement in which the difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than a seventh threshold;

[0463] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0464] The values ​​of the above thresholds can be configured by the network device or pre-agreed.

[0465] In some embodiments, the N predictions selected by the terminal from all predictions of the first model include at least one of the following:

[0466] The first forecast within the monitoring and reporting period;

[0467] The first prediction from the monitoring start time;

[0468] A forecast whose predicted result is different from the predicted result of the previous forecast;

[0469] The difference between the maximum value of the predicted result and the maximum value of the predicted result of the previous prediction is greater than the ninth threshold;

[0470] The difference between the minimum value of the predicted result and the minimum value of the predicted result of the previous prediction is greater than the tenth threshold;

[0471] The difference between the average value of the predicted results and the average value of the predicted results of the previous prediction is greater than the eleventh threshold;

[0472] The difference between the standard deviation of the predicted result and the standard deviation of the predicted result of the previous prediction is greater than the twelfth threshold;

[0473] The difference between the prediction result variance and the prediction result variance of the previous prediction is greater than the prediction of the thirteenth threshold.

[0474] The values ​​of the above thresholds can be configured by the network device or pre-agreed.

[0475] In some embodiments, the terminal indicates the measurement order of the N measurements in all measurements through a bitmap, or indicates measurement identification information of the N measurements.

[0476] For example, assuming that the UE completes 100 measurements on the reference signal within the monitoring reporting period T ms, and finally reports 30 measurement results, the UE can indicate which 30 of the 100 measurements the reported 30 measurement results correspond to through a bitmap.

[0477] In some embodiments, the terminal reports the monitoring-related information of the first model to the network device, including:

[0478] The terminal reports the monitoring-related information of the first model to the network device through the first resource;

[0479] The first resource includes any one of the following:

[0480] Resources configured by the network device for reporting monitoring-related information of the first model;

[0481] PUSCH resources already available to the terminal;

[0482] The terminal requests PUSCH resources through a scheduling request (SR).

[0483] In this implementation, the resources used by the UE for reporting can be determined as follows:

[0484] a) If the reporting resources are configured by the network device, the UE reports the measurement results on the resources configured by the network device;

[0485] b) If the reporting resources are not configured by the network device and the UE already has PUSCH resources, the UE can report information on the existing PUSCH resources, such as the configured grant uplink resources;

[0486] c) If the reporting resources are not configured by the network device and the UE does not have PUSCH resources, the UE can request PUSCH resources through SR. After the network device schedules the PUSCH resources, the UE reports the stored measurement results on the PUSCH resources scheduled by the network device.

[0487] The following describes the specific implementation processes of various performance monitoring solutions involved in the embodiments of the present disclosure:

[0488] Solution 1: Assume the model is deployed on the base station side

[0489] Step 1-1: The base station configures the performance monitoring reporting period T.

[0490] Step 1-2: The base station sends a reference signal for performance monitoring, such as sending a reference signal on the Top-K beam inferred by the model.

[0491] Step 1-3: If the base station sends a reference signal on the Top-K beam inferred by the model, the base station stores the result of the model inference, such as the predicted value of L1-RSRP of the Top-K beam.

[0492] Step 1-4: The UE measures the reference signal sent by the base station and stores the L1-RSRP of the measured reference signal.

[0493] Repeat steps 1-1 to 1-4 above until the time meets the reporting time of the reporting cycle T.

[0494] Step 1-5: The UE reports the measurement results stored in T. The content reported by the UE corresponds to the L1-RSRP measurement value of the Top-K beams that sent the reference signal on the Top-K beams inferred by the model within time T. The resources used by the UE report can be determined as follows:

[0495] a) If the reporting resources are configured by the base station, the UE reports the measurement results within time T using the resources configured by the base station;

[0496] b) If the reporting resources are not configured by the base station, the UE reports the measurement results within time T through the existing PUSCH resources or the PUSCH resources requested by the SR.

[0497] Step 1-6: After receiving the measurement results reported by the UE, the base station calculates the performance monitoring indicators:

[0498] a) The base station calculates the average of the absolute values ​​of the differences between the predicted values ​​of L1-RSRP of the Top-K beams stored within time T and the measured values ​​reported by the UE; or

[0499] b) The base station selects the results for performance monitoring indicator calculation based on the UE's reporting results. The selection method may include the following methods:

[0500] i. If the measurement is the first measurement within the reporting period, select the measurement result;

[0501] ii. If it is not the first measurement, the base station determines whether there is a different measurement result of L1-RSRP in the measurement reported by the UE and the previous measurement reported. If so, the measurement result is selected;

[0502] iii. The base station determines whether the difference between the maximum, minimum, or average value of the reference signal quality measured in the previous measurement is greater than X dB; if so, the measurement result is selected;

[0503] iv. The base station determines whether the difference between the standard deviation / variance of the reference signal quality measured this time and the standard deviation / variance of the reference signal quality measured previously is greater than Y dB or Y%. If so, the measurement result of this time is selected.

[0504] The base station selects the inference result corresponding to the measurement result selected according to the above principle from the stored inference results, and calculates the average of the absolute values ​​of the differences between the predicted value of L1-RSRP of the Top-K beam and the measurement value reported by the UE based on the selected inference result and the selected measurement result.

[0505] Steps 1-7: The base station determines the performance of the base station-side model based on the performance monitoring indicators (also called model monitoring indicators), and performs operations such as model activation, model deactivation, or fallback to the traditional beam management method:

[0506] a) If the calculated monitoring index value of one model is less than the monitoring index value of another model, the model corresponding to the smaller monitoring index value is activated, and the model corresponding to the larger monitoring index value is deactivated;

[0507] b) If the calculated monitoring indicator value of a model is greater than a threshold, the model is deactivated or the traditional beam management method is reverted.

[0508] Solution 2: Assume the model is deployed on the base station side

[0509] Step 2-1: Same as step 1-1 in solution 1.

[0510] Step 2-2: Same as steps 1-2 in solution 1.

[0511] Step 2-3: Same as steps 1-3 in solution 1.

[0512] Step 2-4: The UE measures the reference signal sent by the base station and determines whether the measurement result needs to be stored:

[0513] a) If the measurement is the first measurement in the reporting period, the UE stores the result;

[0514] b) If it is not the first measurement, the UE determines whether there is a measurement result of L1-RSRP different from that of the previous measurement. If so, the UE stores the result;

[0515] Alternatively, the UE determines whether a difference between the maximum value, minimum value, or average value of the reference signal quality measured this time and that measured previously is greater than X dB; if so, the UE stores the result;

[0516] Alternatively, the UE determines whether the difference between the standard deviation / variance of the reference signal quality measured this time and the standard deviation / variance of the reference signal quality measured last time is greater than Y dB or Y%. If so, the UE stores the measurement result.

[0517] Repeat steps 2-1 to 2-4 above until the time meets the reporting time of the reporting cycle T.

[0518] Step 2-5: The UE reports the measurement results stored within T. The content reported by the UE corresponds to the measured values ​​of L1-RSRP of the Top-K beams inferred by the model that transmit reference signals on the Top-K beams that meet the conditions in step 2-4 within time T, and the measurement times within T to which these measurement results correspond. The UE can report on existing PUSCH resources or on PUSCH resources requested through SR.

[0519] Step 2-6: After receiving the measurement results reported by the UE, the base station selects the model inference results corresponding to the measurement results reported by the UE from the stored model inference results based on the UE's reported information and calculates the performance monitoring indicator. For example, the base station calculates the average of the absolute values ​​of the differences between the stored predicted values ​​of L1-RSRP for the Top-K beams and the measured values ​​reported by the UE during time T.

[0520] Steps 2-7: Same as steps 1-7 in solution 1.

[0521] Solution 3: Assume the model is deployed on the UE side

[0522] Step 3-1: The base station configures the performance monitoring reporting period T.

[0523] Step 3-2: The base station sends a reference signal for performance monitoring. For example, the reference signal is sent on the Top-K beams derived from the model inference. The model inference result is reported to the base station by the UE.

[0524] Step 3-3: The UE measures the reference signal sent by the base station and stores the measurement results and the corresponding model inference results. The UE can store part of the measurement results and inference results according to the following method:

[0525] a) The UE determines whether the measurement result needs to be stored, and also stores the model inference result corresponding to the stored measurement result:

[0526] i. If the measurement is the first measurement within the reporting period, the UE stores the result;

[0527] ii. If it is not the first measurement, the UE determines whether there is a different measurement result of L1-RSRP in the previous measurement. If so, the UE stores the result;

[0528] Alternatively, the UE determines whether a difference between the maximum value, minimum value, or average value of the reference signal quality measured this time and that measured previously is greater than X dB; if so, the UE stores the result;

[0529] Alternatively, the UE determines whether a difference between a standard deviation / variance of the reference signal quality measured this time and a standard deviation / variance of the reference signal quality measured previously is greater than Y dB or Y%. If so, the UE stores the measurement result.

[0530] b) The UE determines whether the inference result needs to be stored, and also stores the measurement result corresponding to the stored inference result:

[0531] i. If the reasoning result is the first reasoning within the reporting period, the UE stores the reasoning result;

[0532] ii. If it is not the first reasoning, the UE determines whether there is a different prediction result of L1-RSRP in the reasoning and the previous reasoning. If so, the UE stores the reasoning result;

[0533] Alternatively, the UE determines whether the difference between the maximum value, minimum value, or average value of the reference signal L1-RSRP predicted value obtained in the current inference and the previous inference is greater than X dB; if so, the UE stores the inference result;

[0534] Alternatively, the UE determines whether a difference between a standard deviation / variance of the L1-RSRP prediction value of the reference signal inferred this time and a standard deviation / variance of the L1-RSRP prediction value of the reference signal inferred previously is greater than Y dB or Y%; if so, the UE stores the measurement inference result;

[0535] The UE stores the measured L1-RSRP value and the inferred L1-RSRP value selected according to method a) or method b), or stores the difference between the selected measured L1-RSRP value and the inferred L1-RSRP value.

[0536] Repeat steps 3-1 to 3-3 until the time reaches the reporting time of the reporting cycle T.

[0537] Step 3-4: The UE reports the results stored within T. The content reported by the UE corresponds to the measurement results and inference results that meet the conditions in step 3-3 within time T, or the difference between the measurement results and the inference results. The UE can report on the existing PUSCH resources or the PUSCH resources requested through the SR.

[0538] Step 3-5: After receiving the measurement results reported by the UE, the base station calculates the performance monitoring indicators:

[0539] a) If the UE reports the measurement result and the inference result, the base station calculates the average of the absolute values ​​of the differences between the reported predicted value and the measured value of L1-RSRP;

[0540] b) If the UE reports the difference between the measurement result and the inference result, the base station calculates the average of the absolute values ​​of the differences.

[0541] Steps 3-6: The base station determines the UE-side model performance based on the performance monitoring indicators and performs operations such as model activation, model deactivation, or fallback to the traditional beam management method:

[0542] a) If the performance monitoring indicator value calculated according to steps 3-5 is less than another performance monitoring indicator value, activate the model corresponding to the smaller performance monitoring indicator value and deactivate the model corresponding to the larger performance monitoring indicator value;

[0543] b) If the performance monitoring indicator value calculated according to steps 3-5 is greater than the threshold, deactivate the model or fall back to the traditional beam management method.

[0544] Solution 4: Assume the model is deployed on the UE side

[0545] Step 4-1: The base station configures the performance monitoring reporting period T.

[0546] Step 4-2: The base station sends a reference signal for performance monitoring, such as sending a reference signal on the beam of SetA.

[0547] Step 4-3: The UE measures the reference signal sent by the base station and makes the following judgment based on the measurement result and the corresponding inference result:

[0548] a) Whether the Top-1 beam inferred by the model belongs to the actual Top-K beam; or,

[0549] b) Whether the Top-K beams inferred by the model include the actual Top-1 beam;

[0550] The actual Top-1 beam is the beam corresponding to the maximum L1-RSRP measurement value in the reference signal sent on the SetA beam; the actual Top-K beam is the beam corresponding to the Top-K L1-RSRP measurement value in the reference signal sent on the SetA beam;

[0551] If not included, the count is increased by 1.

[0552] Repeat steps 4-1 to 4-3 until the time meets the reporting time of the reporting cycle T.

[0553] Step 4-4: The UE reports the number of times the model-inferred Top-1 beam does not belong to the actual Top-K beam within T, or the number of times the model-inferred Top-K beam does not include the actual Top-1 beam. The resources used by the UE report can be determined as follows:

[0554] a) If the reporting resources are configured by the base station, the UE reports the results on the resources configured by the base station;

[0555] b) If the reporting resources are not configured, the UE reports information on the existing PUSCH resources or the PUSCH resources requested through SR.

[0556] Step 4-5: After receiving the measurement results reported by the UE, the base station calculates the performance monitoring indicators:

[0557] a) If the UE reports the number of times the model-inferred Top-1 beam does not belong to the actual Top-K beams, the base station calculates the model prediction accuracy (which can be expressed as Top-1 / K%) based on the number of times reported by the UE and the number of times the SetA reference signal was sent within T;

[0558] b) If the UE reports the number of times that the model-inferred Top-K beams do not include the actual Top-1 beam, the base station calculates the model prediction accuracy (which can be expressed as Top-K / 1%) based on the number of times reported by the UE and the number of times the SetA reference signal is sent within T.

[0559] Steps 4-6: The base station determines the UE-side model performance based on the performance monitoring indicators and performs operations such as model activation, model deactivation, or fallback to the traditional beam management method, which is the same as steps 3-6 in solution three.

[0560] Solution 5: Assume the model is deployed on the UE side

[0561] Step 5-1: The base station indicates the performance monitoring start time.

[0562] Step 5-2: The base station sends a reference signal for performance monitoring, such as sending a reference signal on the beam of SetA or Top-K of model inference.

[0563] Step 5-3: The UE measures the reference signal sent by the base station and performs the following operations based on the measurement results and the corresponding inference results:

[0564] a) Determine whether the Top-1 beam inferred by the model belongs to the actual Top-K beam; or,

[0565] b) Determine whether the Top-K beams inferred by the model include the actual Top-1 beam;

[0566] The actual Top-1 beam is the beam corresponding to the maximum L1-RSRP measurement value in the reference signal sent on the SetA beam; the actual Top-K beam is the beam corresponding to the Top-K L1-RSRP measurement value in the reference signal sent on the SetA beam;

[0567] If not included, the count is increased by 1; or

[0568] c) Calculate the difference between the predicted value and the measured value of L1-RSRP of the Top-K beams inferred by the model.

[0569] Repeat steps 5-1 to 5-3 until the following conditions are met, then proceed to step 5-4.

[0570] Step 5-4: UE reports information to the base station:

[0571] a) If the UE performs calculation according to step a) in step 5-3, and the number of times the model-inferred Top-1 beam does not belong to the actual Top-K beam is greater than or equal to N times, the UE reports the information;

[0572] b) If the UE performs calculation according to b) in step 5-3, and the number of times that the model-inferred Top-K beams do not include the actual Top-1 beam is greater than or equal to N times, the UE reports the information;

[0573] c) If the UE performs calculation according to c) in step 5-3, when the average absolute value of the difference between the predicted value and the measured value of L1-RSRP of the Top-K beams inferred by the model is greater than W dB, the UE reports the information.

[0574] In addition, the content reported by the UE may include the model ID that meets the above conditions, and may also include the specific value of the performance monitoring indicator, such as the average of the absolute value of the difference between the predicted value and the measured value of L1-RSRP of the Top-K beam inferred by the reporting model.

[0575] The UE can report on the existing PUSCH resources or the PUSCH resources requested through SR.

[0576] Step 5-5: The base station determines the UE-side model performance based on the results reported by the UE, and performs operations such as model activation, model deactivation, or fallback to the traditional beam management method, which is the same as steps 3-6 in solution three.

[0577] Solution 6: Assume the model is deployed on the UE side

[0578] Step 6-1: The base station indicates the performance monitoring start time.

[0579] Step 6-2: The base station sends a reference signal for performance monitoring, such as sending a reference signal on the Top-K beam of the model inference.

[0580] Step 6-3: The UE measures the reference signal sent by the base station and determines whether the measurement result and inference result need to be stored:

[0581] a) The UE determines whether the measurement result needs to be stored, and also stores the model inference result corresponding to the stored measurement result:

[0582] i. If the measurement is the first measurement within the reporting period, the UE stores the result;

[0583] ii. If it is not the first measurement, the UE determines whether there is a different measurement result of L1-RSRP in the previous measurement. If so, the UE stores the result;

[0584] Alternatively, the UE determines whether a difference between the maximum value, minimum value, or average value of the reference signal quality measured this time and that measured previously is greater than X dB; if so, the UE stores the result;

[0585] Alternatively, the UE determines whether a difference between a standard deviation / variance of the reference signal quality measured this time and a standard deviation / variance of the reference signal quality measured previously is greater than Y dB or Y%. If so, the UE stores the measurement result.

[0586] Alternatively, b) the UE determines whether the inference result needs to be stored, and also stores the performance monitoring measurement result corresponding to the stored inference result:

[0587] i. If the reasoning result is the first reasoning within the reporting period, the UE stores the reasoning result;

[0588] ii. If it is not the first reasoning, the UE determines whether there is a different prediction result of L1-RSRP in the reasoning and the previous reasoning. If so, the UE stores the reasoning result;

[0589] Alternatively, the UE determines whether the difference between the maximum value, minimum value, or average value of the reference signal L1-RSRP predicted value obtained in the current inference and the previous inference is greater than X dB; if so, the UE stores the inference result;

[0590] Alternatively, the UE determines whether the difference between the standard deviation / variance of the L1-RSRP prediction value of the reference signal inferred this time and the standard deviation / variance of the L1-RSRP prediction value of the reference signal inferred previously is greater than Y dB or Y%; if so, the UE stores the measurement inference result.

[0591] The UE stores the measured L1-RSRP value selected according to the above method a) or b) and the inferred L1-RSRP value, or stores the difference between the selected measured L1-RSRP value and the inferred L1-RSRP value.

[0592] Repeat steps 6-1 to 6-3 above.

[0593] Step 6-4: If the storage capacity of a certain result exceeds the maximum storage capacity of the UE, the UE discards the oldest result among the saved results and stores the current result.

[0594] Step 6-5: When the system performance indicators monitored by the base station (such as throughput and beam failure probability) deteriorate, the base station triggers the UE to report the performance monitoring results, and then determines whether the system performance deterioration is caused by the degradation of model performance.

[0595] Step 6-6: After receiving the aperiodic reporting signaling triggered by the base station, the UE reports the stored results. The UE can report the stored results on the existing PUSCH resources or the PUSCH resources requested through the SR.

[0596] Steps 6-7: After receiving the measurement results reported by the UE, the base station calculates the performance monitoring indicators, which is the same as steps 3-5 in solution three.

[0597] Steps 6-8: The base station determines the UE-side model performance based on the performance monitoring indicators and performs operations such as model activation, model deactivation, or fallback to the traditional beam management method, which is the same as steps 3-6 in solution three.

[0598] In order to better understand the technical solutions of the embodiments of the present disclosure, specific embodiments are provided below for illustrative purposes.

[0599] Example 1 (corresponding to solution 1):

[0600] In this embodiment, an AI / ML model is deployed on the base station side for spatial prediction. The AI / ML model predicts the optimal beam in SetA based on the measurement results of SetB beams. The input of the AI / ML model is the L1-RSRP of SetB beams, and the output is the predicted RSRP of all beams in SetA. SetA includes 32 transmit beams, SetB includes 8 transmit beams, and SetB is a subset of SetA.

[0601] The base station periodically transmits the reference signal corresponding to SetB beams. After the UE provides feedback on the L1-RSRP of SetB beams, the base station performs inference to obtain the predicted L1-RSRP for the 32 beams of SetA and selects the three beams with the largest predicted L1-RSRP values ​​as the top-3 beams. The base station transmits the reference signal corresponding to the top-3 beams inferred by the AI / ML model, and the UE measures the L1-RSRP of these top-3 beams. The reference signal corresponding to the top-3 beams transmitted by the base station may only be the top-3 reference signal for certain inference results, and there is no need to monitor every inference result. For example, the base station transmits the reference signal corresponding to the inferred top-3 beams at regular intervals, or only when the inference result of the model differs significantly from the previous inference result.

[0602] Assume that the performance monitoring metric is the difference between the measured and predicted L1-RSRP values ​​for the top three beams predicted by the AI ​​model. The base station configures the UE's reporting period to be T ms. The reported content is the L1-RSRP measurement value of the reference signal for the top three beams configured by the base station within T ms. The reported content is carried by Media Access Control-Control Element (MAC-CE) signaling. The base station sends the reference signal for the top three beams inferred by the model 100 times within T ms and stores the predicted L1-RSRP values ​​for the top three beams inferred by the model. After completing each T ms measurement, if the UE already has PUSCH resources, it reports them using the available PUSCH resources, such as the configured uplink grant resources. If the UE does not have PUSCH resources, it requests PUSCH resources through SR. After the base station schedules PUSCH resources, the UE reports the L1-RSRP measurement values ​​of the reference signals of the top-3 beams measured 100 times within T ms using the PUSCH resources scheduled by the base station, as shown in Figure 5.

[0603] After receiving the L1-RSRP measurement values ​​of the reference signals of the top-3 beams within Tms reported by the UE, the base station selects results with different L1-RSRP measurement values ​​for the top-3 beams. If the L1-RSRP value of a group of top-3 beams is the same as that of the previous group of top-3 beams, that group of measurement results is discarded. Otherwise, the group of measurement results is retained. Assume that the base station ultimately selects 30 measurement results from 100 top-3 beam measurement results. The base station selects the model inference results corresponding to the retained 30 measurement results from the 100 stored model inference results. Based on these 30 measurement results and the inference results, the base station calculates the difference between the average predicted and measured L1-RSRP values ​​for the top-3 beams and uses this difference to determine the performance of the AI / ML model. For example, if the average absolute value of the difference between the predicted and measured L1-RSRP values ​​for these 30 top-3 beams is greater than 1 dB, the current AI model performance is considered to have deteriorated and the AI ​​model is deactivated.

[0604] Example 2 (corresponding to scheme 2):

[0605] In this embodiment, an AI / ML model is deployed on the base station side for airspace prediction, and the input and output of the AI / ML model are the same as those in Example 1.

[0606] The base station periodically transmits the reference signal corresponding to the SetB beam. After the UE provides feedback on the L1-RSRP of the SetB beam, the base station infers the predicted L1-RSRP for the 32 beams of SetA and selects the three beams with the largest predicted L1-RSRP values ​​as the top-3 beams. The base station transmits the reference signal corresponding to the top-3 beams inferred by the AI / ML model, and the UE measures the L1-RSRP of the top-3 beams. After each L1-RSRP measurement of the top-3 beams, the UE determines whether the measurement result is consistent with the previous measurement result of the top-3 beams. If so, the measurement result is not stored. If not, the measurement result is stored.

[0607] After completing each T ms measurement, the UE reports the stored measurement results and indicates which measurements within the T ms the UE reports. For example, if the UE completes 100 measurements of the top-3 beams within T ms and ultimately reports 30 measurement results, the base station can use a bitmap to indicate which 30 of the 100 measurements the UE reports.

[0608] If the UE already has PUSCH resources, the UE reports on the existing PUSCH resources, such as the configured grant uplink resources; if the UE does not have PUSCH resources, the UE requests PUSCH resources through SR. After the base station schedules PUSCH resources, the UE reports the stored measurement results on the PUSCH resources scheduled by the base station.

[0609] After receiving the 30 measurement results reported by the UE, the base station selects the model inference results corresponding to these 30 measurement results from the stored model inference results. Based on these 30 measurement results and the inference results, the difference between the average top-3 beam L1-RSRP predicted value and the measured value is calculated to determine the performance of the AI / ML model. For example, if the average absolute value of the difference between the 30 top-3 beam L1-RSRP predicted value and the measured value is greater than 1dB, the current AI model performance is considered to have deteriorated and the AI ​​model is deactivated.

[0610] Example 3 (corresponding to scheme 4):

[0611] In this embodiment, an AI / ML model is deployed on the UE for spatial prediction. The AI / ML model predicts the optimal beam in SetA based on the measurement results of SetB. The AI / ML model input is the L1-RSRP of SetB, and the output is the predicted RSRP of all beams in SetA. SetA includes 32 transmit beams, SetB includes 8 transmit beams, and SetB is a subset of SetA.

[0612] The base station periodically transmits reference signals corresponding to the SetB and SetA beams, and the reference signal period corresponding to the SetA beam can be different from the reference signal period corresponding to the SetB beam. The UE measures the L1-RSRP of the SetB reference signal. The AI / ML model uses the L1-RSRP of SetB as input for inference, and outputs the predicted top-3 beams in the SetA beam set. The UE feeds back the index value corresponding to the inferred top-3 beams to the base station. The UE measures the L1-RSRP of the SetA reference signal, determines the actual top-1 beam based on the SetA measurement results, and determines whether the top-3 beams inferred by the corresponding model include the actual top-1 beam. The index of the reference signal with the largest L1-RSRP measurement value in the SetA beam reference signal is the index value corresponding to the actual top-1 beam.

[0613] Every Tms, the UE reports back to the base station the measurement results of the top-3 beams for that Tms. The model-inferred top-3 beams do not include the number of times the actual top-1 beam was detected. If the UE already has PUSCH resources, it reports on the existing PUSCH resources, such as configured uplink grant resources. If the UE does not have PUSCH resources, it requests PUSCH resources through SR. After the base station schedules PUSCH resources, the UE reports on the PUSCH resources scheduled by the base station.

[0614] The base station receives the number of times within Tms reported by the UE that the UE-side model's predicted Top-3 beams do not include the actual Top-1 beam. The base station calculates the model's prediction accuracy. For example, if the UE reports 30 times and 100 SetA measurements are completed within Tms, the accuracy is 70%. The base station determines that the model accuracy is below a threshold (e.g., 80%) and deactivates the current AI model.

[0615] Example 4 (corresponding to Scheme 5):

[0616] In this embodiment, an AI / ML model is deployed on the UE for spatial prediction. The AI / ML model predicts the optimal beam in SetA based on the measurement results of SetB beams. The AI / ML model input is the L1-RSRP of SetB beams, and the model output is the index value corresponding to the top-two beams in SetA. SetA includes 32 transmit beams, and SetB includes 8 transmit beams.

[0617] The base station instructs the UE to start performance monitoring, such as instructing the UE to start a timer for performance monitoring. The performance monitoring indicator is the Top-K / 1 beam prediction accuracy (the probability that the predicted Top-K beam includes the actual Top-1 beam). The base station periodically sends the reference signals corresponding to the SetB and SetA beams. The UE measures the L1-RSRP of the SetB reference signal, uses the L1-RSRP of SetB as the model input for inference, and outputs the index value corresponding to the Top-2 beam of SetA. The UE measures the reference signal corresponding to the SetA beam, and the index of the reference signal with the largest L1-RSRP measurement value among the SetA beam reference signals is the index value corresponding to the actual Top-1 beam. The UE determines whether the index value corresponding to the Top-2 beam predicted by the model includes the index value corresponding to the actual Top-1 beam. If not, the UE side counts 1 time. Before the performance monitoring timer expires (the timer expiration period is determined by the base station configuration or protocol), if the model-inferred top-2 beams do not include the actual top-1 beam for a cumulative total of 10 times, the UE reports to the MAC-CE, informing the base station that the model-inferred top-2 beams have not included the actual top-1 beam for 10 times, as shown in Figure 5.

[0618] If multiple AI / ML models for spatial beam prediction are deployed on the UE side, and during performance monitoring, the UE not only monitors the performance of the currently activated AI / ML model but also tests the performance of other inactivated models for spatial beam prediction, then the IDs of other inactivated AI / ML models can be reported in the MAC-CE. The performance of the reported inactivated AI / ML model must meet the Top-2 / 1 beam prediction accuracy requirement greater than the threshold or be better than the activated AI / ML model within the monitoring time. After receiving the MAC-CE reported by the UE, the base station determines the performance of the AI / ML model and decides whether to deactivate the current model and whether to activate other AI / ML models reported in the MAC-CE that meet the Top-2 / 1 beam prediction accuracy threshold requirement.

[0619] Example 5 (corresponding to Scheme 6):

[0620] In this embodiment, an AI / ML model is deployed on the UE side for airspace prediction, and the assumptions about the input, output, and performance monitoring indicators of the AI / ML model are the same as those in Example 1.

[0621] The base station periodically transmits the reference signal corresponding to the SetB beam. The UE measures the L1-RSRP of the SetB reference signal and infers it to obtain the predicted L1-RSRP for the 32 beams of SetA. The three beams with the largest predicted L1-RSRP values ​​are designated as the top-3 beams. The UE feeds back the index values ​​corresponding to the top-3 beams to the base station. For certain inference results, the base station transmits the reference signal corresponding to the top-3 beams inferred by the AI / ML model, and the UE measures the L1-RSRP of the top-3 beams. After each L1-RSRP measurement of the top-3 beams, the UE determines whether the current measurement result is consistent with the previous measurement result of the top-3 beams. If the result is consistent, the measurement result is not stored. If the result is inconsistent, the measurement result is stored. If storing a measurement result would exceed the UE's storage capacity, the UE discards the oldest measurement result and stores the current measurement result.

[0622] The base station triggers the UE to perform non-periodic reporting through MAC-CE signaling. The reported content is the L1-RSRP prediction value and measurement value of the top-3 beams inferred by the model stored by the UE. The reported content is carried by MAC-CE signaling or Radio Resource Control (RRC) signaling. If the UE already has PUSCH resources, the UE reports on the existing PUSCH resources, such as the configured grant uplink resources; if the UE does not have PUSCH resources, the UE requests PUSCH resources through SR. After the base station schedules PUSCH resources, the UE reports on the PUSCH resources scheduled by the base station.

[0623] After receiving the results reported by the UE, the base station calculates the difference between the average top-3 beam L1-RSRP predicted value and the measured value to determine the performance of the AI / ML model. The base station can trigger aperiodic reporting on demand. For example, when network performance indicators are good, the base station may not trigger reporting. When network performance indicators degrade, the base station can trigger aperiodic reporting to monitor the performance of the AI / ML model over a recent period of time and determine whether the AI / ML model is causing the network performance degradation.

[0624] The embodiments of the present disclosure focus on how the UE reports multiple performance monitoring related information to assist the base station side in determining the performance of the base station side model or the UE side model. The embodiments of the present disclosure propose that the UE feeds back multiple performance monitoring related information (such as measurement results of multiple measurements) together, which can reduce the feedback overhead of the UE. The embodiments of the present disclosure also propose that the performance monitoring related information reported by the UE can be obtained after screening according to certain criteria, which can further reduce the feedback overhead of the UE and reduce the computing resource overhead of the base station. The embodiments of the present disclosure also propose that the base station selects the corresponding information from the performance monitoring related information reported by the UE to calculate the performance monitoring indicators. The embodiments of the present disclosure also propose that the base station decides whether to trigger the UE to report the performance monitoring related information non-periodically based on whether the monitored system performance indicators have deteriorated, and then determines whether the system performance deterioration is caused by the degradation of model performance.

[0625] In summary, the embodiments of the present disclosure can reduce unnecessary overhead while ensuring the completion of the performance monitoring function, thereby improving the performance of the model applied to the communication network.

[0626] Please refer to Figure 6, which is a flow chart of a performance monitoring method provided by an embodiment of the present disclosure. As shown in Figure 6, the performance monitoring method includes the following steps:

[0627] Step 601: The network device receives monitoring-related information of the first model reported by the terminal;

[0628] Step 602: The network device performs performance monitoring of the first model based on the monitoring-related information.

[0629] In some embodiments, the method further comprises:

[0630] The network device sends first information to the terminal, where the first information is used to indicate at least one of the following:

[0631] The monitoring and reporting period of the first model;

[0632] The monitoring start time of the first model;

[0633] The prediction result of the first model needs to meet the first preset condition.

[0634] In some embodiments, the reporting condition is determined by predefined performance monitoring related parameters, and the performance monitoring related parameters include at least one of the following:

[0635] The monitoring and reporting period of the first model;

[0636] The monitoring start time of the first model;

[0637] The prediction result of the first model needs to meet the first preset condition.

[0638] In some embodiments, the method further comprises:

[0639] The network device sends reporting trigger information to the terminal, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model.

[0640] In some embodiments, the network device sending the reporting trigger information to the terminal includes:

[0641] When the network device monitors that the system performance indicator meets the second preset condition, the network device sends reporting trigger information to the terminal.

[0642] In some embodiments, the first preset condition includes at least one of the following:

[0643] The number of times the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal;

[0644] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, where the actual optimal K beams are determined based on a measurement result of the terminal;

[0645] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

[0646] In some embodiments, the network device performs performance monitoring of the first model based on the monitoring-related information, including:

[0647] The network device calculates the monitoring index of the first model based on the monitoring-related information.

[0648] In some embodiments, when the first model is deployed on the network device, the monitoring-related information includes measurement results of the terminal performing N measurements on the reference signal, where N is an integer greater than 1;

[0649] The network device calculates the monitoring index of the first model based on the monitoring-related information, including:

[0650] The network device selects N predictions corresponding to the N measurements from all predictions of the first model;

[0651] The network device calculates the monitoring indicator of the first model according to the prediction results of the N predictions and the measurement results of the N measurements;

[0652] or,

[0653] The network device calculates the monitoring index of the first model based on the monitoring-related information, including:

[0654] The network device selects M measurements from the N measurements, where M is an integer greater than 1;

[0655] The network device selects M predictions corresponding to the M measurements from all predictions of the first model;

[0656] The network device calculates the monitoring indicator of the first model according to the prediction results of the M predictions and the measurement results of the M measurements.

[0657] In some embodiments, the M measurements filtered out by the network device from the N measurements include at least one of the following:

[0658] The first measurement within the monitoring and reporting period;

[0659] The first measurement from the monitoring start time;

[0660] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0661] a measurement in which the difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold;

[0662] a measurement in which the difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than a fifth threshold;

[0663] a measurement in which the difference between the average value of the measurement results and the average value of the measurement results of the previous measurement is greater than a sixth threshold;

[0664] a measurement in which the difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than a seventh threshold;

[0665] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0666] In some embodiments, when the first model is deployed on the terminal, the monitoring-related information includes at least one of the following:

[0667] measurement results of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1;

[0668] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0669] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0670] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0671] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0672] the difference, the absolute value of the difference, or the average of the absolute values ​​of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the actual layer 1 reference signal received power values ​​of the optimal K beams;

[0673] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the actual layer 1 reference signal received power value of the optimal beam, or an absolute value of the difference, or an average of the absolute values ​​of the difference;

[0674] Identification information of the first model.

[0675] In some embodiments, the network device receives the monitoring-related information of the first model reported by the terminal, including:

[0676] The network device receives, through a first resource, monitoring-related information of the first model reported by the terminal;

[0677] The first resource includes any one of the following:

[0678] Resources configured by the network device for reporting monitoring-related information of the first model;

[0679] Physical uplink shared channel PUSCH resources already available to the terminal;

[0680] The terminal requests PUSCH resources through a scheduling request SR.

[0681] In some embodiments, the method further comprises:

[0682] The network device performs a first process according to the monitoring indicator of the first model, where the first process includes any one of the following:

[0683] activating the first model;

[0684] Deactivate the first model.

[0685] It should be noted that this embodiment is an implementation of the network device or base station corresponding to the embodiments shown in Figures 4 to 5. Its specific implementation can refer to the relevant descriptions of the embodiments shown in Figures 4 to 5. In order to avoid repetition, this embodiment will not be repeated, and the same beneficial effects can still be achieved.

[0686] Please refer to Figure 7, which is a structural diagram of a terminal provided by an embodiment of the present disclosure. As shown in Figure 7, it includes a memory 720, a transceiver 700 and a processor 710:

[0687] The memory 720 is used to store computer programs; the transceiver 700 is used to send and receive data under the control of the processor 710; the processor 710 is used to read the computer program in the memory 720 and perform the following operations:

[0688] Determining whether a reporting condition for monitoring-related information of the first model is met;

[0689] When the reporting condition is met, the monitoring-related information of the first model is reported to the network device.

[0690] In the disclosed embodiment, the terminal determines whether a reporting condition for monitoring-related information of a first model is satisfied; if the reporting condition is satisfied, the terminal reports the monitoring-related information of the first model to the network device. Through the above process, if the reporting condition for monitoring-related information of the first model is satisfied, the terminal reports the monitoring-related information of the first model to the network device. In this way, the terminal can report multiple performance monitoring-related information to the network device at the same time, which can reduce the reporting overhead of the terminal during the performance monitoring process.

[0691] In FIG7 , the bus architecture may include any number of interconnected buses and bridges, specifically various circuits connected together by one or more processors represented by processor 710 and memory represented by memory 720. The bus architecture may also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 700 may be a plurality of components, including a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium, such as a wireless channel, a wired channel, an optical cable, and the like. For different user devices, the user interface 730 may also be an interface capable of connecting external or internal devices as required, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, and the like.

[0692] The processor 710 is responsible for managing the bus architecture and general processing, and the memory 720 can store data used by the processor 710 when performing operations.

[0693] In some embodiments, the processor 710 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or a complex programmable logic device (CPLD). The processor may also adopt a multi-core architecture.

[0694] The processor calls the computer program stored in the memory to execute any of the methods provided by the embodiments of the present disclosure according to the obtained executable instructions. The processor and the memory can also be arranged physically separately.

[0695] In some embodiments, the processor 710 is configured to read the computer program in the memory and perform the following operations:

[0696] Determine, based on first information sent by the network device, a reporting condition for monitoring-related information of the first model, where the first information is used to indicate at least one of the following:

[0697] The monitoring and reporting period of the first model;

[0698] The monitoring start time of the first model;

[0699] The prediction result of the first model needs to meet the first preset condition.

[0700] In some embodiments, the processor 710 is configured to read the computer program in the memory and perform the following operations:

[0701] Determine a reporting condition for monitoring-related information of the first model based on predefined performance monitoring-related parameters, where the performance monitoring-related parameters include at least one of the following:

[0702] The monitoring and reporting period of the first model;

[0703] The monitoring start time of the first model;

[0704] The prediction result of the first model needs to meet the first preset condition.

[0705] In some embodiments, the reporting condition includes at least one of the following:

[0706] The reporting time of the monitoring reporting cycle of the first model arrives;

[0707] The terminal receives reporting trigger information from the network device, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model;

[0708] The terminal detects that the prediction result of the first model meets a first preset condition.

[0709] In some embodiments, the first preset condition includes at least one of the following:

[0710] The number of times the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal;

[0711] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, where the actual optimal K beams are determined based on a measurement result of the terminal;

[0712] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

[0713] In some embodiments, the monitoring-related information reported by the terminal includes at least one of the following:

[0714] measurement results of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1;

[0715] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0716] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0717] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0718] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0719] the difference, the absolute value of the difference, or the average of the absolute values ​​of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the actual layer 1 reference signal received power values ​​of the optimal K beams;

[0720] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the actual layer 1 reference signal received power value of the optimal beam, or an absolute value of the difference, or an average of the absolute values ​​of the difference;

[0721] Identification information of the first model.

[0722] In some embodiments, the N measurements are all measurements; or, N measurements selected from all measurements; or, N measurements corresponding to N predictions selected from all predictions of the first model.

[0723] In some embodiments, the processor 710 is configured to read the computer program in the memory and perform the following operations:

[0724] The measurement order of the N measurements in all measurements is indicated by a bitmap, or the measurement identification information of the N measurements is indicated.

[0725] In some embodiments, the N measurements selected from all measurements include at least one of the following:

[0726] The first measurement within the monitoring and reporting period of the first model;

[0727] The first measurement from the monitoring start time of the first model;

[0728] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0729] a measurement in which the difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold;

[0730] a measurement in which the difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than a fifth threshold;

[0731] a measurement in which the difference between the average value of the measurement results and the average value of the measurement results of the previous measurement is greater than a sixth threshold;

[0732] a measurement in which the difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than a seventh threshold;

[0733] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0734] In some embodiments, the N predictions selected from all predictions of the first model include at least one of the following:

[0735] The first forecast within the monitoring and reporting period;

[0736] The first prediction from the monitoring start time;

[0737] A forecast whose predicted result is different from the predicted result of the previous forecast;

[0738] The difference between the maximum value of the predicted result and the maximum value of the predicted result of the previous prediction is greater than the ninth threshold;

[0739] The difference between the minimum value of the predicted result and the minimum value of the predicted result of the previous prediction is greater than the tenth threshold;

[0740] The difference between the average value of the predicted results and the average value of the predicted results of the previous prediction is greater than the eleventh threshold;

[0741] The difference between the standard deviation of the predicted result and the standard deviation of the predicted result of the previous prediction is greater than the twelfth threshold;

[0742] The difference between the prediction result variance and the prediction result variance of the previous prediction is greater than the prediction of the thirteenth threshold.

[0743] In some embodiments, the processor 710 is configured to read the computer program in the memory and perform the following operations:

[0744] reporting the monitoring-related information of the first model to the network device through the first resource;

[0745] The first resource includes any one of the following:

[0746] Resources configured by the network device for reporting monitoring-related information of the first model;

[0747] Physical uplink shared channel PUSCH resources already available to the terminal;

[0748] The terminal requests PUSCH resources through a scheduling request SR.

[0749] In some embodiments, the processor 710 is configured to read the computer program in the memory and perform the following operations:

[0750] When storing the measurement result of the Lth measurement, if the required storage space exceeds the remaining storage space of the terminal, the earliest stored measurement result is discarded and the measurement result of the Lth measurement is stored.

[0751] It should be noted here that the above-mentioned terminal provided in the embodiment of the present disclosure can implement all the method steps implemented in the above-mentioned terminal-side method embodiment, and can achieve the same technical effects. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0752] Please refer to Figure 8, which is a structural diagram of a network device provided by an embodiment of the present disclosure. As shown in Figure 8, it includes a memory 820, a transceiver 800 and a processor 810:

[0753] The memory 820 is used to store computer programs; the transceiver 800 is used to send and receive data under the control of the processor 810; the processor 810 is used to read the computer program in the memory 820 and perform the following operations:

[0754] receiving monitoring-related information of the first model reported by the terminal;

[0755] Based on the monitoring-related information, performance monitoring of the first model is performed.

[0756] In FIG8 , the bus architecture may include any number of interconnected buses and bridges, specifically various circuits linked together by one or more processors represented by processor 810 and memory represented by memory 820. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 800 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium, such as a wireless channel, a wired channel, an optical cable, and the like. The processor 810 is responsible for managing the bus architecture and general processing, and the memory 820 may store data used by the processor 810 when performing operations.

[0757] In some embodiments, the processor 810 can be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD), and the processor can also adopt a multi-core architecture.

[0758] The processor calls the computer program stored in the memory to execute any of the methods provided by the embodiments of the present disclosure according to the obtained executable instructions. The processor and the memory can also be arranged physically separately.

[0759] In some embodiments, the processor 810 is configured to read the computer program in the memory and perform the following operations:

[0760] Sending first information to the terminal, where the first information is used to indicate at least one of the following:

[0761] The monitoring and reporting period of the first model;

[0762] The monitoring start time of the first model;

[0763] The prediction result of the first model needs to meet the first preset condition.

[0764] In some embodiments, the reporting condition is determined by predefined performance monitoring related parameters, and the performance monitoring related parameters include at least one of the following:

[0765] The monitoring and reporting period of the first model;

[0766] The monitoring start time of the first model;

[0767] The prediction result of the first model needs to meet the first preset condition.

[0768] In some embodiments, the processor 810 is configured to read the computer program in the memory and perform the following operations:

[0769] Sending reporting trigger information to the terminal, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model.

[0770] In some embodiments, the processor 810 is configured to read the computer program in the memory and perform the following operations:

[0771] When it is monitored that the system performance indicator meets the second preset condition, reporting trigger information is sent to the terminal.

[0772] In some embodiments, the first preset condition includes at least one of the following:

[0773] The number of times the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal;

[0774] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, where the actual optimal K beams are determined based on a measurement result of the terminal;

[0775] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

[0776] In some embodiments, the processor 810 is configured to read the computer program in the memory and perform the following operations:

[0777] Based on the monitoring-related information, a monitoring indicator of the first model is calculated.

[0778] In some embodiments, when the first model is deployed on the network device, the monitoring-related information includes measurement results of the terminal performing N measurements on the reference signal, where N is an integer greater than 1;

[0779] The processor 810 is configured to read the computer program in the memory and perform the following operations:

[0780] Filtering N predictions corresponding to the N measurements from all predictions of the first model;

[0781] Calculating a monitoring indicator of the first model according to the prediction results of the N predictions and the measurement results of the N measurements;

[0782] or,

[0783] Filtering M measurements from the N measurements, where M is an integer greater than 1;

[0784] Filtering M predictions corresponding to the M measurements from all predictions of the first model;

[0785] The monitoring index of the first model is calculated according to the prediction results of the M predictions and the measurement results of the M measurements.

[0786] In some embodiments, the M measurements selected from the N measurements include at least one of the following:

[0787] The first measurement within the monitoring and reporting period;

[0788] The first measurement from the monitoring start time;

[0789] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0790] a measurement in which the difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold;

[0791] a measurement in which the difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than a fifth threshold;

[0792] a measurement in which the difference between the average value of the measurement results and the average value of the measurement results of the previous measurement is greater than a sixth threshold;

[0793] a measurement in which the difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than a seventh threshold;

[0794] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0795] In some embodiments, when the first model is deployed on the terminal, the monitoring-related information includes at least one of the following:

[0796] measurement results of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1;

[0797] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0798] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0799] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0800] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0801] the difference, the absolute value of the difference, or the average of the absolute values ​​of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the actual layer 1 reference signal received power values ​​of the optimal K beams;

[0802] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the actual layer 1 reference signal received power value of the optimal beam, or an absolute value of the difference, or an average of the absolute values ​​of the difference;

[0803] Identification information of the first model.

[0804] In some embodiments, the processor 810 is configured to read the computer program in the memory and perform the following operations:

[0805] receiving, through a first resource, monitoring-related information of the first model reported by the terminal;

[0806] The first resource includes any one of the following:

[0807] Resources configured by the network device for reporting monitoring-related information of the first model;

[0808] Physical uplink shared channel PUSCH resources already available to the terminal;

[0809] The terminal requests PUSCH resources through a scheduling request SR.

[0810] In some embodiments, the processor 810 is configured to read the computer program in the memory and perform the following operations:

[0811] Performing a first process according to the monitoring indicator of the first model, where the first process includes any one of the following:

[0812] activating the first model;

[0813] Deactivate the first model.

[0814] It should be noted here that the above-mentioned network device provided in the embodiment of the present disclosure can implement all the method steps implemented in the above-mentioned network device side method embodiment, and can achieve the same technical effect. The parts and beneficial effects that are the same as the method embodiment in this embodiment will not be described in detail here.

[0815] Please refer to FIG9 , which is a structural diagram of a terminal provided in an embodiment of the present disclosure. As shown in FIG9 , the terminal 900 includes:

[0816] A first determining module 901 is configured to determine whether a reporting condition for monitoring-related information of a first model is met;

[0817] The reporting module 902 is configured to report the monitoring-related information of the first model to the network device when the reporting conditions are met.

[0818] In some embodiments, terminal 900 further includes:

[0819] A second determining module is configured to determine a reporting condition for monitoring-related information of the first model based on first information sent by the network device, where the first information is used to indicate at least one of the following:

[0820] The monitoring and reporting period of the first model;

[0821] The monitoring start time of the first model;

[0822] The prediction result of the first model needs to meet the first preset condition.

[0823] In some embodiments, terminal 900 further includes:

[0824] A third determining module is configured to determine a reporting condition for monitoring-related information of the first model based on predefined performance monitoring-related parameters, where the performance monitoring-related parameters include at least one of the following:

[0825] The monitoring and reporting period of the first model;

[0826] The monitoring start time of the first model;

[0827] The prediction result of the first model needs to meet the first preset condition.

[0828] In some embodiments, the reporting condition includes at least one of the following:

[0829] The reporting time of the monitoring reporting cycle of the first model arrives;

[0830] The terminal 900 receives reporting trigger information from the network device, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model;

[0831] The terminal 900 detects that the prediction result of the first model meets a first preset condition.

[0832] In some embodiments, the first preset condition includes at least one of the following:

[0833] The number of times that the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal 900;

[0834] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, where the actual optimal K beams are determined based on a measurement result of the terminal 900;

[0835] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal 900 is greater than or equal to the third threshold.

[0836] In some embodiments, the monitoring-related information reported by the terminal 900 includes at least one of the following:

[0837] measurement results of N measurements performed by terminal 900 on the reference signal, where N is an integer greater than 1;

[0838] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0839] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0840] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0841] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0842] the difference, the absolute value of the difference, or the average of the absolute values ​​of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the actual layer 1 reference signal received power values ​​of the optimal K beams;

[0843] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the actual layer 1 reference signal received power value of the optimal beam, or an absolute value of the difference, or an average of the absolute values ​​of the difference;

[0844] Identification information of the first model.

[0845] In some embodiments, the N measurements are all measurements; or, N measurements selected from all measurements; or, N measurements corresponding to N predictions selected from all predictions of the first model.

[0846] In some embodiments, the terminal 900 indicates the measurement order of the N measurements in all measurements through a bitmap, or indicates measurement identification information of the N measurements.

[0847] In some embodiments, the N measurements selected from all measurements include at least one of the following:

[0848] The first measurement within the monitoring and reporting period of the first model;

[0849] The first measurement from the monitoring start time of the first model;

[0850] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0851] a measurement in which the difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold;

[0852] a measurement in which the difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than a fifth threshold;

[0853] a measurement in which the difference between the average value of the measurement results and the average value of the measurement results of the previous measurement is greater than a sixth threshold;

[0854] a measurement in which the difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than a seventh threshold;

[0855] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0856] In some embodiments, the N predictions selected from all predictions of the first model include at least one of the following:

[0857] The first forecast within the monitoring and reporting period;

[0858] The first prediction from the monitoring start time;

[0859] A forecast whose predicted result is different from the predicted result of the previous forecast;

[0860] The difference between the maximum value of the predicted result and the maximum value of the predicted result of the previous prediction is greater than the ninth threshold;

[0861] The difference between the minimum value of the predicted result and the minimum value of the predicted result of the previous prediction is greater than the tenth threshold;

[0862] The difference between the average value of the predicted results and the average value of the predicted results of the previous prediction is greater than the eleventh threshold;

[0863] The difference between the standard deviation of the predicted result and the standard deviation of the predicted result of the previous prediction is greater than the twelfth threshold;

[0864] The difference between the prediction result variance and the prediction result variance of the previous prediction is greater than the prediction of the thirteenth threshold.

[0865] In some embodiments, the reporting module 902 is specifically configured to:

[0866] reporting the monitoring-related information of the first model to the network device through the first resource;

[0867] The first resource includes any one of the following:

[0868] Resources configured by the network device for reporting monitoring-related information of the first model;

[0869] Physical uplink shared channel PUSCH resources already available on the terminal 900;

[0870] The terminal 900 requests PUSCH resources through a scheduling request SR.

[0871] In some embodiments, terminal 900 further includes:

[0872] The processing module is configured to discard the earliest stored measurement result and store the measurement result of the Lth measurement if the required storage space exceeds the remaining storage space of the terminal 900 when storing the measurement result of the Lth measurement.

[0873] It should be noted here that the above-mentioned terminal provided in the embodiment of the present disclosure can implement all the method steps implemented in the above-mentioned terminal-side method embodiment, and can achieve the same technical effects. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0874] Please refer to FIG10 , which is a structural diagram of a network device provided in an embodiment of the present disclosure. As shown in FIG10 , the network device 1000 includes:

[0875] Receiving module 1001, configured to receive monitoring-related information of a first model reported by a terminal;

[0876] The monitoring module 1002 is configured to perform performance monitoring of the first model based on the monitoring-related information.

[0877] In some embodiments, the network device 1000 further includes:

[0878] A first sending module is configured to send first information to the terminal, where the first information is used to indicate at least one of the following:

[0879] The monitoring and reporting period of the first model;

[0880] The monitoring start time of the first model;

[0881] The prediction result of the first model needs to meet the first preset condition.

[0882] In some embodiments, the reporting condition is determined by predefined performance monitoring related parameters, and the performance monitoring related parameters include at least one of the following:

[0883] The monitoring and reporting period of the first model;

[0884] The monitoring start time of the first model;

[0885] The prediction result of the first model needs to meet the first preset condition.

[0886] In some embodiments, the network device 1000 further includes:

[0887] The second sending module is used to send reporting trigger information to the terminal, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model.

[0888] In some embodiments, the network device 1000 further includes:

[0889] The third sending module is used to send reporting trigger information to the terminal when it is monitored that the system performance indicator meets the second preset condition.

[0890] In some embodiments, the first preset condition includes at least one of the following:

[0891] The number of times the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal;

[0892] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, where the actual optimal K beams are determined based on a measurement result of the terminal;

[0893] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

[0894] In some embodiments, the monitoring module 1002 includes:

[0895] A calculation unit is used to calculate the monitoring index of the first model based on the monitoring related information.

[0896] In some embodiments, when the first model is deployed on the network device 1000, the monitoring-related information includes measurement results of the terminal performing N measurements on the reference signal, where N is an integer greater than 1;

[0897] The calculation unit includes:

[0898] a first screening subunit, configured to screen out N predictions corresponding to the N measurements from all predictions of the first model;

[0899] A first calculation subunit, configured to calculate a monitoring indicator of the first model based on the prediction results of the N predictions and the measurement results of the N measurements;

[0900] or,

[0901] The calculation unit includes:

[0902] A second screening subunit, configured to screen out M measurements from the N measurements, where M is an integer greater than 1;

[0903] a third screening subunit, configured to screen out M predictions corresponding to the M measurements from all predictions of the first model;

[0904] The second calculation subunit is used to calculate the monitoring index of the first model according to the prediction results of the M predictions and the measurement results of the M measurements.

[0905] In some embodiments, the M measurements selected from the N measurements include at least one of the following:

[0906] The first measurement within the monitoring and reporting period;

[0907] The first measurement from the monitoring start time;

[0908] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0909] a measurement in which the difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold;

[0910] a measurement in which the difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than a fifth threshold;

[0911] a measurement in which the difference between the average value of the measurement results and the average value of the measurement results of the previous measurement is greater than a sixth threshold;

[0912] a measurement in which the difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than a seventh threshold;

[0913] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0914] In some embodiments, when the first model is deployed on the terminal, the monitoring-related information includes at least one of the following:

[0915] measurement results of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1;

[0916] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0917] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0918] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0919] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0920] the difference, the absolute value of the difference, or the average of the absolute values ​​of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the actual layer 1 reference signal received power values ​​of the optimal K beams;

[0921] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the actual layer 1 reference signal received power value of the optimal beam, or an absolute value of the difference, or an average of the absolute values ​​of the difference;

[0922] Identification information of the first model.

[0923] In some embodiments, the receiving module 1001 is specifically configured to:

[0924] receiving, through a first resource, monitoring-related information of the first model reported by the terminal;

[0925] The first resource includes any one of the following:

[0926] Resources configured by the network device 1000 for reporting monitoring-related information of the first model;

[0927] Physical uplink shared channel PUSCH resources already available to the terminal;

[0928] The terminal requests PUSCH resources through a scheduling request SR.

[0929] In some embodiments, the network device 1000 further includes:

[0930] A processing module is configured to perform a first process according to the monitoring indicator of the first model, where the first process includes any one of the following:

[0931] activating the first model;

[0932] Deactivate the first model.

[0933] It should be noted here that the above-mentioned network device provided in the embodiment of the present disclosure can implement all the method steps implemented in the above-mentioned network device side method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0934] It should be noted that the division of units in the embodiments of the present disclosure is schematic and is merely a logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of the present disclosure may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0935] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0936] An embodiment of the present disclosure also provides a processor-readable storage medium, which stores a computer program, and the computer program is used to enable the processor to execute the terminal-side performance monitoring method provided by the embodiment of the present disclosure, or the computer program is used to enable the processor to execute the network device-side performance monitoring method provided by the embodiment of the present disclosure.

[0937] The processor-readable storage medium may be a non-transitory processor-readable storage medium.

[0938] The processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disk, hard disk, magnetic tape, magneto-optical disk (MO)), etc.), optical storage (such as compact disk (CD), digital video disc (DVD), Blu-ray disc (BD), high-definition versatile disc (HVD), etc.), and semiconductor memory (such as read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), non-volatile memory (NAND FLASH), solid-state drive (SSD)), etc.

[0939] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0940] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer-executable instructions. These computer-executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0941] These processor-executable instructions may also be stored in a processor-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the processor-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0942] These processor-executable instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0943] It should be noted that it should be understood that the division of the above modules is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. Moreover, these modules can all be implemented in the form of software called by a processing element; or they can all be implemented in the form of hardware; or some modules can be implemented in the form of software called by a processing element, and some modules can be implemented in the form of hardware. For example, the determination module can be a separately established processing element, or it can be integrated into a chip of the above-mentioned device. In addition, it can also be stored in the memory of the above-mentioned device in the form of program code, and called by a processing element of the above-mentioned device to perform the functions of the above-mentioned determination module. The implementation of other modules is similar. In addition, these modules can all or partly be integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed by the hardware integrated logic circuit in the processor element or by instructions in the form of software.

[0944] For example, each module, unit, sub-unit or sub-module may be one or more integrated circuits configured to implement the above method, such as one or more application-specific integrated circuits (ASICs), one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented by scheduling program code through a processing element, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0945] The terms "first," "second," and the like in the specification and claims of the present disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present disclosure described herein may be implemented in a sequence other than that illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units need not be limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or devices. In addition, the use of "and / or" in the specification and claims to indicate at least one of the connected objects, for example, A and / or B and / or C, means that seven situations are included: A alone, B alone, C alone, both A and B present, both B and C present, both A and C present, and all A, B, and C present. Similarly, the use of "at least one of A and B" in the specification and claims should be understood to mean "A alone, B alone, or both A and B present."

[0946] Obviously, those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include these modifications and variations.

Claims

1. A performance monitoring method, comprising: The terminal determines whether a reporting condition of monitoring-related information of the first model is met; When the reporting condition is met, the terminal reports the monitoring-related information of the first model to the network device.

2. The method according to claim 1, further comprising: The terminal determines, according to first information sent by the network device, a reporting condition for monitoring-related information of the first model, where the first information is used to indicate at least one of the following: The monitoring and reporting period of the first model; The monitoring start time of the first model; The prediction result of the first model needs to meet a first preset condition.

3. The method according to claim 1, further comprising: The terminal determines, according to predefined performance monitoring related parameters, a reporting condition for monitoring related information of the first model, where the performance monitoring related parameters include at least one of the following: The monitoring and reporting period of the first model; The monitoring start time of the first model; The prediction result of the first model needs to meet a first preset condition.

4. The method according to any one of claims 1 to 3, wherein: The reporting conditions include at least one of the following: The reporting time of the monitoring reporting cycle of the first model arrives; The terminal receives reporting trigger information from the network device, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model; The terminal detects that the prediction result of the first model meets a first preset condition.

5. The method according to claim 2 or 3, wherein: The first preset condition includes at least one of the following: The number of times that the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal; The number of times that the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, and the actual optimal K beams are determined based on the measurement result of the terminal; The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

6. The method according to any one of claims 1 to 3, wherein: The monitoring related information reported by the terminal includes at least one of the following: a measurement result of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1; prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements; The difference between the measurement results of the N measurements and the prediction results of the N predictions; The number of times the optimal K beams predicted by the first model do not include the actual optimal beam; The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams; a difference or an absolute value of the difference or an average value of the absolute value of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the actual optimal K beams; a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the layer 1 reference signal received power value of the actual optimal beam, or an absolute value of the difference, or an average value of the absolute values ​​of the difference; Identification information of the first model.

7. The method according to claim 6, wherein: The N measurements are all measurements of the terminal; or, N measurements selected by the terminal from all measurements; or, N measurements corresponding to N predictions selected by the terminal from all predictions of the first model.

8. The method according to claim 7, wherein: The terminal indicates the measurement order of the N measurements in all measurements through a bit map, or indicates measurement identification information of the N measurements.

9. The method according to claim 7, wherein: The N measurements screened by the terminal from all measurements include at least one of the following: The first measurement within the monitoring and reporting period of the first model; The first measurement of the first model from the start of monitoring; A measurement in which the result of a measurement differs from the result of a previous measurement; The difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold value; The difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than the fifth threshold value; A measurement in which the difference between the average value of the measurement result and the average value of the measurement result of the previous measurement is greater than a sixth threshold; The difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than the seventh threshold value; A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

10. The method according to claim 7, wherein: The N predictions selected by the terminal from all predictions of the first model include at least one of the following: The first forecast within the monitoring and reporting period; The first prediction from the start time of the monitoring; A forecast whose predicted result is different from the predicted result of the previous forecast; The prediction that the difference between the maximum value of the predicted result and the maximum value of the predicted result of the previous prediction is greater than the ninth threshold; The difference between the minimum value of the predicted result and the minimum value of the predicted result of the previous prediction is greater than the prediction of the tenth threshold; The difference between the average of the predicted results and the average of the predicted results of the previous prediction is greater than the eleventh threshold; The difference between the standard deviation of the predicted result and the standard deviation of the predicted result of the previous prediction is greater than the prediction of the twelfth threshold; The difference between the prediction result variance and the prediction result variance of the previous prediction is greater than the prediction of the thirteenth threshold.

11. The method according to any one of claims 1 to 3, wherein: The terminal reports the monitoring-related information of the first model to the network device, including: The terminal reports the monitoring related information of the first model to the network device through the first resource; The first resource includes any of the following: A resource configured by the network device for reporting monitoring-related information of the first model; Physical uplink shared channel PUSCH resources already available to the terminal; The terminal requests PUSCH resources via a scheduling request SR.

12. The method according to any one of claims 1 to 3, further comprising: When the terminal stores the measurement result of the Lth measurement, if the required storage space exceeds the remaining storage space of the terminal, the terminal discards the earliest stored measurement result and stores the measurement result of the Lth measurement.

13. A performance monitoring method, comprising: The network device receives monitoring related information of the first model reported by the terminal; The network device performs performance monitoring of the first model based on the monitoring-related information.

14. The method according to claim 13, further comprising: The network device sends first information to the terminal, where the first information is used to indicate at least one of the following: The monitoring and reporting period of the first model; The monitoring start time of the first model; The prediction result of the first model needs to meet a first preset condition.

15. The method according to claim 13 or 14, further comprising: The network device sends reporting trigger information to the terminal, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model.

16. The method according to claim 15, wherein: The network device sends reporting trigger information to the terminal, including: When the network device monitors that the system performance indicator meets the second preset condition, the network device sends reporting trigger information to the terminal.

17. The method according to claim 14, wherein: The first preset condition includes at least one of the following: The number of times that the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal; The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to At a second threshold, the actual optimal K beams are determined based on a measurement result of the terminal; The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

18. The method according to claim 13 or 14, wherein: The network device performs performance monitoring of the first model based on the monitoring related information, including: The network device calculates the monitoring index of the first model based on the monitoring related information.

19. The method according to claim 18, wherein: In the case where the first model is deployed on the network device, the monitoring-related information includes measurement results of the terminal measuring the reference signal N times, where N is an integer greater than 1; The network device calculates the monitoring index of the first model based on the monitoring related information, including: The network device selects N predictions corresponding to the N measurements from all predictions of the first model; The network device calculates the monitoring index of the first model according to the prediction results of the N predictions and the measurement results of the N measurements; or, The network device calculates the monitoring index of the first model based on the monitoring related information, including: The network device selects M measurements from the N measurements, where M is an integer greater than 1; The network device selects M predictions corresponding to the M measurements from all predictions of the first model; The network device calculates the monitoring index of the first model according to the prediction results of the M predictions and the measurement results of the M measurements.

20. The method according to claim 19, wherein: The M measurements screened out by the network device from the N measurements include at least one of the following: The first measurement within the monitoring and reporting period; The first measurement from the start time of the monitoring; A measurement in which the result of a measurement differs from the result of a previous measurement; The difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold value; The difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than the fifth threshold value; A measurement in which the difference between the average value of the measurement result and the average value of the measurement result of the previous measurement is greater than a sixth threshold; The difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than the seventh threshold value; A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

21. The method according to claim 13 or 14, wherein: In the case where the first model is deployed on the terminal, the monitoring-related information includes at least one of the following: a measurement result of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1; prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements; The difference between the measurement results of the N measurements and the prediction results of the N predictions; The number of times the optimal K beams predicted by the first model do not include the actual optimal beam; The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams; a difference or an absolute value of the difference or an average value of the absolute value of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the actual optimal K beams; a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the layer 1 reference signal received power value of the actual optimal beam, or an absolute value of the difference, or an average value of the absolute values ​​of the difference; Identification information of the first model.

22. The method according to claim 13 or 14, wherein: The network device receives the monitoring-related information of the first model reported by the terminal, including: The network device receives, through a first resource, monitoring-related information of the first model reported by the terminal; The first resource includes any of the following: A resource configured by the network device for reporting monitoring-related information of the first model; Physical uplink shared channel PUSCH resources already available to the terminal; The terminal requests PUSCH resources via a scheduling request SR.

23. The method according to claim 18, further comprising: The network device performs a first process according to the monitoring indicator of the first model, where the first process includes any one of the following: activating the first model; Deactivate the first model.

24. A terminal, comprising: A memory, a transceiver, and a processor, wherein: A memory for storing a computer program; a transceiver for transmitting and receiving data under the control of the processor; and a processor for reading the computer program in the memory and performing the following operations: Determining whether a reporting condition of monitoring-related information of the first model is met; When the reporting condition is met, the monitoring related information of the first model is reported to the network device.

25. The terminal according to claim 24, wherein: A processor is configured to read the computer program in the memory and perform the following operations: Determine, according to the first information sent by the network device, a reporting condition of the monitoring-related information of the first model, wherein the first information is used to indicate at least one of the following: The monitoring and reporting period of the first model; The monitoring start time of the first model; The prediction result of the first model needs to meet a first preset condition.

26. The terminal according to claim 24, wherein: A processor is configured to read the computer program in the memory and perform the following operations: Determine a reporting condition for monitoring-related information of the first model according to predefined performance monitoring-related parameters, where the performance monitoring-related parameters include at least one of the following: The monitoring and reporting period of the first model; The monitoring start time of the first model; The prediction result of the first model needs to meet a first preset condition.

27. The terminal according to any one of claims 24 to 26, wherein: The reporting conditions include at least one of the following: The reporting time of the monitoring reporting cycle of the first model arrives; The terminal receives reporting trigger information from the network device, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model; The terminal detects that the prediction result of the first model meets a first preset condition.

28. The terminal according to claim 25 or 26, wherein: The first preset condition includes at least one of the following: The number of times that the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal; The number of times that the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, and the actual optimal K beams are determined based on the measurement result of the terminal; The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

29. The terminal according to any one of claims 24 to 26, wherein: The monitoring related information reported by the terminal includes at least one of the following: a measurement result of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1; prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements; The difference between the measurement results of the N measurements and the prediction results of the N predictions; The number of times the optimal K beams predicted by the first model do not include the actual optimal beam; The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams; a difference or an absolute value of the difference or an average value of the absolute value of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the actual optimal K beams; a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the layer 1 reference signal received power value of the actual optimal beam, or an absolute value of the difference, or an average value of the absolute values ​​of the difference; Identification information of the first model.

30. The terminal according to claim 29, wherein: The N measurements are all measurements; or, N measurements selected from all measurements; or, N measurements corresponding to N predictions selected from all predictions of the first model.

31. The terminal according to claim 30, wherein: A processor is configured to read the computer program in the memory and perform the following operations: The measurement order of the N measurements in all measurements is indicated by a bit map, or the measurement identification information of the N measurements is indicated.

32. The terminal according to claim 30, wherein: The N measurements selected from all measurements include at least one of the following: The first measurement within the monitoring and reporting period of the first model; The first measurement of the first model from the start of monitoring; A measurement in which the result of a measurement differs from the result of a previous measurement; The difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold value; The difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than the fifth threshold value; A measurement in which the difference between the average value of the measurement result and the average value of the measurement result of the previous measurement is greater than a sixth threshold; The difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than the seventh threshold value; A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

33. The terminal according to claim 30, wherein: The N predictions selected from all predictions of the first model include at least one of the following: The first forecast within the monitoring and reporting period; The first prediction from the start time of the monitoring; A forecast whose predicted result is different from the predicted result of the previous forecast; The prediction that the difference between the maximum value of the predicted result and the maximum value of the predicted result of the previous prediction is greater than the ninth threshold; The difference between the minimum value of the predicted result and the minimum value of the predicted result of the previous prediction is greater than the prediction of the tenth threshold; The difference between the average of the predicted results and the average of the predicted results of the previous prediction is greater than the eleventh threshold; The difference between the standard deviation of the predicted result and the standard deviation of the predicted result of the previous prediction is greater than the prediction of the twelfth threshold; The difference between the prediction result variance and the prediction result variance of the previous prediction is greater than the prediction of the thirteenth threshold.

34. The terminal according to any one of claims 24 to 26, wherein: A processor is configured to read the computer program in the memory and perform the following operations: Reporting the monitoring related information of the first model to the network device through the first resource; The first resource includes any of the following: A resource configured by the network device for reporting monitoring-related information of the first model; Physical uplink shared channel PUSCH resources already available to the terminal; The terminal requests PUSCH resources via a scheduling request SR.

35. The terminal according to any one of claims 24 to 26, wherein: A processor is configured to read the computer program in the memory and perform the following operations: When storing the measurement result of the Lth measurement, if the required storage space exceeds the remaining storage space of the terminal, the earliest stored measurement result is discarded and the measurement result of the Lth measurement is stored.

36. A network device comprising: A memory, a transceiver, and a processor, wherein: A memory for storing a computer program; a transceiver for transmitting and receiving data under the control of the processor; and a processor for reading the computer program in the memory and performing the following operations: Receiving monitoring related information of the first model reported by the terminal; Based on the monitoring-related information, performance monitoring of the first model is performed.

37. The network device according to claim 36, wherein: A processor is configured to read the computer program in the memory and perform the following operations: Sending first information to the terminal, where the first information is used to indicate at least one of the following: The monitoring and reporting period of the first model; The monitoring start time of the first model; The prediction result of the first model needs to meet a first preset condition.

38. The network device according to claim 36 or 37, wherein: A processor is configured to read the computer program in the memory and perform the following operations: The network device sends reporting trigger information to the terminal, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model.

39. The network device according to claim 38, wherein: A processor is configured to read the computer program in the memory and perform the following operations: When it is monitored that the system performance indicator meets the second preset condition, reporting trigger information is sent to the terminal.

40. The network device according to claim 37, wherein: The first preset condition includes at least one of the following: The number of times that the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal; The number of times that the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, and the actual optimal K beams are determined based on the measurement result of the terminal; The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

41. The network device according to claim 36 or 37, wherein: A processor is configured to read the computer program in the memory and perform the following operations: Based on the monitoring-related information, a monitoring index of the first model is calculated.

42. The network device according to claim 41, wherein: In the case where the first model is deployed on the network device, the monitoring-related information includes measurement results of the terminal measuring the reference signal N times, where N is an integer greater than 1; A processor is configured to read the computer program in the memory and perform the following operations: Filtering N predictions corresponding to the N measurements from all predictions of the first model; According to the prediction results of the N predictions and the measurement results of the N measurements, the first A monitoring indicator of the model; or, Selecting M measurements from the N measurements, where M is an integer greater than 1; Filtering M predictions corresponding to the M measurements from all predictions of the first model; The monitoring index of the first model is calculated according to the prediction results of the M predictions and the measurement results of the M measurements.

43. The network device according to claim 42, wherein: The M measurements selected from the N measurements include at least one of the following: The first measurement within the monitoring and reporting period; The first measurement from the start time of the monitoring; A measurement in which the result of a measurement differs from the result of a previous measurement; The difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold value; The difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than the fifth threshold value; A measurement in which the difference between the average value of the measurement result and the average value of the measurement result of the previous measurement is greater than a sixth threshold; The difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than the seventh threshold value; A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

44. The network device according to claim 36 or 37, wherein: In the case where the first model is deployed on the terminal, the monitoring-related information includes at least one of the following: a measurement result of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1; prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements; The difference between the measurement results of the N measurements and the prediction results of the N predictions; The number of times the optimal K beams predicted by the first model do not include the actual optimal beam; The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams; a difference or an absolute value of the difference or an average value of the absolute value of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the actual optimal K beams; a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the layer 1 reference signal received power value of the actual optimal beam, or an absolute value of the difference, or an average value of the absolute values ​​of the difference; Identification information of the first model.

45. The network device according to claim 36 or 37, wherein: A processor is configured to read the computer program in the memory and perform the following operations: Receiving, through a first resource, monitoring-related information of the first model reported by the terminal; The first resource includes any of the following: A resource configured by the network device for reporting monitoring-related information of the first model; Physical uplink shared channel PUSCH resources already available to the terminal; The terminal requests PUSCH resources via a scheduling request SR.

46. ​​The network device according to claim 41, wherein: A processor is configured to read the computer program in the memory and perform the following operations: According to the monitoring indicator of the first model, a first process is performed, where the first process includes any one of the following: activating the first model; Deactivate the first model.

47. A terminal, comprising: A first determination module, used to determine whether a reporting condition of monitoring-related information of a first model is met; A reporting module is used to report the monitoring related information of the first model to the network device when the reporting conditions are met.

48. The terminal according to claim 47, further comprising: A second determining module is configured to determine a reporting condition of the monitoring-related information of the first model according to first information sent by the network device, wherein the first information is used to indicate at least one of the following: The monitoring and reporting period of the first model; The monitoring start time of the first model; The prediction result of the first model needs to meet a first preset condition.

49. The terminal according to claim 47, further comprising: A third determination module is configured to determine a reporting condition for monitoring-related information of the first model according to predefined performance monitoring-related parameters, wherein the performance monitoring-related parameters include at least one of the following: The monitoring and reporting period of the first model; The monitoring start time of the first model; The prediction result of the first model needs to meet a first preset condition.

50. The terminal according to any one of claims 47 to 49, wherein: The reporting conditions include at least one of the following: The reporting time of the monitoring reporting cycle of the first model arrives; The terminal receives reporting trigger information from the network device, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model; The terminal detects that the prediction result of the first model meets a first preset condition.

51. The terminal according to claim 48 or 49, wherein: The first preset condition includes at least one of the following: The number of times that the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal; The number of times that the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, and the actual optimal K beams are determined based on the measurement result of the terminal; The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

52. The terminal according to any one of claims 47 to 49, wherein: The monitoring related information reported by the terminal includes at least one of the following: a measurement result of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1; prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements; The difference between the measurement results of the N measurements and the prediction results of the N predictions; The number of times the optimal K beams predicted by the first model do not include the actual optimal beam; The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams; a difference or an absolute value of the difference or an average value of the absolute value of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the actual optimal K beams; a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the layer 1 reference signal received power value of the actual optimal beam, or an absolute value of the difference, or an average value of the absolute values ​​of the difference; Identification information of the first model.

53. The terminal according to claim 52, wherein: The N measurements are all measurements; or, N measurements selected from all measurements; or, N measurements corresponding to N predictions selected from all predictions of the first model.

54. The terminal according to claim 53, wherein: The terminal indicates the measurement order of the N measurements in all measurements through a bit map, or indicates measurement identification information of the N measurements.

55. The terminal according to claim 53, wherein: The N measurements selected from all measurements include at least one of the following: The first measurement within the monitoring and reporting period of the first model; The first measurement of the first model from the start of monitoring; A measurement in which the result of a measurement differs from the result of a previous measurement; The difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold value; The difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than the fifth threshold value; A measurement in which the difference between the average value of the measurement result and the average value of the measurement result of the previous measurement is greater than a sixth threshold; The difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than the seventh threshold value; A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

56. The terminal according to claim 53, wherein: The N predictions selected from all predictions of the first model include at least one of the following: The first forecast within the monitoring and reporting period; The first prediction from the start time of the monitoring; A forecast whose predicted result is different from the predicted result of the previous forecast; The prediction that the difference between the maximum value of the predicted result and the maximum value of the predicted result of the previous prediction is greater than the ninth threshold; The difference between the minimum value of the predicted result and the minimum value of the predicted result of the previous prediction is greater than the prediction of the tenth threshold; The difference between the average of the predicted results and the average of the predicted results of the previous prediction is greater than the eleventh threshold; The difference between the standard deviation of the predicted result and the standard deviation of the predicted result of the previous prediction is greater than the prediction of the twelfth threshold; The difference between the prediction result variance and the prediction result variance of the previous prediction is greater than the prediction of the thirteenth threshold.

57. The terminal according to any one of claims 47 to 49, wherein: The reporting module is specifically used for: Reporting the monitoring related information of the first model to the network device through the first resource; The first resource includes any of the following: A resource configured by the network device for reporting monitoring-related information of the first model; Physical uplink shared channel PUSCH resources already available to the terminal; The terminal requests PUSCH resources via a scheduling request SR.

58. The terminal according to any one of claims 47 to 49, further comprising: The processing module is used for discarding the earliest stored measurement result and storing the measurement result of the Lth measurement if the required storage space exceeds the remaining storage space of the terminal when storing the measurement result of the Lth measurement.

59. A network device comprising: A receiving module, used for receiving monitoring related information of the first model reported by the terminal; A monitoring module is used to perform performance monitoring of the first model based on the monitoring related information.

60. The network device according to claim 59, further comprising: A first sending module is configured to send first information to the terminal, where the first information is used to indicate at least one of the following: The monitoring and reporting period of the first model; The monitoring start time of the first model; The prediction result of the first model needs to meet a first preset condition.

61. The network device according to claim 59 or 60, further comprising: The second sending module is used to send reporting trigger information to the terminal, where the reporting trigger information is used to instruct reporting the monitoring related information of the first model.

62. The network device according to claim 61, further comprising: The third sending module is used to send reporting trigger information to the terminal when it is monitored that the system performance indicator meets the second preset condition.

63. The network device according to claim 60, wherein: The first preset condition includes at least one of the following: The number of times that the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal; The number of times that the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, and the actual optimal K beams are determined based on the measurement result of the terminal; The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

64. The network device according to claim 59 or 60, wherein: The monitoring module comprises: A calculation unit is used to calculate the monitoring index of the first model based on the monitoring related information.

65. The network device according to claim 64, when the first model is deployed in the network device, the monitoring-related information includes measurement results of the terminal measuring the reference signal N times, where N is an integer greater than 1; The computing unit comprises: A first screening subunit, configured to screen out N predictions corresponding to the N measurements from all predictions of the first model; A first calculation subunit, configured to calculate a monitoring indicator of the first model according to the prediction results of the N predictions and the measurement results of the N measurements; or, The computing unit comprises: A second screening subunit, configured to screen out M measurements from the N measurements, where M is an integer greater than 1; A third screening subunit, configured to screen out M predictions corresponding to the M measurements from all predictions of the first model; The second calculation subunit is used to calculate the monitoring index of the first model according to the prediction results of the M predictions and the measurement results of the M measurements.

66. The network device according to claim 65, wherein: The M measurements selected from the N measurements include at least one of the following: The first measurement within the monitoring and reporting period; The first measurement from the start time of the monitoring; A measurement in which the result of a measurement differs from the result of a previous measurement; The difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold value; The difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than the fifth threshold value; A measurement in which the difference between the average value of the measurement result and the average value of the measurement result of the previous measurement is greater than a sixth threshold; The difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than the seventh threshold value; A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

67. The network device according to claim 59 or 60, wherein: In the case where the first model is deployed on the terminal, the monitoring-related information includes at least one of the following: a measurement result of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1; The prediction results of the N predictions of the first model, the N predictions being the same as the N measurements The corresponding predictions; The difference between the measurement results of the N measurements and the prediction results of the N predictions; The number of times the optimal K beams predicted by the first model do not include the actual optimal beam; The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams; a difference or an absolute value of the difference or an average value of the absolute value of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the actual optimal K beams; a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the layer 1 reference signal received power value of the actual optimal beam, or an absolute value of the difference, or an average value of the absolute values ​​of the difference; Identification information of the first model.

68. The network device according to claim 59 or 60, wherein: The receiving module is specifically used for: Receiving, through a first resource, monitoring-related information of the first model reported by the terminal; The first resource includes any of the following: A resource configured by the network device for reporting monitoring-related information of the first model; Physical uplink shared channel PUSCH resources already available to the terminal; The terminal requests PUSCH resources via a scheduling request SR.

69. The network device according to claim 64, further comprising: A processing module is used to perform a first process according to the monitoring indicator of the first model, where the first process includes any one of the following: activating the first model; Deactivate the first model.

70. A processor-readable storage medium, wherein the processor-readable storage medium stores a computer program, wherein the computer program is used to cause the processor to execute the performance monitoring method described in any one of claims 1 to 12, or wherein the computer program is used to cause the processor to execute the performance monitoring method described in any one of claims 13 to 23.

Images