Model monitoring method and communication equipment

Abstract

The invention provides a model monitoring method and communication equipment, and the method comprises the steps: determining whether a reporting condition of monitoring related information of a first model is satisfied or not through a terminal; and the terminal reports the monitoring related information of the first model to network equipment under the condition that the reporting condition is met. Through the above process, the terminal can report the related information of multiple times of model monitoring to the network device together, so that the reporting overhead of the terminal in the model monitoring process can be reduced.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a model monitoring method and communication equipment. Background Art

[0002] In order to reduce the measurement overhead of the communication system, a solution using artificial intelligence (AI) or machine learning (ML) is proposed, that is, to make relevant predictions based on a small number of measurement results or historical measurement results through AI / ML models. In order to ensure the performance of AI / ML applied to communication systems, it is necessary to perform model monitoring on the AI / ML model. During the model monitoring process, the terminal is required to report the measurement results and / or prediction results to the network device. At present, there is no clear solution on how to report model monitoring related information during the model monitoring process. Summary of the invention

[0003] The embodiments of the present application provide a model monitoring method and a communication device to solve the problem of how to report model monitoring related information during the model monitoring process.

[0004] The present application embodiment provides a model monitoring method, including:

[0005] The terminal determines whether a reporting condition of monitoring-related information of the first model is met;

[0006] When the reporting condition is met, the terminal reports the monitoring-related information of the first model to the network device.

[0007] Optionally, the method further comprises:

[0008] The terminal determines, according to first information sent by the network device, a reporting condition for monitoring-related information of the first model, where the first information is used to indicate at least one of the following:

[0009] The monitoring and reporting period of the first model;

[0010] The monitoring start time of the first model;

[0011] The prediction result of the first model needs to meet a first preset condition.

[0012] Optionally, the method further comprises:

[0013] The terminal determines, according to a predefined model monitoring related parameter, a reporting condition of the monitoring related information of the first model, wherein the model monitoring related parameter includes at least one of the following:

[0014] The monitoring and reporting period of the first model;

[0015] The monitoring start time of the first model;

[0016] The prediction result of the first model needs to meet a first preset condition.

[0017] Optionally, the reporting condition includes at least one of the following:

[0018] The reporting time of the monitoring reporting cycle of the first model arrives;

[0019] The terminal receives reporting trigger information from the network device, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model;

[0020] The terminal detects that the prediction result of the first model meets a first preset condition.

[0021] Optionally, the first preset condition includes at least one of the following:

[0022] The number of times that the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal;

[0023] The number of times that the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, and the actual optimal K beams are determined based on the measurement result of the terminal;

[0024] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

[0025] Optionally, the monitoring-related information reported by the terminal includes at least one of the following:

[0026] a measurement result of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1;

[0027] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0028] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0029] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0030] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0031] a difference or an absolute value of the difference or an average value of the absolute value of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the actual optimal K beams;

[0032] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the layer 1 reference signal received power value of the actual optimal beam, or an absolute value of the difference, or an average value of the absolute values ​​of the difference;

[0033] Identification information of the first model.

[0034] Optionally, the N measurements are all measurements of the terminal; or, the N measurements selected by the terminal from all measurements; or, the N measurements corresponding to the N predictions selected by the terminal from all predictions of the first model.

[0035] Optionally, the terminal indicates a measurement order of the N measurements in all measurements through a bit map, or indicates measurement identification information of the N measurements.

[0036] Optionally, the N measurements screened by the terminal from all measurements include at least one of the following:

[0037] The first measurement within the monitoring and reporting period of the first model;

[0038] The first measurement of the first model from the start of monitoring;

[0039] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0040] The difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold value;

[0041] The difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than the fifth threshold value;

[0042] A measurement in which the difference between the average value of the measurement result and the average value of the measurement result of the previous measurement is greater than a sixth threshold;

[0043] The difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than the seventh threshold value;

[0044] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0045] Optionally, the N predictions selected by the terminal from all predictions of the first model include at least one of the following:

[0046] The first forecast within the monitoring and reporting period;

[0047] The first prediction from the start time of the monitoring;

[0048] A forecast whose predicted result is different from the predicted result of the previous forecast;

[0049] The prediction that the difference between the maximum value of the predicted result and the maximum value of the predicted result of the previous prediction is greater than the ninth threshold;

[0050] The difference between the minimum value of the predicted result and the minimum value of the predicted result of the previous prediction is greater than the prediction of the tenth threshold;

[0051] The difference between the average of the predicted results and the average of the predicted results of the previous prediction is greater than the eleventh threshold;

[0052] The difference between the standard deviation of the predicted result and the standard deviation of the predicted result of the previous prediction is greater than the prediction of the twelfth threshold;

[0053] The difference between the prediction result variance and the prediction result variance of the previous prediction is greater than the prediction of the thirteenth threshold.

[0054] Optionally, the terminal reports monitoring-related information of the first model to the network device, including:

[0055] The terminal reports the monitoring related information of the first model to the network device through the first resource;

[0056] The first resource includes any of the following:

[0057] A resource configured by the network device for reporting monitoring-related information of the first model;

[0058] Physical uplink shared channel PUSCH resources already available to the terminal;

[0059] The terminal requests PUSCH resources via a scheduling request SR.

[0060] Optionally, the method further comprises:

[0061] When the terminal stores the measurement result of the Lth measurement, if the required storage space exceeds the remaining storage space of the terminal, the terminal discards the earliest stored measurement result and stores the measurement result of the Lth measurement.

[0062] The present application embodiment provides a model monitoring method, including:

[0063] The network device receives monitoring related information of the first model reported by the terminal;

[0064] The network device performs model monitoring of the first model based on the monitoring-related information.

[0065] Optionally, the method further comprises:

[0066] The network device sends first information to the terminal, where the first information is used to indicate at least one of the following:

[0067] The monitoring and reporting period of the first model;

[0068] The monitoring start time of the first model;

[0069] The prediction result of the first model needs to meet a first preset condition.

[0070] Optionally, the reporting condition is determined by predefined model monitoring related parameters, and the model monitoring related parameters include at least one of the following:

[0071] The monitoring and reporting period of the first model;

[0072] The monitoring start time of the first model;

[0073] The prediction result of the first model needs to meet a first preset condition.

[0074] Optionally, the method further comprises:

[0075] The network device sends reporting trigger information to the terminal, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model.

[0076] Optionally, the network device sending reporting trigger information to the terminal includes:

[0077] When the network device monitors that the system performance indicator meets the second preset condition, the network device sends reporting trigger information to the terminal.

[0078] Optionally, the first preset condition includes at least one of the following:

[0079] The number of times that the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal;

[0080] The number of times that the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, and the actual optimal K beams are determined based on the measurement result of the terminal;

[0081] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

[0082] Optionally, the network device performs model monitoring of the first model based on the monitoring related information, including:

[0083] The network device calculates the monitoring index of the first model based on the monitoring-related information.

[0084] Optionally, when the first model is deployed on the network device, the monitoring-related information includes measurement results of the terminal measuring the reference signal N times, where N is an integer greater than 1;

[0085] The network device calculates the monitoring index of the first model based on the monitoring related information, including:

[0086] The network device selects N predictions corresponding to the N measurements from all predictions of the first model;

[0087] The network device calculates the monitoring index of the first model according to the prediction results of the N predictions and the measurement results of the N measurements;

[0088] or,

[0089] The network device calculates the monitoring index of the first model based on the monitoring related information, including:

[0090] The network device selects M measurements from the N measurements, where M is an integer greater than 1;

[0091] The network device selects M predictions corresponding to the M measurements from all predictions of the first model;

[0092] The network device calculates the monitoring index of the first model according to the prediction results of the M predictions and the measurement results of the M measurements.

[0093] Optionally, the M measurements screened out by the network device from the N measurements include at least one of the following:

[0094] The first measurement within the monitoring and reporting period;

[0095] The first measurement from the start time of the monitoring;

[0096] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0097] The difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold value;

[0098] The difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than the fifth threshold value;

[0099] A measurement in which the difference between the average value of the measurement result and the average value of the measurement result of the previous measurement is greater than a sixth threshold;

[0100] The difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than the seventh threshold value;

[0101] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0102] Optionally, when the first model is deployed on the terminal, the monitoring-related information includes at least one of the following:

[0103] a measurement result of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1;

[0104] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0105] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0106] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0107] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0108] a difference or an absolute value of the difference or an average value of the absolute value of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the actual optimal K beams;

[0109] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the layer 1 reference signal received power value of the actual optimal beam, or an absolute value of the difference, or an average value of the absolute values ​​of the difference;

[0110] Identification information of the first model.

[0111] Optionally, the network device receives monitoring-related information of the first model reported by the terminal, including:

[0112] The network device receives, through a first resource, monitoring-related information of the first model reported by the terminal;

[0113] The first resource includes any of the following:

[0114] A resource configured by the network device for reporting monitoring-related information of the first model;

[0115] Physical uplink shared channel PUSCH resources already available to the terminal;

[0116] The terminal requests PUSCH resources via a scheduling request SR.

[0117] Optionally, the method further comprises:

[0118] The network device performs a first process according to the monitoring indicator of the first model, where the first process includes any one of the following:

[0119] activating the first model;

[0120] Deactivate the first model.

[0121] The embodiment of the present application further provides a terminal, including: a memory, a transceiver, and a processor, wherein:

[0122] A memory for storing a computer program; a transceiver for transmitting and receiving data under the control of the processor; and a processor for reading the computer program in the memory and performing the following operations:

[0123] Determining whether a reporting condition of monitoring-related information of the first model is met;

[0124] When the reporting condition is met, the monitoring related information of the first model is reported to the network device.

[0125] Optionally, the processor is configured to read the computer program in the memory and perform the following operations:

[0126] Determine, according to the first information sent by the network device, a reporting condition of the monitoring-related information of the first model, wherein the first information is used to indicate at least one of the following:

[0127] The monitoring and reporting period of the first model;

[0128] The monitoring start time of the first model;

[0129] The prediction result of the first model needs to meet a first preset condition.

[0130] Optionally, the processor is configured to read the computer program in the memory and perform the following operations:

[0131] Determine a reporting condition for monitoring-related information of the first model according to predefined model monitoring-related parameters, wherein the model monitoring-related parameters include at least one of the following:

[0132] The monitoring and reporting period of the first model;

[0133] The monitoring start time of the first model;

[0134] The prediction result of the first model needs to meet a first preset condition.

[0135] Optionally, the reporting condition includes at least one of the following:

[0136] The reporting time of the monitoring reporting cycle of the first model arrives;

[0137] The terminal receives reporting trigger information from the network device, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model;

[0138] The terminal detects that the prediction result of the first model meets a first preset condition.

[0139] Optionally, the first preset condition includes at least one of the following:

[0140] The number of times that the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal;

[0141] The number of times that the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, and the actual optimal K beams are determined based on the measurement result of the terminal;

[0142] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

[0143] Optionally, the monitoring-related information reported by the terminal includes at least one of the following:

[0144] a measurement result of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1;

[0145] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0146] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0147] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0148] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0149] a difference or an absolute value of the difference or an average value of the absolute value of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the actual optimal K beams;

[0150] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the layer 1 reference signal received power value of the actual optimal beam, or an absolute value of the difference, or an average value of the absolute values ​​of the difference;

[0151] Identification information of the first model.

[0152] Optionally, the N measurements are all measurements; or, N measurements selected from all measurements; or, N measurements corresponding to N predictions selected from all predictions of the first model.

[0153] Optionally, the processor is configured to read the computer program in the memory and perform the following operations:

[0154] The measurement order of the N measurements in all measurements is indicated by a bit map, or the measurement identification information of the N measurements is indicated.

[0155] Optionally, the N measurements screened from all measurements include at least one of the following:

[0156] The first measurement within the monitoring and reporting period of the first model;

[0157] The first measurement of the first model from the start of monitoring;

[0158] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0159] The difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold value;

[0160] The difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than the fifth threshold value;

[0161] A measurement in which the difference between the average value of the measurement result and the average value of the measurement result of the previous measurement is greater than a sixth threshold;

[0162] The difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than the seventh threshold value;

[0163] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0164] Optionally, the N predictions selected from all predictions of the first model include at least one of the following:

[0165] The first forecast within the monitoring and reporting period;

[0166] The first prediction from the start time of the monitoring;

[0167] A forecast whose predicted result is different from the predicted result of the previous forecast;

[0168] The prediction that the difference between the maximum value of the predicted result and the maximum value of the predicted result of the previous prediction is greater than the ninth threshold;

[0169] The difference between the minimum value of the predicted result and the minimum value of the predicted result of the previous prediction is greater than the prediction of the tenth threshold;

[0170] The difference between the average of the predicted results and the average of the predicted results of the previous prediction is greater than the eleventh threshold;

[0171] The difference between the standard deviation of the predicted result and the standard deviation of the predicted result of the previous prediction is greater than the prediction of the twelfth threshold;

[0172] The difference between the prediction result variance and the prediction result variance of the previous prediction is greater than the prediction of the thirteenth threshold.

[0173] Optionally, the processor is configured to read the computer program in the memory and perform the following operations:

[0174] Reporting the monitoring related information of the first model to the network device through the first resource;

[0175] The first resource includes any of the following:

[0176] A resource configured by the network device for reporting monitoring-related information of the first model;

[0177] Physical uplink shared channel PUSCH resources already available to the terminal;

[0178] The terminal requests PUSCH resources via a scheduling request SR.

[0179] Optionally, the processor is configured to read the computer program in the memory and perform the following operations:

[0180] When storing the measurement result of the Lth measurement, if the required storage space exceeds the remaining storage space of the terminal, the earliest stored measurement result is discarded and the measurement result of the Lth measurement is stored.

[0181] The embodiment of the present application further provides a network device, including: a memory, a transceiver, and a processor, wherein:

[0182] A memory for storing a computer program; a transceiver for transmitting and receiving data under the control of the processor; and a processor for reading the computer program in the memory and performing the following operations:

[0183] Receiving monitoring related information of the first model reported by the terminal;

[0184] Based on the monitoring-related information, model monitoring of the first model is performed.

[0185] Optionally, the processor is configured to read the computer program in the memory and perform the following operations:

[0186] Sending first information to the terminal, where the first information is used to indicate at least one of the following:

[0187] The monitoring and reporting period of the first model;

[0188] The monitoring start time of the first model;

[0189] The prediction result of the first model needs to meet a first preset condition.

[0190] Optionally, the reporting condition is determined by predefined model monitoring related parameters, and the model monitoring related parameters include at least one of the following:

[0191] The monitoring and reporting period of the first model;

[0192] The monitoring start time of the first model;

[0193] The prediction result of the first model needs to meet a first preset condition.

[0194] Optionally, the method further comprises:

[0195] The network device sends reporting trigger information to the terminal, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model.

[0196] Optionally, the processor is configured to read the computer program in the memory and perform the following operations:

[0197] When it is monitored that the system performance indicator meets the second preset condition, reporting trigger information is sent to the terminal.

[0198] Optionally, the first preset condition includes at least one of the following:

[0199] The number of times that the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal;

[0200] The number of times that the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, and the actual optimal K beams are determined based on the measurement result of the terminal;

[0201] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

[0202] Optionally, the processor is configured to read the computer program in the memory and perform the following operations:

[0203] Based on the monitoring-related information, a monitoring index of the first model is calculated.

[0204] Optionally, when the first model is deployed on the network device, the monitoring-related information includes measurement results of the terminal measuring the reference signal N times, where N is an integer greater than 1;

[0205] A processor is configured to read the computer program in the memory and perform the following operations:

[0206] Filtering N predictions corresponding to the N measurements from all predictions of the first model;

[0207] Calculating a monitoring indicator of the first model according to the prediction results of the N predictions and the measurement results of the N measurements;

[0208] or,

[0209] Selecting M measurements from the N measurements, where M is an integer greater than 1;

[0210] Filtering M predictions corresponding to the M measurements from all predictions of the first model;

[0211] The monitoring index of the first model is calculated according to the prediction results of the M predictions and the measurement results of the M measurements.

[0212] Optionally, the M measurements screened out from the N measurements include at least one of the following:

[0213] The first measurement within the monitoring and reporting period;

[0214] The first measurement from the start time of the monitoring;

[0215] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0216] The difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold value;

[0217] The difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than the fifth threshold value;

[0218] A measurement in which the difference between the average value of the measurement result and the average value of the measurement result of the previous measurement is greater than a sixth threshold;

[0219] The difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than the seventh threshold value;

[0220] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0221] Optionally, when the first model is deployed on the terminal, the monitoring-related information includes at least one of the following:

[0222] a measurement result of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1;

[0223] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0224] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0225] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0226] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0227] a difference or an absolute value of the difference or an average value of the absolute value of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the actual optimal K beams;

[0228] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the layer 1 reference signal received power value of the actual optimal beam, or an absolute value of the difference, or an average value of the absolute values ​​of the difference;

[0229] Identification information of the first model.

[0230] Optionally, the processor is configured to read the computer program in the memory and perform the following operations:

[0231] Receiving, through a first resource, monitoring-related information of the first model reported by the terminal;

[0232] The first resource includes any of the following:

[0233] A resource configured by the network device for reporting monitoring-related information of the first model;

[0234] Physical uplink shared channel PUSCH resources already available to the terminal;

[0235] The terminal requests PUSCH resources via a scheduling request SR.

[0236] Optionally, the processor is configured to read the computer program in the memory and perform the following operations:

[0237] According to the monitoring indicator of the first model, a first process is performed, where the first process includes any one of the following:

[0238] activating the first model;

[0239] Deactivate the first model.

[0240] The present application also provides a terminal, including:

[0241] A first determination module, used to determine whether a reporting condition of monitoring-related information of a first model is met;

[0242] A reporting module is used to report the monitoring related information of the first model to the network device when the reporting conditions are met.

[0243] Optionally, the terminal further includes:

[0244] A second determining module is configured to determine a reporting condition of the monitoring-related information of the first model according to first information sent by the network device, wherein the first information is used to indicate at least one of the following:

[0245] The monitoring and reporting period of the first model;

[0246] The monitoring start time of the first model;

[0247] The prediction result of the first model needs to meet a first preset condition.

[0248] Optionally, the terminal further includes:

[0249] The third determination module is configured to determine a reporting condition for the monitoring-related information of the first model according to predefined model monitoring-related parameters, wherein the model monitoring-related parameters include at least one of the following:

[0250] The monitoring and reporting period of the first model;

[0251] The monitoring start time of the first model;

[0252] The prediction result of the first model needs to meet a first preset condition.

[0253] Optionally, the reporting condition includes at least one of the following:

[0254] The reporting time of the monitoring reporting cycle of the first model arrives;

[0255] The terminal receives reporting trigger information from the network device, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model;

[0256] The terminal detects that the prediction result of the first model meets a first preset condition.

[0257] Optionally, the first preset condition includes at least one of the following:

[0258] The number of times that the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal;

[0259] The number of times that the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, and the actual optimal K beams are determined based on the measurement result of the terminal;

[0260] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

[0261] Optionally, the monitoring-related information reported by the terminal includes at least one of the following:

[0262] a measurement result of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1;

[0263] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0264] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0265] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0266] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0267] a difference or an absolute value of the difference or an average value of the absolute value of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the actual optimal K beams;

[0268] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the layer 1 reference signal received power value of the actual optimal beam, or an absolute value of the difference, or an average value of the absolute values ​​of the difference;

[0269] Identification information of the first model.

[0270] Optionally, the N measurements are all measurements; or, N measurements selected from all measurements; or, N measurements corresponding to N predictions selected from all predictions of the first model.

[0271] Optionally, the terminal indicates a measurement order of the N measurements in all measurements through a bit map, or indicates measurement identification information of the N measurements.

[0272] Optionally, the N measurements screened from all measurements include at least one of the following:

[0273] The first measurement within the monitoring and reporting period of the first model;

[0274] The first measurement of the first model from the start of monitoring;

[0275] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0276] The difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold value;

[0277] The difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than the fifth threshold value;

[0278] A measurement in which the difference between the average value of the measurement result and the average value of the measurement result of the previous measurement is greater than a sixth threshold;

[0279] The difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than the seventh threshold value;

[0280] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0281] Optionally, the N predictions selected from all predictions of the first model include at least one of the following:

[0282] The first forecast within the monitoring and reporting period;

[0283] The first prediction from the start time of the monitoring;

[0284] A forecast whose predicted result is different from the predicted result of the previous forecast;

[0285] The prediction that the difference between the maximum value of the predicted result and the maximum value of the predicted result of the previous prediction is greater than the ninth threshold;

[0286] The difference between the minimum value of the predicted result and the minimum value of the predicted result of the previous prediction is greater than the prediction of the tenth threshold;

[0287] The difference between the average of the predicted results and the average of the predicted results of the previous prediction is greater than the eleventh threshold;

[0288] The difference between the standard deviation of the predicted result and the standard deviation of the predicted result of the previous prediction is greater than the prediction of the twelfth threshold;

[0289] The difference between the prediction result variance and the prediction result variance of the previous prediction is greater than the prediction of the thirteenth threshold.

[0290] Optionally, the reporting module is specifically used to:

[0291] Reporting the monitoring related information of the first model to the network device through the first resource;

[0292] The first resource includes any of the following:

[0293] A resource configured by the network device for reporting monitoring-related information of the first model;

[0294] Physical uplink shared channel PUSCH resources already available to the terminal;

[0295] The terminal requests PUSCH resources via a scheduling request SR.

[0296] Optionally, the terminal further includes:

[0297] The processing module is used for discarding the earliest stored measurement result and storing the measurement result of the Lth measurement if the required storage space exceeds the remaining storage space of the terminal when storing the measurement result of the Lth measurement.

[0298] The embodiment of the present application also provides a network device, including:

[0299] A receiving module, used for receiving monitoring related information of the first model reported by the terminal;

[0300] A monitoring module is used to perform model monitoring of the first model based on the monitoring related information.

[0301] Optionally, the network device further includes:

[0302] A first sending module is configured to send first information to the terminal, where the first information is used to indicate at least one of the following:

[0303] The monitoring and reporting period of the first model;

[0304] The monitoring start time of the first model;

[0305] The prediction result of the first model needs to meet a first preset condition.

[0306] Optionally, the reporting condition is determined by predefined model monitoring related parameters, and the model monitoring related parameters include at least one of the following:

[0307] The monitoring and reporting period of the first model;

[0308] The monitoring start time of the first model;

[0309] The prediction result of the first model needs to meet a first preset condition.

[0310] Optionally, the network device further includes:

[0311] The second sending module is used to send reporting trigger information to the terminal, where the reporting trigger information is used to instruct reporting the monitoring related information of the first model.

[0312] Optionally, the network device further includes:

[0313] The third sending module is used to send reporting trigger information to the terminal when it is monitored that the system performance indicator meets the second preset condition.

[0314] Optionally, the first preset condition includes at least one of the following:

[0315] The number of times that the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal;

[0316] The number of times that the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, and the actual optimal K beams are determined based on the measurement result of the terminal;

[0317] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

[0318] Optionally, the monitoring module includes:

[0319] A calculation unit is used to calculate the monitoring index of the first model based on the monitoring related information.

[0320] Optionally, when the first model is deployed on the network device, the monitoring-related information includes measurement results of the terminal measuring the reference signal N times, where N is an integer greater than 1;

[0321] The computing unit comprises:

[0322] A first screening subunit, configured to screen out N predictions corresponding to the N measurements from all predictions of the first model;

[0323] A first calculation subunit, configured to calculate a monitoring indicator of the first model according to the prediction results of the N predictions and the measurement results of the N measurements;

[0324] or,

[0325] The computing unit comprises:

[0326] A second screening subunit, configured to screen out M measurements from the N measurements, where M is an integer greater than 1;

[0327] A third screening subunit, configured to screen out M predictions corresponding to the M measurements from all predictions of the first model;

[0328] The second calculation subunit is used to calculate the monitoring index of the first model according to the prediction results of the M predictions and the measurement results of the M measurements.

[0329] Optionally, the M measurements screened out from the N measurements include at least one of the following:

[0330] The first measurement within the monitoring and reporting period;

[0331] The first measurement from the start time of the monitoring;

[0332] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0333] The difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold value;

[0334] The difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than the fifth threshold value;

[0335] A measurement in which the difference between the average value of the measurement result and the average value of the measurement result of the previous measurement is greater than a sixth threshold;

[0336] The difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than the seventh threshold value;

[0337] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0338] Optionally, when the first model is deployed on the terminal, the monitoring-related information includes at least one of the following:

[0339] a measurement result of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1;

[0340] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0341] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0342] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0343] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0344] a difference or an absolute value of the difference or an average value of the absolute value of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the actual optimal K beams;

[0345] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the layer 1 reference signal received power value of the actual optimal beam, or an absolute value of the difference, or an average value of the absolute values ​​of the difference;

[0346] Identification information of the first model.

[0347] Optionally, the receiving module is specifically used for:

[0348] Receiving, through a first resource, monitoring-related information of the first model reported by the terminal;

[0349] The first resource includes any of the following:

[0350] A resource configured by the network device for reporting monitoring-related information of the first model;

[0351] Physical uplink shared channel PUSCH resources already available to the terminal;

[0352] The terminal requests PUSCH resources via a scheduling request SR.

[0353] Optionally, the network device further includes:

[0354] A processing module is used to perform a first process according to the monitoring indicator of the first model, where the first process includes any one of the following:

[0355] activating the first model;

[0356] Deactivate the first model.

[0357] An embodiment of the present application also provides a processor-readable storage medium, which stores a computer program, and the computer program is used to enable the processor to execute any one of the model monitoring methods on the terminal side provided in the embodiment of the present application, or the computer program is used to enable the processor to execute any one of the model monitoring methods on the network device side provided in the embodiment of the present application.

[0358] In the embodiment of the present application, the terminal determines whether the reporting condition of the monitoring-related information of the first model is met; if the reporting condition is met, the terminal reports the monitoring-related information of the first model to the network device. Through the above process, if the reporting condition of the monitoring-related information of the first model is met, the terminal reports the monitoring-related information of the first model to the network device, so that the terminal can report multiple model monitoring-related information to the network device together, which can reduce the reporting overhead of the terminal during the model monitoring process. BRIEF DESCRIPTION OF THE DRAWINGS

[0359] Figure 1 It is a schematic diagram of the structure of the network architecture applicable to the implementation of this application;

[0360] Figure 2 It is a model monitoring flow chart corresponding to the base station side deployment model;

[0361] Figure 3 It is the model monitoring flow chart corresponding to the terminal-side deployment model;

[0362] Figure 4 is a flow chart of a model monitoring method provided in an embodiment of the present application;

[0363] Figure 5 It is a schematic diagram of a base station sending a reference signal and a UE reporting a measurement value during a model monitoring process provided in an embodiment of the present application;

[0364] Figure 6 is a flow chart of a model monitoring method provided in an embodiment of the present application;

[0365] Figure 7 is a structural diagram of a terminal provided in an embodiment of the present application;

[0366] Figure 8is a structural diagram of a network device provided in an embodiment of the present application;

[0367] Fig. 9 is a structural diagram of another terminal provided in an embodiment of the present application;

[0368] Fig.10 It is a structural diagram of another network device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0369] In order to make the technical problems, technical solutions and advantages to be solved by the present application clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0370] In the embodiments of the present application, the term "and / or" describes the association relationship of the associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0371] In the embodiments of the present application, the term "plurality" refers to two or more than two, and other quantifiers are similar.

[0372] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0373] The embodiments of the present application provide a model monitoring method, a model monitoring method, a terminal, a network device and a storage medium, which are used to solve the problem of how to report model monitoring related information during the model monitoring process.

[0374] Among them, the method and the device are based on the same application concept. Since the method and the device solve the problem in a similar principle, the implementation of the device and the method can refer to each other, and the repeated parts will not be repeated.

[0375] The technical solution provided in the embodiment of the present application can be applicable to a variety of systems, especially 5G systems and 6G systems. For example, the applicable system can be a global system of mobile communication (GSM) system, a code division multiple access (CDMA) system, a wideband code division multiple access (WCDMA) general packet radio service (GPRS) system, a long term evolution (LTE) system, a LTE frequency division duplex (FDD) system, a LTE time division duplex (TDD) system, an advanced long term evolution (LTE-A) system, a universal mobile telecommunication system (UMTS), a world-wide interoperability for microwave access (WiMAX) system, a 5G NR system, a 6G system, etc. These various systems include terminals and network equipment. The system may also include a core network part, such as an evolved packet system (EPS), a 5G system (5G System, 5GS), etc.

[0376] See also Figure 1 , Figure 1 It is a schematic diagram of the network architecture applicable to the implementation of this application, such as Figure 1 As shown, it includes a terminal 11 and a network device 12, wherein:

[0377] Among them, the terminal 11 involved in the embodiment of the present application may be a device that provides voice and / or data connectivity to the user, a handheld device with a wireless connection function, or other processing devices connected to a wireless modem. In different systems, the name of the terminal may also be different. For example, in a 5G system, the terminal may be called a user equipment (UE). A wireless terminal can communicate with one or more core networks (CN) via a radio access network (RAN). The wireless terminal can be a mobile terminal, such as a mobile phone (or a "cellular" phone) and a computer with a mobile terminal. For example, it can be a portable, pocket-sized, handheld, computer-built-in or vehicle-mounted mobile device that exchanges language and / or data with a radio access network. For example, personal communication service (PCS) phones, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDA), Redcap terminals and other devices. A wireless terminal may also be referred to as a system, a subscriber unit, a subscriber station, a mobile station, a mobile station, a remote station, an access point, a remote terminal, an access terminal, a user terminal, a user agent, and a user device, but is not limited thereto in the embodiments of the present application.

[0378] The network device 12 involved in the embodiment of the present application may be a base station, which may include multiple cells providing services for the terminal. Depending on the specific application scenario, the base station may also be called an access point, or may be a device in the access network that communicates with the wireless terminal through one or more sectors on the air interface, or other names. The network device may be used to interchange received air frames with Internet Protocol (IP) packets, and serve as a router between the wireless terminal and the rest of the access network, wherein the rest of the access network may include an Internet Protocol (IP) communication network. The network device may also coordinate the attribute management of the air interface. For example, the network device involved in the embodiments of the present application may be a network device (Base Transceiver Station, BTS) in the Global System for Mobile communications (Global System for Mobile communications, GSM) or Code Division Multiple Access (Code Division Multiple Access, CDMA), or a network device (NodeB) in Wide-band Code Division Multiple Access (WCDMA), or an evolutionary network device (evolutional Node B, eNB or e-NodeB) in the long term evolution (long term evolution, LTE) system, a 5G base station (gNB) in the 5G network architecture (next generation system), a base station in 6G, or a home evolved Node B (Home evolved Node B, HeNB), a relay node, a home base station (femto), a pico base station (pico), etc., which is not limited in the embodiments of the present application. In some network structures, the network device may include a centralized unit (CU) and a distributed unit (DU), and the centralized unit and the distributed unit may also be arranged geographically separately.

[0379] In the embodiment of the present application, the network device and the terminal can each use one or more antennas for multiple input multiple output (MIMO) transmission, and the MIMO transmission can be single user MIMO (SU-MIMO) or multi-user MIMO (MU-MIMO). According to the form and number of antenna combinations, MIMO transmission can be two-dimensional MIMO (2D-MIMO), three-dimensional MIMO (3D-MIMO), full-dimensional MIMO (FD-MIMO) or massive MIMO, and can also be diversity transmission, precoded transmission, beamforming transmission, etc.

[0380] Before describing the embodiments of the present application, the following briefly introduces the related technologies:

[0381] In the NR system, in order to combat the path loss in high-frequency scenarios, the transmitter and receiver obtain matching beam pairs through beam management (BM) to improve the beamforming gain. In the downlink beam management process of the related technology, the base station selects a suitable transmit beam (Tx beam) for subsequent communication based on the beam measurement information reported by the UE, and indicates the selected beam information to the UE. The main defect of the related technology is that in order to obtain the optimal beam pair, the base station needs to send a channel state information reference signal (Channel State Information Reference Signal, CSI-RS) / synchronization signal block (Synchronization Signal Block, SSB) on all Tx beams, and the reference signal resource consumption is large. At the same time, the UE needs to use all receive beams (Rx beams) to measure the CSI-RS / SSB sent on all Tx beams separately, and the measurement overhead is large. To solve these problems, it is proposed to use AI / ML technology to predict the optimal beam (pair) based on the measurement of some beams (pairs) or the measurement results of historical beams (pairs), thereby saving reference signal resource overhead and UE measurement overhead, and reducing UE measurement delay.

[0382] For example, the 3rd Generation Partnership Project (3GPP) Release 18 (Rel-18) Air Interface AI topic identifies the following two sub-use cases as sub-use cases for AI beam management:

[0383] BM-case1: Spatial beam prediction, that is, predicting the best K (also called Top-K) beams (pairs) in set A (SetA) based on set B (SetB) measured at a certain moment;

[0384] BM-case2: Time domain beam prediction, that is, predicting the Top-K beams (pairs) of SetA at the next N' moments based on SetB measured at the historical N moments;

[0385] Among them, SetA can be understood as the full set of beams, which may include 32 transmit beams. SetB can be understood as the subset of beams, that is, a subset of SetA, which may include 8 transmit beams.

[0386] The output of the AI / ML model can be the ID of the predicted Top-K beams (pairs) of SetA, or the predicted Reference Signal Received Power (RSRP) of the beams (pairs) of SetA. The latter can select the Top-K beam pairs based on the size of the predicted RSRP.

[0387] As data-driven algorithms, AI / ML algorithms have generalization problems and lack theoretical guarantees for their performance. In order to ensure the performance of AI / ML applied to communication systems, it is necessary to monitor the AI / ML models.

[0388] Different AI / ML models can use different indicators for model monitoring. The model monitoring indicators currently supported by 3GPP include beam prediction indicators, or indicators such as the difference between the measured RSRP and the predicted RSRP. Among them, the beam prediction indicator is applicable to models whose output is the RSRP of Top-K beam (pair) id or SetA; the difference between the measured RSRP and the predicted RSRP is applicable to models whose output is the RSRP of SetA.

[0389] The beam prediction indicators may include the best K / best (can be called Top-K / 1) beam prediction accuracy, the Top-1 / K beam prediction accuracy, and the best beam prediction accuracy within the X dB range, which are defined as follows:

[0390] Top-K / 1 beam prediction accuracy: the probability that the predicted Top-K beam includes the actual optimal (can be called Top-1) beam;

[0391] Top-1 / K beam prediction accuracy: the probability that the predicted Top-1 beam belongs to the actual Top-K beam;

[0392] The accuracy of optimal beam prediction within X dB range is the probability that the difference between the predicted Layer 1 RSRP (L1-RSRP) measurement value of the Top-1 beam and the actual L1-RSRP measurement value of the Top-1 beam is within the X dB range.

[0393] The above-mentioned “actual Top-1 / Top-K beam” is obtained by the UE measuring a certain reference signal set, such as the UE measuring the SetA reference signal set and obtaining the beam corresponding to the Top-1 or Top-K L1-RSRP.

[0394] The difference between the measured RSRP and the predicted RSRP refers to the difference between the measured RSRP and the predicted RSRP of the beam subset (such as SetB beam or Top-K beam predicted by the model). The advantage of using this indicator is that only the beam subset needs to be sent and measured, and all beams of SetA do not need to be sent and measured, thereby saving reference signal resource overhead and UE measurement overhead.

[0395] The AI ​​model can be deployed on the base station side or on the UE side. If the model is deployed on the base station side, the base station generally monitors the model, that is, the base station compares the model prediction results (also known as inference results) with the benchmark results to determine whether the current model is applicable. If the model is deployed on the UE side, the UE can report the model inference results and measurement results to the base station, and the base station compares the model inference results with the benchmark results; or, the UE can directly report the model monitoring results to the base station, and the base station determines whether the current model is applicable.

[0396] Taking the deployment of AI / ML model on the base station side as an example, if the output of the AI / ML model is the predicted L1-RSRP of the SetA beam, the difference between the predicted L1-RSRP measurement value and the predicted L1-RSRP value of the Top-K beam is used as the model monitoring indicator. The model monitoring process is as follows: Figure 2 If the AI / ML model is deployed on the UE side, the above model monitoring process is as follows Figure 3 shown.

[0397] It can be seen that no matter whether the AI / ML model is deployed on the base station side or on the UE side, when the model is monitored by the base station side, the UE needs to report the measurement results and / or inference results to the base station. Figure 2 and Figure 3Both steps 6 and 7 involve the UE measuring the Top-K beams predicted by the model and feeding back the relevant results to the base station. In the existing layer 1 beam quality (such as L1-RSRP, L1-SINR) reporting, the base station configures or indicates the reporting resources and configures the measurement reference signal associated with the reporting. The UE feeds back the measurement results on the Physical Uplink Control Channel (PUCCH) or the Physical Uplink Shared Channel (PUSCH) through layer 1 signaling. Each time, the feedback is a single measurement result of one or some beams or an average of multiple measurement results at multiple times.

[0398] In the model monitoring process, in order to ensure the robustness of model monitoring, the performance of the AI / ML model will be judged based on multiple monitoring results of the AI / ML model over a period of time. According to the existing layer 1 beam reporting process, each time the UE measures the beam set used for model monitoring (such as the Top-K beams inferred by the model), it will feedback to the base station through layer 1 signaling. However, model monitoring judges the performance of the model based on the monitoring results over a period of time, and the latency requirements are lower than other layer 1 measurements. On the other hand, multiple measurement results of the UE may be similar, and the results of model monitoring are also similar. If all these similar results are reported to the base station, information redundancy may be caused and reporting overhead may be wasted.

[0399] In view of this, an embodiment of the present application proposes a model monitoring method. During the model monitoring process, the UE reports the measurement results of multiple model monitorings together. Furthermore, the UE can filter the measurement results and filter out the measurement results used for model monitoring for reporting. This method can reduce unnecessary reporting overhead while ensuring the completion of the model monitoring function.

[0400] The following describes the model monitoring method, model monitoring method, terminal and network device provided in the embodiments of the present application in conjunction with the accompanying drawings.

[0401] See also Figure 4 , Figure 4 : is a flow chart of a model monitoring method provided in an embodiment of the present application. Figure 4 As shown, the model monitoring method includes the following steps:

[0402] Step 401: The terminal determines whether a reporting condition of monitoring-related information of a first model is met;

[0403] Step 402: When the reporting condition is met, the terminal reports the monitoring-related information of the first model to the network device.

[0404] In an embodiment of the present application, the first model can be deployed on the terminal side or on the network device side. Regardless of which side the first model is deployed on, the network device needs to monitor the first model based on the information reported by the terminal.

[0405] The monitoring-related information of the first model (referred to as model monitoring-related information) can be understood as information required to monitor the first model, such as the measurement result of the terminal, the prediction result of the model, or the comparison result of the two. The content of the model monitoring-related information reported by the UE to the network device may also be different depending on the location where the first model is deployed and the monitoring indicators of the first model.

[0406] In the embodiment of the present application, the terminal determines whether the reporting condition of the monitoring-related information of the first model is met; if the reporting condition is met, the terminal reports the monitoring-related information of the first model to the network device. Through the above process, if the reporting condition of the monitoring-related information of the first model is met, the terminal reports the model monitoring-related information to the network device, so that the terminal can report multiple model monitoring-related information to the network device together, which can reduce the reporting overhead of the terminal during the model monitoring process.

[0407] In the embodiment of the present application, before the reporting condition is met, the terminal can store the model monitoring related information (such as measurement results) that needs to be reported. When the reporting condition is met, the terminal reports the stored model monitoring related information to the network device.

[0408] In some embodiments, the method further comprises:

[0409] When the terminal stores the measurement result of the Lth measurement, if the required storage space exceeds the remaining storage space of the terminal, the terminal discards the earliest stored measurement result and stores the measurement result of the Lth measurement.

[0410] In the embodiment of the present application, satisfying the reporting condition can be understood as the UE being triggered to report model monitoring related information. The triggering methods may include the following three:

[0411] Method 1: Periodic or semi-continuous reporting of network device configuration;

[0412] Method 2: Event trigger reporting;

[0413] Method 3: Aperiodic reporting triggered by network devices.

[0414] Among them, mode 2 is when a certain event occurs (or after it occurs), the UE reports the model monitoring related information. Mode 3 is applicable when the network equipment detects that the system performance indicators (such as throughput, beam failure probability) have deteriorated, and the network equipment triggers the UE to report the model monitoring related information non-periodically.

[0415] The reporting mechanisms under the above-mentioned modes reduce unnecessary reporting by the terminal and save reporting overhead.

[0416] After the UE reports the model monitoring related information through the above-mentioned methods 1 to 3, the network device can perform model monitoring of the first model. For example, the network device can calculate the monitoring index of the first model according to the model monitoring related information reported by the UE. The network device can determine the performance of the model based on the calculation result, and perform corresponding processing based on the determination result, such as activating the model, deactivating the model, and falling back to the traditional beam management method.

[0417] In some embodiments, the method further comprises:

[0418] The terminal determines, according to first information sent by the network device, a reporting condition for monitoring-related information of the first model, where the first information is used to indicate at least one of the following:

[0419] The monitoring and reporting period of the first model;

[0420] The monitoring start time of the first model;

[0421] The prediction result of the first model needs to meet a first preset condition.

[0422] In this implementation manner, the reporting condition of the model monitoring related information may be indicated to the UE via the first information sent by the network device.

[0423] In some embodiments, the method further comprises:

[0424] The terminal determines, according to a predefined model monitoring related parameter, a reporting condition of the monitoring related information of the first model, wherein the model monitoring related parameter includes at least one of the following:

[0425] The monitoring and reporting period of the first model;

[0426] The monitoring start time of the first model;

[0427] The prediction result of the first model needs to meet a first preset condition.

[0428] In this implementation, when the model monitoring related parameters are predefined (such as agreed upon in the protocol), the terminal may also determine the reporting conditions of the model monitoring related information based on the predefined model monitoring related parameters.

[0429] In some embodiments, the reporting condition includes at least one of the following:

[0430] The reporting time of the monitoring reporting cycle of the first model arrives; (corresponding to the triggering method of method 1)

[0431] The terminal receives reporting trigger information from the network device, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model; (corresponding to the triggering method of method 3)

[0432] The terminal detects that the prediction result of the first model satisfies the first preset condition. (Corresponding to the triggering method of method 2)

[0433] Among them, in the event-triggered reporting method (i.e., method 2), the event that triggers the reporting may be, for example, that the model performance is lower than a certain predetermined condition (such as the prediction result of the model meets the first preset condition). That is, when the model performance is lower than a certain predetermined condition, the UE reports the model monitoring related information to the network device.

[0434] In some embodiments, the first preset condition includes at least one of the following:

[0435] The number of times that the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal;

[0436] The number of times that the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, and the actual optimal K beams are determined based on the measurement result of the terminal;

[0437] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

[0438] It should be noted that the first preset condition can be configured or indicated by the network device (such as indicated by the first information) or predefined (such as agreed upon in a protocol).

[0439] In some embodiments, the monitoring-related information reported by the terminal includes at least one of the following:

[0440] a measurement result of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1;

[0441] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0442] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0443] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0444] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0445] a difference or an absolute value of the difference or an average value of the absolute value of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the actual optimal K beams;

[0446] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the layer 1 reference signal received power value of the actual optimal beam, or an absolute value of the difference, or an average value of the absolute values ​​of the difference;

[0447] Identification information of the first model.

[0448] As mentioned above, the monitoring-related information reported by the terminal may also have different contents when the model is deployed at different locations or the model monitoring indicators are different. For example, when the model is deployed on the terminal side, the monitoring-related information reported by the terminal may include the terminal's measurement results and the model's prediction results, etc.; or, the terminal's measurement results, the model's prediction results, and the comparison results of the two, etc.; or, the model's identification information. When the model is deployed on the network device side, the monitoring-related information reported by the terminal may include the terminal's measurement results.

[0449] The measurement result of the terminal can be obtained by measuring the reference signal sent by the network device. For example, the network device sends the reference signal on the full set of beams, or the network device sends the reference signal on the optimal K beams predicted by the model.

[0450] In some embodiments, the N measurements are all measurements of the terminal; or, the N measurements selected by the terminal from all measurements; or, the N measurements corresponding to the N predictions selected by the terminal from all predictions of the first model.

[0451] The terminal can report all the measurement results to the network device, or report the measurement results to the network device after filtering, or report the model prediction results to the network device after filtering. In other words, both the measurement results and prediction results used for model monitoring can be filtered results. One way is that the UE reports all measurement results (and / or model prediction results), and the network device filters them; another way is that the UE performs the filtering, and the UE only reports the filtered measurement results (and / or model prediction results).

[0452] In some embodiments, the N measurements screened by the terminal from all measurements include at least one of the following:

[0453] The first measurement within the monitoring and reporting period of the first model;

[0454] The first measurement of the first model from the start of monitoring;

[0455] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0456] The difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold value;

[0457] The difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than the fifth threshold value;

[0458] A measurement in which the difference between the average value of the measurement result and the average value of the measurement result of the previous measurement is greater than a sixth threshold;

[0459] The difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than the seventh threshold value;

[0460] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0461] The values ​​of the above thresholds can be configured or pre-agreed by the network device.

[0462] In some embodiments, the N predictions selected by the terminal from all predictions of the first model include at least one of the following:

[0463] The first forecast within the monitoring and reporting period;

[0464] The first prediction from the start time of the monitoring;

[0465] A forecast whose predicted result is different from the predicted result of the previous forecast;

[0466] The prediction that the difference between the maximum value of the predicted result and the maximum value of the predicted result of the previous prediction is greater than the ninth threshold;

[0467] The difference between the minimum value of the predicted result and the minimum value of the predicted result of the previous prediction is greater than the prediction of the tenth threshold;

[0468] The difference between the average of the predicted results and the average of the predicted results of the previous prediction is greater than the eleventh threshold;

[0469] The difference between the standard deviation of the predicted result and the standard deviation of the predicted result of the previous prediction is greater than the prediction of the twelfth threshold;

[0470] The difference between the prediction result variance and the prediction result variance of the previous prediction is greater than the prediction of the thirteenth threshold.

[0471] The values ​​of the above thresholds can be configured or pre-agreed by the network device.

[0472] In some embodiments, the terminal indicates the measurement order of the N measurements in all measurements through a bit map, or indicates measurement identification information of the N measurements.

[0473] For example, assuming that the UE completes 100 measurements on the reference signal within the monitoring reporting period T ms, and finally the UE reports 30 measurement results, the UE can indicate through a bitmap which 30 of the 100 measurements the reported 30 measurement results correspond to.

[0474] In some embodiments, the terminal reports the monitoring-related information of the first model to the network device, including:

[0475] The terminal reports the monitoring related information of the first model to the network device through the first resource;

[0476] The first resource includes any of the following:

[0477] A resource configured by the network device for reporting monitoring-related information of the first model;

[0478] PUSCH resources already available to the terminal;

[0479] The terminal requests PUSCH resources through a Scheduling Request (SR).

[0480] In this implementation manner, the resources used for UE reporting may be determined in the following manner:

[0481] a) If the reporting resources are configured by the network device, the UE reports the measurement results on the resources configured by the network device;

[0482] b) If the reporting resources are not configured by the network device and the UE already has PUSCH resources, the UE can report information on the existing PUSCH resources, such as the configured grant uplink resources;

[0483] c) If the reporting resources are not configured by the network device and the UE does not have PUSCH resources, the UE can request PUSCH resources through SR. After the network device schedules the PUSCH resources, the UE reports the stored measurement results on the PUSCH resources scheduled by the network device.

[0484] The following describes the specific implementation processes of various model monitoring solutions involved in the embodiments of the present application:

[0485] Solution 1: Assume the model is deployed on the base station side

[0486] Step 1-1: The base station configures the model monitoring reporting period T.

[0487] Step 1-2: The base station sends a reference signal for model monitoring, such as sending a reference signal on a Top-K beam inferred by the model.

[0488] Step 1-3: If the base station sends a reference signal on the Top-K beam inferred by the model, the base station stores the result of the model inference, such as the predicted value of L1-RSRP of the Top-K beam.

[0489] Step 1-4: The UE measures the reference signal sent by the base station and stores the L1-RSRP of the measured reference signal.

[0490] Repeat the above steps 1-1 to 1-4 until the time meets the reporting time of the reporting cycle T.

[0491] Step 1-5: The UE reports the measurement results stored in T. The content reported by the UE corresponds to the L1-RSRP measurement value of the Top-K beam that sent the reference signal on the Top-K beam inferred by the model within time T. The resources used by the UE report can be determined as follows:

[0492] a) If the reporting resources are configured by the base station, the UE reports the measurement results within time T through the resources configured by the base station;

[0493] b) If the reporting resources are not configured by the base station, the UE reports the measurement results within time T through the existing PUSCH resources or the PUSCH resources requested by the SR.

[0494] Step 1-6: After receiving the measurement results reported by the UE, the base station calculates the model monitoring indicators:

[0495] a) The base station calculates the average of the absolute values ​​of the differences between the predicted values ​​of L1-RSRP of the Top-K beams stored within time T and the measured values ​​reported by the UE; or,

[0496] b) The base station selects the result for calculating the model monitoring index according to the report result of the UE, and the selection method may include the following methods:

[0497] i. If the measurement is the first measurement within the reporting period, select the measurement result;

[0498] ii. If it is not the first measurement, the base station determines whether there is a different measurement result of L1-RSRP in the measurement reported by the UE and the previous measurement reported, and if so, the measurement result is selected;

[0499] iii. The base station determines whether the difference between the maximum value, minimum value or average value of the reference signal quality measured this time and the previous measurement is greater than X dB; if so, the measurement result is selected;

[0500] iv. The base station determines whether the difference between the standard deviation / variance of the reference signal quality measured this time and the standard deviation / variance of the reference signal quality measured last time is greater than Y dB or Y%; if so, the measurement result of this time is selected.

[0501] The base station selects the inference result corresponding to the measurement result selected according to the above principle from the stored inference results, and calculates the average value of the absolute value of the difference between the predicted value of L1-RSRP of the Top-K beam and the measurement value reported by the UE based on the selected inference result and the selected measurement result.

[0502] Step 1-7: The base station determines the performance of the base station-side model based on the model monitoring indicators, and performs operations such as model activation, model deactivation, or fallback to the traditional beam management method:

[0503] a) If the value of the monitoring index of one model is less than the value of the monitoring index of another model, the model corresponding to the smaller value of the monitoring index is activated, and the model corresponding to the larger value of the monitoring index is deactivated;

[0504] b) If the calculated monitoring indicator value of a model is greater than a threshold, the model is deactivated or falls back to the traditional beam management method.

[0505] Solution 2: Assume the model is deployed on the base station side

[0506] Step 2-1: Same as step 1-1 in solution 1.

[0507] Step 2-2: Same as step 1-2 in solution 1.

[0508] Step 2-3: Same as steps 1-3 in solution 1.

[0509] Step 2-4: The UE measures the reference signal sent by the base station and determines whether the measurement result needs to be stored:

[0510] a) If the measurement is the first measurement in the reporting period, the UE stores the result;

[0511] b) If it is not the first measurement, the UE determines whether there is a measurement result of L1-RSRP different from that of the previous measurement. If so, the UE stores the result;

[0512] Alternatively, the UE determines whether the difference between the maximum value, the minimum value, or the average value of the reference signal quality measured this time and the previous measurement is greater than X dB; if so, the UE stores the result;

[0513] Alternatively, the UE determines whether the difference between the standard deviation / variance of the reference signal quality measured this time and the standard deviation / variance of the reference signal quality measured last time is greater than Y dB or Y %; if so, the UE stores the measurement result.

[0514] Repeat the above steps 2-1 to 2-4 until the time meets the reporting time of the reporting cycle T.

[0515] Step 2-5: The UE reports the measurement results stored in T. The content reported by the UE corresponds to the measurement values ​​that meet the conditions in step 2-4 in the L1-RSRP of the Top-K beam that transmits the reference signal on the Top-K beam inferred by the model within time T, and which measurements within T these measurement results correspond to. The UE can report on the existing PUSCH resources or the PUSCH resources requested through SR.

[0516] Step 2-6: After receiving the measurement result reported by the UE, the base station selects the model inference result corresponding to the measurement result reported by the UE from the stored model inference results according to the reporting information of the UE, and calculates the model monitoring index. For example, the base station calculates the average of the absolute values ​​of the difference between the predicted value of L1-RSRP of the Top-K beam stored within time T and the measurement value reported by the UE.

[0517] Steps 2-7: Same as steps 1-7 in solution 1.

[0518] Solution 3: Assume the model is deployed on the UE side

[0519] Step 3-1: The base station configures the model monitoring reporting period T.

[0520] Step 3-2: The base station sends a reference signal for model monitoring. For example, the reference signal is sent on the Top-K beam inferred by the model, and the model inference result is reported to the base station by the UE.

[0521] Step 3-3: The UE measures the reference signal sent by the base station and stores the measurement results and the corresponding model inference results. The UE can store part of the measurement results and the inference results according to the following method:

[0522] a) The UE determines whether the measurement result needs to be stored, and also stores the model inference result corresponding to the stored measurement result:

[0523] i. If the measurement is the first measurement in the reporting period, the UE stores the result;

[0524] ii. If it is not the first measurement, the UE determines whether there is a measurement result different from the previous measurement of L1-RSRP. If so, the UE stores the result;

[0525] Alternatively, the UE determines whether the difference between the maximum value, the minimum value, or the average value of the reference signal quality measured this time and the previous measurement is greater than X dB; if so, the UE stores the result;

[0526] Alternatively, the UE determines whether the difference between the standard deviation / variance of the reference signal quality measured this time and the standard deviation / variance of the reference signal quality measured last time is greater than Y dB or Y%; if so, the UE stores the measurement result;

[0527] b) The UE determines whether the inference result needs to be stored, and also stores the measurement result corresponding to the stored inference result:

[0528] i. If the inference result is the first inference within the reporting period, the UE stores the inference result;

[0529] ii. If it is not the first reasoning, the UE determines whether there is a different prediction result of L1-RSRP in the reasoning and the previous reasoning. If so, the UE stores the reasoning result;

[0530] Alternatively, the UE determines whether the difference between the maximum value, minimum value or average value of the reference signal L1-RSRP prediction value in the current inference and the previous inference is greater than X dB; if so, the UE stores the inference result;

[0531] Alternatively, the UE determines whether the difference between the standard deviation / variance of the L1-RSRP prediction value of the reference signal inferred this time and the standard deviation / variance of the L1-RSRP prediction value of the reference signal inferred last time is greater than Y dB or Y%; if so, the UE stores the measurement inference result this time;

[0532] The UE stores the measured L1-RSRP value and the inferred L1-RSRP value selected according to method a) or method b), or stores the difference between the selected measured L1-RSRP value and the inferred L1-RSRP value.

[0533] Repeat the above steps 3-1 to 3-3 until the time meets the reporting time of the reporting cycle T.

[0534] Step 3-4: The UE reports the results stored in T. The content reported by the UE corresponds to the measurement results and inference results that meet the conditions in step 3-3 within the time T, or reports the difference between the measurement results and the inference results. The UE can report on the existing PUSCH resources or on the PUSCH resources requested by the SR.

[0535] Step 3-5: After receiving the measurement results reported by the UE, the base station calculates the model monitoring indicators:

[0536] a) If the UE reports the measurement result and the inference result, the base station calculates the average of the absolute values ​​of the difference between the reported predicted value of L1-RSRP and the measured value;

[0537] b) If the UE reports the difference between the measurement result and the inference result, the base station calculates the average of the absolute values ​​of the difference.

[0538] Step 3-6: The base station determines the UE-side model performance based on the model monitoring indicators and performs operations such as model activation, model deactivation, or fallback to the traditional beam management method:

[0539] a) If the model monitoring index value calculated according to step 3-5 is less than another model monitoring index value, activate the model corresponding to the smaller value of the model monitoring index, and deactivate the model corresponding to the larger value of the model monitoring index;

[0540] b) If the model monitoring indicator value calculated according to steps 3-5 is greater than the threshold, deactivate the model or fall back to the traditional beam management method.

[0541] Solution 4: Assume the model is deployed on the UE side

[0542] Step 4-1: The base station configures the model monitoring reporting period T.

[0543] Step 4-2: The base station sends a reference signal for model monitoring, such as sending a reference signal on the beam of SetA.

[0544] Step 4-3: The UE measures the reference signal sent by the base station and makes the following judgments based on the measurement results and the corresponding inference results:

[0545] a) Whether the Top-1 beam inferred by the model belongs to the actual Top-K beam; or,

[0546] b) Whether the Top-K beams inferred by the model include the actual Top-1 beam;

[0547] The above-mentioned actual Top-1 beam is the beam corresponding to the maximum L1-RSRP measurement value in the reference signal sent on the SetA beam; the actual Top-K beam is the beam corresponding to the Top-KL1-RSRP measurement value in the reference signal sent on the SetA beam;

[0548] If not included, the count is increased by 1.

[0549] Repeat the above steps 4-1 to 4-3 until the time meets the reporting time of the reporting cycle T.

[0550] Step 4-4: The UE reports the number of times that the Top-1 beam inferred by the model within T does not belong to the actual Top-K beam, or the number of times that the Top-K beam inferred by the model does not include the actual Top-1 beam. The resources used by the UE report can be determined as follows:

[0551] a) If the reporting resources are configured by the base station, the UE reports the results on the resources configured by the base station;

[0552] b) If the reporting resources are not configured, the UE reports information on the existing PUSCH resources or the PUSCH resources requested through SR.

[0553] Step 4-5: After receiving the measurement results reported by the UE, the base station calculates the model monitoring indicators:

[0554] a) If the UE reports the number of times that the Top-1 beam inferred by the model does not belong to the actual Top-K beam, the base station calculates the model prediction accuracy (which can be expressed as Top-1 / K%) based on the number of times reported by the UE and the number of times the SetA reference signal is sent within T;

[0555] b) If the UE reports the number of times that the Top-K beams inferred by the model do not include the actual Top-1 beam, the base station calculates the model prediction accuracy (which can be expressed as Top-K / 1%) based on the number of times reported by the UE and the number of times the SetA reference signal is sent within T.

[0556] Step 4-6: The base station determines the UE-side model performance based on the model monitoring indicators, and performs operations such as model activation, model deactivation, or fallback to the traditional beam management method, which is the same as steps 3-6 in solution three.

[0557] Solution 5: Assume the model is deployed on the UE side

[0558] Step 5-1: The base station instructs the model to monitor the start time.

[0559] Step 5-2: The base station sends a reference signal for model monitoring, such as sending a reference signal on a beam of SetA or Top-K of model inference.

[0560] Step 5-3: The UE measures the reference signal sent by the base station, and performs the following operations based on the measurement result and the corresponding inference result:

[0561] a) Determine whether the Top-1 beam inferred by the model belongs to the actual Top-K beam; or,

[0562] b) Determine whether the Top-K beams inferred by the model include the actual Top-1 beam;

[0563] The above-mentioned actual Top-1 beam is the beam corresponding to the maximum L1-RSRP measurement value in the reference signal sent on the SetA beam; the actual Top-K beam is the beam corresponding to the Top-KL1-RSRP measurement value in the reference signal sent on the SetA beam;

[0564] If not included, the count is increased by 1; or,

[0565] c) Calculate the difference between the predicted value and the measured value of L1-RSRP of the Top-K beams inferred by the model.

[0566] Repeat steps 5-1 to 5-3 above until the following conditions are met, and then execute step 5-4.

[0567] Step 5-4: UE reports information to the base station:

[0568] a) If the UE performs calculation according to a) in step 5-3, when the number of times that the Top-1 beam inferred by the model does not belong to the actual Top-K beam is greater than or equal to N times, the UE reports the information;

[0569] b) If the UE performs calculation according to b) in step 5-3, when the number of times that the Top-K beams inferred by the model do not include the actual Top-1 beam is greater than or equal to N times, the UE reports the information;

[0570] c) If the UE performs calculation according to c) in step 5-3, when the average absolute value of the difference between the predicted value and the measured value of L1-RSRP of the Top-K beam inferred by the model is greater than W dB, the UE reports the information.

[0571] In addition, the content reported by the UE may include the model ID that meets the above conditions, and may also include the specific value of the model monitoring indicator, such as the average of the absolute value of the difference between the predicted value and the measured value of L1-RSRP of the Top-K beam inferred by the reporting model.

[0572] The UE can report on the existing PUSCH resources or the PUSCH resources requested through SR.

[0573] Step 5-5: The base station determines the UE-side model performance based on the results reported by the UE, and performs operations such as model activation, model deactivation, or fallback to the traditional beam management method, which is the same as steps 3-6 in solution three.

[0574] Solution 6: Assume the model is deployed on the UE side

[0575] Step 6-1: The base station instructs the model to monitor the start time.

[0576] Step 6-2: The base station sends a reference signal for model monitoring, such as sending a reference signal on a Top-K beam of model inference.

[0577] Step 6-3: The UE measures the reference signal sent by the base station and determines whether the measurement result and inference result need to be stored:

[0578] a) The UE determines whether the measurement result needs to be stored, and also stores the model inference result corresponding to the stored measurement result:

[0579] i. If the measurement is the first measurement in the reporting period, the UE stores the result;

[0580] ii. If it is not the first measurement, the UE determines whether there is a measurement result different from the previous measurement of L1-RSRP. If so, the UE stores the result;

[0581] Alternatively, the UE determines whether the difference between the maximum value, the minimum value, or the average value of the reference signal quality measured this time and the previous measurement is greater than X dB; if so, the UE stores the result;

[0582] Alternatively, the UE determines whether the difference between the standard deviation / variance of the reference signal quality measured this time and the standard deviation / variance of the reference signal quality measured last time is greater than Y dB or Y%; if so, the UE stores the measurement result;

[0583] Alternatively, b) the UE determines whether the inference result needs to be stored, and also stores the model monitoring measurement result corresponding to the stored inference result:

[0584] i. If the inference result is the first inference within the reporting period, the UE stores the inference result;

[0585] ii. If it is not the first reasoning, the UE determines whether there is a different prediction result of L1-RSRP in the reasoning and the previous reasoning. If so, the UE stores the reasoning result;

[0586] Alternatively, the UE determines whether the difference between the maximum value, minimum value or average value of the reference signal L1-RSRP prediction value in the current inference and the previous inference is greater than X dB; if so, the UE stores the inference result;

[0587] Alternatively, the UE determines whether the difference between the standard deviation / variance of the L1-RSRP prediction value of the reference signal inferred this time and the standard deviation / variance of the L1-RSRP prediction value of the reference signal inferred last time is greater than Y dB or Y%; if so, the UE stores the measurement inference result this time.

[0588] The UE stores the measured L1-RSRP value selected according to the above method a) or b) and the inferred L1-RSRP value, or stores the difference between the selected measured L1-RSRP value and the inferred L1-RSRP value.

[0589] Repeat steps 6-1 to 6-3 above.

[0590] Step 6-4: If the storage capacity of a certain result exceeds the maximum storage capacity of the UE, the UE discards the earliest result among the saved results and stores the current result.

[0591] Step 6-5: When the system performance indicators monitored by the base station (such as throughput rate and beam failure probability) deteriorate, the base station triggers the UE to report the model monitoring related results, and then determines whether the system performance deterioration is caused by the degradation of model performance.

[0592] Step 6-6: After receiving the aperiodic reporting signaling triggered by the base station, the UE reports the stored result. The UE can report the stored result in the existing PUSCH resources or the PUSCH resources requested through the SR.

[0593] Step 6-7: After receiving the measurement results reported by the UE, the base station calculates the model monitoring indicators, which is the same as steps 3-5 in solution three.

[0594] Step 6-8: The base station determines the UE-side model performance based on the model monitoring indicators, and performs operations such as model activation, model deactivation, or fallback to the traditional beam management method, which is the same as steps 3-6 in solution three.

[0595] In order to better understand the technical solutions of the embodiments of the present application, specific embodiments are provided below for illustrative purposes.

[0596] Embodiment 1 (corresponding scheme 1):

[0597] In this embodiment, the base station side deploys an AI / ML model for airspace prediction. The AI / ML model predicts the optimal beam in SetA beam based on the measurement results of SetB beam. The input of the AI / ML model is the L1-RSRP of SetB beam, and the output is the predicted RSRP of all beams in SetA. SetA includes 32 transmit beams, SetB includes 8 transmit beams, and SetB is a subset of SetA.

[0598] The base station periodically sends the reference signal corresponding to the SetB beam. After the UE feeds back the L1-RSRP of the SetB beam, the base station performs inference to obtain the predicted L1-RSRP of the 32 beams of SetA, and uses the three beams with the largest predicted L1-RSRP values ​​as the Top-3 beams. The base station sends the reference signal corresponding to the Top-3 beams inferred by the AI / ML model, and the UE measures the L1-RSRP of the Top-3 beams. The reference signal corresponding to the Top-3 beams sent by the base station may only be the Top-3 reference signal of some inference results, and there is no need to monitor the results of each inference. For example, the base station sends the reference signal corresponding to the inferred Top-3 beams at regular intervals, or the inference result of the model is significantly different from the previous inference result. The base station sends the reference signal corresponding to the inferred Top-3 beams only when it sends the reference signal corresponding to the inferred Top-3 beams.

[0599] Assume that the model monitoring indicator is the difference between the measured value and the predicted value of L1-RSRP of the Top-3 beams predicted by the AI ​​model. The base station configures the UE's reporting period to be T ms, and the reporting content is the L1-RSRP measurement value of the reference signal of the Top-3 beams configured by the base station within T ms. The reporting content is carried by MAC-CE signaling. The base station sends the reference signal of the Top-3 beams inferred by the model a total of 100 times within Tms, and the base station stores the 100 predicted values ​​of L1-RSRP of the Top-3 beams inferred by the model. After each measurement of T ms is completed, if the UE already has PUSCH resources, the UE reports on the existing PUSCH resources, such as the configured grant uplink resources; if the UE does not have PUSCH resources, the UE requests PUSCH resources through SR. After the base station schedules PUSCH resources, the UE reports the L1-RSRP measurement value of the reference signal of the Top-3 beams measured 100 times within T ms on the PUSCH resources scheduled by the base station. Figure 5 shown.

[0600] After receiving the L1-RSRP measurement value of the reference signal of the Top-3 beams within Tms reported by the UE, the base station selects the results with different L1-RSRP measurement values ​​of the Top-3 beams. If the L1-RSRP of a group of Top-3 beams is the same as the L1-RSRP of the previous group of Top-3 beams, the group of measurement results is discarded, otherwise the reorganized measurement results are retained. Assume that the base station finally selects 30 measurement results from the 100 Top-3 beam measurement results. The base station selects the model inference results corresponding to the 30 retained measurement results from the 100 stored model inference results, calculates the difference between the average Top-3 beam L1-RSRP prediction value and the measurement value based on these 30 measurement results and the inference results, and judges the performance of the AI / ML model. For example, when the average of the absolute value of the difference between the L1-RSRP prediction value and the measurement value of these 30 Top-3 beams is greater than 1dB, it is considered that the performance of the current AI model has deteriorated and the AI ​​model is deactivated.

[0601] Embodiment 2 (corresponding to scheme 2):

[0602] In this embodiment, an AI / ML model is deployed on the base station side for airspace prediction, and the input and output of the AI / ML model are the same as those in Embodiment 1.

[0603] The base station periodically sends the reference signal corresponding to the SetB beam. After the UE feeds back the L1-RSRP of the SetB beam, the base station infers and obtains the predicted L1-RSRP of the 32 beams of SetA, and uses the three beams with the largest predicted L1-RSRP values ​​as the Top-3 beams. The base station sends the reference signal corresponding to the Top-3 beams inferred by the AI / ML model, and the UE measures the L1-RSRP of the Top-3 beams. Each time the UE measures the L1-RSRP of the Top-3 beams, it determines whether the measurement result is consistent with the previous measurement result of the Top-3 beams. If they are consistent, the measurement result is not stored. If they are inconsistent, the result is stored.

[0604] Each time the measurement of T ms is completed, the UE reports the stored measurement results and indicates which measurements within Tms the measurement results reported by the UE correspond to. For example, if the UE completes 100 measurements of the Top-3 beams within T ms, and finally the UE reports 30 measurement results, the bitmap can be used to indicate which 30 of the 100 measurements the 30 measurement results reported by the base station correspond to.

[0605] If the UE already has PUSCH resources, the UE reports on the existing PUSCH resources, such as the configured grant uplink resources; if the UE does not have PUSCH resources, the UE requests PUSCH resources through SR. After the base station schedules the PUSCH resources, the UE reports the stored measurement results on the PUSCH resources scheduled by the base station.

[0606] After receiving the 30 measurement results reported by the UE, the base station selects the model inference results corresponding to the 30 measurement results from the stored model inference results, calculates the difference between the average Top-3 beam L1-RSRP prediction value and the measured value based on the 30 measurement results and the inference results, and judges the performance of the AI / ML model. For example, when the average of the absolute value of the difference between the 30 Top-3 beam L1-RSRP prediction values ​​and the measured values ​​is greater than 1dB, it is considered that the performance of the current AI model has deteriorated and the AI ​​model is deactivated.

[0607] Embodiment 3 (corresponding to scheme 4):

[0608] In this embodiment, the UE side deploys an AI / ML model for airspace prediction, and the AI / ML model predicts the optimal beam in SetA beam based on the measurement results of SetB beam. The input of the AI / ML model is the L1-RSRP of SetB beam, and the output is the predicted RSRP of all beams in SetA. SetA includes 32 transmit beams, SetB includes 8 transmit beams, and SetB is a subset of SetA.

[0609] The base station periodically sends reference signals corresponding to the SetB beam and the SetA beam, and the period of the reference signal corresponding to the SetA beam may be different from the period of the reference signal corresponding to the SetB beam. The UE measures the L1-RSRP of the SetB reference signal, and the input of the AI / ML model is the L1-RSRP of SetB for inference, and the output is the predicted Top-3 beams in the SetA beam set. The UE feeds back the index value corresponding to the inferred Top-3 beams to the base station. The UE measures the L1-RSRP of the SetA reference signal, determines the actual Top-1 beam based on the measurement results of SetA, and determines whether the Top-3 beams inferred by the corresponding model include the actual Top-1 beam. Among them, the index of the reference signal with the largest L1-RSRP measurement value in the SetA beam reference signal is the index value corresponding to the actual Top-1 beam.

[0610] Every Tms, the UE feeds back the measurement results of the Top-3 beams of this Tms to the base station. The model-inferred Top-3 beams do not include the number of actual Top-1 beams. If the UE already has PUSCH resources, the UE reports on the existing PUSCH resources, such as the configured grant uplink resources; if the UE does not have PUSCH resources, the UE requests PUSCH resources through SR. After the base station schedules PUSCH resources, the UE reports on the PUSCH resources scheduled by the base station.

[0611] The base station receives the number of Top-3 beams predicted by the UE-side model within Tms reported by the UE that do not include the actual Top-1 beam, and calculates the model prediction accuracy. For example, if the number of times reported by the UE is 30, and 100 SetA measurements are completed within Tms, the accuracy is 70%. The base station determines that the model accuracy is lower than the threshold (such as 80%) and deactivates the current AI model.

[0612] Embodiment 4 (corresponding to scheme 5):

[0613] In this embodiment, the UE side deploys an AI / ML model for airspace prediction. The AI / ML model predicts the optimal beam in SetA beam based on the measurement results of SetB beam. The input of the AI / ML model is the L1-RSRP of SetB beam, and the model output is the index value corresponding to the top-2 beam in SetA. SetA includes 32 transmit beams, and SetB includes 8 transmit beams.

[0614] The base station instructs the UE to start model monitoring, such as instructing the UE to start the timer for model monitoring. The model monitoring indicator is the Top-K / 1 beam prediction accuracy (the probability that the predicted Top-K beam includes the actual Top-1 beam). The base station periodically sends the reference signals corresponding to the SetB and SetA beams. The UE measures the L1-RSRP of the SetB reference signal, uses the L1-RSRP of SetB as the model input for inference, and outputs the index value corresponding to the Top-2 beam of SetA. The UE measures the reference signal corresponding to the SetA beam, and the index of the reference signal with the largest L1-RSRP measurement value in the SetA beam reference signal is the index value corresponding to the actual Top-1 beam. The UE determines whether the index value corresponding to the Top-2 beam predicted by the model includes the index value corresponding to the actual Top-1 beam. If not, the UE side counts 1 time. Before the model monitoring timer times out (the timer timeout is configured by the base station or specified by the protocol), if the number of times the model-inferred Top-2 beam does not include the actual Top-1 beam reaches 10, the UE reports to the MAC-CE to inform the base station that the model-inferred Top-2 beam does not include the actual Top-1 beam 10 times. Figure 5 shown.

[0615] If multiple AI / ML models for spatial beam prediction are deployed on the UE side, and in model monitoring, in addition to monitoring the performance of the currently activated AI / ML model, the UE also tests the performance of other inactivated models for spatial beam prediction, then the IDs of other inactivated AI / ML models can be reported in MAC-CE. The performance of the reported inactivated AI / ML model meets the Top-2 / 1 beam prediction accuracy greater than the threshold or is better than the activated AI / ML model within the monitoring time. After receiving the MAC-CE reported by the UE, the base station determines the performance of the AI / ML model and decides whether to deactivate the current model and whether to activate other AI / ML models reported in the MAC-CE that meet the Top-2 / 1 beam prediction accuracy threshold requirements.

[0616] Embodiment 5 (corresponding to scheme 6):

[0617] In this embodiment, an AI / ML model is deployed on the UE side for airspace prediction, and the assumptions of the input, output, and model monitoring indicators of the AI / ML model are the same as those in Embodiment 1.

[0618] The base station periodically sends the reference signal corresponding to the SetB beam. The UE measures the L1-RSRP of the SetB reference signal and then performs inference to obtain the predicted L1-RSRP of the 32 beams of SetA, and uses the three beams with the largest predicted L1-RSRP values ​​as the Top-3 beams. The UE feeds back the index value corresponding to the Top-3 beams to the base station. For certain inference results, the base station sends the reference signal corresponding to the Top-3 beams inferred by the AI / ML model, and the UE measures the L1-RSRP of the Top-3 beams. Each time the UE measures the L1-RSRP of the Top-3 beams, it determines whether the measurement result is consistent with the previous measurement result of the Top-3 beams. If they are consistent, the measurement result is not stored. If they are inconsistent, the measurement result is stored. If storing a certain measurement result exceeds the storage capacity of the UE, the UE discards the earliest measurement result and saves the measurement result.

[0619] The base station triggers the UE to perform non-periodic reporting through MAC-CE signaling. The reported content is the L1-RSRP prediction value and measurement value of the Top-3 beams inferred by the model stored in the UE. The reported content is carried by MAC-CE signaling or RRC signaling. If the UE already has PUSCH resources, the UE reports on the existing PUSCH resources, such as the configured grant uplink resources; if the UE does not have PUSCH resources, the UE requests PUSCH resources through SR. After the base station schedules PUSCH resources, the UE reports on the PUSCH resources scheduled by the base station.

[0620] After receiving the results reported by the UE, the base station calculates the difference between the average top-3 beam L1-RSRP prediction value and the measured value to determine the performance of the AI / ML model. The base station can trigger non-periodic reporting on demand. For example, when the network performance indicators are good, the base station may not trigger reporting; when the network performance indicators decline, the base station may trigger non-periodic reporting to monitor the performance of the AI / ML model in the recent period of time, so as to determine whether the AI / ML model causes the network performance to decline.

[0621] The embodiments of the present application focus on how the UE reports multiple model monitoring related information to assist the base station in determining the performance of the base station side model or the UE side model. The embodiments of the present application propose that the UE feeds back multiple model monitoring related information (such as measurement results of multiple measurements) together, which can reduce the feedback overhead of the UE. The embodiments of the present application also propose that the model monitoring related information reported by the UE can be obtained after screening according to certain criteria, which can further reduce the feedback overhead of the UE and reduce the computing resource overhead of the base station. The embodiments of the present application also propose that the base station selects the corresponding information calculation model monitoring indicators from the model monitoring related information reported by the UE. The embodiments of the present application also propose that the base station decides whether to trigger the UE to report the model monitoring related information non-periodically based on whether the monitored system performance indicators have deteriorated, and then determines whether the system performance deterioration is caused by the decline in model performance.

[0622] In summary, the embodiments of the present application can reduce unnecessary overhead while ensuring the completion of the model monitoring function, thereby improving the performance of the model applied to the communication network.

[0623] See also Figure 6 , Figure 6 : is a flow chart of a model monitoring method provided in an embodiment of the present application. Figure 6 As shown, the model monitoring method includes the following steps:

[0624] Step 601: The network device receives monitoring related information of the first model reported by the terminal;

[0625] Step 602: The network device performs model monitoring of the first model based on the monitoring-related information.

[0626] Optionally, the method further comprises:

[0627] The network device sends first information to the terminal, where the first information is used to indicate at least one of the following:

[0628] The monitoring and reporting period of the first model;

[0629] The monitoring start time of the first model;

[0630] The prediction result of the first model needs to meet a first preset condition.

[0631] Optionally, the reporting condition is determined by predefined model monitoring related parameters, and the model monitoring related parameters include at least one of the following:

[0632] The monitoring and reporting period of the first model;

[0633] The monitoring start time of the first model;

[0634] The prediction result of the first model needs to meet a first preset condition.

[0635] Optionally, the method further comprises:

[0636] The network device sends reporting trigger information to the terminal, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model.

[0637] Optionally, the network device sending reporting trigger information to the terminal includes:

[0638] When the network device monitors that the system performance indicator meets the second preset condition, the network device sends reporting trigger information to the terminal.

[0639] Optionally, the first preset condition includes at least one of the following:

[0640] The number of times that the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal;

[0641] The number of times that the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, and the actual optimal K beams are determined based on the measurement result of the terminal;

[0642] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

[0643] Optionally, the network device performs model monitoring of the first model based on the monitoring related information, including:

[0644] The network device calculates the monitoring index of the first model based on the monitoring-related information.

[0645] Optionally, when the first model is deployed on the network device, the monitoring-related information includes measurement results of the terminal measuring the reference signal N times, where N is an integer greater than 1;

[0646] The network device calculates the monitoring index of the first model based on the monitoring related information, including:

[0647] The network device selects N predictions corresponding to the N measurements from all predictions of the first model;

[0648] The network device calculates the monitoring index of the first model according to the prediction results of the N predictions and the measurement results of the N measurements;

[0649] or,

[0650] The network device calculates the monitoring index of the first model based on the monitoring related information, including:

[0651] The network device selects M measurements from the N measurements, where M is an integer greater than 1;

[0652] The network device selects M predictions corresponding to the M measurements from all predictions of the first model;

[0653] The network device calculates the monitoring index of the first model according to the prediction results of the M predictions and the measurement results of the M measurements.

[0654] Optionally, the M measurements screened out by the network device from the N measurements include at least one of the following:

[0655] The first measurement within the monitoring and reporting period;

[0656] The first measurement from the start time of the monitoring;

[0657] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0658] The difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold value;

[0659] The difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than the fifth threshold value;

[0660] A measurement in which the difference between the average value of the measurement result and the average value of the measurement result of the previous measurement is greater than a sixth threshold;

[0661] The difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than the seventh threshold value;

[0662] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0663] Optionally, when the first model is deployed on the terminal, the monitoring-related information includes at least one of the following:

[0664] a measurement result of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1;

[0665] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0666] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0667] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0668] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0669] a difference or an absolute value of the difference or an average value of the absolute value of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the actual optimal K beams;

[0670] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the layer 1 reference signal received power value of the actual optimal beam, or an absolute value of the difference, or an average value of the absolute values ​​of the difference;

[0671] Identification information of the first model.

[0672] Optionally, the network device receives monitoring-related information of the first model reported by the terminal, including:

[0673] The network device receives, through a first resource, monitoring-related information of the first model reported by the terminal;

[0674] The first resource includes any of the following:

[0675] A resource configured by the network device for reporting monitoring-related information of the first model;

[0676] Physical uplink shared channel PUSCH resources already available to the terminal;

[0677] The terminal requests PUSCH resources via a scheduling request SR.

[0678] Optionally, the method further comprises:

[0679] The network device performs a first process according to the monitoring indicator of the first model, where the first process includes any one of the following:

[0680] activating the first model;

[0681] Deactivate the first model.

[0682] It should be noted that this embodiment is Figures 4 to 5 The implementation methods of the corresponding network devices or base stations in the embodiments shown in the figure can be found in the specific implementation methods. Figures 4 to 5In order to avoid repeated description, the relevant description of the embodiment shown will not be repeated in this embodiment, and the same beneficial effects can still be achieved.

[0683] See also Figure 7 , Figure 7 is a structural diagram of a terminal provided in an embodiment of the present application. Figure 7 As shown, it includes a memory 720, a transceiver 700 and a processor 710:

[0684] The memory 720 is used to store computer programs; the transceiver 700 is used to send and receive data under the control of the processor 710; the processor 710 is used to read the computer program in the memory 720 and perform the following operations:

[0685] Determining whether a reporting condition of monitoring-related information of the first model is met;

[0686] When the reporting condition is met, the monitoring related information of the first model is reported to the network device.

[0687] In the embodiment of the present application, the terminal determines whether the reporting condition of the monitoring-related information of the first model is met; if the reporting condition is met, the terminal reports the monitoring-related information of the first model to the network device. Through the above process, if the reporting condition of the monitoring-related information of the first model is met, the terminal reports the monitoring-related information of the first model to the network device, so that the terminal can report multiple model monitoring-related information to the network device together, which can reduce the reporting overhead of the terminal during the model monitoring process.

[0688] Among them, Figure 7 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors represented by processor 710 and various circuits of memory represented by memory 720 are linked together. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 700 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium, including wireless channels, wired channels, optical cables, and other transmission media. For different user devices, the user interface 730 may also be an interface capable of externally and internally connecting required devices, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, and the like.

[0689] The processor 710 is responsible for managing the bus architecture and general processing, and the memory 720 can store data used by the processor 710 when performing operations.

[0690] Optionally, the processor 710 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD), and the processor may also adopt a multi-core architecture.

[0691] The processor calls the computer program stored in the memory to execute any of the methods provided in the embodiments of the present application according to the obtained executable instructions. The processor and the memory can also be arranged physically separately.

[0692] Optionally, the processor 710 is configured to read the computer program in the memory and perform the following operations:

[0693] Determine, according to the first information sent by the network device, a reporting condition of the monitoring-related information of the first model, wherein the first information is used to indicate at least one of the following:

[0694] The monitoring and reporting period of the first model;

[0695] The monitoring start time of the first model;

[0696] The prediction result of the first model needs to meet a first preset condition.

[0697] Optionally, the processor 710 is configured to read the computer program in the memory and perform the following operations:

[0698] Determine a reporting condition for monitoring-related information of the first model according to predefined model monitoring-related parameters, wherein the model monitoring-related parameters include at least one of the following:

[0699] The monitoring and reporting period of the first model;

[0700] The monitoring start time of the first model;

[0701] The prediction result of the first model needs to meet a first preset condition.

[0702] Optionally, the reporting condition includes at least one of the following:

[0703] The reporting time of the monitoring reporting cycle of the first model arrives;

[0704] The terminal receives reporting trigger information from the network device, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model;

[0705] The terminal detects that the prediction result of the first model meets a first preset condition.

[0706] Optionally, the first preset condition includes at least one of the following:

[0707] The number of times that the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal;

[0708] The number of times that the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, and the actual optimal K beams are determined based on the measurement result of the terminal;

[0709] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

[0710] Optionally, the monitoring-related information reported by the terminal includes at least one of the following:

[0711] a measurement result of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1;

[0712] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0713] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0714] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0715] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0716] a difference or an absolute value of the difference or an average value of the absolute value of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the actual optimal K beams;

[0717] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the layer 1 reference signal received power value of the actual optimal beam, or an absolute value of the difference, or an average value of the absolute values ​​of the difference;

[0718] Identification information of the first model.

[0719] Optionally, the N measurements are all measurements; or, N measurements selected from all measurements; or, N measurements corresponding to N predictions selected from all predictions of the first model.

[0720] Optionally, the processor 710 is configured to read the computer program in the memory and perform the following operations:

[0721] The measurement order of the N measurements in all measurements is indicated by a bit map, or the measurement identification information of the N measurements is indicated.

[0722] Optionally, the N measurements screened from all measurements include at least one of the following:

[0723] The first measurement within the monitoring and reporting period of the first model;

[0724] The first measurement of the first model from the start of monitoring;

[0725] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0726] The difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold value;

[0727] The difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than the fifth threshold value;

[0728] A measurement in which the difference between the average value of the measurement result and the average value of the measurement result of the previous measurement is greater than a sixth threshold;

[0729] The difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than the seventh threshold value;

[0730] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0731] Optionally, the N predictions selected from all predictions of the first model include at least one of the following:

[0732] The first forecast within the monitoring and reporting period;

[0733] The first prediction from the start time of the monitoring;

[0734] A forecast whose predicted result is different from the predicted result of the previous forecast;

[0735] The prediction that the difference between the maximum value of the predicted result and the maximum value of the predicted result of the previous prediction is greater than the ninth threshold;

[0736] The difference between the minimum value of the predicted result and the minimum value of the predicted result of the previous prediction is greater than the prediction of the tenth threshold;

[0737] The difference between the average of the predicted results and the average of the predicted results of the previous prediction is greater than the eleventh threshold;

[0738] The difference between the standard deviation of the predicted result and the standard deviation of the predicted result of the previous prediction is greater than the prediction of the twelfth threshold;

[0739] The difference between the prediction result variance and the prediction result variance of the previous prediction is greater than the prediction of the thirteenth threshold.

[0740] Optionally, the processor 710 is configured to read the computer program in the memory and perform the following operations:

[0741] Reporting the monitoring related information of the first model to the network device through the first resource;

[0742] The first resource includes any of the following:

[0743] A resource configured by the network device for reporting monitoring-related information of the first model;

[0744] Physical uplink shared channel PUSCH resources already available to the terminal;

[0745] The terminal requests PUSCH resources via a scheduling request SR.

[0746] Optionally, the processor 710 is configured to read the computer program in the memory and perform the following operations:

[0747] When storing the measurement result of the Lth measurement, if the required storage space exceeds the remaining storage space of the terminal, the earliest stored measurement result is discarded and the measurement result of the Lth measurement is stored.

[0748] It should be noted here that the above-mentioned terminal provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned terminal side method embodiment, and can achieve the same technical effect. The parts and beneficial effects that are the same as the method embodiment in this embodiment will not be described in detail here.

[0749] See also Figure 8 , Figure 8 is a structural diagram of a network device provided in an embodiment of the present application. Figure 8 As shown, it includes a memory 820, a transceiver 800 and a processor 810:

[0750] The memory 820 is used to store computer programs; the transceiver 800 is used to send and receive data under the control of the processor 810; the processor 810 is used to read the computer program in the memory 820 and perform the following operations:

[0751] Receiving monitoring related information of the first model reported by the terminal;

[0752] Based on the monitoring-related information, model monitoring of the first model is performed.

[0753] Among them, Figure 8 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors represented by processor 810 and various circuits of memory represented by memory 820 are linked together. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 800 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium, including transmission media such as wireless channels, wired channels, and optical cables. For different user devices, the user interface 830 may also be an interface capable of externally and internally connecting required devices, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, and the like.

[0754] The processor 810 is responsible for managing the bus architecture and general processing, and the memory 820 can store data used by the processor 810 when performing operations.

[0755] Optionally, the processor 810 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD), and the processor may also adopt a multi-core architecture.

[0756] The processor calls the computer program stored in the memory to execute any of the methods provided in the embodiments of the present application according to the obtained executable instructions. The processor and the memory can also be arranged physically separately.

[0757] Optionally, the processor 810 is configured to read the computer program in the memory and perform the following operations:

[0758] Sending first information to the terminal, where the first information is used to indicate at least one of the following:

[0759] The monitoring and reporting period of the first model;

[0760] The monitoring start time of the first model;

[0761] The prediction result of the first model needs to meet a first preset condition.

[0762] Optionally, the reporting condition is determined by predefined model monitoring related parameters, and the model monitoring related parameters include at least one of the following:

[0763] The monitoring and reporting period of the first model;

[0764] The monitoring start time of the first model;

[0765] The prediction result of the first model needs to meet a first preset condition.

[0766] Optionally, the processor 810 is configured to read the computer program in the memory and perform the following operations:

[0767] Sending reporting trigger information to the terminal, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model.

[0768] Optionally, the processor 810 is configured to read the computer program in the memory and perform the following operations:

[0769] When it is monitored that the system performance indicator meets the second preset condition, reporting trigger information is sent to the terminal.

[0770] Optionally, the first preset condition includes at least one of the following:

[0771] The number of times that the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal;

[0772] The number of times that the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, and the actual optimal K beams are determined based on the measurement result of the terminal;

[0773] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

[0774] Optionally, the processor 810 is configured to read the computer program in the memory and perform the following operations:

[0775] Based on the monitoring-related information, a monitoring index of the first model is calculated.

[0776] Optionally, when the first model is deployed on the network device, the monitoring-related information includes measurement results of the terminal measuring the reference signal N times, where N is an integer greater than 1;

[0777] The processor 810 is configured to read the computer program in the memory and perform the following operations:

[0778] Filtering N predictions corresponding to the N measurements from all predictions of the first model;

[0779] Calculating a monitoring indicator of the first model according to the prediction results of the N predictions and the measurement results of the N measurements;

[0780] or,

[0781] Selecting M measurements from the N measurements, where M is an integer greater than 1;

[0782] Filtering M predictions corresponding to the M measurements from all predictions of the first model;

[0783] The monitoring index of the first model is calculated according to the prediction results of the M predictions and the measurement results of the M measurements.

[0784] Optionally, the M measurements screened out from the N measurements include at least one of the following:

[0785] The first measurement within the monitoring and reporting period;

[0786] The first measurement from the start time of the monitoring;

[0787] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0788] The difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold value;

[0789] The difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than the fifth threshold value;

[0790] A measurement in which the difference between the average value of the measurement result and the average value of the measurement result of the previous measurement is greater than a sixth threshold;

[0791] The difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than the seventh threshold value;

[0792] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0793] Optionally, when the first model is deployed on the terminal, the monitoring-related information includes at least one of the following:

[0794] a measurement result of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1;

[0795] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0796] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0797] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0798] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0799] a difference or an absolute value of the difference or an average value of the absolute value of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the actual optimal K beams;

[0800] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the layer 1 reference signal received power value of the actual optimal beam, or an absolute value of the difference, or an average value of the absolute values ​​of the difference;

[0801] Identification information of the first model.

[0802] Optionally, the processor 810 is configured to read the computer program in the memory and perform the following operations:

[0803] Receiving, through a first resource, monitoring-related information of the first model reported by the terminal;

[0804] The first resource includes any of the following:

[0805] A resource configured by the network device for reporting monitoring-related information of the first model;

[0806] Physical uplink shared channel PUSCH resources already available to the terminal;

[0807] The terminal requests PUSCH resources via a scheduling request SR.

[0808] Optionally, the processor 810 is configured to read the computer program in the memory and perform the following operations:

[0809] According to the monitoring indicator of the first model, a first process is performed, where the first process includes any one of the following:

[0810] activating the first model;

[0811] Deactivate the first model.

[0812] It should be noted here that the above-mentioned network device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned network device side method embodiment, and can achieve the same technical effect. The parts and beneficial effects that are the same as the method embodiment in this embodiment will not be described in detail here.

[0813] See also Fig. 9 , Fig. 9 is a structural diagram of a terminal provided in an embodiment of the present application, such as Fig. 9 As shown, the terminal 900 includes:

[0814] A first determination module 901 is used to determine whether a reporting condition of monitoring related information of a first model is met;

[0815] The reporting module 902 is used to report the monitoring related information of the first model to the network device when the reporting conditions are met.

[0816] Optionally, the terminal 900 further includes:

[0817] A second determining module is configured to determine a reporting condition of the monitoring-related information of the first model according to first information sent by the network device, wherein the first information is used to indicate at least one of the following:

[0818] The monitoring and reporting period of the first model;

[0819] The monitoring start time of the first model;

[0820] The prediction result of the first model needs to meet a first preset condition.

[0821] Optionally, the terminal 900 further includes:

[0822] The third determination module is configured to determine a reporting condition for the monitoring-related information of the first model according to predefined model monitoring-related parameters, wherein the model monitoring-related parameters include at least one of the following:

[0823] The monitoring and reporting period of the first model;

[0824] The monitoring start time of the first model;

[0825] The prediction result of the first model needs to meet a first preset condition.

[0826] Optionally, the reporting condition includes at least one of the following:

[0827] The reporting time of the monitoring reporting cycle of the first model arrives;

[0828] The terminal 900 receives reporting trigger information from the network device, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model;

[0829] The terminal 900 detects that the prediction result of the first model meets the first preset condition.

[0830] Optionally, the first preset condition includes at least one of the following:

[0831] The number of times that the optimal K beams predicted by the first model do not include the actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal 900;

[0832] The number of times that the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, and the actual optimal K beams are determined based on the measurement result of the terminal 900;

[0833] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal 900 is greater than or equal to the third threshold.

[0834] Optionally, the monitoring-related information reported by the terminal 900 includes at least one of the following:

[0835] a measurement result of performing N measurements on the reference signal by the terminal 900, where N is an integer greater than 1;

[0836] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0837] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0838] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0839] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0840] a difference or an absolute value of the difference or an average value of the absolute value of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the actual optimal K beams;

[0841] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the layer 1 reference signal received power value of the actual optimal beam, or an absolute value of the difference, or an average value of the absolute values ​​of the difference;

[0842] Identification information of the first model.

[0843] Optionally, the N measurements are all measurements; or, N measurements selected from all measurements; or, N measurements corresponding to N predictions selected from all predictions of the first model.

[0844] Optionally, the terminal 900 indicates the measurement order of the N measurements in all measurements through a bit map, or indicates measurement identification information of the N measurements.

[0845] Optionally, the N measurements screened from all measurements include at least one of the following:

[0846] The first measurement within the monitoring and reporting period of the first model;

[0847] The first measurement of the first model from the start of monitoring;

[0848] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0849] The difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold value;

[0850] The difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than the fifth threshold value;

[0851] A measurement in which the difference between the average value of the measurement result and the average value of the measurement result of the previous measurement is greater than a sixth threshold;

[0852] The difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than the seventh threshold value;

[0853] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0854] Optionally, the N predictions selected from all predictions of the first model include at least one of the following:

[0855] The first forecast within the monitoring and reporting period;

[0856] The first prediction from the start time of the monitoring;

[0857] A forecast whose predicted result is different from the predicted result of the previous forecast;

[0858] The prediction that the difference between the maximum value of the predicted result and the maximum value of the predicted result of the previous prediction is greater than the ninth threshold;

[0859] The difference between the minimum value of the predicted result and the minimum value of the predicted result of the previous prediction is greater than the prediction of the tenth threshold;

[0860] The difference between the average of the predicted results and the average of the predicted results of the previous prediction is greater than the eleventh threshold;

[0861] The difference between the standard deviation of the predicted result and the standard deviation of the predicted result of the previous prediction is greater than the prediction of the twelfth threshold;

[0862] The difference between the prediction result variance and the prediction result variance of the previous prediction is greater than the prediction of the thirteenth threshold.

[0863] Optionally, the reporting module 902 is specifically used for:

[0864] Reporting the monitoring related information of the first model to the network device through the first resource;

[0865] The first resource includes any of the following:

[0866] A resource configured by the network device for reporting monitoring-related information of the first model;

[0867] Physical uplink shared channel PUSCH resources already available in the terminal 900;

[0868] Terminal 900 requests PUSCH resources via a scheduling request SR.

[0869] Optionally, the terminal 900 further includes:

[0870] The processing module is used to discard the earliest stored measurement result and store the measurement result of the Lth measurement if the required storage space exceeds the remaining storage space of the terminal 900 when storing the measurement result of the Lth measurement.

[0871] It should be noted here that the above-mentioned terminal provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned terminal side method embodiment, and can achieve the same technical effect. The parts and beneficial effects that are the same as the method embodiment in this embodiment will not be described in detail here.

[0872] See also Fig.10 , Fig.10is a structural diagram of a network device provided in an embodiment of the present application, such as Fig.10 As shown, the network device 1000 includes:

[0873] The receiving module 1001 is used to receive monitoring related information of the first model reported by the terminal;

[0874] The monitoring module 1002 is used to perform model monitoring of the first model based on the monitoring related information.

[0875] Optionally, the network device 1000 further includes:

[0876] A first sending module is configured to send first information to the terminal, where the first information is used to indicate at least one of the following:

[0877] The monitoring and reporting period of the first model;

[0878] The monitoring start time of the first model;

[0879] The prediction result of the first model needs to meet a first preset condition.

[0880] Optionally, the reporting condition is determined by predefined model monitoring related parameters, and the model monitoring related parameters include at least one of the following:

[0881] The monitoring and reporting period of the first model;

[0882] The monitoring start time of the first model;

[0883] The prediction result of the first model needs to meet a first preset condition.

[0884] Optionally, the network device 1000 further includes:

[0885] The second sending module is used to send reporting trigger information to the terminal, where the reporting trigger information is used to instruct reporting the monitoring related information of the first model.

[0886] Optionally, the network device 1000 further includes:

[0887] The third sending module is used to send reporting trigger information to the terminal when it is monitored that the system performance indicator meets the second preset condition.

[0888] Optionally, the first preset condition includes at least one of the following:

[0889] The number of times that the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal;

[0890] The number of times that the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, and the actual optimal K beams are determined based on the measurement result of the terminal;

[0891] The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

[0892] Optionally, the monitoring module 1002 includes:

[0893] A calculation unit is used to calculate the monitoring index of the first model based on the monitoring related information.

[0894] Optionally, when the first model is deployed on the network device 1000, the monitoring-related information includes measurement results of the terminal measuring the reference signal N times, where N is an integer greater than 1;

[0895] The computing unit comprises:

[0896] A first screening subunit, configured to screen out N predictions corresponding to the N measurements from all predictions of the first model;

[0897] A first calculation subunit, configured to calculate a monitoring indicator of the first model according to the prediction results of the N predictions and the measurement results of the N measurements;

[0898] or,

[0899] The computing unit comprises:

[0900] A second screening subunit, configured to screen out M measurements from the N measurements, where M is an integer greater than 1;

[0901] A third screening subunit, configured to screen out M predictions corresponding to the M measurements from all predictions of the first model;

[0902] The second calculation subunit is used to calculate the monitoring index of the first model according to the prediction results of the M predictions and the measurement results of the M measurements.

[0903] Optionally, the M measurements screened out from the N measurements include at least one of the following:

[0904] The first measurement within the monitoring and reporting period;

[0905] The first measurement from the start time of the monitoring;

[0906] A measurement in which the result of a measurement differs from the result of a previous measurement;

[0907] The difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold value;

[0908] The difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than the fifth threshold value;

[0909] A measurement in which the difference between the average value of the measurement result and the average value of the measurement result of the previous measurement is greater than a sixth threshold;

[0910] The difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than the seventh threshold value;

[0911] A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

[0912] Optionally, when the first model is deployed on the terminal, the monitoring-related information includes at least one of the following:

[0913] a measurement result of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1;

[0914] prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements;

[0915] The difference between the measurement results of the N measurements and the prediction results of the N predictions;

[0916] The number of times the optimal K beams predicted by the first model do not include the actual optimal beam;

[0917] The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams;

[0918] a difference or an absolute value of the difference or an average value of the absolute value of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the actual optimal K beams;

[0919] a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the layer 1 reference signal received power value of the actual optimal beam, or an absolute value of the difference, or an average value of the absolute values ​​of the difference;

[0920] Identification information of the first model.

[0921] Optionally, the receiving module 1001 is specifically configured to:

[0922] Receiving, through a first resource, monitoring-related information of the first model reported by the terminal;

[0923] The first resource includes any of the following:

[0924] Resources configured by the network device 1000 for reporting monitoring-related information of the first model;

[0925] Physical uplink shared channel PUSCH resources already available to the terminal;

[0926] The terminal requests PUSCH resources via a scheduling request SR.

[0927] Optionally, the network device 1000 further includes:

[0928] A processing module is used to perform a first process according to the monitoring indicator of the first model, where the first process includes any one of the following:

[0929] activating the first model;

[0930] Deactivate the first model.

[0931] It should be noted here that the above-mentioned network device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned network device side method embodiment, and can achieve the same technical effect. The parts and beneficial effects that are the same as the method embodiment in this embodiment will not be described in detail here.

[0932] It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0933] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the relevant technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0934] An embodiment of the present application also provides a processor-readable storage medium, which stores a computer program, and the computer program is used to enable the processor to execute the model monitoring method on the terminal side provided by the embodiment of the present application, or the computer program is used to enable the processor to execute the model monitoring method on the network device side provided by the embodiment of the present application.

[0935] The processor-readable storage medium may be a non-transitory readable storage medium.

[0936] The processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as compact disks (CD), digital video disks (DVD), Blu-ray Discs (BD), high-definition versatile disks (HVD), etc.), and semiconductor memory (such as read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), non-volatile memory (NAND FLASH), solid-state disk (Solid State Disk or Solid State Drive, SSD)), etc.

[0937] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0938] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer executable instructions. These computer executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0939] These processor executable instructions may also be stored in a processor readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the processor readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0940] These processor-executable instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0941] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A model monitoring method, characterized in that: include: The terminal determines whether a reporting condition of monitoring-related information of the first model is met; When the reporting condition is met, the terminal reports the monitoring-related information of the first model to the network device.

2. The method according to claim 1, characterized in that The method further comprises: The terminal determines, according to first information sent by the network device, a reporting condition for monitoring-related information of the first model, where the first information is used to indicate at least one of the following: The monitoring and reporting period of the first model; The monitoring start time of the first model; The prediction result of the first model needs to meet a first preset condition.

3. The method according to claim 1, characterized in that The method further comprises: The terminal determines, according to a predefined model monitoring related parameter, a reporting condition of the monitoring related information of the first model, wherein the model monitoring related parameter includes at least one of the following: The monitoring and reporting period of the first model; The monitoring start time of the first model; The prediction result of the first model needs to meet a first preset condition.

4. The method according to any one of claims 1 to 3, characterized in that The reporting conditions include at least one of the following: The reporting time of the monitoring reporting cycle of the first model arrives; The terminal receives reporting trigger information from the network device, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model; The terminal detects that the prediction result of the first model meets a first preset condition.

5. The method according to claim 2 or 3, characterized in that: The first preset condition includes at least one of the following: The number of times that the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal; The number of times that the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, and the actual optimal K beams are determined based on the measurement result of the terminal; The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

6. The method according to any one of claims 1 to 3, characterized in that The monitoring related information reported by the terminal includes at least one of the following: a measurement result of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1; prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements; The difference between the measurement results of the N measurements and the prediction results of the N predictions; The number of times the optimal K beams predicted by the first model do not include the actual optimal beam; The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams; a difference or an absolute value of the difference or an average value of the absolute value of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the actual optimal K beams; a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the layer 1 reference signal received power value of the actual optimal beam, or an absolute value of the difference, or an average value of the absolute values ​​of the difference; Identification information of the first model.

7. The method according to claim 6, characterized in that The N measurements are all measurements of the terminal; or, N measurements selected by the terminal from all measurements; or, N measurements corresponding to N predictions selected by the terminal from all predictions of the first model.

8. The method according to claim 7, characterized in that The terminal indicates the measurement order of the N measurements in all measurements through a bit map, or indicates measurement identification information of the N measurements.

9. The method according to claim 7, characterized in that: The N measurements screened by the terminal from all measurements include at least one of the following: The first measurement within the monitoring and reporting period of the first model; The first measurement of the first model from the start of monitoring; A measurement in which the result of a measurement differs from the result of a previous measurement; The difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold value; The difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than the fifth threshold value; A measurement in which the difference between the average value of the measurement result and the average value of the measurement result of the previous measurement is greater than a sixth threshold; The difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than the seventh threshold value; A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

10. The method according to claim 7, characterized in that The N predictions selected by the terminal from all predictions of the first model include at least one of the following: The first forecast within the monitoring and reporting period; The first prediction from the start time of the monitoring; A forecast whose predicted result is different from the predicted result of the previous forecast; The prediction that the difference between the maximum value of the predicted result and the maximum value of the predicted result of the previous prediction is greater than the ninth threshold; The difference between the minimum value of the predicted result and the minimum value of the predicted result of the previous prediction is greater than the prediction of the tenth threshold; The difference between the average of the predicted results and the average of the predicted results of the previous prediction is greater than the eleventh threshold; The difference between the standard deviation of the predicted result and the standard deviation of the predicted result of the previous prediction is greater than the prediction of the twelfth threshold; The difference between the prediction result variance and the prediction result variance of the previous prediction is greater than the prediction of the thirteenth threshold.

11. The method according to any one of claims 1 to 3, characterized in that The terminal reports the monitoring-related information of the first model to the network device, including: The terminal reports the monitoring related information of the first model to the network device through the first resource; The first resource includes any of the following: A resource configured by the network device for reporting monitoring-related information of the first model; Physical uplink shared channel PUSCH resources already available to the terminal; The terminal requests PUSCH resources via a scheduling request SR.

12. The method according to any one of claims 1 to 3, characterized in that The method further comprises: When the terminal stores the measurement result of the Lth measurement, if the required storage space exceeds the remaining storage space of the terminal, the terminal discards the earliest stored measurement result and stores the measurement result of the Lth measurement.

13. A model monitoring method, characterized in that: include: The network device receives monitoring related information of the first model reported by the terminal; The network device performs model monitoring of the first model based on the monitoring-related information.

14. The method according to claim 13, characterized in that The method further comprises: The network device sends first information to the terminal, where the first information is used to indicate at least one of the following: The monitoring and reporting period of the first model; The monitoring start time of the first model; The prediction result of the first model needs to meet a first preset condition.

15. The method according to claim 13 or 14, characterized in that The method further comprises: The network device sends reporting trigger information to the terminal, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model.

16. The method according to claim 15, characterized in that The network device sends reporting trigger information to the terminal, including: When the network device monitors that the system performance indicator meets the second preset condition, the network device sends reporting trigger information to the terminal.

17. The method according to claim 14, characterized in that The first preset condition includes at least one of the following: The number of times that the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal; The number of times that the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, and the actual optimal K beams are determined based on the measurement result of the terminal; The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

18. The method according to claim 13 or 14, characterized in that The network device performs model monitoring of the first model based on the monitoring related information, including: The network device calculates the monitoring index of the first model based on the monitoring-related information.

19. The method according to claim 18, characterized in that In the case where the first model is deployed on the network device, the monitoring-related information includes measurement results of the terminal measuring the reference signal N times, where N is an integer greater than 1; The network device calculates the monitoring index of the first model based on the monitoring related information, including: The network device selects N predictions corresponding to the N measurements from all predictions of the first model; The network device calculates the monitoring index of the first model according to the prediction results of the N predictions and the measurement results of the N measurements; or, The network device calculates the monitoring index of the first model based on the monitoring related information, including: The network device selects M measurements from the N measurements, where M is an integer greater than 1; The network device selects M predictions corresponding to the M measurements from all predictions of the first model; The network device calculates the monitoring index of the first model according to the prediction results of the M predictions and the measurement results of the M measurements.

20. The method according to claim 19, characterized in that The M measurements screened out by the network device from the N measurements include at least one of the following: The first measurement within the monitoring and reporting period; The first measurement from the start time of the monitoring; A measurement in which the result of a measurement differs from the result of a previous measurement; The difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold value; The difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than the fifth threshold value; A measurement in which the difference between the average value of the measurement result and the average value of the measurement result of the previous measurement is greater than a sixth threshold; The difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than the seventh threshold value; A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

21. The method according to claim 13 or 14, characterized in that In the case where the first model is deployed on the terminal, the monitoring-related information includes at least one of the following: a measurement result of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1; prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements; The difference between the measurement results of the N measurements and the prediction results of the N predictions; The number of times the optimal K beams predicted by the first model do not include the actual optimal beam; The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams; a difference or an absolute value of the difference or an average value of the absolute value of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the actual optimal K beams; a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the layer 1 reference signal received power value of the actual optimal beam, or an absolute value of the difference, or an average value of the absolute values ​​of the difference; Identification information of the first model.

22. The method according to claim 13 or 14, characterized in that The network device receives the monitoring-related information of the first model reported by the terminal, including: The network device receives, through a first resource, monitoring-related information of the first model reported by the terminal; The first resource includes any of the following: A resource configured by the network device for reporting monitoring-related information of the first model; Physical uplink shared channel PUSCH resources already available to the terminal; The terminal requests PUSCH resources via a scheduling request SR.

23. The method according to claim 18, characterized in that The method further comprises: The network device performs a first process according to the monitoring indicator of the first model, where the first process includes any one of the following: activating the first model; Deactivate the first model.

24. A terminal, characterized in that: include: A memory, a transceiver, and a processor, wherein: A memory for storing a computer program; a transceiver for transmitting and receiving data under the control of the processor; and a processor for reading the computer program in the memory and performing the following operations: Determining whether a reporting condition of monitoring-related information of the first model is met; When the reporting condition is met, the monitoring related information of the first model is reported to the network device.

25. The terminal according to claim 24, characterized in that A processor is configured to read the computer program in the memory and perform the following operations: Determine, according to the first information sent by the network device, a reporting condition of the monitoring-related information of the first model, wherein the first information is used to indicate at least one of the following: The monitoring and reporting period of the first model; The monitoring start time of the first model; The prediction result of the first model needs to meet a first preset condition.

26. The terminal according to claim 24, characterized in that A processor is configured to read the computer program in the memory and perform the following operations: Determine a reporting condition for monitoring-related information of the first model according to predefined model monitoring-related parameters, wherein the model monitoring-related parameters include at least one of the following: The monitoring and reporting period of the first model; The monitoring start time of the first model; The prediction result of the first model needs to meet a first preset condition.

27. The terminal according to any one of claims 24 to 26, characterized in that: The reporting conditions include at least one of the following: The reporting time of the monitoring reporting cycle of the first model arrives; The terminal receives reporting trigger information from the network device, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model; The terminal detects that the prediction result of the first model meets a first preset condition.

28. The terminal according to claim 25 or 26, characterized in that: The first preset condition includes at least one of the following: The number of times that the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal; The number of times that the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, and the actual optimal K beams are determined based on the measurement result of the terminal; The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

29. The terminal according to any one of claims 24 to 26, characterized in that: The monitoring related information reported by the terminal includes at least one of the following: a measurement result of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1; prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements; The difference between the measurement results of the N measurements and the prediction results of the N predictions; The number of times the optimal K beams predicted by the first model do not include the actual optimal beam; The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams; a difference or an absolute value of the difference or an average value of the absolute value of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the actual optimal K beams; a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the layer 1 reference signal received power value of the actual optimal beam, or an absolute value of the difference, or an average value of the absolute values ​​of the difference; Identification information of the first model.

30. The terminal according to claim 29, characterized in that The N measurements are all measurements; or, N measurements selected from all measurements; or, N measurements corresponding to N predictions selected from all predictions of the first model.

31. The terminal according to claim 30, characterized in that A processor is configured to read the computer program in the memory and perform the following operations: The measurement order of the N measurements in all measurements is indicated by a bit map, or the measurement identification information of the N measurements is indicated.

32. The terminal according to claim 30, characterized in that The N measurements selected from all measurements include at least one of the following: The first measurement within the monitoring and reporting period of the first model; The first measurement of the first model from the start of monitoring; A measurement in which the result of a measurement differs from the result of a previous measurement; The difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold value; The difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than the fifth threshold value; A measurement in which the difference between the average value of the measurement result and the average value of the measurement result of the previous measurement is greater than a sixth threshold; The difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than the seventh threshold value; A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

33. The terminal according to claim 30, characterized in that The N predictions selected from all predictions of the first model include at least one of the following: The first forecast within the monitoring and reporting period; The first prediction from the start time of the monitoring; A forecast whose predicted result is different from the predicted result of the previous forecast; The prediction that the difference between the maximum value of the predicted result and the maximum value of the predicted result of the previous prediction is greater than the ninth threshold; The difference between the minimum value of the predicted result and the minimum value of the predicted result of the previous prediction is greater than the prediction of the tenth threshold; The difference between the average of the predicted results and the average of the predicted results of the previous prediction is greater than the eleventh threshold; The difference between the standard deviation of the predicted result and the standard deviation of the predicted result of the previous prediction is greater than the prediction of the twelfth threshold; The difference between the prediction result variance and the prediction result variance of the previous prediction is greater than the prediction of the thirteenth threshold.

34. The terminal according to any one of claims 24 to 26, characterized in that: A processor is configured to read the computer program in the memory and perform the following operations: Reporting the monitoring related information of the first model to the network device through the first resource; The first resource includes any of the following: A resource configured by the network device for reporting monitoring-related information of the first model; Physical uplink shared channel PUSCH resources already available to the terminal; The terminal requests PUSCH resources via a scheduling request SR.

35. The terminal according to any one of claims 24 to 26, characterized in that: A processor is configured to read the computer program in the memory and perform the following operations: When storing the measurement result of the Lth measurement, if the required storage space exceeds the remaining storage space of the terminal, the earliest stored measurement result is discarded and the measurement result of the Lth measurement is stored.

36. A network device, characterized in that: include: A memory, a transceiver, and a processor, wherein: A memory for storing a computer program; a transceiver for transmitting and receiving data under the control of the processor; and a processor for reading the computer program in the memory and performing the following operations: Receiving monitoring related information of the first model reported by the terminal; Based on the monitoring-related information, model monitoring of the first model is performed.

37. The network device according to claim 36, characterized in that: A processor is configured to read the computer program in the memory and perform the following operations: Sending first information to the terminal, where the first information is used to indicate at least one of the following: The monitoring and reporting period of the first model; The monitoring start time of the first model; The prediction result of the first model needs to meet a first preset condition.

38. The network device according to claim 36 or 37, characterized in that: A processor is configured to read the computer program in the memory and perform the following operations: The network device sends reporting trigger information to the terminal, where the reporting trigger information is used to instruct reporting of monitoring related information of the first model.

39. The network device according to claim 38, characterized in that: A processor is configured to read the computer program in the memory and perform the following operations: When it is monitored that the system performance indicator meets the second preset condition, reporting trigger information is sent to the terminal.

40. The network device according to claim 37, characterized in that: The first preset condition includes at least one of the following: The number of times that the optimal K beams predicted by the first model do not include an actual optimal beam is greater than or equal to a first threshold, where K is an integer greater than or equal to 1, and the actual optimal beam is determined based on a measurement result of the terminal; The number of times that the optimal beam predicted by the first model does not belong to the actual optimal K beams is greater than or equal to a second threshold, and the actual optimal K beams are determined based on the measurement result of the terminal; The average of the absolute values ​​of the differences between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the optimal K beams measured by the terminal is greater than or equal to a third threshold.

41. The network device according to claim 36 or 37, characterized in that: A processor is configured to read the computer program in the memory and perform the following operations: Based on the monitoring-related information, a monitoring index of the first model is calculated.

42. The network device according to claim 41, characterized in that In the case where the first model is deployed on the network device, the monitoring-related information includes measurement results of the terminal measuring the reference signal N times, where N is an integer greater than 1; A processor is configured to read the computer program in the memory and perform the following operations: Filtering N predictions corresponding to the N measurements from all predictions of the first model; Calculating a monitoring indicator of the first model according to the prediction results of the N predictions and the measurement results of the N measurements; or, Selecting M measurements from the N measurements, where M is an integer greater than 1; Filtering M predictions corresponding to the M measurements from all predictions of the first model; The monitoring index of the first model is calculated according to the prediction results of the M predictions and the measurement results of the M measurements.

43. The network device according to claim 42, characterized in that: The M measurements selected from the N measurements include at least one of the following: The first measurement within the monitoring and reporting period; The first measurement from the start time of the monitoring; A measurement in which the result of a measurement differs from the result of a previous measurement; The difference between the maximum value of the measurement result and the maximum value of the measurement result of the previous measurement is greater than a fourth threshold value; The difference between the minimum value of the measurement result and the minimum value of the measurement result of the previous measurement is greater than the fifth threshold value; A measurement in which the difference between the average value of the measurement result and the average value of the measurement result of the previous measurement is greater than a sixth threshold; The difference between the standard deviation of the measurement result and the standard deviation of the measurement result of the previous measurement is greater than the seventh threshold value; A measurement in which a difference between a measurement result variance and a measurement result variance of a previous measurement is greater than an eighth threshold.

44. The network device according to claim 36 or 37, characterized in that: In the case where the first model is deployed on the terminal, the monitoring-related information includes at least one of the following: a measurement result of N measurements performed by the terminal on the reference signal, where N is an integer greater than 1; prediction results of N predictions of the first model, the N predictions being predictions corresponding to the N measurements; The difference between the measurement results of the N measurements and the prediction results of the N predictions; The number of times the optimal K beams predicted by the first model do not include the actual optimal beam; The number of times the optimal beam predicted by the first model does not belong to the actual optimal K beams; a difference or an absolute value of the difference or an average value of the absolute value of the difference between the layer 1 reference signal received power values ​​of the optimal K beams predicted by the first model and the layer 1 reference signal received power values ​​of the actual optimal K beams; a difference between the layer 1 reference signal received power value of the optimal beam predicted by the first model and the layer 1 reference signal received power value of the actual optimal beam, or an absolute value of the difference, or an average value of the absolute values ​​of the difference; Identification information of the first model.

45. The network device according to claim 36 or 37, characterized in that: A processor is configured to read the computer program in the memory and perform the following operations: Receiving, through a first resource, monitoring-related information of the first model reported by the terminal; The first resource includes any of the following: A resource configured by the network device for reporting monitoring-related information of the first model; Physical uplink shared channel PUSCH resources already available to the terminal; The terminal requests PUSCH resources via a scheduling request SR.

46. ​​The network device according to claim 41, characterized in that A processor is configured to read the computer program in the memory and perform the following operations: According to the monitoring indicator of the first model, a first process is performed, where the first process includes any one of the following: activating the first model; Deactivate the first model.

47. A terminal, characterized in that: include: A determination module, used to determine whether a reporting condition of monitoring related information of the first model is met; A reporting module is used to report the monitoring related information of the first model to the network device when the reporting conditions are met.

48. A network device, characterized in that: include: A receiving module, used for receiving monitoring related information of the first model reported by the terminal; A monitoring module is used to perform model monitoring of the first model based on the monitoring related information.

49. A processor-readable storage medium, characterized in that: The processor-readable storage medium stores a computer program, wherein the computer program is used to enable the processor to execute the model monitoring method according to any one of claims 1 to 12, or the computer program is used to enable the processor to execute the model monitoring method according to any one of claims 13 to 24.

Images