Model performance monitoring method and device and storage medium

Abstract

The invention relates to a model performance monitoring method and device and a storage medium. The method comprises: receiving first information sent by a network device, the first information comprising configuration of a reference signal resource, the reference signal resource being used by a terminal device for beam measurement; performing beam measurement according to the first information to obtain a beam measurement result; and sending performance monitoring data to the network device according to the beam measurement result, the performance monitoring data being used for determining the performance of the first AI model. Thus, the terminal device can perform beam measurement according to the first information sent by the network device and send the performance monitoring data to the network device according to the beam measurement result, and the network device can determine the performance of the first AI model according to the performance monitoring data, thereby realizing performance monitoring of the AI model.

Description

Model performance monitoring method, device and storage medium Technical Field

[0001] The present disclosure relates to the field of communication technology, and in particular to a model performance monitoring method, device, and storage medium. Background Art

[0002] In wireless communication systems, multiple transmission receive points (MTRPs) require beams from multiple transmission receive points (TRPs) to simultaneously serve terminal devices. During beam measurement, an artificial intelligence (AI) model can be used to predict the beam corresponding to each TRP. The accuracy of beam prediction depends on the performance of the AI ​​model. Therefore, how to monitor the performance of the AI ​​model has become a pressing issue.

[0003] Summary of the Invention

[0004] The embodiments of the present disclosure provide a model performance monitoring method, device, and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, a model performance monitoring method is proposed, the method comprising:

[0006] receiving first information sent by a network device, where the first information includes configuration of a reference signal resource, where the reference signal resource is used by the terminal device to perform beam measurement;

[0007] Performing beam measurement according to the first information to obtain a beam measurement result;

[0008] Based on the beam measurement result, performance monitoring data is sent to the network device, where the performance monitoring data is used to determine the performance of the first AI model.

[0009] According to a second aspect of an embodiment of the present disclosure, a model performance monitoring method is proposed, the method comprising:

[0010] Sending first information to a terminal device, where the first information includes a configuration of a reference signal resource, where the reference signal resource is used by the terminal device to perform beam measurement;

[0011] Receive performance monitoring data sent by the terminal device according to the beam measurement result, where the beam measurement result is obtained by the terminal device performing beam measurement based on the first information, and the performance monitoring data is used to determine the performance of the first AI model.

[0012] According to a third aspect of an embodiment of the present disclosure, a terminal device is provided, including:

[0013] a transceiver module configured to receive first information sent by a network device, where the first information includes a configuration of a reference signal resource, where the reference signal resource is used by the terminal device to perform beam measurement;

[0014] a processing module, configured to perform beam measurement according to the first information to obtain a beam measurement result;

[0015] The transceiver module is further configured to send performance monitoring data to the network device based on the beam measurement result, and the performance monitoring data is used to determine the performance of the first AI model.

[0016] According to a fourth aspect of an embodiment of the present disclosure, a network device is provided, including:

[0017] a transceiver module configured to send first information to a terminal device, where the first information includes a configuration of a reference signal resource, where the reference signal resource is used by the terminal device to perform beam measurement;

[0018] The transceiver module is further configured to receive performance monitoring data sent by the terminal device according to the beam measurement result, where the beam measurement result is obtained by the terminal device performing beam measurement according to the first information, and the performance monitoring data is used to determine the performance of the first AI model.

[0019] According to a fifth aspect of an embodiment of the present disclosure, a communication device is proposed, comprising: one or more processors; wherein the communication device can be used to execute an optional implementation of the first aspect or the second aspect.

[0020] According to a sixth aspect of an embodiment of the present disclosure, a storage medium is proposed, which stores instructions. When the instructions are executed on a communication device, the communication device executes the method described in the optional implementation of the first aspect or the second aspect.

[0021] The technical solution provided by the embodiment of the present disclosure may include the following beneficial effects: receiving first information sent by a network device, the first information including the configuration of reference signal resources, the reference signal resources being used by the terminal device to perform beam measurement; performing beam measurement based on the first information to obtain a beam measurement result; and sending performance monitoring data to the network device based on the beam measurement result, the performance monitoring data being used to determine the performance of the first AI model. In this way, the terminal device can perform beam measurement based on the first information sent by the network device, and send performance monitoring data to the network device based on the beam measurement result. The network device can determine the performance of the first AI model based on the performance monitoring data, thereby realizing performance monitoring of the AI ​​model.

[0022] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following drawings required for describing the embodiments are introduced. The following drawings are merely some embodiments of the present disclosure and do not impose specific limitations on the protection scope of the present disclosure.

[0024] FIG1 is a schematic diagram showing the architecture of a communication system according to an embodiment of the present disclosure.

[0025] FIG2A is an interactive schematic diagram illustrating a model performance monitoring method according to an embodiment of the present disclosure.

[0026] FIG2B is an interactive schematic diagram illustrating a model performance monitoring method according to an embodiment of the present disclosure.

[0027] FIG2C is an interactive schematic diagram illustrating a model performance monitoring method according to an embodiment of the present disclosure.

[0028] FIG3A is a flow chart illustrating a method for monitoring model performance according to an embodiment of the present disclosure.

[0029] FIG3B is a flow chart illustrating a method for monitoring model performance according to an embodiment of the present disclosure.

[0030] FIG3C is a flow chart illustrating a method for monitoring model performance according to an embodiment of the present disclosure.

[0031] FIG4A is a flow chart illustrating a method for monitoring model performance according to an embodiment of the present disclosure.

[0032] FIG4B is a flow chart illustrating a method for monitoring model performance according to an embodiment of the present disclosure.

[0033] FIG4C is a flow chart illustrating a method for monitoring model performance according to an embodiment of the present disclosure.

[0034] FIG5 is an interactive schematic diagram illustrating a model performance monitoring method according to an embodiment of the present disclosure.

[0035] FIG6A is a schematic structural diagram of a terminal device proposed in an embodiment of the present disclosure.

[0036] FIG6B is a schematic structural diagram of a network device proposed in an embodiment of the present disclosure.

[0037] FIG7A is a schematic structural diagram of a communication device proposed in an embodiment of the present disclosure.

[0038] FIG7B is a schematic diagram of the structure of the chip proposed in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0039] The embodiments of the present disclosure provide a model performance monitoring method, device, and storage medium.

[0040] In a first aspect, an embodiment of the present disclosure provides a model performance monitoring method, the method comprising:

[0041] receiving first information sent by a network device, where the first information includes configuration of a reference signal resource, where the reference signal resource is used by the terminal device to perform beam measurement;

[0042] Performing beam measurement according to the first information to obtain a beam measurement result;

[0043] Based on the beam measurement result, performance monitoring data is sent to the network device, where the performance monitoring data is used to determine the performance of the first AI model.

[0044] In the above embodiment, the terminal device can perform beam measurement based on the first information sent by the network device, and send performance monitoring data to the network device based on the beam measurement results. The network device can determine the performance of the first AI model based on the performance monitoring data, thereby realizing performance monitoring of the AI ​​model.

[0045] In conjunction with some embodiments of the first aspect, in some embodiments,

[0046] The first AI model is a model for performing beam prediction, and the first information includes at least one of the following:

[0047] a first reference signal resource set, where reference signal resources in the first reference signal resource set correspond to beams to be measured;

[0048] a second reference signal resource set, where the reference signal resources in the second reference signal resource set correspond to the beam to be predicted;

[0049] a third reference signal resource set, wherein the third reference signal resource set includes resources used for interference measurement;

[0050] The relationship between the first reference signal resource set and the second reference signal resource set.

[0051] In the above embodiment, configuration of reference signal resources for beam measurement is provided so that the terminal device can perform beam measurement.

[0052] In conjunction with some embodiments of the first aspect, in some embodiments, the first AI model is a model for performing spatial beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following:

[0053] The first reference signal resource set is a subset of the second reference signal resource set;

[0054] The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same.

[0055] In the above embodiment, a relationship between the measurement beam and the prediction beam corresponding to the spatial beam prediction model is provided so that the terminal device can accurately perform beam measurement.

[0056] In conjunction with some embodiments of the first aspect, in some embodiments, the first AI model is a model for performing time-domain beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following:

[0057] The first reference signal resource set is a subset of the second reference signal resource set;

[0058] The first reference signal resource set is the same as the second reference signal resource set;

[0059] The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same.

[0060] In the above embodiment, a relationship between the measurement beam and the predicted beam corresponding to the time domain beam prediction model is provided so that the terminal device can accurately perform beam measurement.

[0061] In combination with some embodiments of the first aspect, in some embodiments, the first reference signal resource set includes multiple fourth reference signal resource sets, and different fourth reference signal resource sets correspond to different transceiver points TRP; the second reference signal resource set includes multiple fifth reference signal resource sets, and different fifth reference signal resource sets correspond to different TRPs.

[0062] In the above embodiment, different TRPs may set different reference signal resource sets so that the terminal device can measure and report the measurement results in a group-based manner.

[0063] In conjunction with some embodiments of the first aspect, in some embodiments, the first AI model is a model for performing spatial beam prediction, and the relationship between the fourth reference signal resource set and the fifth reference signal resource set includes at least one of the following:

[0064] The fourth reference signal resource set is a subset of the fifth reference signal resource set;

[0065] The beam corresponding to the fourth reference signal resource set is a wide beam, the beam corresponding to the fifth reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the fourth reference signal resource set and the fifth reference signal resource set are the same.

[0066] In the above embodiment, the terminal device can perform beam measurement and reporting based on the reference signal resources of each TRP, thereby improving the communication performance of the terminal device based on multi-beam transmission.

[0067] In conjunction with some embodiments of the first aspect, in some embodiments, the first AI model is a model for performing time-domain beam prediction, and the relationship between the fourth reference signal resource set and the fifth reference signal resource set includes at least one of the following:

[0068] The fourth reference signal resource set is a subset of the fifth reference signal resource set;

[0069] The fourth reference signal resource set is the same as the fifth reference signal resource set;

[0070] The beam corresponding to the fourth reference signal resource set is a wide beam, the beam corresponding to the fifth reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the fourth reference signal resource set and the fifth reference signal resource set are the same.

[0071] In the above embodiment, the terminal device can perform beam measurement and reporting based on the reference signal resources of each TRP, thereby improving the communication performance of the terminal device based on multi-beam transmission.

[0072] In conjunction with some embodiments of the first aspect, in some embodiments, the performance monitoring data includes at least one of the following:

[0073] the performance value of the first AI model;

[0074] first data, the first data including at least one of the following: input data of the first AI model, output data of the first AI model, and measurement data corresponding to the output data, where the output data is data output by the first AI model based on the input data;

[0075] a specified event, where the specified event is triggered based on a comparison result between a performance value of the first AI model and a first threshold value or a first offset value;

[0076] First operation information, where the first operation information is used to indicate a management operation to be performed on the first AI model, where the management operation includes any one of the following: activating the first AI model, deactivating the first AI model, switching the first AI model, and not using the AI ​​model.

[0077] In the above embodiment, the terminal device can send one or more performance monitoring data to the network device so that the network device can perform performance monitoring on the first A model or perform inference through the first AI model.

[0078] In conjunction with some embodiments of the first aspect, in some embodiments, the performance value includes at least one of the following:

[0079] Beam prediction accuracy;

[0080] beam pair prediction accuracy, where the beam pairs include beam pairs that the terminal device can simultaneously receive and / or simultaneously transmit;

[0081] a beam quality difference, where the beam quality difference is a difference between a measured beam quality of a first beam and a measured beam quality of a second beam, where the first beam is the beam with the strongest predicted beam quality and the second beam is the beam with the strongest measured beam quality;

[0082] A predicted beam quality difference is a difference between a predicted beam quality of the first beam and a measured beam quality of the first beam.

[0083] In the above embodiment, the network device can monitor the performance of the first AI model through one or more of the performance values.

[0084] In combination with some embodiments of the first aspect, in some embodiments, the beam prediction accuracy is the accuracy of including an actual optimal beam in the predicted at least one beam.

[0085] In the above embodiment, the network device may determine the performance of the first AI model based on the accuracy of the predicted actual optimal beam.

[0086] In combination with some embodiments of the first aspect, in some embodiments, the beam pair prediction accuracy is the accuracy of at least one predicted beam pair including an actual optimal beam pair.

[0087] In the above embodiment, the network device may determine the performance of the first AI model based on the accuracy of the predicted actual optimal beam pair.

[0088] In conjunction with some embodiments of the first aspect, in some embodiments, the designated event includes at least one of the following: a first event, a second event, a third event, a fourth event, a fifth event, a sixth event, a seventh event, and an eighth event;

[0089] The sending the performance monitoring data to the network device according to the beam measurement result includes at least one of the following:

[0090] Determining, according to the beam measurement result, that the beam prediction accuracy is less than a first accuracy threshold, and sending the first event to the network device;

[0091] Determining, according to the beam measurement result, that the beam prediction accuracy is greater than a second accuracy threshold, and sending the second event to the network device;

[0092] Determining, according to the beam measurement result, that the beam pair prediction accuracy is less than a third accuracy threshold, and sending the third event to the network device;

[0093] Determining, according to the beam measurement result, that the beam pair prediction accuracy is greater than a fourth accuracy threshold, and sending the fourth event to the network device;

[0094] Determining, according to the beam measurement result, that the beam quality difference is less than a first difference threshold, and sending the fifth event to the network device;

[0095] determining, according to the beam measurement result, that the beam quality difference is greater than a second difference threshold, and sending the sixth event to the network device;

[0096] determining, according to the beam measurement result, that the predicted beam quality difference is less than a third difference threshold, and sending the seventh event to the network device;

[0097] Determine, according to the beam measurement result, that the predicted beam quality difference is greater than a fourth difference threshold, and send the eighth event to the network device.

[0098] In the above embodiment, the terminal device can judge the prediction result based on the beam measurement result, and trigger different events based on the judgment result, so that the network device can determine the performance of the first AI model.

[0099] In conjunction with some embodiments of the first aspect, in some embodiments, sending the performance monitoring data to the network device according to the beam measurement result includes:

[0100] determining the output data and the measurement data corresponding to the output data according to the beam measurement result;

[0101] determining the first operation information according to the output data and measurement data corresponding to the output data;

[0102] The first operation information is sent to the network device.

[0103] In the above embodiment, the terminal device can determine the decision on the first AI model based on the output data and the measurement data corresponding to the output data, and inform the network device.

[0104] In conjunction with some embodiments of the first aspect, in some embodiments, determining the first operation information according to the output data and the measurement data corresponding to the output data includes:

[0105] The first AI model is in an inactive state, and it is determined based on the output data and the measurement data corresponding to the output data that the performance of the first AI model meets the performance requirement, and the first operation information is determined to activate the first AI model; or

[0106] The first AI model is in an activated state. It is determined based on the output data and the measurement data corresponding to the output data that the performance of the first AI model does not meet performance requirements, and the first operation information is determined to be deactivating the first AI model.

[0107] In the above embodiment, the terminal device can determine the management operation of the AI ​​model according to the current state of the first AI model.

[0108] In conjunction with some embodiments of the first aspect, in some embodiments, the first AI model is a model for performing spatial beam prediction, and the input data includes at least one of the following:

[0109] beam qualities of the N beams corresponding to the first reference signal resource set, the beam qualities comprising layer 1 reference signal received power L1-RSRP or layer 1 signal to interference plus noise ratio L1-SINR, where N is a positive integer;

[0110] identifiers of reference signal resources corresponding to the N beams corresponding to the first reference signal resource set;

[0111] The second information is used to indicate that the beams contained in at least one group in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

[0112] In the above embodiment, the input data may include multiple different types, so that the performance of the first AI model can be monitored through different data, or more predictions can be made through the first AI model.

[0113] In conjunction with some embodiments of the first aspect, in some embodiments, the first AI model is a model for performing spatial beam prediction, and the output data includes at least one of the following:

[0114] At least one group;

[0115] identifiers of two reference signal resources corresponding to each group, wherein the reference signal resources are reference signal resources in the second reference signal resource set;

[0116] a beam quality corresponding to an identifier of each reference signal resource;

[0117] at least one third beam;

[0118] an identifier of a reference signal resource corresponding to each of the third beams, wherein the reference signal resource is a reference signal resource in the second reference signal resource set;

[0119] a beam quality corresponding to each of the third beams;

[0120] The third information is used to indicate that the beams contained in at least one group in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

[0121] In the above embodiment, the output data may include multiple different types, so as to monitor the performance of the first AI model or train the first AI model through different data.

[0122] In conjunction with some embodiments of the first aspect, in some embodiments, the first AI model is a model for performing time-domain beamforming prediction, and the input data includes at least one of the following:

[0123] at least one historical time;

[0124] beam qualities of N beams corresponding to the first reference signal resource set corresponding to each historical time, where the beam qualities include L1-RSRP or L1-SINR, where N is a positive integer;

[0125] identifiers of reference signal resources corresponding to the N beams corresponding to the first reference signal resource set corresponding to each historical time;

[0126] The fourth information is used to indicate that the beams contained in at least one group in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

[0127] In the above embodiment, the input data may include multiple different types, so that the performance of the first AI model can be monitored through different data, or more predictions can be made through the first AI model.

[0128] In conjunction with some embodiments of the first aspect, in some embodiments, the first AI model is a model for performing time-domain beam prediction, and the output data includes at least one of the following:

[0129] At least one future time, where the future time is a time corresponding to a beam predicted by the first AI model;

[0130] at least one group corresponding to each of the future times;

[0131] identifiers of two reference signal resources corresponding to each group corresponding to each of the future times, wherein the reference signal resources are reference signal resources in the second reference signal resource set;

[0132] a beam quality corresponding to an identifier of each reference signal resource corresponding to each future time;

[0133] at least one fourth beam corresponding to each of the future times;

[0134] an identifier of a reference signal resource corresponding to each of the fourth beams corresponding to each of the future time periods, wherein the reference signal resource is a reference signal resource in the second reference signal resource set;

[0135] The beam quality corresponding to each fourth beam at each future time;

[0136] The fifth information is used to indicate that the beams contained in at least one group corresponding to at least one future time in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

[0137] In the above embodiment, the output data may include multiple different types, so as to monitor the performance of the first AI model or train the first AI model through different data.

[0138] In a second aspect, an embodiment of the present disclosure provides a model performance monitoring method, the method comprising:

[0139] Sending first information to a terminal device, where the first information includes a configuration of a reference signal resource, where the reference signal resource is used by the terminal device to perform beam measurement;

[0140] Receive performance monitoring data sent by the terminal device according to the beam measurement result, where the beam measurement result is obtained by the terminal device performing beam measurement based on the first information, and the performance monitoring data is used to determine the performance of the first AI model.

[0141] In conjunction with some embodiments of the first aspect, in some embodiments, the first AI model is a model for performing beam prediction, and the first information includes at least one of the following:

[0142] a first reference signal resource set, where reference signal resources in the first reference signal resource set correspond to beams to be measured;

[0143] a second reference signal resource set, where the reference signal resources in the second reference signal resource set correspond to the beam to be predicted;

[0144] a third reference signal resource set, wherein the third reference signal resource set includes resources used for interference measurement;

[0145] The relationship between the first reference signal resource set and the second reference signal resource set.

[0146] In conjunction with some embodiments of the first aspect, in some embodiments, the first AI model is a model for performing spatial beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following:

[0147] The first reference signal resource set is a subset of the second reference signal resource set;

[0148] The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same.

[0149] In conjunction with some embodiments of the second aspect, in some embodiments, the first AI model is a model for performing time-domain beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following:

[0150] The first reference signal resource set is a subset of the second reference signal resource set;

[0151] The first reference signal resource set is the same as the second reference signal resource set;

[0152] The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same.

[0153] In combination with some embodiments of the second aspect, in some embodiments, the first reference signal resource set includes multiple fourth reference signal resource sets, and different fourth reference signal resource sets correspond to different transceiver points TRP; the second reference signal resource set includes multiple fifth reference signal resource sets, and different fifth reference signal resource sets correspond to different TRPs.

[0154] In conjunction with some embodiments of the second aspect, in some embodiments, the first AI model is a model for performing spatial beam prediction, and the relationship between the fourth reference signal resource set and the fifth reference signal resource set includes at least one of the following:

[0155] The fourth reference signal resource set is a subset of the fifth reference signal resource set;

[0156] The beam corresponding to the fourth reference signal resource set is a wide beam, the beam corresponding to the fifth reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the fourth reference signal resource set and the fifth reference signal resource set are the same.

[0157] In conjunction with some embodiments of the second aspect, in some embodiments, the first AI model is a model for performing time-domain beam prediction, and the relationship between the fourth reference signal resource set and the fifth reference signal resource set includes at least one of the following:

[0158] The fourth reference signal resource set is a subset of the fifth reference signal resource set;

[0159] The fourth reference signal resource set is the same as the fifth reference signal resource set;

[0160] The beam corresponding to the fourth reference signal resource set is a wide beam, the beam corresponding to the fifth reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the fourth reference signal resource set and the fifth reference signal resource set are the same.

[0161] In conjunction with some embodiments of the second aspect, in some embodiments, the performance monitoring data includes at least one of the following:

[0162] The performance value of the first AI model;

[0163] first data, the first data including at least one of the following: input data of the first AI model, output data of the first AI model, and measurement data corresponding to the output data, where the output data is data output by the first AI model based on the input data;

[0164] a specified event, where the specified event is triggered based on a comparison result between a performance value of the first AI model and a first threshold value or a first offset value;

[0165] First operation information, where the first operation information is used to instruct a management operation to be performed on the first AI model, where the management operation includes activating the first AI model, deactivating the first AI model, switching the first AI model, and not using the AI ​​model.

[0166] In conjunction with some embodiments of the second aspect, in some embodiments, the performance value includes at least one of the following:

[0167] Beam prediction accuracy;

[0168] beam pair prediction accuracy, where the beam pairs include beam pairs that the terminal device can simultaneously receive and / or simultaneously transmit;

[0169] a beam quality difference, where the beam quality difference is a difference between a measured beam quality of a first beam and a measured beam quality of a second beam, where the first beam is the beam with the strongest predicted beam quality and the second beam is the beam with the strongest measured beam quality;

[0170] A predicted beam quality difference is a difference between a predicted beam quality of the first beam and a measured beam quality of the first beam.

[0171] In combination with some embodiments of the second aspect, in some embodiments, the beam prediction accuracy is the accuracy of including the actual optimal beam in the predicted at least one beam.

[0172] In combination with some embodiments of the second aspect, in some embodiments, the beam pair prediction accuracy includes the accuracy of including the actual optimal beam pair in the predicted at least one beam pair.

[0173] In conjunction with some embodiments of the second aspect, in some embodiments, the designated event includes at least one of the following: a first event, a second event, a third event, a fourth event, a fifth event, a sixth event, a seventh event, and an eighth event;

[0174] The receiving performance monitoring data sent by the terminal device according to the beam measurement result includes at least one of the following:

[0175] Receiving the first event sent by the terminal device, where the first event is triggered when the terminal device determines, based on the beam measurement result, that the beam prediction accuracy is less than a first accuracy threshold;

[0176] receiving the second event sent by the terminal device, where the second event is triggered when the terminal device determines, based on the beam measurement result, that the beam prediction accuracy is greater than a second accuracy threshold;

[0177] receiving the third event sent by the terminal device, where the third event is triggered when the terminal device determines, based on the beam measurement result, that the beam pair prediction accuracy is less than a third accuracy threshold;

[0178] receiving the fourth event sent by the terminal device, where the fourth event is triggered when the terminal device determines, based on the beam measurement result, that the beam pair prediction accuracy is greater than a fourth accuracy threshold;

[0179] receiving the fifth event sent by the terminal device, where the fifth event is triggered when the terminal device determines, based on the beam measurement result, that the beam quality difference is less than a first difference threshold;

[0180] receiving the sixth event sent by the terminal device, where the sixth event is triggered when the terminal device determines, based on the beam measurement result, that the beam quality difference is greater than a second difference threshold;

[0181] receiving the seventh event sent by the terminal device, where the seventh event is triggered when the terminal device determines, based on the beam measurement result, that the predicted beam quality difference is less than a third difference threshold;

[0182] Receive the eighth event sent by the terminal device, where the eighth event is triggered when the terminal device determines, based on the beam measurement result, that the predicted beam quality difference is greater than a fourth difference threshold.

[0183] In combination with some embodiments of the second aspect, in some embodiments, the first operation information is determined by the terminal device according to the output data and measurement data corresponding to the output data.

[0184] In conjunction with some embodiments of the second aspect, in some embodiments, the first AI model is a model for performing spatial beam prediction, and the input data includes at least one of the following:

[0185] beam qualities of the N beams corresponding to the first reference signal resource set, the beam qualities comprising layer 1 reference signal received power L1-RSRP or layer 1 signal to interference plus noise ratio L1-SINR, where N is a positive integer;

[0186] identifiers of reference signal resources corresponding to the N beams corresponding to the first reference signal resource set;

[0187] The second information is used to indicate that the beams contained in at least one group in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

[0188] In conjunction with some embodiments of the second aspect, in some embodiments, the first AI model is a model for performing spatial beam prediction, and the output data includes at least one of the following:

[0189] At least one group;

[0190] identifiers of two reference signal resources corresponding to each group, wherein the reference signal resources are reference signal resources in the second reference signal resource set;

[0191] a beam quality corresponding to an identifier of each reference signal resource;

[0192] at least one third beam;

[0193] an identifier of a reference signal resource corresponding to each of the third beams, wherein the reference signal resource is a reference signal resource in the second reference signal resource set;

[0194] a beam quality corresponding to each of the third beams;

[0195] The third information is used to indicate that the beams contained in at least one group in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

[0196] In conjunction with some embodiments of the second aspect, in some embodiments, the first AI model is a model for performing time-domain beamforming prediction, and the input data includes at least one of the following:

[0197] at least one historical time;

[0198] beam qualities of N beams corresponding to the first reference signal resource set corresponding to each historical time, where the beam qualities include L1-RSRP or L1-SINR, where N is a positive integer;

[0199] identifiers of reference signal resources corresponding to the N beams corresponding to the first reference signal resource set corresponding to each historical time;

[0200] The fourth information is used to indicate that the beams contained in at least one group in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

[0201] In conjunction with some embodiments of the second aspect, in some embodiments, the first AI model is a model for performing time-domain beam prediction, and the output data includes at least one of the following:

[0202] multiple future times, where the future times are times corresponding to beams predicted by the first AI model;

[0203] at least one group corresponding to each of the future times;

[0204] identifiers of two reference signal resources corresponding to each group corresponding to each of the future times, wherein the reference signal resources are reference signal resources in the second reference signal resource set;

[0205] a beam quality corresponding to an identifier of each reference signal resource corresponding to each future time;

[0206] at least one fourth beam corresponding to each of the future times;

[0207] an identifier of a reference signal resource corresponding to each of the fourth beams corresponding to each of the future time periods, wherein the reference signal resource is a reference signal resource in the second reference signal resource set;

[0208] a beam quality corresponding to each of the fourth beams corresponding to each of the future time periods;

[0209] The fifth information is used to indicate that the beams contained in at least one group corresponding to at least one future time in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

[0210] In a third aspect, an embodiment of the present disclosure provides a model performance monitoring method, the method comprising:

[0211] The network device sends first information to the terminal device, where the first information includes configuration of a reference signal resource, where the reference signal resource is used by the terminal device to perform beam measurement;

[0212] The terminal device performs beam measurement according to the first information to obtain a beam measurement result;

[0213] The terminal device sends performance monitoring data to the network device based on the beam measurement result, and the performance monitoring data is used to determine the performance of the first AI model.

[0214] In a fourth aspect, an embodiment of the present disclosure proposes a terminal device, which may include at least one of a transceiver module and a processing module; wherein the terminal device can be used to execute the optional implementation method of the first aspect.

[0215] In a fifth aspect, an embodiment of the present disclosure proposes a network device, which may include at least one of a transceiver module and a processing module; wherein the network device can be used to execute the optional implementation method of the second aspect.

[0216] In a sixth aspect, an embodiment of the present disclosure proposes a terminal device, which may include: one or more processors; wherein the terminal device can be used to execute the optional implementation method of the first aspect.

[0217] In a seventh aspect, an embodiment of the present disclosure proposes a network device, which may include: one or more processors; wherein, the network device can be used to execute the optional implementation method of the second aspect.

[0218] In an eighth aspect, an embodiment of the present disclosure proposes a communication device, which may include: one or more processors; wherein the communication device can be used to execute an optional implementation of the first aspect or the second aspect.

[0219] In the ninth aspect, an embodiment of the present disclosure proposes a communication system, which may include: a terminal device and a network device; wherein, the terminal device is configured to execute the method described in the optional implementation manner of the first aspect, and the network device is configured to execute the method described in the optional implementation manner of the second aspect.

[0220] In a tenth aspect, an embodiment of the present disclosure proposes a storage medium storing instructions, which, when executed on a communication device, enables the communication device to execute the method described in the optional implementation of the first aspect or the second aspect.

[0221] In an eleventh aspect, an embodiment of the present disclosure proposes a program product, which, when executed by a communication device, enables the communication device to execute the method described in the optional implementation manner of the first aspect or the second aspect.

[0222] In a twelfth aspect, an embodiment of the present disclosure proposes a computer program, which, when executed on a computer, enables the computer to execute the method described in the optional implementation of the first or second aspect.

[0223] In a thirteenth aspect, an embodiment of the present disclosure provides a chip or a chip system, wherein the chip or chip system includes a processing circuit configured to execute the method described in the optional implementation of the first aspect or the second aspect.

[0224] It is understandable that the aforementioned network devices, terminal devices, communication devices, communication systems, storage media, program products, computer programs, chips, or chip systems can all be used to execute the methods proposed in the embodiments of the present disclosure. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding methods and will not be repeated here.

[0225] The present disclosure provides a model performance monitoring method, device, and storage medium. In some embodiments, the terms "model performance monitoring method" and "information processing method" and "communication method" are interchangeable; "model performance monitoring device" and "information processing device" and "communication device" are interchangeable; and "model performance monitoring system" and "communication system" are interchangeable.

[0226] The embodiments of the present disclosure are not exhaustive and are merely illustrative of some embodiments, and are not intended to be a specific limitation on the scope of protection of the present disclosure. In the absence of contradiction, each step in a certain embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a certain embodiment can also be implemented as an independent embodiment, and the order of the steps in a certain embodiment can be arbitrarily exchanged. In addition, the optional implementation methods in a certain embodiment can be arbitrarily combined; in addition, the embodiments can be arbitrarily combined. For example, some or all steps of different embodiments can be arbitrarily combined, and a certain embodiment can be arbitrarily combined with the optional implementation methods of other embodiments.

[0227] In each embodiment of the present disclosure, unless otherwise specified or provided for by logic, the terms and / or descriptions between the embodiments are consistent and can be referenced by each other. The technical features in different embodiments can be combined to form a new embodiment based on their inherent logical relationships.

[0228] The terms used in the embodiments of the present disclosure are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure.

[0229] In the embodiments of the present disclosure, unless otherwise specified, elements expressed in the singular, such as "a", "an", "the", "above", "said", "the", "the", etc., may mean "one and only one", or "one or more", "at least one", etc. For example, when using articles such as "a", "an", "the" in English in translation, the noun following the article may be understood as a singular expression or a plural expression.

[0230] In some embodiments, "plurality" may refer to two or more.

[0231] In some embodiments, the terms "at least one," "one or more," "a plurality of," "multiple," etc. may be used interchangeably.

[0232] In some embodiments, descriptions such as "at least one of A and B," "A and / or B," "A in one case, B in another case," or "in response to one case A, in response to another case B" may include the following technical solutions depending on the situation: in some embodiments, A (A is executed independently of B); in some embodiments, B (B is executed independently of A); in some embodiments, execution is selected from A and B (A and B are selectively executed); and in some embodiments, A and B (both A and B are executed). The above is also applicable when there are more branches such as A, B, and C.

[0233] In some embodiments, "A or B" and other descriptions may include the following technical solutions depending on the situation: in some embodiments, A (A is executed independently of B); in some embodiments, B (B is executed independently of A); in some embodiments, execution is selected from A and B (A and B are selectively executed). The above is also applicable when there are more branches such as A, B, C, etc.

[0234] The prefixes such as "first" and "second" in the embodiments of the present disclosure are only used to distinguish different description objects and do not constitute any restriction on the position, order, priority, quantity or content of the description objects. For the statement of the description object, please refer to the description in the context of the claims or embodiments, and no unnecessary restriction should be constituted due to the use of prefixes. For example, if the description object is a "field", the ordinal number before the "field" in the "first field" and the "second field" does not limit the position or order between the "fields". "First" and "second" do not limit whether the "fields" they modify are in the same message, nor do they limit the order of the "first field" and the "second field". For another example, if the description object is a "level", the ordinal number before the "level" in the "first level" and the "second level" does not limit the priority between the "levels". For another example, the number of description objects is not limited by the ordinal number and can be one or more. Taking "first device" as an example, the number of "devices" can be one or more. In addition, the objects modified by different prefixes can be the same or different. For example, if the description object is "device", then the "first device" and the "second device" can be the same device or different devices, and their types can be the same or different; for another example, if the description object is "information", then the "first information" and the "second information" can be the same information or different information, and their contents can be the same or different.

[0235] In some embodiments, “including A,” “comprising A,” “used to indicate A,” and “carrying A” can be interpreted as directly carrying A or indirectly indicating A.

[0236] In some embodiments, terms such as "in response to...", "in response to determining...", "in the case of...", "at the time of...", "when...", "if...", "if...", etc. can be used interchangeably.

[0237] In some embodiments, terms such as "greater than", "greater than or equal to", "not less than", "more than", "more than or equal to", "not less than", "higher than", "higher than or equal to", "not less than", and "above" can be replaced with each other, and terms such as "less than", "less than or equal to", "not greater than", "less than", "less than or equal to", "not more than", "lower than", "lower than or equal to", "not higher than", and "below" can be replaced with each other.

[0238] In some embodiments, devices and the like can be interpreted as physical or virtual, and their names are not limited to those described in the embodiments. Terms such as "device," "equipment," "device," "circuit," "network element," "node," "function," "unit," "section," "system," "network," "chip," "chip system," "entity," and "subject" can be used interchangeably.

[0239] In some embodiments, "network" can be interpreted as devices included in the network (eg, access network equipment, core network equipment, etc.).

[0240] In some embodiments, the terms "Access Network Device (AN Device)", "Radio Access Network Device (RAN Device)", "Base Station (BS)", "Radio Base Station (Radio Base Station)", "Fixed Station (Fixed Station)", "Node (Node)", "Access Point (Access Point)", "Transmission Point (TP)", "Reception Point (RP)", "Transmission and / or Reception Point (TRP))", "Panel (Panel)", "Antenna Panel (Antenna Panel)", "Antenna Array (Antenna Array)" "Cell (Cell)", "Macro Cell (Macro Cell)", "Small Cell (Small Cell)", "Femto Cell (Femto Cell)", "Pico Cell (Pico Cell)" "Sector (Sector)", "Cell Group (Cell Group)", "Serving Cell", "Carrier (Carrier)", "Component Carrier (Component Carrier)", "Bandwidth Part (BWP)" and the like can be used interchangeably.

[0241] In some embodiments, the terms "terminal", "terminal device", "user equipment (UE)", "user terminal" "mobile station (MS)", "mobile terminal (MT)", subscriber station (Subscriber Station), mobile unit (Mobile Unit), subscriber unit (Subscriber Unit), wireless unit (Wireless Unit), remote unit (Remote Unit), mobile device (Mobile Device), wireless device (Wireless Device), wireless communication device (Wireless Communication Device), remote device (Remote Device), mobile subscriber station (Mobile Subscriber Station), access terminal (Access Terminal), mobile terminal (Mobile Terminal), wireless terminal (Wireless Terminal), remote terminal (Remote Terminal), handset (Handset), user agent (User Agent), mobile client (Mobile Client), client (Client) and the like can be used interchangeably.

[0242] In some embodiments, the access network device, the core network device, or the network device can be replaced by a terminal. For example, the various embodiments of the present disclosure can also be applied to a structure in which the communication between the access network device, the core network device, or the network device and the terminal is replaced by communication between multiple terminals (for example, device-to-device (D2D), vehicle-to-everything (V2X), etc.). In this case, it is also possible to set the structure in which the terminal has all or part of the functions of the access network device. In addition, terms such as "uplink" and "downlink" can also be replaced by terms corresponding to communication between terminals (for example, "side"). For example, uplink channels, downlink channels, etc. can be replaced by side channels or direct channels, and uplinks, downlinks, etc. can be replaced by side links or direct links.

[0243] In some embodiments, the terminal may be replaced by an access network device, a core network device, or a network device. In this case, the access network device, the core network device, or the network device may have a structure that has all or part of the functions of the terminal.

[0244] In some embodiments, obtaining data, information, etc. may comply with the laws and regulations of the country where the data is obtained.

[0245] In some embodiments, data, information, etc. may be obtained with the user's consent.

[0246] In addition, each element, each row, or each column in the table of the embodiment of the present disclosure can be implemented as an independent embodiment, and the combination of any elements, any rows, and any columns can also be implemented as an independent embodiment.

[0247] FIG1 is a schematic diagram illustrating an architecture of a communication system according to an embodiment of the present disclosure. As shown in FIG1 , the communication system 100 may include a terminal device 101 and a network device 102 .

[0248] In some embodiments, the terminal device 101 may include at least one of a mobile phone, a wearable device, an Internet of Things device, a car with communication capabilities, a smart car, a tablet computer, a computer with wireless transceiver capabilities, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal device in industrial control, a wireless terminal device in self-driving, a wireless terminal device in remote medical surgery, a wireless terminal device in smart grid, a wireless terminal device in transportation safety, a wireless terminal device in smart city, and a wireless terminal device in smart home, but is not limited thereto.

[0249] In some embodiments, the network device 102 may include at least one of an access network device and a core network device.

[0250] In some embodiments, the access network device may be a node or device that accesses the terminal device to the wireless network. The access network device may include an evolved NodeB (eNB), a next generation evolved NodeB (ng-eNB), a next generation NodeB (gNB), a node B (NB), a home node B (HNB), a home evolved nodeB (HeNB), a wireless backhaul device, a radio network controller (RNC), a base station controller (BSC), a base transceiver station (BTS), a base band unit (BBU), a mobile switching center, a base station in a 6G communication system, an open base station (Open RAN), a cloud base station (Cloud RAN), a base station in other communication systems, and at least one of an access node in a Wi-Fi system, but is not limited thereto.

[0251] In some embodiments, the technical solution of the present disclosure can be applied to the Open RAN architecture. In this case, the interfaces between or within the access network devices involved in the embodiments of the present disclosure can be transformed into internal interfaces of the Open RAN, and the processes and information interactions between these internal interfaces can be implemented through software or programs.

[0252] In some embodiments, the access network device can be composed of a centralized unit (CU) and a distributed unit (DU), where the CU can also be called a control unit (Control Unit). The CU-DU structure can be used to split the protocol layer of the access network device, with the functions of some protocol layers centrally controlled by the CU, and the functions of the remaining part or all of the protocol layers distributed in the DU, which is centrally controlled by the CU, but is not limited to this.

[0253] In some embodiments, the core network device may be a single device, or may be multiple devices or a group of devices. The core network may include at least one of an Evolved Packet Core (EPC), a 5G Core Network (5GCN), and a Next Generation Core (NGC).

[0254] It can be understood that the communication system described in the embodiment of the present disclosure is for the purpose of more clearly illustrating the technical solution of the embodiment of the present disclosure, and does not constitute a limitation on the technical solution proposed in the embodiment of the present disclosure. Ordinary technicians in this field can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solution proposed in the embodiment of the present disclosure is also applicable to similar technical problems.

[0255] The following embodiments of the present disclosure may be applied to the communication system 100 shown in FIG1 , or a portion thereof, but are not limited thereto. The entities shown in FIG1 are examples. The communication system may include all or part of the entities shown in FIG1 , or may include other entities outside of FIG1 . The number and form of the entities are arbitrary, and the entities may be physical or virtual. The connection relationship between the entities is an example. The entities may be connected or disconnected, and the connection may be in any manner, including direct or indirect, wired or wireless.

[0256] The embodiments of the present disclosure can be applied to Long Term Evolution (LTE), LTE-Advanced (LTE-A), LTE-Beyond (LTE-B), SUPER 3G, IMT-Advanced, 4th generation mobile communication system (4G), 5th generation mobile communication system (5G), 5G New Radio (NR), Future Radio Access (FRA), New-Radio Access Technology (RAT), New Radio (NR), New Radio Access (NX), Future generation radio access (FX), Global System for Mobile communications (GSM (registered trademark)), CDMA2000, Ultra Mobile Broadband (UMB), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.20, Ultra-WideBand (UWB), Bluetooth (registered trademark), Public Land Mobile Network (PLMN) networks, Device-to-Device (D2D) systems, Machine-to-Machine (M2M) systems, Internet of Things (IoT) systems, Vehicle-to-Everything (V2X), systems utilizing other communication methods, and next-generation systems based on and extending these methods. Furthermore, multiple systems may be combined (for example, a combination of LTE or LTE-A with 5G).

[0257] In some embodiments of the present disclosure, the communication system may introduce a first artificial intelligence (AI) model, or other models for prediction. The first AI model may be one or more models, and the first AI model may include one or more functions. The first AI model may be deployed on the terminal device side or on the network device side.

[0258] In NR communications, for the FR2 (frequency range 2) communication band, since high-frequency channels attenuate quickly, beam-based transmission and reception can be used to ensure coverage.

[0259] In some embodiments, the network device may configure a reference signal resource set for beam measurement, and the terminal device may measure the reference signal resources in the reference signal resource set and report the IDs of X reference signal resources with relatively strong signal quality in the measurement results, as well as the Layer 1 Reference Signal Receiving Power (L1-RSRP) and / or Layer 1 Signal to Interference plus Noise Ratio (L1-SINR) of each of the X reference signal resources. The reference signal resource set configured by the network device includes X reference signal resources, each of which corresponds to a different transmit beam of the network device. For each reference signal resource, the terminal device needs to measure the reference signal resource through all receive beams, determine the beam measurement quality corresponding to each receive beam, and determine the strongest beam measurement quality from multiple beam measurement qualities. In the above measurement process, if the number of transmit beams of the network device is M and the number of receive beams of the terminal device is N, the number of beam pairs that the terminal device needs to measure is M*N.

[0260] In some embodiments, beam prediction is performed through an AI model. For example, for spatial beam prediction, the terminal device may measure only a portion of the beam pairs. For example, the beam pairs measured by the terminal device may be 1 / 8, 1 / 4, etc. of the M*N beam pairs. The beam measurement quality of the measured portion of the beam pairs is input into the AI ​​model, and the beam quality of the M*N beam pairs is predicted by the AI ​​model. For example, for spatial beam prediction, the terminal device may measure only a portion of the beams. For example, the beams measured by the terminal device may be 1 / 8, 1 / 4, etc. of the M transmit beams. The beam measurement quality of the measured portion of the beams is input into the AI ​​model, and the beam quality of the M transmit beams is predicted by the AI ​​model. For time domain beam prediction, the terminal device may measure the beam quality of the beam pairs at historical times to obtain the beam historical measurement quality. Based on the beam historical measurement quality, the AI ​​model is used to predict the beam quality of the beam pairs at future times. Similarly, for time domain beam prediction, the beam pairs can also be replaced with transmit beams.

[0261] In some embodiments, for spatial domain beam prediction, the measurement results of the beams in beam set setA can be predicted based on the measurement results of the beams in beam set setB; for time domain beam prediction, the measurement results of the beams in setA at future times can be predicted based on the measurement results of the beams in setB at historical times.

[0262] In some embodiments, for spatial beam prediction, the terminal device can measure the L1-RSRP of each beam in setB, input the measured multiple L1-RSRPs into the AI ​​model, and obtain the L1-RSRP of each beam in setA.

[0263] The relationship between setB and setA may include at least one of the following:

[0264] setB may be a subset of setA. For example, setA includes 32 reference signals (each reference signal corresponds to a beam direction), and setB includes N reference signals, where N<32, for example, N=8.

[0265] The beam corresponding to setB is a wide beam, and the beam corresponding to setA is a narrow beam. For example, setA includes 32 reference signals, each reference signal corresponds to a beam direction, and the range covered by the 32 reference signals is 120 degrees. setB includes N reference signals, for example, N=8, and the range covered by N reference signals is also 120 degrees. That is to say, the beam directions of multiple reference signals in setB cover the beam directions of multiple reference signals in setA. It can also be understood that the 32 / N reference signals in setA and the same reference signal in setB are in a quasi co-location (QCL) Type D relationship.

[0266] In some embodiments, for time domain beam prediction, the terminal device can measure the L1-RSRP of each beam in setB at historical time, input the measured multiple L1-RSRPs into the AI ​​model, and predict the L1-RSRP of each beam in setA at future time.

[0267] The relationship between setB and setA may include at least one of the following:

[0268] setB can be a subset of setA;

[0269] setB is the same as setA;

[0270] The beam corresponding to setB is a wide beam, and the beam corresponding to setA is a narrow beam.

[0271] In some embodiments, the output data of the AI ​​model mainly includes L1-RSRP and / or beam (pair) ID. However, in the MTRP scenario, multiple TRP beams may be required to serve the terminal device at the same time. In this case, the terminal device is required to perform group-based beam reporting. The terminal device needs to measure all beams. Its reference signal resource overhead is relatively large, and the complexity of the terminal device measurement is also relatively high. In this case, the AI ​​model can be used to predict the beam corresponding to each TRP, and the accuracy of the beam prediction depends on the performance of the AI ​​model. Therefore, how to monitor the performance of the AI ​​model has become an urgent problem to be solved.

[0272] FIG2A is an interactive diagram illustrating a model performance monitoring method according to an embodiment of the present disclosure. The method may be executed by the above-mentioned communication system. As shown in FIG2A , the method may include:

[0273] Step S2101: The network device sends first information to the terminal device.

[0274] In some embodiments, the terminal device may receive the first information. For example, the terminal device may receive the first information sent by the network device. For another example, the terminal device may also receive the first information sent by another entity.

[0275] In some embodiments, the name of the first information is not limited, and may be, for example, "measurement request information", "measurement configuration information", "measurement indication information", etc.

[0276] In some embodiments, the first AI model is a model for performing beam prediction, and the first information may include at least one of the following:

[0277] a first reference signal resource set, where reference signal resources in the first reference signal resource set correspond to beams to be measured;

[0278] a second reference signal resource set, where reference signal resources in the second reference signal resource set correspond to beams to be predicted;

[0279] a third reference signal resource set, the third reference signal resource set including resources for interference measurement;

[0280] A relationship between the first reference signal resource set and the second reference signal resource set.

[0281] Among them, the "beam to be measured" is the definition when the model is actually used. The "beam to be measured" can be understood as the beam that needs to be actually measured as the input of the AI ​​model. The beam here can be understood as a transmitting beam, or a transmitting and receiving beam pair.

[0282] In some embodiments, a beam may correspond to a reference signal resource, which may correspond to a reference signal, and each reference signal may correspond to a beam direction. For each beam, a network device may configure a reference signal resource corresponding to the reference signal, and a terminal device may measure the beam based on the reference signal.

[0283] In some embodiments, if the beam measurement result is L1-SINR, the first information may include the third reference signal resource set.

[0284] In some embodiments, the third reference signal resource set may include two resource sets used for interference measurement, one of which corresponds to the first reference signal resource set, and the other corresponds to the second reference signal resource set.

[0285] In some embodiments, the third reference signal resource may include a beam corresponding to each beam corresponding to the first reference signal resource set and the second reference signal resource set and used for interference measurement.

[0286] In some embodiments, if the first AI model is a model for performing spatial beamforming, the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following:

[0287] The first reference signal resource set is a subset of the second reference signal resource set;

[0288] The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same.

[0289] In some embodiments, if the first reference signal resource set is a subset of the second reference signal resource set, the first information may only include the second reference signal resource set.

[0290] The first reference signal resource set is a subset of the second reference signal resource set, indicating that the beam to be measured corresponding to the first reference signal resource set is a subset of the beam to be predicted corresponding to the second reference signal resource set. For example, if the beam to be predicted corresponding to the second reference signal resource set includes 32 beams, and the beam to be measured corresponding to the first reference signal resource set may include 4 beams among the 32 beams to be predicted corresponding to the second reference signal resource set, then the first reference signal resource set is a subset of the second reference signal resource set.

[0291] The first reference signal resource set is a subset of the second reference signal resource set, and can be represented by which reference signal resources in the second reference signal resource set the first reference signal resource set corresponds to. For example, the beam set corresponding to the second reference signal resource set is setA, and setA includes 32 beams (for convenience of explanation, the 32 beams can be recorded as beam 1, beam 2, beam 3, beam 4, beam 5, ..., beam 30, beam 31, beam 32), and the beam set corresponding to the first reference signal resource set is setB, and setB includes 4 beams (for convenience of explanation, the 4 beams can be recorded as beam 1, beam 2, beam 3, beam 4), beam 1 in setB corresponds to beam 8 in setA, beam 2 in setB corresponds to beam 16 in setA, beam 3 in setB corresponds to beam 24 in setA, and beam 4 in setB corresponds to beam 32 in setA. From the correspondence between the beams in setB and the beams in setA, it can be seen that the 4 beams in setB are a subset of the 32 beams in setA.

[0292] It should be noted that the number of beams corresponding to the first reference signal resource set and the number of beams corresponding to the second reference signal resource set are for exemplary purposes only, and the correspondence between the beams corresponding to the first reference signal resource set and the beams corresponding to the second reference signal resource set is also for exemplary purposes only. The embodiments of the present disclosure do not limit this.

[0293] The beam coverage range corresponding to the first reference signal resource set is the same as that corresponding to the second reference signal resource set, which can be expressed as each wide beam corresponding to the first reference signal resource set can cover multiple narrow beams corresponding to the second reference signal resource set. For example, if the beam set corresponding to the first reference signal resource set is setB, setB includes 8 wide beams (for the convenience of explanation, the 8 beams can be recorded as beam 1, beam 2, beam 3, beam 4, ..., beam 8), the beam set corresponding to the second reference signal resource set is setA, setA includes 32 narrow beams (for the convenience of explanation, the 32 beams can be recorded as beam 1, beam 2, beam 3, beam 4, beam 5, ..., beam 30, beam 31, beam 32 ), the correspondence between each wide beam corresponding to the first reference signal resource set and the narrow beam corresponding to the second reference signal resource set covered by the wide beam can be expressed as beam 1 in setB covers beam 1, beam 2, beam 3 to beam 4 in setA, beam 2 in setB covers beam 5, beam 6, beam 7 to beam 8 in setA, and so on, beam 8 in setB covers beam 29, beam 30, beam 31 to beam 32 in setA.

[0294] In some embodiments, if the first reference signal resource set is a subset of the second reference signal resource set, the network device may send the second reference signal resource set to the terminal device.

[0295] It should be noted that “the network device may send the second reference signal resource set to the terminal device” may be interpreted as the network device sending the second reference signal resource set to the terminal device but not sending the first reference signal resource set to the terminal device.

[0296] In some embodiments, if the first AI model is a model for performing time-domain beamforming, the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following:

[0297] The first reference signal resource set is a subset of the second reference signal resource set;

[0298] The first reference signal resource set is the same as the second reference signal resource set;

[0299] The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same.

[0300] In some embodiments, if the first reference signal resource set is a subset of the second reference signal resource set, the first information may only include the second reference signal resource set.

[0301] The first reference signal resource set is identical to the second reference signal resource set, indicating that the beam to be measured corresponding to the first reference signal resource set is identical to the beam to be predicted corresponding to the second reference signal resource set. For the time-domain beam prediction model, the beam measurement results for each beam corresponding to the first reference signal resource set obtained from historical measurements can be used to predict the beam measurement results for each beam corresponding to the second reference signal resource set in the future. This allows the terminal device to avoid performing any measurements in the future.

[0302] In some embodiments, the first reference signal resource set may include multiple fourth reference signal resource sets, and different fourth reference signal resource sets correspond to different transceiver points TRP; the second reference signal resource set may include multiple fifth reference signal resource sets, and different fifth reference signal resource sets correspond to different TRPs.

[0303] For example, if the TRP includes a first TRP and a second TRP, the first reference signal resource may include a fourth reference signal resource set A and a fourth reference signal resource set B, and the second reference signal resource set includes a fifth reference signal resource set A and a fifth reference signal resource set B, then the fourth reference signal resource set A corresponds to the fifth reference signal resource set A, the fourth reference signal resource set B corresponds to the fifth reference signal resource set B, the fourth reference signal resource set A and the fifth reference signal resource set A correspond to the first TRP, and the fourth reference signal resource set B and the fifth reference signal resource set B correspond to the second TRP. That is, the fourth reference signal resource set A is the beam to be measured corresponding to the first TRP, the fifth reference signal resource set A is the beam to be predicted corresponding to the first TRP; the fourth reference signal resource set B is the beam to be measured corresponding to the second TRP, and the fifth reference signal resource set B is the beam to be predicted corresponding to the second TRP.

[0304] In some embodiments, "different fourth reference signal resource sets correspond to different TRPs" and "different fifth reference signal resource sets correspond to different TRPs" can be understood as different fifth reference signal resource sets corresponding to different fourth reference signal resource sets, that is, the fifth reference signal resource set corresponds one-to-one to the fourth reference signal resource set.

[0305] In some embodiments, if the first AI model is a model for performing spatial beamforming, the relationship between the fourth reference signal resource set and the fifth reference signal resource set includes at least one of the following:

[0306] The fourth reference signal resource set is a subset of the fifth reference signal resource set;

[0307] The beam corresponding to the fourth reference signal resource set is a wide beam, the beam corresponding to the fifth reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the fourth reference signal resource set and the fifth reference signal resource set are the same.

[0308] In some embodiments, if the first AI model is a model for performing time-domain beamforming, the relationship between the fourth reference signal resource set and the fifth reference signal resource set includes at least one of the following:

[0309] The fourth reference signal resource set is a subset of the fifth reference signal resource set;

[0310] The fourth reference signal resource set is the same as the fifth reference signal resource set;

[0311] The beam corresponding to the fourth reference signal resource set is a wide beam, the beam corresponding to the fifth reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the fourth reference signal resource set and the fifth reference signal resource set are the same.

[0312] It should be noted that, for the description of the relationship between the fourth reference signal resource set and the fifth reference signal resource set, reference may be made to the description of the relationship between the first reference signal resource set and the second reference signal resource set, which will not be repeated here.

[0313] It should also be noted that when the fourth reference signal resource and the fifth reference signal resource correspond to the same TRP, the fourth reference signal resource and the fifth reference signal resource have the above-mentioned relationship.

[0314] Step S2102: The terminal device performs beam measurement based on the first information to obtain a beam measurement result.

[0315] In some embodiments, the beam measurements may include beam quality.

[0316] In some embodiments, the beam quality may include L1-RSRP or L1-SINR.

[0317] In some embodiments, the beam measurement result may include a first beam measurement result and a second beam measurement result. The first beam measurement result and the second beam measurement result may be L1-RSRP. The terminal device may perform beam measurement based on the beam corresponding to the first reference signal resource set to obtain the first beam measurement result; the terminal device may perform beam measurement based on the beam corresponding to the second reference signal resource set to obtain the second beam measurement result.

[0318] In some embodiments, the first beam measurement result and the second beam measurement result may be L1-SINR. The terminal device may perform beam measurement based on the beams corresponding to the first reference signal resource set and the third reference signal resource set to obtain the first beam measurement result, and perform beam measurement based on the beams corresponding to the second reference signal resource set and the third reference signal resource set to obtain the second beam measurement result.

[0319] In some embodiments, the first beam measurement result may be L1-RSRP, and the second beam measurement result may be L1-SINR. The terminal device may perform beam measurement based on the beam corresponding to the first reference signal resource set to obtain the first beam measurement result, and perform beam measurement based on the beams corresponding to the second reference signal resource set and the third reference signal resource set to obtain the second beam measurement result.

[0320] In some embodiments, the first beam measurement result may include L1-RSRP or L1-SINR of N beams corresponding to the first reference signal resource set, where N is a positive integer.

[0321] In some embodiments, the first beam measurement result may include L1-RSRP or L1-SINR of the N beams corresponding to the first reference signal resource set, and identifiers of the N reference signal resources.

[0322] Here, "N" can be interpreted as all beams corresponding to the first reference signal resource set, or can be interpreted as part of the beams corresponding to the first reference signal resource set, which is not limited in the embodiments of the present disclosure.

[0323] In some embodiments, the second beam measurement result may include identifiers of two reference signal resources corresponding to each group in at least one group in the second reference signal resource set, wherein the group may be a beam group.

[0324] In some embodiments, the second beam measurement result may include identifiers of two reference signal resources corresponding to each group in at least one group in the second reference signal resource set. The group may be a beam group supported by the terminal for simultaneous reception, a beam group supported by the terminal for simultaneous transmission, or a beam group supported by the terminal for both simultaneous reception and transmission.

[0325] In some embodiments, the second beam measurement result may include identifiers of two reference signal resources corresponding to each group in at least one group in the second reference signal resource set, and L1-RSRP or L1-SINR corresponding to each reference signal resource identifier, where the group may be a beam group.

[0326] In some embodiments, the second beam measurement result may include identifiers of two reference signal resources corresponding to each group in at least one group in the second reference signal resource set, and L1-RSRP or L1-SINR corresponding to each reference signal resource identifier. The group may be a beam group that the terminal supports simultaneous reception, a beam group that the terminal supports simultaneous transmission, or a beam group that the terminal supports both simultaneous reception and transmission.

[0327] In some embodiments, the second beam measurement result may include a reference signal resource identifier of at least one beam in the second reference signal resource set, where the beam cannot be grouped with other beams.

[0328] In some embodiments, the second beam measurement result may include an identifier of a reference signal resource of at least one beam in the second reference signal resource set, and the L1-RSRP or L1-SINR corresponding to the identifier of the reference signal resource, where the beam cannot form a group with other beams.

[0329] Step S2103: The terminal device determines the output data and the measurement data corresponding to the output data based on the beam measurement result.

[0330] In some embodiments, the output data and the measurement data corresponding to the output data can be used to determine the performance of the first AI model.

[0331] In some embodiments, the terminal device can determine the input data of the first AI model based on the beam measurement results.

[0332] In some embodiments, if the first AI model is a model for performing spatial beamforming, the input data of the first AI model may include at least one of the following:

[0333] L1-RSRPs of N beams corresponding to the first reference signal resource set, where N is a positive integer;

[0334] identifiers of reference signal resources corresponding to the N beams corresponding to the first reference signal resource set;

[0335] The second information is used to indicate that the beams contained in at least one group in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

[0336] In some embodiments, the identifier of the reference signal resource may be a synchronization signal block (SSB) ID or a channel state information reference signal (CSI-RS) ID.

[0337] In some embodiments, the second information may be indicated by the network device to the terminal device, or may be determined autonomously by the terminal device.

[0338] In some embodiments, if the first AI model can predict that the terminal device supports two beams that are simultaneously received and / or simultaneously transmitted as a group, the input data may include second information, indicating that the first AI model is expected to output only two beams that the terminal device supports being received simultaneously as a group; or indicating that the first AI model is expected to output only two beams that the terminal device supports being sent simultaneously as a group; or instructing the first AI model to output two beams that the terminal device supports being received and sent simultaneously as a group. If the first AI model can only predict that the terminal device supports two beams that are simultaneously received as a group, or the first AI model can only predict that the terminal device supports two beams that are simultaneously transmitted as a group, or the first AI model can only predict that the terminal device supports two beams that are simultaneously received and sent as a group, then the input data may not include the second information.

[0339] In some embodiments, if the first AI model is a model for performing time-domain beamforming, the input data of the first AI model may include at least one of the following:

[0340] at least one historical time;

[0341] L1-RSRP or L1-SINR of N beams corresponding to the first reference signal resource set corresponding to each historical time, where N is a positive integer;

[0342] identifiers of the reference signal resources corresponding to the N beams corresponding to the first reference signal resource set corresponding to each historical time;

[0343] The fourth information is used to indicate that the beams contained in at least one group in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

[0344] In some embodiments, the historical time may be a time when measurement is performed on the first reference signal resource set.

[0345] In some embodiments, the historical time may include multiple times, and each historical time may be a time when a measurement is performed on the first reference signal resource set.

[0346] In some embodiments, the first reference signal resources corresponding to different historical times may be the same or different, which is not limited in the embodiments of the present disclosure.

[0347] The fourth information may be included in the input data or not included in the input data, and the details are the same as those described for the second information.

[0348] In some embodiments, the fourth information may be indicated by the network device to the terminal device, or may be determined autonomously by the terminal device.

[0349] In some embodiments, after the terminal device determines the input data of the first AI model, it can input the input data into the first AI model to obtain output data output by the first AI model.

[0350] In some embodiments, if the first AI model is a model for performing spatial beamforming, the output data of the first AI model may include at least one of the following:

[0351] At least one group;

[0352] identifiers of two reference signal resources corresponding to each group, wherein the reference signal resources are reference signal resources in the second reference signal resource set;

[0353] a beam quality corresponding to an identifier of each reference signal resource;

[0354] at least one third beam;

[0355] an identifier of a reference signal resource corresponding to each of the third beams, wherein the reference signal resource is a reference signal resource in the second reference signal resource set;

[0356] a beam quality corresponding to each of the third beams;

[0357] The third information is used to indicate that the beams contained in at least one group in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

[0358] In some embodiments, the groups may be beam groups.

[0359] In some embodiments, the two reference signal resources corresponding to each group may be two beams in the second reference signal resource set.

[0360] In some embodiments, the two reference signal resources corresponding to each group may be two beams in two fifth reference signal resource sets respectively.

[0361] For example, if the beam group includes reference signal resource A and reference signal resource B, reference signal resource A can be a beam in the fifth reference signal resource set A, and reference signal resource B can be a beam in the fifth reference signal resource set B.

[0362] In some embodiments, if the output data includes a third beam, the output data does not include any beam reported in a group with the third beam.

[0363] In some embodiments, the terminal device may separately indicate that the beams contained in each group are two beams supported by the terminal device that can be received and / or transmitted simultaneously.

[0364] In some embodiments, the terminal device may simultaneously indicate that the beams contained in each group are two beams supported by the terminal device that can be received and / or transmitted simultaneously.

[0365] In some embodiments, if the first AI model can predict that the terminal device supports two beams that are simultaneously received and / or simultaneously transmitted as a group, and the input data does not include the second information, the output data may include the third information. If the first AI model can predict that the terminal device supports two beams that are simultaneously received and / or simultaneously transmitted as a group, and the input data includes the second information, the output data does not need to include the third information. If the first AI model can only predict that the terminal device supports two beams that are simultaneously received as a group, or the first AI model can only predict that the terminal device supports two beams that are simultaneously transmitted as a group, or the first AI model can only predict that the terminal device supports two beams that are simultaneously received and transmitted as a group, then the output data may not include the third information.

[0366] It should be noted that the output data includes all the information predicted by the first AI model.

[0367] In some embodiments, if the first AI model is a model for performing time-domain beamforming, the output data of the first AI model may include at least one of the following:

[0368] At least one future time, where the future time is a time corresponding to a beam predicted by the first AI model;

[0369] at least one group corresponding to each of the future times;

[0370] identifiers of two reference signal resources corresponding to each group corresponding to each of the future times, wherein the reference signal resources are reference signal resources in the second reference signal resource set;

[0371] a beam quality corresponding to an identifier of each reference signal resource corresponding to each future time;

[0372] at least one fourth beam corresponding to each of the future times;

[0373] an identifier of a reference signal resource corresponding to each of the fourth beams corresponding to each of the future time periods, wherein the reference signal resource is a reference signal resource in the second reference signal resource set;

[0374] The beam quality corresponding to each fourth beam at each future time;

[0375] The fifth information is used to indicate that the beams contained in at least one group corresponding to at least one future time in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

[0376] In some embodiments, the future time may be the time when beam prediction is performed by the first AI model.

[0377] In some embodiments, the future time may include multiple times, and the first AI model may be used to perform beam prediction once at each future time.

[0378] In some embodiments, the terminal device may report output data corresponding to each future time separately.

[0379] In some embodiments, at each future time, the terminal device may respectively indicate that the beams contained in each group are two beams supported by the terminal device that can be received and / or transmitted simultaneously.

[0380] In some embodiments, at each future time, the terminal device may simultaneously indicate that the beams included in each group are two beams supported by the terminal device that can be received and / or transmitted simultaneously.

[0381] In some embodiments, the terminal device may simultaneously indicate multiple future times, and the beams contained in multiple groups are two beams supported by the terminal device that can be received and / or transmitted simultaneously.

[0382] In some embodiments, the fifth information may be included in the output data or not, and the details are the same as those described for the third information.

[0383] In some embodiments, the measurement data corresponding to the output data may be measurement data of the beam corresponding to the output data in the second beam measurement result.

[0384] Step S2104: The terminal device determines first operation information according to the output data and the measurement data corresponding to the output data.

[0385] In some embodiments, the first operation information may be used to indicate a management operation to be performed on the first AI model.

[0386] In some embodiments, the management operation includes any one of the following: activating the first AI model, deactivating the first AI model, switching the first AI model, and not using the AI ​​model.

[0387] In some embodiments, “switching the first AI model” can be interpreted as deactivating the first AI model and activating the second AI model, where the second AI model can be any model other than the first AI model, which is not limited in the embodiments of the present disclosure.

[0388] In some embodiments, “not using the AI ​​model” can be interpreted as not using any AI model, and can also be called fallback, that is, falling back to the traditional mode (a mode that does not use the AI ​​model).

[0389] In some embodiments, in response to the first AI model being in an inactive state, it is determined that the performance of the first AI model meets performance requirements based on the output data and the measurement data corresponding to the output data, and the first operation information is determined to activate the first AI model.

[0390] In some embodiments, in response to the first AI model being in an activated state, it is determined based on the output data and the measurement data corresponding to the output data that the performance of the first AI model does not meet the performance requirements, and the first operation information is determined to be deactivating the first AI model.

[0391] In some embodiments, if the difference between the output data and the measurement data corresponding to the output data is less than or equal to a first difference threshold, it can be determined that the performance of the first AI model meets the performance requirements; if the difference between the output data and the measurement data corresponding to the output data is greater than the first difference threshold, it can be determined that the performance of the first AI model does not meet the performance requirements.

[0392] It should be noted that the above method of determining whether the performance of the first AI model meets the performance requirements is for illustration only and is not limited to this in the embodiments of the present disclosure.

[0393] Step S2105: The terminal device sends first operation information to the network device.

[0394] In some embodiments, the network device may receive the first operation information. For example, the network device may receive the first operation information sent by the terminal device. For another example, the network device may also receive the first operation information sent by another entity.

[0395] In some embodiments, the name of the first operation information is not limited, and may be, for example, "model operation report", "model operation instruction", "model operation information", "model processing information", etc.

[0396] In some embodiments, the first AI model is deployed on the terminal device side, and the terminal device sends the first operation information to the network device, which can inform the network device of the terminal device's processing decision on the first AI model.

[0397] Using the above method, the terminal device can perform beam measurement based on the first information sent by the network device to obtain a beam measurement result, determine the output data of the first AI model and the measurement data corresponding to the output data based on the beam measurement result, and determine the first operation information for the first AI model based on the output data and the measurement data corresponding to the output data, and inform the network device of the first operation information, thereby realizing performance monitoring of the first AI model.

[0398] The method involved in the embodiments of the present disclosure may include at least one of the above steps S2101 to S2105. For example, step S2101 can be implemented as an independent embodiment, step S2105 can be implemented as an independent embodiment, and steps S2102+S2103+S2104 can be implemented as independent embodiments, but are not limited thereto.

[0399] In some embodiments, steps S2101 to S2105 are all optional. For example, steps S2101 and S2105 are optional, and one or more of these steps may be omitted or replaced in different embodiments. For another example, steps S2102 and S2103 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0400] In some embodiments, reference may be made to other optional implementations described before or after the description corresponding to FIG. 2A .

[0401] In some embodiments, the names of information, etc. are not limited to the names described in the embodiments, and terms such as "information", "message", "signal", "signaling", "report", "configuration", "indication", "instruction", "command", "channel", "parameter", "domain", "field", "symbol", "symbol", "codeword", "codebook", "codeword", "codepoint", "bit", "data", "program", and "chip" can be used interchangeably.

[0402] In some embodiments, "obtain", "get", "get", "receive", "transmit", "bidirectional transmission", "send and / or receive" can be interchangeable, and can be interpreted as receiving from other entities, obtaining from protocols, obtaining from higher layers, obtaining by self-processing, autonomous implementation, etc.

[0403] In some embodiments, terms such as "send", "transmit", "report", "download", "transmit", "bidirectional transmission", "send and / or receive" can be used interchangeably.

[0404] In some embodiments, terms such as "certain", "preset", "preset", "setting", "indicated", "a certain", "any", and "first" can be interchangeable. "Specific A", "preset A", "preset A", "setting A", "indicated A", "a certain A", "any A", and "first A" can be interpreted as A pre-specified in a protocol, etc., or as A obtained through setting, configuration, or indication, etc., or as specific A, a certain A, any A, or first A, etc., but not limited to this.

[0405] FIG2B is an interactive diagram illustrating a model performance monitoring method according to an embodiment of the present disclosure. The method may be executed by the above-mentioned communication system. As shown in FIG2B , the method may include:

[0406] Step S2201: The network device sends first information to the terminal device.

[0407] The optional implementation of step S2201 can refer to the optional implementation of step S2101 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0408] Step S2202: The terminal device performs beam measurement based on the first information to obtain a beam measurement result.

[0409] The optional implementation of step S2202 can refer to the optional implementation of step S2102 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0410] Step S2203: The terminal device sends a specified event to the network device according to the beam measurement result.

[0411] In some embodiments, the specified event may be used to determine the performance of the first AI model.

[0412] In some embodiments, the designated event may be triggered based on a comparison result between the performance value of the first AI model and a first threshold value or a first offset value.

[0413] In some embodiments, the first threshold value or the first offset value may be specified by a protocol, configured by a network device, or an empirical value, which is not limited in the embodiments of the present disclosure.

[0414] In some embodiments, the designated event may include at least one of the following: a first event, a second event, a third event, a fourth event, a fifth event, a sixth event, a seventh event, and an eighth event.

[0415] It should be noted that the above-mentioned designated events are for illustration only, and the embodiments of the present disclosure do not limit the specific events included in the designated events.

[0416] In some embodiments, the terminal device determines that the beam prediction accuracy is less than a first accuracy threshold based on the beam measurement result, and sends a first event to the network device.

[0417] In some embodiments, it is determined based on the beam measurement result that the beam prediction accuracy is greater than a second accuracy threshold, and a second event is sent to the network device.

[0418] In some embodiments, it is determined based on the beam measurement result that the beam pair prediction accuracy is less than a third accuracy threshold, and the third event is sent to the network device.

[0419] In some embodiments, it is determined based on the beam measurement result that the beam pair prediction accuracy is greater than a fourth accuracy threshold, and the fourth event is sent to the network device.

[0420] In some embodiments, it is determined based on the beam measurement result that the beam quality difference is less than a first difference threshold, and the fifth event is sent to the network device.

[0421] In some embodiments, it is determined based on the beam measurement result that the beam quality difference is greater than a second difference threshold, and the sixth event is sent to the network device.

[0422] In some embodiments, it is determined based on the beam measurement result that the predicted beam quality difference is less than a third difference threshold, and the seventh event is sent to the network device.

[0423] In some embodiments, it is determined based on the beam measurement result that the predicted beam quality difference is greater than a fourth difference threshold, and the eighth event is sent to the network device.

[0424] For example, the first accuracy threshold can be 80%, the second accuracy threshold can be 90%, the first difference threshold can be 1dB, and the second difference threshold can be 3dB. The first event is triggered when the beam prediction accuracy is less than 80%, the second event is triggered when the beam prediction accuracy is greater than 90%, the fifth event is triggered when the beam quality difference is less than 1dB, and the seventh event is triggered when the beam quality difference is greater than 3dB.

[0425] In some embodiments, when the beam measurement result can trigger multiple designated events, multiple designated events can be sent to the network device.

[0426] For example, if it is determined based on the beam measurement result that the beam prediction accuracy is less than or equal to the first accuracy threshold, and the beam pair prediction accuracy is less than or equal to the second accuracy threshold, the first event and the third event can be sent to the network device; if it is determined based on the beam measurement result that the beam quality difference is less than or equal to the first difference threshold, and the predicted beam quality difference is less than or equal to the second difference threshold, the fifth event and the seventh event can be sent to the network device.

[0427] In some embodiments, after the terminal device determines that the designated event is triggered, it can report the ID of the designated event to the network device.

[0428] In some embodiments, the terminal device may also report the performance value corresponding to the specified event to the network device.

[0429] In some embodiments, the above steps are all optional steps.

[0430] FIG2C is an interactive diagram illustrating a model performance monitoring method according to an embodiment of the present disclosure. The method may be executed by the above-mentioned communication system. As shown in FIG2C , the method may include:

[0431] Step S2301: The network device sends first information to the terminal device.

[0432] The optional implementation of step S2301 can refer to the optional implementation of step S2101 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0433] Step S2302: The terminal device performs beam measurement based on the first information to obtain a beam measurement result.

[0434] The optional implementation of step S2302 can refer to the optional implementation of step S2102 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0435] Step S2303: The terminal device sends performance monitoring data to the network device based on the beam measurement result.

[0436] In some embodiments, the network device may receive performance monitoring data. For example, the network device may receive performance monitoring data sent by a terminal device. For another example, the network device may also receive performance monitoring data sent by other entities.

[0437] In some embodiments, the name of the performance monitoring data is not limited, and may be, for example, "performance report", "performance monitoring report", "model monitoring report", "model performance monitoring report", etc.

[0438] In some embodiments, the performance monitoring data may be used to determine the performance of the first AI model.

[0439] In some embodiments, the performance monitoring data includes at least one of the following:

[0440] The performance value of the first AI model;

[0441] first data, the first data including at least one of the following: input data of the first AI model, output data of the first AI model, and measurement data corresponding to the output data, where the output data is data output by the first AI model based on the input data;

[0442] a specified event, where the specified event is triggered based on a comparison result between a performance value of the first AI model and a first threshold value or a first offset value;

[0443] First operation information, where the first operation information is used to indicate a management operation to be performed on the first AI model, where the management operation includes any one of the following: activating the first AI model, deactivating the first AI model, switching the first AI model, and not using the AI ​​model.

[0444] In some embodiments, the performance value is used to indicate a performance indicator of the first AI model.

[0445] It should be noted that the input data, the output data, and the measurement data corresponding to the output data can refer to the definition in step S2103, and will not be repeated here.

[0446] It should also be noted that the designated event can refer to the definition in step S2203, and the first operation information can refer to the definition in step S2104, which will not be repeated here.

[0447] In some embodiments, the performance value may include at least one of the following:

[0448] Beam prediction accuracy;

[0449] beam pair prediction accuracy, where the beam pairs include beam pairs that the terminal device can simultaneously receive and / or simultaneously transmit;

[0450] a beam quality difference, where the beam quality difference is a difference between a measured beam quality of a first beam and a measured beam quality of a second beam, where the first beam is the beam with the strongest predicted beam quality and the second beam is the beam with the strongest measured beam quality;

[0451] A predicted beam quality difference is a difference between a predicted beam quality of the first beam and a measured beam quality of the first beam.

[0452] In some embodiments, the terminal device includes two panels (antenna panels) and can report a paired beam pair to the network device, where the beam pair includes a beam with the strongest beam quality.

[0453] In some embodiments, the beam prediction accuracy may be whether the predicted at least one beam pair includes the actual strongest beam.

[0454] For example, if the predicted at least one beam pair includes the actual strongest beam, the beam prediction is accurate; if the predicted at least one beam pair does not include the actual strongest beam, the beam prediction is inaccurate.

[0455] In some embodiments, the actual strongest beam may be the beam with the strongest measured L1-RSRP and / or L1-SINR.

[0456] In some embodiments, the beam pair prediction accuracy may be determined by determining whether the at least one predicted beam pair includes the actual optimal beam pair. The actual optimal beam pair includes beam 1 with the highest measured L1-RSRP and / or L1-SINR, and its paired beam 2. The predicted beam pair may include the prediction accuracy of multiple beam pairs, for example, the multiple beam pairs may include a first beam pair and a second beam pair.

[0457] The predicted first beam pair includes beam A and beam B, and the predicted second beam pair includes beam C and beam D. The actually measured optimal beam pair is also called the actually measured first beam pair, and this first beam pair includes beam E and beam F. Beam E is the beam with the highest measured L1-SINR. Beam F must meet at least one of the following conditions: Beam F has the highest L1-SINR among multiple beams that can be paired with beam A; the L1-SINR of beam F is greater than a first threshold; and the difference between the L1-SINRs of beam F and beam E is less than a first offset. An accurate beam pair prediction means that beams A and B are the same as beams E and beam F, or beams C and beam D are the same as beams E and beam F. Otherwise, the beam pair prediction is inaccurate. The beam pair prediction accuracy can be understood as counting M model outputs, where the number of accurate model outputs for the beam pair prediction is N, and the beam pair prediction accuracy is N / M.

[0458] The second best beam pair actually measured includes beam X and beam Y. Beam X is the beam with the strongest L1-SINR, excluding beams E and F. Beam Y must meet at least one of the following conditions: beam Y is the beam with the strongest L1-SINR, excluding beams E and F, among multiple beams that can be paired with beam X; the L1-SINR of beam Y is greater than a first threshold; and the difference between the L1-SINRs of beam Y and beam X is less than a first offset.

[0459] It should be noted that if the beam pairs actually measured also include a third beam pair, a fourth beam pair, etc., the method for determining the beam pairs of the third beam pair and the fourth beam pair can be determined by referring to the beam pairs of the above-mentioned first beam pair and the second beam pair, and will not be repeated here.

[0460] In some embodiments, the terminal device can determine the performance value of the first AI model based on the beam measurement result, and send performance monitoring data containing the performance value to the network device.

[0461] In some embodiments, the terminal device can determine the performance value of the first AI model based on the beam measurement result, determine a specified event based on the performance value, and send performance monitoring data including the specified event to the network device.

[0462] In some embodiments, the terminal device may determine the first data based on the beam measurement result, and send performance monitoring data including the first data to the network device.

[0463] In some embodiments, if the first AI model is deployed on the terminal device side, the first data sent by the terminal device to the network device may include the output data and the measurement data corresponding to the output data.

[0464] In some embodiments, if the first AI model is deployed on the network device side, the first data sent by the terminal device to the network device may include measurement data corresponding to the input data and the output data.

[0465] In some embodiments, the input data and the measurement data may be in the same performance monitoring report or in different performance monitoring reports.

[0466] In some embodiments, the terminal device may determine the first data based on the beam measurement result, determine the first operation information based on the first data, and send performance monitoring data containing the first operation information to the network device.

[0467] It should be noted that the terminal device can send one or more items of performance monitoring data to the network device, and the embodiments of the present disclosure are not limited to this.

[0468] FIG3A is a flow chart of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG3A , the embodiment of the present disclosure relates to a model performance monitoring method, which can be executed by a terminal device. The method may include:

[0469] Step S3101: Obtain first information.

[0470] The optional implementation of step S3101 can refer to the optional implementation of step S2101 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0471] In some embodiments, the terminal device may receive the first information sent by the network device, but is not limited thereto. The terminal device may also receive the first information sent by other entities.

[0472] In some embodiments, the terminal device may obtain first information specified by the protocol.

[0473] In some embodiments, the terminal device may obtain the first information from an upper layer(s).

[0474] In some embodiments, step S3101 may be omitted, and the terminal device may autonomously implement the reference signal resources indicated by the first information, or the above function may be default or by default.

[0475] Step S3102: Perform beam measurement according to the first information to obtain a beam measurement result.

[0476] The optional implementation of step S3102 can refer to the optional implementation of step S2102 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0477] Step S3103: Determine output data and measurement data corresponding to the output data according to the beam measurement result.

[0478] The optional implementation of step S3103 can refer to the optional implementation of step S2103 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0479] Step S3104: Determine first operation information according to the output data and the measurement data corresponding to the output data.

[0480] The optional implementation of step S3104 can refer to the optional implementation of step S2104 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0481] Step S3105: Send first operation information.

[0482] The optional implementation of step S3105 can refer to the optional implementation of step S2105 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0483] The method involved in the embodiments of the present disclosure may include at least one of the above steps S3101 to S3105. For example, step S3101 can be implemented as an independent embodiment, step S3105 can be implemented as an independent embodiment, and steps S3102+S3103+S3104 can be implemented as independent embodiments, but are not limited thereto.

[0484] In some embodiments, the above steps S3101 to S3105 are all optional steps. For example, steps S3101 and S3105 are optional, and one or more of these steps may be omitted or replaced in different embodiments.

[0485] In some embodiments, reference may be made to other optional implementations described before or after the description corresponding to FIG. 3A .

[0486] FIG3B is a flow chart of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG3B , the embodiment of the present disclosure relates to a model performance monitoring method, which can be executed by a terminal device. The method may include:

[0487] Step S3201: Obtain first information.

[0488] The optional implementation of step S3201 can be found in step S2201 of FIG. 2B and other related parts of the embodiment involved in FIG. 2B , which will not be described in detail here.

[0489] Step S3202: Perform beam measurement according to the first information to obtain a beam measurement result.

[0490] The optional implementation of step S3202 can be found in step S2202 of FIG. 2B and other related parts of the embodiment involved in FIG. 2B , which will not be described in detail here.

[0491] Step S3203: Send a specified event according to the beam measurement result.

[0492] The optional implementation of step S3203 can be found in step S2203 of FIG. 2B and other related parts of the embodiment involved in FIG. 2B , which will not be described in detail here.

[0493] FIG3C is a flow chart of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG3C , the embodiment of the present disclosure relates to a model performance monitoring method, which can be executed by a terminal device. The method may include:

[0494] Step S3301: Obtain first information.

[0495] The optional implementation of step S3301 can be found in step S2101 of Figure 2A, step S2201 of Figure 2B, step S3101 of Figure 3A, and the optional implementation of step S3201 of Figure 3B, as well as other related parts in the embodiments involved in Figures 2A, 2B, 3A, and 3B, which will not be repeated here.

[0496] Step S3302: Perform beam measurement according to the first information to obtain a beam measurement result.

[0497] The optional implementation of step S3302 can be found in step S2102 of Figure 2A, step S2202 of Figure 2B, step S3102 of Figure 3A, and the optional implementation of step S3202 of Figure 3B, as well as other related parts in the embodiments involved in Figures 2A, 2B, 3A, and 3B, which will not be repeated here.

[0498] Step S3303: Send performance monitoring data according to the beam measurement result.

[0499] For optional implementations of step S3303, please refer to step S2105 in Figure 2A, step S2203 in Figure 2B, step S3105 in Figure 3A, and the optional implementations of step S3203 in Figure 3B, as well as other related parts in the embodiments involved in Figures 2A, 2B, 3A, and 3B, which will not be repeated here.

[0500] In some embodiments, the first AI model is a model for performing beam prediction, and the first information includes at least one of the following:

[0501] a first reference signal resource set, where reference signal resources in the first reference signal resource set correspond to beams to be measured;

[0502] a second reference signal resource set, where the reference signal resources in the second reference signal resource set correspond to the beam to be predicted;

[0503] a third reference signal resource set, wherein the third reference signal resource set includes resources used for interference measurement;

[0504] The relationship between the first reference signal resource set and the second reference signal resource set.

[0505] In some embodiments, the first AI model is a model for performing spatial beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following:

[0506] The first reference signal resource set is a subset of the second reference signal resource set;

[0507] The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same.

[0508] In some embodiments, the first AI model is a model for performing time-domain beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following:

[0509] The first reference signal resource set is a subset of the second reference signal resource set;

[0510] The first reference signal resource set is the same as the second reference signal resource set;

[0511] The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same.

[0512] In some embodiments, the first reference signal resource set includes multiple fourth reference signal resource sets, and different fourth reference signal resource sets correspond to different transceiver points TRP; the second reference signal resource set includes multiple fifth reference signal resource sets, and different fifth reference signal resource sets correspond to different TRPs.

[0513] In some embodiments, the first AI model is a model for performing spatial beam prediction, and the relationship between the fourth reference signal resource set and the fifth reference signal resource set includes at least one of the following:

[0514] The fourth reference signal resource set is a subset of the fifth reference signal resource set;

[0515] The beam corresponding to the fourth reference signal resource set is a wide beam, the beam corresponding to the fifth reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the fourth reference signal resource set and the fifth reference signal resource set are the same.

[0516] In some embodiments, the first AI model is a model for performing time-domain beam prediction, and the relationship between the fourth reference signal resource set and the fifth reference signal resource set includes at least one of the following:

[0517] The fourth reference signal resource set is a subset of the fifth reference signal resource set;

[0518] The fourth reference signal resource set is the same as the fifth reference signal resource set;

[0519] The beam corresponding to the fourth reference signal resource set is a wide beam, the beam corresponding to the fifth reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the fourth reference signal resource set and the fifth reference signal resource set are the same.

[0520] In some embodiments, the performance monitoring data includes at least one of the following:

[0521] The performance value of the first AI model;

[0522] first data, the first data including output data of the first AI model and / or measurement data corresponding to the output data, the output data being data output by the first AI model based on input data;

[0523] a specified event, where the specified event is triggered based on a comparison result between a performance value of the first AI model and a first threshold value or a first offset value;

[0524] First operation information, where the first operation information is used to indicate a management operation to be performed on the first AI model, where the management operation includes any one of the following: activating the first AI model, deactivating the first AI model, switching the first AI model, and not using the AI ​​model.

[0525] In some embodiments, the performance value includes at least one of the following:

[0526] Beam prediction accuracy;

[0527] beam pair prediction accuracy, where the beam pairs include beam pairs that the terminal device can simultaneously receive and / or simultaneously transmit;

[0528] a beam quality difference, where the beam quality difference is a difference between a measured beam quality of a first beam and a measured beam quality of a second beam, where the first beam is the beam with the strongest predicted beam quality and the second beam is the beam with the strongest measured beam quality;

[0529] A predicted beam quality difference is a difference between a predicted beam quality of the first beam and a measured beam quality of the first beam.

[0530] In some embodiments, the beam prediction accuracy is the accuracy of including an actual optimal beam in the predicted at least one beam.

[0531] In some embodiments, the beam pair prediction accuracy is an accuracy rate of at least one predicted beam pair including an actual optimal beam pair.

[0532] In some embodiments, the designated event includes at least one of the following: a first event, a second event, a third event, a fourth event, a fifth event, a sixth event, a seventh event, and an eighth event;

[0533] The sending the performance monitoring data to the network device according to the beam measurement result includes at least one of the following:

[0534] Determining, according to the beam measurement result, that the beam prediction accuracy is less than or equal to a first accuracy threshold, and sending the first event to the network device;

[0535] Determining, according to the beam measurement result, that the beam prediction accuracy is greater than the first accuracy threshold, and sending the second event to the network device;

[0536] Determining, according to the beam measurement result, that the beam pair prediction accuracy is less than or equal to a second accuracy threshold, and sending the third event to the network device;

[0537] Determining, according to the beam measurement result, that the beam pair prediction accuracy is greater than the second accuracy threshold, and sending the fourth event to the network device;

[0538] Determining, according to the beam measurement result, that the beam quality difference is less than or equal to a first difference threshold, and sending the fifth event to the network device;

[0539] determining, according to the beam measurement result, that the beam quality difference is greater than the first difference threshold, and sending the sixth event to the network device;

[0540] Determining, according to the beam measurement result, that the predicted beam quality difference is less than or equal to a second difference threshold, and sending the seventh event to the network device;

[0541] Determine, according to the beam measurement result, that the predicted beam quality difference is greater than the second difference threshold, and send the eighth event to the network device.

[0542] In some embodiments, sending the performance monitoring data to the network device according to the beam measurement result includes:

[0543] determining the output data and the measurement data corresponding to the output data according to the beam measurement result;

[0544] determining the first operation information according to the output data and measurement data corresponding to the output data;

[0545] The first operation information is sent to the network device.

[0546] In some embodiments, determining the first operation information according to the output data and the measurement data corresponding to the output data includes:

[0547] In response to the first AI model being in an inactive state, determining, based on the output data and measurement data corresponding to the output data, that performance of the first AI model meets performance requirements, and determining that the first operation information is to activate the first AI model; or

[0548] In response to the first AI model being in an activated state, it is determined based on the output data and the measurement data corresponding to the output data that the performance of the first AI model does not meet performance requirements, and the first operation information is determined to be deactivating the first AI model.

[0549] In some embodiments, the first AI model is a model for performing spatial beam prediction, and the input data includes at least one of the following:

[0550] beam qualities of the N beams corresponding to the first reference signal resource set, the beam qualities comprising layer 1 reference signal received power L1-RSRP or layer 1 signal to interference plus noise ratio L1-SINR, where N is a positive integer;

[0551] identifiers of reference signal resources corresponding to the N beams corresponding to the first reference signal resource set;

[0552] The second information is used to indicate that the beams contained in at least one group in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

[0553] In some embodiments, the first AI model is a model for performing spatial beam prediction, and the output data includes at least one of the following:

[0554] At least one group;

[0555] identifiers of two reference signal resources corresponding to each group, wherein the reference signal resources are reference signal resources in the second reference signal resource set;

[0556] a beam quality corresponding to an identifier of each reference signal resource;

[0557] at least one third beam;

[0558] an identifier of a reference signal resource corresponding to each of the third beams, wherein the reference signal resource is a reference signal resource in the second reference signal resource set;

[0559] a beam quality corresponding to each of the third beams;

[0560] The third information is used to indicate that the beams contained in at least one group in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

[0561] In some embodiments, the first AI model is a model for performing time-domain beamforming prediction, and the input data includes at least one of the following:

[0562] at least one historical time;

[0563] beam qualities of N beams corresponding to the first reference signal resource set corresponding to each historical time, where the beam qualities include L1-RSRP or L1-SINR, where N is a positive integer;

[0564] identifiers of reference signal resources corresponding to the N beams corresponding to the first reference signal resource set corresponding to each historical time;

[0565] The fourth information is used to indicate that the beams contained in at least one group in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

[0566] In some embodiments, the first AI model is a model for performing time-domain beam prediction, and the output data includes at least one of the following:

[0567] At least one future time, where the future time is a time corresponding to a beam predicted by the first AI model;

[0568] at least one group corresponding to each of the future times;

[0569] identifiers of two reference signal resources corresponding to each group corresponding to each of the future times, wherein the reference signal resources are reference signal resources in the second reference signal resource set;

[0570] a beam quality corresponding to an identifier of each reference signal resource corresponding to each future time;

[0571] at least one fourth beam corresponding to each of the future times;

[0572] an identifier of a reference signal resource corresponding to each of the fourth beams corresponding to each of the future time periods, wherein the reference signal resource is a reference signal resource in the second reference signal resource set;

[0573] The beam quality corresponding to each fourth beam at each future time;

[0574] The fifth information is used to indicate that the beams contained in at least one group corresponding to at least one future time in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

[0575] FIG4A is a flow chart of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG4A , the embodiment of the present disclosure relates to a model performance monitoring method, which can be executed by a network device. The method may include:

[0576] Step S4101: Send the first information.

[0577] The optional implementation of step S4101 can refer to the optional implementation of step S2101 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0578] Step S4102: Obtain first operation information.

[0579] The optional implementation of step S4102 can refer to the optional implementation of step S2105 in Figure 2A and other related parts in the embodiment involved in Figure 2A, which will not be repeated here.

[0580] In some embodiments, the network device may receive the first operation information sent by the terminal device, but is not limited thereto. The network device may also receive the first operation information sent by other entities.

[0581] In some embodiments, the above steps are all optional steps.

[0582] FIG4B is a flow chart of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG4B , the embodiment of the present disclosure relates to a model performance monitoring method, which can be executed by a network device. The method may include:

[0583] Step S4201: Send the first message.

[0584] The optional implementation of step S4201 can refer to the optional implementation of step S2101 in Figure 2A, the optional implementation of step S4101 in Figure 4A, and other related parts in the embodiments involved in Figures 2 and 4A, which will not be repeated here.

[0585] Step S4202: Get the specified event.

[0586] The optional implementation of step S4202 can refer to the optional implementation of step S2203 in Figure 2B and other related parts in the embodiment involved in Figure 2B, which will not be repeated here.

[0587] In some embodiments, the network device may receive a designated event sent by a terminal device, but is not limited thereto. The network device may also receive a designated event sent by other entities.

[0588] In some embodiments, the above steps are all optional steps.

[0589] FIG4C is a flow chart of a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG4C , the embodiment of the present disclosure relates to a model performance monitoring method, which can be executed by a network device. The method may include:

[0590] Step S4301: Send the first message.

[0591] The optional implementation of step S4301 can be found in step S2101 of FIG. 2A , step S2201 of FIG. 2B , the optional implementation of step S4101 of FIG. 4A , and other related parts in the embodiments involved in FIG. 2A , FIG. 2B , and FIG. 4A , which will not be repeated here.

[0592] Step S4302: Obtain performance monitoring data.

[0593] The optional implementation of step S4302 can be found in step S2105 of Figure 2A, step S2203 of Figure 2B, step S4102 of Figure 4A, the optional implementation of step S4202 of Figure 4B, and other related parts in the embodiments involved in Figures 2A, 2B, 4A, and 4B, which will not be repeated here.

[0594] In some embodiments, the above steps are all optional steps.

[0595] In some embodiments, the first AI model is a model for performing beam prediction, and the first information includes at least one of the following:

[0596] a first reference signal resource set, where reference signal resources in the first reference signal resource set correspond to beams to be measured;

[0597] a second reference signal resource set, where the reference signal resources in the second reference signal resource set correspond to the beam to be predicted;

[0598] a third reference signal resource set, wherein the third reference signal resource set includes resources used for interference measurement;

[0599] The relationship between the first reference signal resource set and the second reference signal resource set.

[0600] In some embodiments, the first AI model is a model for performing spatial beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following:

[0601] The first reference signal resource set is a subset of the second reference signal resource set;

[0602] The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same.

[0603] In some embodiments, the first AI model is a model for performing time-domain beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following:

[0604] The first reference signal resource set is a subset of the second reference signal resource set;

[0605] The first reference signal resource set is the same as the second reference signal resource set;

[0606] The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same.

[0607] In some embodiments, the first reference signal resource set includes multiple fourth reference signal resource sets, and different fourth reference signal resource sets correspond to different transceiver points TRP; the second reference signal resource set includes multiple fifth reference signal resource sets, and different fifth reference signal resource sets correspond to different TRPs.

[0608] In some embodiments, the first AI model is a model for performing spatial beam prediction, and the relationship between the fourth reference signal resource set and the fifth reference signal resource set includes at least one of the following:

[0609] The fourth reference signal resource set is a subset of the fifth reference signal resource set;

[0610] The beam corresponding to the fourth reference signal resource set is a wide beam, the beam corresponding to the fifth reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the fourth reference signal resource set and the fifth reference signal resource set are the same.

[0611] In some embodiments, the first AI model is a model for performing time-domain beam prediction, and the relationship between the fourth reference signal resource set and the fifth reference signal resource set includes at least one of the following:

[0612] The fourth reference signal resource set is a subset of the fifth reference signal resource set;

[0613] The fourth reference signal resource set is the same as the fifth reference signal resource set;

[0614] The beam corresponding to the fourth reference signal resource set is a wide beam, the beam corresponding to the fifth reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the fourth reference signal resource set and the fifth reference signal resource set are the same.

[0615] In some embodiments, the performance monitoring data includes at least one of the following:

[0616] the performance value of the first AI model;

[0617] first data, the first data including output data of the first AI model and / or measurement data corresponding to the output data, the output data being data output by the first AI model based on input data;

[0618] a specified event, where the specified event is triggered based on a comparison result between a performance value of the first AI model and a first threshold value or a first offset value;

[0619] First operation information, where the first operation information is used to instruct a management operation to be performed on the first AI model, where the management operation includes activating the first AI model, deactivating the first AI model, switching the first AI model, and not using the AI ​​model.

[0620] In some embodiments, the performance value includes at least one of the following:

[0621] Beam prediction accuracy;

[0622] beam pair prediction accuracy, where the beam pairs include beam pairs that the terminal device can simultaneously receive and / or simultaneously transmit;

[0623] a beam quality difference, where the beam quality difference is a difference between a measured beam quality of a first beam and a measured beam quality of a second beam, where the first beam is the beam with the strongest predicted beam quality and the second beam is the beam with the strongest measured beam quality;

[0624] A predicted beam quality difference is a difference between a predicted beam quality of the first beam and a measured beam quality of the first beam.

[0625] In some embodiments, the beam prediction accuracy is the accuracy of including an actual optimal beam in the predicted at least one beam.

[0626] In some embodiments, the beam pair prediction accuracy rate includes an accuracy rate of including an actual optimal beam pair in the predicted at least one beam pair.

[0627] In some embodiments, the designated event includes at least one of the following: a first event, a second event, a third event, a fourth event, a fifth event, a sixth event, a seventh event, and an eighth event;

[0628] The receiving performance monitoring data sent by the terminal device according to the beam measurement result includes at least one of the following:

[0629] Receiving the first event sent by the terminal device, where the first event is triggered when the terminal device determines, based on the beam measurement result, that the beam prediction accuracy is less than or equal to a first accuracy threshold;

[0630] receiving the second event sent by the terminal device, where the second event is triggered when the terminal device determines, based on the beam measurement result, that the beam prediction accuracy is greater than the first accuracy threshold;

[0631] receiving the third event sent by the terminal device, where the third event is triggered when the terminal device determines, based on the beam measurement result, that the beam pair prediction accuracy is less than or equal to a second accuracy threshold;

[0632] receiving the fourth event sent by the terminal device, where the fourth event is triggered when the terminal device determines, based on the beam measurement result, that the beam pair prediction accuracy is greater than the second accuracy threshold;

[0633] receiving the fifth event sent by the terminal device, where the fifth event is triggered when the terminal device determines, based on the beam measurement result, that the beam quality difference is less than or equal to a first difference threshold;

[0634] receiving the sixth event sent by the terminal device, where the sixth event is triggered when the terminal device determines, based on the beam measurement result, that the beam quality difference is greater than the first difference threshold;

[0635] receiving the seventh event sent by the terminal device, where the seventh event is triggered when the terminal device determines, based on the beam measurement result, that the predicted beam quality difference is less than or equal to a second difference threshold;

[0636] Receive the eighth event sent by the terminal device, where the eighth event is triggered when the terminal device determines, based on the beam measurement result, that the predicted beam quality difference is greater than the second difference threshold.

[0637] In some embodiments, the first operation information is determined by the terminal device according to the output data and measurement data corresponding to the output data.

[0638] In some embodiments, the first AI model is a model for performing spatial beam prediction, and the input data includes at least one of the following:

[0639] beam qualities of the N beams corresponding to the first reference signal resource set, the beam qualities comprising layer 1 reference signal received power L1-RSRP or layer 1 signal to interference plus noise ratio L1-SINR, where N is a positive integer;

[0640] identifiers of reference signal resources corresponding to the N beams corresponding to the first reference signal resource set;

[0641] The second information is used to indicate that the beams contained in at least one group in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

[0642] In some embodiments, the first AI model is a model for performing spatial beam prediction, and the output data includes at least one of the following:

[0643] At least one group;

[0644] identifiers of two reference signal resources corresponding to each group, wherein the reference signal resources are reference signal resources in the second reference signal resource set;

[0645] a beam quality corresponding to an identifier of each reference signal resource;

[0646] at least one third beam;

[0647] an identifier of a reference signal resource corresponding to each of the third beams, wherein the reference signal resource is a reference signal resource in the second reference signal resource set;

[0648] a beam quality corresponding to each of the third beams;

[0649] The third information is used to indicate that the beams contained in at least one group in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

[0650] In some embodiments, the first AI model is a model for performing time-domain beamforming prediction, and the input data includes at least one of the following:

[0651] at least one historical time;

[0652] beam qualities of N beams corresponding to the first reference signal resource set corresponding to each historical time, where the beam qualities include L1-RSRP or L1-SINR, where N is a positive integer;

[0653] identifiers of reference signal resources corresponding to the N beams corresponding to the first reference signal resource set corresponding to each historical time;

[0654] The fourth information is used to indicate that the beams contained in at least one group in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

[0655] In some embodiments, the first AI model is a model for performing time-domain beam prediction, and the output data includes at least one of the following:

[0656] multiple future times, where the future times are times corresponding to beams predicted by the first AI model;

[0657] at least one group corresponding to each of the future times;

[0658] identifiers of two reference signal resources corresponding to each group corresponding to each of the future times, wherein the reference signal resources are reference signal resources in the second reference signal resource set;

[0659] a beam quality corresponding to an identifier of each reference signal resource corresponding to each future time;

[0660] at least one fourth beam corresponding to each of the future times;

[0661] an identifier of a reference signal resource corresponding to each of the fourth beams corresponding to each of the future time periods, wherein the reference signal resource is a reference signal resource in the second reference signal resource set;

[0662] a beam quality corresponding to each of the fourth beams corresponding to each of the future time periods;

[0663] The fifth information is used to indicate that the beams contained in at least one group corresponding to at least one future time in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

[0664] FIG5 is an interactive diagram illustrating a model performance monitoring method according to an embodiment of the present disclosure. As shown in FIG5 , an embodiment of the present disclosure relates to a model performance monitoring method, which can be executed by a communication system. The method may include:

[0665] Step S5101: The network device sends first information to the terminal device.

[0666] Optional implementations of step S5101 may refer to the optional implementations of step S2101 in FIG. 2A , step S3101 in FIG. 3A , step S4101 in FIG. 4A , and other related parts in the embodiments involved in FIG. 2A , FIG. 3A , and FIG. 4A , which will not be repeated here.

[0667] Step S5102: The terminal device performs beam measurement based on the first information to obtain a beam measurement result.

[0668] The optional implementation of step S5102 can be found in step S2102 of FIG. 2A , the optional implementation of step S3102 of FIG. 3A , and other related parts in the embodiments involved in FIG. 2A and FIG. 3A , which will not be described in detail here.

[0669] Step S5103: The terminal device sends performance monitoring data to the network device based on the beam measurement result.

[0670] Optional implementations of step S5103 may refer to step S2105 in FIG. 2A , optional implementations of step S3105 in FIG. 3A , and other related parts in the embodiments involved in FIG. 2A and FIG. 3A , which will not be described in detail here.

[0671] In some embodiments, the above method may include the method described in the embodiments of the above communication system, terminal equipment, network equipment, etc., which will not be repeated here.

[0672] In some embodiments, the first AI model is used to perform a spatial beam prediction model, and the input data of the first AI model may include at least one of the following:

[0673] L1-RSRP or L1-SINR of the beams in setB1 and setB2 (or the beam identifier ID, i.e., reference signal resource ID: SSB ID or CSI-RS resource ID)

[0674] L1-RSRP or L1-SINR of the beam in setB (or the beam identifier ID, i.e., reference signal resource ID: SSB ID or CSI-RS resource ID, can also be added). setB indicates that setB1 and setB2 are not distinguished and are mixed into one setB.

[0675] When the input data is L1-SINR, each reference signal resource in setBi is configured with a corresponding reference signal resource for measuring interference;

[0676] It is desired that the two beams included in the output beam group (or beam pair) are two beams that the terminal supports simultaneous reception and / or transmission.

[0677] Among them, setB1 and setB2 correspond to different reference signal resource sets, that is, correspond to different TRPs, setB1 and setA1 correspond to the same TRP, and setB2 and setA2 correspond to the same TRP.

[0678] The relationship between set Bi and set Ai may include at least one of the following: set Bi is a subset of set Ai, set Bi is a wide beam and set Ai is a narrow beam (one wide beam of set Bi covers multiple narrow beams of set Ai).

[0679] When setB does not distinguish between setB1 and setB2, setA does not distinguish between setA1 and setA2.

[0680] In some embodiments, the beam may be a beam, QCL Type D, spatial setting, spatial filter, spatial relation info, Transmission Configuration Indication (TCI) state.

[0681] In some embodiments, the first AI model is a model for performing spatial beam prediction. The output data of the first AI model may include at least one of the following:

[0682] N beam pairs, and two reference signal resource IDs corresponding to each beam pair. Among them, the two reference signals are two in set A, or one in set A1 and one in set A2 respectively;

[0683] N beam pairs, two reference signal resource IDs corresponding to each beam pair, and the L1-SINR corresponding to each reference signal resource ID;

[0684] M beams, that is, there is no other beam that can be paired with it, so it is reported in a single form;

[0685] The L1-SINR corresponding to the beam.

[0686] In some embodiments, when the first AI model is a model for performing temporal beam prediction, compared with the input data of the first AI model for performing spatial beam prediction, the input data of the first AI model for performing temporal beam prediction further includes multiple historical times, and each historical time includes an input data of the first AI model for performing spatial beam prediction. When set Bi < set Ai, that is, when set Bi is a subset of set Ai, set Bi of multiple historical times remains unchanged, or the beams included in set Bi of multiple historical times are different. For example, multiple set Bis can be combined into one set Ai.

[0687] In some embodiments, when the first AI model is a model for performing temporal beam prediction, the relationship between set Bi and set Ai may also include that set Bi is the same as set Ai.

[0688] In some embodiments, the first AI model is a model for performing time domain beam prediction, and the output data of the first AI model, compared with the output data of the first AI model for performing spatial domain beam prediction, also includes multiple future times, and each future time includes a portion of the output data of the first AI model when it is used to perform spatial domain beam prediction.

[0689] In some embodiments, the terminal device may receive first information sent by the network device, determine a reference signal resource based on the first information, obtain a performance monitoring report, and send the performance monitoring report to the network device.

[0690] In some embodiments, the performance monitoring report may include data used for model performance monitoring, and the performance monitoring data may include data used to calculate a performance metric, or a calculated performance metric, or an event triggered based on comparison of the performance metric with a threshold, or an operational decision made for model management (deactivating the model, activating the model, switching the model, or fallback to a non-AI mode).

[0691] In some embodiments, the performance metric used for model performance monitoring may include at least one of the following:

[0692] Top-1 beam prediction accuracy;

[0693] beam pair prediction accuracy;

[0694] L1-SINR difference

[0695] Predicted L1-SINR difference.

[0696] In some embodiments, a terminal device includes two panels. When reporting paired beam pairs, the beam pair may include the beam with the strongest L1-SINR. If the predicted beam pairs include the beam with the strongest actual L1-SINR, the Top-1 beam prediction accuracy is accurate.

[0697] In some embodiments, the predicted beam pair is a beam pair that the actual terminal can simultaneously receive and / or simultaneously transmit.

[0698] In some embodiments, the beam pair may include a first beam pair, and the first beam pair may include a first beam and a second beam. If the first beam is the beam with the strongest L1-SINR in set A1, the prediction accuracy of the beam pair can be determined by at least one of the following methods: whether the second beam paired with the first beam is the beam with the strongest L1-SINR among the multiple beams in set A2 that can be paired with the first beam, whether the L1-SINR of the second beam is greater than a first threshold value, and whether the difference between the L1-SINR of the second beam and the first beam is less than a first offset value.

[0699] In some embodiments, the beam pair may also include a second beam pair, the second beam pair may include a third beam and a fourth beam, the third beam may be the beam with the strongest L1-SINR after removing the first beam and the second beam. Similar to the above-mentioned second beam, it can be determined whether the fourth beam is the beam with the strongest L1-SINR among multiple beams paired with the third beam in another set, or whether the L1-SINR of the fourth beam is greater than the first threshold value, or whether the difference between the L1-SINR of the fourth beam and the third beam is less than the first offset value.

[0700] In some embodiments, the L1-SINR difference may be a difference between the actual L1-SINR of the beam with the strongest predicted L1-SINR and the actual L1-SINR of the beam with the strongest actual L1-SINR.

[0701] In some embodiments, the Predicted L1-SINR difference may be a difference between the predicted L1-SINR of the beam with the strongest predicted L1-SINR and the actual L1-SINR of the beam with the strongest predicted L1-SINR.

[0702] It should be noted that the above L1-SINR may also be replaced by L1-RSRP, or the above processing may be performed in combination with L1-SINR and L1-RSRP.

[0703] In some embodiments, data used to calculate performance metrics for model performance monitoring may include at least one of the following:

[0704] When the model is on the terminal device side, the terminal device needs to report the predicted value output by the model and the corresponding measurement value of each predicted value. The output predicted value can refer to the description of the embodiment shown in Figure 2A above, and the measurement value can be a measurement value corresponding to each output value;

[0705] When the model resides on the network device, the model output values ​​are also on the network device, so the terminal device only needs to report the measured values ​​for each value corresponding to the predicted value output by the model. Furthermore, for the input of the network device-side model, the terminal device also needs to report the model input data. However, the model input data and the measured values ​​used for model performance monitoring can be in the same report (performance monitoring report) or in different reports.

[0706] In some embodiments, the terminal device may report an event triggered based on a performance metric.

[0707] Network devices configure events. For example, event 1 is triggered when the Top-1 beam prediction accuracy is less than 80%; event 2 is triggered when the Top-1 beam prediction accuracy is greater than 90%; event 3 is triggered when the L1-SINR value difference is less than 1dB; event 4 is triggered when the L1-SINR value difference is greater than 3dB... Therefore, the terminal device can determine whether to trigger and which event to trigger based on the predicted value output by the terminal device-side model and the actual measured value, and then report the corresponding event ID and further report the value of the performance metric corresponding to the triggering event.

[0708] In some embodiments, the terminal device can make a judgment based on the predicted value of the terminal device side model and the actual measured value to determine whether it is necessary to activate or deactivate or switch the AI ​​model or function (the above AI model performance monitoring can be based on the performance monitoring of the model or function), and inform the network device side of the terminal device of the decision.

[0709] In some embodiments, if the model is in an activated state and is found to have poor performance, it is deactivated.

[0710] In some embodiments, if the model is in an inactive state and is found to have good performance, it is activated.

[0711] In some embodiments, the reference signal resource configuration information may include reference signal resources within setB and setA. If different TRPs are distinguished, the reference signal resource configuration information may include reference signal resources within setB1, setB2, setA1, and setA2. If setB is a subset of setA, the reference signal resource configuration information may only include the reference signal resources of setA. If setBi is a subset of setAi, the reference signal resource configuration information may only include the reference signal resources of setAi. If the input data and output data of the first AI model are L1-SINR, the reference signal resource configuration information also includes the reference signal resources corresponding to set B and set A for interference measurement.

[0712] In some embodiments of the present disclosure, a communication system is provided, which may include a network device and a terminal device, wherein the network device can execute the model performance monitoring method executed by the network device in the aforementioned embodiment of the present disclosure; the terminal device can execute the model performance monitoring method executed by the terminal device in the aforementioned embodiment of the present disclosure.

[0713] The embodiments of the present disclosure further provide an apparatus for implementing any of the above methods. For example, an apparatus is provided, comprising units or modules for implementing each step performed by a terminal in any of the above methods. For another example, another apparatus is provided, comprising units or modules for implementing each step performed by a network device (e.g., an access network device, a core network function node, a core network device, etc.) in any of the above methods.

[0714] It should be understood that the division of the various units or modules in the above device is merely a division of logical functions. In actual implementation, they may be fully or partially integrated into a physical entity, or they may be physically separated. In addition, the units or modules in the device may be implemented in the form of a processor calling software: for example, the device includes a processor, the processor is connected to a memory, and the memory stores instructions. The processor calls the instructions stored in the memory to implement any of the above methods or implement the functions of the various units or modules of the above device, wherein the processor is, for example, a general-purpose processor, such as a central processing unit (CPU) or a microprocessor, and the memory is a memory within the device or a memory outside the device. Alternatively, the units or modules in the device can be implemented in the form of hardware circuits, and the functions of some or all of the units or modules can be realized by designing the hardware circuits. The above-mentioned hardware circuits can be understood as one or more processors; for example, in one implementation, the above-mentioned hardware circuit is an application-specific integrated circuit (ASIC), and the functions of some or all of the above units or modules are realized by designing the logical relationship of the components in the circuit; for example, in another implementation, the above-mentioned hardware circuit can be realized by a programmable logic device (PLD). Taking a field programmable gate array (FPGA) as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by configuring the configuration file, thereby realizing the functions of some or all of the above units or modules. All units or modules of the above devices can be realized in the form of software called by the processor, or in the form of hardware circuits, or in part by the form of software called by the processor, and the rest by hardware circuits.

[0715] In the embodiments of the present disclosure, the processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), or a digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationship of a hardware circuit. The logical relationship of the above-mentioned hardware circuit is fixed or reconfigurable. For example, the processor is a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units or modules. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc.

[0716] Figure 6A is a structural diagram of a terminal device proposed in an embodiment of the present disclosure. As shown in Figure 6A, the terminal device 101 may include at least one of a transceiver module 6101, a processing module 6102, etc. In some embodiments, the transceiver module 6101 is configured to receive first information sent by a network device, wherein the first information includes the configuration of a reference signal resource, and the reference signal resource is used for the terminal device to perform beam measurement; the processing module 6102 is configured to perform beam measurement according to the first information to obtain a beam measurement result; the transceiver module 6101 is further configured to send performance monitoring data to the network device according to the beam measurement result, and the performance monitoring data is used to determine the performance of the first AI model. Optionally, the transceiver module 6101 can be used to execute at least one of the communication steps such as sending and / or receiving (for example, step S2101, step S2105, but not limited to this) performed by the terminal device 101 in any of the above methods, which will not be repeated here. Optionally, the processing module 6102 can be used to execute at least one of the other steps (such as step S2102, step S2103, step S2104, but not limited to these) performed by the terminal device 101 in any of the above methods, which will not be repeated here.

[0717] In some embodiments, the transceiver module may include a transmitting module and / or a receiving module, and the transmitting module and the receiving module may be separate or integrated. Optionally, the transceiver module may be interchangeable with the transceiver.

[0718] Figure 6B is a structural diagram of a network device proposed in an embodiment of the present disclosure. As shown in Figure 6B, the network device 102 may include: at least one of a transceiver module 6201, a processing module 6202, etc. In some embodiments, the transceiver module 6201 is configured to send first information to the terminal device, wherein the first information includes the configuration of reference signal resources, and the reference signal resources are used for the terminal device to perform beam measurement; the transceiver module 6201 is also configured to receive performance monitoring data sent by the terminal device according to the beam measurement result, and the beam measurement result is obtained by the terminal device performing beam measurement according to the first information, and the performance monitoring data is used to determine the performance of the first AI model. Optionally, the transceiver module 6201 can be used to execute at least one of the communication steps such as sending and / or receiving (for example, step S2101, step S2105, but not limited to this) performed by the network device 102 in any of the above methods, which will not be repeated here.

[0719] In some embodiments, the transceiver module may include a transmitting module and / or a receiving module, and the transmitting module and the receiving module may be separate or integrated. Optionally, the transceiver module may be interchangeable with the transceiver.

[0720] In some embodiments, the processing module can be a single module or can include multiple submodules. Optionally, the multiple submodules respectively execute all or part of the steps required to be executed by the processing module. Optionally, the processing module can be interchangeable with the processor.

[0721] Figure 7A is a schematic diagram of the structure of a communication device 7100 proposed in an embodiment of the present disclosure. Communication device 7100 can be a network device (e.g., an access network device, a core network device, etc.), a terminal (e.g., a user device, etc.), a chip, a chip system, or a processor that supports a first device to implement any of the above methods, or a chip, a chip system, or a processor that supports a terminal to implement any of the above methods. Communication device 7100 can be used to implement the methods described in the above method embodiments. For details, please refer to the description of the above method embodiments.

[0722] As shown in FIG7A , the communication device 7100 includes one or more processors 7101. The processor 7101 may be a general-purpose processor or a dedicated processor, for example, a baseband processor or a central processing unit. The baseband processor may be used to process communication protocols and communication data, and the central processing unit may be used to control a communication device (e.g., a base station, a baseband chip, an IoT device, an IoT device chip, a DU or CU, etc.), execute programs, and process program data. The communication device 7100 is used to perform any of the above methods.

[0723] In some embodiments, the communication device 7100 further includes one or more memories 7102 for storing instructions. Optionally, all or part of the memories 7102 may be located outside the communication device 7100.

[0724] In some embodiments, the communication device 7100 further includes one or more transceivers 7103. When the communication device 7100 includes one or more transceivers 7103, the transceiver 7103 performs at least one of the communication steps such as sending and / or receiving in the above method (for example, step S2101 and step S4101, but not limited thereto), and the processor 7101 performs at least one of the other steps (for example, step S2102, but not limited thereto).

[0725] In some embodiments, a transceiver may include a receiver and / or a transmitter. The receiver and transmitter may be separate or integrated. Optionally, the terms transceiver, transceiver unit, transceiver, and transceiver circuit may be used interchangeably; the terms transmitter, transmitting unit, transmitter, and transmitting circuit may be used interchangeably; and the terms receiver, receiving unit, receiver, and receiving circuit may be used interchangeably.

[0726] In some embodiments, the communication device 7100 may include one or more interface circuits. Optionally, the interface circuits are connected to the memory 7102 and may be used to receive signals from the memory 7102 or other devices, or to send signals to the memory 7102 or other devices. For example, the interface circuits may read instructions stored in the memory 7102 and send the instructions to the processor 7101.

[0727] The communication device 7100 described in the above embodiments may be a first device or an IoT device, but the scope of the communication device 7100 described in the present disclosure is not limited thereto, and the structure of the communication device 7100 may not be limited by FIG. 7A . The communication device may be an independent device or may be part of a larger device. For example, the communication device may be: 1) an independent integrated circuit IC, or a chip, or a chip system or subsystem; (2) a collection of one or more ICs, optionally, the above IC collection may also include a storage component for storing data or programs; (3) an ASIC, such as a modem; (4) a module that can be embedded in other devices; (5) a receiver, an IoT device, an intelligent IoT device, a cellular phone, a wireless device, a handheld device, a mobile unit, an in-vehicle device, a first device, a cloud device, an artificial intelligence device, etc.; (6) others, etc.

[0728] 7B is a schematic diagram of the structure of a chip 7200 proposed in an embodiment of the present disclosure. If the communication device 7100 can be a chip or a chip system, please refer to the schematic diagram of the structure of the chip 7200 shown in FIG7B , but the present disclosure is not limited thereto.

[0729] The chip 7200 includes one or more processors 7201 , and the chip 7200 is configured to execute any of the above methods.

[0730] In some embodiments, the chip 7200 further includes one or more interface circuits 7203. Optionally, the interface circuit 7203 is connected to the memory 7202. The interface circuit 7203 can be used to receive signals from the memory 7202 or other devices, and can be used to send signals to the memory 7202 or other devices. For example, the interface circuit 7203 can read instructions stored in the memory 7202 and send the instructions to the processor 7201.

[0731] In some embodiments, the interface circuit 7203 executes at least one of the communication steps such as sending and / or receiving in the above method (for example, step S2101, step S4101, but not limited to this), and the processor 7201 executes at least one of the other steps (for example, step S2102, but not limited to this).

[0732] In some embodiments, terms such as interface circuit, interface, transceiver pin, and transceiver may be used interchangeably.

[0733] In some embodiments, the chip 7200 further includes one or more memories 7202 for storing instructions. Alternatively, all or part of the memory 7202 may be located outside the chip 7200.

[0734] The embodiments of the present disclosure further provide a storage medium having instructions stored thereon. When the instructions are executed on the communication device 7100, the communication device 7100 executes any of the above methods. Optionally, the storage medium is an electronic storage medium. Optionally, the storage medium is a computer-readable storage medium, but is not limited thereto, and may also be a storage medium readable by other devices. Optionally, the storage medium may be a non-transitory storage medium, but is not limited thereto, and may also be a temporary storage medium.

[0735] The present disclosure also provides a program product, which, when executed by the communication device 7100, enables the communication device 7100 to perform any of the above methods. Optionally, the program product may be a computer program product.

[0736] The embodiments of the present disclosure also provide a computer program, which, when executed on a computer, enables the computer to execute any one of the above methods.

Claims

1. A model performance monitoring method, characterized in that: The method comprises: Receiving first information sent by a network device, where the first information includes configuration of a reference signal resource, where the reference signal resource is used by the terminal device to perform beam measurement; Performing beam measurement according to the first information to obtain a beam measurement result; Based on the beam measurement result, performance monitoring data is sent to the network device, and the performance monitoring data is used to determine the performance of the first AI model.

2. The method according to claim 1, characterized in that The first AI model is a model for performing beam prediction, and the first information includes at least one of the following: a first reference signal resource set, wherein the reference signal resources in the first reference signal resource set correspond to the beam to be measured; a second reference signal resource set, wherein the reference signal resources in the second reference signal resource set correspond to beams to be predicted; a third reference signal resource set, wherein the third reference signal resource set includes resources used for interference measurement; The relationship between the first reference signal resource set and the second reference signal resource set.

3. The method according to claim 2, characterized in that The first AI model is a model for performing spatial beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following: The first reference signal resource set is a subset of the second reference signal resource set; The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same.

4. The method according to claim 2, characterized in that: The first AI model is a model for performing time-domain beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following: The first reference signal resource set is a subset of the second reference signal resource set; The first reference signal resource set is the same as the second reference signal resource set; The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same.

5. The method according to any one of claims 2 to 4, characterized in that: The first reference signal resource set includes multiple fourth reference signal resource sets, and different fourth reference signal resource sets correspond to different transceiver points TRP; The second reference signal resource set includes multiple fifth reference signal resource sets, and different fifth reference signal resource sets correspond to different TRPs.

6. The method according to claim 5, characterized in that The first AI model is a model for performing spatial beam prediction, and the relationship between the fourth reference signal resource set and the fifth reference signal resource set includes at least one of the following: The fourth reference signal resource set is a subset of the fifth reference signal resource set; The beam corresponding to the fourth reference signal resource set is a wide beam, the beam corresponding to the fifth reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the fourth reference signal resource set and the fifth reference signal resource set are the same.

7. The method according to claim 5, characterized in that The first AI model is a model for performing time domain beam prediction, and the relationship between the fourth reference signal resource set and the fifth reference signal resource set includes at least one of the following: The fourth reference signal resource set is a subset of the fifth reference signal resource set; The fourth reference signal resource set is the same as the fifth reference signal resource set; The beam corresponding to the fourth reference signal resource set is a wide beam, the beam corresponding to the fifth reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the fourth reference signal resource set and the fifth reference signal resource set are the same.

8. The method according to any one of claims 2 to 7, characterized in that: The performance monitoring data includes at least one of the following: The performance value of the first AI model; first data, the first data including at least one of the following: input data of the first AI model, output data of the first AI model, and measurement data corresponding to the output data, where the output data is data output by the first AI model according to the input data; A specified event, wherein the specified event is triggered based on a comparison result between a performance value of the first AI model and a first threshold value or a first offset value; First operation information, where the first operation information is used to indicate a management operation to be performed on the first AI model, where the management operation includes any one of the following: activating the first AI model, deactivating the first AI model, switching the first AI model, and not using the AI ​​model.

9. The method according to claim 8, characterized in that The performance value includes at least one of the following: Beam prediction accuracy; A beam pair prediction accuracy rate, wherein the beam pair includes a beam pair that the terminal device can simultaneously receive and / or simultaneously transmit; A beam quality difference, where the beam quality difference is a difference between a measured beam quality of a first beam and a measured beam quality of a second beam, where the first beam is a beam with the strongest predicted beam quality and the second beam is a beam with the strongest measured beam quality; A predicted beam quality difference is a difference between a predicted beam quality of the first beam and a measured beam quality of the first beam.

10. The method according to claim 9, characterized in that The beam prediction accuracy rate is the accuracy rate of including an actual optimal beam in at least one predicted beam.

11. The method according to claim 9 or 10, characterized in that: The beam pair prediction accuracy rate is an accuracy rate of including an actual optimal beam pair in the predicted at least one beam pair.

12. The method according to any one of claims 9 to 11, characterized in that: The designated event includes at least one of the following: a first event, a second event, a third event, a fourth event, a fifth event, a sixth event, a seventh event, and an eighth event; The sending the performance monitoring data to the network device according to the beam measurement result includes at least one of the following: Determine, according to the beam measurement result, that the beam prediction accuracy is less than a first accuracy threshold, and send the first event to the network device; Determine, according to the beam measurement result, that the beam prediction accuracy is greater than a second accuracy threshold, and send the second event to the network device; Determine, according to the beam measurement result, that the beam pair prediction accuracy is less than a third accuracy threshold, and send the third event to the network device; Determine, according to the beam measurement result, that the beam pair prediction accuracy is greater than a fourth accuracy threshold, and send the fourth event to the network device; Determine, according to the beam measurement result, that the beam quality difference is less than a first difference threshold, and send the fifth event to the network device; Determine, according to the beam measurement result, that the beam quality difference is greater than a second difference threshold, and send the sixth event to the network device; Determine, according to the beam measurement result, that the predicted beam quality difference is less than a third difference threshold, and send the seventh event to the network device; Determine, according to the beam measurement result, that the predicted beam quality difference is greater than a fourth difference threshold, and send the eighth event to the network device.

13. The method according to any one of claims 8 to 12, characterized in that: The sending the performance monitoring data to the network device according to the beam measurement result includes: Determine the output data and the measurement data corresponding to the output data according to the beam measurement result; determining the first operation information according to the output data and the measurement data corresponding to the output data; The first operation information is sent to the network device.

14. The method according to claim 13, characterized in that The determining, according to the output data and the measurement data corresponding to the output data, the first operation information comprises: The first AI model is in an inactive state, and it is determined that the performance of the first AI model meets the performance requirement according to the output data and the measurement data corresponding to the output data, and the first operation information is determined to activate the first AI model; or The first AI model is in an activated state. It is determined according to the output data and the measurement data corresponding to the output data that the performance of the first AI model does not meet the performance requirement, and the first operation information is determined to be to deactivate the first AI model.

15. The method according to any one of claims 8 to 14, characterized in that: The first AI model is a model for performing spatial beam prediction, and the input data includes at least one of the following: beam qualities of the N beams corresponding to the first reference signal resource set, the beam qualities comprising layer 1 reference signal received power L1-RSRP or layer 1 signal to interference plus noise ratio L1-SINR, where N is a positive integer; identifiers of reference signal resources corresponding to the N beams corresponding to the first reference signal resource set; The second information is used to indicate that the beams contained in at least one group in the group-based beam information output by the first AI model are two beams supported by the terminal device and can be received and / or sent simultaneously.

16. The method according to any one of claims 8 to 15, characterized in that: The first AI model is a model for performing spatial beam prediction, and the output data includes at least one of the following: at least one group; identifiers of two reference signal resources corresponding to each group, wherein the reference signal resources are reference signal resources in the second reference signal resource set; a beam quality corresponding to an identifier of each of the reference signal resources; at least one third beam; an identifier of a reference signal resource corresponding to each of the third beams, wherein the reference signal resource is a reference signal resource in the second reference signal resource set; a beam quality corresponding to each of the third beams; The third information is used to indicate that the beams contained in at least one group in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

17. The method according to any one of claims 8 to 14, characterized in that: The first AI model is a model for performing time-domain beam prediction, and the input data includes at least one of the following: at least one historical time; beam quality of N beams corresponding to the first reference signal resource set corresponding to each of the historical times, the beam quality comprising L1-RSRP or L1-SINR, where N is a positive integer; identifiers of reference signal resources corresponding to the N beams corresponding to the first reference signal resource set corresponding to each of the historical times; The fourth information is used to indicate that the beams contained in at least one group in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

18. The method according to any one of claims 8 to 14, characterized in that: The first AI model is a model for performing time-domain beam prediction, and the output data includes at least one of the following: At least one future time, where the future time is a beam corresponding time for beam prediction by the first AI model; at least one group corresponding to each of the future times; identifiers of two reference signal resources corresponding to each group corresponding to each of the future times, wherein the reference signal resources are reference signal resources in the second reference signal resource set; a beam quality corresponding to an identifier of each of the reference signal resources corresponding to each of the future times; at least one fourth beam corresponding to each of the future times; an identifier of a reference signal resource corresponding to each of the fourth beams corresponding to each of the future times, wherein the reference signal resource is a reference signal resource in the second reference signal resource set; The beam quality corresponding to each of the fourth beams at each of the future times; The fifth information is used to indicate that the beams contained in at least one group corresponding to at least one future time in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

19. A model performance monitoring method, characterized in that: The method comprises: Sending first information to a terminal device, where the first information includes configuration of a reference signal resource, where the reference signal resource is used by the terminal device to perform beam measurement; Receive performance monitoring data sent by the terminal device according to the beam measurement result, where the beam measurement result is obtained by the terminal device performing beam measurement according to the first information, and the performance monitoring data is used to determine the performance of the first AI model.

20. The method according to claim 19, characterized in that The first AI model is a model for performing beam prediction, and the first information includes at least one of the following: A first reference signal resource set, wherein the reference signal resources in the first reference signal resource set correspond to the beam to be measured; a second reference signal resource set, wherein the reference signal resources in the second reference signal resource set correspond to beams to be predicted; a third reference signal resource set, wherein the third reference signal resource set includes resources used for interference measurement; The relationship between the first reference signal resource set and the second reference signal resource set.

21. The method according to claim 20, characterized in that The first AI model is a model for performing spatial beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following: The first reference signal resource set is a subset of the second reference signal resource set; The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same.

22. The method according to claim 20, characterized in that The first AI model is a model for performing time-domain beam prediction, and the relationship between the first reference signal resource set and the second reference signal resource set includes at least one of the following: The first reference signal resource set is a subset of the second reference signal resource set; The first reference signal resource set is the same as the second reference signal resource set; The beam corresponding to the first reference signal resource set is a wide beam, the beam corresponding to the second reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the first reference signal resource set and the second reference signal resource set are the same.

23. The method according to any one of claims 20 to 22, characterized in that: The first reference signal resource set includes multiple fourth reference signal resource sets, and different fourth reference signal resource sets correspond to different transceiver points TRP; The second reference signal resource set includes multiple fifth reference signal resource sets, and different fifth reference signal resource sets correspond to different TRPs.

24. The method according to claim 23, characterized in that The first AI model is a model for performing spatial beam prediction, and the relationship between the fourth reference signal resource set and the fifth reference signal resource set includes at least one of the following: The fourth reference signal resource set is a subset of the fifth reference signal resource set; The beam corresponding to the fourth reference signal resource set is a wide beam, the beam corresponding to the fifth reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the fourth reference signal resource set and the fifth reference signal resource set are the same.

25. The method according to claim 23, characterized in that The first AI model is a model for performing time domain beam prediction, and the relationship between the fourth reference signal resource set and the fifth reference signal resource set includes at least one of the following: The fourth reference signal resource set is a subset of the fifth reference signal resource set; The fourth reference signal resource set is the same as the fifth reference signal resource set; The beam corresponding to the fourth reference signal resource set is a wide beam, the beam corresponding to the fifth reference signal resource set is a narrow beam, and the beam coverage ranges corresponding to the fourth reference signal resource set and the fifth reference signal resource set are the same.

26. The method according to any one of claims 20 to 25, characterized in that: The performance monitoring data includes at least one of the following: The performance value of the first AI model; first data, the first data including at least one of the following: input data of the first AI model, output data of the first AI model, and measurement data corresponding to the output data, where the output data is data output by the first AI model according to the input data; A specified event, wherein the specified event is triggered based on a comparison result between a performance value of the first AI model and a first threshold value or a first offset value; First operation information, where the first operation information is used to indicate a management operation to be performed on the first AI model, where the management operation includes activating the first AI model, deactivating the first AI model, switching the first AI model, and not using the AI ​​model.

27. The method according to claim 26, characterized in that The performance value includes at least one of the following: Beam prediction accuracy; A beam pair prediction accuracy rate, wherein the beam pair includes a beam pair that the terminal device can simultaneously receive and / or simultaneously transmit; A beam quality difference, where the beam quality difference is a difference between a measured beam quality of a first beam and a measured beam quality of a second beam, where the first beam is a beam with the strongest predicted beam quality and the second beam is a beam with the strongest measured beam quality; A predicted beam quality difference is a difference between a predicted beam quality of the first beam and a measured beam quality of the first beam.

28. The method according to claim 27, characterized in that The beam prediction accuracy rate is the accuracy rate of including an actual optimal beam in at least one predicted beam.

29. The method according to claim 27 or 28, characterized in that The beam pair prediction accuracy rate includes an accuracy rate of including an actual optimal beam pair in the predicted at least one beam pair.

30. The method according to any one of claims 27 to 29, characterized in that: The designated event includes at least one of the following: a first event, a second event, a third event, a fourth event, a fifth event, a sixth event, a seventh event, and an eighth event; The receiving performance monitoring data sent by the terminal device according to the beam measurement result includes at least one of the following: Receiving the first event sent by the terminal device, where the first event is triggered when the terminal device determines, based on the beam measurement result, that the beam prediction accuracy is less than a first accuracy threshold; receiving the second event sent by the terminal device, where the second event is triggered when the terminal device determines, based on the beam measurement result, that the beam prediction accuracy is greater than a second accuracy threshold; receiving the third event sent by the terminal device, where the third event is triggered when the terminal device determines, based on the beam measurement result, that the prediction accuracy of the beam pair is less than a third accuracy threshold; receiving the fourth event sent by the terminal device, where the fourth event is triggered when the terminal device determines, based on the beam measurement result, that the beam pair prediction accuracy is greater than a fourth accuracy threshold; receiving the fifth event sent by the terminal device, where the fifth event is triggered when the terminal device determines, according to the beam measurement result, that the beam quality difference is less than a first difference threshold; receiving the sixth event sent by the terminal device, where the sixth event is triggered when the terminal device determines, according to the beam measurement result, that the beam quality difference is greater than a second difference threshold; receiving the seventh event sent by the terminal device, where the seventh event is triggered when the terminal device determines, based on the beam measurement result, that the predicted beam quality difference is less than a third difference threshold; Receive the eighth event sent by the terminal device, where the eighth event is triggered when the terminal device determines, based on the beam measurement result, that the predicted beam quality difference is greater than a fourth difference threshold.

31. The method according to any one of claims 26 to 30, characterized in that: The first operation information is determined by the terminal device according to the output data and measurement data corresponding to the output data.

32. The method according to any one of claims 26 to 31, characterized in that: The first AI model is a model for performing spatial beam prediction, and the input data includes at least one of the following: beam qualities of the N beams corresponding to the first reference signal resource set, the beam qualities comprising layer 1 reference signal received power L1-RSRP or layer 1 signal to interference plus noise ratio L1-SINR, where N is a positive integer; identifiers of reference signal resources corresponding to the N beams corresponding to the first reference signal resource set; The second information is used to indicate that the beams contained in at least one group in the group-based beam information output by the first AI model are two beams supported by the terminal device and can be received and / or sent simultaneously.

33. The method according to any one of claims 26 to 32, characterized in that: The first AI model is a model for performing spatial beam prediction, and the output data includes at least one of the following: at least one group; identifiers of two reference signal resources corresponding to each group, wherein the reference signal resources are reference signal resources in the second reference signal resource set; a beam quality corresponding to an identifier of each of the reference signal resources; at least one third beam; an identifier of a reference signal resource corresponding to each of the third beams, wherein the reference signal resource is a reference signal resource in the second reference signal resource set; a beam quality corresponding to each of the third beams; The third information is used to indicate that the beams contained in at least one group in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

34. The method according to any one of claims 26 to 31, characterized in that: The first AI model is a model for performing time-domain beam prediction, and the input data includes at least one of the following: at least one historical time; beam quality of N beams corresponding to the first reference signal resource set corresponding to each of the historical times, the beam quality comprising L1-RSRP or L1-SINR, where N is a positive integer; identifiers of reference signal resources corresponding to the N beams corresponding to the first reference signal resource set corresponding to each of the historical times; The fourth information is used to indicate that the beams contained in at least one group in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

35. The method according to any one of claims 26 to 31, characterized in that: The first AI model is a model for performing time-domain beam prediction, and the output data includes at least one of the following: multiple future times, where the future times are times corresponding to beams predicted by the first AI model; at least one group corresponding to each of the future times; identifiers of two reference signal resources corresponding to each group corresponding to each of the future times, wherein the reference signal resources are reference signal resources in the second reference signal resource set; a beam quality corresponding to an identifier of each of the reference signal resources corresponding to each of the future times; at least one fourth beam corresponding to each of the future times; an identifier of a reference signal resource corresponding to each of the fourth beams corresponding to each of the future times, wherein the reference signal resource is a reference signal resource in the second reference signal resource set; a beam quality corresponding to each of the fourth beams corresponding to each of the future time; The fifth information is used to indicate that the beams contained in at least one group corresponding to at least one future time in the group-based beam information output by the first AI model are two beams supported by the terminal device that can be received and / or sent simultaneously.

36. A terminal device, characterized in that: include: A transceiver module is configured to receive first information sent by a network device, where the first information includes a configuration of a reference signal resource, where the reference signal resource is used by the terminal device to perform beam measurement; A processing module, configured to perform beam measurement according to the first information to obtain a beam measurement result; The transceiver module is also configured to send performance monitoring data to the network device based on the beam measurement result, and the performance monitoring data is used to determine the performance of the first AI model.

37. A network device, characterized in that: include: A transceiver module is configured to send first information to a terminal device, where the first information includes a configuration of a reference signal resource, where the reference signal resource is used by the terminal device to perform beam measurement; The transceiver module is also configured to receive performance monitoring data sent by the terminal device according to the beam measurement result, where the beam measurement result is obtained by the terminal device performing beam measurement according to the first information, and the performance monitoring data is used to determine the performance of the first AI model.

38. A communication device, characterized in that: The invention is characterized by comprising: one or more processors; The communication device is used to execute the communication method described in any one of claims 1 to 18 or claims 19 to 35.

39. A storage medium storing instructions, characterized in that: When the instruction is executed on a terminal device, the terminal device executes the communication method as described in any one of claims 1 to 18, or when the instruction is executed on a network device, the network device executes the communication method as described in any one of claims 19 to 35.

Images