Performance evaluation method for artificial intelligence prediction model, terminal, and medium

Abstract

The present disclosure relates to a performance evaluation method for an artificial intelligence prediction model, a terminal, and a storage medium. The performance evaluation method comprises: predicting a first signal quality by means of a first model, wherein the first signal quality is a predicted value of a first reference signal transmitted by means of a first beam; obtaining first difference information, wherein the first difference information is used for representing a difference between the first signal quality and a second signal quality, and the second signal quality is a reference value of the signal quality of the first reference signal; and determining prediction performance of the first model on the basis of the first difference information. According to the method, the performance of the first model can be accurately evaluated.

Description

Performance evaluation methods, terminals, and media for artificial intelligence prediction models Technical Field

[0001] The present disclosure relates to the field of communication technologies, and in particular to a performance evaluation method, terminal, and storage medium for an artificial intelligence prediction model. Background Art

[0002] The terminal can use the AI ​​prediction model to predict the optimal beam and the signal quality of the reference signal transmitted through a specific beam. Therefore, it is necessary to evaluate the accuracy of the prediction and, based on this accuracy, evaluate the performance of the AI ​​prediction model.

[0003] Summary of the Invention

[0004] In order to evaluate the performance of an artificial intelligence prediction model, the embodiments of the present disclosure provide a performance evaluation method, a terminal, and a storage medium for an artificial intelligence prediction model.

[0005] According to a first aspect of an embodiment of the present disclosure, a performance evaluation method for an artificial intelligence prediction model is provided, which is applied to a terminal, and the method includes:

[0006] Predicting a first signal quality using a first model, where the first signal quality is a predicted value of a first reference signal transmitted via a first beam;

[0007] Acquire first difference information, where the first difference information is used to indicate a difference between the first signal quality and a second signal quality, where the second signal quality is a reference value of a signal quality of the first reference signal;

[0008] The prediction performance of the first model is determined based on the first difference information.

[0009] According to a second aspect of an embodiment of the present disclosure, a performance evaluation method for an artificial intelligence prediction model is provided, which is applied to a terminal, and the method includes:

[0010] Predicting a first beam using a first model, wherein the first beam is a beam with the best signal quality in the first set predicted by the terminal;

[0011] determining a third signal quality, where the third signal quality is a signal quality of a first reference signal transmitted through the first beam;

[0012] Obtaining second difference information, where the second difference information is used to indicate a difference between the third signal quality and a fourth signal quality, where the fourth signal quality is a reference value of a signal quality of the first reference signal transmitted through a second beam, wherein the second beam is a beam with the best signal quality in the first set measured by the terminal, or the second beam is a best beam determined from the first set based on actual signal quality, or the second beam is a default beam;

[0013] The prediction performance of the first model is determined based on the second difference information.

[0014] According to a third aspect of an embodiment of the present disclosure, a terminal is provided, including:

[0015] The processing module is configured to:

[0016] Predicting a first signal quality using a first model, where the first signal quality is a predicted value of a first reference signal transmitted via a first beam;

[0017] Acquire first difference information, where the first difference information is used to indicate a difference between the first signal quality and a second signal quality, where the second signal quality is a reference value of a signal quality of the first reference signal;

[0018] The prediction performance of the first model is determined based on the first difference information.

[0019] According to a fourth aspect of an embodiment of the present disclosure, a terminal is provided, including:

[0020] The processing module is configured to:

[0021] Predicting a first beam using a first model, wherein the first beam is a beam with the best signal quality in the first set predicted by the terminal;

[0022] determining a third signal quality, where the third signal quality is a signal quality of a first reference signal transmitted through the first beam;

[0023] Obtaining second difference information, where the second difference information is used to indicate a difference between the third signal quality and a fourth signal quality, where the fourth signal quality is a reference value of a signal quality of the first reference signal transmitted through a second beam, wherein the second beam is a beam with the best signal quality in the first set measured by the terminal, or the second beam is a best beam determined from the first set based on actual signal quality, or the second beam is a default beam;

[0024] The prediction performance of the first model is determined based on the second difference information.

[0025] According to a fifth aspect of an embodiment of the present disclosure, a terminal is provided, including:

[0026] one or more processors;

[0027] The terminal is used to execute the method described in the first aspect or the second aspect.

[0028] According to a sixth aspect of an embodiment of the present disclosure, a storage medium is provided, wherein the storage medium stores instructions, and when the instructions are executed on a communication device, the communication device executes the method described in the first aspect or the second aspect.

[0029] The method provided in the embodiments of the present disclosure compares the predicted or measured signal quality with the signal quality of a reference standard to obtain difference information between the two, and evaluates the prediction performance of the model based on the difference information. The performance of the first model can be evaluated using different beams in different application scenarios or at different time points to obtain more accurate evaluation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following drawings required for describing the embodiments are introduced. The following drawings are merely some embodiments of the present disclosure and do not impose specific limitations on the protection scope of the present disclosure.

[0031] FIG1 is a schematic diagram of a communication system architecture provided according to an embodiment of the present disclosure.

[0032] FIG2 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

[0033] FIG3 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

[0034] FIG4 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

[0035] FIG5 is a flowchart of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

[0036] FIG6 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

[0037] FIG7 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

[0038] FIG8 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

[0039] FIG9 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

[0040] FIG10 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

[0041] FIG11 is a flowchart of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

[0042] FIG12 is a flowchart of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

[0043] FIG13 is a structural diagram of a performance evaluation device for an artificial intelligence prediction model provided according to an embodiment of the present disclosure.

[0044] FIG14 is a structural diagram of a performance evaluation device for an artificial intelligence prediction model provided according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0045] The embodiments of the present disclosure provide a performance evaluation method, terminal, and storage medium for an artificial intelligence prediction model.

[0046] In a first aspect, an embodiment of the present disclosure provides a performance evaluation method for an artificial intelligence prediction model, applied to a terminal, the method comprising:

[0047] Predicting a first signal quality using a first model, where the first signal quality is a predicted value of a first reference signal transmitted via a first beam;

[0048] Acquire first difference information, where the first difference information is used to indicate a difference between the first signal quality and a second signal quality, where the second signal quality is a reference value of a signal quality of the first reference signal;

[0049] The prediction performance of the first model is determined based on the first difference information.

[0050] In the above embodiment, the predicted or measured signal quality is compared with the signal quality of the reference standard to obtain difference information between the two. The prediction performance of the model is evaluated based on the difference information. Different beams can be used for evaluation in different application scenarios or at different time points to obtain more accurate evaluation results.

[0051] In conjunction with some embodiments of the first aspect, in some embodiments, predicting the first signal quality using the first model includes:

[0052] measuring, in a first time domain unit, a signal quality of the first reference signal transmitted through the second beam to obtain a first measurement value of the first reference signal;

[0053] Taking a first measurement value of the first reference signal as input, and predicting the first signal quality in a second time domain unit by using the first model;

[0054] The time domain unit is one of the following: time slot, half time slot, symbol.

[0055] In the above embodiment, the current signal quality of the target beam is predicted based on the historical signal quality of other beams. The correlation between beams, such as the carrier aggregation relationship, can be used to obtain a reasonable prediction result.

[0056] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes:

[0057] The signal quality of the first reference signal transmitted through the first beam in the second time domain unit is measured to obtain the second signal quality.

[0058] In the above embodiment, the measured signal quality is used as the reference quality, so that the difference information represents the relative difference, which can better reflect the impact of the application scenario on the signal quality in different application scenarios.

[0059] In combination with some embodiments of the first aspect, in some embodiments, the second signal quality is a true value of the first reference signal transmitted through the first beam in the second time domain unit.

[0060] In the above embodiment, the actual signal quality is used as the reference signal quality, so that the first difference information represents the absolute difference. This makes the first difference information unaffected by the application scenario and can better reflect the actual difference.

[0061] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes:

[0062] obtaining the second signal quality;

[0063] sending the first signal quality and the second signal quality to a test device;

[0064] The obtaining of the first difference information includes:

[0065] The first difference information sent by the test device is received, where the first difference information is determined by the test device according to the first signal quality and the second signal quality.

[0066] In the above embodiment, the first signal quality and the second signal quality are provided to the test device, which is applicable to an application scenario in which the terminal has the capability of calculating the second signal quality.

[0067] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes:

[0068] sending the first signal quality to a test device;

[0069] The obtaining of the first difference information includes:

[0070] The first difference information sent by the test device is received, where the first difference information is determined by the test device according to the first signal quality and the second signal quality, and the second signal quality is determined by the test device.

[0071] In the above embodiment, only the first signal quality is provided to the test device, which is applicable to an application scenario where the test device has the ability to calculate the second signal quality.

[0072] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes:

[0073] Predicting a first beam using a first model, wherein the first beam is a beam with the best signal quality in the first set predicted by the terminal;

[0074] determining a third signal quality, where the third signal quality is a signal quality of a first reference signal transmitted through the first beam;

[0075] Obtaining second difference information, where the second difference information is used to indicate a difference between the third signal quality and a fourth signal quality, where the fourth signal quality is a reference value of a signal quality of the first reference signal transmitted through a second beam, wherein the second beam is a beam with the best signal quality in the first set measured by the terminal, or the second beam is a best beam determined from the first set based on actual signal quality, or the second beam is a default beam;

[0076] The prediction performance of the first model is determined based on the second difference information.

[0077] In the above embodiment, the optimal beam is predicted, and the first signal quality is determined according to the optimal beam, so that the performance of the model in predicting the signal quality and the performance of the predicted beam can be evaluated simultaneously.

[0078] In conjunction with some embodiments of the first aspect, in some embodiments, determining the third signal quality includes:

[0079] The third signal quality is predicted using the first model.

[0080] In conjunction with some embodiments of the first aspect, in some embodiments, the determining the third signal quality, the method further includes:

[0081] measuring a signal quality of the first reference signal corresponding to each beam in the first set to obtain a measurement result;

[0082] According to the identification information of the first beam, it is determined that the third signal quality is a measurement result corresponding to the identification information of the first beam in the measurement results.

[0083] In the above embodiment, the measurement results can be used for reasonable processing required in the evaluation process.

[0084] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes:

[0085] sending the third signal quality to a test device;

[0086] The obtaining of the second difference information includes:

[0087] receiving the second difference information sent by the test device, wherein the fourth signal quality is a true value of the signal quality of the first reference signal transmitted through the second beam determined by the test device;

[0088] In the above embodiment, only the first signal quality is provided to the test device, which is applicable to an application scenario in which the test device has the ability to obtain the second beam and the ability to determine the fourth signal quality.

[0089] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes:

[0090] predicting the third signal quality by using the first model;

[0091] sending the third signal quality and the measurement result to a test device;

[0092] The obtaining of the second difference information includes:

[0093] The second difference information sent by the test device is received, wherein the fourth signal quality is the signal quality corresponding to the second beam in the measurement result.

[0094] In the above embodiment, providing the first signal quality and the measurement result to the test device is applicable to an application scenario where the test device has the ability to obtain the second beam but does not have the ability to determine the fourth signal quality.

[0095] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes:

[0096] Sending identification information of the first beam and the measurement result to a test device;

[0097] The obtaining of the second difference information includes:

[0098] The second difference information sent by the test device is received, wherein the first signal quality is the signal quality corresponding to the first beam in the measurement result, and the fourth signal quality is the signal quality corresponding to the second beam in the measurement result.

[0099] The above embodiment is applicable to an application scenario in which the test device has the ability to obtain the second beam but does not have the ability to calculate the fourth signal quality.

[0100] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes:

[0101] predicting the third signal quality by using the first model;

[0102] Determining the fourth signal quality according to the measurement result, where the fourth signal quality is the signal quality corresponding to the second beam in the measurement result;

[0103] sending the third signal quality and the fourth signal quality to a test device;

[0104] The obtaining of the second difference information includes:

[0105] Receive the second difference information sent by the testing device.

[0106] The above embodiment is applicable to applications where the test device does not have the ability to obtain the second beam and does not have the ability to calculate the fourth signal quality.

[0107] scene.

[0108] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes:

[0109] determining a third signal quality and a fourth signal quality according to the measurement result, where the third signal quality is a measurement result corresponding to the identification information of the first beam in the measurement result, and the fourth signal quality is a signal quality corresponding to the second beam in the measurement result;

[0110] sending the third signal quality and the fourth signal quality to a test device;

[0111] The obtaining of the second difference information includes:

[0112] Receive the second difference information sent by the testing device.

[0113] The above embodiment is applicable to applications where the test device does not have the ability to obtain the second beam and does not have the ability to calculate the fourth signal quality.

[0114] scene.

[0115] In a second aspect, an embodiment of the present disclosure provides a terminal, including:

[0116] The processing module is configured to:

[0117] Predicting a first signal quality using a first model, where the first signal quality is a predicted value of a first reference signal transmitted via a first beam;

[0118] Acquire first difference information, where the first difference information is used to indicate a difference between the first signal quality and a second signal quality, where the second signal quality is a reference value of a signal quality of the first reference signal;

[0119] The prediction performance of the first model is determined based on the first difference information.

[0120] In a third aspect, an embodiment of the present disclosure provides a terminal, including:

[0121] The processing module is configured to:

[0122] Predicting a first beam using a first model, wherein the first beam is a beam with the best signal quality in the first set predicted by the terminal;

[0123] determining a third signal quality, where the third signal quality is a signal quality of a first reference signal transmitted through the first beam;

[0124] Obtaining second difference information, where the second difference information is used to indicate a difference between the third signal quality and a fourth signal quality, where the fourth signal quality is a reference value of a signal quality of the first reference signal transmitted through a second beam, wherein the second beam is a beam with the best signal quality in the first set measured by the terminal, or the second beam is a best beam determined from the first set based on actual signal quality, or the second beam is a default beam;

[0125] The prediction performance of the first model is determined based on the second difference information.

[0126] In a fourth aspect, an embodiment of the present disclosure provides a terminal, comprising: one or more processors; wherein the above-mentioned terminal is used to execute the method described in the first aspect or the second aspect.

[0127] In a fifth aspect, an embodiment of the present disclosure provides a storage medium, wherein the storage medium stores instructions. When the instructions are executed on a communication device, the communication device executes the method described in the first aspect or the second aspect.

[0128] In a sixth aspect, an embodiment of the present disclosure provides a program product. When the program product is executed by a communication device, the communication device executes the method described in the first aspect or the second aspect.

[0129] In a seventh aspect, an embodiment of the present disclosure provides a computer program, which, when executed on a computer, enables the computer to execute the method described in the first aspect or the second aspect.

[0130] In an eighth aspect, an embodiment of the present disclosure provides a chip or a chip system, wherein the chip or chip system includes a processing circuit configured to execute the method according to the first or second aspect.

[0131] It is understandable that the above-mentioned terminals, storage media, program products, computer programs, chips or chip systems are all used to execute the methods proposed in the embodiments of the present disclosure. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding methods and will not be repeated here.

[0132] The embodiments of the present disclosure are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be understood as limiting the present disclosure.

[0133] As shown in Figure 1, the method provided in the embodiment of the present disclosure can be applied to a wireless communication system 100, which may include a terminal 101 and a test device 102. It should be noted that the wireless communication system 100 may also include other devices, and this application does not limit the devices included in the wireless communication system 100.

[0134] It should be understood that the above wireless communication system 100 is applicable to both low-frequency scenarios and high-frequency scenarios. Application scenarios of the wireless communication system 100 include, but are not limited to, long-term evolution (LTE) systems, LTE frequency division duplex (FDD) systems, LTE time division duplex (TDD) systems, worldwide interoperability for micro wave access (WiMAX) communication systems, cloud radio access networks (CRAN) systems, future fifth-generation (5G) systems, new radio (NR) communication systems, or future evolved public land mobile networks (PLMN) systems, Internet of Things systems, etc.

[0135] The terminal 101 shown above may be a terminal, an access terminal, a terminal unit, a terminal station, a mobile station (MS), a remote station, a remote terminal, a mobile terminal, a wireless communication device, a terminal agent, an Internet of Things terminal, etc. The terminal 101 may have wireless transceiver functions, and may be capable of communicating (e.g., wirelessly communicating) with one or more network devices of one or more communication systems and receiving network services provided by the network devices.

[0136] Among them, the terminal 101 can be a cellular phone, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA) device, a handheld device with wireless communication function, a computing device or other processing device connected to a wireless modem, an in-vehicle device, a wearable device, a terminal in a future 5G network, or a terminal in a future evolved PLMN network, etc.

[0137] Terminal 101 may also be an Internet of Things (IoT) terminal. IoT terminals have lower complexity, manufacturing costs, and maintenance costs than conventional terminals. IoT terminals may not require batteries and instead receive power from electromagnetic signals. IoT terminals may also include a battery with a small amount of electrical storage capacity. This battery does not require manual charging, but instead obtains energy from external sources, such as electromagnetic waves, thermal energy, kinetic energy, and so on.

[0138] When applying an AI prediction model, the terminal can predict the RSRP of some beams based on the measured RSRP of other beams. This AI prediction model may introduce additional prediction errors. To determine the prediction performance of the AI ​​prediction model, it is necessary to calculate difference information. This difference information can be the difference between the predicted RSRP and the reference RSRP, indicating the difference between the predicted value and the actual value.

[0139] Then the prediction accuracy includes measurement error and prediction error, so it is necessary to consider both prediction error and measurement error.

[0140] As shown in Tables 1 and 2 below, Table 1 shows the accuracy requirements of SSB based on L1-RSRP absolute accuracy in FR1, and Table 2 shows the accuracy requirements of SSB based on L1-RSRP absolute accuracy in FR2.

[0141] Table 1

[0142] Table 2

[0143] In the following embodiments, the signal quality may be reference signal receiving power (RSRP), such as L1-RSRP.

[0144] FIG2 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure. As shown in FIG2 , the method includes the following steps:

[0145] In step S2101, the terminal predicts a first signal quality using a first model.

[0146] In some embodiments, the first model is an artificial intelligence model.

[0147] In some embodiments, the first model is an artificial intelligence predictive model.

[0148] In some embodiments, the first signal quality is a predicted value of a first reference signal transmitted via the first beam.

[0149] In some embodiments, the first reference signal is a downlink reference signal.

[0150] In some embodiments, the first beam is any beam in the first set.

[0151] In some embodiments, the first set is a beam set.

[0152] In some embodiments, the first set is a preset set, and the beams included in the first set are pre-set.

[0153] In some embodiments, predicting the first signal quality using the first model includes:

[0154] The method comprises measuring the signal quality of the first reference signal transmitted through the second beam in a first time domain unit to obtain a first measurement value of the first reference signal; using the first measurement value of the first reference signal as input, and predicting the first signal quality in the second time domain unit through a first model; wherein the time domain unit is one of: time slot, half time slot, and symbol.

[0155] In one example, the signal quality of the first reference signal transmitted through the first beam (beam1) is measured in the first time slot (slot1) to obtain a first measurement value of the first reference signal. The first measurement value of the first reference signal is used as input to predict the first signal quality of the second beam (beam2) in the second time slot (slot2) through the first model.

[0156] In another example, the signal quality of the first reference signal transmitted through the first beam (beam3) is measured in the first time slot (slot1) to obtain a first measurement value of the first reference signal. The first measurement value of the first reference signal is used as input to predict the first signal quality of the second beam (beam4) in the second time slot (slot2) through the first model.

[0157] Step S2102: The terminal determines a second signal quality.

[0158] In some embodiments, the terminal measures the signal quality of the first reference signal to obtain the measured second signal quality.

[0159] In some embodiments, the second signal quality is a reference value of the first reference signal transmitted via the first beam.

[0160] In some embodiments, the terminal obtains a known second signal quality.

[0161] In some embodiments, the second signal quality is a true signal quality.

[0162] Optionally, the true signal quality is an accurate signal quality.

[0163] Alternatively, the true signal quality may be referred to as genie-aided signal quality.

[0164] Step S2103: The testing device sends a second signal quality to the terminal.

[0165] In some embodiments, the second signal quality is a true signal quality.

[0166] Optionally, the true signal quality is an accurate signal quality.

[0167] Alternatively, the true signal quality may be referred to as genie-aided signal quality.

[0168] Step S2104: The terminal determines first difference information according to the first signal quality and the second signal quality.

[0169] In some embodiments, the first difference information represents a difference between the first signal quality and the second signal quality.

[0170] In some embodiments, the first difference information is the difference between the predicted RSRP and the actual RSRP.

[0171] Step S2105: The terminal determines the prediction performance of the first model based on the first difference information.

[0172] The method involved in the embodiment of the present disclosure may include step S2101, step S2103, step S2105, and step S2106. In this case, step S2102 is omitted.

[0173] The method involved in the embodiment of the present disclosure may further include step S2101, step S2102, step S2105, and step S2106. In this case, step S2103 is omitted.

[0174] FIG3 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure. As shown in FIG3 , the method includes the following steps:

[0175] Step S3101: The terminal predicts a first signal quality using a first model.

[0176] The optional implementation of step S3101 can refer to the optional implementation of step S2101 in Figure 2 and other related parts in the embodiment involved in Figure 2, which will not be repeated here.

[0177] Step S3102: The terminal obtains a second signal quality.

[0178] For optional implementations of step S3102 , reference may be made to the optional implementations of step S2102 or step S2103 in FIG. 2 .

[0179] Step S3103: The terminal sends the first signal quality and the second signal quality to the test device.

[0180] Step S3104: The testing device determines first difference information according to the first signal quality and the second signal quality.

[0181] When step S2102 of FIG. 2 is used in step S3102 , the first difference information is relative difference information.

[0182] When step S2103 of FIG. 2 is used in step S3102 , the first difference information is absolute difference information.

[0183] Step S3105: The testing device sends first difference information to the terminal.

[0184] In some embodiments, the first difference information is the difference between the predicted RSRP and the actual RSRP.

[0185] Step S3106: The terminal determines the prediction performance of the first model based on the first difference information.

[0186] FIG4 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure. As shown in FIG4 , the method includes the following steps:

[0187] Step S4101: The terminal predicts a first signal quality using a first model.

[0188] The optional implementation of step S4101 can refer to the optional implementation of step S2101 in Figure 2 and other related parts in the embodiment involved in Figure 2, which will not be repeated here.

[0189] Step S4102: The terminal sends a first signal quality to the test device.

[0190] Step S4103: The testing device obtains the second signal quality.

[0191] In some embodiments, the test device simulates the behavior of a base station, that is, sends a first reference signal to the terminal through different beams, so that the test device can calculate the second signal quality.

[0192] In some embodiments, the second signal quality is a true signal quality.

[0193] Optionally, the true signal quality is an accurate signal quality.

[0194] Alternatively, the true signal quality may be referred to as genie-aided signal quality.

[0195] Step S4104: The testing device determines first difference information according to the first signal quality and the second signal quality.

[0196] In some embodiments, the first difference information is absolute difference information.

[0197] Step S4105: The testing device sends first difference information to the terminal.

[0198] Step S4106: The terminal determines the prediction performance of the first model based on the first difference information.

[0199] FIG5 is an interactive diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure, which is applied to a terminal, as shown in FIG5 , and includes the following steps:

[0200] In step S5101, the terminal predicts a first beam using a first model.

[0201] In some embodiments, the first beam is a beam with the best signal quality in the first set predicted by the terminal.

[0202] In some embodiments, the signal quality of the first reference signal transmitted via the first beam is the best in the first set.

[0203] In some embodiments, the first beam is the best beam in the first set.

[0204] In some embodiments, the first set is a preset set, and the beams included in the first set are pre-set.

[0205] In some embodiments, the index of the predicted first beam is i.

[0206] Step S5102: The terminal predicts a third signal quality using the first model.

[0207] In some embodiments, the third signal quality is a signal quality of a first reference signal transmitted via the first beam.

[0208] In some embodiments, the predicted quality of the third signal is recorded as Predicted L1-RSRP i .

[0209] Step S5103: The terminal measures the signal quality of the first reference signal corresponding to each beam in the first set to obtain a measurement result.

[0210] In some embodiments, the measurement result includes a signal quality of a first reference signal corresponding to each beam in the first set.

[0211] Step S5104: The terminal determines a third signal quality according to the measurement result.

[0212] In some embodiments, based on the identification information of the first beam, the third signal quality is determined to be a measurement result corresponding to the identification information of the first beam in the measurement results.

[0213] Step S5104: The terminal obtains second difference information.

[0214] In some embodiments, the second difference information is used to represent a difference between the third signal quality and the fourth signal quality, where the fourth signal quality is a reference value of the signal quality of the first reference signal transmitted via the second beam.

[0215] In some embodiments, the second beam is a beam with the best signal quality in the first set measured by the terminal.

[0216] In some embodiments, the second beam is the best beam determined from the first set based on actual signal quality.

[0217] In some embodiments, the second beam is a default beam. For example, the index of the second beam is q, where q is a default value. The index of the second beam can be called the best genie-aided beam index.

[0218] Step S5105: The terminal determines the prediction performance of the first model based on the second difference information.

[0219] In the method according to the embodiment of the present disclosure, step S5102 or step S5103 may be omitted.

[0220] FIG6 is an interactive diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure. As shown in FIG6 , the method includes the following steps:

[0221] In step S6101, the terminal predicts a first beam according to a first model.

[0222] The optional implementation of step S6101 can refer to the optional implementation of step S5101 in Figure 5 and other related parts in the embodiment involved in Figure 5, which will not be repeated here.

[0223] Step S6102: The terminal predicts the third signal quality according to the first model.

[0224] The optional implementation of step S6102 can refer to the optional implementation of step S5102 in Figure 5 and other related parts in the embodiment involved in Figure 5, which will not be repeated here.

[0225] Step S6103: The terminal sends a third signal quality to the test device.

[0226] In some embodiments, the terminal sends the third signal quality and identification information of the first beam to the test device.

[0227] Step S6104: The testing device determines the second beam and the fourth signal quality.

[0228] In some embodiments, the test equipment determines the second beam.

[0229] In some embodiments, the second beam is the best beam determined from the first set based on actual signal quality.

[0230] In some embodiments, the test device simulates the behavior of the base station, that is, sends the first reference signal to the terminal through different beams, so that the test device can calculate the fourth signal quality.

[0231] In some embodiments, the second beam is a default beam. For example, the index of the second beam is q, where q is a default value. The index of the second beam can be called the best genie-aided beam index.

[0232] In some embodiments, the three signal qualities are recorded as Predicted L1-RSRP i .

[0233] In some embodiments, the fourth signal quality is recorded as Genie L1-RSRP q .

[0234] Step S6105: The testing device determines second difference information.

[0235] In some embodiments, the testing device determines that the second difference information is the difference between the third signal quality and the fourth signal quality. The second difference information is the difference between the third signal quality predicted by the terminal and the fourth signal quality calculated by the first reference signal corresponding to the actual optimal beam (i.e., the second beam), which can be considered as absolute difference information.

[0236] In some embodiments, the second difference information is: Predicted L1-RSRP i -Genie L1-RSRP q .

[0237] Step S6106: The testing device sends second difference information to the terminal.

[0238] Step S6107: The terminal determines the prediction performance of the first model based on the second difference information.

[0239] FIG7 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure. As shown in FIG7 , the method includes the following steps:

[0240] In step S7101, the terminal predicts a first beam using a first model.

[0241] The optional implementation of step S7101 can refer to the optional implementation of step S5101 in Figure 5 and other related parts in the embodiment involved in Figure 5, which will not be repeated here.

[0242] Step S7102: The terminal predicts a third signal quality using the first model.

[0243] For optional implementations of step S7102, reference may be made to the optional implementations of step S5102 in FIG5 and other related parts in the embodiment involved in FIG5, which will not be described in detail here.

[0244] In step S7103 , the terminal measures the signal quality of the first reference signal corresponding to each beam in the first set to obtain a measurement result.

[0245] The optional implementation of step S7103 can refer to the optional implementation of step S5103 in Figure 5 and other related parts in the embodiment involved in Figure 5, which will not be repeated here.

[0246] Step S7104: The terminal sends the third signal quality and the measurement result to the test device.

[0247] In some embodiments, the terminal sends the third signal quality, the measurement result, and identification information of the first beam to the test device.

[0248] Step S7105: The testing device determines a second beam, and determines a fourth signal quality based on the second beam and the measurement result.

[0249] In some embodiments, the second beam is the best beam determined from the first set based on actual signal quality.

[0250] In some embodiments, the second beam is a default beam. For example, the index of the second beam is q, where q is a default value. The index of the second beam can be called the best genie-aided beam index.

[0251] In some embodiments, the test device determines that the fourth signal quality is a measured signal quality corresponding to the second beam in the measurement result.

[0252] In some embodiments, the third signal quality is recorded as Predicted L1-RSRP i .

[0253] In some embodiments, the fourth signal quality is recorded as Measured L1-RSRP q .

[0254] Step S7106: The testing device determines second difference information.

[0255] In some embodiments, the testing device determines the second difference information as the difference between the third signal quality and the fourth signal quality. The second difference information is the difference between the predicted third signal quality and the measured fourth signal quality corresponding to the actual optimal beam (i.e., the second beam), which can be considered as absolute difference information.

[0256] In some embodiments, the second difference information is: Predicted L1-RSRP i -Measured L1-RSRP q .

[0257] Step S7107: The testing device sends second difference information to the terminal.

[0258] Step S7108: The terminal determines the prediction performance of the first model based on the second difference information.

[0259] FIG8 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure. As shown in FIG8 , the method includes the following steps:

[0260] In step S8101, the terminal predicts a first beam using a first model.

[0261] The optional implementation of step S8101 can refer to the optional implementation of step S5101 in Figure 5 and other related parts in the embodiment involved in Figure 5, which will not be repeated here.

[0262] Step S8102: The terminal measures the signal quality of the first reference signal corresponding to each beam in the first set to obtain a measurement result.

[0263] The optional implementation of step S8102 can refer to the optional implementation of step S5103 in Figure 5 and other related parts in the embodiment involved in Figure 5, which will not be repeated here.

[0264] Step S8103: The terminal determines a third signal quality according to the measurement result.

[0265] In some embodiments, the third signal quality is determined to be a measurement result corresponding to the identification information of the first beam in the measurement results.

[0266] Step S8104: The terminal sends the measurement result to the test device.

[0267] In some embodiments, the terminal sends the measurement result and an identification of the first beam to the test device.

[0268] Step S8105: The testing device determines a second beam, and determines a fourth signal quality based on the second beam and the measurement result.

[0269] In some embodiments, the second beam is the best beam determined from the first set based on actual signal quality. For example, the index of the second beam is q, which can be called the best genie-aided beam index.

[0270] In some embodiments, the second beam is a default beam.

[0271] In some embodiments, the test device determines that the fourth signal quality is a measured signal quality corresponding to the second beam in the measurement result.

[0272] In some embodiments, the third signal quality is recorded as Measured L1-RSRP i .

[0273] In some embodiments, the fourth signal quality is recorded as Measured L1-RSRP q .

[0274] Step S8106: The testing device determines the second difference information.

[0275] In some embodiments, when the testing device receives the measurement result and the identifier of the first beam, it determines the third signal quality based on the identifier of the first beam and the measurement result, and determines the second difference information based on the third signal quality and the fourth signal quality. The second difference information is the difference between the measured third signal quality and the measured fourth signal quality corresponding to the actual optimal beam (i.e., the second beam), and can be considered as absolute difference information.

[0276] In some embodiments, the second difference information is: Measured L1-RSRP i -Measured L1-RSRP q .

[0277] Step S8107: The testing device sends second difference information to the terminal.

[0278] Step S8108: The terminal determines the prediction performance of the first model based on the second difference information.

[0279] In the method according to the embodiment of the present disclosure, step S8103 may be omitted.

[0280] FIG9 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure. As shown in FIG9 , the method includes the following steps:

[0281] In step S9101, the terminal predicts a first beam based on a first model.

[0282] The optional implementation of step S9101 can refer to the optional implementation of step S5101 in Figure 5 and other related parts in the embodiment involved in Figure 5, which will not be repeated here.

[0283] Step S9102: The terminal predicts a third signal quality based on the first model.

[0284] For optional implementations of step S9102, reference may be made to the optional implementations of step S5102 in FIG5 and other related parts in the embodiment involved in FIG5, which will not be described in detail here.

[0285] Step S9103: The terminal measures the signal quality of the first reference signal corresponding to each beam in the first set to obtain a measurement result.

[0286] The optional implementation of step S9103 can refer to the optional implementation of step S5103 in Figure 5 and other related parts in the embodiment involved in Figure 5, which will not be repeated here.

[0287] Step S9104: The terminal determines a fourth signal quality according to the measurement result.

[0288] In some embodiments, the fourth signal quality is the best signal quality in the measurement results. The index of the second beam corresponding to the fourth signal quality is k.

[0289] In one example, the third signal quality is recorded as Predicted L1-RSRPi, and the fourth signal quality is recorded as Measured L1-RSRP k .

[0290] In some embodiments, the fourth signal quality is the signal quality corresponding to the second beam in the measurement result. The second beam is the best beam determined from the first set based on the actual signal quality. For example, the index of the second beam is q, and the index of the second beam can be called the best genie-aided beam index.

[0291] In one example, the third signal quality is recorded as Predicted L1-RSRPi, and the fourth signal quality is recorded as Measured L1-RSRPq.

[0292] Step S9105: The terminal sends the third signal quality and the fourth signal quality to the test device.

[0293] In some embodiments, the terminal sends the third signal quality, the fourth signal quality, and identification information of the first beam to the test device.

[0294] Step S9106: The testing device determines the second difference information.

[0295] In some embodiments, the testing device determines the second difference information as the difference between the received third signal quality and the received fourth signal quality. The second difference information is the difference between the predicted third signal quality and the measured optimal signal quality, i.e., the fourth signal quality, and can be considered as relative difference information.

[0296] In some embodiments, the second difference information is: Predicted L1-RSRP i -Measured L1-RSRP q .

[0297] Step S9107: The testing device sends second difference information to the terminal.

[0298] Step S9108: The terminal determines the prediction performance of the first model based on the second difference information.

[0299] FIG10 is an interactive schematic diagram of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure. As shown in FIG10 , the method includes the following steps:

[0300] Step S10101: The terminal predicts a first beam based on a first model.

[0301] The optional implementation of step S10101 can refer to the optional implementation of step S5101 in Figure 5 and other related parts in the embodiment involved in Figure 5, which will not be repeated here.

[0302] Step S10102: The terminal measures the signal quality of the first reference signal corresponding to each beam in the first set to obtain a measurement result.

[0303] The optional implementation of step S10103 can refer to the optional implementation of step S5103 in Figure 5 and other related parts in the embodiment involved in Figure 5, which will not be repeated here.

[0304] Step S10103: The terminal determines the third signal quality and the fourth signal quality according to the measurement result.

[0305] In some embodiments, the third signal quality is the measurement quality corresponding to the first beam in the measurement result, the fourth signal quality is the signal quality with the best signal quality in the measurement result, and the index of the second beam corresponding to the fourth signal quality is k.

[0306] In one example, the third signal quality is recorded as Measured L1-RSRP i The fourth signal quality is recorded as Measured L1-RSRP k .

[0307] In some embodiments, the fourth signal quality is the signal quality corresponding to the second beam in the measurement result. The second beam is the best beam determined from the first set based on the actual signal quality. For example, the index of the second beam is q, and the index of the second beam can be called the best genie-aided beam index.

[0308] In one example, the third signal quality is recorded as Measured L1-RSRPi, and the fourth signal quality is recorded as Measured L1-RSRPq.

[0309] Step S10104: The terminal sends the third signal quality and the fourth signal quality to the test device.

[0310] In some embodiments, the terminal sends the third signal quality, the fourth signal quality, and identification information of the first beam to the test device.

[0311] Step S10105: The testing device determines second difference information.

[0312] In some embodiments, the testing device determines the second difference information as the difference between the received third signal quality and the received fourth signal quality. The second difference information is the difference between the measured third signal quality and the measured optimal signal quality, i.e., the fourth signal quality, and can be considered as relative difference information.

[0313] In some embodiments, the second difference information is: Measured L1-RSRP i -Measured L1-RSRP q .

[0314] Step S10106: The testing device sends second difference information to the terminal.

[0315] Step S10107: The terminal determines the prediction performance of the first model based on the second difference information.

[0316] FIG11 is a flowchart of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure, which is applied to a terminal. As shown in FIG11 , the method includes the following steps:

[0317] Step S11101: predict a first signal quality using a first model.

[0318] The first signal quality is a predicted value of the first reference signal transmitted through the first beam.

[0319] In some embodiments, predicting the first signal quality using the first model includes:

[0320] measuring, in a first time domain unit, a signal quality of the first reference signal transmitted through the second beam to obtain a first measurement value of the first reference signal;

[0321] Taking a first measurement value of the first reference signal as input, and predicting the first signal quality in a second time domain unit by using the first model;

[0322] The time domain unit is one of the following: time slot, half time slot, symbol.

[0323] In some embodiments, the historical quality is a measured signal quality of a first reference signal transmitted through at least one beam other than the first beam.

[0324] Step S11102: determine the first difference information.

[0325] The first difference information is used to indicate the difference between the first signal quality and the second signal quality, and the second signal quality is a reference value of the signal quality of the first reference signal.

[0326] In some embodiments, the reference quality is a measured signal quality; the method further includes: measuring a signal quality of the first reference signal transmitted via the first beam in the second time domain unit to obtain the second signal quality.

[0327] In some embodiments, the reference quality is a real signal quality; the method further comprises: determining the real quality of the first reference signal; and obtaining a second signal quality.

[0328] In some embodiments, the method further comprises: sending the first signal quality and the second signal quality to the test device;

[0329] Determining first difference information includes:

[0330] First difference information sent by the test device is received, where the first difference information is determined by the test device according to the first signal quality and the second signal quality.

[0331] In some embodiments, the method further comprises:

[0332] sending the first signal quality to a test device;

[0333] Determining first difference information includes:

[0334] First difference information sent by a test device is received, where the first difference information is determined by the test device according to the first signal quality and the second signal quality, and the second signal quality is determined by the test device.

[0335] Step S11103: Determine the prediction performance of the first model based on the first difference information.

[0336] FIG12 is a flowchart of a performance evaluation method for an artificial intelligence prediction model provided according to an embodiment of the present disclosure, which is applied to a terminal. As shown in FIG12 , the method includes the following steps:

[0337] Step S12101: predict a first beam using a first model.

[0338] The first beam is the beam with the best signal quality in the first set predicted by the terminal.

[0339] Step S12102: Determine the third signal quality.

[0340] The third signal quality is the signal quality of the first reference signal transmitted through the first beam.

[0341] In some embodiments, determining the third signal quality includes: predicting the third signal quality using a first model.

[0342] In some embodiments, determining the third signal quality includes: measuring a signal quality of the first reference signal corresponding to each beam in the first set to obtain a measurement result;

[0343] According to the identification information of the first beam, it is determined that the third signal quality is a measurement result corresponding to the identification information of the first beam in the measurement results.

[0344] Step S12102: Determine the second difference information.

[0345] In some embodiments, the second difference information is used to represent a difference between the third signal quality and the fourth signal quality, where the fourth signal quality is the signal quality of the first reference signal transmitted through the second beam.

[0346] In some embodiments, the second beam is the beam with the best signal quality in the first set measured by the terminal, or the second beam is the best beam determined from the first set based on actual signal quality, or the second beam is a default beam.

[0347] In some embodiments, before step S12102, the method further includes:

[0348] sending the third signal quality to a test device;

[0349] The obtaining of the second difference information includes:

[0350] The second difference information sent by the test device is received, where the fourth signal quality is a true value of the signal quality of the first reference signal transmitted through the second beam determined by the test device.

[0351] In some embodiments, before step S12102, the method further includes:

[0352] predicting the third signal quality by using the first model;

[0353] sending the third signal quality and the measurement result to a test device;

[0354] The obtaining of the second difference information includes:

[0355] The second difference information sent by the test device is received, wherein the fourth signal quality is the signal quality corresponding to the second beam in the measurement result.

[0356] In some embodiments, before step S12102, the method further includes:

[0357] Sending identification information of the first beam and the measurement result to a test device;

[0358] The obtaining of the second difference information includes:

[0359] The second difference information sent by the test device is received, wherein the first signal quality is the signal quality corresponding to the first beam in the measurement result, and the fourth signal quality is the signal quality corresponding to the second beam in the measurement result.

[0360] In some embodiments, before step S12102, the method further includes:

[0361] predicting the third signal quality by using the first model;

[0362] determining the fourth signal quality according to the measurement results, where the fourth signal quality is the best signal quality in the measurement results;

[0363] sending the third signal quality and the fourth signal quality to a test device;

[0364] The obtaining of the second difference information includes:

[0365] Receive the second difference information sent by the testing device.

[0366] In some embodiments, before step S12102, the method further includes:

[0367] determining a third signal quality and a fourth signal quality according to the measurement results, where the third signal quality is a measurement result corresponding to the identification information of the first beam in the measurement results, and the fourth signal quality is a signal quality with the best signal quality in the measurement results;

[0368] sending the third signal quality and the fourth signal quality to a test device;

[0369] The obtaining of the second difference information includes:

[0370] Receive the second difference information sent by the testing device.

[0371] Step S12103: Determine the prediction performance of the first model based on the second difference information.

[0372] Receive the second difference information sent by the testing device.

[0373] The embodiments of the present disclosure further provide an apparatus for implementing any of the above methods. For example, an apparatus is provided, comprising units or modules for implementing each step performed by a terminal in any of the above methods. For another example, another apparatus is provided, comprising units or modules for implementing each step performed by a network device (e.g., an access network device, a core network function node, a core network device, etc.) in any of the above methods.

[0374] It should be understood that the division of the various units or modules in the above device is merely a division of logical functions. In actual implementation, they may be fully or partially integrated into a physical entity, or they may be physically separated. In addition, the units or modules in the device may be implemented in the form of a processor calling software: for example, the device includes a processor, the processor is connected to a memory, and the memory stores instructions. The processor calls the instructions stored in the memory to implement any of the above methods or implement the functions of the various units or modules of the above device, wherein the processor is, for example, a general-purpose processor, such as a central processing unit (CPU) or a microprocessor, and the memory is a memory within the device or a memory outside the device. Alternatively, the units or modules in the device can be implemented in the form of hardware circuits, and the functions of some or all of the units or modules can be realized by designing the hardware circuits. The above-mentioned hardware circuits can be understood as one or more processors; for example, in one implementation, the above-mentioned hardware circuit is an application-specific integrated circuit (ASIC), which realizes the functions of some or all of the above units or modules by designing the logical relationship of the components in the circuit; for example, in another implementation, the above-mentioned hardware circuit can be realized by a programmable logic device (PLD). Taking a field programmable gate array (FPGA) as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by configuring the configuration file, thereby realizing the functions of some or all of the above units or modules. All units or modules of the above devices can be realized in the form of software called by the processor, or in the form of hardware circuits, or in part by the form of software called by the processor, and the rest by hardware circuits.

[0375] In the embodiments of the present disclosure, the processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction reading and execution capabilities, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), or a digital signal processor (DSP). In another implementation, the processor can implement certain functions through the logical relationship of the hardware circuit. The logical relationship of the above-mentioned hardware circuit is fixed or reconfigurable. For example, the processor is a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and implementing the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units or modules. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc.

[0376] FIG13 is a schematic structural diagram of a communication device according to an embodiment of the present disclosure, which is applied to a terminal. As shown in FIG13 , the communication device 13 may include: a processing module 1301 and a transceiver module 1302 .

[0377] In some embodiments, the processing module 1301 is configured to:

[0378] Predicting a first signal quality using a first model, where the first signal quality is a predicted value of a first reference signal transmitted via a first beam;

[0379] Acquire first difference information, where the first difference information is used to indicate a difference between the first signal quality and a second signal quality, where the second signal quality is a reference value of a signal quality of the first reference signal;

[0380] The prediction performance of the first model is determined based on the first difference information.

[0381] In some embodiments, the processing module 1301 is further configured to: measure, in a first time domain unit, a signal quality of the first reference signal transmitted through the second beam to obtain a first measurement value of the first reference signal;

[0382] Taking a first measurement value of the first reference signal as input, and predicting the first signal quality in a second time domain unit by using the first model;

[0383] The time domain unit is one of the following: time slot, half time slot, symbol.

[0384] In some embodiments, the processing module 1301 is further configured to: measure a signal quality of the first reference signal transmitted via the first beam in the second time domain unit to obtain a second signal quality, where the second signal quality is a true value of the first reference signal transmitted via the first beam in the second time domain unit.

[0385] In some embodiments, the processing module 1301 is further configured to: obtain the second signal quality;

[0386] The transceiver module 1302 is configured to: send the first signal quality and the second signal quality to the test device;

[0387] First difference information sent by the test device is received, where the first difference information is determined by the test device according to the first signal quality and the second signal quality.

[0388] In some embodiments, the transceiver module 1302 is further configured to: send the first signal quality to the test device; receive the first difference information sent by the test device, the first difference information is determined by the test device based on the first signal quality and the second signal quality, and the second signal quality is determined by the test device.

[0389] In some embodiments, the processing module 1301 is configured to:

[0390] Predicting a first beam using a first model, wherein the first beam is a beam with the best signal quality in the first set predicted by the terminal;

[0391] determining a third signal quality, where the third signal quality is a signal quality of a first reference signal transmitted through the first beam;

[0392] Obtaining second difference information, where the second difference information is used to indicate a difference between the third signal quality and a fourth signal quality, where the fourth signal quality is a reference value of a signal quality of the first reference signal transmitted through a second beam, wherein the second beam is a beam with the best signal quality in the first set measured by the terminal, or the second beam is a best beam determined from the first set based on actual signal quality, or the second beam is a default beam;

[0393] The prediction performance of the first model is determined based on the second difference information.

[0394] In some embodiments, the processing module 1301 is further configured to: predict the third signal quality using the first model.

[0395] In some embodiments, the processing module 1301 is further configured to: measure a signal quality of the first reference signal corresponding to each beam in the first set to obtain a measurement result;

[0396] According to the identification information of the first beam, it is determined that the third signal quality is a measurement result corresponding to the identification information of the first beam in the measurement results.

[0397] In some embodiments, the transceiver module 1302 is further configured to: send the third signal quality to the test device;

[0398] The second difference information sent by the test device is received, where the fourth signal quality is a true value of the signal quality of the first reference signal transmitted through the second beam determined by the test device.

[0399] In some embodiments, the processing module 1301 is further configured to: predict the third signal quality using the first model;

[0400] In some embodiments, the transceiver module 1302 is further configured to: send the third signal quality and the measurement result to the test device;

[0401] The second difference information sent by the test device is received, wherein the fourth signal quality is the signal quality corresponding to the second beam in the measurement result.

[0402] In some embodiments, the transceiver module 1302 is further configured to: send identification information of the first beam and the measurement result to a test device;

[0403] The second difference information sent by the test device is received, wherein the first signal quality is the signal quality corresponding to the first beam in the measurement result, and the fourth signal quality is the signal quality corresponding to the second beam in the measurement result.

[0404] In some embodiments, the processing module 1301 is further configured to: predict the third signal quality using the first model;

[0405] Determine the fourth signal quality according to the measurement result, where the fourth signal quality is the signal quality corresponding to the second beam in the measurement result;

[0406] The transceiver module 1302 is further configured to: send the third signal quality and the fourth signal quality to the test device; and receive the second difference information sent by the test device.

[0407] In some embodiments, the processing module 1301 is further configured to: determine a third signal quality and a fourth signal quality based on the measurement result, where the third signal quality is a measurement result corresponding to the identification information of the first beam in the measurement result, and the fourth signal quality is a signal quality corresponding to the second beam in the measurement result;

[0408] The transceiver module 1302 is further configured to: send the third signal quality and the fourth signal quality to the test device; and receive the second difference information sent by the test device.

[0409] Figure 8A is a schematic diagram of the structure of a communication device 8100 according to an embodiment of the present disclosure. Communication device 8100 can be a network device (e.g., an access network device, a core network device, etc.), a terminal (e.g., a user device, etc.), a chip, a chip system, or a processor that implements any of the above methods as a network device, or a chip, a chip system, or a processor that implements any of the above methods as a terminal. Communication device 8100 can be used to implement the methods described in the above method embodiments. For details, please refer to the description of the above method embodiments.

[0410] As shown in Figure 14, the communication device 8100 includes one or more processors 8101. The processor 8101 can be a general-purpose processor or a dedicated processor, for example, a baseband processor or a central processing unit. The baseband processor can be used to process communication protocols and communication data, and the central processing unit can be used to control the communication device (such as a base station, a baseband chip, a terminal device, a terminal device chip, a DU or a CU, etc.), execute programs, and process program data. The communication device 8100 is used to perform any of the above methods.

[0411] In some embodiments, the communication device 8100 further includes one or more memories 8102 for storing instructions. Optionally, all or part of the memories 8102 may be located outside the communication device 8100.

[0412] In some embodiments, the communication device 8100 further includes one or more transceivers 8103. When the communication device 8100 includes one or more transceivers 8103, the transceiver 8103 performs at least one of the communication steps such as sending and / or receiving in the above method (for example, step S2102 and step S2103, but not limited thereto), and the processor 8101 performs at least one of the other steps (for example, step S2101, but not limited thereto).

[0413] In some embodiments, a transceiver may include a receiver and / or a transmitter. The receiver and transmitter may be separate or integrated. Optionally, the terms transceiver, transceiver unit, transceiver, and transceiver circuit may be used interchangeably; the terms transmitter, transmitting unit, transmitter, and transmitting circuit may be used interchangeably; and the terms receiver, receiving unit, receiver, and receiving circuit may be used interchangeably.

[0414] In some embodiments, the communication device 8100 may include one or more interface circuits 8104. Optionally, the interface circuit 8104 is connected to the memory 8102. The interface circuit 8104 may be configured to receive signals from the memory 8102 or other devices, and may be configured to send signals to the memory 8102 or other devices. For example, the interface circuit 8104 may read instructions stored in the memory 8102 and send the instructions to the processor 8101.

[0415] The communication device 8100 described in the above embodiment may be a network device or a terminal, but the scope of the communication device 8100 described in the present disclosure is not limited thereto, and the structure of the communication device 8100 may not be limited by FIG. 14 . The communication device may be an independent device or may be part of a larger device. For example, the communication device may be: 1) an independent integrated circuit IC, or a chip, or a chip system or subsystem; (2) a collection of one or more ICs, optionally, the above IC collection may also include a storage component for storing data or programs; (3) an ASIC, such as a modem; (4) a module that can be embedded in other devices; (5) a receiver, a terminal device, an intelligent terminal device, a cellular phone, a wireless device, a handheld device, a mobile unit, an in-vehicle device, a network device, a cloud device, an artificial intelligence device, etc.; (8) others, etc. Industrial Applicability

[0416] The performance of the first model can be evaluated using different beams in different application scenarios or at different time points to obtain more accurate evaluation results.

Claims

1. A performance evaluation method for an artificial intelligence prediction model, applied to a terminal, the method comprising: Predicting a first signal quality by using a first model, where the first signal quality is a predicted value of a first reference signal transmitted by a first beam; Acquire first difference information, where the first difference information is used to indicate a difference between the first signal quality and a second signal quality, where the second signal quality is a reference value of a signal quality of the first reference signal; The prediction performance of the first model is determined according to the first difference information.

2. The method of claim 1, wherein: The predicting the first signal quality by using the first model includes: Measuring, in a first time domain unit, a signal quality of the first reference signal transmitted through the second beam to obtain a first measurement value of the first reference signal; Taking a first measurement value of the first reference signal as input, predicting the first signal quality in a second time domain unit by using the first model; The time domain unit is one of: time slot, half time slot, symbol.

3. The method of claim 2, wherein: The method further comprises: The signal quality of the first reference signal transmitted through the first beam in the second time domain unit is measured to obtain the second signal quality.

4. The method of claim 2, wherein: The second signal quality is a true value of the first reference signal transmitted through the first beam in the second time domain unit.

5. The method according to any one of claims 1 to 4, wherein: The method further comprises: obtaining the second signal quality; sending the first signal quality and the second signal quality to a test device; The obtaining of the first difference information includes: The first difference information sent by the test device is received, where the first difference information is determined by the test device according to the first signal quality and the second signal quality.

6. The method according to any one of claims 1 to 4, wherein: The method further comprises: sending the first signal quality to a test device; The obtaining of the first difference information includes: The first difference information sent by the test device is received, where the first difference information is determined by the test device according to the first signal quality and the second signal quality, and the second signal quality is determined by the test device.

7. A performance evaluation method for an artificial intelligence prediction model, applied to a terminal, the method comprising: Predicting a first beam by using a first model, wherein the first beam is a beam with the best signal quality in the first set predicted by the terminal; determining a third signal quality, where the third signal quality is a signal quality of a first reference signal transmitted through the first beam; Acquire second difference information, where the second difference information is used to indicate a difference between the third signal quality and the fourth signal quality, where the fourth signal quality is a reference value of the signal quality of the first reference signal transmitted through the second beam, wherein the second beam is a beam with the best signal quality in the first set measured by the terminal, or the second beam is a best beam determined from the first set based on actual signal quality, or the second beam is a default beam; The prediction performance of the first model is determined according to the second difference information.

8. The method of claim 7, wherein: The determining the third signal quality comprises: The third signal quality is predicted by using the first model.

9. The method of claim 7, wherein: The determining of the third signal quality, the method further comprises: Measuring a signal quality of the first reference signal corresponding to each beam in the first set to obtain a measurement result; According to the identification information of the first beam, it is determined that the third signal quality is a measurement result in the measurement results corresponding to the identification information of the first beam.

10. The method of claim 8, wherein: The method further comprises: sending the third signal quality to a test device; The obtaining of the second difference information includes: The second difference information sent by the test device is received, wherein the fourth signal quality is a true value of the signal quality of the first reference signal transmitted through the second beam determined by the test device.

11. The method of claim 9, wherein: The method further comprises: Predicting the third signal quality by using the first model; sending the third signal quality and the measurement result to a test device; The obtaining of the second difference information includes: The second difference information sent by the test device is received, wherein the fourth signal quality is the signal quality corresponding to the second beam in the measurement result.

12. The method of claim 9, wherein: The method further comprises: Sending identification information of the first beam and the measurement result to a test device; The obtaining of the second difference information includes: The second difference information sent by the test device is received, wherein the first signal quality is the signal quality corresponding to the first beam in the measurement result, and the fourth signal quality is the signal quality corresponding to the second beam in the measurement result.

13. The method of claim 9, wherein: The method further comprises: Predicting the third signal quality by using the first model; Determine the fourth signal quality according to the measurement result, where the fourth signal quality is the signal quality corresponding to the second beam in the measurement result; sending the third signal quality and the fourth signal quality to a test device; The obtaining of the second difference information includes: The second difference information sent by the test device is received.

14. The method of claim 9, wherein: The method further comprises: determining a third signal quality and a fourth signal quality according to the measurement result, wherein the third signal quality is a measurement result corresponding to the identification information of the first beam in the measurement result, and the fourth signal quality is a signal quality corresponding to the second beam in the measurement result; sending the third signal quality and the fourth signal quality to a test device; The obtaining of the second difference information includes: The second difference information sent by the test device is received.

15. A terminal, comprising: The processing module is configured as follows: Predicting a first signal quality by using a first model, where the first signal quality is a predicted value of a first reference signal transmitted by a first beam; Acquire first difference information, where the first difference information is used to indicate a difference between the first signal quality and a second signal quality, where the second signal quality is a reference value of a signal quality of the first reference signal; The prediction performance of the first model is determined according to the first difference information.

16. A terminal, comprising: The processing module is configured as follows: Predicting a first beam by using a first model, wherein the first beam is a beam with the best signal quality in the first set predicted by the terminal; determining a third signal quality, where the third signal quality is a signal quality of a first reference signal transmitted through the first beam; Acquire second difference information, where the second difference information is used to indicate a difference between the third signal quality and the fourth signal quality, where the fourth signal quality is a reference value of the signal quality of the first reference signal transmitted through the second beam, wherein the second beam is a beam with the best signal quality in the first set measured by the terminal, or the second beam is a best beam determined from the first set based on actual signal quality, or the second beam is a default beam; The prediction performance of the first model is determined according to the second difference information.

17. A terminal, comprising: one or more processors; The terminal is used to execute the method according to any one of claims 1 to 6 or 7 to 14.

18. A storage medium storing instructions, which, when executed on a communication device, causes the communication device to execute the method according to any one of claims 1 to 6 or 7 to 14.

Images