Evaluation method, apparatus and storage medium

Abstract

The present disclosure provides an evaluation method, an apparatus, and a storage medium. The method comprises: receiving a predicted value sent by a terminal, wherein the predicted value is used for indicating parameters related to a beam; and determining an evaluation result on the basis of the predicted value and a true value, wherein the evaluation result is used for evaluating the accuracy of the predicted value. The present disclosure can evaluate the accuracy of parameters related to a beam predicted by an AI model, improving the availability of the AI model in a communication system.

Description

Evaluation method and device, storage medium TECHNICAL FIELD

[0001] The present disclosure relates to the field of artificial intelligence, and in particular, to an evaluation method and device, and a storage medium. BACKGROUND

[0002] With the development of artificial intelligence (AI) technology, an AI model can be applied to predict an optimal transmission beam, so that signaling or data is received using a receiving beam corresponding to the predicted optimal transmission beam, thereby improving the communication performance of a communication system.

[0003] SUMMARY

[0004] To improve the accuracy of a prediction result, an evaluation method and device, and a storage medium are provided in embodiments of the present disclosure.

[0005] According to a first aspect of embodiments of the present disclosure, an evaluation method is provided, comprising:

[0006] receiving a prediction value sent by a terminal; wherein the prediction value is used to indicate a parameter related to a beam;

[0007] determining an evaluation result based on the prediction value and a true value, the evaluation result being used to evaluate the accuracy of the prediction value.

[0008] According to a second aspect of embodiments of the present disclosure, an evaluation method is provided, comprising:

[0009] sending a prediction value to a first device; wherein the prediction value is used to indicate a parameter related to a beam;

[0010] wherein the prediction value and a true value are used by the first device to determine an evaluation result, the evaluation result being used to evaluate the accuracy of the prediction value.

[0011] According to a third aspect of embodiments of the present disclosure, a first device is provided, comprising:

[0012] a transceiver module configured to receive a prediction value sent by a terminal; wherein the prediction value is used to indicate a parameter related to a beam;

[0013] a processing module configured to determine an evaluation result based on the prediction value and a true value, the evaluation result being used to evaluate the accuracy of the prediction value.

[0014] According to a fourth aspect of embodiments of the present disclosure, a terminal is provided, comprising:

[0015] a transceiver module configured to send a prediction value to a first device; wherein the prediction value is used to indicate a parameter related to a beam;

[0016] The predicted value is used by the first device to determine an evaluation result, and the evaluation result is used to evaluate accuracy of the predicted value.

[0017] According to a fifth aspect of embodiments of the present disclosure, a communication device is provided, comprising

[0018] a processing module configured to determine an evaluation result based on a predicted value and a true value, wherein the predicted value is used to indicate a parameter related to a beam, and the evaluation result is used to evaluate accuracy of the predicted value.

[0019] According to a sixth aspect of embodiments of the present disclosure, a communication device is provided, wherein an artificial intelligence (AI) model is deployed on the device, and the device comprises:

[0020] a processing module configured to obtain a predicted value output by the AI model, wherein the predicted value is used to indicate a parameter related to a beam.

[0021] The processing module is further configured to train the AI model based on the predicted value and a true value until a stop training condition is met, to obtain a trained AI model, wherein the stop training condition comprises that accuracy of the predicted value output by the AI model reaches a first accuracy.

[0022] According to a seventh aspect of embodiments of the present disclosure, a first device is provided, comprising:

[0023] one or more processors;

[0024] The processor is configured to perform the evaluation method of any one of the first aspect.

[0025] According to an eighth aspect of embodiments of the present disclosure, a terminal is provided, comprising:

[0026] one or more processors;

[0027] The processor is configured to perform the evaluation method of the second aspect.

[0028] According to a ninth aspect of embodiments of the present disclosure, a communication system is provided, comprising:

[0029] a first device configured to implement the evaluation method of any one of the first aspect;

[0030] a terminal configured to implement the evaluation method of the second aspect.

[0031] According to a tenth aspect of embodiments of the present disclosure, a storage medium is provided, wherein the storage medium stores instructions, and when the instructions run on a communication device, the communication device is caused to perform the evaluation method of any one of the first aspect or the second aspect.

[0032] According to a first aspect of the embodiments of the present disclosure, there is provided an evaluation method, comprising: receiving a predicted value sent by a terminal, the predicted value being used to indicate a parameter related to a beam; and determining an evaluation result based on the predicted value and a true value, wherein the evaluation result is used to evaluate accuracy of the predicted value.

[0033] In the embodiments of the present disclosure, the first device can receive a predicted value sent by a terminal, the predicted value can be used to indicate a parameter related to a beam, and based on the predicted value and a true value, an evaluation result can be determined, wherein the evaluation result is used to evaluate the accuracy of the predicted value. The present disclosure can evaluate the accuracy of the AI model in predicting the beam-related parameter, and improve the usability of the AI model in the communication system.

[0034] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0035] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0036] FIG. 1 is one exemplary schematic diagram of an architecture of a communication system according to embodiments of the present disclosure.

[0037] FIG. 2A is one exemplary interaction schematic diagram of an evaluation method according to embodiments of the present disclosure.

[0038] FIG. 2B is one exemplary interaction schematic diagram of an evaluation method according to embodiments of the present disclosure.

[0039] FIG. 3A is one exemplary interaction schematic diagram of an evaluation method according to embodiments of the present disclosure.

[0040] FIG. 3B is one exemplary interaction schematic diagram of an evaluation method according to embodiments of the present disclosure.

[0041] FIG. 4A is one exemplary schematic diagram of a scenario with only NLOS according to embodiments of the present disclosure.

[0042] FIG. 4B is one exemplary schematic diagram of a scenario with only LOS according to embodiments of the present disclosure.

[0043] FIG. 4C is one exemplary schematic diagram of a scenario with coexistence of LOS and NLOS according to embodiments of the present disclosure.

[0044] FIG. 4D is one exemplary schematic diagram of selecting the best transmission beam according to embodiments of the present disclosure.

[0045] FIG. 5A is one exemplary block diagram of a first device according to embodiments of the present disclosure.

[0046] FIG. 5B is one exemplary block diagram of a terminal according to embodiments of the present disclosure.

[0047] FIG. 5C is one exemplary block diagram of a communication device according to embodiments of the present disclosure.

[0048] FIG. 5D is one exemplary block diagram of a communication device according to embodiments of the present disclosure.

[0049] FIG. 6A is one exemplary interaction diagram of a communication device according to embodiments of the present disclosure.

[0050] FIG. 6B is one exemplary interaction diagram of a chip according to embodiments of the present disclosure. DETAILED DESCRIPTION

[0051] The exemplary embodiments will be described in detail herein with reference to the attached drawings. In the following description, the same numbers are used to indicate the same or similar elements, unless otherwise represented. The embodiments described in the following exemplary embodiments do not represent all the implementations consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0052] Embodiments of the present disclosure provide an evaluation method and device, and a storage medium.

[0053] In a first aspect, embodiments of the present disclosure provide an evaluation method, comprising:

[0054] receiving a predicted value sent by a terminal; wherein the predicted value is used to indicate a parameter related to a beam;

[0055] determining an evaluation result based on the predicted value and a true value, wherein the evaluation result is used to evaluate the accuracy of the predicted value.

[0056] In the above embodiments, the first device can receive a predicted value sent by a terminal, which can be used to indicate a parameter related to a beam. Based on the predicted value and a true value, an evaluation result can be determined, wherein the evaluation result is used to evaluate the accuracy of the predicted value. The present disclosure can evaluate the accuracy of the AI model in predicting the beam-related parameter, thereby improving the usability of the AI model in the communication system.

[0057] In some embodiments in combination with the first aspect, in some embodiments, the predicted value comprises:

[0058] a predicted index value, wherein the predicted index value is an index value of a best transmission beam predicted by the terminal;

[0059] The true value comprises a first set, and the first set comprises any of the following:

[0060] a first index value; wherein the first index value is an index value of the best transmission beam;

[0061] a first index value and one or more second index values; wherein the first index value is an index value of the best transmission beam.

[0062] In the above embodiment, when the predicted value is a predicted index value, the true value can be a first set, and the first set can include the first index value or the first index value and the second index value, which explicitly determines the specific determination manner of the true value, and has high usability.

[0063] In combination with some embodiments of the first aspect, in some embodiments, the method further includes:

[0064] determining the transmission beam with the largest RSRP value as the best transmission beam, and determining the index value of the best transmission beam as the first index value; or

[0065] determining the best transmission beam based on the first position information of the transmission antenna, the second position information of the reception antenna, and the first spatial configuration information of the transmission antenna, and determining the index value of the best transmission beam as the first index value.

[0066] In the above embodiment, the first device can determine the index value of the best transmission beam in the above manner, which is simple and has high usability.

[0067] In combination with some embodiments of the first aspect, in some embodiments, the method further includes:

[0068] from the transmission beams other than the best transmission beam, selecting one or more transmission beams in descending order of RSRP values of each transmission beam, and determining the index value of the selected transmission beam as the second index value; or

[0069] from the transmission beams other than the best transmission beam, selecting one or more transmission beams adjacent to the best transmission beam, and determining the index value of the selected transmission beam as the second index value.

[0070] In the above embodiment, the second index value can be determined in the above manner, so that the first set is obtained, which has high usability.

[0071] In combination with some embodiments of the first aspect, in some embodiments, the accuracy is equal to a first quotient value of a first number and a first total number, the first number is a number of times that the predicted index value belongs to the first set, and the first total number is a total number of times that the predicted index value sent by the terminal is received.

[0072] In the above embodiments, the evaluation result can be determined in the above manner, the accuracy of the predicted value is quantified, and the usability of the AI model in the communication system is improved.

[0073] In some embodiments of the first aspect, in some embodiments, the predicted value comprises:

[0074] a predicted reference signal received power (RSRP) value, the predicted RSRP value being an RSRP value of a best beam predicted by the terminal;

[0075] The true value comprises a first RSRP value interval.

[0076] In the above embodiments, when the predicted value is a predicted RSRP value, the true value can comprise a first RSRP value interval.

[0077] In some embodiments of the first aspect, in some embodiments, the first RSRP value interval is associated with at least one of:

[0078] a first RSRP value;

[0079] a second RSRP value, the second RSRP value being a measured RSRP value of the best transmit beam at the receiving end;

[0080] a third RSRP value, the third RSRP value being an RSRP value of the best transmit beam with beamforming gain.

[0081] In the above embodiments, the first RSRP value interval can be associated with at least one of the above, which is simple and has high usability.

[0082] In some embodiments of the first aspect, in some embodiments, the first RSRP value interval comprises at least one of:

[0083] a minimum value of the first RSRP value interval is equal to a first value, and a maximum value of the first RSRP value interval is equal to a second value; wherein the first value is a difference between the first RSRP value and x, and the second value is a sum of the first RSRP value and x;

[0084] a minimum value of the first RSRP value interval is equal to a third value, and a maximum value of the first RSRP value interval is equal to a fourth value; wherein the third value is a difference between the second RSRP value and x, and the fourth value is a sum of the second RSRP value and x;

[0085] a minimum value of the first RSRP value interval is equal to a third value, and a maximum value of the first RSRP value interval is equal to a second value; wherein the third value is a difference between the second RSRP value and x, and the second value is a sum of the first RSRP value and x;

[0086] a minimum value of the first RSRP value interval is equal to a fifth value, and a maximum value of the first RSRP value interval is equal to a sixth value; wherein the fifth value is a difference value of the third RSRP value and x, and the sixth value is a sum value of the third RSRP value and x;

[0087] a minimum value of the first RSRP value interval is equal to a fifth value, and a maximum value of the first RSRP value interval is equal to a sixth value; wherein the fifth value is a difference value of the third RSRP value and x, and the sixth value is a sum value of the third RSRP value and x;

[0088] a minimum value of the first RSRP value interval is equal to a fifth value, and a maximum value of the first RSRP value interval is equal to a sixth value; wherein the fifth value is a difference value of the third RSRP value and x, and the sixth value is a sum value of the third RSRP value and x;

[0089] In the above embodiments, the first RSRP value interval can be determined in the above manner, which improves the reliability of the evaluation result.

[0090] In combination with some embodiments of the first aspect, in some embodiments, the x is a non-negative number, and / or the x is determined based on the accuracy of the beam measurement.

[0091] In the above embodiments, the value of x can be determined in the above manner, which has high availability.

[0092] In combination with some embodiments of the first aspect, in some embodiments, the accuracy is equal to a second quotient value of a second number and a second total number; wherein the second number is a number of times that the predicted RSRP value belongs to a second RSRP value interval, the second RSRP value interval is determined based on the first RSRP value interval and a preset value, and the second total number is a total number of times that the predicted RSRP value is received from the terminal.

[0093] In the above embodiments, the evaluation result can be determined in the above manner, which quantifies the accuracy of the predicted value and improves the availability of the AI model in the communication system.

[0094] In a second aspect, the embodiments of the present disclosure provide an evaluation method, characterized in that, comprising:

[0095] predicting a value for a first device; wherein the predicted value is used to indicate a parameter related to a beam;

[0096] wherein the predicted value and the true value are used by the first device to determine an evaluation result, and the evaluation result is used to evaluate the accuracy of the predicted value.

[0097] In the above embodiments, the terminal can send the predicted value to the first device, and the first device can evaluate the accuracy of the predicted value based on the predicted value and the true value, thereby improving the usability of the AI model in the communication system.

[0098] In some embodiments in combination with the second aspect, in some embodiments, the predicted value comprises at least one of:

[0099] a predicted index value, the predicted index value being an index value of the best transmission beam predicted by the terminal;

[0100] a predicted reference signal received power (RSRP) value, the predicted RSRP value being an RSRP value of the best transmission beam predicted by the terminal.

[0101] In a third aspect, the embodiments of the present disclosure provide a first device, comprising:

[0102] a transceiver module configured to receive a predicted value sent by a terminal; wherein the predicted value is used to indicate a parameter related to a beam;

[0103] a processing module configured to determine an evaluation result based on the predicted value and a true value, the evaluation result being used to evaluate the accuracy of the predicted value.

[0104] In a fourth aspect, the embodiments of the present disclosure provide a terminal, comprising:

[0105] a transceiver module configured to send a predicted value to a first device; wherein the predicted value is used to indicate a parameter related to a beam;

[0106] wherein the predicted value is used by the first device to determine an evaluation result, the evaluation result being used to evaluate the accuracy of the predicted value.

[0107] In a fifth aspect, the embodiments of the present disclosure provide a communication device, comprising:

[0108] a processing module configured to determine an evaluation result based on a predicted value and a true value; wherein the predicted value is used to indicate a parameter related to a beam; and the evaluation result is used to evaluate the accuracy of the predicted value.

[0109] In some embodiments in combination with the fifth aspect, in some embodiments, the predicted value comprises:

[0110] a predicted index value, the predicted index value being an index value of the best transmission beam predicted by the terminal;

[0111] the true value comprises a first set, the first set comprising any of:

[0112] a first index value, wherein the first index value is an index value of the best transmission beam;

[0113] a first index value and one or more second index values; wherein the first index value is an index value of a best transmission beam.

[0114] In some embodiments combined with the fifth aspect, in some embodiments, the predicted value comprises:

[0115] a predicted reference signal received power, RSRP, value, the predicted RSRP value being an RSRP value of a best beam predicted by the terminal;

[0116] the true value comprises a first RSRP value interval.

[0117] In some embodiments combined with the fifth aspect, in some embodiments, the device is any one of:

[0118] a terminal;

[0119] a network device.

[0120] In a sixth aspect, the embodiments of the present disclosure provide a communication device, an artificial intelligence, AI, model being deployed on the device, comprising:

[0121] a processing module configured to obtain a predicted value output by the AI model; wherein the predicted value is used to indicate a parameter related to a beam;

[0122] the processing module is further configured to train the AI model based on the predicted value and a true value until a stop training condition is met, to obtain a trained AI model; wherein the stop training condition comprises that an accuracy of the predicted value output by the AI model reaches a first accuracy.

[0123] In some embodiments combined with the sixth aspect, in some embodiments, the input value of the AI model comprises:

[0124] one or more measured RSRP values of transmission beams at a receiving end.

[0125] In some embodiments combined with the sixth aspect, in some embodiments, the predicted value comprises:

[0126] a predicted index value, the predicted index value being an index value of a best transmission beam predicted by the terminal;

[0127] the true value comprises a first set, the first set comprising any one of:

[0128] a first index value; wherein the first index value is an index value of a best transmission beam.

[0129] a first index value and one or more second index values; wherein the first index value is an index value of a best transmission beam.

[0130] In some embodiments combining with the sixth aspect, in some embodiments, the predicted value comprises:

[0131] a predicted reference signal received power, RSRP, value, the predicted RSRP value being an RSRP value of a best beam predicted by the terminal;

[0132] the true value comprises a first RSRP value interval.

[0133] In some embodiments combining with the sixth aspect, in some embodiments, the device is any one of the following:

[0134] a terminal;

[0135] a network device.

[0136] In a seventh aspect, the embodiments of the present disclosure provide a first device, comprising:

[0137] one or more processors;

[0138] wherein the processor is configured to perform the evaluation method according to any one of the first aspect.

[0139] In an eighth aspect, the embodiments of the present disclosure provide a terminal, comprising:

[0140] one or more processors;

[0141] wherein the processor is configured to perform the evaluation method according to the second aspect.

[0142] In a ninth aspect, the embodiments of the present disclosure provide a communication system, comprising:

[0143] a first device configured to implement the evaluation method according to any one of the first aspect;

[0144] a terminal configured to implement the evaluation method according to the second aspect.

[0145] In a tenth aspect, the embodiments of the present disclosure provide a storage medium, the storage medium storing instructions, when the instructions are run on a communication device, causing the communication device to perform the evaluation method according to any one of the first aspect or the second aspect.

[0146] In an eleventh aspect, the embodiments of the present disclosure provide a computer program product, comprising a computer program configured to implement the evaluation method according to any one of the first aspect or the second aspect when executed by a processor.

[0147] It can be understood that the first device, the terminal, the communication device, the communication system, the storage medium, and the computer program described above are used to execute the method proposed in the embodiments of the present disclosure. Therefore, the beneficial effects achieved thereby can refer to the beneficial effects in the corresponding method, which will not be described here.

[0148] The embodiments of the present disclosure propose an evaluation method and device, and a storage medium. In some embodiments, the evaluation method and the terms such as the evaluation method and processing method of the beam prediction result can be replaced with each other, the evaluation device and the terms such as the evaluation device and processing device of the beam prediction result can be replaced with each other, and the terms such as the communication system and processing system can be replaced with each other.

[0149] The embodiments of the present disclosure are not exhaustive, but only illustrate some embodiments, and are not specific limitations on the protection scope of the present disclosure. In the case of no contradiction, each step in an embodiment can be implemented as an independent embodiment, and the steps can be combined arbitrarily, for example, the scheme after removing some steps in an embodiment can also be implemented as an independent embodiment, and the order of the steps in an embodiment can be exchanged arbitrarily, in addition, the optional implementation manners in an embodiment can be combined arbitrarily; in addition, the embodiments can be combined arbitrarily, for example, the steps of different embodiments or part or all of the steps of different embodiments can be combined arbitrarily, an embodiment can be combined with the optional implementation manners of other embodiments.

[0150] In the embodiments of the present disclosure, the terms and / or descriptions between the embodiments are consistent and can be referred to each other if there is no special description and logical conflict, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0151] The terms used in the embodiments of the present disclosure are only for the purpose of describing specific embodiments, and not as a limitation on the present disclosure.

[0152] In the embodiments of the present disclosure, unless otherwise specified, the elements expressed in singular form, such as "one", "a", "the", "above", "the", "the", "this", etc., can represent "one and only one", or "one or more", "at least one", etc. For example, in the case of using articles such as "a", "an", "the" in English, the noun after the article can be understood as singular expression, or can be understood as plural expression.

[0153] In the embodiments of the present disclosure, "a plurality of" means two or more.

[0154] In some embodiments, the terms "at least one of," "one or more of," "a plurality of," "multiple," and the like can be used interchangeably.

[0155] In some embodiments, the recitations "at least one of A, B," "A and / or B," "in one case A, in another case B," "in response to a case A, in response to a case B," and the like can include the following technical solutions according to the case: in some embodiments A (A is executed regardless of B); in some embodiments B (B is executed regardless of A); in some embodiments, A and B are selectively executed (A and B are selectively executed); in some embodiments, A and B (A and B are executed). When there are more branches such as A, B, C, and the like, the above is similar.

[0156] In some embodiments, the recitations "A or B" and the like can include the following technical solutions according to the case: in some embodiments A (A is executed regardless of B); in some embodiments B (B is executed regardless of A); in some embodiments, A and B are selectively executed (A and B are selectively executed). When there are more branches such as A, B, C, and the like, the above is similar.

[0157] The prefix words "first", "second", and the like in the embodiments of the present disclosure are merely used to distinguish different description objects, and do not constitute a limitation on the position, order, priority, quantity, or content of the description objects. The description of the description objects should refer to the description in the context of the claims or embodiments, and should not constitute an additional limitation because of the use of the prefix words. For example, the description objects are "fields", and the ordinal words before "fields" in "first field" and "second field" do not limit the position or order between "fields", and "first" and "second" do not limit whether the "fields" modified thereby are in the same message or not, nor limit the order of "first field" and "second field". For another example, the description objects are "levels", and the ordinal words before "levels" in "first level" and "second level" do not limit the priority between "levels". For another example, the quantity of the description objects is not limited by the ordinal words, and can be one or more. For example, "first device", wherein the quantity of "devices" can be one or more. In addition, the objects modified by different prefix words can be the same or different, for example, the description objects are "devices", and "first device" and "second device" can be the same device or different devices, and the types thereof can be the same or different; for another example, the description objects are "information", and "first information" and "second information" can be the same information or different information, and the content thereof can be the same or different.

[0158] In some embodiments, "comprising", "containing", "instructed to", "carrying" can be interpreted as directly carrying A, or indirectly indicating A.

[0159] In some embodiments, the apparatus and device can be interpreted as physical or virtual, and the name thereof is not limited to the name described in the embodiments, and in some cases can also be understood as "equipment", "device", "circuit", "network element", "node", "function", "unit", "section", "system", "network", "entity", "subject", etc.

[0160] In some embodiments, the data, information, etc. can be obtained in accordance with the laws and regulations of the country where the data is located.

[0161] In some embodiments, the data, information, etc. can be obtained after obtaining the consent of the user.

[0162] In addition, each element, each row, or each column in the table of the embodiments of the present disclosure can be implemented as an independent embodiment, and any combination of any element, any row, or any column can also be implemented as an independent embodiment.

[0163] FIG. 1 is a schematic diagram of the architecture of a communication system according to an embodiment of the present disclosure.

[0164] As shown in FIG. 1, the communication system 100 includes a terminal 101, a first device 102.

[0165] In some embodiments, the terminal 101 includes at least one of a mobile phone, a wearable device, an Internet of Things device, a communication-enabled car, a smart car, a tablet computer (Pad), a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal device in industrial control, a wireless terminal device in self-driving, a wireless terminal device in remote medical surgery, a wireless terminal device in smart grid, a wireless terminal device in transportation safety, a wireless terminal device in smart city, a wireless terminal device in smart home, etc., but is not limited thereto.

[0166] In some embodiments, the AI model deployed on the terminal 101 has input values which are not limited and can be index values of multiple transmission beams and / or multiple reception beams, and / or measurement values, wherein the measurement values include but are not limited to Reference Signal Receiving Power (RSRP) values of the transmission beams and / or the reception beams, angle values (for example, the azimuth angle value of the receiving end relative to the plane where the transmitting end antenna is located), etc. The output values of the AI model can include but are not limited to prediction values.

[0167] Exemplarily, the prediction values can include but are not limited to at least one of the following:

[0168] a predicted index value;

[0169] a predicted Reference Signal Receiving Power (RSRP) value.

[0170] The predicted index value can be an index value of the best transmission beam predicted by the terminal 101.

[0171] The predicted RSRP value can be an RSRP value of the best transmission beam predicted by the terminal 101.

[0172] In some embodiments, the first device 102 can include but is not limited to at least one of a network device, such as an access network device, a core network device, etc.

[0173] In some embodiments, the above-mentioned access network device is, for example, a node or device for accessing terminals to a wireless network, and the access network device can include at least one of an evolved NodeB (eNB), a next generation eNB (ng-eNB), a next generation NodeB (gNB), a node B (NB), a home node B (HNB), a home evolved node B (HeNB), a wireless backhaul device, a radio network controller (RNC), a base station controller (BSC), a base transceiver station (BTS), a base band unit (BBU), a mobile switching center, a base station in a 6G communication system, an open base station (Open RAN), a cloud base station (Cloud RAN), a base station in other communication systems, an access node in a Wi-Fi system, but is not limited thereto.

[0174] In some embodiments, the above-mentioned access network device 102-1 can be composed of a central unit (CU) and a distributed unit (DU), wherein the CU can also be referred to as a control unit (control unit), and the CU-DU structure can split the protocol layers of the access network device, and some of the protocol layers are controlled by the CU, and the rest or all of the protocol layers are distributed in the DU and controlled by the CU, but is not limited thereto.

[0175] In some embodiments, the above-mentioned core network device can be one device, including one or more network elements, etc., and can also be multiple devices or device groups. The network element can be virtual or physical. The core network includes, for example, at least one of an evolved packet core (EPC), a 5G core network (5GCN), and a next generation core (NGC).

[0176] In some embodiments, the first device 102 can be a specific device for determining the evaluation result. The evaluation result is used to evaluate the accuracy of the predicted value. The present disclosure does not limit this.

[0177] In some embodiments, the first device 102 can be a specific device independent of the network device, and the communication system 100 can further include a network device 103. The network device 103 can serve as a sending end, and the terminal 101 can serve as a receiving end. The AI model deployed on the terminal 101 can output the predicted value described above and provide the predicted value to the first device 102, and the first device 102 can determine the accuracy of the predicted value.

[0178] In some embodiments, the technical solutions of the present disclosure can be applied to the Open RAN architecture, at which time the interfaces between the access network devices or within the access network devices involved in the embodiments of the present disclosure can become internal interfaces of the Open RAN, and the processes and information interactions between these internal interfaces can be implemented through software or programs.

[0179] It can be understood that the communication system described in the embodiments of the present disclosure is for more clearly illustrating the technical solutions of the embodiments of the present disclosure, and does not constitute a limitation on the technical solutions proposed in the embodiments of the present disclosure. It can be known by those skilled in the art that, as the system architecture evolves and new business scenarios appear, the technical solutions proposed in the embodiments of the present disclosure are also applicable to similar technical problems.

[0180] The following embodiments of the present disclosure can be applied to the communication system 100 shown in FIG. 1 or part of the subject, but are not limited thereto. The subjects shown in FIG. 1 are exemplary, and the communication system can include all or part of the subjects in FIG. 1, or other subjects other than FIG. 1. The number and form of each subject is arbitrary, each subject can be real or virtual, the connection relationship between each subject is exemplary, each subject can not be connected or can be connected, the connection can be in any way, can be direct connection or indirect connection, can be wired connection or wireless connection.

[0181] Embodiments of the present disclosure can be applied to Long Term Evolution (LTE), LTE-Advanced (LTE-A), LTE-Beyond (LTE-B), SUPER 3G, IMT-Advanced, 4th generation mobile communication system (4G), 5th generation mobile communication system (5G), 5G new radio (NR), 6th generation mobile communication system (6G), Future Radio Access (FRA), New-Radio Access Technology (RAT), New Radio (NR), New radio access (NX), Future generation radio access (FX), Global System for Mobile communications (GSM (registered trademark)), CDMA2000, Ultra Mobile Broadband (UMB), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.20, Ultra-WideBand (UWB), Bluetooth (Bluetooth (registered trademark)), Public Land Mobile Network (PLMN) network, systems using other communication methods, next-generation systems expanded based on them, and the like. In addition, a plurality of systems can be combined (for example, a combination of LTE or LTE-A and 5G, and the like).

[0182] In some embodiments, an AI-based Layer 1-Reference Signal Receiving Power (L1-RSRP) report can predict an optimal transmission (TX) beam index based on some measurement values.

[0183] The index value of the optimal transmission beam predicted by the AI model deployed on the terminal can be a candidate option for L1-RSRP reporting. However, in the conventional non-artificial intelligence beam reporting, the prediction of the beam-related parameters is not involved, and therefore there is no scheme for evaluating the predicted value.

[0184] To evaluate the accuracy of the predicted value, the present disclosure provides the following evaluation method and device, and storage medium.

[0185] FIG. 2A is an interaction schematic diagram of an evaluation method according to an embodiment of the present disclosure. As shown in FIG. 2A, the embodiment of the present disclosure relates to an evaluation method, and the method comprises:

[0186] In step S2100, the terminal 101 obtains a measurement value.

[0187] In some embodiments, the network device 103 can act as a sending end, and the terminal 101 can act as a receiving end. The terminal 101 can measure the parameters of the received beam to obtain the measurement value.

[0188] In one example, the measurement value can include but is not limited to the RSRP value of one or more transmission beams at the receiving end (i.e., the terminal 101).

[0189] In one example, the measurement value can include but is not limited to the index value of one or more transmission beams and / or the index value of one or more reception beams, wherein the terminal 101 can obtain the index value through but not limited to physical layer signaling. Of course, other signaling can also be used, such as radio resource control signaling, media access control unit, etc.

[0190] In step S2101, the terminal 101 sends the predicted value to the first device 102.

[0191] In some embodiments, the first device 102 can be a network device.

[0192] In some embodiments, the first device 102 can be a specific device for evaluating the accuracy of the predicted value output by the AI model deployed on the terminal 101.

[0193] In some embodiments, the terminal 101 obtains the predicted value through a pre-trained AI model.

[0194] In some embodiments, the input value of the AI model can include but is not limited to the above-mentioned measurement value, including at least one of the following:

[0195] The index value of the transmission (TX) beam;

[0196] The index value of the reception (RX) beam;

[0197] The RSRP value of the transmission beam measured at the receiving end.

[0198] The above is only an example, and the measurement value can also include other parameters, which are not limited in the present disclosure.

[0199] In one example, the transmission beam can refer to the beam used by the network device as the transmitting end.

[0200] The receiving beam can refer to the beam used by the terminal 101 as the receiving end.

[0201] In some embodiments, the output value of the AI model can be a predicted value.

[0202] For example, the predicted value can include a predicted index value, which is the index value of the best transmission beam predicted by the terminal 101.

[0203] For example, the predicted value can include a predicted RSRP value, which is the RSRP value of the best transmission beam predicted by the terminal 101.

[0204] For example, the predicted value includes a predicted index value and a predicted RSRP value. Alternatively, the predicted value can also include other values, which are not limited in the present disclosure.

[0205] In some embodiments, the first device 102 receives the predicted value.

[0206] In some embodiments, step S2101 can not be performed, and the first device 102 can obtain a manually input predicted value.

[0207] In step S2102, the first device 102 determines an evaluation result based on the predicted value and the true value.

[0208] In some embodiments, the first device 102 can first determine the true value.

[0209] In one example, the above true value can be manually set.

[0210] In one example, the above true value can be obtained by the first device 102 from other devices, where the other devices can be devices in the same area or within a distance less than a preset value from the terminal 101.

[0211] In one example, when the predicted value includes a predicted index value, the beam true value can be an index true value of the ideal best transmission beam, or the beam true value can be an index true value of the measured best transmission beam.

[0212] It should be noted that the ideal optimal transmission beam can be understood as a transmission beam that theoretically produces the maximum RSRP value on all transmission beams and / or reception beams. Alternatively, the ideal optimal transmission beam can be understood as a transmission beam that theoretically produces the maximum RSRP value among all transmission beams corresponding to a specific reception beam. The specific reception beam is one or more of all reception beams, i.e., the specific reception beam is a subset of all reception beams.

[0213] The measured optimal transmission beam can be understood as a transmission beam that actually measures the maximum RSRP value on all transmission beams and / or reception beams.

[0214] Exemplarily, when the true value adopts the index value of the ideal optimal transmission beam, and the ideal optimal transmission beam is for a specific reception beam, the following cases can exist:

[0215] Case 1: The index value of the ideal optimal transmission beam in a non-line-of-sight (NLOS) scenario.

[0216] The NLOS scenario refers to a scenario in which the signal transmitted by the sending end experiences one or more cluster reflections to form different paths to the receiving end (i.e., the terminal 101), as shown in FIG. 4A. Each cluster can be composed of one or more network devices.

[0217] In the NLOS scenario, the ideal optimal transmission beam for a specific reception beam is variable.

[0218] For example, in FIG. 4A, for the reception beam RX#1, the corresponding ideal optimal transmission beam should be TX#1, and for the reception beam RX#2, the corresponding ideal optimal transmission beam should be TX#2, i.e., if the index value of the reception beam changes, the index value of the ideal optimal transmission beam also changes.

[0219] In the embodiments of the present disclosure, the index value of the ideal optimal transmission beam corresponding to the specific reception beam depends on the cluster position and the direction of the reception beam.

[0220] Since the information of the reception beam depends on the implementation of the terminal 101, the first device 102 does not know the information of the reception beam, and therefore cannot determine the index value of the ideal optimal transmission beam.

[0221] If the ideal optimal transmission beam adopts a transmission beam that theoretically produces the maximum RSRP value on all transmission beams and / or reception beams, the ideal optimal transmission beam can be selected by scanning all reception beams, and the index value of the ideal optimal transmission beam is the theoretically global optimal true value.

[0222] But in practical application, considering that the terminal 101 can only perform a limited number of receive beam scanning, for example, can only scan 8 receive beams, which depends on the different receive beam angle set corresponding to the implementation of the terminal 101. Among them, the different receive beam angle set can correspond to the index true value of the ideal best transmit beam. In addition, the first device 102 cannot determine whether the cluster exists and the specific location of the cluster, and cannot determine the ideal best transmit beam index true value.

[0223] Therefore, in the multipath case, the first device 102 cannot determine the index value of the ideal best transmit beam. Alternatively, a set composed of index values can be provided, which can include the index value of the ideal best transmit beam, and can include the index value of other alternative transmit beams.

[0224] Exemplarily, the true value can include a first set composed of a first index value and one or more second index values; wherein the first index value is the index value of the ideal best transmit beam.

[0225] Exemplarily, the first device 102 can determine the transmit beam with the largest RSRP value as the ideal best transmit beam, and determine the index value of the ideal best transmit beam as the first index value.

[0226] Further, the first device 102 can select one or more transmit beams in other transmit beams in descending order of the RSRP value of each transmit beam, and determine the index value of the selected transmit beam as the second index value.

[0227] The first index value and the second index value together constitute the first set.

[0228] For example, the index value of the transmit beam with the largest RSRP value is n, that is, the first index value is n, and the index values of the two transmit beams selected in descending order of the RSRP value in other transmit beams are m and s, that is, the second index values are m and n, and the first set includes {m, n, s}.

[0229] Exemplarily, the first device 102 can determine the transmit beam with the largest RSRP value as the ideal best transmit beam, and determine the index value of the ideal best transmit beam as the first index value.

[0230] Further, considering that the ideal best transmit beam may have an angle deviation, therefore, the first device 102 can determine the index value of one or more transmit beams near the ideal best transmit beam as the second index value.

[0231] The first index value and the second index value jointly form a first set.

[0232] For example, the first index value is n, the second index value is n-1, n+1, etc., and the first set includes {n-1, n, n+1}.

[0233] Correspondingly, the first device 102 can determine an evaluation result based on the predicted index value sent by the terminal 101, where the evaluation result can be used to indicate the accuracy of the predicted index value.

[0234] For example, the accuracy of the predicted index value = the first number of times / the first total number of times, where the first number of times is the number of times that the predicted index value belongs to the first set, and the first total number of times is the total number of times that the terminal 101 sends the predicted index value.

[0235] For example, assuming that the total number of times that the terminal 101 sends the predicted index value is 100 times, i.e., the first total number of times is 100, and among them, 98 times of the predicted index value belongs to the first set, and the other 2 times of the predicted index value does not belong to the first set, then the accuracy of the predicted index value = 98%.

[0236] Case 2: The index value of the ideal best transmission beam in the line of sight (LOS) scenario or the NLOS+LOS scenario.

[0237] The LOS scenario refers to a scenario in which the signal sent by the sending end can directly reach the receiving end (i.e., the terminal 101) without passing through a cluster, as shown in FIG. 4B.

[0238] The LOS+NLOS scenario refers to a scenario in which two paths exist simultaneously, as shown in FIG. 4C.

[0239] It can be understood that, regardless of the scenario shown in FIG. 4B or FIG. 4C, the index value of the ideal best transmission beam is unchanged and depends on the first position information of the sending antenna, the second position information of the receiving antenna, and the first spatial configuration information of the sending antenna. Specifically, based on the first position information, the second position information, and the first spatial configuration information, an angle value of a straight-line path from the sending end to the receiving end relative to the antenna plane of the sending end can be determined, for example, an angle value a of the straight-line path relative to the y-axis of the antenna plane and / or an angle value b of the straight-line path relative to the x-axis of the antenna plane, as shown in FIG. 4D.

[0240] At this time, the first set can only include the first index value. That is, the first device 102 can determine the optimal transmission beam based on the first position information of the transmission antenna, the second position information of the reception antenna, and the first spatial configuration information of the transmission antenna, and determine the index value of the optimal transmission beam as the first index value. Further, the first device 102 can determine the accuracy = the first number of times / the first total number of times, where the first number of times is the number of times that the predicted index value belongs to the first set, and the first total number of times is the total number of times that the terminal 101 sends the predicted index value.

[0241] For example, assuming that the index value of the ideal optimal transmission beam is k, and the first set is {k}, if the total number of times that the terminal 101 sends the predicted index value is 100, and 80 times of the predicted index value is k, the first device 102 can determine the accuracy of the predicted value to be 80%.

[0242] Of course, the first set can also include the second index value, and the second index value can be determined in a manner similar to the foregoing embodiments. The first device 102 can determine the accuracy = the first number of times / the first total number of times, and the specific implementation process is not described herein.

[0243] In one example, the true value can directly adopt the index value of the measured optimal transmission beam.

[0244] Correspondingly, the first set can include the index value of the measured optimal transmission beam.

[0245] The first device 102 can determine the accuracy = the first number of times / the first total number of times, where the first number of times is the number of times that the predicted index value belongs to the first set, and the first total number of times is the total number of times that the terminal 101 sends the predicted index value.

[0246] For example, the index value of the measured optimal transmission beam is k', and the first set is {k'}, if the total number of times that the terminal 101 sends the predicted index value is 100, and 85 times of the predicted index value is k', the first device 102 can determine the accuracy of the predicted value to be 85%.

[0247] In one example, when the predicted value includes a predicted RSRP value, the true value can include a first RSRP value interval.

[0248] Exemplarily, the RSRP value involved in the present disclosure will be introduced first.

[0249] The first RSRP value is the ideal RSRP value of the ideal optimal transmission beam at the receiving end (i.e., the terminal 101). The first RSRP value can be represented as P1.

[0250] The second RSRP value is the measured RSRP value of the measured optimal transmission beam at the receiving end (i.e., the terminal 101). The second RSRP value can be represented as P2.

[0251] The third RSRP value is an RSRP value of the best transmission beam with beamforming gain. Exemplarily, the third RSRP value can be denoted as P3. P3 can be calculated by using the following formula 1 or 2: P3 = P0 + G0 Formula 1

[0252] wherein P0 is the first power value, the first power value is a transmission power value of the ideal best reception beam. G0 is the first gain value of beamforming. P3 ∈ [P0 + Gmin, P0 + Gmax] Formula 2

[0253] wherein P0 is the first power value, the first power value is a transmission power value of the ideal best reception beam. Gmin is the minimum gain value of beamforming, and P0 + Gmax is the maximum gain value of beamforming.

[0254] The fourth RSRP value is a measured RSRP value of the ideal best transmission beam at the receiving end, and the fourth RSRP value can be denoted as P4.

[0255] The fifth RSRP value is an ideal RSRP value of the measured best transmission beam at the receiving end, and the fifth RSRP value can be denoted as P5.

[0256] The first RSRP value interval can be associated with at least one of the first RSRP value, the second RSRP value, the third RSRP value, the fourth RSRP value, and the fifth RSRP value.

[0257] Exemplarily, the first RSRP value interval can be [P1-x, P1+x].

[0258] Exemplarily, the first RSRP value interval can be [P2-x, P2+x].

[0259] Exemplarily, the first RSRP value interval can be [P2-x, P1+x].

[0260] Exemplarily, the first RSRP value interval can be [P1-x, P2+x].

[0261] Exemplarily, the first RSRP value interval can be [P3-x, P3+x]. Wherein P3 = P0 + G0.

[0262] Exemplarily, the first RSRP value interval can be [P3-x, P3+x]. Wherein P3 ∈ [P0 + Gmin, P0 + Gmax].

[0263] Exemplarily, the first RSRP value interval can be [P3-x-y, P3+x]. Wherein, P3 e [P0+Gmin, P0+Gmax]. Wherein, y can be a beam mismatch difference value. y can be defined based on a protocol or indicated by the terminal 101, which is not limited in the disclosure.

[0264] Exemplarily, the first RSRP value interval can be [P4-x, P4+x].

[0265] Exemplarily, the first RSRP value interval can be [P5-x, P5+x].

[0266] Exemplarily, the first RSRP value interval can be [P4-x, P5+x].

[0267] Exemplarily, the first RSRP value interval can be [P5-x, P4+x].

[0268] The above is only an exemplary description, and the disclosure does not limit the first RSRP value interval.

[0269] Wherein, the above x can be a non-negative number, i.e. x≥0.

[0270] Wherein, the above x can be determined based on the accuracy of beam measurement. Of course, x can also be determined based on other information, which is not limited in the disclosure.

[0271] Exemplarily, the first device 102 can determine the accuracy of the predicted RSRP value = the second number of times / the second total number of times. Wherein, the second number of times is the number of times that the predicted RSRP value belongs to the second RSRP value interval, and the second total number of times is the total number of times that the terminal 101 sends the predicted RSRP value.

[0272] Wherein, the second RSRP value interval is determined based on the first RSRP value interval and a preset value.

[0273] The preset value can be a fixed value agreed by a protocol. The fixed value can be a real number, for example, can be 1, 2 or other values, which is not limited in the disclosure.

[0274] The minimum value of the second RSRP value interval can be equal to the minimum value of the first RSRP value interval minus the preset value. The maximum value of the second RSRP value interval can be equal to the maximum value of the first RSRP value interval plus the preset value.

[0275] For example, the first RSRP value interval can be [P1-x, P1+x], and the preset value is Δ. The second RSRP value interval is [P1-x-Δ, P1+x+Δ].

[0276] The above is only an example. The minimum value of the second RSRP value interval can also be equal to the minimum value of the first RSRP value interval + a preset value. The maximum value of the second RSRP value interval can also be equal to the maximum value of the first RSRP value interval - a preset value. The present disclosure is not limited in this regard.

[0277] For example, the terminal 101 sends 100 predicted RSRP values, 90 of which belong to the second RSRP value interval, and the accuracy is 90%.

[0278] In some embodiments, the names of information and the like are not limited to the names described in the embodiments. The terms "information", "message", "signal", "signaling", "report", "configuration", "indication", "instruction", "command", "channel", "parameter", "domain", "field", "symbol", "codebook", "codeword", "codepoint", "bit", "data", "program", "chip", and the like can be replaced with each other.

[0279] In some embodiments, the terms "send", "transmit", "report", "issue", "transmit", "bidirectional transmission", "send and / or receive", and the like can be replaced with each other.

[0280] In some embodiments, the terms "acquire", "obtain", "get", "receive", "transmit", "bidirectional transmission", "send and / or receive" can be replaced with each other, and can be interpreted as receiving from other subjects, acquiring from protocols, obtaining from higher layers, obtaining by self-processing, autonomously implementing, and the like.

[0281] In some embodiments, the terms "certain", "preset", "preset", "set", "indicated", "certain", "arbitrary", "first", and the like can be replaced with each other. "Certain A", "preset A", "preset A", "set A", "indicated A", "certain A", "arbitrary A", "first A" can be interpreted as A specified in advance in protocols and the like, can be interpreted as A obtained by setting, configuring, or indicating, and the like, and can be interpreted as certain A, certain A, arbitrary A, or first A, but is not limited thereto.

[0282] In some embodiments, the evaluation method according to the embodiments of the present disclosure can include at least one of steps S2100-S2102. For example, step S2100 can be implemented as an independent embodiment, step S2101 can be implemented as an independent embodiment, step S2102 can be implemented as an independent embodiment, steps S2101 and S2102 can be implemented as an independent embodiment, steps S2100-S2102 can be implemented as an independent embodiment, but the present disclosure is not limited thereto.

[0283] In some embodiments, step S2100 is optional, and one or more of the steps can be omitted or replaced in different embodiments. For example, step S2100 can not be performed when the terminal 101 obtains the measurement value without performing measurement.

[0284] In some embodiments, step S2101 is optional, and one or more of the steps can be omitted or replaced in different embodiments. For example, step S2101 can not be performed when the prediction value is sent by another execution subject.

[0285] In some embodiments, step S2102 is optional, and one or more of the steps can be omitted or replaced in different embodiments. For example, step S2102 can not be performed when the measurement value in the L1-RSRP report reported by the terminal is obtained in a non-AI manner.

[0286] In some embodiments, steps S2101-S2102 are optional, and one or more of the steps can be omitted or replaced in different embodiments.

[0287] In some embodiments, the execution order of steps S2101-S2102 is not limited.

[0288] In the above embodiments, the first device can receive a prediction value sent by the terminal, the prediction value can be used to indicate a parameter related to a beam, and based on the prediction value and the true value, an evaluation result can be determined, wherein the evaluation result is used to evaluate the accuracy of the prediction value. The present disclosure can evaluate the accuracy of the AI model in predicting the beam-related parameter, thereby improving the usability of the AI model in the communication system.

[0289] In some embodiments, the scheme shown in FIG. 2A can also be applicable to evaluating the prediction value sent by the network device, thereby determining the evaluation result. The specific implementation is similar to the scheme of FIG. 2A, which will not be described here.

[0290] FIG. 2B is an interaction diagram of an evaluation method according to an embodiment of the present disclosure. As shown in FIG. 2B, the present disclosure relates to an evaluation method, and the above method includes:

[0291] Step S2201, the terminal 101 acquires a measurement value.

[0292] In some embodiments, the implementation of step S2201 is similar to the aforementioned step S2100, which will not be repeated here.

[0293] Step S2202, the terminal 101 determines an evaluation result based on the predicted value and the true value.

[0294] In some embodiments, the first device 102 can also be the terminal 101 itself. That is, the terminal 101 can itself determine the evaluation result based on the predicted value and the true value.

[0295] In some embodiments, the implementation of step S2202 is similar to the aforementioned step S2102, which will not be repeated here.

[0296] In some embodiments, steps S2201 to S2202 are optional, and one or more of these steps can be omitted or replaced in different embodiments.

[0297] In some embodiments, the execution order of steps S2201 to S2202 is not limited.

[0298] In the above embodiments, the terminal can directly determine the measurement value and determine the evaluation result based on the predicted value and the true value, wherein the evaluation result is used to evaluate the accuracy of the predicted value. The present disclosure can evaluate the accuracy of the AI model in predicting beam-related parameters, thereby improving the usability of the AI model in the communication system.

[0299] FIG. 3A is an interaction diagram of an evaluation method according to an embodiment of the present disclosure. As shown in FIG. 3A, the present embodiment relates to an evaluation method, which can be performed by the first device 102, and the above method comprises:

[0300] Step S3101, acquiring a predicted value.

[0301] In some embodiments, the first device 102 can acquire the predicted value from the terminal 101, but is not limited thereto, and can also receive a predicted value sent by other subjects.

[0302] In some embodiments, the first device 102 acquires the predicted value determined according to a predefined rule.

[0303] In some embodiments, the first device 102 processes to obtain the predicted value.

[0304] In some embodiments, step S3101 is omitted, and the first device 102 autonomously implements the function indicated by the predicted value, or the first device 102 acquires the predicted value based on a predefined rule or protocol agreement, or the above function is default or default.

[0305] In some embodiments, the optional implementation of step S3101 can refer to the optional implementation of step S2101 in FIG. 2 and other associated parts in the embodiments involved in FIG. 2, which will not be repeated here.

[0306] Step S3102, determining the evaluation result.

[0307] In some embodiments, the optional implementation of step S3102 can refer to the optional implementation of step S2102 in FIG. 2 and other associated parts in the embodiments involved in FIG. 2, which will not be repeated here.

[0308] In some embodiments, steps S3101 to S3102 are optional, and one or more of these steps can be omitted or replaced in different embodiments.

[0309] In some embodiments, the execution order of steps S3101 to S3102 is not limited.

[0310] In the above embodiments, the first device can evaluate the accuracy of the prediction value output by the AI model, improving the usability of the AI model in the communication system.

[0311] FIG. 3B is an interaction diagram of an evaluation method according to an embodiment of the present disclosure. As shown in FIG. 3B, the embodiments of the present disclosure relate to an evaluation method, which can be executed by the terminal 101. The above method includes:

[0312] Step S3201, sending a prediction value.

[0313] In some embodiments, the terminal 101 can send the prediction value to the first device 102.

[0314] In some embodiments, the first device 102 receives the prediction value.

[0315] In some embodiments, the optional implementation of step S3201 can refer to the optional implementation of step S2101 in FIG. 2 and other associated parts in the embodiments involved in FIG. 2, which will not be repeated here.

[0316] In the above embodiments, the terminal can send the prediction value to the first device, so that the first device determines the accuracy of the prediction value. The usability of the AI model in the communication system is improved.

[0317] The above process is further illustrated as follows.

[0318] The beam prediction accuracy can be divided into two types:

[0319] Type 1, based on the difference of beam index values between predicted Top-1 or Top-K beams and ground truth Top-1 or Top-K beams.

[0320] Type 2, based on the difference of beam index values between predicted Top-1 beam and ground truth Top-1 beam.

[0321] Type 1, for the prediction accuracy of beam index values, compare the predicted best TX beam index value with the index ground truth of the best TX beam.

[0322] There are two possible choices for the index ground truth of the best TX beam:

[0323] Option 1, the index value of the ideal best Tx beam.

[0324] Option 2, the index value of the measured best TX beam.

[0325] For Option 1, the ground truth is the index value of the ideal best Tx beam in reality. It will be divided into two cases:

[0326] Case 1: Ground truth in non-line-of-sight (NLOS) only.

[0327] Since the first device 102 can know all channel model related information and TX side implementation except the RX beam information of the terminal, it can adopt Top-1 Tx beam, which has two options related to RX beam assumption:

[0328] Option A (basic), Top-1 Tx beam is the Tx beam that produces the maximum L1-RSRP on all Tx and Rx beams.

[0329] Option B (optional), Top-1 Tx beam is the Tx beam that produces the maximum L1-RSRP on all Tx beams with a specific Rx beam.

[0330] Specific Rx beam needs to be reported. Note: Specific Rx beam is a subset of all Rx beams.

[0331] For Option B, the index value of the ideal best TX beam is selected by the specific RX beam. The ideal best TX beam is suboptimal.

[0332] As shown in FIG. 4A. If the index value of the RX beam is assumed to be 1, the corresponding ideal best TX beam is the one with TX beam #1. And if the index value of the RX beam is assumed to be 2, the ideal best TX beam can be the one with index value 2. If the index value of the RX beam changes, the index value of the ideal best TX beam can change accordingly. Therefore, for a specific RX beam case, the index value of the best TX beam is determined by the cluster location and the RX beam direction.

[0333] For the first device 102, it does not know the detailed RX beam information because it is a terminal implementation. It can also not know where the cluster is if the cluster is not artificial. Without the information of the RX beam direction and the cluster location, the first device 102 cannot know what the corresponding ideal best TX beam is.

[0334] For option A, the index value of the ideal best TX beam is selected by considering all RX beam scans. The index value of the ideal best TX beam is a theoretical global optimization. However, in reality, the terminal can only apply a limited RX beam scan, for example, the number of scanned RX beams N = 8. Then, there are many different RX beam angle sets from different terminal implementations. Different RX angle sets can also correspond to different index values of the best TX beam. In addition, the first device 102 also does not know whether the cluster exists, so the first device 102 does not know what the index value of the ideal best TX beam is.

[0335] In summary, in the multipath case, the first device 102 cannot know the exact index value of the ideal TX beam.

[0336] In this case, some margin needs to be added, similar to the traditional case, where the ideal RSRP is considered by considering the range of RX beamforming gain. Here, the index value of the ideal TX beam can also be a set of TX beam index values. For example, the ideal TX beam set can be a combination of several TX beam index values. On the other hand, assuming that the terminal has the possibility to report the RX beam, the first device 102 has the possibility to know the index value of the best TX beam.

[0337] The test index is: the percentage of the predicted best transmission beam index value being one of the K ideal best beam index values (K >= 1).

[0338] There are several options for the K ideal best beam index values:

[0339] - Option 1, the top K ideal best beam index values determined by RSRP values, selecting the beam index values corresponding to the K largest RSRP values.

[0340] Option 2: Select the beam index value corresponding to the maximum RSRP, for example, n. Then select other K indexes around it, for example, n+1, n-1, etc.

[0341] In the case of only LOS or LOS+NLOS, the LOS ray is the strongest, as shown in FIG. 4B or FIG. 4C. For the LOS ray, the best TX beam direction is only determined by the locations of the transmitter and receiver. The index value of the best Tx beam is independent of which RX beam is selected. Therefore, if the first device 102 knows the terminal location, the first device 102 can know the index value of the ideal best TX beam.

[0342] The test metric is: the percentage of predicted beam index values that are the index value of the ideal best beam.

[0343] The index value of the ideal best beam is determined by the angle of the TX-RX relative to the TX antenna configuration, as shown in FIG. 4D. Since the TE itself knows the configuration of the TX antenna, the first device 102 can select the index value of the best TX beam according to the vertical direction and the azimuth angle.

[0344] Option 2, another possible solution is to use the index value of the measured best TX beam as the true value.

[0345] The test metric is: the percentage of predicted beam index values that are the measured best beam index value.

[0346] Type 2, compare the predicted best TX beam RSRP value with the best TX beam RSRP value.

[0347] However, as mentioned earlier, in some cases, the first device 102 can not know the index value of the ideal best TX beam, therefore, some margin of beam mismatch needs to be considered. In addition, the first device 102 does not know the RX beamforming gain. There are several options for the test metric:

[0348] Option 1, it is considered that the maximum RSRP value in the first K predicted beams is greater than (ideal RSRP-x of the ideal best transmission beam at the receiving end) decibels (dB), and less than (ideal RSRP+x of the ideal best transmission beam at the receiving end) dB.

[0349] Wherein, the relevant measurement accuracy can be considered to determine x.

[0350] Option 2, it is considered that the maximum RSRP value in the first K predicted beams is greater than (measured RSRP-x of the measured best transmission beam at the receiving end) dB, and less than (measured RSRP+x of the measured best transmission beam at the receiving end) dB.

[0351] Where x can be determined considering the related measurement accuracy.

[0352] Option 3 is considered that the maximum RSRP value among the top K predicted beams is greater than (ideal best transmit beam’s ideal RSRP at the receiver - x) dB and less than (measured best transmit beam’s measured RSRP at the receiver + x) dB.

[0353] Where x can be determined considering the related measurement accuracy.

[0354] Option 4 is considered that the maximum RSRP value among the top K predicted beams is greater than (measured best transmit beam’s measured RSRP at the receiver - x) dB and less than (ideal best transmit beam’s ideal RSRP at the receiver + x) dB.

[0355] Where x can be determined considering the related measurement accuracy.

[0356] Option 5 is considered that the maximum RSRP value among the top K predicted beams is greater than (P3 - x) dB and less than (P3 + x) dB.

[0357] Where x can be determined considering the related measurement accuracy.

[0358] Where P3 is the RSRP of the strongest beam with fixed RX beamforming gain, P3 = P0 + G0, P0 is the transmit power of the ideal best RX beam.

[0359] Option 6 is considered that the maximum RSRP value among the top K predicted beams is greater than (P3 - x) dB and less than (P3 + x) dB.

[0360] Where x can be determined considering the related measurement accuracy.

[0361] Where P3 is the RSRP range considering RX beamforming gain, i.e., P3 ∈ [P0 + Gmin, P0 + Gmax], P0 is the transmit power of the ideal best RX beam.

[0362] Option 7 is considered that the maximum RSRP value among the top K predicted beams is greater than (P3 - x - y) dB and less than (P3 + x) dB.

[0363] Where x can be determined considering the related measurement accuracy.

[0364] Where the beam mismatch margin is y dB.

[0365] Where P3 is the RSRP range considering RX beamforming gain, i.e., P3 ∈ [P0 + Gmin, P0 + Gmax], P0 is the transmit power of the ideal best RX beam.

[0366] wherein the test indicator is: a percentage of predicted RSRP values belonging to a RSRP value interval. The RSRP value interval here is determined based on the above RSRP interval ± a preset value.

[0367] The embodiments of the present disclosure also propose a device for implementing any of the above methods, for example, a device is proposed, which comprises units or modules for implementing the steps performed by each node (e.g., terminal, first device) in any of the above methods.

[0368] It should be understood that the division of each unit or module in the above device is only a logical function division, and all or part of them can be integrated into one physical entity or physically separated in actual implementation. In addition, the units or modules in the device can be implemented in the form of processor calling software: for example, the device comprises a processor connected with a memory, the memory stores instructions, and the processor calls the instructions stored in the memory to implement any of the above methods or to realize the functions of the units or modules of the device, wherein the processor is, for example, a general processor such as a central processing unit (CPU) or a microprocessor, and the memory is a memory in the device or a memory outside the device. Alternatively, the units or modules in the device can be implemented in the form of hardware circuit, and the functions of part or all of the units or modules can be realized by the design of hardware circuit, and the hardware circuit can be understood as one or more processors; for example, in one implementation, the hardware circuit is an application-specific integrated circuit (ASIC), and the functions of part or all of the units or modules are realized by the design of the logical relationship of elements in the circuit; for another example, in another implementation, the hardware circuit is a programmable logic device (PLD), and taking a field programmable gate array (FPGA) as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by a configuration file, so as to realize the functions of part or all of the units or modules. All units or modules of the above device can be implemented in the form of processor calling software, or all units or modules can be implemented in the form of hardware circuit, or part of the units or modules can be implemented in the form of processor calling software, and the remaining part can be implemented in the form of hardware circuit.

[0369] In the embodiments of the present disclosure, the processor is a circuit with signal processing capability. In one implementation, the processor can be a circuit with instruction reading and running capability, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a microprocessor), a digital signal processor (DSP), or the like. In another implementation, the processor can implement certain functions through a logical relationship of hardware circuit, and the logical relationship of the hardware circuit is fixed or can be reconfigured. For example, the processor is a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as an FPGA. In the reconfigurable hardware circuit, the processor loads a configuration document to implement the configuration of the hardware circuit. It can be understood that the processor loads instructions to implement the functions of the above part or all units or modules. In addition, the hardware circuit can also be designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), or the like.

[0370] FIG. 5A is a structural schematic diagram of a first device according to an embodiment of the present disclosure. As shown in FIG. 5A, the first device 5100 can include a transceiver module 5101 and a processing module 5102.

[0371] In some embodiments, the transceiver module 5101 described above is configured to receive a predicted value sent by a terminal; wherein the predicted value is used to indicate a parameter related to a beam.

[0372] In some embodiments, the processing module 5102 described above is configured to determine an evaluation result based on the predicted value and a true value, and the evaluation result is used to evaluate the accuracy of the predicted value.

[0373] In some embodiments, the transceiver module 5101 is configured to perform at least one of the communication steps (for example, step S2101, but not limited thereto) of the receiving and / or sending performed by the first device 5100 in any of the above methods, and details are not described herein again.

[0374] In some embodiments, the processing module 5102 described above is configured to perform at least one of the other steps (for example, step S2102, but not limited thereto) performed by the first device 5100 in any of the above methods, which will not be repeated here.

[0375] FIG. 5B is a structural schematic diagram of a terminal according to an embodiment of the present disclosure. As shown in FIG. 5B, the terminal 5200 can include a transceiver module 5201.

[0376] In some embodiments, the transceiver module 5201 described above is configured to send a prediction value to the first device; wherein the prediction value is used to indicate a parameter related to a beam.

[0377] The prediction value is used by the first device to determine an evaluation result, and the evaluation result is used to evaluate the accuracy of the prediction value.

[0378] In some embodiments, the transceiver module 5201 described above is configured to perform at least one of the communication steps (for example, step S2101, but not limited thereto) performed by the terminal 5200 in any of the above methods, which will not be repeated here.

[0379] FIG. 5C is a structural schematic diagram of a communication device according to an embodiment of the present disclosure. As shown in FIG. 5C, the communication device 5300 can include a processing module 5301.

[0380] In some embodiments, the processing module 5301 described above is configured to determine an evaluation result based on a prediction value and a true value; wherein the prediction value is used to indicate a parameter related to a beam; and the evaluation result is used to evaluate the accuracy of the prediction value.

[0381] In some embodiments, the prediction value includes:

[0382] a prediction index value, the prediction index value being an index value of a best transmission beam predicted by the terminal;

[0383] The true value includes a first set, and the first set includes any of the following:

[0384] a first index value, wherein the first index value is an index value of a best transmission beam;

[0385] a first index value and one or more second index values, wherein the first index value is an index value of a best transmission beam.

[0386] In some embodiments, the prediction value includes:

[0387] a prediction reference signal receiving power (RSRP) value, the prediction RSRP value being an RSRP value of a best beam predicted by the terminal;

[0388] The true value includes a first RSRP value interval.

[0389] In some embodiments, the communication device 5300 can be a terminal, and the processing module 5301 is configured to perform at least one of other steps performed by the terminal in any of the above methods (for example, step S2201, step S2202, but not limited thereto), which will not be repeated here.

[0390] In some embodiments, the communication device 5300 can be a network device, including but not limited to the aforementioned access network device or core network device. The processing module 5301 is configured to perform at least one of other steps performed by the network device in any of the above methods (for example, step S2201, step S2202 (the subject of execution can be replaced by the network device), but not limited thereto), which will not be repeated here.

[0391] FIG. 5D is a structural schematic diagram of a communication device according to an embodiment of the present disclosure. As shown in FIG. 5D, the communication device 5400 can include a processing module 5401.

[0392] In some embodiments, the processing module 5401 is configured to obtain a predicted value output by the AI model; wherein the predicted value is used to indicate a parameter related to a beam.

[0393] The processing module 5401 is further configured to train the AI model based on the predicted value and a true value until a stop training condition is met, to obtain a trained AI model; wherein the stop training condition includes that the accuracy of the predicted value output by the AI model reaches a first accuracy.

[0394] In some embodiments, the input value of the AI model includes:

[0395] The RSRP value of one or more transmission beams measured at the receiving end.

[0396] In some embodiments, the predicted value includes:

[0397] A predicted index value, the predicted index value being an index value of a best transmission beam predicted by the terminal;

[0398] The true value includes a first set, and the first set includes any of the following:

[0399] A first index value, wherein the first index value is an index value of a best transmission beam;

[0400] A first index value and one or more second index values, wherein the first index value is an index value of a best transmission beam.

[0401] In some embodiments, the predicted value includes:

[0402] a predicted reference signal received power (RSRP) value, the predicted RSRP value being an RSRP value of a best beam predicted by the terminal;

[0403] The true value includes a first RSRP value interval.

[0404] In some embodiments, the device is any one of the following:

[0405] a terminal;

[0406] a network device.

[0407] It should be noted that the communication device 5400 can supervise the training of the AI model based on the predicted value output by the AI model deployed by itself, and take the true value as a label. When the stop training condition is met, the trained AI model is obtained. The stop training condition can include but is not limited to the accuracy of the predicted value output by the AI model reaching a first accuracy.

[0408] The first accuracy can be determined based on experience value, or can be manually set, and the present disclosure does not limit this.

[0409] The terminal or the network device can train the AI model deployed by itself in this way, so as to obtain an AI model meeting the first accuracy. The reliability and accuracy of the output of the AI model are improved. In some embodiments, when the above communication device is a network device, it can include but is not limited to the aforementioned access network device or core network device.

[0410] In some embodiments, the transceiver module can include a sending module and / or a receiving module, and the sending module and the receiving module can be separate or integrated together. Optionally, the transceiver module can be mutually replaced with the transceiver.

[0411] In some embodiments, the processing module can be one module, or can include multiple sub-modules. Optionally, the multiple sub-modules perform all or part of the steps required to be performed by the processing module. Optionally, the processing module can be mutually replaced with the processor.

[0412] FIG. 6A is a structural schematic diagram of a communication device 6100 according to an embodiment of the present disclosure. The communication device 6100 can be a first device, a chip, a chip system, or a processor supporting the first device to implement any one of the above methods, and can also be a terminal or a chip, a chip system, or a processor supporting the terminal to implement any one of the above methods. The communication device 6100 can be used to implement the methods described in the above method embodiments, and specific implementation can be referred to the descriptions in the above method embodiments.

[0413] As shown in FIG. 6A, the communication device 6100 includes one or more processors 6101. The processor 6101 can be a general processor or a special purpose processor, etc., for example, a baseband processor or a central processing unit. The baseband processor can be used to process communication protocols and communication data, the central processing unit can be used to control the communication device (e.g., a first device, a terminal, etc.), execute programs, and process data of the programs. Optionally, the communication device 6100 is configured to perform any of the above methods. Optionally, the one or more processors 6101 are configured to invoke instructions to cause the communication device 6100 to perform any of the above methods.

[0414] In some embodiments, the communication device 6100 further includes one or more transceivers 6102. When the communication device 6100 includes one or more transceivers 6102, the transceiver 6102 performs at least one of the communication steps (e.g., step S2101, but not limited to) in the above methods, and the processor 6101 performs at least one of the other steps (e.g., step S2100, step S2102, step S2201, step S2202, but not limited to). In optional embodiments, the transceiver can include a receiver and / or a transmitter, which can be separate or integrated together. Optionally, the terms transceiver, transceiving unit, transceiver, transceiving circuit, interface circuit, interface, etc. can be replaced with each other, the terms transmitter, transmitting unit, transmitter, transmitting circuit, etc. can be replaced with each other, and the terms receiver, receiving unit, receiver, receiving circuit, etc. can be replaced with each other.

[0415] In some embodiments, the communication device 6100 further includes one or more memories 6102 for storing data. Optionally, all or part of the memory 6102 can also be outside the communication device 6100. In optional embodiments, the communication device 6100 can include one or more interface circuits 6104. Optionally, the interface circuit 6104 is connected to the memory 6102, and the interface circuit 6104 can be used to receive data from the memory 6102 or other devices, and can be used to send data to the memory 6102 or other devices. For example, the interface circuit 6104 can read data stored in the memory 6102 and send the data to the processor 6101.

[0416] The communication device 6100 described in the above embodiments can be a network device, but the scope of the communication device 6100 described in the present disclosure is not limited thereto, and the structure of the communication device 6100 can not be limited by FIG. 6A. The communication device can be a standalone device or can be part of a larger device. For example, the communication device can be: 1) a standalone integrated circuit (IC), or a chip, or a chip system or subsystem; (2) a set of one or more ICs, which can optionally also include storage components for storing data, programs; (3) an ASIC, such as a modem; (4) a module that can be embedded in other devices; (5) a receiver, terminal device, intelligent terminal device, cellular phone, wireless device, handset, mobile unit, car-mounted device, network device, cloud device, artificial intelligence device, and the like; (6) other devices, and the like.

[0417] FIG. 6B is a structural schematic diagram of a chip 6200 according to an embodiment of the present disclosure. For the case where the communication device 6100 can be a chip or a chip system, the structural schematic diagram of the chip 6200 shown in FIG. 6B can be referred to, but is not limited thereto.

[0418] The chip 6200 includes one or more processors 6201. The chip 6200 is configured to perform any of the above methods.

[0419] In some embodiments, the chip 6200 further includes one or more interface circuits 6202. Optionally, the terms interface circuit, interface, transceiver pin, and the like can be replaced with each other. In some embodiments, the chip 6200 further includes one or more memories 6203 for storing data. Optionally, all or part of the memory 6203 can be outside the chip 6200. Optionally, the interface circuit 6202 is connected to the memory 6203, and the interface circuit 6202 can be configured to receive data from the memory 6203 or other devices, and the interface circuit 6202 can be configured to send data to the memory 6203 or other devices. For example, the interface circuit 6202 can read data stored in the memory 6203 and send the data to the processor 6201.

[0420] In some embodiments, the interface circuit 6202 performs at least one of the communication steps (for example, step S2101, but not limited thereto) of transmitting and / or receiving in the above methods. The interface circuit 6202 performing the communication steps such as transmitting and / or receiving in the above methods means that the interface circuit 6202 performs data interaction between the processor 6201, the chip 6200, the memory 6203, or a transceiver device. In some embodiments, the processor 6201 performs at least one of other steps (for example, step S2100, step S2102, step S2201, step S2202, but not limited thereto).

[0421] The modules and / or devices described in each embodiment of the virtual device, physical device, chip, etc. can be combined or separated according to circumstances. Optionally, part or all of the steps can also be performed by multiple modules and / or devices in cooperation, which is not limited here.

[0422] The disclosure further provides a storage medium having stored instructions which, when executed on the communication device 6100, cause the communication device 6100 to perform any of the above methods. Optionally, the storage medium is an electronic storage medium. Optionally, the storage medium is a computer-readable storage medium, but is not limited to this, and it can also be a storage medium readable by other devices. Optionally, the storage medium can be a non-transitory storage medium, but is not limited to this, and it can also be a transitory storage medium.

[0423] The disclosure further provides a program product which, when executed by the communication device 6100, causes the communication device 6100 to perform any of the above methods. Optionally, the program product is a computer program product.

[0424] The disclosure further provides a computer program which, when executed on a computer, causes the computer to perform any of the above methods.

[0425] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the features disclosed herein. The disclosure is intended to cover any variations, uses or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice in the art to which the disclosure pertains. The specification and examples are to be regarded as illustrative only, and the true scope and spirit of the disclosure are indicated by the following claims.

[0426] It should be understood that the present disclosure is not limited to the precise structures herein described and illustrated above, and that various modifications and changes in the aspects can be made by those skilled in the art without departing from the scope of this disclosure. The scope of the present disclosure is limited only by the claims that follow.

Claims

1. An evaluation method, characterized in that: include: receiving a prediction value sent by a terminal; wherein the prediction value is used to indicate a parameter related to the beam; An evaluation result is determined based on the predicted value and the true value, and the evaluation result is used to evaluate the accuracy of the predicted value.

2. The method according to claim 1, characterized in that The predicted values ​​include: A predicted index value, where the predicted index value is an index value of an optimal transmit beam predicted by the terminal; The truth value includes a first set, and the first set includes any of the following: A first index value; wherein the first index value is the index value of the best transmit beam; A first index value and one or more second index values; wherein the first index value is the index value of the optimal transmit beam.

3. The method according to claim 2, characterized in that The method further comprises: Determine the transmission beam with the largest RSRP value as the optimal transmission beam, and determine the index value of the optimal transmission beam as the first index value; or The optimal transmit beam is determined based on the first position information of the transmit antenna, the second position information of the receive antenna, and the first spatial configuration information of the transmit antenna, and the index value of the optimal transmit beam is determined as the first index value.

4. The method according to claim 2 or 3, characterized in that The method further comprises: Selecting one or more transmit beams from other transmit beams other than the optimal transmit beam in descending order of RSRP values ​​of each transmit beam, and determining the index value of the selected transmit beam as the second index value; or Among the transmission beams other than the best transmission beam, one or more transmission beams adjacent to the best transmission beam are selected, and the index values ​​of the selected transmission beams are determined as the second index value.

5. The method according to any one of claims 2 to 4, characterized in that: The accuracy is equal to a first quotient of a first number and a first total number, wherein the first number is the number of times the predicted index value belongs to the first set, and the first total number is the total number of times the predicted index value sent by the terminal is received.

6. The method according to any one of claims 1 to 5, characterized in that The predicted values ​​include: Predicting a reference signal received power (RSRP) value, where the predicted RSRP value is an RSRP value of an optimal beam predicted by the terminal; The true value includes a first RSRP value interval.

7. The method according to claim 6, characterized in that The first RSRP value interval is associated with at least one of the following: a first RSRP value; a second RSRP value, where the second RSRP value is a measured RSRP value of the best transmit beam at the receiving end; A third RSRP value is an RSRP value of an optimal transmit beam having a beamforming gain.

8. The method according to claim 6 or 7, characterized in that The first RSRP value range includes at least one of the following: The minimum value of the first RSRP value interval is equal to a first value, and the maximum value of the first RSRP value interval is equal to a second value; wherein the first value is a difference between the first RSRP value and x, and the second value is a sum of the first RSRP value and x; The minimum value of the first RSRP value interval is equal to a third value, and the maximum value of the first RSRP value interval is equal to a fourth value; wherein the third value is a difference between the second RSRP value and x, and the fourth value is a sum of the second RSRP value and x; The minimum value of the first RSRP value interval is equal to a third value, and the maximum value of the first RSRP value interval is equal to a second value; wherein the third value is a difference between the second RSRP value and x, and the second value is a sum of the first RSRP value and x; The minimum value of the first RSRP value interval is equal to the first value, and the maximum value of the first RSRP value interval is equal to the fourth value; wherein the first value is the difference between the first RSRP value and x, and the fourth value is the sum of the second RSRP value and x; The minimum value of the first RSRP value interval is equal to the fifth value, and the maximum value of the first RSRP value interval is equal to the sixth value; wherein the fifth value is the difference between the third RSRP value and x, and the sixth value is the sum of the third RSRP value and x; The minimum value of the first RSRP value interval is equal to the seventh value, and the maximum value of the first RSRP value interval is the sixth value; wherein the seventh value is the difference between the third RSRP value and x, y, and the sixth value is the sum of the third RSRP value and x; wherein y is the beam mismatch difference.

9. The method according to claim 8, characterized in that The x is a non-negative number, and / or the x is determined based on the accuracy of beam measurement.

10. The method according to any one of claims 6 to 9, characterized in that: The accuracy is equal to the second quotient of the second number and the second total number; wherein the second number is the number of times the predicted RSRP value belongs to a second RSRP interval, the second RSRP value interval is determined based on the first RSRP value interval and a preset value, and the second total number is the total number of times the predicted RSRP value sent by the terminal is received.

11. An evaluation method, characterized in that: include: Predicting a value to the first device; wherein the predicted value is used to indicate a parameter related to the beam; The predicted value and the true value are used by the first device to determine an evaluation result, and the evaluation result is used to evaluate the predicted value. The precision of the value.

12. The method according to claim 11, characterized in that The predicted value includes at least one of the following: A predicted index value, where the predicted index value is an index value of an optimal transmit beam predicted by the terminal; A predicted reference signal received power (RSRP) value is obtained, where the predicted RSRP value is an RSRP value of an optimal transmit beam predicted by the terminal.

13. A first device, characterized in that: include: a transceiver module configured to receive a prediction value sent by a terminal; wherein the prediction value is used to indicate a parameter related to the beam; The processing module is configured to determine an evaluation result based on the predicted value and the true value, where the evaluation result is used to evaluate the accuracy of the predicted value.

14. A terminal, characterized in that: include: a transceiver module configured to predict a value to the first device; wherein the predicted value is used to indicate a parameter related to the beam; The predicted value is used by the first device to determine an evaluation result, and the evaluation result is used to evaluate the accuracy of the predicted value.

15. A communication device, characterized in that: include: a processing module configured to determine an evaluation result based on a predicted value and a true value; wherein the predicted value is used to indicate a parameter related to the beam; The evaluation result is used to evaluate the accuracy of the predicted value.

16. The device according to claim 15, characterized in that The predicted values ​​include: A predicted index value, where the predicted index value is an index value of an optimal transmit beam predicted by the terminal; The truth value includes a first set, and the first set includes any of the following: A first index value; wherein the first index value is the index value of the best transmit beam; A first index value and one or more second index values; wherein the first index value is the index value of the optimal transmit beam.

17. The device according to claim 15 or 16, characterized in that The predicted values ​​include: Predicting a reference signal received power (RSRP) value, where the predicted RSRP value is an RSRP value of an optimal beam predicted by the terminal; The true value includes a first RSRP value interval.

18. The device according to any one of claims 15 to 17, characterized in that The device is any of the following: terminal; Network equipment.

19. A communication device, characterized in that: The artificial intelligence (AI) model deployed on the device includes: a processing module configured to obtain a predicted value output by the AI ​​model; wherein the predicted value is used to indicate a parameter related to the beam; The processing module is also configured to train the AI ​​model based on the predicted value and the true value until a stop training condition is met to obtain a trained AI model; wherein the stop training condition includes that the accuracy of the predicted value output by the AI ​​model reaches a first accuracy.

20. The device according to claim 19, characterized in that The input values ​​of the AI ​​model include: Measured RSRP value of one or more transmit beams at the receiving end.

21. The device according to claim 19 or 20, characterized in that The predicted values ​​include: A predicted index value, where the predicted index value is an index value of an optimal transmit beam predicted by the terminal; The truth value includes a first set, and the first set includes any of the following: A first index value; wherein the first index value is the index value of the best transmit beam; A first index value and one or more second index values; wherein the first index value is the index value of the optimal transmit beam.

22. The device according to any one of claims 19 to 21, characterized in that The predicted values ​​include: Predicting a reference signal received power (RSRP) value, where the predicted RSRP value is an RSRP value of an optimal beam predicted by the terminal; The true value includes a first RSRP value interval.

23. The device according to any one of claims 19 to 22, characterized in that The device is any of the following: terminal; Network equipment.

24. A first device, characterized in that: include: one or more processors; The processor is configured to execute the evaluation method according to any one of claims 1 to 10.

25. A terminal, characterized in that: include: one or more processors; Wherein, the processor is used to execute the evaluation method described in claim 11 or 12.

26. A communication system, characterized in that: include: a first device, wherein the first device is configured to implement the evaluation method according to any one of claims 1 to 10; A terminal configured to implement the evaluation method according to claim 11 or 12.

27. A storage medium storing instructions, characterized in that: When the instruction is executed on a communication device, the communication device is caused to execute the evaluation method according to any one of claims 1 to 10 or 11 to 12.

28. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, it is used to implement the evaluation method according to any one of claims 1 to 10 or 11 to 12.

Images