Performance evaluation method, first device, and second device

Abstract

The present application relates to a performance evaluation method, a first device, a second device, a chip, a computer-readable storage medium, a computer program product, a computer program, and a communication system. The performance evaluation method comprises: under a first condition, a first device uses a first evaluation mode corresponding to the first condition to evaluate the performance of a first scheme. According to the technical solution in embodiments of the present application, the evaluation performance can be improved and the evaluation overhead can be controlled.

Description

Performance evaluation method, first device and second device Technical Field

[0001] The present application relates to the field of communications, and more specifically, to a performance evaluation method, a first device, a second device, a chip, a computer-readable storage medium, a computer program product, a computer program, and a communication system. Background Art

[0002] Communications systems involve numerous wireless communication solutions, requiring evaluation to manage them, such as switching, updating, or rolling back solutions. For certain solutions, a simple approach is to build an evaluation mechanism to obtain results and trigger subsequent processes. However, considerations are needed to improve evaluation performance and control evaluation overhead.

[0003] Summary of the Invention

[0004] This embodiment of the present application provides a performance evaluation method, including:

[0005] Under the first condition, the first device evaluates the performance of the first solution using a first evaluation method corresponding to the first condition.

[0006] This embodiment of the present application provides a performance evaluation method, including:

[0007] The second device sends third indication information to the first device; wherein the third indication information is used to enable the first device to evaluate the performance of the first solution under the first condition by using a first evaluation method corresponding to the first condition.

[0008] An embodiment of the present application provides a first device, including:

[0009] The first processing module is configured to evaluate the performance of the first solution under a first condition by using a first evaluation method corresponding to the first condition.

[0010] An embodiment of the present application provides a second device, including:

[0011] The third communication module is used to send third indication information to the first device; wherein the third indication information is used to enable the first device to evaluate the performance of the first solution under the first condition using a first evaluation method corresponding to the first condition.

[0012] An embodiment of the present application provides a first device, comprising: a transceiver, a processor, and a memory. The memory is configured to store a computer program, the transceiver is configured to communicate with other devices, and the processor is configured to call and execute the computer program stored in the memory, so that the first device performs the aforementioned performance evaluation method.

[0013] An embodiment of the present application provides a second device, comprising: a transceiver, a processor, and a memory. The memory is configured to store a computer program, the transceiver is configured to communicate with other devices, and the processor is configured to call and execute the computer program stored in the memory, so that the second device executes the aforementioned performance evaluation method.

[0014] An embodiment of the present application provides a chip for implementing the above-mentioned performance evaluation method.

[0015] Specifically, the chip includes: a processor, which is used to call and run a computer program from a memory, so that a device equipped with the chip executes the above-mentioned performance evaluation method.

[0016] An embodiment of the present application provides a computer-readable storage medium for storing a computer program. When the computer program is executed by a device, the device executes the above-mentioned performance evaluation method.

[0017] An embodiment of the present application provides a computer program product, including computer program instructions, which enable a computer to execute the above-mentioned performance evaluation method.

[0018] An embodiment of the present application provides a computer program, which, when executed on a computer, enables the computer to execute the above-mentioned performance evaluation method.

[0019] An embodiment of the present application provides a communication system, including a first device and a second device for executing the above-mentioned performance evaluation method.

[0020] In this embodiment of the present application, under a first condition, the first device uses a first evaluation method corresponding to the first condition to evaluate the performance of the first solution, i.e., corresponding evaluation methods are used for different conditions. Because the impact of different conditions on the evaluation of the first solution is taken into account, a more appropriate evaluation method can be used to evaluate the first solution, thereby improving evaluation performance and controlling evaluation overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] FIG1 is a schematic diagram of a communication system according to an embodiment of the present application.

[0022] FIG2A is a schematic diagram of an artificial intelligence-based CSI feedback solution in an embodiment of the present application.

[0023] FIG2B is a schematic diagram of an artificial intelligence-based channel estimation solution in an embodiment of the present application.

[0024] FIG2C is a schematic diagram of an artificial intelligence-based positioning solution in an embodiment of the present application.

[0025] FIG2D is a schematic diagram of an artificial intelligence-based beam management solution in an embodiment of the present application.

[0026] FIG3 is a schematic flowchart of a performance evaluation method according to an embodiment of the present application.

[0027] FIG4A is a schematic diagram of determining a first condition in an embodiment of the present application.

[0028] FIG4B is a schematic diagram of determining a first evaluation method in an embodiment of the present application.

[0029] FIG4C is another schematic diagram of determining the first condition in an embodiment of the present application.

[0030] FIG4D is a schematic diagram of another method for determining the first evaluation method in an embodiment of the present application.

[0031] FIG5 is a schematic flowchart of a performance evaluation method according to another embodiment of the present application.

[0032] FIG6 is a schematic block diagram of a first device according to an embodiment of the present application.

[0033] FIG7 is a schematic block diagram of a first device according to another embodiment of the present application.

[0034] FIG8 is a schematic block diagram of a first device according to yet another embodiment of the present application.

[0035] FIG9 is a schematic block diagram of a second device according to an embodiment of the present application.

[0036] FIG10 is a schematic block diagram of a communication device according to an embodiment of the present application.

[0037] FIG11 is a schematic block diagram of a chip according to an embodiment of the present application.

[0038] FIG12 is a schematic block diagram of a communication system according to an embodiment of the present application. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0040] The technical solutions of the embodiments of the present application can be applied to various communication systems, such as: Long Term Evolution (LTE) system, Advanced long term evolution (LTE-A) system, New Radio (NR) system, NR system evolution system, LTE on unlicensed spectrum (LTE-U) system, NR on unlicensed spectrum (NR-based access to unlicensed spectrum, NR-U) system, Non-Terrestria lNetworks (NTN) system, Universal Mobile Telecommunication System (UMTS), Wireless Local Area Networks (WLAN), Wireless Fidelity (WiFi), Fifth Generation (5th-Generation, 5G) system or other communication systems.

[0041] Generally speaking, traditional communication systems support a limited number of connections and are easy to implement. However, with the development of communication technology, mobile communication systems will not only support traditional communications, but will also support, for example, device-to-device (D2D) communication, machine-to-machine (M2M) communication, machine-type communication (MTC), vehicle-to-vehicle (V2V) communication, or vehicle-to-everything (V2X) communication, etc. The embodiments of the present application can also be applied to these communication systems.

[0042] In one implementation, the communication system in the embodiment of the present application may be applied to a carrier aggregation (CA) scenario, a dual connectivity (DC) scenario, or a standalone (SA) networking scenario.

[0043] In one embodiment, the communication system in the embodiment of the present application can be applied to an unlicensed spectrum, wherein the unlicensed spectrum can also be considered as a shared spectrum; or, the communication system in the embodiment of the present application can also be applied to an authorized spectrum, wherein the authorized spectrum can also be considered as an unshared spectrum.

[0044] The embodiments of the present application describe various embodiments in conjunction with network devices and terminal devices, wherein the terminal device may also be referred to as user equipment (UE), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent or user device, etc.

[0045] The terminal device can be a station (STAION, ST) in a WLAN, a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA) device, a handheld device with wireless communication capabilities, a computing device or other processing device connected to a wireless modem, a vehicle-mounted device, a wearable device, a terminal device in a next-generation communication system such as an NR network, or a terminal device in a future evolved Public Land Mobile Network (PLMN) network, etc.

[0046] In an embodiment of the present application, the terminal device can be deployed on land, including indoors or outdoors, handheld, wearable or vehicle-mounted; it can also be deployed on the water surface (such as ships, etc.); it can also be deployed in the air (such as airplanes, balloons and satellites, etc.).

[0047] In an embodiment of the present application, the terminal device may be a mobile phone, a tablet computer, a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal device in industrial control, a wireless terminal device in self-driving, a wireless terminal device in remote medical, a wireless terminal device in a smart grid, a wireless terminal device in transportation safety, a wireless terminal device in a smart city, or a wireless terminal device in a smart home, etc.

[0048] As an example and not a limitation, in the embodiment of the present application, the terminal device may also be a wearable device. Wearable devices may also be called wearable smart devices, which are a general term for wearable devices that are intelligently designed and developed using wearable technology for daily wear, such as glasses, gloves, watches, clothing, and shoes. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothes or accessories. Wearable devices are not only hardware devices, but also achieve powerful functions through software support, data interaction, and cloud interaction. Broadly speaking, wearable smart devices include those that are fully functional, large in size, and can achieve complete or partial functions without relying on smartphones, such as smart watches or smart glasses, as well as those that only focus on a certain type of application function and need to be used in conjunction with other devices such as smartphones, such as various smart bracelets and smart jewelry for vital sign monitoring.

[0049] In an embodiment of the present application, the network device may be a device for communicating with a mobile device. The network device may be an access point (AP) in a WLAN, an evolved base station (eNB or eNodeB) in LTE, or a relay station or access point, or a vehicle-mounted device, a wearable device, and a network device (gNB) in an NR network, or a network device in a future evolved PLMN network or a network device in an NTN network, etc.

[0050] As an example and not a limitation, in an embodiment of the present application, the network device may have a mobile feature, for example, the network device may be a mobile device. Alternatively, the network device may be a satellite or a balloon station. For example, the satellite may be a low earth orbit (LEO) satellite, a medium earth orbit (MEO) satellite, a geostationary earth orbit (GEO) satellite, a high elliptical orbit (HEO) satellite, etc. Optionally, the network device may also be a base station set up in a location such as land or water.

[0051] In an embodiment of the present application, the network device can provide services for a cell, and the terminal device communicates with the network device through the transmission resources used by the cell (for example, frequency domain resources, or spectrum resources). The cell can be a cell corresponding to the network device (for example, a base station). The cell can belong to a macro base station or a base station corresponding to a small cell. The small cells here may include: metro cells, micro cells, pico cells, femto cells, etc. These small cells have the characteristics of small coverage and low transmission power, and are suitable for providing high-speed data transmission services.

[0052] FIG1 exemplarily illustrates a communication system 100. The communication system includes a network device 110 and two terminal devices 120. In one embodiment, the communication system 100 may include multiple network devices 110, and each network device 110 may include a different number of terminal devices 120 within its coverage area, which is not limited in this embodiment of the present application.

[0053] It should be understood that in the embodiments of the present application, a device having a communication function in a network / system may be referred to as a communication device. Taking the communication system shown in Figure 1 as an example, the communication device may include a network device and a terminal device having a communication function. The network device and the terminal device may be specific devices in the embodiments of the present application and will not be described in detail here. The communication device may also include other devices in the communication system, such as a network controller, a mobility management entity, and other network entities, which are not limited in the embodiments of the present application.

[0054] It should be understood that the terms "system" and "network" are often used interchangeably herein. The term "and / or" is simply a description of an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " generally indicates that the related objects are in an "or" relationship.

[0055] It should be understood that the "indication" mentioned in the embodiments of this application can be a direct indication, an indirect indication, or an indication of an association. For example, "A indicates B" can mean that A directly indicates B, for example, B can be obtained through A; it can also mean that A indirectly indicates B, for example, A indicates C, and B can be obtained through C; it can also mean that there is an association between A and B.

[0056] In the description of the embodiments of the present application, the term "corresponding" may indicate a direct or indirect correspondence between the two, or an association relationship between the two, or a relationship between indication and being indicated, configuration and being configured, etc.

[0057] To facilitate understanding of the technical solutions of the embodiments of the present application, the following uses an AI (Artificial Intelligence)-based wireless communication solution as an example to illustrate the technical problems to be solved by the embodiments of the present application. It should be understood that the AI-based wireless communication solution is an application example of the technical solutions of the embodiments of the present application and does not limit the scope of protection of the embodiments of the present application. That is, the technical solutions of the embodiments of the present application can also be applied to other wireless communication solutions.

[0058] (1) AI-based wireless communication solutions

[0059] Currently, AI-based solutions are increasingly being used in wireless communication systems. For example, as shown in Figure 2A, CSI (Channel-State Information) feedback is implemented based on artificial intelligence. Among them, through the introduction of AI encoders and AI decoders, compressed feedback of AI-based CSI information is implemented. For another example, as shown in Figure 2B, the channel estimation problem is solved based on AI, and high-performance estimation of a given channel is achieved through the AI ​​channel estimator. For another example, as shown in Figure 2C, the positioning problem is solved based on AI, and high-precision positioning results are obtained by relying on positioning channel information through an AI-based positioning algorithm. For another example, as shown in Figure 2D, the beam management problem is solved based on AI, and through an AI-based beam management algorithm, based on known beam information, preferred or more refined beam information is obtained, or predictions of beam information at future moments are obtained.

[0060] 2. Performance Evaluation of AI / ML Solutions

[0061] In current research, the performance evaluation of AI / ML solutions is primarily defined based on the inference performance of the AI / ML solutions. For example, for CSI compression and recovery solutions, the CSI recovery accuracy achievable by a specific AI / ML solution under specific compression feedback bit conditions can be used as a performance evaluation indicator for the solution. For example, the difference or similarity between ideal CSI information or the CSI information to be compressed and the CSI information obtained through compression and recovery can be used as a performance evaluation indicator. For CSI prediction solutions, similar to CSI compression and recovery solutions, the achievable CSI prediction accuracy of a specific AI / ML solution can be used as a performance evaluation indicator for the solution, for example, by evaluating the difference or similarity between ideal CSI information or target CSI information and the predicted CSI information. In the above-mentioned solutions, the difference evaluation between CSI information is, for example, the NMSE (Normalized Mean Square Error) or MSE (Mean Squared Error) evaluation between CSI information. The similarity evaluation between CSI information is, for example, based on GCS or SGCS.

[0062] Current research on AI-based wireless communication solutions has demonstrated some performance gains and effective results. However, it should be noted that practical applications of these solutions require consideration of how they address performance assurance issues. For example, for AI / ML-based CSI feedback or prediction, AI / ML-based beam management, beam prediction, AI / ML-based positioning, AI / ML-based channel estimation, AI / ML-based mobility management, AI / ML-based resource management, AI / ML-based encoding and decoding, and AI / ML-based modulation and demodulation, in actual systems, model performance evaluation is required to trigger corresponding model management processes, such as switching, updating, and fallback of the aforementioned solutions.

[0063] In the above evaluation process, for a certain type of solution, a simple approach is to build a corresponding evaluation mechanism, such as an evaluation window, to obtain a stable evaluation result through multiple evaluations, and then trigger the subsequent corresponding process. However, it should be pointed out here that if a fixed evaluation method (such as evaluation window, evaluation sample) is adopted under different usage conditions, there will be problems such as excessive evaluation overhead (resource overhead, time overhead) or poor evaluation performance. For example, for the same CSI compression and recovery solution, if the evaluation method in a high-speed scenario is the same as that in a low-speed scenario, or the evaluation method in a scenario where the channel environment is relatively stable and the channel environment is relatively complex and changing, it may cause problems such as excessive evaluation overhead or poor evaluation performance.

[0064] The technical solutions of the embodiments of the present application are mainly intended to solve the above-mentioned problems.

[0065] FIG3 is a schematic flow chart of a performance evaluation method according to an embodiment of the present application. The method can optionally be applied to the system shown in FIG1 , but is not limited thereto. The method includes at least part of the following contents.

[0066] S310: Under a first condition, the first device evaluates the performance of the first solution using a first evaluation method corresponding to the first condition.

[0067] Exemplarily, the first condition may be a condition that affects the performance evaluation effect of the first solution. For example, the first condition may be a condition used to characterize the scenario environment in which the first solution is used. Since the same evaluation method has different effects on the first solution in different scenario environments, setting the first condition enables the first device to determine that it is in a specific scenario environment based on the first condition, and thus use a specific evaluation method to evaluate the performance of the first solution.

[0068] Similarly, under the second condition, the first device uses a second evaluation method corresponding to the second condition to evaluate the performance of the first solution. That is, different conditions can correspond to different evaluation methods. In other words, the first device can adjust the evaluation method for the first solution according to different conditions.

[0069] Exemplarily, the first device may include a network device or a terminal device.

[0070] Exemplarily, the first scheme may include any wireless communication solution, such as a CSI feedback scheme, a beam management scheme, a positioning scheme, a channel estimation scheme, a mobility management scheme, a resource management scheme, a coding and decoding scheme, a modulation and demodulation scheme, etc. in a wireless communication system.

[0071] Exemplarily, the first evaluation method may be an evaluation mechanism based on one or more information such as an evaluation window, the number of evaluation samples, the length of evaluation time, and the type of evaluation.

[0072] For example, for a CSI feedback scheme, a first evaluation method may be to collect K CSI feedback information within an evaluation time of length L in a first evaluation window, and evaluate the CSI feedback scheme based on the difference between each of the K CSI feedback information and the corresponding ideal CSI, where K is an integer greater than or equal to 1, and L is a positive number. It can be seen that the first evaluation method in this example is an evaluation mechanism based on the first evaluation window, the number of evaluation samples K, the evaluation time length L, and an evaluation type based on difference comparison.

[0073] Exemplarily, evaluating the performance of the first solution may include determining whether the processing result of the first solution meets the requirements, for example, determining a performance evaluation index based on the processing result of the first solution, and determining whether the performance evaluation index is greater than a threshold value or less than a threshold value, etc.

[0074] According to the performance evaluation method of the above embodiment, under a first condition, the first device uses a first evaluation method corresponding to the first condition to evaluate the performance of the first solution. That is, corresponding evaluation methods are used for different conditions. Because the impact of different conditions on the evaluation of the first solution is taken into account, a more appropriate evaluation method can be used to evaluate the first solution, thereby improving evaluation performance and controlling evaluation overhead.

[0075] In some embodiments, the first solution includes an AI-based wireless communication solution. Optionally, the first solution may include a wireless communication solution based on machine learning (ML). Since the AI / ML model will achieve good performance for familiar scenarios (i.e., scenarios that have been encountered or similar scenarios during the training phase), but the performance for unfamiliar scenarios may be low, the performance evaluation of the AI / ML-based wireless communication solution is particularly important. According to the above embodiment, the accuracy of the performance evaluation of the AI / ML-based wireless communication solution can be improved, so that the AI / ML solution can be reasonably applied and the AI / ML capabilities can be fully utilized to improve the performance of the wireless communication system.

[0076] In some embodiments, the first solution includes at least one of the following:

[0077] CSI feedback program;

[0078] CSI prediction scheme;

[0079] beam management schemes;

[0080] Beam prediction scheme;

[0081] Positioning plan;

[0082] Channel estimation scheme;

[0083] Mobility management solutions;

[0084] resource management plans;

[0085] Codec scheme;

[0086] Modem scheme.

[0087] Exemplarily, the first solution may include an alternative to a traditional wireless communication solution, which is implemented based on AI / ML. For example, the first solution may include an AI / ML-based CSI feedback solution, an AI / ML-based beam management solution, an AI / ML-based positioning solution, an AI / ML-based channel estimation solution, an AI / ML-based mobility management solution, an AI / ML-based resource management solution, an AI / ML-based encoding and decoding solution, or an AI / ML-based modulation and demodulation solution.

[0088] For example, the first solution may include an AI / ML-based information prediction solution to assist in communication decision-making, such as an AI / ML-based CSI prediction solution, a beam prediction solution, or a service change prediction solution.

[0089] It can be understood that in actual applications, the first solution may also include two or more of the above solutions. For example, the first evaluation method is to evaluate the first solution based on the reference signal measurement result of the first time window. The first solution may include an AI / ML-based CSI feedback solution and a beam management solution, that is, under the first condition (for example, the same scenario environment), the CSI feedback solution and the beam management solution are both evaluated based on the reference signal measurement result of the first time window. In specific implementation, the specific content of the first solution can be set according to the actual scenario requirements, solution characteristics, or according to the system / protocol agreement, and they are not listed one by one here.

[0090] In some embodiments, the first condition is related to at least one of the following information A to G:

[0091] A.Scene environment of the first option.

[0092] For example, the scenario environment can be determined based on one or more information including channel complexity, channel variability, frequency selection status, multipath status, the cell in which the first device is located, and the scope of use of the solution. In other words, different scenario environments in which the first solution is used due to conditions such as channel complexity, variability, frequency selection status, multipath status, and other environmental and scenario information, as well as different cells and different scopes of use of the first solution, can serve as the first condition or one of the elements of the first condition.

[0093] For example, the first condition may include determining based on a preset rule that the channel complexity is high and / or the channel variability is high. For another example, the first condition may include determining based on a preset rule that the channel complexity is low and / or the channel variability is high.

[0094] B. Configuration information of a first measurement behavior; wherein the first measurement behavior is used to evaluate the first solution.

[0095] Specifically, for the first scheme that needs to perform a measurement action (for example, it is necessary to measure a specific signal or it is necessary to extract specific information through measurement) to complete the performance evaluation, the first condition can be set to be related to the configuration information of the first measurement behavior used to evaluate the first scheme. For example, when the first scheme is an AI-based CSI compression, recovery, prediction and other use cases, the performance evaluation of the first scheme needs to make CSI measurement-related measurements, such as measurements based on CSI-RS (Channel State Information Reference Signal); when the first scheme is an AI-based beam management and other use cases, the performance evaluation of the first scheme needs to make beam measurement-related measurements; when the first scheme is an AI-based positioning scheme, the performance evaluation of the first scheme needs to make positioning-related measurements, such as measurements based on PRS (Positioning Reference Signal) and measurements based on CSI-RS.

[0096] Exemplarily, the configuration information for the first measurement behavior may include one or more of the following: the number, period, time domain density, frequency domain density, interval, number of ports, number of beams, and number of reference sites of measurement reference signals. That is, for a specific first solution, different measurements may affect measurement accuracy due to differences in measurement reference signals, such as the number, period, time domain density, frequency domain density, interval, number of ports, number of beams, and number of reference sites of measurement reference signals. These different measurement configurations or different measurement accuracies may serve as the first condition or one of the elements of the first condition.

[0097] For example, the first condition may include the number, period, time domain density, frequency domain density, interval, number of ports, number of beams or number of reference sites of the measurement reference signal being greater than a preset threshold value or less than a preset threshold value, etc.

[0098] C. The measurement opportunity corresponding to the first measurement behavior.

[0099] Here, the measurement opportunity corresponding to the first measurement behavior refers to a measurement opportunity available for the first measurement behavior. Specifically, if the execution of the first measurement behavior requires a measurement opportunity, the first condition may also be set to be related to the measurement opportunity available for the first measurement opportunity.

[0100] Exemplarily, the measurement opportunity corresponding to the first measurement behavior may include a measurement window and / or a measurement interval (Gap) of a reference signal, wherein the measurement window of the reference signal includes a measurement window of an SSB (e.g., SMTC), a measurement window of a CSI-RS, a measurement window of a PRS, a measurement window of a DMRS, etc. Alternatively, the measurement opportunity corresponding to the first measurement behavior may include whether the first measurement behavior conflicts with other measurements, whether LBT (Listen Before Talk) is successful (affecting the sending or receiving of the measurement signal of the first measurement behavior), and the ability of the first device to process the first measurement behavior.

[0101] That is to say, due to the impact of the measurement window of the reference signal on the first measurement behavior used for the performance evaluation of the first scheme, the impact of the measurement Gap on the first measurement behavior, the impact of whether the first measurement behavior conflicts with other measurements on the first measurement behavior, the impact of the success or failure of LBT on the first measurement behavior, the impact of the measurement processing capability of the first device, etc., the differences in available measurement opportunities can be used as the first condition or one of the elements of the first condition.

[0102] For example, the first condition may include that the size of the measurement window of the reference signal is greater than a preset threshold, or the first measurement behavior does not conflict with other measurements, or the first condition may include that LBT is successful, etc.

[0103] D. The moving speed of the terminal device related to the first scenario.

[0104] For example, the terminal device associated with the first solution may be the first device, or may be another device other than the first device. For example, if the first device is a terminal device, the first condition may be related to the moving speed of the terminal device. For another example, if the first device is a network device, the first condition may be related to the moving speed of the terminal device scheduled by the network device.

[0105] Exemplarily, the moving speed may be represented by a moving speed level, such as a level corresponding to ultra-high speed, high speed, medium speed, low speed, etc., which may be used as the first condition or one of the elements of the first condition.

[0106] For example, the first condition may include that the moving speed of the terminal device is greater than a preset threshold value or less than a preset threshold value, etc.

[0107] E. The processing capability of the first device.

[0108] Exemplarily, the processing capability may include the AI / ML processing capability of the first device or the capability of the first device to process information to be evaluated (such as measurement result processing, evaluation data acquisition, evaluation result calculation, etc.).

[0109] That is to say, if there are differences in the AI / ML processing capabilities of the first device, or the capabilities of the first device to process the information to be evaluated, the above different capabilities will affect the evaluation window size, duration, etc. involved in the evaluation plan. Therefore, the difference in processing capabilities can be used as the first condition or one of the elements of the first condition.

[0110] For example, the first condition may include determining, based on a preset rule, whether the AL / ML model processing capability of the first device is strong or weak.

[0111] F. The accuracy of a single evaluation of the first solution by the first device.

[0112] When evaluating the first solution, even for the same problem and using the same evaluation method, the accuracy of a single evaluation may vary across scenarios, devices, data, and models. For example, when evaluating CSI feedback, the accuracy of a single evaluation on device 1 may be 90% reliable, while the accuracy of a single evaluation on device 2 may be 80%. This difference in accuracy between the two evaluations can serve as the first condition or a factor of the first condition.

[0113] For example, the first condition may include that the accuracy of a single evaluation of the first solution by the first device is greater than a preset threshold value or less than a preset threshold value, etc.

[0114] G. Evaluation type of the first option.

[0115] Exemplarily, the evaluation types may include evaluation based on label comparison, evaluation based on feature statistics of input information / output information, evaluation based on distribution statistics of input information / output information, evaluation based on equivalent substitution model, etc.

[0116] Different evaluation types will affect the evaluation results of the same evaluation method. Therefore, different evaluation types mentioned above can be used as the first condition or one of the elements of the first condition.

[0117] For example, the first condition may include that the evaluation type of the first solution is an evaluation based on label comparison. Alternatively, the first condition may include that the evaluation type of the first solution is an evaluation based on feature statistics of input information.

[0118] It can be understood that in actual applications, the first condition may also be related to two or more of the above information. For example, the first condition is to determine, based on a preset rule, that the channel complexity is high and the moving speed of the terminal device is greater than a preset threshold value. For another example, the first condition is that the first measurement behavior does not conflict with other measurements and the moving speed of the terminal device is less than a preset threshold value. During specific implementation, the specific content of the first condition can be set according to the actual scenario requirements, solution characteristics, or according to the system / protocol agreement, and they are not listed one by one here.

[0119] In some embodiments, the first condition corresponds to the first information of the first evaluation method. That is, the first condition corresponds to the first evaluation method because the first condition corresponds to specific first information and thus corresponds to the first evaluation method. That is, different conditions correspond to different first information and thus correspond to different evaluation methods.

[0120] In some embodiments, the first information includes at least one of the following:

[0121] Type of assessment;

[0122] Evaluation window size;

[0123] Evaluation sample size;

[0124] Length of time for evaluation.

[0125] In other words, different evaluation methods may refer to one or more differences such as different evaluation types, different evaluation window sizes, different numbers of evaluation samples, and different evaluation time lengths.

[0126] For example, assuming that the first information includes the evaluation type and the evaluation window size, the first condition can correspond to an evaluation mechanism based on label comparison within an evaluation window of length L, and other conditions such as the second condition can correspond to an evaluation mechanism based on label comparison within an evaluation window of length L1, or the second condition can correspond to an evaluation mechanism based on distribution statistics within an evaluation window of length L1.

[0127] For another example, assuming that the first information includes the evaluation window size and the number of evaluation samples, the first condition may correspond to an evaluation mechanism with a number of samples of K within an evaluation window of length L, and the second condition may correspond to an evaluation mechanism with a number of samples of K1 within an evaluation window of length L1.

[0128] From the above examples, it can be seen that the first information may include any one or more of the above information. During specific implementation, the specific content of the first information can be set according to actual scenario requirements, solution characteristics, or system / protocol agreements, which are not listed one by one here.

[0129] In some embodiments, the first condition may directly correspond to a set value of the first information, or different conditions may correspond to different values ​​of the first information. For example, the first condition may correspond to an evaluation window size of 1ms (milliseconds), while the second condition may correspond to an evaluation window size of 2ms. In other words, in some embodiments, adjusting the evaluation method based on different conditions may refer to directly adjusting the first information in the evaluation method.

[0130] In some embodiments, the first condition may correspond to a method for adjusting the first information, or different conditions may correspond to different methods for adjusting the first information. Specifically, the first information of the first evaluation method is obtained by adjusting the first information of the second evaluation method based on the first condition. In other words, adjusting the evaluation method based on different conditions may be based on a defined, configured, or currently used second evaluation method, and then adjusting the first information of the second evaluation method according to the first condition to obtain the first information of the first evaluation method corresponding to the first condition.

[0131] In some embodiments, the first information of the first evaluation method is obtained by adjusting the first information of the second evaluation method based on the scaling factor or correction coefficient corresponding to the first condition. In other words, the first condition corresponds to a specific scaling factor or correction coefficient, and thus corresponds to a specific adjustment method. Therefore, based on the second evaluation method, the first information of the second evaluation method can be adjusted according to the specific scaling factor or correction coefficient to obtain the first information of the first evaluation method corresponding to the first condition.

[0132] Based on the above embodiments for the first solution, first condition, first evaluation method, and the relationship between different first conditions and first evaluation methods, some application examples are provided below. It should be noted that only a few examples are provided here, and the impact of different first conditions on evaluation methods under more different first solutions is not fully enumerated. However, in actual operation, the impact of different first conditions on different first evaluation methods can be applied to the performance evaluation of different solutions.

[0133] Application Example 1:

[0134] For different environments and scenarios, for example, in environments and scenarios with high channel complexity, rapid changes, strong frequency selection, and rich multipath characteristics, a larger evaluation window, longer evaluation time, or more evaluation samples can be adjusted or updated. For another example, in environments and scenarios with low channel complexity, slow changes, weak frequency selection, and a simpler multipath characteristic, a smaller evaluation window, shorter evaluation time, or fewer evaluation samples can be adjusted or updated.

[0135] Specifically:

[0136] For example, when the first scheme is an AI / ML-based CSI feedback scheme, under a first condition (e.g., a first scenario, such as unstable channel changes and a complex environment), the first device expands the evaluation window by changing the window length from L to L1; or increasing the length L2 based on the given evaluation window length L; or expanding the given evaluation window L by a scaling factor k to obtain a new evaluation window L*(1+k), where k is greater than or equal to 0; or expanding the given evaluation window L by a scaling factor m to obtain a new evaluation window L*m, where m is greater than 1. Performance evaluation of the AI / ML-based CSI feedback scheme is performed based on the expanded evaluation window.

[0137] Alternatively, when the first scheme is an AI / ML-based CSI feedback scheme, under the first condition (for example, the second scenario, such as stable channel changes and a relatively simple environment), the first device reduces the evaluation window by changing the window length from L to L3; or reducing the length of L4 based on a given evaluation window length L; or reducing the given evaluation window L by a scaling factor k to obtain a new evaluation window L*(1-k), where k is greater than or equal to 0 and k is less than 1; or reducing the given evaluation window L by a scaling factor m to obtain a new evaluation window L*m, where m is less than 1.

[0138] Application Example 2:

[0139] For different measurement configurations and measurement accuracies, for example, if the measurement accuracy is high, fewer evaluation samples, or a shorter evaluation time, or a shorter evaluation window may be used in the first evaluation scheme. Conversely, if the measurement accuracy is low, more evaluation samples, or a longer evaluation time, or a longer evaluation window may be used in the first evaluation scheme.

[0140] Specifically:

[0141] For example, when the first scheme is an AI / ML-based beam management scheme (such as beam prediction), under a first condition (for example, the measurement accuracy of CSI-RS or other reference signals is low, for example, due to a small number of measurement reference signals, low time domain density, low frequency domain density, small number of ports, small number of beams, and other configuration differences, or due to the limited measurement capability of the first device itself, resulting in low measurement accuracy), the first device increases the number of evaluation samples by increasing the sample size N1 on the basis of a given evaluation sample number N; or by scaling factor k on the basis of a given evaluation sample number N, expanding to obtain a new evaluation sample size N*(1+k), where k is greater than or equal to 0; or by scaling factor m on the basis of a given evaluation sample size N, expanding to obtain a new evaluation sample size N*m, where m is greater than 1. The performance evaluation of the AI / ML-based beam management scheme is performed based on the expanded evaluation sample size.

[0142] Alternatively, when the first scheme is an AI / ML-based beam management scheme (such as beam prediction), under a first condition (for example, the measurement accuracy of CSI-RS or other reference signals is high, for example, due to a large number of measurement reference signals, a high time domain density, a high frequency domain density, a large number of ports, a large number of beams, and other configuration differences, or due to the first device's own strong measurement capability, resulting in high measurement accuracy), the first device may reduce the number of evaluation samples by reducing the N2 sample size based on a given evaluation sample number N; or by scaling the given evaluation sample number N by a scaling factor k to obtain a new evaluation sample size N*(1-k), where k is less than 1 and greater than or equal to 0; or by scaling the given evaluation sample size N by a scaling factor m to obtain a new evaluation sample size N*m, where m is less than 1; used for performance evaluation of the AI / ML-based beam management scheme.

[0143] Application Example 3:

[0144] For different measurement opportunities, for example, if there are fewer measurement opportunities, a longer evaluation time, a longer evaluation window, and a lower evaluation reliability may be used in the first evaluation plan. Conversely, if there are more measurement opportunities, a shorter evaluation time and a shorter evaluation window may be used in the first evaluation plan.

[0145] Specifically:

[0146] For example, when the first solution is a CSI feedback, beam management, channel estimation, positioning, etc. solution based on AI / ML, when performing the performance evaluation of the first solution, corresponding SSB, CSI-RS, PT-RS, DMRS, PRS, etc. measurements need to be performed. Under the first condition (the measurement of the reference signal used for the performance evaluation of the first solution conflicts with other measurement behaviors (such as mobility measurement, measurement of different tasks, measurement of different reference signals), or measurement configuration (such as SMTC, measurement GAP, etc.), the first device increases the evaluation time, or increases the number of evaluation samples, or increases the evaluation window. The method adopted is to increase the L1 length based on the given evaluation window length (or duration) L; or based on the given evaluation window L, by scaling factor k, a new evaluation window L*(1+k) is obtained, where k is greater than or equal to 0; or based on the given evaluation window L, by scaling factor m, a new evaluation window L*m is obtained, where m is greater than 1; used for performance evaluation of CSI feedback, beam management, channel estimation, positioning, etc. solutions based on AI / ML.

[0147] Alternatively, the method may be to increase the number of evaluation samples N by N1; or to expand the number of evaluation samples N by a scaling factor k to obtain a new evaluation sample size N*(1+k), where k is greater than or equal to 0; or to expand the number of evaluation samples N by a scaling factor m to obtain a new evaluation sample size N*m, where m is greater than 1. Performance evaluation of AI / ML-based CSI feedback, beam management, channel estimation, positioning, and other solutions can be performed based on the expanded evaluation window or evaluation sample size.

[0148] Application Example 4:

[0149] For different single evaluation accuracies, for example, if the single evaluation accuracy is low, more evaluation samples, or a longer evaluation time, or a longer evaluation window can be used in the first evaluation scheme. Conversely, if the single evaluation accuracy is high, fewer evaluation samples, or a shorter evaluation time, or a shorter evaluation window can be used in the first evaluation scheme.

[0150] Specifically:

[0151] For example, when the first solution is an AI / ML-based positioning solution, under the first condition (for example, the single evaluation accuracy of the first AI positioning model is low, for example, due to a higher probability of a larger positioning error, etc.), the first device may increase the number of evaluation samples by increasing the sample size N1 based on the given number of evaluation samples N; this is used for performance evaluation of the AI / ML-based positioning solution.

[0152] Alternatively, when the first solution is an AI / ML-based positioning solution, under a first condition (for example, the single evaluation accuracy of the first AI positioning model is higher, for example, the probability of a larger positioning error is smaller, etc.), the first device may reduce the number of evaluation samples by reducing the sample size N1 based on a given number of evaluation samples N; this is used for performance evaluation of the AI / ML-based positioning solution.

[0153] Application Example 5:

[0154] For different speed information, for example, based on the degree of scene change caused by speed, for example, high speed causes rapid scene changes, while low speed causes slow scene changes, then high speed requires more evaluation samples, or a longer evaluation time, or a longer evaluation window in the first evaluation scheme to obtain accurate evaluation results. Low speed scenes can use fewer evaluation samples, or a shorter evaluation time, or a shorter evaluation window in the first evaluation scheme.

[0155] Specifically:

[0156] For example, when the first scheme is CSI feedback, beam management, channel estimation and other schemes based on AI / ML, under the first condition (for example, UE moves at high speed), the first device may increase the number of evaluation samples by increasing the sample size N1 based on the given number of evaluation samples N; or adjusting the evaluation window W to the evaluation window W1, where W1 is greater than W; and used for performance evaluation of CSI feedback, beam management, channel estimation and other schemes based on AI / ML.

[0157] Alternatively, when the first scheme is an AI / ML-based CSI feedback, beam management, channel estimation and other schemes, under the first condition (for example, UE moves at low speed), the first device can reduce the number of evaluation samples by reducing the N2 sample size based on a given number of evaluation samples N; or adjusting the evaluation window W to the evaluation window W2, where W2 is less than W; and used for performance evaluation of AI / ML-based CSI feedback, beam management, channel estimation and other schemes.

[0158] Application Example 6:

[0159] For different speed information, for example, considering the real-time requirements for solution updates due to speed, for example, high speeds lead to rapid scene changes, and the demand for rapid solution updates is also strong; low speeds lead to slow scene changes, and the demand for rapid solution updates is also weak. In this case, high-speed scenarios can use fewer evaluation samples, or a shorter evaluation time, or a shorter evaluation window in the first evaluation solution. Conversely, low-speed scenarios can use more evaluation samples, or a longer evaluation time, or a longer evaluation window in the first evaluation solution.

[0160] Specifically:

[0161] For example, when the first scheme is CSI feedback, beam management, channel estimation and other schemes based on AI / ML, under the first condition (for example, UE moves at high speed), the first device can reduce the number of evaluation samples by reducing the number of samples N2 based on the given number of evaluation samples; or adjusting the evaluation window W to the evaluation window W2, where W2 is less than W; and used for performance evaluation of CSI feedback, beam management, channel estimation and other schemes based on AI / ML.

[0162] Alternatively, when the first scheme is a CSI feedback, beam management, channel estimation and other schemes based on AI / ML, under the first condition (for example, UE moves at low speed), the first device may increase the number of evaluation samples by increasing the sample size N1 based on the given number of evaluation samples N; or adjusting the evaluation window W to the evaluation window W1, where W1 is greater than W; and used for performance evaluation of CSI feedback, beam management, channel estimation and other schemes based on AI / ML.

[0163] Application Example 7:

[0164] For different device processing capabilities, for example, if the UE has stronger capabilities, fewer evaluation samples, or a shorter evaluation time, or a shorter evaluation window can be used in the first evaluation scheme. Conversely, if the UE has weaker capabilities, more evaluation samples, or a longer evaluation time, or a longer evaluation window can be used in the first evaluation scheme.

[0165] Specifically:

[0166] For example, when the first scheme is CSI feedback, beam management, channel estimation, positioning, encoding and decoding, modulation and demodulation and other schemes based on AI / ML, under the first condition (for example, the UE has strong performance evaluation capabilities, for example, the UE has a model that can be used for performance evaluation, and the model has good performance evaluation capabilities), the first device can reduce the number of evaluation samples by reducing the number of N1 samples based on a given number of evaluation samples N; or adjusting the evaluation window W to the evaluation window W1, where W1 is less than W; used for performance evaluation of CSI feedback, beam management, channel estimation, positioning, encoding and decoding, modulation and demodulation and other schemes based on AI / ML.

[0167] Alternatively, when the first scheme is an AI / ML-based CSI feedback, beam management, channel estimation, positioning, encoding and decoding, modulation and demodulation scheme, under the first condition (for example, the UE does not have strong performance evaluation capabilities), the first device needs to increase the number of evaluation samples by increasing the sample size N2 on the basis of a given number of evaluation samples N; or adjusting the evaluation window W to the evaluation window W2, where W2 is greater than W; used for performance evaluation of AI / ML-based CSI feedback, beam management, channel estimation, positioning, encoding and decoding, modulation and demodulation schemes.

[0168] Different implementations are provided below for the configuration of the first condition and the first evaluation method.

[0169] In some embodiments, the above-mentioned performance evaluation method further includes: the first device receiving first indication information from the second device, wherein the first indication information is used to indicate a first condition or indicate that the first condition is met.

[0170] For example, the second device may send first indication information to the first device to indicate the specific content of the first condition corresponding to the first evaluation method, so that the first device can independently determine whether the first condition is met and use the first evaluation method under the first condition. Alternatively, the second device may determine whether the second condition is met. If the second device determines that the first condition is met, the second device sends the first indication information to the first device, so that the first device uses the first evaluation method under the first condition that is currently met.

[0171] In some embodiments, if the first device is a terminal device and the second device is a network device, the first indication information can be carried by a broadcast message, MIB (Master Information Block), SIB (System Information Block), downlink message in initial access, DCI (Downlink Control Information), MAC CE (MAC Control Element), RRC (Radio Resource Control) message or RRC reconfiguration message.

[0172] In some embodiments, if the first device is a network device and the second device is a terminal device, the first indication information can be carried by an uplink message in the initial access, UE capability reporting, UCI (Uplink Control Information) or RRC message.

[0173] In some embodiments, the performance evaluation method may further include: the first device determining a first condition based on a first rule. Optionally, the first rule is a pre-configured rule for determining the first condition.

[0174] In some embodiments, the first device includes a first terminal device, and the first rule is pre-agreed or configured by a first network device. In other embodiments, the first device includes a first network device, and the first rule is pre-agreed.

[0175] In some embodiments, the performance evaluation method may further include: the first device receiving second indication information from the second device, wherein the second indication information is used to indicate the first evaluation method. In other words, the first evaluation method is determined by the second device and indicated to the first device.

[0176] It should be noted that the second indication information is used to indicate the first evaluation method. When the second device determines that the first condition is satisfied, the second indication information may be sent to the first device to indicate the first evaluation method currently being used. Alternatively, the second device may send the second indication information to the first device to indicate the first evaluation method corresponding to the first condition, and the first device may determine whether to use the first evaluation method based on the current condition.

[0177] In some embodiments, if the first device is a terminal device and the second device is a network device, the second indication information can be carried by a broadcast message, MIB, SIB, a downlink message in the initial access, DCI, MAC CE, RRC message or RRC reconfiguration message.

[0178] In some embodiments, if the first device is a network device and the second device is a terminal device, the second indication information can be carried by an uplink message in the initial access, a UE capability report, a UCI or an RRC message.

[0179] In some embodiments, the performance evaluation method may further include: the first device determining the first evaluation method based on a second rule. Optionally, the second rule is a pre-configured rule for determining the first evaluation method.

[0180] In some embodiments, the first device includes a second terminal device, and the second rule is pre-agreed or configured by the second network device. In other embodiments, the first device includes a second network device, and the second rule is pre-agreed.

[0181] During specific implementation, any of the above embodiments may be selected or combined according to actual needs. The following describes the method for determining the first condition and the method for determining the first evaluation method from the perspective of interaction between the terminal device and the network device. Specifically, the method for determining the first condition and / or the method for determining the first evaluation method may include one or more of the following methods:

[0182] Method 1: As shown in Figure 4A, in this method, the performance evaluation of the first scheme is implemented on the UE side, and the first condition is indicated to the UE by the network device, for example, through broadcast, MIB, SIB, downlink message in initial access, DCI, MAC CE, RRC message, RRC reconfiguration message, etc. to indicate the UE's current first condition information.

[0183] Mode 2: In this mode, the performance evaluation of the first solution is implemented on the UE side, and the first condition is determined by the UE, for example, through a protocol agreement, or based on a first condition determination rule preconfigured by the network.

[0184] Method 3: As shown in Figure 4B, in this method, the performance evaluation of the first scheme is implemented on the UE side, and the network instructs the UE to adopt the first evaluation method, for example, through broadcast, MIB, SIB, downlink message in initial access, DCI, MAC CE, RRC message, RRC reconfiguration message, etc.

[0185] Mode 4: In this mode, the performance evaluation of the first solution is implemented on the UE side, and the first evaluation mode is determined by the UE, for example, by a protocol agreement, or based on a first evaluation mode determination rule preconfigured by the network.

[0186] Method 5: As shown in Figure 4C, in this method, the performance evaluation of the first scheme is implemented on the network side, and the first condition is reported by the UE to the network device, for example, by reporting the current first condition information through uplink messages in initial access, UE capability reporting, UCI, RRC messages, etc.

[0187] Mode 6: In this mode, the performance evaluation of the first solution is implemented on the network side, and the first condition is determined by the network device, for example, through a protocol agreement.

[0188] Method 7: As shown in Figure 4D, in this method, the performance evaluation of the first scheme is implemented on the network side, and the UE reports the first evaluation method adopted to the network device, for example, through the uplink message in the initial access, UE capability reporting, UCI, RRC message, etc.

[0189] Mode 8: In this mode, the performance evaluation of the first solution is implemented on the network side, and the first evaluation mode is determined by the network device, for example, by a protocol agreement.

[0190] FIG5 is a schematic flow chart of a performance evaluation method according to another embodiment of the present application. The method can optionally be applied to the system shown in FIG1 , but is not limited thereto. The method includes at least part of the following contents.

[0191] S510. The second device sends third indication information to the first device; wherein the third indication information is used to enable the first device to evaluate the performance of the first solution under the first condition by using a first evaluation method corresponding to the first condition.

[0192] Exemplarily, the third indication information may be used by the first device to determine whether the first condition is met, and may also be used by the first device to determine the first evaluation method.

[0193] In some embodiments, the third indication information is used to indicate the first condition or to indicate that the first condition is satisfied. Specifically, the third indication information may include the first indication information in the aforementioned embodiment. Optionally, the second device may send the third indication information to the first device to indicate the specific content of the first condition corresponding to the first evaluation method, so that the first device can independently determine whether the first condition is satisfied and use the first evaluation method under the first condition. Alternatively, the second device may determine whether the second condition is satisfied. If the second device determines that the first condition is satisfied, the second device sends the first indication information to the first device, so that the first device uses the first evaluation method under the currently satisfied first condition.

[0194] In some embodiments, the third indication information is used to indicate a first rule, and the first rule is used by the first device to determine the first condition.

[0195] In some embodiments, the third indication information is used to indicate the first evaluation method. Specifically, the third indication information may include the second indication information in the aforementioned embodiment. Optionally, the second indication information may be used to indicate the first evaluation method currently in use, i.e., the second device may send the second indication information to the first device under the first condition to cause the terminal device to use the first evaluation method. Alternatively, the second indication information may be used to indicate the first evaluation method corresponding to the first condition, in which case the first device may use the first evaluation method under the first condition.

[0196] In some implementations, the third indication information is used to indicate a second rule, and the second rule is used by the first device to determine the first evaluation method.

[0197] In some embodiments, if the first device is a terminal device and the second device is a network device, the third indication information can be carried by a broadcast message, MIB, SIB, a downlink message in initial access, DCI, MAC CE, RRC message or RRC reconfiguration message.

[0198] In some embodiments, if the first device is a network device and the second device is a terminal device, the third indication information can be carried by an uplink message in the initial access, a UE capability report, a UCI or an RRC message.

[0199] It can be seen that according to the technical solution of the embodiment of the present application, the first device evaluates the first solution under the first condition using the first evaluation method corresponding to the first condition. Different first conditions correspond to different first evaluation methods. Different first conditions are used to confirm the adjustment and update of different first evaluation methods. It can be achieved under different conditions that affect the evaluation of the first solution. Avoid unnecessary evaluation overhead, such as resource overhead and evaluation time overhead, while also ensuring that the evaluation effect is effective and accurate. When more evaluation overhead is required under the first condition, the corresponding first evaluation method is used to ensure the effectiveness of the evaluation. When excessive evaluation overhead is not required under the first condition, the corresponding first evaluation method is used to avoid unnecessary evaluation overhead.

[0200] FIG6 is a schematic block diagram of a first device 600 according to an embodiment of the present application. The first device 600 may include:

[0201] The first processing module 610 is configured to evaluate the performance of the first solution under a first condition by using a first evaluation method corresponding to the first condition.

[0202] In one embodiment, the first solution includes an artificial intelligence-based wireless communication solution.

[0203] In one embodiment, the first scheme includes at least one of the following: a CSI feedback scheme; a CSI prediction scheme; a beam management scheme; a beam prediction scheme; a positioning scheme; a channel estimation scheme; a mobility management scheme; a resource management scheme; a coding and decoding scheme; and a modulation and demodulation scheme.

[0204] In one embodiment, the first condition is related to at least one of the following:

[0205] The scenario environment of the first solution;

[0206] Configuration information of a first measurement behavior; wherein the first measurement behavior is used to evaluate the first solution;

[0207] a measurement opportunity corresponding to the first measurement behavior;

[0208] The moving speed of the terminal device related to the first solution;

[0209] the processing capability of the first device;

[0210] the accuracy of a single evaluation of the first solution by the first device;

[0211] The evaluation type of the first option.

[0212] In one embodiment, the first condition corresponds to first information of the first evaluation method, and the first information includes at least one of the following: evaluation type; evaluation window size; number of evaluation samples; and evaluation time length.

[0213] In one embodiment, the first information of the first evaluation method is obtained by adjusting the first information of the second evaluation method based on the first condition.

[0214] In one embodiment, the first information of the first evaluation method is obtained by adjusting the second evaluation method based on a scaling factor or a correction coefficient corresponding to the first condition.

[0215] In one embodiment, as shown in FIG7 , the first device 600 further includes a first communication module 710 configured to:

[0216] First indication information is received from a second device, wherein the first indication information is used to indicate a first condition or indicate that the first condition is met.

[0217] In one embodiment, the first processing module 610 is further configured to:

[0218] Based on the first rule, it is determined that the first condition is met.

[0219] In one embodiment, a first terminal device is included, and the first rule is pre-agreed or configured by a first network device.

[0220] In one embodiment, a first network device is included, and the first rule is pre-agreed.

[0221] In one embodiment, as shown in FIG8 , the first device further includes a second communication module 810 configured to:

[0222] Second indication information is received from a second device, wherein the second indication information is used to indicate the first evaluation method.

[0223] In one embodiment, the first processing module 610 is further configured to:

[0224] Based on the second rule, the first evaluation method is determined.

[0225] In one embodiment, a second terminal device is included, and the second rule is pre-agreed or configured by a second network device.

[0226] In one embodiment, a second network device is included, and the second rule is pre-agreed.

[0227] The first device 600 of the embodiment of the present application can implement the corresponding functions of the first device in the aforementioned method embodiment. The processes, functions, implementation methods and beneficial effects corresponding to the various modules (sub-modules, units or components, etc.) in the first device 600 can be found in the corresponding descriptions in the above-mentioned method embodiments, which will not be repeated here. It should be noted that the functions described in the various modules (sub-modules, units or components, etc.) in the first device 600 of the application embodiment can be implemented by different modules (sub-modules, units or components, etc.) or by the same module (sub-module, unit or component, etc.).

[0228] FIG9 is a schematic block diagram of a second device 900 according to an embodiment of the present application. The second device 900 may include:

[0229] The third communication module 910 is used to send third indication information to the first device; wherein the third indication information is used to enable the first device to evaluate the performance of the first solution under the first condition using the first evaluation method corresponding to the first condition.

[0230] In one embodiment, the third indication information is used to indicate the first condition or indicate that the first condition is met.

[0231] In one implementation, the third indication information is used to indicate a first rule, and the first rule is used by the first device to determine whether the first condition is satisfied.

[0232] In one implementation, the third indication information is used to indicate the first evaluation method.

[0233] In one implementation, the third indication information is used to indicate a second rule, and the second rule is used by the first device to determine the first evaluation method.

[0234] The second device 900 of the embodiment of the present application can implement the corresponding functions of the second device in the aforementioned method embodiment. The processes, functions, implementation methods and beneficial effects corresponding to the various modules (sub-modules, units or components, etc.) in the second device 900 can be found in the corresponding descriptions in the above-mentioned method embodiments, which will not be repeated here. It should be noted that the functions described by the various modules (sub-modules, units or components, etc.) in the second device 900 of the application embodiment can be implemented by different modules (sub-modules, units or components, etc.), or by the same module (sub-module, unit or component, etc.).

[0235] Figure 10 is a schematic structural diagram of a communication device 1000 according to an embodiment of the present application. The communication device 1000 includes a processor 1010, which can call and execute a computer program from a memory to enable the communication device 1000 to implement the method in the embodiment of the present application.

[0236] In one embodiment, the communication device 1000 may further include a memory 1020. The processor 1010 may call and execute a computer program from the memory 1020 to enable the communication device 1000 to implement the method in the embodiment of the present application.

[0237] The memory 1020 may be a separate device independent of the processor 1010 , or may be integrated into the processor 1010 .

[0238] In one embodiment, the communication device 1000 may further include a transceiver 1030 , and the processor 1010 may control the transceiver 1030 to communicate with other devices. Specifically, the transceiver 1030 may send information or data to other devices, or receive information or data sent by other devices.

[0239] The transceiver 1030 may include a transmitter and a receiver. The transceiver 1030 may further include an antenna, and the number of antennas may be one or more.

[0240] In one embodiment, the communication device 1000 may be the first device of the embodiment of the present application, and the communication device 1000 may implement the corresponding processes implemented by the first device in each method of the embodiment of the present application. For the sake of brevity, they will not be repeated here.

[0241] In one embodiment, the communication device 1000 may be the second device of the embodiment of the present application, and the communication device 1000 may implement the corresponding processes implemented by the second device in each method of the embodiment of the present application. For the sake of brevity, they will not be repeated here.

[0242] 11 is a schematic structural diagram of a chip 1100 according to an embodiment of the present application. The chip 1100 includes a processor 1110, which can call and execute a computer program from a memory to implement the method according to the embodiment of the present application.

[0243] In one embodiment, the chip 1100 may further include a memory 1120. The processor 1110 may call and execute a computer program from the memory 1120 to implement the method executed by the terminal device or the network device in the embodiment of the present application.

[0244] The memory 1120 may be a separate device independent of the processor 1110 , or may be integrated into the processor 1110 .

[0245] In one embodiment, the chip 1100 may further include an input interface 1130. The processor 1110 may control the input interface 1130 to communicate with other devices or chips, and specifically, may obtain information or data sent by other devices or chips.

[0246] In one embodiment, the chip 1100 may further include an output interface 1140. The processor 1110 may control the output interface 1140 to communicate with other devices or chips, and specifically, may output information or data to other devices or chips.

[0247] In one embodiment, the chip can be applied to the first device in the embodiment of the present application, and the chip can implement the corresponding processes implemented by the first device in each method of the embodiment of the present application. For the sake of brevity, it will not be repeated here.

[0248] In one embodiment, the chip can be applied to the second device in the embodiments of the present application, and the chip can implement the corresponding processes implemented by the second device in each method of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0249] The chips used in the first device and the second device may be the same chip or different chips.

[0250] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0251] The processor mentioned above may be a general-purpose processor, a digital signal processor (DSP), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or other programmable logic devices, transistor logic devices, discrete hardware components, etc. The general-purpose processor mentioned above may be a microprocessor or any conventional processor, etc.

[0252] The memory mentioned above may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM).

[0253] It should be understood that the above-mentioned memories are exemplary but not restrictive. For example, the memories in the embodiments of the present application may also be static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM RAM (DR RAM), etc. In other words, the memories in the embodiments of the present application are intended to include, but are not limited to, these and any other suitable types of memories.

[0254] FIG12 is a schematic block diagram of a communication system 1200 according to an embodiment of the present application. The communication system 1200 includes a first device 600 and a second device 900 .

[0255] The first device 600 is configured to evaluate the performance of the first solution under the first condition by using a first evaluation method corresponding to the first condition.

[0256] The second device 900 is used to send third indication information to the first device 600; wherein the third indication information is used to enable the first device 600 to evaluate the performance of the first solution under the first condition by using the first evaluation method corresponding to the first condition.

[0257] The first device 600 can be used to implement the corresponding functions implemented by the first device in the above method, and the second device 900 can be used to implement the corresponding functions implemented by the second device in the above method. For the sake of brevity, they are not described here in detail.

[0258] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function in accordance with the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server or data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode to another website, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0259] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0260] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0261] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present application should be included within the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A performance evaluation method, comprising: Under the first condition, the first device evaluates the performance of the first solution using a first evaluation method corresponding to the first condition.

2. The method according to claim 1, wherein: The first solution includes an artificial intelligence-based wireless communication solution.

3. The method according to claim 1 or 2, wherein: The first solution includes at least one of the following: Channel state information CSI feedback scheme; CSI prediction scheme; Beam management solutions; Beam prediction scheme; Positioning plan; Channel estimation scheme; Mobility management solutions; resource management scheme; Codec scheme; Modem scheme.

4. The method according to any one of claims 1 to 3, wherein: The first condition is related to at least one of the following: The scene environment of the first solution; Configuration information of a first measurement behavior; wherein the first measurement behavior is used to evaluate the first solution; a measurement opportunity corresponding to the first measurement behavior; The moving speed of the terminal device related to the first solution; the processing capability of the first device; A single evaluation accuracy of the first solution by the first device; The evaluation type of the first option.

5. The method according to any one of claims 1 to 4, wherein: The first condition corresponds to first information of the first evaluation method, and the first information includes at least one of the following: Type of assessment; Evaluation window size; Evaluation sample size; Length of time for evaluation.

6. The method according to claims 1-5, wherein: The first information of the first evaluation method is obtained by adjusting the first information of the second evaluation method based on the first condition.

7. The method according to claim 6, wherein: The first information of the first evaluation method is obtained by adjusting the first information of the second evaluation method based on a scaling factor or a correction coefficient corresponding to the first condition.

8. The method according to any one of claims 1 to 7, wherein: The method further comprises: The first device receives first indication information from the second device, wherein the first indication information is used to indicate the first condition or indicate that the first condition is met.

9. The method according to any one of claims 1 to 7, wherein: The method further comprises: The first device determines the first condition based on a first rule.

10. The method according to claim 9, wherein: The first device includes a first terminal device, and the first rule is pre-agreed or configured by a first network device.

11. The method according to claim 9, wherein: The first device includes a first network device, and the first rule is pre-agreed.

12. The method according to any one of claims 1 to 11, wherein: The method further comprises: The first device receives second indication information from a second device, wherein the second indication information is used to indicate the first evaluation method.

13. The method according to any one of claims 1 to 11, wherein: The method further comprises: The first device determines the first evaluation method based on a second rule.

14. The method according to claim 13, wherein: The first device includes a second terminal device, and the second rule is pre-agreed or configured by a second network device.

15. The method according to claim 13, wherein: The first device includes a second network device, and the second rule is pre-agreed.

16. A performance evaluation method comprising: The second device sends third indication information to the first device; wherein the third indication information is used to enable the first device to evaluate the performance of the first solution under the first condition by using the first evaluation method corresponding to the first condition.

17. The method according to claim 16, wherein: The third indication information is used to indicate the first condition or to indicate that the first condition is met.

18. The method according to claim 16, wherein: The third indication information is used to indicate a first rule, and the first rule is used by the first device to determine the first condition.

19. The method according to any one of claims 16 to 18, wherein: The third indication information is used to indicate the first evaluation method.

20. The method according to any one of claims 16 to 18, wherein: The third indication information is used to indicate a second rule, and the second rule is used by the first device to determine the first evaluation method.

21. A first device, comprising: The first processing module is used to evaluate the performance of the first solution under a first condition by using a first evaluation method corresponding to the first condition.

22. The first device according to claim 21, wherein The first solution includes an artificial intelligence-based wireless communication solution.

23. The first device according to claim 21 or 22, wherein: The first solution includes at least one of the following: CSI feedback program; CSI prediction scheme; Beam management solutions; Beam prediction scheme; Positioning plan; Channel estimation scheme; Mobility management solutions; resource management scheme; Codec scheme; Modem scheme.

24. The first device according to any one of claims 21 to 23, wherein: The first condition is related to at least one of the following: The scene environment of the first solution; Configuration information of a first measurement behavior; wherein the first measurement behavior is used to evaluate the first solution; a measurement opportunity corresponding to the first measurement behavior; The moving speed of the terminal device related to the first solution; the processing capability of the first device; A single evaluation accuracy of the first solution by the first device; The evaluation type of the first option.

25. The first device according to any one of claims 21 to 24, wherein: The first condition corresponds to first information of the first evaluation method, and the first information includes at least one of the following: Type of assessment; Evaluation window size; Evaluation sample size; Length of time for evaluation.

26. The first device according to claims 21-25, wherein: The first information of the first evaluation method is obtained by adjusting the first information of the second evaluation method based on the first condition.

27. The first device according to claim 26, wherein: The first information of the first evaluation method is obtained by adjusting the first information of the second evaluation method based on a scaling factor or a correction coefficient corresponding to the first condition.

28. The first device according to any one of claims 21 to 27, wherein: The first device further includes a first communication module, and the first communication module is used to: First indication information is received from a second device, wherein the first indication information is used to indicate the first condition or to indicate that the first condition is met.

29. The first device according to any one of claims 21 to 27, wherein: The first processing module is also used for: Based on a first rule, the first condition is determined.

30. The first device according to claim 29, wherein: The first device includes a first terminal device, and the first rule is pre-agreed or configured by a first network device.

31. The first device according to claim 29, wherein: The first device includes a first network device, and the first rule is pre-agreed.

32. The first device according to any one of claims 21 to 31, wherein: The first device further includes a second communication module, wherein the second communication module is configured to: Second indication information is received from a second device, wherein the second indication information is used to indicate the first evaluation method.

33. The first device according to any one of claims 21 to 31, wherein: The first processing module is also used for: Based on the second rule, the first evaluation method is determined.

34. The first device according to claim 33, wherein: A second terminal device is included, and the second rule is pre-agreed or configured by a second network device.

35. The first device according to claim 33, wherein: A second network device is included, and the second rule is pre-agreed.

36. A second device, comprising: The third communication module is used to send third indication information to the first device; wherein the third indication information is used to enable the first device to evaluate the performance of the first solution under the first condition by using the first evaluation method corresponding to the first condition.

37. The second device according to claim 36, wherein: The third indication information is used to indicate the first condition or to indicate that the first condition is met.

38. The second device according to claim 36, wherein: The third indication information is used to indicate a first rule, and the first rule is used by the first device to determine the first condition.

39. The second device according to any one of claims 36-38, wherein: The third indication information is used to indicate the first evaluation method.

40. The second device according to any one of claims 36-38, wherein: The third indication information is used to indicate a second rule, and the second rule is used by the first device to determine the first evaluation method.

41. A first device, comprising: A transceiver, a processor and a memory, wherein the memory is used to store a computer program, the transceiver is used to communicate with other devices, and the processor is used to call and run the computer program stored in the memory so that the first device executes the method as described in any one of claims 1 to 15.

42. A second device, comprising: A transceiver, a processor and a memory, wherein the memory is used to store a computer program, the transceiver is used to communicate with other devices, and the processor is used to call and run the computer program stored in the memory so that the second device executes the method as described in any one of claims 16 to 20.

43. A chip, comprising: A processor, configured to call and run a computer program from a memory, so that a device equipped with the chip executes a method as claimed in any one of claims 1 to 15.

44. A chip, comprising: A processor, configured to call and run a computer program from a memory, so that a device equipped with the chip executes a method as claimed in any one of claims 16 to 20.

45. A computer-readable storage medium for storing a computer program, which, when executed by a device, causes the device to perform the method according to any one of claims 1 to 15.

46. ​​A computer-readable storage medium for storing a computer program, which, when executed by a device, causes the device to perform the method according to any one of claims 16 to 20.

47. A computer program product comprising computer program instructions for causing a computer to perform the method of any one of claims 1 to 15.

48. A computer program product comprising computer program instructions for causing a computer to perform the method of any one of claims 16 to 20.

49. A computer program causing a computer to execute the method according to any one of claims 1 to 15.

50. A computer program causing a computer to execute the method of any one of claims 16 to 20.

51. A communication system comprising: A first device, configured to perform the method according to any one of claims 1 to 15; A second device, configured to execute the method according to any one of claims 6 to 20.

Images