Model test method, devices, and storage medium

Abstract

The present disclosure relates to a model test method, devices, and a storage medium. The method comprises: determining whether predicted data output by a first model meets a first condition, wherein the first condition is a performance requirement for the predicted data, the predicted data is data obtained by means of the first model performing prediction according to first measured data, the first measured data is data obtained after a terminal device performs measurement on first information, and the first information is information sent by a network device to the terminal device. Thus, a first model can be tested or verified to determine whether the first model meets a first condition, thereby determining the performance and reliability of the model, achieving online verification and testing of the model, and improving the flexibility and reliability of model management.

Description

Model testing methods, equipment and storage media Technical Field

[0001] This disclosure relates to the field of communication technology, and in particular to a model testing method, device and storage medium. Background Technology

[0002] With the advancement of communication technology, predictive models, such as Artificial Intelligence (AI) models, Machine Learning (ML) models, or other models, have been introduced into communication systems. These models provide predictive data for certain scenarios to improve network performance. However, the reliability of this predictive data significantly impacts network performance; therefore, determining the reliability of predictive data from predictive models is a pressing issue that needs to be addressed.

[0003] Summary of the Invention

[0004] This disclosure provides a model testing method, apparatus, and storage medium.

[0005] According to a first aspect of the embodiments of this disclosure, a model testing method is proposed, the method comprising:

[0006] Determine whether the predicted data output by the first model meets the first condition; wherein, the first condition is the performance requirement of the predicted data, the predicted data is the data predicted by the first model based on the first measurement data, the first measurement data is the data obtained by the terminal device after measuring the first information, and the first information is the information sent by the network device to the terminal device.

[0007] According to a second aspect of the embodiments of this disclosure, a model testing method is proposed, the method comprising:

[0008] Using a first model, predictions are made based on first information to obtain the predicted data; wherein, the first information is information sent from the network device to the terminal device;

[0009] Send a second message, the second message including the prediction data, the prediction data being used to determine whether the first model meets a first condition, the first condition being the performance requirements of the prediction data.

[0010] According to a third aspect of the embodiments of this disclosure, a model testing method is proposed, the method comprising:

[0011] The terminal device uses a first model to make a prediction based on first information to obtain the prediction data; wherein, the first information is information sent from the network device to the terminal device;

[0012] The terminal device sends second information to the network device. The second information includes the prediction data. The prediction data is used to determine whether the first model meets the first condition, where the first condition is the performance requirement of the prediction data.

[0013] The network device determines whether the predicted data output by the first model meets the first condition; wherein, the predicted data is the data predicted by the first model based on the first measurement data, the first measurement data is the data obtained by the terminal device after measuring the first information, and the first information is the information sent by the network device to the terminal device.

[0014] According to a fourth aspect of the embodiments of this disclosure, a network device is provided, comprising:

[0015] The processing module is configured to determine whether the predicted data output by the first model meets a first condition; wherein the first condition is the performance requirement of the predicted data, the predicted data is the data predicted by the first model based on the first measurement data, the first measurement data is the data obtained by the terminal device after measuring the first information, and the first information is the information sent by the network device to the terminal device.

[0016] According to a fifth aspect of the embodiments of this disclosure, a terminal device is provided, comprising:

[0017] The processing module is configured to use a first model to make a prediction based on first information to obtain the prediction data; wherein, the first information is information sent from the network device to the terminal device;

[0018] The transceiver module is configured to send second information, the second information including the prediction data, the prediction data being used to determine whether the first model meets a first condition, the first condition being the performance requirements of the prediction data.

[0019] According to a sixth aspect of the present disclosure, a network device is provided, comprising: one or more processors; wherein the network device can be used to perform an optional implementation of the first aspect.

[0020] According to a seventh aspect of the present disclosure, a terminal device is provided, comprising: one or more processors; wherein the terminal device can be used to execute an optional implementation of the second aspect.

[0021] According to an eighth aspect of the present disclosure, a communication system is provided, which may include a network device and a terminal device; wherein the network device is configured to perform the method described in the optional implementation of the first aspect, and the terminal device is configured to perform the method described in the optional implementation of the second aspect.

[0022] According to a ninth aspect of the present disclosure, a storage medium is provided that stores instructions that, when executed on a communication device, cause the communication device to perform the method as described in an optional implementation of the first or second aspect.

[0023] The technical solution provided by this disclosure can include the following beneficial effects: determining whether the predicted data output by the first model meets a first condition; wherein, the first condition is the performance requirement of the predicted data, the predicted data is the data predicted by the first model based on first measurement data, the first measurement data is the data obtained by the terminal device after measuring first information, and the first information is the information sent by the network device to the terminal device. In this way, the first model can be tested or verified to determine whether it meets the first condition, thereby determining the model's performance and reliability, realizing online verification and testing of the model, and improving the flexibility and reliability of model management.

[0024] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings required for the description of the embodiments are introduced below. The following drawings are only some embodiments of this disclosure and do not impose specific limitations on the protection scope of this disclosure.

[0026] Figure 1A is a schematic diagram of the architecture of a communication system according to an embodiment of the present disclosure.

[0027] Figure 1B is a schematic diagram of a beam prediction scenario according to an embodiment of the present disclosure.

[0028] Figure 2A is a schematic flowchart illustrating a model testing method according to an embodiment of the present disclosure.

[0029] Figure 2B is a schematic flowchart illustrating a model testing method according to an embodiment of the present disclosure.

[0030] Figure 3A is a schematic flowchart illustrating a model testing method according to an embodiment of the present disclosure.

[0031] Figure 3B is a schematic flowchart illustrating a model testing method according to an embodiment of the present disclosure.

[0032] Figure 3C is a schematic flowchart illustrating a model testing method according to an embodiment of the present disclosure.

[0033] Figure 3D is a schematic flowchart illustrating a model testing method according to an embodiment of the present disclosure.

[0034] Figure 4A is a schematic flowchart illustrating a model testing method according to an embodiment of the present disclosure.

[0035] Figure 4B is a schematic flowchart illustrating a model testing method according to an embodiment of the present disclosure.

[0036] Figure 4C is a schematic flowchart illustrating a model testing method according to an embodiment of the present disclosure.

[0037] Figure 4D is a schematic flowchart illustrating a model testing method according to an embodiment of the present disclosure.

[0038] Figure 5 is a flowchart illustrating a model testing method according to an embodiment of the present disclosure.

[0039] Figure 6 is a schematic flowchart illustrating a model testing method according to an embodiment of the present disclosure.

[0040] Figure 7A is a schematic diagram of the structure of a terminal device according to an embodiment of the present disclosure.

[0041] Figure 7B is a schematic diagram of the structure of a network device according to an embodiment of the present disclosure.

[0042] Figure 8A is a schematic diagram of the structure of a communication device according to an embodiment of the present disclosure.

[0043] Figure 8B is a schematic diagram of the structure of a chip according to an embodiment of the present disclosure. Detailed Implementation

[0044] This disclosure provides a model testing method, apparatus, and storage medium.

[0045] In a first aspect, embodiments of this disclosure propose a model testing method, the method comprising:

[0046] Determine whether the predicted data output by the first model meets the first condition; wherein, the first condition is the performance requirement of the predicted data, the predicted data is the data predicted by the first model based on the first measurement data, the first measurement data is the data obtained by the terminal device after measuring the first information, and the first information is the information sent by the network device to the terminal device.

[0047] In the above embodiments, the first model can be tested or verified to determine whether the first model meets the first condition, thereby determining the performance and reliability of the model, realizing online verification and testing of the model, and improving the flexibility and reliability of model management.

[0048] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes:

[0049] The terminal device sends a second message, which includes the prediction data.

[0050] In the above embodiments, the first model can be tested or verified based on the predicted data or the first measurement data, thereby adapting to the model testing needs in different scenarios.

[0051] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes:

[0052] Send a third message to the terminal device; the third message is used to indicate whether the first model can be used by the terminal device.

[0053] In the above embodiments, if the first model meets the first condition, the terminal device can be instructed to determine that the first model can be used, thereby ensuring the reliability of the models added or updated by the terminal device.

[0054] In conjunction with some embodiments of the first aspect, in some embodiments, the method further includes:

[0055] Receive fourth information; the fourth information is used to trigger the network device to send first information, the fourth information including the first model identifier of the first model;

[0056] Based on the fourth information, it is determined that the first model meets the test start conditions, and the first information is sent to the terminal device.

[0057] In the above embodiments, the terminal device can actively trigger model testing, thereby improving the flexibility of model testing.

[0058] In conjunction with some embodiments of the first aspect, in some embodiments, the test initiation condition includes at least one of the following:

[0059] The first model has not been tested;

[0060] The first model is not in the model management set; the model management set includes models that have already been tested.

[0061] The fourth piece of information is used to instruct the execution of a test on the first model.

[0062] In the above embodiments, starting the model test when the test start conditions are met can avoid invalid tests and improve test reliability.

[0063] In conjunction with some embodiments of the first aspect, in some embodiments, the fourth information includes at least one of the following:

[0064] Model update indication information;

[0065] Add instruction information to the model;

[0066] Model test instruction information.

[0067] In the above embodiments, model testing can be triggered by at least one of model update instruction information, model add instruction information, and model test instruction information.

[0068] In conjunction with some embodiments of the first aspect, in some embodiments, the first model is a model for performing beam prediction, the prediction data including the predicted signal quality of the first beam, the first beam being at least one beam configured by the network device for the terminal device;

[0069] The first condition includes the difference between the predicted signal quality and the first signal quality within a first difference range; wherein, the first signal quality is the signal quality of the first beam generated by the network device according to the beam configuration information, or, the first signal quality is the signal quality of the first beam actually measured by the terminal device.

[0070] In the above embodiments, the model performing beam prediction can be tested based on the difference in signal quality, thereby improving the reliability of the model in beam prediction scenarios.

[0071] In conjunction with some embodiments of the first aspect, in some embodiments, the first model is a model for performing beam prediction, and the prediction data includes a second beam that satisfies the signal conditions obtained by the beam prediction;

[0072] The signal conditions include at least one of the following:

[0073] The signal quality of the second beam is greater than or equal to a preset signal quality threshold;

[0074] The second beam is the beam with the strongest signal quality among the multiple beams configured by the network device for the terminal device.

[0075] In the above embodiments, the model performing beam prediction can be tested based on the accuracy of the predicted beam, thereby improving the reliability of the model in beam prediction scenarios.

[0076] In conjunction with some embodiments of the first aspect, in some embodiments, the first condition is that the number of times the prediction success condition is satisfied in the N beam predictions performed is greater than or equal to M; where N is a natural number and M is a natural number less than or equal to N; the prediction success condition is that the second beam is the same as the third beam, and the second beam is the beam that satisfies the signal condition obtained by performing beam prediction.

[0077] The third beam is a beam that satisfies the signal conditions, determined by the network device based on beam configuration information; or, the third beam is a beam that satisfies the signal conditions, determined based on second measurement data, whereby the terminal device measures multiple beams configured for it by the network device.

[0078] In the above embodiments, the model performing beam prediction can be tested based on the prediction accuracy of multiple predictions, thereby improving the reliability of the model in beam prediction scenarios.

[0079] In conjunction with some embodiments of the first aspect, in some embodiments, the signal quality includes at least one of the following: RSRP, SINR, RSSI, and RSRQ.

[0080] In the above embodiments, the first model can be tested based on at least one of a variety of signal qualities to improve the reliability of the model in beam prediction scenarios.

[0081] In conjunction with some embodiments of the first aspect, in some embodiments, the first information includes at least one of the following:

[0082] First data used to test the first model;

[0083] The first signal is used to test the first model.

[0084] In the above embodiments, the model can be tested using the first data and / or the first signal, thereby increasing the application scenarios for model testing.

[0085] Secondly, embodiments of this disclosure propose a model testing method, the method comprising:

[0086] Using a first model, predictions are made based on first information to obtain the predicted data; wherein, the first information is information sent from the network device to the terminal device;

[0087] Send a second message, the second message including the prediction data, the prediction data being used to determine whether the first model meets a first condition, the first condition being the performance requirements of the prediction data.

[0088] In the above embodiments, the first model can be tested or verified to determine whether the first model meets the first condition, thereby determining the performance and reliability of the model, realizing online verification and testing of the model, and improving the flexibility and reliability of model management.

[0089] In conjunction with some embodiments of the second aspect, in some embodiments, the method further includes:

[0090] Receive the first information;

[0091] The step of using the first model to make predictions based on the first information to obtain the predicted data includes:

[0092] The first information is measured to obtain the first measurement data;

[0093] The first measurement data is input into the first model to obtain the predicted data output by the first model.

[0094] In conjunction with some embodiments of the second aspect, in some embodiments, the method further includes:

[0095] Receive third information; the third information is used to indicate whether the first model can be used by the terminal device;

[0096] The third information is used to determine whether the first model can be used.

[0097] In conjunction with some embodiments of the second aspect, in some embodiments, the method further includes:

[0098] Send a fourth message; wherein the fourth message is used to trigger the network device to send the first message, and the fourth message includes the first model identifier of the first model;

[0099] In conjunction with some embodiments of the second aspect, in some embodiments, the fourth information includes at least one of the following:

[0100] Model update indication information;

[0101] Add instruction information to the model;

[0102] Model test instruction information.

[0103] In conjunction with some embodiments of the second aspect, in some embodiments, the first model is a model for performing beam prediction, the prediction data including the predicted signal quality of a first beam, the first beam being at least one beam configured by the network device for the terminal device;

[0104] The first condition includes the difference between the predicted signal quality and the first signal quality within a first difference range; wherein, the first signal quality is the signal quality of the first beam generated by the network device according to the beam configuration information, or, the first signal quality is the signal quality of the first beam actually measured by the terminal device.

[0105] In conjunction with some embodiments of the second aspect, in some embodiments, the first model is a model for performing beam prediction, and the prediction data includes a second beam that satisfies the signal conditions obtained by the beam prediction;

[0106] The signal conditions include at least one of the following:

[0107] The signal quality of the second beam is greater than or equal to a preset signal quality threshold;

[0108] The second beam is the beam with the strongest signal quality among the multiple beams configured by the network device for the terminal device.

[0109] In conjunction with some embodiments of the second aspect, in some embodiments, the first condition is that the number of times the prediction success condition is satisfied in the N beam predictions performed is greater than or equal to M; where N is a natural number and M is a natural number less than or equal to N; the prediction success condition is that the second beam is the same as the third beam, and the second beam is the beam that satisfies the signal condition obtained by performing beam prediction.

[0110] The third beam is a beam that satisfies the signal conditions, determined by the network device based on beam configuration information; or, the third beam is a beam that satisfies the signal conditions, determined based on second measurement data, whereby the terminal device measures multiple beams configured for it by the network device.

[0111] In conjunction with some embodiments of the second aspect, in some embodiments, the signal quality includes at least one of the following: RSRP, SINR, RSSI, and RSRQ.

[0112] In conjunction with some embodiments of the second aspect, in some embodiments, the first information includes at least one of the following:

[0113] First data used to test the first model;

[0114] The first signal is used to test the first model.

[0115] Thirdly, embodiments of this disclosure propose a model testing method, the method comprising:

[0116] The terminal device uses a first model to make a prediction based on first information to obtain the prediction data; wherein, the first information is information sent from the network device to the terminal device;

[0117] The terminal device sends second information to the network device. The second information includes the prediction data. The prediction data is used to determine whether the first model meets the first condition, where the first condition is the performance requirement of the prediction data.

[0118] The network device determines whether the predicted data output by the first model meets the first condition; wherein, the predicted data is the data predicted by the first model based on the first measurement data, the first measurement data is the data obtained by the terminal device after measuring the first information, and the first information is the information sent by the network device to the terminal device.

[0119] In the above embodiments, the first model can be tested or verified to determine whether the first model meets the first condition, thereby determining the performance and reliability of the model, realizing online verification and testing of the model, and improving the flexibility and reliability of model management.

[0120] Fourthly, embodiments of this disclosure provide a network device that may include at least one of a transceiver module and a processing module; wherein the network device may be used to perform an optional implementation of the first aspect.

[0121] Fifthly, embodiments of this disclosure provide a terminal device, which may include at least one of a transceiver module and a processing module; wherein the terminal device may be used to execute an optional implementation of the second aspect.

[0122] In a sixth aspect, embodiments of this disclosure provide a network device that may include one or more processors; wherein the network device may be used to perform an optional implementation of the first aspect.

[0123] In a seventh aspect, embodiments of this disclosure provide a terminal device that may include one or more processors; wherein the terminal device may be used to execute an optional implementation of the second aspect.

[0124] Eighthly, embodiments of this disclosure provide a communication system that may include a terminal device and a network device; wherein the network device is configured to perform the method described in the optional implementation of the first aspect, and the terminal device is configured to perform the method described in the optional implementation of the second aspect.

[0125] In a ninth aspect, embodiments of this disclosure provide a storage medium storing instructions that, when executed on a communication device, cause the communication device to perform the method as described in an optional implementation of the first or second aspect.

[0126] In a tenth aspect, embodiments of this disclosure provide a program product that, when executed by a communication device, causes the communication device to perform the method as described in an optional implementation of the first or second aspect.

[0127] In one aspect, embodiments of this disclosure provide a computer program that, when run on a computer, causes the computer to perform the methods described in an optional implementation of the first or second aspect.

[0128] In a twelfth aspect, embodiments of this disclosure provide a chip or chip system. The chip or chip system includes processing circuitry configured to perform the methods described in optional implementations of the first or second aspect.

[0129] It is understood that the aforementioned terminal devices, network devices, communication devices, communication systems, storage media, program products, computer programs, chips, or chip systems can all be used to execute the methods proposed in the embodiments of this disclosure. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects in the corresponding methods, and will not be repeated here.

[0130] This disclosure provides a model testing method, apparatus, and storage medium. In some embodiments, the terms "model testing method" and "information processing method," "communication method," etc., can be used interchangeably; the terms "model testing apparatus" and "information processing apparatus," "communication apparatus," "communication device," etc., can be used interchangeably; and the terms "information processing system," "communication system," etc., can be used interchangeably.

[0131] This disclosure is not exhaustive, but merely illustrative of some embodiments, and is not intended to limit the scope of protection of this disclosure. Unless otherwise specified, each step in a particular embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a particular embodiment can also be implemented as an independent embodiment, and the order of the steps in a particular embodiment can be arbitrarily interchanged. Furthermore, the optional implementation methods in a particular embodiment can be arbitrarily combined; moreover, the embodiments can be arbitrarily combined, for example, some or all steps of different embodiments can be arbitrarily combined, and a particular embodiment can be arbitrarily combined with the optional implementation methods of other embodiments.

[0132] In each of the disclosed embodiments, unless otherwise specified or in case of logical conflict, the terminology and / or descriptions of the embodiments are consistent and can be referenced by each other. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0133] The terminology used in the embodiments of this disclosure is for the purpose of describing particular embodiments only and is not intended to limit the scope of this disclosure.

[0134] In this embodiment of the disclosure, unless otherwise stated, elements expressed in the singular form, such as "a," "an," "the," "the," "the," "the," "the," "the," "this," etc., can mean "one and only one," or "one or more," "at least one," etc. For example, when using articles such as "a," "an," "the," etc. in translation, the noun following the article can be understood as either a singular expression or a plural expression.

[0135] In the embodiments disclosed herein, "multiple" refers to two or more.

[0136] In some embodiments, the terms “a plurality of”, “multiple”, “at least one of”, “one or more”, etc., may be used interchangeably.

[0137] In some embodiments, the notation "at least one of A and B", "A and / or B", "A in one case, B in another", "in response to one case A, in response to another case B", etc., may include the following technical solutions depending on the situation: in some embodiments, A (execute A regardless of B); in some embodiments, B (execute B regardless of A); in some embodiments, execution is selected from A and B (A and B are selectively executed); in some embodiments, A and B (both A and B are executed). The same applies when there are more branches such as A, B, C, etc.

[0138] In some embodiments, the notation "A or B" may include the following technical solutions, depending on the situation: in some embodiments, A (execution of A regardless of B); in some embodiments, B (execution of B regardless of A); in some embodiments, execution is selected from A and B (A and B are selectively executed). The same applies when there are more branches such as A, B, C, etc.

[0139] The prefixes "first," "second," etc., used in the embodiments of this disclosure are merely for distinguishing different descriptive objects and do not impose restrictions on the position, order, priority, quantity, or content of the descriptive objects. The description of the descriptive objects is found in the claims or the context of the embodiments, and the use of prefixes should not constitute unnecessary restrictions. For example, if the descriptive object is a "field," the ordinal numbers preceding "field" in "first field" and "second field" do not restrict the position or order of the "fields." "First" and "second" do not restrict whether the "fields" they modify are in the same message, nor do they restrict the order of "first field" and "second field." Similarly, if the descriptive object is a "level," the ordinal numbers preceding "level" in "first level" and "second level" do not restrict the priority between "levels." Furthermore, the number of descriptive objects is not limited by ordinal numbers and can be one or more. For example, in "first device," the number of "devices" can be one or more. Furthermore, the objects modified by different prefixes can be the same or different. For example, if the object being described is "device", then "first device" and "second device" can be the same device or different devices, and their types can be the same or different. Similarly, if the object being described is "information", then "first information" and "second information" can be the same information or different information, and their content can be the same or different.

[0140] In some embodiments, “including A,” “containing A,” “for indicating A,” and “carrying A” can be interpreted as directly carrying A or indirectly indicating A.

[0141] In some embodiments, the terms “in response to…”, “in response to determining…”, “in the case of…”, “when…”, “if…”, “if…”, etc., can be used interchangeably.

[0142] In some embodiments, the terms “greater than,” “greater than or equal to,” “not less than,” “more than,” “more than or equal to,” “not less than,” “higher than,” “higher than or equal to,” “not lower than,” and “above” can be used interchangeably, as can the terms “less than,” “less than or equal to,” “not greater than,” “less than,” “less than or equal to,” “not more than,” “lower than,” “lower than or equal to,” “not higher than,” and “below”.

[0143] In some embodiments, the apparatus and device may be interpreted as physical or virtual, and their names are not limited to the names recorded in the embodiments. In some cases, they may also be understood as "equipment", "device", "circuit", "network element", "node", "function", "unit", "section", "system", "network", "chip", "chip system", "entity", "body", etc.

[0144] In some embodiments, "network" can be interpreted as devices included in the network, such as access network devices, core network devices, etc.

[0145] In some embodiments, "Access Network Device (AN Device)" may also be referred to as "Radio Access Network Device (RAN Device)," "Base Station (BS)," "Radio Base Station," or "Fixed Station." In some embodiments, it may also be understood as "Node," "Access Point," "Transmission Point (TP)," "Reception Point (RP)," "Transmission / Reception Point (TRP)," "Panel," "Antenna Panel," "Antenna Array," "Cell," "Macro Cell," "Small Cell," "Femto Cell," "Pico Cell," "Sector," "Cell Group," "Serving Cell," "Carrier," "Component Carrier," or "Bandwidth Part (BWP)," etc.

[0146] In some embodiments, "terminal" or "terminal device" may be referred to as "user equipment (UE)," "user terminal," "mobile station (MS)," "mobile terminal (MT)," "subscriber station," "mobile unit," "subscriber unit," "wireless unit," "remote unit," "mobile device," "wireless device," "wireless communication device," "remote device," "mobile subscriber station," "access terminal," "mobile terminal," "wireless terminal," "remote terminal," "handset," "user agent," "mobile client," "client," etc.

[0147] In some embodiments, the acquisition of data, information, etc., may comply with the laws and regulations of the country where the location is situated.

[0148] In some embodiments, data, information, etc., may be obtained with the user's consent.

[0149] Furthermore, each element, each row, or each column in the table of this disclosure can be implemented as an independent embodiment, and any combination of any element, any row, or any column can also be implemented as an independent embodiment.

[0150] Figure 1A is a schematic diagram of the architecture of a communication system according to an embodiment of the present disclosure. As shown in Figure 1A, the communication system 100 may include a terminal device 101 and a network device 102.

[0151] In some embodiments, terminal device 101 may include at least one of, but is not limited to, a mobile phone, a wearable device, an Internet of Things device, a car with communication capabilities, a smart car, a tablet computer, a computer with wireless transceiver capabilities, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal device in industrial control, a wireless terminal device in self-driving, a wireless terminal device in remote medical surgery, a wireless terminal device in a smart grid, a wireless terminal device in transportation safety, a wireless terminal device in a smart city, and a wireless terminal device in a smart home.

[0152] In some embodiments, network device 102 may include at least one of access network device and core network device.

[0153] In some embodiments, the access network device may be a node or device that connects a terminal to a wireless network. The access network device may include, but is not limited to, at least one of the following in a 5G communication system: evolved Node B (eNB), next-generation eNB (ng-eNB), next-generation Node B (gNB), node B (NB), home node B (HNB), home evolved node B (HeNB), radio backhaul device, radio network controller (RNC), base station controller (BSC), base transceiver station (BTS), base band unit (BBU), mobile switching center, base station in a 6G communication system, open RAN, cloud RAN, base station in other communication systems, and access node in a Wi-Fi system.

[0154] In some embodiments, the technical solutions of this disclosure can be applied to the Open RAN architecture. In this case, the interfaces between or within access network devices involved in the embodiments of this disclosure can be transformed into internal interfaces of Open RAN. The processes and information interactions between these internal interfaces can be implemented by software or programs.

[0155] In some embodiments, the access network device may be composed of a central unit (CU) and a distributed unit (DU). The CU may also be called a control unit. The CU-DU structure can separate the protocol layer of the access network device. Some of the protocol layer functions are centrally controlled by the CU, while the remaining part or all of the protocol layer functions are distributed in the DU and centrally controlled by the CU. However, this is not the only possibility.

[0156] In some embodiments, the core network equipment may be a single device, multiple devices, or a group of devices. The core network may include at least one of the following: Evolved Packet Core (EPC), 5G Core Network (5GCN), and Next Generation Core (NGC).

[0157] It is understood that the communication system described in this disclosure is for the purpose of more clearly illustrating the technical solutions of this disclosure, and does not constitute a limitation on the technical solutions proposed in this disclosure. As those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions proposed in this disclosure are also applicable to similar technical problems.

[0158] The following embodiments of this disclosure can be applied to the communication system 100 shown in FIG1A, or to some of the main bodies, but are not limited thereto. The main bodies shown in FIG1A are examples. The communication system may include all or some of the main bodies in FIG1A, or it may include other main bodies outside of FIG1A. The number and form of each main body are arbitrary. Each main body may be physical or virtual. The connection relationship between the main bodies is an example. The main bodies may not be connected or may be connected. The connection can be in any way, it can be a direct connection or an indirect connection, it can be a wired connection or a wireless connection.

[0159] The embodiments disclosed herein can be applied to Long Term Evolution (LTE), LTE-Advanced (LTE-A), LTE-Beyond (LTE-B), SUPER 3G, IMT-Advanced, 4th generation mobile communication system (4G), 5th generation mobile communication system (5G), 5G new radio (NR), Future Radio Access (FRA), New-Radio Access Technology (RAT), New Radio (NR), New radio access (NX), Future generation radio access (FX), Global System for Mobile communications (GSM), CDMA2000, Ultra Mobile Broadband (UMB), IEEE 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), and IEEE 802.20, Ultra-Wideband (UWB), Bluetooth (a registered trademark), Public Land Mobile Network (PLMN) networks, Device-to-Device (D2D) systems, Machine-to-Machine (M2M) systems, Internet of Things (IoT) systems, Vehicle-to-Everything (V2X) systems, systems utilizing other communication methods, and next-generation systems built upon them, etc. Furthermore, multiple systems can be combined (e.g., a combination of LTE or LTE-A with 5G).

[0160] In some embodiments of this disclosure, the communication system described above may incorporate a first model, which may be an AI model, an ML model, an AI / ML model, or other models used for prediction. The first model may be one or more models.

[0161] In this communication system, the process and performance of certain typical scenarios can be simplified and improved based on the prediction of the first model. These typical scenarios may include CSI (Channel State Information) feedback scenarios, beam management scenarios, positioning scenarios, or other application scenarios. Among them:

[0162] In CSI feedback scenarios, the terminal device can predict accurate and effective CSI feedback values ​​based on the first model. This allows the network device to select a suitable MCS (Modulation and Coding Scheme) for downlink data transmission based on the feedback CSI value, reducing the BLER (Block Error Rate) of downlink data transmission. In beam management scenarios, the terminal device can predict the optimal beam / beam group based on the first model, such as the top-1 strongest beam / beam group. This allows the network device to select the best beam for subsequent transmission. In positioning scenarios, the terminal device can predict positioning information based on the first model, thereby improving positioning accuracy. This positioning information may include at least one of the following: Reference Signal Time Difference (RSTD), Positioning Reference Signal-Reference Signal Receiving Power (PRS-RSRP), positioning accuracy, Line of Sight (LOS) indication, and Non-Line of Sight (NLOS) indication.

[0163] Figure 1B is a schematic diagram of a beam prediction scenario according to an embodiment of the present disclosure. As shown in Figure 1B, the terminal device can measure a portion of the beams to obtain first measurement data. This first measurement data may include the channel quality of the measured portion of the beams. Inputting this first measurement data into a first model yields prediction data output by the first model. For example, the first model can perform inference to predict the channel quality of other beams besides the measured portion. The prediction data may include at least one of the following: channel quality of some or all beams, optimal beam, and channel quality of the optimal beam.

[0164] Based on the aforementioned first measurement data and / or prediction data, the network device can determine the beam with the best channel quality from all beams for communication.

[0165] In some embodiments, the first model can be deployed on a terminal device, and the portion of the beam measured by the terminal device can be referred to as first information. The prediction data obtained by the terminal device based on the first model can be referred to as second information. The terminal device can report the prediction data to the network device.

[0166] In some embodiments, the first model can be deployed on a network device, and the portion of the beam measured by the terminal device can be referred to as first information, while the first measurement data obtained by the terminal device can be referred to as second information. The terminal device can report the first measurement data to the network device, and the network device can input the first measurement data into the first model to obtain prediction data.

[0167] In related technologies, if a terminal device updates or adds a new first model, and this model is not in the model management monitoring list, the performance and reliability of the model's prediction results have not been tested, and the terminal device may not be able to directly use the model for result prediction. Therefore, online verification and testing of models are urgent problems to be solved.

[0168] Figure 2A is a flowchart illustrating a model testing method according to an embodiment of the present disclosure. This method can be executed by the aforementioned communication system. As shown in Figure 2A, the method may include:

[0169] Step S2101: The terminal device sends the fourth information to the network device.

[0170] In some embodiments, the network device may receive fourth information. For example, the network device may receive fourth information sent by a terminal device. As another example, the network device may also receive fourth information sent by other entities.

[0171] In some embodiments, the fourth information can be used to trigger the network device to send the first information.

[0172] In some embodiments, the fourth information may be used to request or instruct the network device to send the first information.

[0173] In some embodiments, the fourth information may be used to indicate the addition or updating of the first model.

[0174] In some embodiments, the fourth information may be used to indicate the testing of the first model.

[0175] In some embodiments, the name of the fourth information is not limited, and may be, for example, "model addition or update instruction information", "model addition instruction information", "model update instruction information", "model online test instruction information", "model test instruction information", "model test request information", etc.

[0176] In some embodiments, the terminal device may send the fourth information to the network device when adding or updating the first model, so as to trigger the network device to start testing the first model.

[0177] In some embodiments, the fourth information may include a first model identifier, which may be an ID identifier corresponding to the first model.

[0178] In some embodiments, the fourth information may include at least one of the following: model update instruction information, model addition instruction information, and model testing instruction information.

[0179] The model update indication information can be used to trigger, request, or indicate an update (or rollback) of the first model.

[0180] Model addition instructions can be used to trigger, request, or instruct the addition (addition) of the first model.

[0181] Model test instruction information can be used to trigger, request, or instruct the testing (training or validation) of the first model. The name of this model test instruction information is not limited; for example, it could be "Model Online Test Instruction Information," "Model Test Request Information," etc.

[0182] In some embodiments, the terminal device may send a fourth message, which may include the aforementioned fourth information. For example, the terminal device may send the fourth message to the network device. Optionally, the network device may receive the fourth message. The fourth message may include at least one of the following: a Radio Resource Control (RRC) message, a Medium Access Control (MAC) Element, Uplink Control Information (UCI), or other messages sent by the terminal device to the network device.

[0183] In some embodiments of this disclosure, the first model described above may be an AI model, an ML model, an AI / ML model, or other models used for prediction. The first model may be one or more models.

[0184] In some embodiments, the first model can be a model for performing beam prediction, which can be applied in beam management scenarios. For example, the first model can be deployed on a terminal device, which can predict the optimal beam / beam group (e.g., the top-1 strongest beam / beam group) based on the first model and report it to the network device. The network device can then select the optimal beam / beam group for subsequent transmission. Similarly, the first model can also be deployed on a network device, which can also predict the optimal beam / beam group based on the first model.

[0185] In some embodiments, the first model can be a model for performing CSI feedback prediction, and this first model can be applied to CSI feedback scenarios. For example, the first model can be deployed on a terminal device, which can predict accurate and effective CSI feedback values ​​based on the first model. A network device can then select a suitable MCS for downlink data transmission based on the CSI feedback values, reducing the BLER of downlink data transmission. Similarly, the first model can also be deployed on a network device, which can also predict accurate and effective CSI feedback values ​​based on the first model.

[0186] In some embodiments, the first model can be a model for performing location prediction, and the first model can be applied in a location scenario. For example, the first model can be deployed on a terminal device, which can predict location information based on the first model to improve location accuracy. The location information may include at least one of RSTD, PRS-RSRP, location accuracy, LOS indication, NLOS indication, etc. Similarly, the first model can also be deployed on a network device, and the network device can also predict the aforementioned location information based on the first model.

[0187] Step S2102: The network device sends the first information to the terminal device.

[0188] In some embodiments, the terminal device may receive first information. For example, the terminal device may receive first information sent by a network device. As another example, the terminal device may also receive first information sent by other entities.

[0189] In some embodiments, the first information may include information for testing the first model.

[0190] In some embodiments, the first information may include first data for testing the first model. The first data may include downlink data, which may be data sent from the network device to the terminal device via a downlink channel. The downlink channel may include at least one of the Physical Downlink Control Channel (PDCCH), Physical Downlink Shared Channel (PDSCH), or other channels specified in the protocol.

[0191] In some embodiments, the first information may include a first signal for testing the first model. The first signal may include at least one of a Channel State Information-Reference Signal (CSI-RS), a Synchronization Signal Block (SSB), a Positioning Reference Signal (PRS), or other signals transmitted by the network device to the terminal device.

[0192] In some embodiments, the first information may include the first data and the first signal described above.

[0193] In some embodiments, the name of the first information is not limited, and may be, for example, "test reference information," "test information," "test data," "test signal," "model test information," "model test data," "model test signal," etc. Similarly, the name of the first signal is not limited, and may be, for example, "test reference signal," "test signal," "model test signal," etc.; the name of the first data is also not limited, and may be, for example, "test reference data," "test data," "model test data," etc.

[0194] In some embodiments, the test performed on the first model described above can be an online test. For example, the first model can be tested online while a commercial network is in operation. The first data mentioned above may include data from the commercial network, and similarly, the first signal mentioned above may include signals from the commercial network. In this way, online testing can be performed after the terminal device or network device is commercially available, so that the model can be verified, updated, or added to in a timely manner.

[0195] In some embodiments, the tests performed on the first model described above may also be offline tests. For example, the first model may be tested offline in a laboratory or online before commercial use.

[0196] In some embodiments, the network device may send a first message, which may include the first information described above. For example, the network device may send the first message to a terminal device. Optionally, the terminal device may receive the first message. The first message may include at least one of the following: an RRC message, a MAC CE, downlink control information (DCI), or other messages sent by the network device to the terminal device.

[0197] In some embodiments of this disclosure, the network device can determine that the first model meets the test start conditions based on the fourth information and send the first information to the terminal device.

[0198] The conditions for starting the test may include at least one of the following:

[0199] The first model has not been tested: for example, the first model has not been trained or its performance has not been validated.

[0200] The first model is not in the model management set: this model management set includes models that have been tested (or trained, or whose performance has been validated). This model management set can also be called a model management list, model monitoring management list, model monitoring list, etc.

[0201] This fourth piece of information is used to instruct the execution of tests on the first model. For example, this first piece of information may be model test instruction information or model online test instruction information.

[0202] In some embodiments, the network device may determine whether it needs to send the first information to the terminal device or whether it needs to send data / signals for online testing of the first model to the terminal device based on the received fourth information.

[0203] For example, if the first model is not in the model monitoring and management list (has not been trained or performance verified), or after the network receives the online test instruction information, the network sends online test data / signals to the UE. The data / signals can be determined according to different scenarios. For example, data (PDSCH) is used for AI / ML-based CSI feedback; signals (CSI-RS / SSB) are used for AL / ML-based beam management; and signals (PRS) are used for AL / ML-based positioning.

[0204] Step S2103: The terminal device uses the first model to make a prediction based on the first information and obtains the prediction data.

[0205] In some embodiments, the terminal device may measure the received first information to obtain first measurement data; input the first measurement data into a first model to obtain prediction data output by the first model.

[0206] For example, the first information is a CSI-RS signal, the first measurement data can be the actual measurement result of the CSI-RS signal, and the prediction data can be the data output by the first model after the first measurement data is input into the first model.

[0207] In one implementation, the first model is deployed on the terminal device side, and the second information may include prediction data, or the second information may include prediction data and first measurement data. The prediction data may be data obtained by the terminal device based on the first model and the first measurement data.

[0208] For example, the terminal device can receive first information, measure the first information to obtain first measurement data, input the first measurement data into a first model, and obtain the prediction data output by the first model.

[0209] For example, the terminal device can perform predictions based on configuration information. For instance, the terminal device can perform predictions using a first model based on the configuration information to obtain prediction results. This configuration information can be information predefined by the protocol, or it can be information configured by the terminal device, or it can be information received by the terminal device from the network device, such as configuration information determined and sent to the terminal device by the network device.

[0210] Step S2104: The terminal device sends the second information to the network device.

[0211] In some embodiments, the network device may receive second information. For example, the network device may receive second information sent by a terminal device. As another example, the network device may also receive second information sent by other entities.

[0212] In some embodiments, the name of the second information is not limited, and may be, for example, "prediction result", "prediction information", "measurement result", "measurement information", "test result", "test information", etc.

[0213] In some embodiments, the second information may include at least one of the predicted data and the first measurement data.

[0214] In some embodiments, the name of the predicted data is not limited, and may be, for example, "prediction result", "model output data", "model output result", etc. Similarly, the name of the first measurement data is not limited, and may be, for example, "first measurement result", "actual measurement result", "actual measurement data", etc.

[0215] The predicted data can be data obtained by the first model based on the first measurement data, and the first measurement data can be data obtained by the terminal device after measuring the first information.

[0216] In some embodiments, the terminal device may send a second message, which may include the second information described above. For example, the terminal device may send the second message to a network device. Optionally, the network device may receive the second message. The second message may include at least one of an RRC message, a MAC CE, a UCI, or other messages sent by the terminal device to the network device.

[0217] In some embodiments of this disclosure, the terminal device uses a first model to predict the result based on the configuration information of the first information configured by the network device, obtains the second information (prediction result), and reports the second information to the network device.

[0218] Step S2105: The network device determines whether the predicted data output by the first model meets the first condition.

[0219] In some embodiments, the network device may determine whether the predicted data output by the first model satisfies the first condition based on the second information. Alternatively, the network device may determine whether the first model satisfies the first condition based on the second information.

[0220] In some embodiments, the first condition may be a performance requirement for the predicted data. This performance requirement may include at least one of an accuracy requirement, a precision requirement, and a reliability requirement.

[0221] In some embodiments, the first condition may be a condition for determining whether the first mode can be used. For example, if the first condition is met, the first mode can be used by the terminal device or the network; if the first condition is not met, the first mode cannot be used by the terminal device or the network.

[0222] In some embodiments, the first model may be a model for performing beam prediction, the prediction data including the predicted signal quality of a first beam, which is at least one beam configured by the network device for the terminal device.

[0223] It should be noted that the first beam can be a beam or a group of beams, and this disclosure does not limit it.

[0224] The first beam can be any one or more beams configured by the network device for the terminal device, and the first beam can also be the beam with the strongest signal quality.

[0225] The first condition may include the difference between the predicted signal quality and the first signal quality within a first difference range; wherein, the predicted signal quality is the signal quality of the first beam predicted by the first model based on the first information; the first signal quality is the signal quality of the first beam generated by the network device based on the beam configuration information, or the first signal quality is the signal quality of the first beam actually measured by the terminal device.

[0226] In some embodiments, the first model may be a model for performing beam prediction, and the prediction data may include a second beam that satisfies the signal conditions obtained from the beam prediction.

[0227] The signal condition may include at least one of the following:

[0228] The signal quality of the second beam is greater than or equal to a preset signal quality threshold.

[0229] The second beam is the beam with the strongest signal quality among the multiple beams configured by the network device for the terminal device.

[0230] It should also be noted that the second beam can be a beam or a group of beams.

[0231] The first condition mentioned above can be that the number of times the prediction success condition is met in the N beam predictions performed is greater than or equal to M; where N is a natural number and M is a natural number less than or equal to N; the prediction success condition is that the second beam is the same as the third beam, and the second beam is the beam obtained by performing beam prediction.

[0232] The aforementioned third beam is a beam that meets the signal conditions, determined by the network device based on beam configuration information; or,

[0233] The aforementioned third beam is a beam that meets the signal conditions, determined based on the second measurement data. The second measurement data is the data obtained by the terminal device after measuring multiple beams configured for the terminal device by the network device.

[0234] For example, a network device configures first information (which may be a first signal) for beam management to a terminal device. The terminal device can infer and predict the predicted signal quality of the first beam based on the first information using a first model. The first beam may be at least one beam or beam group configured by the network device for the terminal device, or it may be the optimal beam or beam group with the strongest signal quality. The terminal device can report prediction data to the network device, which may include the predicted signal quality of the first beam. Alternatively, the second information may include the first beam and its predicted signal quality. The network device can evaluate whether the performance requirements of the prediction data reported by the terminal device meet a first condition, which may include predefined performance indicator requirements.

[0235] In one implementation, the performance requirement may be a signal quality accuracy requirement, whereby the signal quality may include at least one of RSRP, SINR, RSSI, and RSRQ. The signal quality accuracy may be the difference between the predicted signal quality and the first signal quality.

[0236] The first signal quality can be the signal quality simulated and generated by the network device based on the beam configuration information, or the first signal quality can be the signal quality of the first beam actually measured by the terminal device (for example, the signal quality of the first beam obtained by the terminal device based on the first information measured by a traditional method).

[0237] If the signal quality accuracy meets the predefined performance index requirements, for example, if the signal quality accuracy is within the first difference range, then it can be determined that the predicted data output by the first model meets the first condition.

[0238] In another implementation, the performance metric requirement can be a predictive reliability requirement. For example, the predicted reliability percentage needs to meet a predefined predicted reliability requirement. If the performance metric is predicted reliability, then the predicted reliability percentage needs to meet a predefined predicted reliability requirement. For example, if N beam predictions are performed, and the prediction success condition is met M times, then the predicted reliability is (M / N)*100%.

[0239] Where N is a natural number, and M is a natural number less than or equal to N.

[0240] The prediction is successful if the second beam is the same as the third beam.

[0241] The second beam is the beam (e.g., the optimal beam or beam group) that meets the signal conditions obtained by the terminal device performing beam prediction.

[0242] The third beam can be a beam that meets the signal conditions, determined by the network device based on beam configuration information. For example, it can be a beam that meets the signal conditions and is simulated and generated by the network device (e.g., an optimal beam or beam group). Alternatively,

[0243] The third beam may be a beam that satisfies the signal conditions, determined based on the second measurement data, which is the data obtained by the terminal device after measuring the multiple beams configured for the terminal device by the network device.

[0244] If the prediction reliability requirement meets the predefined performance index requirements, for example, if the number of times the prediction success condition is met in the N beam predictions performed is greater than or equal to M, then it can be determined that the prediction data output by the first model meets the first condition.

[0245] It should be noted that the predicted data output by the first model satisfies the first condition, which can also be expressed as "the first model is reliable".

[0246] Step S2106: The network device sends third information to the terminal device.

[0247] In some embodiments, the terminal device may receive third information. For example, the terminal device may receive third information sent by a network device. As another example, the terminal device may also receive third information sent by other entities.

[0248] In some embodiments, the third information may be used to indicate whether the first model can be used by the terminal device.

[0249] In some embodiments, the third information may be used to indicate whether the terminal device is able to use the first model.

[0250] For example, a network device may send the third information if it determines that the prediction result of the first model meets the first condition. The third information is used to indicate that the first model can be used by the terminal device.

[0251] For example, a network device may choose not to send the third information if it determines that the prediction result of the first model does not meet the first condition.

[0252] For example, if a network device determines that the prediction result of the first model meets the first condition, the third information it sends may be a first specific value, which can be used to indicate that the first model can be used by the terminal device.

[0253] For example, if the terminal network device determines that the prediction result of the first model does not meet the first condition, the third information sent may be a second specific value. This third information, being a second specific value, can be used to indicate that the first model cannot be used by the terminal device. The first specific value and the second specific value are different; for example, the first specific value is 1 and the second specific value is 0; or, the first specific value is 0 and the second specific value is 1.

[0254] In some embodiments, the third information may be used to instruct the terminal device to add, update, or delete the first model described above.

[0255] In some embodiments, the third information may be used to instruct the terminal device to add or update the first model described above.

[0256] In some embodiments, the third information may be used to indicate that the terminal device is capable of using the first model described above.

[0257] In some embodiments, the third information may be used to indicate that the terminal device is permitted to use the first model described above.

[0258] In some embodiments, the third information may be used to instruct the terminal device to delete the first model described above.

[0259] In some embodiments, the third information may be used to indicate that the terminal device cannot use the first model described above.

[0260] In some embodiments, the name of the third information is not limited, and may be, for example, "add model indicator information", "update model indicator information", "delete model indicator information", "can use model indicator information", "allow use of model indicator information", etc.

[0261] In some embodiments, the network device determines that the prediction data output by the first model meets the first condition and sends third information to the terminal device, the third information being used to instruct the terminal device to add or update the first model.

[0262] For example, the network evaluates whether the prediction data output by the first model meets the first condition (i.e., the predefined performance requirements) based on the second information reported by the terminal device. If it does, the network device can send third information to the terminal device, which can be used to instruct the terminal device to add or update the first model.

[0263] Optionally, after receiving the third information, the terminal device can add or update the first model.

[0264] In other embodiments, the network device determines that the prediction data output by the first model does not meet the first condition and sends third information to the terminal device, the third information being used to instruct the terminal device to delete the first model.

[0265] Optionally, after receiving the third information, the terminal device can delete the first model.

[0266] In some embodiments, the network device may send a third message, which may include the aforementioned third information. For example, the network device may send a third message to a terminal device. Optionally, the terminal device may receive the third message. The third message may include at least one of an RRC message, a MAC CE, a DCI, or other messages sent by the network device to the terminal device.

[0267] Step S2107: The terminal device determines whether the first model can be used.

[0268] In some embodiments, the terminal device may determine whether the first model can be used based on third information.

[0269] For example, upon receiving the third information, the terminal device can determine that the first model is usable. Similarly, the network device can send the third information if it determines that the prediction result of the first model meets the first condition.

[0270] For example, a terminal device can determine that the first model cannot be used if it does not receive the third information. Similarly, a network device can refuse to send the third information if it determines that the prediction result of the first model does not meet the first condition.

[0271] For example, a terminal device can determine that the first model can be used if the value of the third information is a first specific value. Similarly, a network device can send the third information as the first specific value if it determines that the prediction result of the first model meets the first condition.

[0272] For example, a terminal device can determine that the first model cannot be used if the value of the third information is a second specific value. Similarly, a network device can send the third information as a second specific value if it determines that the prediction result of the first model does not meet the first condition. The first specific value and the second specific value are different; for example, the first specific value is 1 and the second specific value is 0; or the first specific value is 0 and the second specific value is 1.

[0273] In some embodiments, the terminal device determines that the first model is usable and can be formally used for prediction, thereby improving the performance and reliability of the prediction.

[0274] In some embodiments, the terminal device may add, update, or delete the first model based on third information.

[0275] In some embodiments, the third information is used to instruct the terminal device to add the first model, and the terminal device can add the first model.

[0276] In some embodiments, the third information is used to instruct the terminal device to update the first model, and the terminal device can update the first model.

[0277] In some embodiments, the third information is used to instruct the terminal device to delete the first model, and the terminal device may delete the first model.

[0278] Using the above method, the first model can be tested or verified to determine whether it meets the first condition, thereby determining the model's performance and reliability. This enables online verification and testing of the model, improving the flexibility and reliability of model management.

[0279] The methods involved in the embodiments of this disclosure may include at least one of the steps S2101 to S2107 described above. For example, step S2105 may be implemented as an independent embodiment, step S2104 may be implemented as an independent embodiment, step S2102+S2105 may be implemented as an independent embodiment, step S2102+S2104 may be implemented as an independent embodiment, step S2105+S2106 may be implemented as an independent embodiment, step S2106+S2107 may be implemented as an independent embodiment, step S2102+S2104+S2105 may be implemented as an independent embodiment, step S2102+S2104+S2105+S2106 may be implemented as an independent embodiment, and step S2101+S2102+S2104+S2105 may be implemented as an independent embodiment, but are not limited thereto.

[0280] In some embodiments, steps S2101 to S2107 can be performed in different orders or simultaneously. For example, steps S2101 and S2104 can be performed in different orders or simultaneously.

[0281] In some embodiments, steps S2101 to S2107 are all optional steps. For example, steps S2101, S2102, S2104, S2106, and S2107 are optional, and one or more of these steps may be omitted or substituted in different embodiments. As another example, steps S2101, S2105, S2106, and S2107 are optional, and one or more of these steps may be omitted or substituted in different embodiments.

[0282] In some embodiments, other alternative implementations may be described before or after the specification corresponding to FIG2A.

[0283] In some embodiments, the names of information, etc., are not limited to the names described in the embodiments. Terms such as "information", "message", "signal", "signaling", "report", "configuration", "indication", "instruction", "command", "channel", "parameter", "domain", "field", "symbol", "symbol", "codebook", "codeword", "codepoint", "bit", "data", "program", and "chip" can be used interchangeably.

[0284] In some embodiments, “get,” “obtain,” “receive,” “transmit,” “bidirectional transmission,” and “send and / or receive” can be used interchangeably and can be interpreted as receiving from other entities, obtaining from protocols, obtaining from higher layers, obtaining through self-processing, or autonomous implementation, among other meanings.

[0285] In some embodiments, terms such as “send,” “transmit,” “report,” “distribute,” “transfer,” “bidirectional transmission,” “send and / or receive” can be used interchangeably.

[0286] Figure 2B is a flowchart illustrating a model testing method according to an embodiment of the present disclosure. This method can be executed by the aforementioned communication system. As shown in Figure 2B, the method may include:

[0287] Step S2201: The network device sends the first information to the terminal device.

[0288] The optional implementation of step S2201 can be found in the optional implementation of step S2102 in Figure 2A, and other related parts in the embodiments involved in Figure 2A, which will not be repeated here.

[0289] In some embodiments, the terminal device may receive first information. For example, the terminal device may receive first information sent by a network device. As another example, the terminal device may also receive first information sent by other entities.

[0290] In some embodiments, the network device may send first information to the terminal device to initiate testing of the first model when a first model is added or updated.

[0291] Step S2202: The terminal device sends the second information to the network device.

[0292] The optional implementation of step S2202 can be found in the optional implementation of step S2104 in Figure 2A, and other related parts in the embodiments involved in Figure 2A, which will not be repeated here.

[0293] In some embodiments, the network device may receive second information. For example, the network device may receive second information sent by a terminal device. As another example, the network device may also receive second information sent by other entities.

[0294] In some embodiments, the second information may include first measurement data. The terminal device may measure the received first information to obtain the first measurement data.

[0295] Step S2203: The network device uses the first model to make a prediction based on the second information and obtains the prediction data.

[0296] In one implementation, the first model can be deployed on the network device side, and the second information can include the aforementioned first measurement data.

[0297] Optionally, the network device can use the predicted data obtained by the first model based on the first measurement data. For example, after receiving the first measurement data, the network device can input the first measurement data into the first model to obtain the predicted data output by the first model.

[0298] For example, a network device can perform predictions based on configuration information. For instance, a network device can perform a prediction using a first model based on the configuration information to obtain a prediction result. This configuration information can be protocol-predefined information, or it can be information configured by the network device, or it can be information received by the network device from a terminal device, such as configuration information determined by the terminal device and sent to the network device.

[0299] In another implementation, the first model can be deployed on both the terminal device and the network device. The second information may include prediction data, or it may include first measurement data, or it may include both prediction data and first measurement data. If the network device receives the first measurement data, it can use the prediction data obtained by making a prediction based on the first model and the first measurement data. If the network device receives the prediction data, it can directly use the prediction data without re-performing the prediction. The communication system may contain a first terminal with the first model deployed and a second terminal without the first model deployed; the network device can be compatible with both types of terminal devices.

[0300] Step S2204: The network device determines whether the predicted data output by the first model meets the first condition.

[0301] The optional implementation of step S2204 can be found in the optional implementation of step S2105 in Figure 2A, as well as other related parts in the embodiments involved in Figure 2A, which will not be repeated here.

[0302] In some embodiments, if the network device determines that the predicted data output by the first model meets a first condition, it can use the first model to make predictions, thereby improving the performance and reliability of the predictions.

[0303] Figure 3A is a flowchart illustrating a model testing method according to an embodiment of the present disclosure. As shown in Figure 3A, this disclosure relates to a model testing method, which can be executed by a network device, and the method includes:

[0304] Step S3101: Obtain the fourth information.

[0305] The optional implementation of step S3101 can be found in the optional implementation of step S2101 in Figure 2A and other related parts in the embodiments involved in Figure 2A, which will not be repeated here.

[0306] In some embodiments, the network device may receive fourth information sent by the terminal device, but is not limited thereto; the network device may also receive fourth information sent by other entities.

[0307] In some embodiments, network devices may obtain fourth information as defined by a protocol.

[0308] In some embodiments, network devices may obtain fourth information from upper layer(s).

[0309] In some embodiments, the network device may process the information to obtain a fourth piece of information.

[0310] In some embodiments, step S3101 can be omitted, and the network device can autonomously implement the function indicated by the fourth information, or the above function can be defaulted or set to default.

[0311] Step S3102: Send the first message.

[0312] The optional implementation of step S3102 can be found in the optional implementation of step S2102 in Figure 2A, the optional implementation of step S2201 in Figure 2B, and other related parts in the embodiments involved in Figures 2A and 2B, which will not be repeated here.

[0313] In some embodiments, the network device may send the first information to the terminal device, but is not limited thereto; the network device may also send the first information to other entities.

[0314] Optionally, the first information can be used by the terminal device to perform measurements to obtain first measurement data. For example, the terminal device can receive the first information and measure the first information to obtain the first measurement data. Optional implementations can be found in the optional implementations of steps S2102 or S2104 in Figure 2A, and other related parts in the embodiments involved in Figure 2A, which will not be repeated here.

[0315] Step S3103: Obtain the second information.

[0316] The optional implementation of step S3103 can be found in the optional implementation of step S2104 in Figure 2A, the optional implementation of step S2202 in Figure 2B, and other related parts in the embodiments involved in Figures 2A and 2B, which will not be repeated here.

[0317] In some embodiments, the network device may receive second information sent by the terminal device, but is not limited thereto; the network device may also receive second information sent by other entities.

[0318] In some embodiments, a network device may obtain second information as defined by a protocol.

[0319] In some embodiments, the network device may obtain second information from the upper layer(s).

[0320] In some embodiments, the network device may process the information to obtain the second information.

[0321] In some embodiments, step S3103 may be omitted, and the network device may autonomously implement the function indicated by the second information, or the above function may be defaulted or set to default.

[0322] Step S3104: Determine whether the predicted data output by the first model meets the first condition.

[0323] The optional implementation of step S3104 can be found in the optional implementation of step S2105 in Figure 2A, the optional implementation of step S2204 in Figure 2B, and other related parts in the embodiments involved in Figures 2A and 2B, which will not be repeated here.

[0324] Step S3105: Send the third message.

[0325] The optional implementation of step S3105 can be found in the optional implementation of step S2106 in Figure 2A, as well as other related parts in the embodiments involved in Figure 2A, which will not be repeated here.

[0326] In some embodiments, the network device may send the third information to the terminal device, but is not limited thereto; the network device may also send the third information to other entities.

[0327] Optionally, the third information can be used by the terminal device to measure and obtain the first measurement data. For example, the terminal device can receive the third information and measure it to obtain the first measurement data. Optional implementations can be found in the optional implementations of steps S2102 or S2104 in Figure 2A, and other related parts in the embodiments involved in Figure 2A, which will not be repeated here.

[0328] Optionally, this third information can be used by the terminal device to determine whether the first model can be used. Optional implementations can be found in the optional implementations of step S2107 in Figure 2A, and other related parts in the embodiments involved in Figure 2A, which will not be repeated here.

[0329] In some embodiments, the network device may use a first model to make predictions based on second information to obtain prediction data. The specific implementation of this step can be found in the optional implementation of step S2203 in Figure 2B, as well as other related parts in the embodiments involved in Figure 2B, and will not be repeated here.

[0330] The methods involved in the embodiments of this disclosure may include at least one of the steps S3101 to S3105 described above. For example, step S3104 may be implemented as an independent embodiment, step S3102+S3104 may be implemented as an independent embodiment, step S3104+S3105 may be implemented as an independent embodiment, step S3102+S3103+S3104 may be implemented as an independent embodiment, and step S3102+S3103+S3104+S3105 may be implemented as an independent embodiment, but are not limited thereto.

[0331] In some embodiments, steps S3101 to S3105 can be performed in different orders or simultaneously. For example, steps S3101 and S3103 can be performed in different orders or simultaneously.

[0332] In some embodiments, steps S3101 to S3105 are all optional steps. For example, steps S3101, S3102, S3103, and S3105 are optional, and one or more of these steps may be omitted or substituted in different embodiments.

[0333] In some embodiments, other optional implementations may be described before or after the specification corresponding to Figure 3.

[0334] Figure 3B is a flowchart illustrating a model testing method according to an embodiment of the present disclosure. As shown in Figure 3B, the present disclosure relates to a model testing method, which can be executed by a network device. The method may include:

[0335] Step S3201: Send the first message.

[0336] The optional implementation of step S3201 can be found in step S2102 of Figure 2A, the optional implementation of step S3102 of Figure 3A, and other related parts in the embodiments involved in Figures 2A and 3A, which will not be repeated here.

[0337] Step S3202: Determine whether the predicted data output by the first model meets the first condition.

[0338] The optional implementation of step S3202 can be found in the optional implementation of step S2105 in Figure 2A, step S2204 in Figure 2B, step S3104 in Figure 3A, and other related parts in the embodiments involved in Figures 2A, 2B, and 3A, which will not be repeated here.

[0339] In some embodiments, the above steps are all optional.

[0340] In some embodiments, the embodiment shown in FIG3B can also be combined with step S3103 in the embodiment shown in FIG3A as a new embodiment.

[0341] In some embodiments, the embodiment shown in FIG3B can also be combined with step S3101 in the embodiment shown in FIG3A as a new embodiment.

[0342] In some embodiments, the embodiment shown in FIG3B can also be combined with steps S3103 and S3105 in the embodiment shown in FIG3A as a new embodiment.

[0343] In some embodiments, the embodiment shown in FIG3B can also be used as a new embodiment along with steps S3101 and S3103 in the embodiment shown in FIG3A.

[0344] In some embodiments, the embodiment shown in FIG3B can also be combined with steps S3101, S3103 and S3105 in the embodiment shown in FIG3A as a new embodiment.

[0345] Figure 3C is a flowchart illustrating a model testing method according to an embodiment of the present disclosure. As shown in Figure 3C, the present disclosure relates to a model testing method, which can be executed by a network device. The method may include:

[0346] Step S3301: Determine whether the predicted data output by the first model meets the first condition.

[0347] The optional implementation of step S3301 can be found in the optional implementation of step S2105 in Figure 2A, step S2204 in Figure 2B, step S3104 in Figure 3A, and other related parts in the embodiments involved in Figures 2A, 2B, and 3A, which will not be repeated here.

[0348] Step S3302: Send the third message.

[0349] The optional implementation of step S3302 can be found in the optional implementation of step S2106 in Figure 2A, step S3105 in Figure 3A, and other related parts in the embodiments involved in Figures 2A and 3A, which will not be repeated here.

[0350] In some embodiments, the above steps are all optional.

[0351] In some embodiments, the embodiment shown in FIG3C can also be combined with step S3103 in the embodiment shown in FIG3A as a new embodiment.

[0352] In some embodiments, the embodiment shown in FIG3C can also be combined with steps S3102 and S3103 in the embodiment shown in FIG3A as a new embodiment.

[0353] In some embodiments, the embodiment shown in FIG3C can also be combined with steps S3102, S3103 and S3103 in the embodiment shown in FIG3A as a new embodiment.

[0354] Figure 3D is a flowchart illustrating a model testing method according to an embodiment of the present disclosure. As shown in Figure 3C, this disclosure relates to a model testing method, which can be executed by a network device. The method may include:

[0355] Step S3401: Determine whether the predicted data output by the first model meets the first condition.

[0356] The optional implementation of step S3401 can be found in the optional implementation of step S2105 in Figure 2A, step S2204 in Figure 2B, step S3104 in Figure 3A, and other related parts in the embodiments involved in Figures 2A, 2B, and 3A, which will not be repeated here.

[0357] In some embodiments, the embodiment shown in FIG3C can also be combined with at least one of steps S3101, S3102, S3103, and S3105 in the embodiment shown in FIG3A as a new embodiment.

[0358] In some embodiments, the first condition is the performance requirement of the predicted data, the predicted data is the data predicted by the first model based on the first measurement data, the first measurement data is the data obtained by the terminal device after measuring the first information, and the first information is the information sent by the network device to the terminal device.

[0359] In some embodiments, the method further includes:

[0360] The terminal device sends a second message, which includes the prediction data.

[0361] In some embodiments, the method further includes:

[0362] Send a third message to the terminal device; the third message is used to indicate whether the first model can be used by the terminal device.

[0363] In some embodiments, the method further includes:

[0364] Receive fourth information; the fourth information is used to trigger the network device to send first information, the fourth information including the first model identifier of the first model;

[0365] Based on the fourth information, it is determined that the first model meets the test start conditions, and the first information is sent to the terminal device.

[0366] In some embodiments, the test initiation conditions include at least one of the following:

[0367] The first model has not been tested;

[0368] The first model is not in the model management set; the model management set includes models that have already been tested.

[0369] The fourth piece of information is used to instruct the execution of a test on the first model.

[0370] In some embodiments, the fourth information includes at least one of the following:

[0371] Model update indication information;

[0372] Add instruction information to the model;

[0373] Model test instruction information.

[0374] In some embodiments, the first model is a model for performing beam prediction, and the prediction data includes the predicted signal quality of a first beam, wherein the first beam is at least one beam configured by the network device for the terminal device.

[0375] The first condition includes the difference between the predicted signal quality and the first signal quality within a first difference range; wherein, the first signal quality is the signal quality of the first beam generated by the network device according to the beam configuration information, or, the first signal quality is the signal quality of the first beam actually measured by the terminal device.

[0376] In some embodiments, the first model is a model for performing beam prediction, and the prediction data includes a second beam that satisfies the signal conditions obtained by the beam prediction.

[0377] The signal conditions include at least one of the following:

[0378] The signal quality of the second beam is greater than or equal to a preset signal quality threshold;

[0379] The second beam is the beam with the strongest signal quality among the multiple beams configured by the network device for the terminal device.

[0380] In some embodiments, the first condition is that the number of times the prediction success condition is met in the N beam predictions performed is greater than or equal to M; where N is a natural number and M is a natural number less than or equal to N; the prediction success condition is that the second beam is the same as the third beam, and the second beam is the beam that meets the signal condition obtained by performing beam prediction.

[0381] The third beam is a beam that satisfies the signal conditions, determined by the network device based on beam configuration information; or, the third beam is a beam that satisfies the signal conditions, determined based on second measurement data, whereby the terminal device measures multiple beams configured for it by the network device.

[0382] In some embodiments, the signal quality includes at least one of the following: RSRP, SINR, RSSI, and RSRQ.

[0383] In some embodiments, the first information includes at least one of the following:

[0384] First data used to test the first model;

[0385] The first signal is used to test the first model.

[0386] Figure 4A is a flowchart illustrating a model testing method according to an embodiment of the present disclosure. As shown in Figure 4A, this disclosure relates to a model testing method, which can be executed by a terminal device. The method may include:

[0387] Step S4101: Send the fourth message.

[0388] The optional implementation of step S4101 can be found in the optional implementation of step S2101 in Figure 2A, and other related parts in the embodiments involved in Figure 2A, which will not be repeated here.

[0389] In some embodiments, the terminal device may send the fourth information to the network device, but is not limited thereto; the terminal device may also send the fourth information to other entities.

[0390] Step S4102: Obtain the first information.

[0391] The optional implementation of step S4102 can be found in the optional implementation of step S2102 in Figure 2A and other related parts in the embodiments involved in Figure 2A, which will not be repeated here.

[0392] In some embodiments, the terminal device may receive first information sent by the network device, but is not limited thereto; the terminal device may also receive first information sent by other entities.

[0393] In some embodiments, the terminal device may obtain first information as defined by the protocol.

[0394] In some embodiments, the terminal device may obtain first information from the upper layer(s).

[0395] In some embodiments, the terminal device may perform processing to obtain the first information.

[0396] In some embodiments, step S4102 may be omitted, and the terminal device may autonomously implement the function indicated by the first information, or the above function may be defaulted or set to default.

[0397] Step S4103: Using the first model, make a prediction based on the first information to obtain the prediction data.

[0398] The optional implementation of step S4103 can be found in the optional implementation of step S2103 in Figure 2A and other related parts in the embodiments involved in Figure 2A, which will not be repeated here.

[0399] Step S4104: Send the second message.

[0400] The optional implementation of step S4104 can be found in the optional implementation of step S2104 in Figure 2A and other related parts in the embodiments involved in Figure 2A, which will not be repeated here.

[0401] In some embodiments, the terminal device may send the second information to the network device, but is not limited thereto; the terminal device may also send the second information to other entities.

[0402] Step S4105: Obtain third information.

[0403] The optional implementation of step S4105 can be found in the optional implementation of step S2106 in Figure 2A, as well as other related parts in the embodiments involved in Figure 2A, which will not be repeated here.

[0404] In some embodiments, the terminal device may receive third information sent by the network device, but is not limited thereto; the terminal device may also receive third information sent by other entities.

[0405] In some embodiments, the terminal device may obtain third information as defined by the protocol.

[0406] In some embodiments, the terminal device may obtain third information from the upper layer(s).

[0407] In some embodiments, the terminal device may process information to obtain third information.

[0408] In some embodiments, step S4102 may be omitted, and the terminal device may autonomously implement the function indicated by the third information, or the above function may be defaulted or set to default.

[0409] Step S4106: Determine whether the first model can be used.

[0410] The optional implementation of step S4106 can be found in the optional implementation of step S2107 in Figure 2A, as well as other related parts in the embodiments involved in Figure 2A, which will not be repeated here.

[0411] The methods involved in the embodiments of this disclosure may include at least one of the steps S4101 to S4106 described above. For example, step S4104 may be implemented as an independent embodiment, step S4103+S4104 may be implemented as an independent embodiment, step S4102+S4104 may be implemented as an independent embodiment, step S4105+S4106 may be implemented as an independent embodiment, step S4102+S4103+S4104 may be implemented as an independent embodiment, step S4101+S4102+S4104 may be implemented as an independent embodiment, and step S4101+S4105+S4106 may be implemented as an independent embodiment, but are not limited thereto.

[0412] In some embodiments, steps S4101 to S4106 can be performed in different orders or simultaneously. For example, steps S4101 and S4104 can be performed in different orders or simultaneously.

[0413] In some embodiments, steps S4101 to S4106 are all optional steps. For example, steps S4101, S4102, S4103, S4105, and S4106 are optional, and one or more of these steps may be omitted or substituted in different embodiments. As another example, steps S4101, S4102, and S4104 are optional, and one or more of these steps may be omitted or substituted in different embodiments.

[0414] In some embodiments, other optional implementations may be described before or after the specification corresponding to Figure 4.

[0415] Figure 4B is a flowchart illustrating a model testing method according to an embodiment of the present disclosure. As shown in Figure 4B, this disclosure relates to a model testing method, which can be executed by a terminal device. The method may include:

[0416] Step S4201: Obtain the first information.

[0417] The optional implementation of step S4201 can be found in step S2102 of Figure 2A, the optional implementation of step S4102 of Figure 4A, and other related parts in the embodiments involved in Figures 2A and 4A, which will not be repeated here.

[0418] Step S4202: Send the second message.

[0419] The optional implementation of step S4202 can be found in step S2104 of Figure 2A, the optional implementation of step S4104 of Figure 4A, and other related parts in the embodiments involved in Figures 2A and 4A, which will not be repeated here.

[0420] In some embodiments, the above steps are all optional.

[0421] In some embodiments, the embodiment shown in FIG4B can also be combined with at least one of steps S4101, S4105, and S4106 in the embodiment shown in FIG4A as a new embodiment.

[0422] Figure 4C is a flowchart illustrating a model testing method according to an embodiment of the present disclosure. As shown in Figure 4C, this disclosure relates to a model testing method, which can be executed by a terminal device. The method may include:

[0423] Step S4301: Obtain third information.

[0424] The optional implementation of step S4301 can be found in the optional implementation of step S2106 in Figure 2A, step S4105 in Figure 4A, and other related parts in the embodiments involved in Figures 2A and 4A, which will not be repeated here.

[0425] Step S4302: Determine whether the first model can be used.

[0426] The optional implementation of step S4302 can be found in the optional implementation of step S2107 in Figure 2A, step S4106 in Figure 4A, and other related parts in the embodiments involved in Figures 2A and 4A, which will not be repeated here.

[0427] In some embodiments, the above steps are all optional.

[0428] In some embodiments, the embodiment shown in FIG4C can also be combined with at least one of steps S4101, S4102, and S4104 in the embodiment shown in FIG4A as a new embodiment.

[0429] Figure 4D is a flowchart illustrating a model testing method according to an embodiment of the present disclosure. As shown in Figure 4D, the present disclosure relates to a model testing method, which can be executed by a terminal device. The method may include:

[0430] Step S4401: Using the first model, make a prediction based on the first information to obtain the prediction data.

[0431] The optional implementation of step S4401 can be found in step S2103 of Figure 2A, the optional implementation of step S4103 of Figure 4A, and other related parts in the embodiments involved in Figures 2A and 4A, which will not be repeated here.

[0432] Step S4402: Send the second message.

[0433] The optional implementation of step S4402 can be found in step S2104 of Figure 2A, the optional implementation of step S4104 of Figure 4A, and other related parts in the embodiments involved in Figures 2A and 4A, which will not be repeated here.

[0434] In some embodiments, the above steps are all optional.

[0435] In some embodiments, the embodiment shown in FIG4B can also be combined with at least one of steps S4101, S4105, and S4106 in the embodiment shown in FIG4A as a new embodiment.

[0436] In some embodiments, the first information is information sent from the network device to the terminal device.

[0437] In some embodiments, the second information includes the prediction data, which is used to determine whether the first model meets a first condition, wherein the first condition is the performance requirement of the prediction data.

[0438] In some embodiments, the method further includes:

[0439] Receive the first information;

[0440] The step of using the first model to make predictions based on the first information to obtain the predicted data includes:

[0441] The first information is measured to obtain the first measurement data;

[0442] The first measurement data is input into the first model to obtain the predicted data output by the first model.

[0443] In some embodiments, the method further includes:

[0444] Receive third information; the third information is used to indicate whether the first model can be used by the terminal device;

[0445] The third information is used to determine whether the first model can be used.

[0446] In some embodiments, the method further includes:

[0447] Send a fourth message; wherein the fourth message is used to trigger the network device to send the first message, and the fourth message includes the first model identifier of the first model;

[0448] In some embodiments, the fourth information includes at least one of the following:

[0449] Model update indication information;

[0450] Add instruction information to the model;

[0451] Model test instruction information.

[0452] In some embodiments, the first model is a model for performing beam prediction, and the prediction data includes the predicted signal quality of a first beam, wherein the first beam is at least one beam configured by the network device for the terminal device.

[0453] The first condition includes the difference between the predicted signal quality and the first signal quality within a first difference range; wherein, the first signal quality is the signal quality of the first beam generated by the network device according to the beam configuration information, or, the first signal quality is the signal quality of the first beam actually measured by the terminal device.

[0454] In some embodiments, the first model is a model for performing beam prediction, and the prediction data includes a second beam that satisfies the signal conditions obtained by the beam prediction.

[0455] The signal conditions include at least one of the following:

[0456] The signal quality of the second beam is greater than or equal to a preset signal quality threshold;

[0457] The second beam is the beam with the strongest signal quality among the multiple beams configured by the network device for the terminal device.

[0458] In some embodiments, the first condition is that the number of times the prediction success condition is met in the N beam predictions performed is greater than or equal to M; where N is a natural number and M is a natural number less than or equal to N; the prediction success condition is that the second beam is the same as the third beam, and the second beam is the beam that meets the signal condition obtained by performing beam prediction.

[0459] The third beam is a beam that satisfies the signal conditions, determined by the network device based on beam configuration information; or, the third beam is a beam that satisfies the signal conditions, determined based on second measurement data, whereby the terminal device measures multiple beams configured for it by the network device.

[0460] In some embodiments, the signal quality includes at least one of the following: RSRP, SINR, RSSI, and RSRQ.

[0461] In some embodiments, the first information includes at least one of the following:

[0462] First data used to test the first model;

[0463] The first signal is used to test the first model.

[0464] Figure 5 is a flowchart illustrating a model testing method according to an embodiment of the present disclosure. As shown in Figure 5, the present disclosure relates to a model testing method, which may include:

[0465] Step S5101: The terminal device uses the first model to make a prediction based on the first information and obtains the prediction data.

[0466] The first information is information sent from the network device to the terminal device.

[0467] The optional implementation of step S5101 can be found in step S2103 of Figure 2A, the optional implementation of step S4103 of Figure 4A, and other related parts in the embodiments involved in Figures 2A and 4A, which will not be repeated here.

[0468] Step S5102: The terminal device sends the second information to the network device.

[0469] The optional implementation of step S5102 can be found in step S2104 of Figure 2A, step S3103 of Figure 3A, step S4104 of Figure 4A, and other related parts in the embodiments involved in Figures 2A, 3A, and 4A, which will not be repeated here.

[0470] In some embodiments, the second information may include the aforementioned prediction data.

[0471] Step S5103: The network device determines whether the predicted data output by the first model meets the first condition.

[0472] The optional implementation of step S5103 can be found in step S2105 of Figure 2A, step S3104 of Figure 3A, and other related parts in the embodiments involved in Figures 2A and 3A, which will not be repeated here.

[0473] In some embodiments, the above methods may include the methods described in the embodiments of the communication system, terminal device, network device, etc., which will not be repeated here.

[0474] Figure 6 is a flowchart illustrating a model testing method according to an embodiment of the present disclosure. As shown in Figure 6, the present disclosure relates to a model testing method, which can be executed by a communication system and may include:

[0475] Step S6101: The terminal device sends the fourth information to the network device.

[0476] The fourth piece of information may be AI / ML model addition instructions, AI / ML model update instructions, or AI / ML model online testing instructions. This fourth piece of information must include at least the first model identifier of the first model.

[0477] For example, the terminal device can send instructions to the network device to add or update an AI / ML model, or the terminal device can send instructions to the network device to test an AI / ML model online. These instructions must include at least the ID of the target model.

[0478] Step S6102: The network device sends the second information to the terminal device based on the fourth information.

[0479] The second piece of information may be data and / or signals used for online testing of the target model.

[0480] For example, the network device can determine whether it needs to send data and / or signals for online testing of the target model to the UE based on the received fourth information. If the target model is not in the model monitoring and management list (i.e., the model has not been trained or its performance has been verified), or after the network device receives the above-mentioned AI / ML model online testing instruction information, the network device can send the second line (online test data and / or signals) to the terminal device. The data and / or signals can be determined according to different scenarios. For example, data (PDSCH) is used for CSI feedback based on the AL / ML model; signals (CSI-RS / SSB) are used for beam management based on the AL / ML model; and signals (PRS) are used for positioning based on the AL / ML model.

[0481] Step S6103: The terminal device makes a prediction and reports the prediction data to the network device.

[0482] This predicted data can also be referred to as the prediction result.

[0483] For example, the terminal device can use the first model to predict the result based on the configuration information of the test data / signals configured by the network, and report the prediction result to the network.

[0484] Step S6104: The network device determines whether the predicted data output by the first model meets the first condition.

[0485] This first condition may include predefined performance requirements.

[0486] For example, a network device can assess whether the predefined performance requirements are met based on the reported prediction results. If they are met, the network device can send an indication message to the terminal device allowing the use of the target model. After receiving the indication message, the terminal device can add / update the first model.

[0487] In some embodiments, the network device configures a test reference signal for beam management to the terminal device. The terminal device infers and predicts the signal strength (L1-RSRP signal strength) of the beam based on an AI / ML model to obtain the strongest beam or beam group. The terminal device reports the predicted best beam or beam group and its signal strength to the network device. The network device evaluates whether the performance requirements of the prediction results reported by the terminal device meet the predefined performance indicators. If the performance indicator is RSRP signal strength accuracy, where signal strength accuracy = predicted result value – ideal value, and the ideal value is the signal strength value simulated by TE according to the configuration parameters, and the signal strength accuracy requirement meets the predefined performance indicator requirements, then the AI / ML model is considered reliable. If the performance indicator is prediction reliability, then the prediction reliability percentage must meet the predefined prediction reliability requirements. For example, if N predictions are performed and the number of successful predictions is M, then the prediction reliability is (M / N)*100%. The successful prediction criterion is whether the best beam or beam group reported by the terminal device is consistent with the ideal best beam or beam group simulated by the network device. If the network device determines and evaluates that the prediction results of the AI / ML model are reliable, it sends an instruction to the terminal device that it is allowed to use / use the model. The terminal device adds / updates the target model according to the received instruction.

[0488] In some embodiments, the network device configures a beam management test reference signal to the terminal device. The terminal device infers and predicts the signal strength of the beam (L1-RSRP signal strength) based on an AI / ML model, deriving the strongest beam or beam group. The terminal device reports the predicted best beam or beam group and its signal strength to the network device. Simultaneously, the terminal device measures the test reference signal using conventional methods and derives the strongest beam / beam group and its signal strength, reporting the predicted best beam or beam group and its signal strength to the network device. The network device evaluates whether the prediction results reported by the terminal device meet predefined performance requirements. If the performance metric is RSRP signal strength accuracy, where signal strength accuracy = predicted result value – conventional measurement value, and the signal strength accuracy requirement meets the predefined performance metric, then the AI / ML model is considered reliable. If the performance metric is prediction reliability, then the prediction reliability percentage must meet the predefined prediction reliability requirement. For example, if N predictions are performed and the number of successful predictions is M, then the prediction reliability is (M / N)*100%. The successful prediction criterion is whether the optimal beam or beam group reported by the terminal device is consistent with the optimal beam or beam group measured by the terminal device using traditional methods. If the network device judges and evaluates the prediction result of the AI / ML model to be reliable, it sends an instruction message to the terminal device allowing the use of the model. The terminal device adds / updates the target model according to the received instruction message.

[0489] In some embodiments of this disclosure, a communication system is provided, which may include a terminal device and a network device, wherein the terminal device may execute the model testing method executed by the terminal device in the foregoing embodiments of this disclosure; and the network device may execute the model testing method executed by the network device in the foregoing embodiments of this disclosure.

[0490] This disclosure also provides an apparatus for implementing any of the above methods. For example, an apparatus is provided that includes units or modules for implementing the steps performed by the terminal device in any of the above methods. Alternatively, another apparatus is provided that includes units or modules for implementing the steps performed by the network device (e.g., access network device, core network functional node, core network device, etc.) in any of the above methods.

[0491] It should be understood that the division of units or modules in the above device is only a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, the units or modules in the device can be implemented by a processor calling software: for example, the device includes a processor connected to a memory containing instructions. The processor calls the instructions stored in the memory to implement any of the above methods or to implement the functions of the units or modules in the above device. The processor can be, for example, a general-purpose processor, such as a Central Processing Unit (CPU) or a microprocessor, and the memory can be internal or external to the device. Alternatively, the units or modules in the device can be implemented in the form of hardware circuits. The functionality of some or all of the units or modules can be achieved through the design of these hardware circuits, which can be understood as one or more processors. For example, in one implementation, the hardware circuit is an Application-Specific Integrated Circuit (ASIC), and the functionality of some or all of the units or modules is achieved through the design of the logical relationships between the components within the circuit. In another implementation, the hardware circuit can be implemented using a Programmable Logic Device (PLD), such as a Field Programmable Gate Array (FPGA), which can include a large number of logic gates. The connection relationships between the logic gates are configured through configuration files, thereby achieving the functionality of some or all of the units or modules. All units or modules of the above device can be implemented entirely through processor-called software, entirely through hardware circuits, or partially through processor-called software with the remaining parts implemented through hardware circuits.

[0492] In this embodiment, the processor is a circuit with signal processing capabilities. In one implementation, the processor can be a circuit with instruction read and execute capabilities, such as a Central Processing Unit (CPU), a microprocessor, a Graphics Processing Unit (GPU) (which can be understood as a microprocessor), or a Digital Signal Processor (DSP). In another implementation, the processor can implement certain functions through the logical relationships of hardware circuits. The logical relationships of the aforementioned hardware circuits are fixed or reconfigurable. For example, the processor is a hardware circuit implemented using an Application-Specific Integrated Circuit (ASIC) or a Programmable Logic Device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units or modules. In addition, it can also be hardware circuits designed for artificial intelligence, which can be understood as ASICs, such as Neural Network Processing Units (NPUs), Tensor Processing Units (TPUs), and Deep Learning Processing Units (DPUs).

[0493] Figure 7A is a schematic diagram of the structure of a terminal device according to an embodiment of this disclosure. As shown in Figure 7A, the terminal device 101 may include at least one of a transceiver module 8101, a processing module 8102, etc. In some embodiments, the processing module 8102 is configured to use a first model to make a prediction based on first information to obtain the prediction data; wherein, the first information is information sent from a network device to the terminal device; the transceiver module 8101 is configured to send second information, the second information including the prediction data, the prediction data being used to determine whether the first model satisfies a first condition, the first condition being the performance requirements of the prediction data. Optionally, the transceiver module 8101 may be used to execute at least one of the communication steps (e.g., steps S2101, S2102, S2104, S2106, but not limited thereto) performed by the terminal device 101 in any of the above methods, which will not be elaborated here. Optionally, the processing module 8102 may be used to execute at least one of the other steps (such as steps S2103, S2105, and S2107, but not limited thereto) executed by the terminal device 101 in any of the above methods, which will not be elaborated here.

[0494] Figure 7B is a schematic diagram of the structure of a network device according to an embodiment of this disclosure. As shown in Figure 7B, the network device 102 may include at least one of a transceiver module 8201, a processing module 8202, etc. In some embodiments, the transceiver module 8201 is configured to send first information; wherein the first information is information for testing a first model; the processing module 8202 is configured to determine whether the prediction data output by the first model satisfies a first condition; wherein the first condition is the performance requirement of the prediction data, the prediction data is data predicted by the first model based on first measurement data, the first measurement data is data obtained by the terminal device after measuring the first information, and the first information is information sent by the network device to the terminal device. Optionally, the transceiver module 8201 may be used to perform at least one of the communication steps such as sending and / or receiving performed by the network device 102 in any of the above methods (e.g., steps S2101, S2102, S2104, S2106, but not limited thereto), which will not be elaborated here. Optionally, the processing module 8202 may be used to perform at least one of the other steps (such as steps S2103, S2105, and S2107, but not limited thereto) performed by the network device 102 in any of the above methods, which will not be elaborated here.

[0495] In some embodiments, the transceiver module may include a transmitting module and / or a receiving module, which may be separate or integrated. Optionally, the transceiver module may be interchangeable with a transceiver.

[0496] In some embodiments, the processing module may be a single module or may include multiple sub-modules. Optionally, the multiple sub-modules may each perform all or part of the steps required by the processing module. Optionally, the processing module may be interchangeable with a processor.

[0497] Figure 8A is a schematic diagram of the structure of the communication device 9100 proposed in an embodiment of this disclosure. The communication device 9100 can be a network device (e.g., access network device, core network device, etc.), a terminal device (e.g., user equipment, etc.), a chip, chip system, or processor that supports the network device in implementing any of the above methods, or a chip, chip system, or processor that supports the terminal device in implementing any of the above methods. The communication device 9100 can be used to implement the methods described in the above method embodiments; for details, please refer to the descriptions in the above method embodiments.

[0498] As shown in Figure 8A, the communication device 9100 includes one or more processors 9101. The processor 9101 can be a general-purpose processor or a dedicated processor, such as a baseband processor or a central processing unit (CPU). The baseband processor can be used to process communication protocols and communication data, while the CPU can be used to control communication devices (e.g., base stations, baseband chips, terminal devices, terminal device chips, DUs or CUs, etc.), execute programs, and process program data. The communication device 9100 is used to execute any of the above methods.

[0499] In some embodiments, the communication device 9100 further includes one or more memories 9102 for storing instructions. Optionally, all or part of the memories 9102 may also be located outside the communication device 9100.

[0500] In some embodiments, the communication device 9100 further includes one or more transceivers 9103. When the communication device 9100 includes one or more transceivers 9103, the transceivers 9103 perform at least one of the communication steps such as sending and / or receiving in the above method (e.g., steps S2101, S2102, S2104, S2106, but not limited thereto), and the processor 9101 performs at least one of other steps (e.g., steps S2103, S2105, S2107, but not limited thereto).

[0501] In some embodiments, a transceiver may include a receiver and / or a transmitter, which may be separate or integrated. Optionally, the terms transceiver, transceiver unit, transceiver, transceiver circuit, etc., may be used interchangeably; the terms transmitter, transmitting unit, transmitter, transmitting circuit, etc., may be used interchangeably; and the terms receiver, receiving unit, receiver, receiving circuit, etc., may be used interchangeably.

[0502] In some embodiments, the communication device 9100 may include one or more interface circuits. Optionally, the interface circuit is connected to the memory 9102, and the interface circuit can be used to receive signals from the memory 9102 or other devices, and can be used to send signals to the memory 9102 or other devices. For example, the interface circuit can read instructions stored in the memory 9102 and send the instructions to the processor 9101.

[0503] The communication device 9100 described in the above embodiments may be a network device or a terminal device, but the scope of the communication device 9100 described in this disclosure is not limited thereto, and the structure of the communication device 9100 may not be limited by FIG8A. The communication device may be a standalone device or may be part of a larger device. For example, the communication device may be: (1) a standalone integrated circuit IC, or chip, or chip system or subsystem; (2) a collection of one or more ICs, optionally, the IC collection may also include storage components for storing data and programs; (3) an ASIC, such as a modem; (4) a module that can be embedded in other devices; (5) a receiver, terminal device, smart terminal device, cellular phone, wireless device, handheld device, mobile unit, vehicle device, network device, cloud device, artificial intelligence device, etc.; (6) others, etc.

[0504] Figure 8B is a schematic diagram of the structure of the chip 9200 proposed in an embodiment of this disclosure. For cases where the communication device 9100 can be a chip or a chip system, the schematic diagram of the chip 9200 shown in Figure 8B can be referenced, but is not limited thereto.

[0505] Chip 9200 includes one or more processors 9201, which are used to perform any of the above methods.

[0506] In some embodiments, chip 9200 further includes one or more interface circuits 9203. Optionally, interface circuit 9203 is connected to memory 9202, and interface circuit 9203 can be used to receive signals from memory 9202 or other devices, and interface circuit 9203 can be used to send signals to memory 9202 or other devices. For example, interface circuit 9203 can read instructions stored in memory 9202 and send the instructions to processor 9201.

[0507] In some embodiments, the interface circuit 9203 performs at least one of the communication steps such as sending and / or receiving in the above method (e.g., steps S2101, S2102, S2104, S2106, but not limited thereto), and the processor 9201 performs at least one of the other steps (e.g., steps S2103, S2105, S2107, but not limited thereto).

[0508] In some embodiments, the terms interface circuit, interface, transceiver pin, transceiver, etc., can be used interchangeably.

[0509] In some embodiments, chip 9200 further includes one or more memories 9202 for storing instructions. Optionally, all or part of the memories 9202 may be located outside of chip 9200.

[0510] This disclosure also proposes a storage medium storing instructions that, when executed on a communication device 9100, cause the communication device 9100 to perform any of the above methods. Optionally, the storage medium is an electronic storage medium. Optionally, the storage medium is a computer-readable storage medium, but not limited thereto; it may also be a storage medium readable by other devices. Optionally, the storage medium may be a non-transitory storage medium, but not limited thereto; it may also be a temporary storage medium.

[0511] This disclosure also provides a program product that, when executed by the communication device 9100, causes the communication device 9100 to perform any of the above methods. Optionally, the program product may be a computer program product.

[0512] This disclosure also proposes a computer program that, when run on a computer, causes the computer to perform any of the above methods.

Claims

1. A model testing method, characterized in that, The method includes: Determine whether the predicted data output by the first model meets the first condition; wherein, the first condition is the performance requirement of the predicted data, the predicted data is the data predicted by the first model based on the first measurement data, the first measurement data is the data obtained by the terminal device after measuring the first information, and the first information is the information sent by the network device to the terminal device.

2. The method according to claim 1, characterized in that, The method further includes: The terminal device sends a second message, which includes the prediction data.

3. The method according to claim 1, characterized in that, The method further includes: Send a third message to the terminal device; the third message is used to indicate whether the first model can be used by the terminal device.

4. The method according to any one of claims 1 to 3, characterized in that, The method further includes: Receive fourth information; wherein the fourth information is used to trigger the network device to send first information, and the fourth information includes the first model identifier of the first model; Based on the fourth information, it is determined that the first model meets the test start conditions, and the first information is sent to the terminal device.

5. The method according to claim 4, characterized in that, The test initiation conditions include at least one of the following: The first model has not been tested; The first model is not in the model management set; the model management set includes models that have already been tested. The fourth piece of information is used to instruct the execution of a test on the first model.

6. The method according to claim 4 or 5, characterized in that, The fourth piece of information includes at least one of the following: Model update indication information; Add instruction information to the model; Model test instruction information.

7. The method according to any one of claims 1 to 6, characterized in that, The first model is a model for performing beam prediction, and the prediction data includes the predicted signal quality of the first beam, which is at least one beam configured by the network device for the terminal device. The first condition includes the difference between the predicted signal quality and the first signal quality within a first difference range; wherein, the first signal quality is the signal quality of the first beam generated by the network device according to the beam configuration information, or, the first signal quality is the signal quality of the first beam actually measured by the terminal device.

8. The method according to any one of claims 1 to 6, characterized in that, The first model is a model for performing beam prediction, and the prediction data includes a second beam that satisfies the signal conditions obtained by the beam prediction; The signal conditions include at least one of the following: The signal quality of the second beam is greater than or equal to a preset signal quality threshold; The second beam is the beam with the strongest signal quality among the multiple beams configured by the network device for the terminal device.

9. The method according to claim 8, characterized in that, The first condition is that the number of times the prediction success condition is met in the N beam predictions is greater than or equal to M; where N is a natural number and M is a natural number less than or equal to N; the prediction success condition is that the second beam and the third beam are the same, and the second beam is the beam that meets the signal condition obtained by performing beam prediction. The third beam is a beam that satisfies the signal conditions, determined by the network device based on beam configuration information; or, the third beam is a beam that satisfies the signal conditions, determined based on second measurement data, whereby the terminal device measures multiple beams configured for it by the network device.

10. The method according to any one of claims 7 to 9, characterized in that, The signal quality includes at least one of the following: RSRP, SINR, RSSI, and RSRQ.

11. The method according to any one of claims 1 to 12, characterized in that, The first information includes at least one of the following: First data used to test the first model; The first signal is used to test the first model.

12. A model testing method, characterized in that, The method includes: Using a first model, predictions are made based on first information to obtain the predicted data; wherein, the first information is information sent from the network device to the terminal device; Send a second message, the second message including the prediction data, the prediction data being used to determine whether the first model meets a first condition, the first condition being the performance requirements of the prediction data.

13. The method according to claim 12, characterized in that, The method further includes: Receive the first information; The step of using the first model to make predictions based on the first information to obtain the predicted data includes: The first information is measured to obtain the first measurement data; The first measurement data is input into the first model to obtain the predicted data output by the first model.

14. The method according to claim 12 or 13, characterized in that, The method further includes: Receive third information; the third information is used to indicate whether the first model can be used by the terminal device; The third information is used to determine whether the first model can be used.

15. The method according to any one of claims 12 to 14, characterized in that, The method further includes: Send a fourth message; wherein the fourth message is used to trigger the network device to send a first message, and the fourth message includes a first model identifier of the first model.

16. The method according to claim 15, characterized in that, The fourth piece of information includes at least one of the following: Model update indication information; Add instruction information to the model; Model test instruction information.

17. The method according to any one of claims 12 to 16, characterized in that, The first model is a model for performing beam prediction, and the prediction data includes the predicted signal quality of the first beam, which is at least one beam configured by the network device for the terminal device. The first condition includes the difference between the predicted signal quality and the first signal quality within a first difference range; wherein, the first signal quality is the signal quality of the first beam generated by the network device according to the beam configuration information, or, the first signal quality is the signal quality of the first beam actually measured by the terminal device.

18. The method according to any one of claims 12 to 16, characterized in that, The first model is a model for performing beam prediction, and the prediction data includes a second beam that satisfies the signal conditions obtained by the beam prediction; The signal conditions include at least one of the following: The signal quality of the second beam is greater than or equal to a preset signal quality threshold; The second beam is the beam with the strongest signal quality among the multiple beams configured by the network device for the terminal device.

19. The method according to claim 18, characterized in that, The first condition is that the number of times the prediction success condition is met in the N beam predictions is greater than or equal to M; where N is a natural number and M is a natural number less than or equal to N; the prediction success condition is that the second beam and the third beam are the same, and the second beam is the beam that meets the signal condition obtained by performing beam prediction. The third beam is a beam that satisfies the signal conditions, determined by the network device based on beam configuration information; or, the third beam is a beam that satisfies the signal conditions, determined based on second measurement data, whereby the terminal device measures multiple beams configured for it by the network device.

20. The method according to any one of claims 17 to 19, characterized in that, The signal quality includes at least one of the following: RSRP, SINR, RSSI, and RSRQ.

21. The method according to any one of claims 12 to 20, characterized in that, The first information includes at least one of the following: First data used to test the first model; The first signal is used to test the first model.

22. A model testing method, characterized in that, The method includes: The terminal device uses a first model to make a prediction based on first information to obtain the prediction data; wherein, the first information is information sent from the network device to the terminal device; The terminal device sends second information to the network device. The second information includes the prediction data. The prediction data is used to determine whether the first model meets the first condition, where the first condition is the performance requirement of the prediction data. The network device determines whether the predicted data output by the first model meets the first condition; wherein, the predicted data is the data predicted by the first model based on the first measurement data, the first measurement data is the data obtained by the terminal device after measuring the first information, and the first information is the information sent by the network device to the terminal device.

23. A network device, characterized in that, include: The processing module is configured to determine whether the predicted data output by the first model meets a first condition; wherein the first condition is the performance requirement of the predicted data, the predicted data is the data predicted by the first model based on the first measurement data, the first measurement data is the data obtained by the terminal device after measuring the first information, and the first information is the information sent by the network device to the terminal device.

24. A terminal device, characterized in that, include: The processing module is configured to use a first model to make a prediction based on first information to obtain the prediction data; wherein, the first information is information sent from the network device to the terminal device; The transceiver module is configured to send second information, the second information including the prediction data, the prediction data being used to determine whether the first model meets a first condition, the first condition being the performance requirements of the prediction data.

25. A network device, characterized in that, include: One or more processors; The terminal device is used to execute the model testing method according to any one of claims 1 to 11.

26. A terminal device, characterized in that, include: One or more processors; The network device is used to execute the model testing method according to any one of claims 12 to 21.

27. A communication system, characterized in that, The communication system includes a terminal device and a network device, wherein the network device is configured to implement the model testing method according to any one of claims 1 to 11, and the terminal device is configured to implement the model testing method according to any one of claims 12 to 21.

28. A storage medium storing instructions, characterized in that, When the instruction is executed on the communication device, the communication device performs the model testing method as described in any one of claims 1 to 11 or claims 12 to 21.

Images