consistency test

Abstract

The present application relates to conformance testing. According to example aspects of the application, there is provided a user equipment configured to, as part of a conformance testing procedure, indicate to a test equipment that the user equipment is configured with a reinforcement learning model for use in communications with a cellular communications network; retrain the reinforcement learning model using exploration data received in the user equipment from the test equipment; and provide to the test equipment a result obtained using the reinforcement learning model trained with the exploration data.

Description

Technical Field

[0001] This disclosure relates to performance testing of user equipment, such as cellular user equipment. Background Technology

[0002] Conformity testing is conducted to ensure that products with radio transceivers operate according to appropriate communication technologies and comply with the limitations specified in the specifications defining those technologies. For example, receiver sensitivity may need to meet minimum requirements under different channel conditions, and products may need to comply with maximum transmit RF power limits to ensure that operating the product does not pose a danger to human health.

[0003] Following fifth-generation cellular technology, sixth-generation cellular technology is expected to present network optimization challenges. Reinforcement learning (RL) is a method for automating the adaptation of network devices to dynamic and complex environments, such as radio access networks (RANs). Summary of the Invention

[0004] The subject matter of the independent claims is provided according to several aspects. Several embodiments are defined in the dependent claims. The scope of protection sought with respect to the various embodiments of the invention is set forth in the independent claims. Embodiments, examples, and features (if any) described in this specification that do not fall within the scope of the independent claims are to be interpreted as examples useful for understanding the various embodiments of the invention.

[0005] According to a first aspect of this disclosure, a user equipment is provided, including at least one processing core and at least one memory storing instructions, the instructions, when executed by the at least one processing core, causing the user equipment to at least: as part of a conformance testing process, instruct a test device that the user equipment is configured to use a reinforcement learning model in communication with a cellular communication network; retrain the reinforcement learning model using exploration data received from the test device in the user equipment; and provide the test device with the results obtained using the reinforcement learning model trained with the exploration data.

[0006] According to a second aspect of this disclosure, a test apparatus is provided, comprising at least one processing core and at least one memory storing instructions, the instructions, when executed by the at least one processing core, causing the test apparatus to at least: as part of a conformance testing process, obtain from a user equipment instructions for a reinforcement learning model configured for use in communication with a cellular communication network; retrain the reinforcement learning model in the test apparatus using exploration data sent from the test apparatus to the user equipment; and compare the results obtained using the reinforcement learning model trained with the exploration data in the test apparatus with the results received from the user equipment in the test apparatus.

[0007] According to a third aspect of this disclosure, a method is provided, comprising: as part of a conformance testing process, instructing a test device that the user equipment is configured to use a reinforcement learning model in communication with a cellular communication network; retraining the reinforcement learning model using exploration data received from the test device in the user equipment; and providing the test device with the results obtained using the reinforcement learning model trained with the exploration data.

[0008] According to a fourth aspect of this disclosure, a method is provided, comprising: as part of a conformance testing process, obtaining from a user equipment an indication of a reinforcement learning model configured for use in communication with a cellular communication network; retraining the reinforcement learning model in the test equipment using exploration data sent from the test equipment to the user equipment; and comparing the results obtained using the reinforcement learning model trained with the exploration data in the test equipment with the results received from the user equipment in the test equipment.

[0009] According to a fifth aspect of this disclosure, a user equipment is provided, comprising components for: as part of a conformance testing process, instructing a test equipment that the user equipment is configured to use a reinforcement learning model in communication with a cellular communication network; retraining the reinforcement learning model using exploration data received from the test equipment in the user equipment; and providing the test equipment with the results obtained using the reinforcement learning model trained with the exploration data.

[0010] According to a sixth aspect of this disclosure, a test apparatus is provided, comprising components for: obtaining from a user equipment an instruction of a reinforcement learning model configured for use in communication with a cellular communication network as part of a conformance testing process; retraining the reinforcement learning model in the test apparatus using exploration data sent from the test apparatus to the user equipment; and comparing the result obtained using the reinforcement learning model trained with the exploration data in the test apparatus with the result received from the user equipment in the test apparatus.

[0011] According to a seventh aspect of this disclosure, a non-transitory computer-readable medium is provided having a computer-readable instruction set thereon, which, when executed by at least one processor, causes a user equipment to at least: as part of a conformance testing process, instruct a test device that the user equipment is configured to use a reinforcement learning model in communication with a cellular communication network; retrain the reinforcement learning model using exploration data received from the test device in the user equipment; and provide the test device with the results obtained using the reinforcement learning model trained with the exploration data.

[0012] According to an eighth aspect of this disclosure, a non-transitory computer-readable medium is provided having a computer-readable instruction set thereon, which, when executed by at least one processor, causes a test device to at least: as part of a conformance testing process, obtain from a user equipment an instruction for a reinforcement learning model configured for use in communication with a cellular communication network; retrain the reinforcement learning model in the test device using exploration data sent from the test device to the user equipment; and compare the result obtained using the reinforcement learning model trained with the exploration data in the test device with the result received from the user equipment in the test device. Attached Figure Description

[0013] Figure 1A An example system according to at least some embodiments of the present invention is shown; Figure 1B An example system according to at least some embodiments of the present invention is shown; Figure 2A Signaling according to at least some embodiments of the present invention is shown; Figure 2B Signaling according to at least some embodiments of the present invention is shown; Figure 2C Signaling according to at least some embodiments of the present invention is shown; Figure 3 Example apparatuses capable of supporting at least some embodiments of the present invention are shown; and Figures 4-7 This is a flowchart of a method according to at least some embodiments of the present invention. Detailed Implementation

[0014] This document discloses a method for enhancing conformance testing of a User Equipment (UE), which verifies the correct operation of a reinforcement learning (RL) model within the UE. Specifically, in conformance testing, the UE can grant access to its RL model to the test device either by providing the model or by providing an identifier of the RL model, which the test device can then use to retrieve the model. Alternatively, if the RL model is to be kept confidential within the UE, the test device is not granted access to the RL model; however, the test device can use a knowledge distillation process to construct a model of the UE's RL model, as will be described in more detail herein.

[0015] Figure 1AAn example system according to at least some embodiments of the present invention is illustrated. A conducted mode conformance test is used to test the UE, wherein a system simulator 120 provides signals {x1, x2, x3, x4} to the UE 110 via a channel emulator 130. For example, in the conducted mode test, each UE antenna port is connected to the test equipment via a cable (such as a 50-ohm coaxial cable), as shown. For the 3GPP 3rd Generation Partnership Project, conducted mode testing is only used for Frequency Range 1 (FR1) test cases and... Figure 1A The example illustration of such a test system shows that channel simulator 130 is used to model a specific implementation of a wireless fading channel in a controlled and repeatable manner for conformance testing purposes. Therefore, for conformance testing purposes, crosstalk generated by the wireless channel of the UE 110 deployed in the field is modeled in channel simulator 120 as follows: .

[0016] For a conduction consistency test system, the signal received at the device under test in the downlink (DL) direction can be modeled as follows: ,in x ={x1, x2, x3, x4} and H L It is an analog channel matrix, which generates y As a signal received at UE 110, the uplink UL signal for testing can be obtained in a similar manner.

[0017] Figure 1B An example system according to at least some embodiments of the present invention is shown. Figure 1B In this study, over-the-air (OTA) conformance testing is used to test the UE. The same reference numerals indicate the same... Figure 1A The same structure is used in OTA mode testing. In OTA mode testing, the UE can be placed inside an anechoic chamber, and the test equipment can be connected to one or more detectors 142, 144 installed at different locations inside the anechoic chamber, allowing the detectors to be fixed or movable within the chamber. OTA mode testing is used in the 3GPP framework for FR1 and Frequency Range 2 (FR2) test cases and... Figure 1B The functions of the channel emulator 130 and system simulator 120 are similar to those discussed above. Figure 1A As explained above. Detectors 142 and 144 (including transceivers) are installed inside an anechoic chamber to wirelessly transmit radio signals to and / or receive radio signals from the UE. Specifically, detectors 142 and 144 form beams 143 and 145, respectively, for communication with the corresponding antenna panels of the UE 110.

[0018] For the OTA testing system, the signal received at UE 110 in DL can be modeled as ,in It is the signal vector to be sent. It is an analog channel matrix. Includes an anechoic chamber and a UE, including a spatial filter, and This is the signal vector received at UE 110. The conformance test described herein can be used... Figure 1A Settings or Figure 1B The system simulator 120 and channel emulator 130 are included in the test apparatus, which is configured to perform conformance tests in the UE 110. The test apparatus generally includes a processor and suitable memory, as well as computer instructions for performing conformance tests with the UE 110.

[0019] Following fifth-generation cellular technology, sixth-generation cellular technology is expected to present network optimization challenges. Reinforcement learning (RL) is a method for automating the adaptation of network devices to dynamic and complex environments, such as radio access networks (RAN). UEs can use RL models to schedule measurements, prioritize beams and cells for measurements, select cells or beams for random access procedures, and perform other RAN tasks. For example, an RL model can be configured to predict which beam the UE will next switch to when it is attached to a cellular network.

[0020] When in use, UEs roam the coverage area of ​​the cellular network, thus monitoring when they approach the edge of the coverage area of ​​the cell to which they are attached. This monitoring may include measurements of signal strength from the serving cell and neighboring cells, allowing for advance anticipation and preparation for handover, resulting in a seamless service experience for the user equipment and its users. Base stations, including those in the RAN, control single or multiple cells and may consist of a centralized unit (CU) and one or more distributed units (DUs) with radio capabilities. UEs typically support uplink and downlink communication, for example, through the use of Time Division Multiple Access (TDMA), Code Division Multiple Access (CDMA), or Orthogonal Frequency Division Multiple Access (OFDM). The core network (CN) performs functions affecting the entire cellular network, such as policy enforcement, handover, and subscriber credential management.

[0021] Reinforcement learning (RL) is a type of machine learning where an agent learns to make decisions by interacting with its environment. The agent receives feedback in the form of rewards or penalties based on its actions, which helps it learn the optimal strategy to achieve its goals. Unlike supervised learning, which trains models on pre-labeled data, RL is learned through trial and error, allowing RL models to adapt and improve their performance over time. This approach is widely used in a variety of applications, including robotics, gaming, and autonomous systems, due to its ability to handle complex decision-making tasks.

[0022] The difference between reinforcement learning and supervised learning is that reinforcement learning does not require pre-labeled input-output training data and does not require explicit correction of suboptimal actions. Instead, the goal of RL is to find a balance between exploring the unknown parameter-space and utilizing the current parameter values, with the aim of maximizing cumulative reward. The search for this balance is known as the exploration-exploitation dilemma.

[0023] RL is a goal-based optimization algorithm where a model performs actions based on observed states or context and is thus rewarded or penalized while learning optimal actions. The phases in RL model operation include an initial training phase, using the model trained in the initial training, retraining the model, and then using the retrained model. In the initial training, near real-time interaction with the wireless environment is allowed via exploration / exploitation. The initial online training of the RL model is performed through exploration and exploitation, as well as by identifying the dynamics of the wireless environment. Once the model has converged, for example, based on a pre-configured convergence threshold, it can be used as a trained model. The distinguishing phase for deploying an RL model compared to supervised learning methods is the initial training, which involves frequent exploration to accelerate model convergence. The decision about whether to explore depends on the exploration strategy used by the RL model and its associated hyperparameters. For example, so-called ε-greedy exploration is one of the most common methods used. In typical RL algorithms, the model explores more frequently during the initial portion of its interaction with the environment to quickly learn optimal actions. However, to continuously learn changes in a non-static environment, the agent must maintain the exploration parameter space to some extent, for example, by exploring periodically at fixed time intervals.

[0024] To ensure the reliability and efficiency of RL-based solutions in real-world deployments, rigorous testing can be conducted in realistic simulation environments, such as UE simulation frameworks. This testing can be based on performance benchmarks, spectrum utilization standards, and interoperability constraints that must be met for seamless integration into 6G cellular communication architectures.

[0025] It is necessary to test UEs using RL models to achieve performance targets to ensure that they can and will meet those targets. Testing RL models is challenging, and relying solely on a limited test dataset may not be sufficient to reliably verify that the RL model will perform adequately to achieve the performance targets. Therefore, testing of RL models (such as conformance testing) achieves the technical benefit of reliably achieving the RF performance required for interoperability with other devices in wireless cellular communication networks.

[0026] As discussed in this paper, the behavior of RL models on the UE side under real-world network conditions can be evaluated by testing RL models in a controlled but representative UE test environment. Adherence to 3GPP specifications ensures that RL models not only achieve predefined performance metrics but also comply with regulatory constraints and industry standards, thereby preventing adverse impacts on network stability and fairness. Due to the complex and dynamic nature of radio environments and algorithmic RL architectures, testing the performance of RL agents in a UE simulation framework according to 3GPP requirements presents several challenges. First, RL models require extensive training and real-time feedback to adapt to diverse network conditions, but accurately simulating these conditions in a controlled test setting is computationally demanding. Ensuring compliance with 3GPP specifications further increases the constraints, as RL models need to balance exploration and exploitation while adhering to regulatory and performance thresholds. These factors, combined with the need for a scalable and efficient simulation environment, make rigorous validation of RL models for 6G networks a complex and crucial task.

[0027] In field deployments, the UE obtains its data from the environment, such as radio channel conditions, UE location, and UE speed. Therefore, it is necessary to simulate this environment in the test setup. Furthermore, the performance of the RL model on the UE side will not be easily evaluated at the test equipment because the online nature of the exploration makes it impossible to obtain ground truth, unlike supervised learning scenarios where clear ground truth can be provided. Due to the continuous nature of RL algorithms, i.e., the exploration phase, evaluating their performance will be limited to a specific snapshot / version of the agent under test; therefore, synchronization between the UE and the test equipment is crucial.

[0028] Test data for RL model testing can be generated using an environment simulator, existing network topology, and / or measurements recorded during the Minimized Road Test (MDT) to obtain source data for generating test data.

[0029] This source data can be fed into the test data generator according to the requirements of the conformance test. For example, if the conformance test aims to test UEs in a city densely populated with many UEs within a cell, the channel model of the test data generator can be configured as the city microenvironment Umi. Then, by feeding the source data as input to the channel model, corresponding test data targeting UE behavior is generated, resulting in the test data as the output of the channel model. The test data can then be fed to the UE in one or more streams during the conformance test to aid in training the RL model.

[0030] The generated test data can then be transmitted to the UE in a pre-configured stream to simulate real-world or field scenarios. For example, the UE's location and the channel conditions that should dominate at that location can be streamed to the UE and then received at the UE's function. Especially in the case of RL models, during the exploration phase, it can be ensured that the model is adequately trained in real time using the data, so that the inference in the utilization phase matches the model running in the test equipment. Real-time data in this paper refers to the test data fed to the UE during conformance testing, and the UE's response to the test data during conformance testing. These responses of the UE may affect further real-time test data fed to the UE during conformance testing, causing the UE to experience an input data stream similar to that when attached to a real cellular network.

[0031] In the first embodiment, the test device is given access to the UE's RL model, allowing it to run a copy of the UE's RL model during conformance testing to verify that the UE is operating correctly using the RL model. In the second embodiment, the test device is not given access to the UE's RL model; instead, it performs knowledge distillation on the UE's RL model to construct a model of the UE's RL model. The test device then uses this constructed model (i.e., the student model) to verify that the UE is operating correctly using the RL model.

[0032] Figure 2A Signaling according to at least some embodiments of the present invention is illustrated. The figure shows signaling of a first embodiment, wherein the UE shares its RL model with a test equipment, and the test equipment uses this shared model to track and verify the performance of the UE. From left to right on the vertical axis are the UE, the test equipment TE, and the model repository (i.e., the storage), labeled "Repo" in the figure. The model repository is optional and is not present in some embodiments, as will be described herein.

[0033] Initially, prior to phase 210, it can be determined that the UE is capable of testing its RL model, as described in this paper. If this is the case, then execution can proceed. Figure 2A The process. In some embodiments, this initial phase is absent. Initially, RL model testing is initiated, for example, by signaling from the TE to the UE to set the UE to test mode. In response, the UE provides the TE with an indication of its model, phase 210. Thus, as part of the conformance testing process, the UE indicates to the TE the reinforcement learning model that the UE is configured to use when operating attached to a cellular communication network. This indication may take the form of an identifier for the model, or it may include the model itself.

[0034] exist Figure 2AIn the example, the indication in phase 210 includes an identifier for the RL model, which the TE uses in phase 220 to retrieve the RL model from the model store. For example, the identifier could be a container / object address, pointer, or hash. When the TE provides the identifier to the model store, the model store provides the model to the TE. For example, phase 230 can be performed via a secure connection. For example, the model store could be a cloud storage service. If the indication in phase 210 includes the UE's own RL model, there may be no model store or interaction with the model store.

[0035] In phase 230, the TE initiates testing of the RL model. The testing phase begins with initial training phase 201, which includes providing training and test inference data to the UE in phase 240. This data can be generated as described above, for example using simulation, modeling, and / or recorded MDT data. This data enables the RL model in the UE to generate inference, phase 250, and the TE also runs a copy of its RL model using the same data to generate inference, phase 260, thus developing the models in both the UE and the TE in a manner similar to the response to the same input data in phase 240.

[0036] In phase 270, the UE provides information about the state of its RL model after the inference in phase 250. The TE can compare this information with the state of a copy of its RL model in phase 280. If it is observed that the RL model has been developed differently in phases 250 and 260, the TE may fail the test for the UE and indicate to the operator that the test has failed. The information provided from the UE in phase 270 may include a copy of the model or a fingerprint (e.g., a hash) of the model, making it possible to determine whether the model in the UE and its copy in the TE remain the same. Alternatively, the TE can be configured to allow slight deviations between the model and its copy without failing the test for the UE. For example, the information in phase 270 may include a set of N parameters of the model. The TE can calculate the N-dimensional distance between the parameters in the UE's model and its copy in the TE, and in response to the absolute value of this N-dimensional distance falling below a pre-configured threshold, the TE can avoid failing the test for the UE in phase 280.

[0037] If the UE does not fail in phase 280, the test proceeds to exploration phase 202. This phase begins with the TE signaling the start of the exploration command in phase 290 and providing real-time exploration data to the UE in phase 2100. These two phases, 290 and 2100, can be combined into a single phase and do not need to include two separate messages.

[0038] In phase 2110, the UE develops (i.e., retrains) its RL model using the exploration data, and in concurrent phase 2120, the TE performs the same operation as a copy of its RL model after the initial training phase 201. In phase 2130, the TE reports the impact of the exploration on the UE's RL model and provides the UE with new exploration data, allowing the exploration to continue in a continuous manner. In phase 2140, the TE compares the information about its RL model provided by the UE with the state of a copy of the RL model running in the TE to determine whether the RL model in the UE deviates from the copy of the model when running with the same exploration data. If this occurs, the TE will fail the UE. Similar to phase 280, the TE can be configured to allow deviations in distance in the N-dimensional parameter space up to a pre-configured threshold without failing the UE, and to fail the UE if the deviation in the N-dimensional parameter space exceeds the threshold distance. The threshold distance can be the same in phases 280 and 2140, or alternatively, phases 280 and 2140 can be configured to use different threshold distances.

[0039] In general, Figure 2A The process involves situations where the UE is willing to allow the TE to run a copy of the RL model that the UE possesses. This provides a technical effect for testing, capable of detecting even very slight deviations in the performance of the corresponding model. On the other hand, the RL model can be confidential in nature, in which case the UE can be configured to not allow the TE to run a copy or to store a copy of the RL model in a repository.

[0040] One approach for compressing or recreating machine learning models is knowledge distillation (KD), where a large, complex teacher model at least partially transfers its learned knowledge to a smaller, more efficient student model. Thus, the student model is a model of the teacher model. By leveraging the KD algorithm in deep learning scenarios, computationally efficient models can be generated, making them more suitable for deployment on resource-constrained devices. This is also a method that can be used to synthesize models that can reasonably and accurately simulate larger models.

[0041] Now, a high-level view of the process is described, depicting data-free knowledge distillation, where knowledge is learned by a smaller student network without access to the original training data, thus avoiding privacy and proprietary issues. First, the teacher model is used with the exploration data to generate inference in the form of soft-labels, which are the training targets used by the student models during supervised learning. The distillation loss can then be estimated by comparing the outputs of the teacher and student models; if the distillation loss (i.e., the difference between the student model's output and the teacher model's output when the models are run on the same data) is acceptablely low, the student model is considered ready. A strategy commonly used in KD, the soft-target approach, aims to train the student model on soft labels / targets, such as the probability distribution of the teacher model's predictions, rather than binary labels / truth values. This allows the student model to generalize better with fewer parameters, resulting in computationally efficient models. Training the student model can also be achieved via a query-based approach, which involves selective knowledge transfer by querying only important or informative knowledge from the teacher model, rather than transferring all of the teacher model's knowledge. This can lead to a more efficient training process by selectively transferring knowledge for critical states and information when needed. Establishing a test framework specifically for handling the exploration and exploitation phases of RL is one of the main pillars of 6G enablers, and therefore, focusing on this topic is of considerable interest.

[0042] Figure 2B Signaling according to at least some embodiments of the present invention is illustrated. This figure shows signaling of a second embodiment, where the UE does not allow the TE to access the UE's RL model. Instead, based on observed measurement reports and hyperparameters from the UE, a test device using the KD method attempts to simulate the behavior of the UE's RL model to test the performance of the underlying RL model. The vertical axis corresponds to the absence of a model store. Figure 2A The vertical axis in.

[0043] Initially, the TE requests information about the UE's RL model, phase 2150. The UE responds by providing meta-information about the RL model in phase 2160, which is referred to in the knowledge distillation field as the set of hyperparameters of the RL model. These hyperparameters describe the architecture of the RL model without disclosing its detailed components. Hyperparameters may describe, for example, one or more of the UE's RL model's algorithm and architecture, action and observation spaces, reward and exploration policies, optimizer, and loss function. It is also possible that the UE will not share any information about its RL model and its configuration; the TE can then determine the KD method it will use to train the model at its end based on the TE's configuration. In phase 2170, the TE determines the training method to perform based on the information it has about the UE, or based on information it receives from the UE, such as in response to the request. Specifically, the TE decides whether to perform a soft-objective knowledge distillation process or a query-based knowledge distillation process. Phase 2170 may also include selecting the pre-trained model that best matches the hyperparameters provided by the UE, if any.

[0044] Phase 203 includes a soft-objective-based knowledge distillation KD process. This phase can be executed if the UE is willing to provide soft objectives to the TE to train the UE's RL model (student model). If the UE is unwilling to do so, the query-based KD process in Phase 204 is executed instead of Phase 203.

[0045] In phase 2180, the TE provides the soft target configuration to the UE. If the UE accepts the soft target configuration, it will return an acknowledgment to the TE in phase 2190. If the UE does not accept the soft target configuration in phase 2180, it will provide a negative acknowledgment in phase 2190, and the TE will respond by sending an alternative soft target configuration to the UE, which the UE can then accept. Therefore, the TE may need to make multiple attempts before the UE accepts the soft target configuration by providing an acknowledgment; however, for clarity, in Figure 2BA single soft target configuration 2180 is shown. If the TE and UE cannot agree on the soft target configuration, a query-based KD process in phase 204 can be used instead of phase 203. Once the TE has confirmation, it will continue to send test data for inference to the UE to train the UE model, phase 2200, against which the UE runs its RL model to generate a set of soft inference targets 2210 for the UE. The TE will use the soft inference targets to update the parameters of the UE's RL model it has, phase 2220. These soft targets can be floating-point numbers reflecting the degree of determinism obtained in the UE's RL model, which the TE model can use as targets, for example, in its own training using supervised learning. In phase 2230, the TE will calculate the distillation loss of the model it must evaluate to determine how close it is to the actual RL model in the UE. In this KD process, the UE's RL model is the teacher model, and the model that the TE has is the student model. Phases 2200-2230 can be repeated until the student model is considered sufficiently effective in modeling the teacher model. This can be determined, for example, by using an appropriate threshold. The test data used for inference is data received by the UE from the TE, and the UE performs inference based on this data. Soft target configuration can define, for example, the precision expressing soft probabilities, the reporting interval of soft probabilities, and / or other characteristics specified by soft probabilities.

[0046] In the query-based KD procedure 204, which runs when the UE is not configured to participate in the soft-target-based KD procedure 203, the TE initially generates query data in phase 2240. This can be a dataset designed to query the characteristics of the UE RL model. For example, the TE can prepare a data stream consisting of different noise levels to understand the behavior of the UE RL model. The TE can also, or alternatively, compare the responses of similar models in other UEs to customize the generation of data in phase 2240. For example, the query data can be, but is not limited to, different interference patterns, fading profiles, beam configurations, channel parameters and / or other channel or environmental parameters. The query data can be constructed in the TE, and it can be TE-specific. The query data can include hierarchical data, such as different noise levels.

[0047] In phase 2250, the TE provides query data to the UE, which serves as test data for inference used by the UE. The UE runs this data through its RL model to return inferred values ​​to the TE, phase 2260. These inferred values ​​can be integers, for example. The TE uses the inferred values ​​to train a student model of its teacher RL model running in the UE, phase 2270; and in phase 2280, the TE determines how closely its model approximates the teacher model running in the UE. Phases 2250-2280 can be run multiple times, referred to as iterations, before the TE determines that its model is sufficiently effective in modeling the UE's RL model.

[0048] Through either a soft-target-based KD process 203 or a query-based KD process 204, the TE obtains a student model that sufficiently approximates the RL model in the UE that the UE is unwilling to share with the TE. As mentioned above, the sufficiency can be determined based on an appropriate threshold. Once the student model is thus sufficiently trained, the TE can indicate this to the UE, which will then know that it does not expect further messages from the TE in stage 203 or 204.

[0049] Figure 2C Signaling according to at least some embodiments of the present invention is illustrated. This figure illustrates signaling according to a second embodiment. The stages of this figure begin at the end of stage 203 or 204, and either of these stages is operational. Figure 2C The process is equivalent to using a student model running in TE to validate the UE RL model, which is trained using KD, as referenced above. Figure 2B As stated above.

[0050] Initially, testing begins in phase 2290. First, initial training phase 205 is executed, where training and testing data are sent to the UE in phase 2300. This data could be similar to that of phase 240. Inference is generated using data communicated in the UE in phase 2300 (phase 2310), and also using data communicated in the student model in the TE (phase 2320). The UE can report the results of this inference in phase 2330, allowing the TE to compare the inference made in the UE with the inference obtained using the student model in phase 2340. If the inferences are sufficiently close, for example, compared using a suitable numerical metric against a threshold amount of difference, the process proceeds to exploration phase 206. If the difference is too large for processing to proceed to phase 206, another iteration of phases 2300-2340 is executed.

[0051] In exploration phase 206, the TE can initially notify the UE that the conformance test will proceed to exploration phase 206, which is phase 2350. In phase 2360, exploration data is provided from the TE to the UE, which is similar to the exploration data in phase 2100, for example. In phases 2370 and 2380, the UE and TE retrain their respective models using the exploration data from phase 2360. In phase 2390, the UE reports the status and / or output of its RL model, allowing the TE to again compare its student model with the teacher model running in the UE. If the UE deviates from the model in the TE by more than the threshold of the KD exploration phase, the TE may fail the UE and indicate this to the user, and / or disable the software in the UE to prevent it from being delivered to the user, because the conformance test failed, and the UE may, for example, radiate more energy than is safe for a human user.

[0052] Figure 3 Exemplary apparatus capable of supporting at least some embodiments of the present invention is shown. Device 300 is shown, which may include, for example, a UE 110, or, where appropriate, a test device configured to perform conformance testing on the UE. Device 300 is an apparatus. Device 300 includes a processor 310, which may include, for example, a single-core or multi-core processor, wherein a single-core processor includes one processing core, and a multi-core processor includes more than one processing core. Processor 310 typically includes a control device. Processor 310 may include more than one processor. When processor 310 includes more than one processor, device 300 may be a distributed device, wherein processing of tasks occurs in more than one physical unit. Processor 310 may be a control device. Processing cores may include, for example, a Cortex-A8 processing core manufactured by ARM Holdings or a Zen processing core designed by Advanced Micro Devices. A processing core or processor may be or may include at least one qubit. Processor 310 may include at least one Qualcomm Snapdragon and / or Intel Atom processor. Processor 310 may include at least one application-specific integrated circuit (ASIC). Processor 310 may include at least one field-programmable gate array (FPGA). Processor 310 may optionally be a component, together with memory and computer instructions, for performing method steps (e.g., instructing, training, providing, failing, and executing) in device 300. Processor 310 may be configured, at least in part, by computer instructions to perform actions.

[0053] A processor may include, or be configured as, one or more circuits configured to perform stages of the methods according to embodiments described herein. As used herein, the term “circuit” may refer to one or more or all of the following: (a) a hardware circuit implementation only, such as an implementation only in analog and / or digital circuitry; and (b) a combination of hardware circuitry and software, such as (if applicable): (i) a combination of analog and / or digital hardware circuitry with software / firmware; and (ii) any part of a hardware processor having software (including digital signal processors, software, and memory that work together to enable a device (such as a UE or test equipment) to perform various functions); and (c) hardware circuitry and / or a processor that requires software (e.g., firmware) to operate, such as a microprocessor or part of a microprocessor, but where the software may be absent when operation does not require it.

[0054] This definition of "circuit" applies to all uses of the term in this application (including in any claim). As another example, as used in this application, the term "circuit" also covers only hardware circuitry or processors (or processors), or a portion of hardware circuitry or processors and their accompanying software and / or firmware implementations. For example, where applicable to a particular claim element, the term "circuit" also covers baseband integrated circuits or processor integrated circuits for mobile devices, or similar integrated circuits in servers, cellular network devices, or other computing or network devices.

[0055] Device 300 may include memory 320. Memory 320 may include random access memory and / or permanent memory. Memory 320 may include at least one RAM chip. Memory 320 may be a computer-readable medium. For example, memory 320 may include solid-state, magnetic, optical, and / or holographic memory. Memory 320 may be at least partially accessible by processor 310. Memory 320 may be at least partially included in processor 310. Memory 320 may be a component for storing information. Memory 320 may include computer instructions configured to be executed by processor 310. When computer instructions configured to cause processor 310 to perform certain actions are stored in memory 320, and device 300 as a whole is configured to operate under the guidance of processor 310 using computer instructions from memory 320, processor 310 and / or at least one of its processing cores may be considered to be configured to perform said certain actions. Memory 320 may be at least partially external to device 300, but accessible by device 300. Memory 320 may be transient or non-transient. As used herein, the term “non-transient” refers to a limitation on the medium itself (i.e., tangible, not signal), rather than a limitation on the persistence of data storage (e.g., RAM versus ROM).

[0056] Device 300 may include a transmitter 330. Device 300 may include a receiver 340. Transmitter 330 and receiver 340 may be configured to transmit and receive information according to at least one cellular or non-cellular standard, respectively. Transmitter 330 may include more than one transmitter. Receiver 340 may include more than one receiver. For example, transmitter 330 and / or receiver 340 may be configured to operate according to Global System for Mobile Communications (GSM), Wideband Code Division Multiple Access (WCDMA), 5G, Long Term Evolution (LTE), IS-95, Wireless Local Area Network (WLAN), Ethernet, and / or Global Microwave Access Interoperability (WiMAX) standards.

[0057] Device 300 may include a near-field communication (NFC) transceiver 350. The NFC transceiver 350 may support at least one NFC technology, such as NFC, Bluetooth, Bluetooth Low Energy, BLE, Wibree, or similar technologies.

[0058] Device 300 may include a user interface (UI) 360. UI 360 may include at least one of the following: a display, a keyboard, a touchscreen, a vibrator arranged to signal to the user by causing device 300 to vibrate, a speaker, or a microphone. The user can operate device 300 via UI 360, for example, to accept incoming telephone calls, initiate telephone or video calls, browse the internet, manage digital files stored in memory 320 or in the cloud accessible via transmitter 330 and receiver 340 or via NFC transceiver 350, and / or play games.

[0059] Device 300 may include or be arranged to accept a user identity module 370. User identity module 370 may include, for example, a subscriber identity module SIM card that can be installed in device 300. User identity module 370 may include subscription information identifying the user of device 300. User identity module 370 may include password information that can be used to verify the identity of the user of device 300 and / or facilitate the encryption of communicated information and billing of the user of device 300 for communications performed via device 300.

[0060] Processor 310 may be equipped with a transmitter arranged to output information from processor 310 to other devices included in device 300 via electrical leads within device 300. Such a transmitter may include a serial bus transmitter arranged to output information to memory 320 for storage, for example, via at least one electrical lead. Alternatively, the transmitter may include a parallel bus transmitter. Similarly, processor 310 may include a receiver arranged to receive information within processor 310 from other devices included in device 300 via electrical leads within device 300. Such a receiver may include a serial bus receiver arranged to receive information from receiver 340, for example, via at least one electrical lead, for processing within processor 310. Alternatively, the receiver may include a parallel bus receiver.

[0061] Device 300 may include Figure 3 Other devices not shown. For example, in the case where device 300 includes a smartphone, it may include at least one digital camera. Some devices 300 may include a rear camera and a front camera, wherein the rear camera is intended for digital photography and the front camera is intended for video calling. Device 300 may include a fingerprint sensor arranged to at least partially authenticate the user of device 300. In some embodiments, device 300 lacks at least one of the above-mentioned devices. For example, some devices 300 may lack an NFC transceiver 350 and / or a user identity module 370.

[0062] Processor 310, memory 320, transmitter 330, receiver 340, NFC transceiver 350, UI 360, and / or user identity module 370 can be interconnected in various ways via electrical leads within device 300. For example, each of the above-described devices can be individually connected to the main bus within device 300 to allow the devices to exchange information. However, as those skilled in the art will understand, this is merely an example, and various ways of interconnecting at least two of the above-described devices can be selected according to embodiments without departing from the scope of the invention.

[0063] Figure 4 This is a flowchart of a method according to at least some embodiments of the present invention. The stages of the method shown can be performed in the UE 110 or in a control device configured to control its functions when installed therein.

[0064] Phase 410 includes: as part of the conformance testing process, the user equipment instructs the test equipment on the reinforcement learning model configured for use in communication with the cellular communication network. Phase 420 includes: retraining the reinforcement learning model using exploration data received from the test equipment in the user equipment. Phase 430 includes: providing the test equipment with the results obtained using the reinforcement learning model trained with the exploration data.

[0065] Figure 5 This is a flowchart of a method according to at least some embodiments of the present invention. The stages of the method shown can be performed in a test apparatus or in a control apparatus configured to control its function when installed therein.

[0066] Phase 510 includes: as part of a conformance testing process, obtaining from the user equipment an indication of the reinforcement learning model configured for use in communication with the cellular communication network. Phase 520 includes: retraining the reinforcement learning model in the test equipment using exploration data sent from the test equipment to the user equipment. Phase 530 includes: comparing the results obtained using the reinforcement learning model trained with exploration data in the test equipment with the results received from the user equipment in the test equipment.

[0067] Figure 6 This is a flowchart of a method according to at least some embodiments of the present invention. The stages of the method shown can be performed in the UE 110 or in a control device configured to control its functions when installed therein.

[0068] Phase 610 includes: as part of a consistency testing process, a knowledge distillation process involving the user equipment and a test equipment regarding a reinforcement learning model, wherein the user equipment is configured to use the reinforcement learning model when communicating with a cellular communication network, and wherein the reinforcement learning model is the teacher model in the knowledge distillation process. Phase 620 includes: after the knowledge distillation process, retraining the reinforcement learning model using exploration data received from the test equipment in the user equipment. Phase 630 includes: providing the test equipment with the results obtained using the reinforcement learning model trained with the exploration data.

[0069] Figure 7 This is a flowchart of a method according to at least some embodiments of the present invention. The stages of the method shown can be performed in a test apparatus or in a control apparatus configured to control its function when installed therein.

[0070] Phase 710 includes: as part of a consistency testing process, a knowledge distillation process involving both the test device and the user device regarding a reinforcement learning model used for communication with the cellular network, wherein the user device's reinforcement learning model serves as the teacher model in the knowledge distillation process. Phase 720 includes: retraining a student model of the reinforcement learning model in the test device using exploration data sent from the test device to the user device. The final phase 730 includes: comparing the results obtained using the student model trained on the test device with the results received from the user device in the test device.

[0071] It should be understood that the embodiments of the invention disclosed herein are not limited to the specific structures, process steps, or materials disclosed herein, but extend to equivalents that will be recognized by those skilled in the art. It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting.

[0072] Throughout this specification, any reference to an embodiment or embodiment means that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of the invention. Therefore, the phrases "in one embodiment" or "in an embodiment" appearing throughout this specification do not necessarily refer to the same embodiment. Precise numerical values ​​are also disclosed where terms such as, for example, approximately or substantially, are used to refer to numerical values.

[0073] As used herein, for convenience, multiple items, structural elements, constituent elements, and / or materials may be presented in a common list. However, these lists should be interpreted as if each member of the list were individually identified as a separate and unique member. Therefore, without indication to the contrary, any individual member of such a list should not be construed as a de facto equivalent of any other member of the same list based solely on their presentation in the common group. Furthermore, various embodiments and examples of the invention may be cited herein together with alternatives to their various components. It should be understood that such embodiments, examples, and alternatives should not be construed as de facto equivalents of each other, but should be considered as separate and autonomous representations of the invention.

[0074] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details, such as examples of length, width, shape, etc., have been provided in the foregoing description to provide a thorough understanding of embodiments of the invention. However, those skilled in the art will recognize that the invention can be practiced without one or more specific details or using other methods, components, materials, etc. In other instances, well-known structures, materials, or operations have not been shown or described in detail to avoid obscuring aspects of the invention.

[0075] While the foregoing examples illustrate the principles of the invention in one or more specific applications, it will be apparent to those skilled in the art that many modifications in form, use, and detail can be made without inventive effort and without departing from the principles and concepts of the invention. Therefore, the invention is not intended to be limited except by the claims set forth below.

[0076] The verbs “comprising” and “including” are used herein as open-ended restrictions, neither excluding nor requiring the presence of any unlisted features. Unless otherwise expressly stated, the features recited in the dependent claims may be freely combined with each other. Furthermore, it should be understood that the use of “a” or “an” (i.e., the singular form) throughout this document does not exclude a plurality.

[0077] As used herein, “at least one of the following: ” and “at least one of ” and similar wording (where the list of two or more elements is connected by “and” or “or”) means at least any one of the elements, or at least any two or more of the elements, or at least all of the elements.

[0078] Industrial applicability At least some embodiments of the present invention can be used in industrial applications in wireless communication.

[0079] List of acronyms RAN Radio Access Network RL reinforcement learning TE testing equipment UE User Equipment List of reference numerals

[0080] Technical terms: Clause 1A. A user equipment comprising at least one processing core and at least one memory storing instructions, the instructions, when executed by the at least one processing core, causing the user equipment to at least: - As part of the conformance testing process, the test equipment is instructed to be configured with a reinforcement learning model for use in communication with the cellular communication network; - Retrain the reinforcement learning model using exploration data received from the test device in the user device; and - Provide the test equipment with the results obtained using the reinforcement learning model trained with the exploration data.

[0081] Clause 2A. The user equipment as described in Clause 1A is configured to execute the instruction by providing the reinforcement learning model to the test equipment or by providing an identifier of the reinforcement learning model to the test equipment.

[0082] Clause 3A. The user equipment as described in Clause 2, wherein the identifier of the reinforcement learning model identifies the reinforcement learning model in the model storage.

[0083] Clause 4A. A user equipment pursuant to any one of Clauses 1A to 3A is configured to receive test data from the test equipment to generate inference, and to provide the test equipment with a first result obtained from the reinforcement learning model using the test data before training the reinforcement learning model with the exploratory data.

[0084] Clause 5A. A user equipment pursuant to any one of Clauses 1A to 4A is configured to: after retraining the reinforcement learning model using the exploration data, receive second exploration data from the test equipment to further train the reinforcement learning model already trained using the second exploration data, and provide the test equipment with the results obtained using the reinforcement learning model trained using the second exploration data.

[0085] Clause 6A. A test apparatus comprising at least one processing core and at least one memory storing instructions, the instructions, when executed by said at least one processing core, causing the test apparatus to at least: - As part of the conformance testing process, obtain indications from the user equipment of the reinforcement learning model configured for use in communication with the cellular communication network; - Retrain the reinforcement learning model on the test device using exploration data sent from the test device to the user device; and - The results obtained using the reinforcement learning model trained with the exploration data in the test device are compared with the results received from the user device in the test device.

[0086] Clause 7A. The test equipment as described in Clause 6A is configured to cause the user equipment to fail in response to the comparison indicating that the deviation between the result obtained using the reinforcement learning model trained with the exploration data in the test equipment and the result received from the user equipment in the test equipment exceeds a threshold deviation.

[0087] Clause 8A. The testing equipment described in Clause 6A or 7A is further configured to: provide test data to the user equipment, obtain a first result from the reinforcement learning model using the test data before retraining the reinforcement learning model using the exploration data, and compare the first result with a result obtained from the user equipment before providing the exploration data.

[0088] Clause 9A. The testing equipment described in Clause 8A is configured to cause the user equipment to fail in response to the comparison indicating that the deviation between the first result and the result obtained from the user equipment before the provision of the exploration data exceeds a second threshold deviation.

[0089] Clause 10A. A test device pursuant to any one of Clauses 6A to 9A, wherein the indication includes the reinforcement learning model, or the indication includes an identifier of the reinforcement learning model.

[0090] Clause 11A. A method comprising: - As part of the conformance testing process, the user equipment indicates to the test equipment the reinforcement learning model that the user equipment is configured to use in communication with the cellular communication network; - Retrain the reinforcement learning model using exploration data received from the test device in the user device; and - Provide the test equipment with the results obtained using the reinforcement learning model trained with the exploration data.

[0091] Clause 12A. The method according to Clause 11A includes: performing the instruction by providing the reinforcement learning model to the test device or by providing an identifier of the reinforcement learning model to the test device.

[0092] Clause 13A. The method according to Clause 12A, wherein the identifier of the reinforcement learning model identifies the reinforcement learning model in the model storage.

[0093] Clause 14A. A method according to any one of Clauses 11A to 13A, comprising: receiving test data from the test device to generate inference, and providing the test device with a first result obtained from the reinforcement learning model using the test data before training the reinforcement learning model using the exploratory data.

[0094] Clause 15A. The method according to any one of Clauses 11A to 14A comprises: after retraining the reinforcement learning model using the exploration data, receiving second exploration data from the test device to further train the reinforcement learning model already trained using the second exploration data, and providing the test device with the results obtained using the reinforcement learning model trained using the second exploration data.

[0095] Clause 16A. A method comprising: - As part of the conformance testing process, the test equipment obtains from the user equipment an indication of the reinforcement learning model configured for use in communication with the cellular communication network; - Retrain the reinforcement learning model on the test device using exploration data sent from the test device to the user device; and - The results obtained using the reinforcement learning model trained with the exploration data in the test device are compared with the results received from the user device in the test device.

[0096] Clause 17A. The method according to Clause 16A includes: in response to the comparison indicating that the deviation between the result obtained using the reinforcement learning model trained with the exploration data in the test device and the result received from the user device in the test device exceeds a threshold deviation, causing the user device to fail.

[0097] Clause 18A. The method according to Clause 16A or 17A further comprises: providing test data to the user equipment, obtaining a first result from the reinforcement learning model using the test data before retraining the reinforcement learning model using the exploration data, and comparing the first result with a result obtained from the user equipment before providing the exploration data.

[0098] Clause 19A. The method according to Clause 18A includes: in response to the comparison indicating that the deviation between the first result and the result obtained from the user equipment before the exploration data is provided exceeds a second threshold deviation, causing the user equipment to fail.

[0099] Clause 20A. The method according to any one of Clauses 16A to 19A, wherein the instruction includes the reinforcement learning model, or the instruction includes an identifier of the reinforcement learning model.

[0100] Clause 21A. A non-transitory computer-readable medium having stored thereon a computer-readable instruction set, which, when executed by at least one processor, causes a user equipment to at least: - As part of the conformance testing process, the test equipment is instructed to be configured with a reinforcement learning model for use in communication with the cellular communication network; - Retrain the reinforcement learning model using exploration data received from the test device in the user device; and - Provide the test equipment with the results obtained using the reinforcement learning model trained with the exploration data.

[0101] Clause 22A. A non-transitory computer-readable medium having stored thereon a computer-readable instruction set, which, when executed by at least one processor, causes a test device to at least: - As part of the conformance testing process, the test equipment obtains from the user equipment an indication of the reinforcement learning model configured for use in communication with the cellular communication network; - Retrain the reinforcement learning model on the test device using exploration data sent from the test device to the user device; and - The results obtained using the reinforcement learning model trained with the exploration data in the test device are compared with the results received from the user device in the test device.

[0102] Clause 1B. A user equipment comprising at least one processing core and at least one memory storing instructions, the instructions causing the user equipment, when executed by said at least one processing core, to at least: - As part of the conformance testing process, participate in a knowledge distillation process for a reinforcement learning model together with the test equipment, wherein the user equipment is configured to use the reinforcement learning model when communicating with a cellular communication network, and wherein the reinforcement learning model is the teacher model in the knowledge distillation process; - Following the knowledge distillation process, the reinforcement learning model is retrained using exploration data received from the test device in the user device; and - Provide the test equipment with the results obtained using the reinforcement learning model trained with the exploration data.

[0103] Clause 2B. The user equipment according to Clause 1B, wherein the user equipment is configured to: provide the test equipment with a soft inference objective for adjusting a student model of the reinforcement learning model, the provision of the soft inference objective being included in the knowledge distillation process, or to be configured to: provide the test equipment with inference values ​​in response to query data received from the test equipment for adjusting the student model of the reinforcement learning model.

[0104] Clause 3B. The user equipment described in Clause 2B is configured to receive the query data from the test equipment after refusing to participate in the knowledge distillation process using the soft target.

[0105] Clause 4B. A user equipment pursuant to any one of Clauses 1B to 3B is configured to: receive test data from the test equipment, and provide the test equipment with a first result obtained from the reinforcement learning model using the test data before training the reinforcement learning model with the exploratory data and after performing the knowledge distillation process.

[0106] Clause 5B. A user equipment pursuant to any one of Clauses 1B to 4B is configured to: after training the reinforcement learning model using the exploration data, receive second exploration data from the test equipment, further train the reinforcement learning model already trained using the second exploration data using the second exploration data, and provide the test equipment with the results obtained using the reinforcement learning model trained using the second exploration data.

[0107] Clause 6B. A test apparatus comprising at least one processing core and at least one memory storing instructions, the instructions, when executed by said at least one processing core, causing the test apparatus to at least: - As part of the consistency testing process, participate in a knowledge distillation process with the user equipment regarding a reinforcement learning model, wherein the reinforcement learning model is used to communicate with a cellular communication network, and wherein the reinforcement learning model of the user equipment is a teacher model in the knowledge distillation process; - Using exploration data sent from the test device to the user device, retrain the student model of the reinforcement learning model in the test device; and - The results obtained using the student model trained with the exploration data in the test device will be compared with the results received from the user device in the test device.

[0108] Clause 7B. The test equipment as described in Clause 6B is configured to cause the user equipment to fail in response to the comparison indicating that the deviation between the result obtained using the reinforcement learning model trained on the exploration data in the test equipment and the result received from the user equipment in the test equipment exceeds a threshold deviation.

[0109] Clause 8B. The testing equipment described in Clause 6B or 7B is further configured to: provide test data to the user equipment, obtain a first result from the student model using the test data before retraining the student model using the exploration data, and compare the first result with a result obtained from the user equipment before providing the exploration data.

[0110] Clause 9B. The testing equipment described in Clause 8B is configured to cause the user equipment to fail in response to the comparison indicating that the deviation between the first result and the result obtained from the user equipment before the provision of the exploration data exceeds a second threshold deviation.

[0111] Clause 10B. The test apparatus according to any one of Clauses 6B to 9B is configured to: build the student model in the test apparatus during the knowledge distillation process.

[0112] Clause 11B. The test apparatus according to any one of Clauses 6B to 10B is further configured to: receive hyperparameters of the reinforcement learning model from the user equipment during the knowledge distillation process, and employ the hyperparameters in the establishment of the student model.

[0113] Clause 12B. A test device according to any one of Clauses 6B to 11B is configured to: receive from the user equipment a soft inference target for adjusting the student model of the reinforcement learning model, the receipt of the soft inference target being included in the knowledge distillation process, or is configured to: receive inference values ​​from the user equipment in response to query data sent from the test device for adjusting the student model of the reinforcement learning model.

[0114] Clause 13B. The test equipment described in Clause 12B is configured to provide the query data to the user equipment after the user equipment has refused to participate in the knowledge distillation process using the soft target.

[0115] Clause 14B. A method comprising: - As part of the conformance testing process, user equipment and test equipment participate in a knowledge distillation process about a reinforcement learning model, wherein the user equipment is configured to use the reinforcement learning model when communicating with a cellular communication network, and wherein the reinforcement learning model is a teacher model in the knowledge distillation process; - Following the knowledge distillation process, the reinforcement learning model is retrained using exploration data received from the test device in the user device; and - Provide the test equipment with the results obtained using the reinforcement learning model trained with the exploration data.

[0116] Clause 15B. The method according to Clause 14B, wherein the user equipment provides the test equipment with a soft inference objective for adjusting the student model of the reinforcement learning model, the soft inference objective being provided by the user equipment during the knowledge distillation process or by the user equipment providing inference values ​​to the test equipment in response to query data received from the test equipment for adjusting the student model of the reinforcement learning model.

[0117] Clause 16B. The method according to Clause 15B includes: receiving the query data from the test device after refusing to participate in the knowledge distillation process using the soft target.

[0118] Clause 17B. The method according to any one of Clauses 13B to 16B comprises: receiving test data from the test device, and providing the test device with a first result obtained from the reinforcement learning model using the test data before training the reinforcement learning model with the exploratory data and after performing the knowledge distillation process.

[0119] Clause 18B. The method according to any one of Clauses 13B to 17B comprises: after training the reinforcement learning model using the exploration data, receiving second exploration data from the test device, further training the reinforcement learning model already trained using the second exploration data using the second exploration data, and providing the test device with the result obtained using the reinforcement learning model trained using the second exploration data.

[0120] Clause 19B. A method comprising: - As part of the conformance testing process, the test equipment and the user equipment participate in a knowledge distillation process regarding a reinforcement learning model, wherein the reinforcement learning model is used to communicate with a cellular communication network, and wherein the reinforcement learning model of the user equipment is the teacher model in the knowledge distillation process; - Using the exploration data sent from the test device to the user device, retrain the student model of the reinforcement learning model on the test device, and - The results obtained using the student model trained with the exploration data in the test device will be compared with the results received from the user device in the test device.

[0121] Clause 20B. The method according to Clause 19B includes: in response to the comparison indicating that the deviation between the result obtained using the reinforcement learning model trained with the exploration data in the test device and the result received from the user device in the test device exceeds a threshold deviation, causing the user device to fail.

[0122] Clause 21B. The method according to Clause 18B or 20B further comprises: providing test data to the user device to obtain a first result from the student model using the test data before retraining the student model using the exploration data, and comparing the first result with a result obtained from the user device before providing the exploration data.

[0123] Clause 22B. The method according to Clause 21B includes: in response to the comparison indicating that the deviation between the first result and the result obtained from the user equipment before the exploration data is provided exceeds a second threshold deviation, causing the user equipment to fail.

[0124] Clause 23B. The method according to any one of Clauses 19B to 22B, comprising: establishing the student model in the test apparatus during the knowledge distillation process.

[0125] Clause 24B. The method according to any one of Clauses 19B to 23B further comprises: receiving hyperparameters of the reinforcement learning model from the user equipment during the knowledge distillation process, and employing the hyperparameters in the construction of the student model.

[0126] Clause 25B. A method according to any one of Clauses 19B to 24B, comprising: during the knowledge distillation process, receiving from the user equipment a soft inference objective for adjusting the student model of the reinforcement learning model, or in response to query data sent from the test equipment, receiving inference values ​​from the user equipment for adjusting the student model of the reinforcement learning model.

[0127] Clause 26B. The method according to Clause 25B includes: providing the query data to the user equipment after the user equipment has refused to participate in the knowledge distillation process using the soft target.

[0128] Clause 27B. A non-transitory computer-readable medium having stored thereon a computer-readable instruction set, which, when executed by at least one processor, causes a user equipment to at least: - As part of the conformance testing process, participate in a knowledge distillation process for a reinforcement learning model together with the test equipment, wherein the user equipment is configured to use the reinforcement learning model when communicating with a cellular communication network, and wherein the reinforcement learning model is the teacher model in the knowledge distillation process; - Following the knowledge distillation process, the reinforcement learning model is retrained using exploration data received from the test device in the user device; and - Provide the test equipment with the results obtained using the reinforcement learning model trained with the exploration data.

[0129] Clause 28B. A non-transitory computer-readable medium having stored thereon a computer-readable instruction set, which, when executed by at least one processor, causes a test device to at least: - As part of the consistency testing process, participate in a knowledge distillation process with the user equipment regarding a reinforcement learning model, wherein the reinforcement learning model is used to communicate with a cellular communication network, and wherein the reinforcement learning model of the user equipment is a teacher model in the knowledge distillation process; - Using the exploration data sent from the test device to the user device, retrain the student model of the reinforcement learning model on the test device, and - The results obtained using the student model trained with the exploration data in the test device will be compared with the results received from the user device in the test device.

Claims

1. A user equipment comprising at least one processing core and at least one memory storing instructions, wherein the instructions, when executed by the at least one processing core, cause the user equipment to at least: - As part of the conformance testing process, the test equipment is instructed to be configured with a reinforcement learning model for use in communication with the cellular communication network; - The reinforcement learning model is retrained using exploration data received from the test device in the user device; as well as - Provide the test equipment with the results obtained using the reinforcement learning model trained with the exploration data.

2. The user equipment of claim 1, configured to execute the instruction by providing the reinforcement learning model to the test equipment or by providing an identifier of the reinforcement learning model to the test equipment.

3. The user equipment according to claim 2, wherein the identifier of the reinforcement learning model identifies the reinforcement learning model in the model storage.

4. The user equipment according to any one of claims 1 to 3, configured to receive test data from the test equipment to generate inference, and to provide the test equipment with a first result obtained from the reinforcement learning model using the test data before training the reinforcement learning model using the exploratory data.

5. The user equipment according to any one of claims 1 to 3, configured to: after retraining the reinforcement learning model using the exploration data, receive second exploration data from the test equipment to further train the reinforcement learning model already trained using the second exploration data, and provide the test equipment with the results obtained using the reinforcement learning model trained using the second exploration data.

6. A test apparatus comprising at least one processing core and at least one memory storing instructions, wherein the instructions, when executed by the at least one processing core, cause the test apparatus to at least: - As part of the conformance testing process, obtain indications from the user equipment of the reinforcement learning model configured for use in communication with the cellular communication network; - Retrain the reinforcement learning model on the test device using exploration data sent from the test device to the user device; as well as - The results obtained using the reinforcement learning model trained with the exploration data in the test device are compared with the results received from the user device in the test device.

7. The testing device of claim 6, configured to cause the user equipment to fail in response to the comparison indicating that the deviation between the result obtained using the reinforcement learning model trained with the exploration data in the testing device and the result received from the user equipment in the testing device exceeds a threshold deviation.

8. The testing apparatus of claim 6, further configured to: provide test data to the user equipment, obtain a first result from the reinforcement learning model using the test data before retraining the reinforcement learning model using the exploration data, and compare the first result with a result obtained from the user equipment before providing the exploration data.

9. The testing apparatus of claim 8, configured to cause the user equipment to fail in response to the comparison indicating that the deviation between the first result and the result obtained from the user equipment before the exploration data is provided exceeds a second threshold deviation.

10. The test apparatus according to any one of claims 6 to 9, wherein the indication includes the reinforcement learning model, or the indication includes an identifier of the reinforcement learning model.

Images