Assessment method, and terminal, storage medium and computer program product
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2026-02-03
- Publication Date
- 2026-08-13
Smart Images
Figure CN2026076858_13082026_PF_FP_ABST
Abstract
Description
Evaluation methods, terminals, storage media, and computer program products
[0001] Cross-references to related applications
[0002] This application is based on and claims priority to Chinese Patent Application No. 202510138917.6, filed on February 7, 2025, the entire contents of which are incorporated herein by reference. Technical Field
[0003] This application relates to the field of terminal technology, and in particular to an evaluation method, terminal, storage medium, and computer program product. Background Technology
[0004] Currently, terminal testing and / or functional testing and / or verification and / or certification and / or supervision and / or evaluation are typically conducted in a laboratory environment only before the terminal is put into market use. However, after the terminal is put into market use, the deployed model and / or functionality may be updated. Therefore, there is still a need for terminal testing and / or verification and / or certification and / or supervision and / or evaluation. Summary of the Invention
[0005] This application provides an evaluation method, terminal, storage medium, and computer program product. The terminal, which has been put into actual use in the market, can perform model and / or function testing and / or verification and / or certification and / or supervision and / or evaluation at appropriate times, meeting the online testing and / or verification and / or certification and / or supervision and / or evaluation needs of models and / or functions, and improving the flexibility of model and / or function testing and / or verification and / or certification and / or supervision and / or evaluation.
[0006] The technical solution of this application embodiment is implemented as follows:
[0007] This application provides an evaluation method applied to a terminal, the terminal being deployed with a first model and / or a first function, the method comprising:
[0008] In the first state, test and / or verify and / or authenticate and / or supervise and / or evaluate whether the first model and / or the first function is available;
[0009] The first state is to activate the terminal's special functions and / or testing functions and / or verification functions and / or authentication functions and / or supervision functions and / or evaluation functions and / or testing mode and / or verification mode and / or authentication mode and / or supervision mode and / or evaluation mode.
[0010] In the above method, when in the first state, the terminal suspends the execution of communication services, or executes part of the communication services.
[0011] In the above method, when in the first state, the portion of communication services that the terminal is allowed to execute includes communication services for the purpose of testing and / or verification and / or certification and / or supervision and / or evaluation, and / or at least a portion of existing network communication services.
[0012] The above method also includes:
[0013] When the business is not busy, it is allowed to enter the first state.
[0014] The above method also includes:
[0015] If the duration of being in an idle state and / or inactive state is not less than a first duration, it is determined that the business is in the non-busy state.
[0016] The above method also includes:
[0017] Entering the first state is permitted when the object is stationary and / or in low-speed motion.
[0018] The above method also includes:
[0019] If the moving speed does not exceed the speed threshold within the second time period, it is determined that the state is stationary and / or in a low-speed motion state.
[0020] In the above method, the testing and / or verification and / or certification and / or supervision and / or evaluation of whether the first model and / or the first function is available includes:
[0021] In a first environment, the first performance and / or first indicator and / or first requirement of the terminal under the first model and / or first function are tested and / or verified and / or certified and / or supervised and / or evaluated to obtain a first result; wherein, the first environment is a laboratory environment and / or a non-laboratory environment;
[0022] Based on the first result, determine whether the first model and / or the first function is available.
[0023] In the above method, the first performance and / or the first indicator and / or the first requirement are associated with the second performance and / or the second indicator and / or the second requirement;
[0024] The terminal does not support testing and / or verification and / or certification and / or supervision and / or evaluation of the second performance and / or the second metric and / or the second requirement, which are used to determine whether the first model and / or the first function is available.
[0025] In the above method, the terminal is deployed with a second model and / or a second function, the second model and / or the second function having been tested and / or verified and / or certified and / or supervised and / or evaluated in the first environment, and the method further includes:
[0026] In the first environment, test and / or verify and / or authenticate and / or supervise and / or evaluate the first performance and / or the first metric and / or the first requirement of the terminal under the second model and / or the second function to obtain a second result;
[0027] The step of determining whether the first model and / or the first function is available based on the first result includes:
[0028] Based on the relationship between the first result and the second result, determine whether the first model and / or the first function is available.
[0029] The above method also includes:
[0030] In the first environment, the first performance and / or the first metric and / or the first requirement of the terminal are tested and / or verified and / or certified and / or supervised and / or evaluated when no model and / or function is activated, to obtain a third result;
[0031] The step of determining whether the first model and / or the first function is available based on the first result includes:
[0032] Based on the relationship between the first result and the third result, determine whether the first model and / or the first function is available.
[0033] In the above method, determining whether the first model and / or the first function is available based on the first result includes:
[0034] Based on the relationship between the first result and the first performance threshold, determine whether the first model and / or the first function is available.
[0035] The above method also includes:
[0036] Send an activation request to the network device; wherein the activation request is used to request the activation of the first state;
[0037] Receive activation information sent by the network device and activate the first state.
[0038] The above method also includes:
[0039] Send an activation response to the network device; wherein the activation response is used to indicate that the first state has been activated.
[0040] The above method also includes:
[0041] Receive an inquiry message sent by a network device; wherein the inquiry message is used to inquire about the capabilities of the terminal;
[0042] Send information about the functions and / or models supported by the terminal to the network device.
[0043] The above method also includes:
[0044] Receive the inference configuration of the first model and / or the first function sent by the network device.
[0045] The above method also includes:
[0046] Send capability information of the first model and / or the first function to the network device.
[0047] The above method also includes:
[0048] Receive the test and / or verification and / or certification and / or supervision and / or evaluation configuration of the first model and / or the first function sent by the network device.
[0049] The above method also includes:
[0050] Receive deactivation information sent by the network device and deactivate the first state.
[0051] The above method also includes:
[0052] Send a deactivation response to the network device; wherein the deactivation response is used to indicate that the first state has been deactivated.
[0053] The above method also includes:
[0054] Send an activation message to the network device; wherein the activation message is used to indicate that the first state has been activated, deactivated, or is about to be activated.
[0055] In the above method, the activation message carries at least the time information for activating and / or deactivating the first state.
[0056] This application provides a terminal, which is configured with a first model and / or a first function, and the terminal includes:
[0057] An evaluation module is used to test and / or verify and / or authenticate and / or supervise and / or evaluate whether the first model and / or the first function is available in the first state.
[0058] The first state is to activate the terminal's special functions and / or testing functions and / or verification functions and / or authentication functions and / or supervision functions and / or evaluation functions and / or testing mode and / or verification mode and / or authentication mode and / or supervision mode and / or evaluation mode.
[0059] This application provides a terminal, including: a processor, a memory, and a communication bus;
[0060] The communication bus is used to realize the communication connection between the processor and the memory;
[0061] The processor is used to execute one or more computer programs stored in the memory to implement an evaluation method applied to the terminal.
[0062] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements an evaluation method applied to a terminal.
[0063] This application provides a computer program product, including a computer program that, when executed by a processor, implements an evaluation method applied to a terminal.
[0064] This application provides an evaluation method, terminal, storage medium, and computer program product. The terminal is deployed with a first model and / or a first function. The evaluation method includes: in a first state, testing and / or verifying and / or certifying and / or supervising and / or evaluating whether the first model and / or the first function is available; wherein, the first state is activating special functions and / or testing functions and / or verification functions and / or certification functions and / or supervision functions and / or evaluation functions and / or testing modes and / or verification modes and / or certification modes and / or supervision modes and / or evaluation modes of the terminal. The technical solution provided by this application, with terminals already in actual market use, can perform model and / or function testing and / or verification and / or certification and / or supervision and / or evaluation at appropriate times, meeting the online testing and / or verification and / or certification and / or supervision and / or evaluation needs of models and / or functions, and improving the flexibility of model and / or function testing and / or verification and / or certification and / or supervision and / or evaluation. Attached Figure Description
[0065] Figure 1 is a flowchart illustrating an evaluation method provided in an embodiment of this application;
[0066] Figure 2 is a schematic diagram of the interaction between a terminal and a network device provided in an embodiment of this application;
[0067] Figure 3 is a schematic diagram of the structure of a terminal provided in an embodiment of this application;
[0068] Figure 4 is a schematic diagram of the structure of a terminal provided in an embodiment of this application. Embodiments of the present invention
[0069] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0070] The technical solutions of this application and how they solve the aforementioned technical problems will be described in detail below through embodiments and in conjunction with the accompanying drawings. The embodiments below can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.
[0071] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.
[0072] Unless otherwise stated, " / " means "or". For example, A / B can mean A or B. In this article, "and / or" is merely a way of describing the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A alone, A and B together, or B alone.
[0073] This application provides an evaluation method applicable to post-deployment testing and / or post-deployment validation / verification and / or post-deployment certification and / or post-deployment monitoring and / or post-deployment evaluation, i.e., testing and / or verification and / or certification and / or monitoring and / or evaluation performed after the terminal (User Equipment, UE) is put into actual use.
[0074] In the embodiments of this application, the terminal may deploy different models and / or functions, or it may be described as the terminal deploying and / or configuring and / or applying different models and / or functions. The models and / or functions can be specifically divided into three types:
[0075] Supported model(s) and / or supported functionality (or supported functionalities) refer to the models and / or functions that the UE can indicate by using UE capability information (via RRC / LPP signalling).
[0076] Applicable models(s) and / or supported functionality (or applicable functionalities) refer to the models and / or functions that the UE is ready to apply for inference. Applicable models and / or functions are a subset of supported models and / or functions. For example, due to low battery power, insufficient memory, or some functions not yet being trained, the models and / or functions supported by the UE may not be applicable.
[0077] Activated model(s) and / or activated functionality or activated functionalities refer to models and / or functions that are already enabled for performing inference.
[0078] The technical solutions provided in this application are intended to test and / or verify and / or authenticate and / or supervise and / or evaluate whether a model and / or function is available. They can also be described as testing and / or verifying and / or authenticating and / or supervising and / or evaluating whether a model and / or function is applicable. Alternatively, they can be described as testing and / or verifying and / or authenticating and / or supervising and / or evaluating whether a model and / or function is activated.
[0079] Figure 1 is a flowchart illustrating an evaluation method provided in an embodiment of this application. As shown in Figure 1, the evaluation method in this embodiment mainly includes the following operations:
[0080] S101. In the first state, test and / or verify and / or authenticate and / or supervise and / or evaluate whether the first model and / or the first function is available; wherein, the first state is to activate the terminal special function and / or test function and / or verification function and / or authentication function and / or supervision function and / or evaluation function and / or test mode and / or verification mode and / or authentication mode and / or supervision mode and / or evaluation mode.
[0081] In embodiments of this application, the terminal, in a first state, tests and / or verifies and / or authenticates and / or supervises and / or evaluates whether the first model and / or the first function is available.
[0082] In the embodiments of this application, the first state is to activate the terminal's special functions(s) and / or test functions(s) and / or validation functions(s) and / or test modes(s) and / or validation modes(s) and / or certification functions(s) and / or certification modes(s) and / or monitoring functions(s) and / or monitoring modes(s) and / or evaluation functions(s) and / or evaluation modes(s). This can also be described as entering the terminal's special functions and / or test functions and / or validation functions and / or certification functions and / or monitoring functions and / or evaluation modes and / or test modes and / or validation modes and / or certification modes and / or monitoring modes and / or evaluation modes. Furthermore, in the first state, the terminal suspends communication services or performs partial communication services. The first state, unlike the general communication service state, means that general communication services cannot be performed, or only some communication services can be performed. The first state can also be understood as the terminal that has been launched on the market and is in actual use entering test mode(s) or online test mode(s), which is different from the state that the terminal is in before it is launched for conformance testing and / or verification and / or certification and / or supervision and / or evaluation.
[0083] It should be noted that, in the embodiments of this application, the special function can also be described as a special conformance test function(s). The test function can also be described as a test mode bearer(s) or a test mode procedure(s), and the test mode can also be described as an online test mode(s).
[0084] In the embodiments of this application, when the terminal is in a first state, the permitted communication services include communication services for the purpose of testing and / or verification and / or certification and / or supervision and / or evaluation, and / or at least a portion of existing network communication services.
[0085] It should be noted that, in the embodiments of this application, when the terminal is in the first state, it can still communicate with the test equipment and / or test server. This communication is not ordinary live network communication, but rather communication for the purpose of testing and / or verification and / or authentication and / or supervision and / or evaluation, and occurs only between the terminal and the test equipment and / or verification equipment and / or test server and / or verification server. Furthermore, even when the terminal is in the first state, it may still perform at least some ordinary live network communication services.
[0086] In embodiments of this application, the terminal is allowed to enter a first state when it is in a non-busy service state. Specifically, the terminal is determined to be in a non-busy service state if the duration of its idle state and / or inactive state is not less than a first duration.
[0087] In embodiments of this application, the terminal may also be allowed to enter the first state while in a stationary and / or low-speed movement state. Specifically, the terminal is determined to be in a stationary and / or low-speed movement state if its movement speed does not exceed a speed threshold within a second time period.
[0088] It should be noted that, in the embodiments of this application, the first duration and the second duration may be the same or different. The specific first duration and second duration, as well as the value of the speed threshold, can be set according to actual needs and application scenarios. This application embodiment does not limit these settings.
[0089] It should be noted that, in the embodiments of this application, the terminal may be allowed to enter the first state if one of the above two conditions is met, or it may be allowed to enter the first state if both conditions are met. The embodiments of this application do not limit this.
[0090] It is understood that in the embodiments of this application, the terminal can enter the first state when it is in a non-busy state, and / or, stationary and / or in low-speed movement. This can effectively avoid interference or conflict between certain existing network communication services and other factors and testing and / or verification and / or authentication and / or supervision and / or evaluation. For example, the first state often occurs during periods of low network service, such as at night.
[0091] In the embodiments of this application, when the terminal is in a first state, it can test and / or verify and / or authenticate and / or supervise and / or evaluate whether the first model and / or the first function is available, or it can be described as testing and / or verifying and / or authenticating and / or supervising and / or evaluating whether the first model and / or the first function is applicable, or it can be described as testing and / or verifying and / or authenticating and / or supervising and / or evaluating whether the first model and / or the first function is activated.
[0092] It should be noted that, in the embodiments of this application, the terminal deploys a first model and / or a first function, which can also be described as the terminal deploying and / or configuring and / or applying a first model and / or a first function. The first model is a model in the terminal that needs to be tested and / or verified and / or certified and / or supervised and / or evaluated, for example, an updated model. Specifically, the first model can be an Artificial Intelligence (AI) model and / or a Machine Learning (ML) model. The first function can be a function in the terminal that needs to be tested and / or verified and / or certified and / or supervised and / or evaluated. The specific first model and first function can be selected according to actual needs, and the embodiments of this application do not limit them.
[0093] In embodiments of this application, terminal testing and / or verification and / or certification and / or supervision and / or evaluation of whether a first model and / or a first function is available includes: testing and / or verifying and / or certifying and / or supervising and / or evaluating the terminal's first performance and / or first indicator and / or first requirement under activated first model and / or first function in a first environment to obtain a first result; wherein the first environment is a laboratory environment and / or a non-laboratory environment; and determining whether the first model and / or first function is available based on the first result.
[0094] It should be noted that, in the embodiments of this application, after the terminal is put into use on the market, it may not have the conditions to conduct laboratory testing and / or verification and / or certification and / or supervision and / or evaluation. In this case, post-deployment testing and / or verification and / or certification and / or supervision and / or evaluation can be carried out in a non-laboratory environment. Of course, the terminal may still have the conditions to conduct laboratory testing and / or verification and / or certification and / or supervision and / or evaluation. In this case, the terminal can conduct post-deployment testing and / or verification and / or certification and / or supervision and / or evaluation in a laboratory environment.
[0095] It should be noted that, in the embodiments of this application, the terminal testing and / or verifying and / or certifying and / or supervising and / or evaluating the terminal's first performance(s) and / or first index(s) and / or first requirement(s) when activating the first model and / or the first function can also be described as the terminal testing and / or verifying and / or certifying and / or supervising and / or evaluating the terminal's first performance(s) and / or first index(s) and / or first requirement(s) when entering the first model and / or the first function, or it can also be described as the terminal testing and / or verifying and / or certifying and / or supervising and / or evaluating the terminal's first performance and / or first index(s) and / or first requirement(s) when using the first model and / or the first function.
[0096] It should be noted that, in the embodiments of this application, the terminal's determination of whether the first model and / or the first function is available can also be described as the terminal determining whether the first model and / or the first function is activatable, or it can also be described as the terminal determining whether the first model and / or the first function can be accessed, or it can also be described as the terminal determining whether the first model and / or the first function is applicable.
[0097] It should be noted that, in the embodiments of this application, the first performance and / or the first indicator and / or the first requirement are associated with the second performance and / or the second indicator and / or the second requirement; the terminal does not support testing and / or verification and / or certification and / or supervision and / or evaluation of the second performance and / or the second indicator and / or the second requirement, and the second performance and / or the second indicator and / or the second requirement are used to determine whether the first model and / or the first function is available.
[0098] It should be noted that, in the embodiments of this application, after the terminal is put into market use, the performance evaluation of the first model and / or the first function can select only some key performance indicators for testing and / or verification and / or certification and / or supervision and / or evaluation to determine its usability. Compared to the terminal conformance testing and / or verification and / or certification and / or supervision and / or evaluation before the terminal is launched, after the terminal is launched, only some indicators can be selected for testing and / or verification and / or certification and / or supervision and / or evaluation, instead of full testing and / or verification and / or certification and / or supervision and / or evaluation. Among several interrelated indicators, a small number of indicators can be selected for testing and / or verification and / or certification and / or supervision and / or evaluation. For example, in the receiving performance and / or receiving indicators and / or receiving requirements, only the basic sensitivity can be selected for testing and / or verification and / or certification and / or supervision and / or evaluation; in the transmitting performance and / or transmitting indicators and / or transmitting requirements, only the maximum transmitting power can be selected for testing and / or verification and / or certification and / or supervision and / or evaluation. The aforementioned second performance and / or second indicator and / or second requirement is used to determine whether the first model and / or first function is available, and can be selected from different performance parameters.
[0099] It should be noted that the testing and / or verification and / or certification and / or supervision and / or evaluation conditions of terminals in actual use are limited. For the first model and / or the first function, although it is expected to determine whether the second performance and / or second indicator and / or second requirement is usable through testing and / or verification and / or certification and / or supervision and / or evaluation, the second performance and / or second indicator and / or second requirement of the terminal cannot be tested and / or verified and / or certified and / or supervised and / or evaluated due to the limitations of the testing environment and / or verification environment and / or testing conditions and / or verification conditions. Under these circumstances, this application proposes to achieve performance testing and / or verification and / or certification and / or supervision and / or evaluation through an "equivalent indirect" testing and / or verification and / or certification and / or supervision and / or evaluation method. The "equivalent indirect" testing and / or verification and / or certification and / or supervision and / or evaluation method refers to a method in which, although it is desirable to test and / or verify and / or certify and / or supervise and / or evaluate the second performance and / or second indicator and / or second requirement, the second performance and / or second indicator and / or second requirement of the terminal already on the market and in actual use is limited by the testing environment and / or verification environment and / or certification environment and / or supervision environment and / or evaluation conditions. Therefore, the second performance and / or second indicator and / or second requirement of the terminal can only be indirectly measured by testing and / or verifying and / or certifying and / or supervising and / or evaluating the first performance and / or first indicator and / or first requirement of the terminal to determine whether the first model and / or first function is available.
[0100] It should be noted that, in the embodiments of this application, the association between the first performance and / or the first indicator and / or the first requirement and the second performance and / or the second indicator and / or the second requirement can refer to a direct association between the first performance and / or the first indicator and / or the first requirement and the second performance and / or the second indicator and / or the second requirement. For example, the first performance and / or the first indicator and / or the first requirement is throughput, and the second performance and / or the second indicator and / or the second requirement is sensitivity. Alternatively, it can refer to an indirect association between the first performance and / or the first indicator and / or the first requirement and the second performance and / or the second indicator and / or the second requirement. For example, the first performance and / or the first indicator and / or the first requirement is associated with performance a, performance a is associated with performance b, and performance b is associated with the second performance and / or the second indicator and / or the second requirement. The specific first performance and / or the first indicator and / or the first requirement and the second performance and / or the second indicator and / or the second requirement are not limited in the embodiments of this application.
[0101] In the embodiments of this application, the terminal determines whether the first model and / or the first function is available based on the first test result, and can choose different ways to implement it, which will be explained below.
[0102] In the embodiments of this application, a first approach is to place the second model and / or the second function in the same environment as the first model and / or the first function, and determine the usability of the first model and / or the first function by comparing the differences between the two regarding the first performance and / or the first indicator and / or the first requirement. The terminal may be equipped with the second model and / or the second function, which has been tested and / or verified and / or certified and / or supervised and / or evaluated in the first environment. The specific second model and / or second function can be selected according to actual needs and application scenarios, and this application embodiment does not limit this selection. The same environment, i.e., the aforementioned first environment, includes the same laboratory environment and / or the same non-laboratory environment, and the same testing and / or verification and / or certification and / or supervision and / or evaluation parameter configuration.
[0103] In embodiments of this application, the terminal may also perform the following operations: in a first environment, activate the second model and / or the second function, test and / or verify and / or authenticate and / or supervise and / or evaluate the terminal's first performance and / or first indicator and / or first requirement, and obtain a second result; based on this, the terminal determines whether the first model and / or the first function is available based on the first result, which may include: determining whether the first model and / or the first function is available based on the relationship between the first result and the second result.
[0104] It should be noted that, in the embodiments of this application, the second result can be obtained at any time during the post-deployment testing and / or post-deployment verification and / or post-deployment certification phases of the terminal. For example, the second result can be obtained at a historical time by activating the second model and / or the second function to test and / or verify and / or authenticate and / or supervise and / or evaluate the terminal's first performance and / or first indicator and / or first requirement, i.e., historical data. Alternatively, it can be obtained when determining whether the first model and / or the first function is available, i.e., after entering the first state, by activating the second model and / or the second function to test and / or verify and / or authenticate and / or supervise and / or evaluate the terminal's first performance and / or first indicator and / or first requirement.
[0105] For example, in an embodiment of this application, the first performance and / or the first metric and / or the first requirement is throughput, and the terminal may perform the following operations:
[0106] 1. The terminal performs throughput testing and / or verification and / or certification and / or supervision and / or evaluation under the use or activated first model and / or first function to obtain the first throughput T1 (first result), including:
[0107] 1a. The terminal enters the connected state;
[0108] 1b. The terminal connects to the first server to perform throughput testing and / or verification and / or certification and / or supervision and / or evaluation to obtain the first throughput T1; the first server can provide the terminal with testing and / or verification and / or certification and / or supervision and / or evaluation services.
[0109] Wherein, the first throughput T1 can be the average throughput obtained from N tests and / or verification and / or certification and / or supervision and / or evaluation, where N is a natural number greater than 1, or it can be the throughput obtained from a single test and / or verification and / or certification and / or supervision and / or evaluation.
[0110] 2. The terminal deactivates the first model and / or the first function, activates the second model and / or the second function, and performs throughput testing and / or verification and / or certification and / or supervision and / or evaluation again under the activated second model and / or the second function to obtain the second throughput T2 (second result).
[0111] Wherein, the second throughput T2 can be the average throughput obtained from M tests and / or verifications and / or certifications and / or supervisions and / or evaluations, where M is a natural number greater than 1, or it can be the throughput obtained from a single test and / or verification and / or certification and / or supervision and / or evaluation.
[0112] 3. Based on the relationship between the first throughput T1 and the second throughput T2, the terminal determines whether the first model and / or the first function is available:
[0113] If the difference between T1 and T2 is not greater than the first throughput difference threshold, then the first model and / or the first function is determined to be available, that is, the first model and / or the first function is an applicable model(s) and / or an applicable function(s) or applicable functionalities; after the network side is configured, the first model and / or the first function can be activated and used, that is, it is converted into an activated model(s) and / or an activated function(s) or activated functionalities for inference;
[0114] If the difference between T1 and T2 is greater than the first throughput difference threshold, the first model and / or the first function is determined to be a supported model(s) and / or a supported function(s) or supported functionalities, but cannot be determined to be an applicable model(s) and / or applicable functionalities.
[0115] It should be noted that, in the embodiments of this application, the terminal may first test and / or verify and / or authenticate and / or supervise and / or evaluate the first performance and / or first indicator and / or first requirement under the activation of the first model and / or first function, and then test and / or verify and / or authenticate and / or supervise and / or evaluate the first performance and / or first indicator and / or first requirement under the activation of the second model and / or second function. Alternatively, it may first test and / or verify and / or authenticate and / or supervise and / or evaluate the first performance and / or first indicator and / or first requirement under the activation of the second model and / or second function, and then test and / or verify and / or authenticate and / or supervise and / or evaluate the first performance and / or first indicator and / or first requirement under the activation of the first model and / or first function. The embodiments of this application do not limit this.
[0116] In the embodiments of this application, the second approach is to use the state in which the terminal does not activate any model and / or function as a benchmark in the same environment, i.e., the first environment, and to determine whether the first model and / or first function is available by the difference between the terminal in the benchmark state and the first performance and / or first indicator and / or first requirement when the first model and / or first function is activated.
[0117] In embodiments of this application, the terminal may further perform the following operations: in a first environment, test and / or verify and / or authenticate and / or supervise and / or evaluate the terminal's first performance and / or first indicator and / or first requirement when no model and / or function is activated, and obtain a third result; based on this, the terminal determines whether the first model and / or first function is available based on the first result, which may include: determining whether the first model and / or first function is available based on the relationship between the first result and the third result.
[0118] It should be noted that, in the embodiments of this application, the third result can be obtained at any time during the post-deployment testing and / or post-deployment verification and / or post-deployment certification phases of the terminal. For example, the third result can be obtained at a historical time when no model and / or function is activated, and the terminal's first performance and / or first indicator and / or first requirement is tested and / or verified and / or certified and / or monitored and / or evaluated—that is, historical data. Alternatively, it can be obtained when determining whether the first model and / or first function is available, that is, after entering the first state, when no model and / or function is activated, and the terminal's first performance and / or first indicator and / or first requirement is tested and / or verified and / or certified and / or monitored and / or evaluated.
[0119] For example, in an embodiment of this application, the first performance and / or the first metric and / or the first requirement is throughput, and the terminal may perform the following operations:
[0120] 1. The terminal performs throughput testing and / or verification and / or certification and / or supervision and / or evaluation under the use or activated first model and / or first function to obtain the first throughput T1 (first result):
[0121] 1a. The terminal enters the connected state;
[0122] 1b. The terminal connects to the first server to perform throughput testing and / or verification and / or certification and / or supervision and / or evaluation to obtain the first throughput T1; the first server can provide the terminal with testing and / or verification and / or certification and / or supervision and / or evaluation services.
[0123] Wherein, the first throughput T1 can be the average throughput obtained from N tests and / or verification and / or certification and / or supervision and / or evaluation, where N is a natural number greater than 1, or it can be the throughput obtained from a single test and / or verification and / or certification and / or supervision and / or evaluation.
[0124] 2. The terminal deactivates the first model and / or the first function, and performs throughput testing and / or verification and / or certification and / or supervision and / or evaluation again without activating any model and / or function to obtain the third throughput T3 (third result).
[0125] The third throughput T3 can be the average throughput obtained from M tests and / or verifications and / or certifications and / or supervisions and / or evaluations, where M is a natural number greater than 1, or it can be the throughput obtained from a single test and / or verification and / or certification and / or supervision and / or evaluation.
[0126] 3. Based on the relationship between the first throughput T1 and the third throughput T3, the terminal determines whether the first model and / or the first function is available:
[0127] If the difference between T1 and T3 is not greater than the second throughput difference threshold, then the first model and / or the first function is determined to be available, that is, the first model and / or the first function is an applicable model(s) and / or an applicable function(s) or applicable functionalities; after the network side is configured, the first model and / or the first function can be activated and used, that is, it is converted into an activated model(s) and / or an activated function(s) or activated functionalities for inference execution;
[0128] If the difference between T1 and T3 is greater than the second throughput difference threshold, the first model and / or the first function is determined to be a supported model(s) and / or a supported function(s) or supported functionalities, but cannot be determined to be an applicable model(s) and / or applicable functionalities.
[0129] It should be noted that, in the embodiments of this application, the terminal may first test and / or verify and / or authenticate and / or supervise and / or evaluate the first performance and / or first indicator and / or first requirement under the activation of the first model and / or first function, and then test and / or verify and / or authenticate and / or supervise and / or evaluate the first performance and / or first indicator and / or first requirement under the activation of any model and / or function. Alternatively, it may first test and / or verify and / or authenticate and / or supervise and / or evaluate the first performance and / or first indicator and / or first requirement under the activation of any model and / or function, and then test and / or verify and / or authenticate and / or supervise and / or evaluate the first performance and / or first indicator and / or first requirement under the activation of the first model and / or first function. The embodiments of this application do not limit this.
[0130] It should be noted that, in the embodiments of this application, the method of determining whether the first model and / or the first function is available based on the difference between throughput T1 and T2, or based on the difference between T1 and T3, is only an optional determination method. For example, the following methods can also be used to determine:
[0131] The expected throughput is determined based on T2 or T3, for example, the expected throughput is twice T2 or T3;
[0132] If T1 reaches the expected throughput, the first model and / or the first function is determined to be available, that is, the first model and / or the first function are available models(s) and / or available functions(s) or available functionalities(s). After the network side is configured, the first model and / or the first function can be activated and used, that is, it is converted into activated models(s) and / or activated functions(s) or activated functionalities(s) for inference execution.
[0133] If T1 does not reach the expected throughput, the first model and / or the first function can be determined as a supported model(s) and / or a supported function(s) or supported functionalities, but cannot be determined as an applicable model(s) and / or applicable functionalities.
[0134] In the embodiments of this application, the first and second methods described above are based on the relative relationship (relative value) between the results of two tests and / or verifications and / or certifications and / or supervisions and / or evaluations to determine whether the first model and / or the first function is available. The third method is to use the absolute value of the first result as the basis for judgment.
[0135] In embodiments of this application, the terminal determines whether the first model and / or the first function is available based on the first result, and may further include: determining whether the first model and / or the first function is available based on the relationship between the first result and the first performance threshold.
[0136] It should be noted that, in the embodiments of this application, the first performance threshold is used to measure the quality of the first performance and / or the first indicator and / or the first requirement. It can be a threshold used in historical testing and / or verification and / or certification and / or supervision and / or evaluation, or it can be a threshold set based on current requirements. The specific value of the first performance threshold is not limited in the embodiments of this application.
[0137] For example, in an embodiment of this application, the first performance and / or the first metric and / or the first requirement is throughput, and the terminal may perform the following operations:
[0138] 1. The terminal performs a throughput test under the current activated first model and / or first function, and obtains the first throughput T1 (first result):
[0139] 1a. The terminal enters the connected state;
[0140] 1b. The terminal connects to the first server to perform throughput testing and obtain the first throughput T1, which is the first result; the first server can provide the terminal with testing and / or verification and / or certification and / or supervision and / or evaluation services.
[0141] Wherein, the first throughput T1 can be the average throughput obtained from N tests and / or verification and / or certification and / or supervision and / or evaluation, where N is a natural number greater than 1, or it can be the throughput obtained from a single test and / or verification and / or certification and / or supervision and / or evaluation.
[0142] 2. The terminal compares the relationship between the first throughput T1 and the throughput threshold (first performance threshold) to determine whether the first model and / or the first function is available:
[0143] If T1 is not less than the throughput threshold, that is, the first model and / or the first function is an applicable model(s) and / or an applicable function(s) or applicable functionalities; after the network side is configured, the first model and / or the first function can be activated and used, that is, converted into an activated model(s) and / or activated function(s) for inference;
[0144] If T1 is less than the throughput threshold, the first model and / or the first function are determined to be supported models(s) and / or supported functions(s), but cannot be determined to be applicable models(s) and / or applicable functions(s).
[0145] It should be noted that, in the embodiments of this application, the three methods described above for determining whether the first model and / or the first function is available are merely exemplary and optional methods. Of course, other methods or other rules may also be used to determine whether the first model and / or the first function is available, and the embodiments of this application do not limit this.
[0146] For example, in embodiments of this application, a fourth result can also be obtained from testing and / or verifying and / or certifying and / or supervising and / or evaluating the first performance of the terminal under the activation of the first model and / or the first function at a historical moment. Based on the relationship between the fourth result and the first result, it can be determined whether the first model and / or the first function is available. The fourth result may be obtained from pre-deployment testing and / or verification and / or certification and / or supervision and / or evaluation, or from testing and / or verification and / or certification and / or supervision and / or evaluation in a laboratory environment and / or a non-laboratory environment. In some embodiments, the fourth result may be obtained through terminal laboratory conformance testing and / or verification and / or certification and / or supervision and / or evaluation, and / or through field testing and / or verification and / or certification and / or supervision and / or evaluation.
[0147] It should be noted that, in the embodiments of this application, the following prerequisite must also be met for the scheme to measure the relationship between the first result and the other results mentioned above: the relevant test data for obtaining the two results must be consistent, that is, the inputs (model input(s)) and / or ground truth(s) of the two tests and / or verification and / or certification and / or supervision and / or evaluation must be consistent.
[0148] In the embodiments of this application, the terminal may also interact with the network side when executing the above scheme, as explained below with reference to Figure 2.
[0149] In the embodiments of this application, exemplarily referring to FIG2, the main interactions between the terminal and the network device involve the following:
[0150] S201. The terminal sends an activation request to the network device; wherein, the activation request is used to request activation of the first state, that is, to request activation of the test mode.
[0151] S202. The terminal receives the activation information sent by the network device and activates the first state, that is, the activation test mode.
[0152] S203. The terminal sends an activation response for the first state to the network device, which is also called the activation response for the test mode (ACK for Activated Test Mode), to indicate that the first state has been activated. This activation response can also be described as indicating that the first state has been activated, or as informing that the first state has been activated.
[0153] S204. The terminal receives an inquiry message sent by the network device; wherein, the inquiry message is used to inquire about the terminal's capabilities (UE Capability Enquiry) and initiate the process of the terminal reporting supported functions and / or supported models.
[0154] S205. The terminal sends UE Capability Information to the network device, which includes information on supported functions and / or supported models(s).
[0155] S206. Network devices configure or reconfigure terminals, including at least the following:
[0156] The terminal is allowed to report terminal auxiliary information (UAI) through other configurations.
[0157] Network devices can provide network-side additional conditions (NW-side additional condition(s)).
[0158] The above configuration and / or reconfiguration can be accomplished through Radio Resource Control (RRC);
[0159] S207. The terminal receives the test mode configuration and / or test configuration and / or online test configuration and / or online test mode configuration sent by the network device for the first model and / or the first function.
[0160] It should be noted that operation S207 can be executed after operation S203 and before operation S204;
[0161] S208. The terminal sends capability information of the first model and / or the first function to the network device. The capability information can also be described as the model(s) for test and / or the function for test or functionalities for test.
[0162] S209. If the network device did not configure the inference configuration(s) of the first model and / or the first function to the terminal in the aforementioned operation S206, the network device may configure the inference configuration of the first model and / or the first function to the terminal in this operation; if the network device configured the inference configuration of the first model and / or the first function to the terminal in the aforementioned operation S206, the network device may also provide the terminal with updated inference configuration(s) in this operation.
[0163] The configuration and / or configuration updates of the terminals by the aforementioned network devices can be completed via RRC signaling;
[0164] S210, Activate the first model and / or the first function;
[0165] S211. The terminal sends an activation response of the first model and / or the first function to the network device to indicate that the first model and / or the first function has been activated. The activation response can also be described as indicating that the first model and / or the first function has been activated, or as informing that the first model and / or the first function has been activated.
[0166] S212, Initiate testing of the first model and / or the first function;
[0167] S213, monitoring of first performance and / or first indicators and / or first requirements;
[0168] S214. First model and / or first function deactivation.
[0169] It should be noted that if there are multiple first models and / or first functions to be tested, then repeat operations S210~S214.
[0170] S215. The terminal receives the deactivation information (Deactivate Test Mode) sent by the network device and deactivates the first state.
[0171] S216. The terminal sends a first-state deactivation response to the network device, which is also called a test mode deactivation response (ACK for Deactivated Test Mode), indicating that the first state has been deactivated. This deactivation response can also be described as indicating that the first state has been deactivated, or as informing that the first state has been deactivated.
[0172] It should be noted that, in the embodiments of this application, as shown by the dotted line in Figure 2, the above operations S203-S206, S209-S211, and S214 are optional operations.
[0173] In the embodiments of this application, the terminal can not only operate S201-S203 as shown in FIG2 to activate the first state by requesting the network device, but also activate and / or deactivate the first state autonomously. In this case, the terminal can directly send an activation message to the network device. The activation message is used to indicate that the first state is activated, deactivated, or about to be activated. This activation message can also be described as indicating that the first state is activated, deactivated, or about to be activated, or as informing that the first state is activated, deactivated, or about to be activated. The activation message carries at least the time information for activating and / or deactivating the first state.
[0174] It should be noted that, in the embodiments of this application, when the terminal directly sends activation messages to the network device, the number of activation messages can be multiple, the content represented by each activation message can be different, and each activation message can be sent by the terminal to the network device at different stages. For example, when the terminal is about to activate the first state, it sends a first activation message to the network device, indicating that the first state is about to be activated. After activating the first state, the terminal sends a second activation message to the network device, indicating that the first state has been activated. After completing the first model and / or first function test and deactivating the first state, the terminal can also send a third activation message to the network device, indicating that the first state has been deactivated.
[0175] Based on the above, it can be seen that the evaluation method provided in this application can meet the needs of real-time online testing and / or verification and / or certification and / or supervision and / or evaluation of the models and / or functions configured in terminals that have been put into actual use in the market, and improves the flexibility of model and / or function testing and / or verification and / or certification and / or supervision and / or evaluation.
[0176] This application provides a terminal. Figure 3 is a schematic diagram of the structure of a terminal provided in this application embodiment. As shown in Figure 3, in this embodiment, the terminal is equipped with a first model and / or a first function, and the terminal includes:
[0177] Evaluation module 301 is used to test and / or verify and / or authenticate and / or supervise and / or evaluate whether the first model and / or the first function is available in the first state;
[0178] The first state is to activate the terminal's special functions and / or testing functions and / or verification functions and / or authentication functions and / or supervision functions and / or evaluation functions and / or testing mode and / or verification mode and / or authentication mode and / or supervision mode and / or evaluation mode.
[0179] In some embodiments of this application, when in the first state, the terminal suspends the execution of communication services, or executes part of the communication services.
[0180] In some embodiments of this application, when in the first state, the terminal is allowed to perform certain communication services, including communication services for testing and / or verification and / or certification and / or supervision and / or evaluation, and / or at least some existing network communication services.
[0181] In some embodiments of this application, the evaluation module 301 is further configured to allow entry into the first state when the system is in a non-busy state.
[0182] In some embodiments of this application, the evaluation module 301 is further configured to determine that the non-busy business state is in effect if the duration of being in an idle state and / or inactive state is not less than a first duration.
[0183] In some embodiments of this application, the evaluation module 301 is further configured to allow entry into the first state when in a stationary and / or low-speed motion state.
[0184] In some embodiments of this application, the evaluation module 301 is further configured to determine that the object is at rest and / or in the low-speed motion state if the moving speed does not exceed a speed threshold within a second time period.
[0185] In some embodiments of this application, the evaluation module 301 is further configured to test and / or verify and / or authenticate and / or supervise and / or evaluate the first performance and / or first indicator and / or first requirement of the terminal under the activation of the first model and / or the first function in a first environment, and obtain a first result; wherein the first environment is a laboratory environment and / or a non-laboratory environment; and based on the first result, determine whether the first model and / or the first function is available.
[0186] In some embodiments of this application, the first performance and / or the first metric and / or the first requirement are associated with the second performance and / or the second metric and / or the second requirement; the terminal does not support testing and / or verification and / or certification and / or supervision and / or evaluation of the second performance and / or the second metric and / or the second requirement, which is used to determine whether the first model and / or the first function is available.
[0187] In some embodiments of this application, the terminal is deployed with a second model and / or a second function, the second model and / or the second function having been tested and / or verified and / or certified and / or supervised and / or evaluated in the first environment; the evaluation module 301 is further configured to, in the first environment, test and / or verify and / or certified and / or supervise and / or evaluate the first performance and / or first indicator and / or first requirement of the terminal under the activation of the second model and / or the second function, to obtain a second result; and based on the relationship between the first result and the second result, determine whether the first model and / or the first function is available.
[0188] In some embodiments of this application, the evaluation module 301 is further configured to, in the first environment, test and / or verify and / or authenticate and / or supervise and / or evaluate the first performance and / or first indicator and / or first requirement of the terminal when no model and / or function is activated, and obtain a third result; and determine whether the first model and / or the first function is available based on the relationship between the first result and the third result.
[0189] In some embodiments of this application, the evaluation module 301 is further configured to determine whether the first model and / or the first function is available based on the relationship between the first result and the first performance threshold.
[0190] In some embodiments of this application, the terminal further includes a transceiver module (not shown in the figure) for sending an activation request to a network device; wherein the activation request is used to request activation of the first state; and to receive activation information sent by the network device to activate the first state.
[0191] In some embodiments of this application, the transceiver module is further configured to send an activation response to the network device; wherein the activation response is used to indicate that the first state has been activated.
[0192] In some embodiments of this application, the transceiver module is further configured to receive an inquiry message sent by a network device; wherein the inquiry message is used to inquire about the capabilities of the terminal;
[0193] Send information about the functions and / or models supported by the terminal to the network device.
[0194] In some embodiments of this application, the transceiver module is further configured to receive the inference configuration of the first model and / or the first function sent by the network device.
[0195] In some embodiments of this application, the transceiver module is further configured to send capability information of the first model and / or the first function to the network device.
[0196] In some embodiments of this application, the transceiver module is further configured to receive test and / or verification and / or authentication and / or supervision and / or evaluation configurations of the first model and / or the first function sent by the network device.
[0197] In some embodiments of this application, the transceiver module is further configured to receive deactivation information sent by the network device to deactivate the first state.
[0198] In some embodiments of this application, the transceiver module is further configured to send a deactivation response to the network device; wherein the deactivation response is used to indicate that the first state has been deactivated.
[0199] In some embodiments of this application, the transceiver module is further configured to send an activation message to the network device; wherein the activation message is used to indicate that the first state has been activated, deactivated, or is about to be activated.
[0200] In some embodiments of this application, the activation message carries at least time information for activating and / or deactivating the first state.
[0201] Based on the same inventive concept, Figure 4 is a second structural schematic diagram of a terminal provided in an embodiment of this application. As shown in Figure 4, in the embodiment of this application, the terminal includes: a processor 401, a memory 402, and a communication bus 403;
[0202] The communication bus 403 is used to realize the communication connection between the processor 401 and the memory 402;
[0203] The processor 401 is used to execute one or more computer programs stored in the memory 402 to implement an evaluation method applied to the terminal.
[0204] This application provides a computer program product, including a computer program that, when executed by a processor, implements an evaluation method applied to a terminal.
[0205] This application provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements an evaluation method applied to a terminal. The computer-readable storage medium can be volatile memory, such as random-access memory (RAM); or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid-state drive (SSD); or it can be a device including one or any combination of the above-mentioned memories, such as a mobile phone, computer, tablet device, personal digital assistant, etc.
[0206] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.
[0207] This application is described with reference to schematic and / or block diagrams of implementations of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the schematic and / or block diagrams, and combinations thereof, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the schematic and / or block diagrams.
[0208] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.
[0209] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide operations for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.
[0210] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. An evaluation method applied to a terminal, said terminal having deployed a first model and / or a first function, said method comprising: In the first state, test and / or verify and / or authenticate and / or supervise and / or evaluate whether the first model and / or the first function is available; The first state is to activate the terminal's special functions and / or testing functions and / or verification functions and / or authentication functions and / or supervision functions and / or evaluation functions and / or testing mode and / or verification mode and / or authentication mode and / or supervision mode and / or evaluation mode.
2. The method according to claim 1, wherein, When in the first state, the terminal suspends communication services or performs partial communication services.
3. The method according to claim 2, wherein, In the first state, the terminal is allowed to execute certain communication services, including communication services for testing and / or verification and / or certification and / or supervision and / or evaluation, and / or at least some existing network communication services.
4. The method according to any one of claims 1-3, wherein, The method further includes: When the business is not busy, it is allowed to enter the first state.
5. The method according to claim 4, wherein, The method further includes: If the duration of being in an idle state and / or inactive state is not less than a first duration, it is determined that the business is in the non-busy state.
6. The method according to any one of claims 1-5, wherein, The method further includes: Entering the first state is permitted when the object is stationary and / or in low-speed motion.
7. The method according to claim 6, wherein, The method further includes: If the moving speed does not exceed the speed threshold within the second time period, it is determined that the state is stationary and / or the low-speed movement state.
8. The method according to any one of claims 1-7, wherein, The testing and / or verification and / or certification and / or supervision and / or evaluation of the availability of the first model and / or the first function includes: In a first environment, the first performance and / or first indicator and / or first requirement of the terminal under the first model and / or first function are tested and / or verified and / or certified and / or supervised and / or evaluated to obtain a first result; wherein, the first environment is a laboratory environment and / or a non-laboratory environment; Based on the first result, determine whether the first model and / or the first function is available.
9. The method according to claim 8, wherein, The first performance and / or the first metric and / or the first requirement are associated with the second performance and / or the second metric and / or the second requirement; The terminal does not support testing and / or verification and / or certification and / or supervision and / or evaluation of the second performance and / or the second metric and / or the second requirement, which are used to determine whether the first model and / or the first function is available.
10. The method according to claim 8, wherein, The terminal is deployed with a second model and / or a second function, the second model and / or the second function having been tested and / or verified and / or certified and / or supervised and / or evaluated in the first environment, and the method further includes: In the first environment, test and / or verify and / or authenticate and / or supervise and / or evaluate the first performance and / or the first metric and / or the first requirement of the terminal under the second model and / or the second function to obtain a second result; The step of determining whether the first model and / or the first function is available based on the first result includes: Based on the relationship between the first result and the second result, determine whether the first model and / or the first function is available.
11. The method according to claim 8, wherein, The method further includes: In the first environment, the first performance and / or the first metric and / or the first requirement of the terminal are tested and / or verified and / or certified and / or supervised and / or evaluated when no model and / or function is activated, to obtain a third result; The step of determining whether the first model and / or the first function is available based on the first result includes: Based on the relationship between the first result and the third result, determine whether the first model and / or the first function is available.
12. The method according to claim 8, wherein, The step of determining whether the first model and / or the first function is available based on the first result includes: Based on the relationship between the first result and the first performance threshold, determine whether the first model and / or the first function is available.
13. The method according to any one of claims 1-12, wherein, The method further includes: Send an activation request to the network device; wherein the activation request is used to request the activation of the first state; Receive activation information sent by the network device and activate the first state.
14. The method according to claim 13, wherein, The method further includes: Send an activation response to the network device; wherein the activation response is used to indicate that the first state has been activated.
15. The method according to any one of claims 1-14, wherein, The method further includes: Receive an inquiry message sent by a network device; wherein the inquiry message is used to inquire about the capabilities of the terminal; Send information about the functions and / or models supported by the terminal to the network device.
16. The method according to any one of claims 1-15, wherein, The method further includes: Receive the inference configuration of the first model and / or the first function sent by the network device.
17. The method according to any one of claims 1-16, wherein, The method further includes: Send capability information of the first model and / or the first function to the network device.
18. The method according to any one of claims 1-17, wherein, The method further includes: Receive the test and / or verification and / or certification and / or supervision and / or evaluation configuration of the first model and / or the first function sent by the network device.
19. The method according to any one of claims 1-18, wherein, The method further includes: Receive deactivation information sent by the network device and deactivate the first state.
20. The method according to claim 19, wherein, The method further includes: Send a deactivation response to the network device; wherein the deactivation response is used to indicate that the first state has been deactivated.
21. The method according to any one of claims 1-20, wherein, The method further includes: Send an activation message to the network device; wherein the activation message is used to indicate that the first state has been activated, deactivated, or is about to be activated.
22. The method according to claim 21, wherein, The activation message carries at least the time information for activating and / or deactivating the first state.
23. A terminal, the terminal having deployed a first model and / or a first function, the terminal comprising: An evaluation module is used to test and / or verify and / or authenticate and / or supervise and / or evaluate whether the first model and / or the first function is available in the first state. The first state is to activate the terminal's special functions and / or testing functions and / or verification functions and / or authentication functions and / or supervision functions and / or evaluation functions and / or testing mode and / or verification mode and / or authentication mode and / or supervision mode and / or evaluation mode.
24. A terminal, comprising: Processor, memory, and communication bus; The communication bus is used to realize the communication connection between the processor and the memory; The processor is configured to execute one or more computer programs stored in the memory to implement the evaluation method as described in any one of claims 1-22.
25. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the evaluation method as described in any one of claims 1-22.
26. A computer program product comprising a computer program, wherein, When the computer program is executed by a processor, it implements the evaluation method as described in any one of claims 1-22.