Model testing method and apparatuses

Abstract

Provided are a model testing method and apparatuses. The method comprises: a target device determines an auxiliary test model used for testing a first model, wherein the auxiliary test model is determined on the basis of a first condition, and the first condition is associated with one or more of the following: a training mode of the first model, an acquisition mode of the first model, a training mode of a peer model of the first model, an acquisition mode of the peer model of the first model, whether the first model is obtainable, whether the peer model of the first model is obtainable, whether the first model has been successfully deployed, whether the peer model of the first model has been successfully deployed, whether a reference model has been successfully deployed, whether a peer model of the reference model has been successfully deployed, and whether a test device supports the peer model of the first model. The target device can select the auxiliary test model on the basis of the first condition, so as to subsequently test the first model on the basis of the auxiliary test model, thus helping to enhance the probability of obtaining auxiliary test models.

Description

Method and device for model testing Technical Field

[0001] The present application relates to the field of communication technology, and more particularly, to a method and apparatus for model testing. Background Art

[0002] At present, in order to ensure the normal use of the model (also called the first model) in the communication system, the model needs to be tested. If the test of the first model passes, communication can be carried out based on the first model. If the test of the first model fails, communication can be carried out based on the traditional method. In some scenarios, the test equipment needs to test the first model based on the auxiliary test model associated with the first model. At present, in the test method for testing the first model, only a relatively rough definition of the method for obtaining the auxiliary test model is given. This rough definition may limit the accuracy of the test method.

[0003] Summary of the Invention

[0004] The present application provides a method and apparatus for model testing. The following introduces various aspects of the present application.

[0005] In a first aspect, a method for model testing is provided, comprising: a target device determines an auxiliary test model for testing a first model, wherein the auxiliary test model is determined based on a first condition, and the first condition is associated with one or more of the following: a training method of the first model; a method for obtaining the first model; a training method of a peer model of the first model; a method for obtaining the peer model of the first model; whether the first model is available; whether the peer model of the first model is available; whether the first model is successfully deployed; whether the peer model of the first model is successfully deployed; whether the reference model is successfully deployed; whether the peer model of the reference model is successfully deployed; and whether the test device supports the peer model of the first model.

[0006] In a second aspect, a device for model testing is provided, comprising: a processing unit, for determining an auxiliary test model for testing a first model, wherein the auxiliary test model is determined based on a first condition, and the first condition is associated with one or more of the following: a training method of the first model; a method for obtaining the first model; a training method of the peer model of the first model; a method for obtaining the peer model of the first model; whether the first model is available; whether the peer model of the first model is available; whether the first model is successfully deployed; whether the peer model of the first model is successfully deployed; whether the reference model is successfully deployed; whether the peer model of the reference model is successfully deployed; whether the test device supports the peer model of the first model.

[0007] In a third aspect, a device for model testing is provided, comprising a processor, a memory, and a communication interface, wherein the memory is used to store one or more computer programs, and the processor is used to call the computer program in the memory so that the terminal device executes part or all of the steps in the method of the first aspect.

[0008] In a fourth aspect, an embodiment of the present application provides a communication system, which includes the above-mentioned device for model testing. In another possible design, the system may also include other devices that interact with the device for model testing in the solution provided in the embodiment of the present application.

[0009] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program enables a communication device (for example, an apparatus for model testing) to perform some or all of the steps in the methods of the above aspects.

[0010] In a sixth aspect, embodiments of the present application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, wherein the computer program is operable to cause a device for model testing to perform some or all of the steps of the methods described in each of the above aspects. In some implementations, the computer program product may be a software installation package.

[0011] In the seventh aspect, an embodiment of the present application provides a chip, which includes a memory and a processor. The processor can call and run a computer program from the memory to implement some or all of the steps described in the methods of the above aspects.

[0012] In an embodiment of the present application, the target device can select an auxiliary test model based on the first condition, so that the first model can be tested subsequently based on the auxiliary test model. Compared with the traditional auxiliary test model acquisition method, it helps to increase the possibility of obtaining the auxiliary test model, thereby improving the accuracy of testing the first model using the test method. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] FIG1 is a wireless communication system 100 used in an embodiment of the present application.

[0014] FIG2 is a schematic diagram of a dual-end solution in a channel state information (CSI) feedback process.

[0015] FIG3 is a schematic flow chart of a method for model testing according to an embodiment of the present application.

[0016] FIG4 is a schematic diagram of a method for obtaining a model in a communication scenario between a terminal device and a network device.

[0017] FIG5 is a schematic flow chart of a method for model testing according to another embodiment of the present application.

[0018] FIG6 is a schematic flow chart of a method for model testing according to another embodiment of the present application.

[0019] FIG7 is a schematic diagram of a device for model testing according to an embodiment of the present application.

[0020] FIG8 is a schematic structural diagram of a device according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] The technical solution in this application will be described below with reference to the accompanying drawings.

[0022] Figure 1 illustrates a wireless communication system 100 used in an embodiment of the present application. The wireless communication system 100 may include a network device 110 and a terminal device 120. The network device 110 may be a device that communicates with the terminal device 120. The network device 110 may provide communication coverage for a specific geographic area and may communicate with the terminal device 120 within the coverage area.

[0023] FIG1 exemplarily shows a network device and two terminals. Optionally, the wireless communication system 100 may include multiple network devices and each network device may include other numbers of terminal devices within its coverage area, which is not limited in the embodiments of the present application.

[0024] Optionally, the wireless communication system 100 may further include other network entities such as a network controller and a mobility management entity, which is not limited in the embodiment of the present application.

[0025] It should be understood that the technical solutions of the embodiments of the present application can be applied to various communication systems, such as: fifth generation (5G) system or new radio (NR), long term evolution (LTE) system, LTE frequency division duplex (FDD) system, LTE time division duplex (TDD), etc. The technical solutions provided in this application can also be applied to future communication systems, such as the sixth generation mobile communication system, satellite communication system, etc.

[0026] The terminal device in the embodiments of the present application may also be referred to as user equipment (UE), access terminal, user unit, user station, mobile station, mobile station (MS), mobile terminal (MT), remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent or user device. The terminal device in the embodiments of the present application may refer to a device that provides voice and / or data connectivity to a user and can be used to connect people, objects and machines, such as a handheld device with wireless connection function, a vehicle-mounted device, etc. The terminal device in the embodiments of the present application can be a mobile phone, a tablet computer, a laptop computer, a PDA, a mobile internet device (MID), a wearable device, a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, etc. Optionally, the UE can be used to act as a base station. For example, the UE can act as a scheduling entity that provides sidelink signals between UEs in V2X or D2D, etc. For example, a cellular phone and a car communicate with each other using sidelink signals. The cellular phone and smart home devices communicate without relaying the communication signal through the base station.

[0027] The network device in the embodiments of the present application may be a device for communicating with a terminal device, and may also be referred to as an access network device or a radio access network device. For example, the network device may be a base station. The network device in the embodiments of the present application may refer to a radio access network (RAN) node (or device) that connects a terminal device to a wireless network. A base station can broadly cover various names as follows, or be replaced with the following names, such as: NodeB, evolved NodeB (eNB), next generation NodeB (gNB), relay station, access point, transmission point (TRP), transmission point (TP), master station MeNB, secondary station SeNB, multi-standard radio (MSR) node, home base station, network controller, access node, wireless node, access point (AP), transmission node, transceiver node, baseband unit (BBU), remote radio unit (RRU), active antenna unit (AAU), remote radio head (RRH), central unit (CU), distributed unit (DU), positioning node, etc. A base station can be a macro base station, a micro base station, a relay node, a donor node or the like, or a combination thereof. A base station can also refer to a communication module, a modem or a chip used to be set in the aforementioned device or apparatus. The base station can also be a mobile switching center and a device that performs base station functions in device-to-device D2D, vehicle-to-everything (V2X), and machine-to-machine (M2M) communications, a network-side device in a 6G network, or a device that performs base station functions in future communication systems. The base station can support networks with the same or different access technologies. The embodiments of this application do not limit the specific technology and specific device form used by the network equipment.

[0028] Base stations can be fixed or mobile. For example, a helicopter or drone can be configured to act as a mobile base station, and one or more cells can move based on the location of the mobile base station. In other examples, a helicopter or drone can be configured to act as a device that communicates with another base station.

[0029] In some deployments, the network device in the embodiments of the present application may refer to a CU or a DU, or the network device may include a CU and a DU. The gNB may also include an AAU.

[0030] The network equipment and terminal devices can be deployed on land, including indoors or outdoors, handheld or vehicle-mounted; they can also be deployed on water; they can also be deployed in the air on aircraft, balloons, and satellites. The embodiments of this application do not limit the scenarios in which the network equipment and terminal devices are located.

[0031] It should be understood that all or part of the functions of the communication device in this application can also be implemented through software functions running on hardware, or through virtualization functions instantiated on a platform (such as a cloud platform).

[0032] Model-Based Solutions for Wireless Communications

[0033] With the development of artificial intelligence technology and / or machine learning technology, artificial intelligence (AI) models and / or machine learning (ML) models have been introduced into communication systems, forming model-based wireless communication solutions. At present, model-based wireless communication solutions can be divided into two categories, one is a single-end solution, and the other is a dual-end solution. For a single-end solution, a model-based solution is usually only set in one device. For example, a model-based solution can be set only in a terminal device. For another example, a model-based solution can be set only in a network device. For a dual-end solution, it is usually necessary for the terminal device and the network device to cooperate with each other, that is, models are set in both the terminal device and the network device. The following describes the dual-end solution in the CSI feedback process in conjunction with Figure 2.

[0034] As shown in Figure 2, in the CSI compression process of the transmitter 210, an encoder based on an AI model (also called "AI encoder 211 or CSI compression model") can be introduced. Correspondingly, a decoder based on an AI model (also called "AI decoder 221 or CSI recovery model") is introduced in the CSI recovery process of the receiver 220. In other words, a model-based solution is provided in both the transmitter 210 and the receiver 220. Therefore, this solution can also be called a "dual-end model solution". In the transmitter 210, the CSI to be transmitted can be compressed and encoded by the AI ​​encoder 211 to obtain compressed CSI. The compressed CSI is then fed back to the receiver 220 through a feedback link. Accordingly, the receiver 220 can decode the compressed CSI through the AI ​​decoder 221 to obtain the recovered CSI.

[0035] In some scenarios, to improve CSI recovery accuracy, the CSI compression model and CSI recovery model are often matched. For example, the CSI compression model and CSI recovery model can be jointly trained. If the CSI compression model and CSI recovery model are previously mismatched, the accuracy of the CSI recovery process may be reduced.

[0036] It should be noted that the above description uses the CSI feedback process as an example to introduce a model-based dual-end solution. In some implementations, the model-based dual-end solution can also be applied to signal modulation or demodulation, source (or channel) encoding or decoding, and other matching communication processes between transmitters and receivers. In other words, in a dual-end solution, models are deployed on both the transmitting and receiving ends of the communication system. Accordingly, the transmitting and receiving models can jointly perform one or more wireless transmission functions.

[0037] Model-based approach to building dual-end solutions

[0038] In actual communication systems, building a model-based, dual-end solution typically requires consideration of the nodes where the model is trained and deployed, particularly the issue of different training and deployment nodes. Currently, the training and acquisition methods for the models used in dual-end solutions (also known as "dual-end models") can be divided into three major categories. The following sections describe the acquisition methods using single-end training, joint training, and sequential training as examples.

[0039] For single-end training, both end models can be trained at one end device, and after the training is completed, the model that needs to be deployed by the other end model (also called "other end model") is sent to the other end model.

[0040] For example, the terminal device trains the dual-end model, and after training is complete, the terminal device can transmit the peer model to the network device for deployment. For another example, the network device trains the dual-end model, and after training is complete, the network device transmits the peer model to the terminal device for deployment.

[0041] For joint training, the dual-end model can be deployed on two different devices respectively, and accordingly, the two devices jointly train the dual-end model.

[0042] For example, one model in the two-end model can be deployed on the terminal device, and the other model in the two-end model can be deployed on the network device. Accordingly, during the joint training process, the network device and the terminal device can jointly train the two-end model. In other words, the terminal device and the network device can jointly complete the information forward and backhaul required for model training, and rely on the information interaction between the network device and the terminal device to jointly complete the joint training of the two-end model.

[0043] For sequential training, the dual-end model can be deployed on two different devices. Accordingly, the model can be trained on one device first. After the training is completed, the device can transmit the data required to train the model of the other device to the other device. After that, the other device can train the model based on the received data.

[0044] For example, the dual-end model can be deployed on the network device and the terminal device respectively. The network device can train one of the dual-end models and send the training data set to the terminal device after the training is completed. Correspondingly, the terminal device can train the terminal device model based on the training data set. Afterwards, the dual-end models trained based on the training data set can be matched and used with each other at the network device and the terminal device respectively.

[0045] For another example, the dual-end model can be deployed on the network device and the terminal device respectively. The terminal device can train one of the dual-end models and send the training data set to the network device after the training is completed. Correspondingly, the network device can train the model of the network device based on the training data set. Afterwards, the dual-end models trained based on the training data set can be matched and used on the network device and the terminal device respectively.

[0046] Testing Methods for Two-End Models

[0047] Currently, to ensure the proper functioning of the dual-ended model within a communication system, it is necessary to test the dual-ended model. If the dual-ended model passes the test, communication can proceed based on the dual-ended model. If the dual-ended model fails the test, communication can proceed based on traditional methods. This dual-ended model testing method differs from traditional testing of specific devices, functions, and solutions. The dual-ended model testing method requires consideration of model matching between the devices where the dual-ended model is deployed.

[0048] Assume that the dual-end model is deployed on the terminal device and the network device respectively, that is, the terminal device is deployed with the first model in the dual-end model, and the network device is deployed with the opposite-end model of the dual-end model. If the first model deployed in the terminal device is tested, it needs to be based on the opposite-end model deployed in the network device. If the test device cannot obtain the opposite-end model to assist in the test of the first model, the uncertainty of the test quality will increase. In other words, if the test result for the first model is poor in this case, it may be due to the improper selection of the model selected by the test device to assist in the test of the first model (also known as the "auxiliary test model"). It is not due to problems with the first model itself (for example, design problems of the first model, implementation problems of the first model, or usage problems of the first model) that cause the test result for the first model to be poor. Therefore, according to the different ways of obtaining the opposite-end model, the test methods of the dual-end model can be divided into three categories. Assume that the model to be tested is deployed on device 1 and the opposite-end model is deployed on device 2.

[0049] In the first test method, device 1 provides a peer model to ensure the accuracy of the test.

[0050] For example, if the model to be tested is deployed on a terminal device, the terminal device can provide the peer model to the test device. For another example, if the model to be tested is deployed on a network device, the network device can provide the peer model to the test device.

[0051] In the second test method, device 2 provides a peer model to ensure the accuracy of the test.

[0052] For example, if the model to be tested is deployed on a terminal device, the network device can provide the peer model to the test device. For another example, if the model to be tested is deployed on a network device, the terminal device can provide the peer model to the test device.

[0053] The third type of testing method defines a common model in the protocol to assist in the testing of the first model to ensure the accuracy of the test.

[0054] For example, if the model to be tested is deployed on a terminal device, the test device can test the model to be tested based on the public model defined by the protocol. For another example, if the model to be tested is deployed on a network device, the test device can test the model to be tested based on the public model defined by the protocol.

[0055] Currently, each of the aforementioned different testing methods has its own problems. For the first type of testing method, the test equipment typically needs to store a large number of models to adapt to the different models to be tested. For the second type of testing method, since the auxiliary test model is provided by device 2 and does not fully match the model to be tested, the accuracy of the testing method may be reduced. For the third type of testing method, the mismatch between the model to be tested and the public model is even more significant, which may significantly reduce the accuracy of the testing method.

[0056] Based on the above introduction, it can be seen that the above three categories of testing methods only roughly define the methods for obtaining auxiliary test models of the model to be tested. This rough definition may limit the accuracy of the testing methods of the model to be tested.

[0057] For example, if the model to be tested is deployed on the terminal device and the peer model is transmitted to the terminal device by the network device, at this time, the terminal device itself does not have the peer model of the model to be tested. Therefore, the first model cannot be tested based on the first type of testing method. In other words, it is difficult to achieve the terminal device providing the peer model as an auxiliary test model to complete the test of the model to be tested.

[0058] For example, if the model to be tested is deployed on a network device, and the model to be tested and the peer model are trained by a terminal device, and then the terminal device transmits the model to be tested to the network device, then the network device does not store the peer model. Therefore, it is impossible to test the first model based on the first type of test method. In other words, it is difficult to implement the network device providing the peer model as an auxiliary test model to complete the test of the model to be tested.

[0059] For example, if the model to be tested is deployed on the terminal device, and the model to be tested and the peer model are also trained by the terminal device, in addition, the terminal device has not transmitted the peer model to the network device before use. At this time, the network device itself does not have the peer model. Therefore, it is impossible to test the first model based on the above-mentioned second type of testing method. In other words, it is difficult for the network device to provide the peer model as an auxiliary test model to complete the test of the model to be tested.

[0060] For example, if the model to be tested is deployed on a network device, and the model to be tested and the peer model are also trained by the network device, in addition, the network device has not transmitted the peer model to the terminal device before use. At this time, the terminal device itself does not have the peer model. Therefore, it is impossible to test the first model based on the above-mentioned second type of testing method. In other words, it is difficult for the terminal device to provide the peer model as an auxiliary test model to complete the test of the model to be tested.

[0061] In response to the above problems, the present application provides a method for model testing, in which the target device can select an auxiliary test model based on a first condition, so that the model to be tested (hereinafter referred to as the "first model") can be subsequently tested based on the auxiliary test model. Compared with the traditional method of obtaining auxiliary test models, this method helps to increase the possibility of obtaining the auxiliary test model, thereby improving the accuracy of testing the first model.

[0062] In some implementations, the first model may be deployed on a network device or the first model may be deployed on a terminal device.

[0063] In some implementations, the auxiliary test model may include one or more of the following: a peer model of the first model; a reference test model; and a predefined model.

[0064] The peer model may be a model that matches the first model, or in other words, the peer model may be a model used in conjunction with the first model. In some implementations, the peer model may be a model trained together with the first model. For example, in a dual-end solution, the peer model may be a model trained together with the first model. In other implementations, the peer model may be a model deployed on a peer device, where the peer device may be understood as a peer device of a device that deploys the first model. For example, the device that deploys the first model may communicate with the peer device.

[0065] Taking the first model as an example of a CSI compression model, the first model is deployed in the transmitter. Accordingly, the peer model can be a CSI recovery model, and the peer model is deployed at the receiver. Taking the first model as an example of a CSI generation model, the first model is deployed in the transmitter. Accordingly, the peer model can be a CSI reconstruction model, and the peer model is deployed at the receiver.

[0066] The above-mentioned reference test model (also referred to as "reference model") may be a model that matches the first model, or in other words, the peer model may be a model used in conjunction with the first model. In some implementations, the reference test model may be a model trained together with the first model. For example, in a dual-end solution, the reference test model may be a model trained together with the first model. In other implementations, the reference test model may be a model that is not deployed on the peer device, where the peer device may be understood as the peer device of the device on which the first model is deployed. For example, the reference test model may be a model that is prepared to be deployed on the peer device but has not yet been deployed on the peer device. For another example, the reference test model may be a model that is prepared to be deployed on the peer device but has not yet been deployed on the peer device.

[0067] Taking the first model as an example of a CSI compression model, the reference test model may be a CSI recovery model, and both the CSI compression model and the CSI recovery model may be trained by the transmitter, wherein the CSI recovery model is not deployed in the receiver. Taking the first model as an example of a CSI generation model, the reference test model may be a CSI reconstruction model, and both the CSI generation model and the CSI reconstruction model may be trained by the receiver, wherein the CSI generation model is not deployed in the transmitter.

[0068] The embodiments of the present application do not limit the training methods of the reference test model and the first model. For example, the reference test model and the first model can be obtained through joint training. For another example, the reference test model can be obtained through single-end training. For another example, the reference test model and the first model can be obtained through sequential training.

[0069] The predefined model may be a predefined model used for the first model test. In some implementations, the predefined model may be a model predefined by a protocol. For example, the predefined model may be the public model described above.

[0070] In the embodiments of the present application, the target device is not specifically limited. In some implementations, the target device may be a test device used to test the first model. In other implementations, the target device may be a device on which the first model is deployed, such as a terminal device or a network device. In other implementations, the target device may also be a device on which the first model is trained, such as a terminal device or a network device. In other implementations, the first model is a dual-end model, and accordingly, the target device may be a device on which the peer-end model is deployed.

[0071] In addition, in an embodiment of the present application, the device for deploying the first model may be the same as the device for training the first model. Of course, the device for deploying the first model may be different from the device for training the first model.

[0072] The following describes a method for model testing according to an embodiment of the present application in conjunction with Figure 3. The method shown in Figure 3 includes step S310.

[0073] In step S310 , the target device determines an auxiliary test model for testing the first model.

[0074] In some implementations, the auxiliary test model is determined based on a first condition, wherein the first condition is associated with one or more of the following: a training method for the first model; a method for obtaining the first model; a training method for a peer model of the first model; a method for obtaining the peer model of the first model; whether the first model is available; whether the peer model of the first model is available; whether the first model is successfully deployed; whether the peer model of the first model is successfully deployed; whether the reference model is successfully deployed; whether the peer model of the reference model is successfully deployed; and whether the test device supports the peer model of the first model.

[0075] Taking the association between the first condition and the training mode as an example, the training mode may include one or more of joint training, single-end training, and sequential training.

[0076] In some implementations, the training method can be associated with a target model, or in other words, the training method is used to train the target model. The target model can include one or more of the following: a first model; a peer model of the first model; or a reference test model of the first model.

[0077] It should be noted that the training methods of the first model and the counterpart model of the first model are matched. In some implementations, the first model and the counterpart model of the first model can be trained based on the same training method. For example, the first model and the counterpart model of the first model can be trained based on a joint training method. Therefore, in some scenarios, in order to reduce the complexity of determining the first condition, only the training method of the first model can be determined. In other scenarios, in order to reduce the complexity of determining the first condition, only the training method of the counterpart model of the first model can be determined. Of course, in an embodiment of the present application, the training method of the first model and the training method of the counterpart model of the first model can also be determined at the same time.

[0078] Taking the association of the first condition with the acquisition method as an example, the acquisition method can be associated with the target model, or in other words, the acquisition method can be the acquisition method of the target model. The target model can include one or more of the following: the first model; the peer model of the first model.

[0079] In some implementations, the acquisition method includes transmitting the target model from a target model providing device to a device that deploys the target model. The target model providing device may be a device that can provide the target model. For example, the providing device may be a device that trains the target model. Of course, in the embodiments of the present application, the providing device may also be a device that only obtains the target model but does not train the target model.

[0080] In the embodiments of the present application, the specific method for transmitting the target model is not limited. In some implementations, the target model can be indicated by indicating information indicating the target model. That is, the device providing the target model can transmit the indicating information of the target model to the device deploying the target model to indicate the target model. The indicating information of the target model can include an identifier of the target model; the indicating information of the target model can also include an identifier of a function corresponding to the target model.

[0081] In other implementations, the target model can be indicated by indicating the target model's training data. Specifically, the target model-providing device can transmit information indicating the target model's training data to the device deploying the target model to indicate the target model. Accordingly, the device deploying the target model can train an initial model of the target model based on the indicated training data to obtain the target model. The information indicating the target model's training data can include an identifier for the target model's training data.

[0082] In other implementations, the target model can be transmitted directly, that is, the target model providing device can transmit the target model to the device deploying the target model. Transmitting the target model may include one or more of the following: transmitting the model parameter information of the target model; transmitting the model structure information of the target model. Taking the transmission of the model parameter information of the target model as an example, the device deploying the target model can obtain the target model based on the combination of the model parameter information and the pre-stored structure information of the target model. Taking the transmission of the model structure information of the target model as an example, the device deploying the target model can obtain the target model based on the combination of the model structure information of the target model and the pre-stored parameter information of the target model. Taking the transmission of the model structure information of the target model and the parameter information of the target model as an example, the device deploying the target model can obtain the target model based on the model structure information of the target model and the parameter information of the target model.

[0083] In other implementations, the training data of the target model can be directly transmitted so that the device deploying the target model can train the target model based on the training data. In other words, the device providing the target model can transmit the training data of the target model to the device deploying the target model, and the device deploying the target model can accordingly obtain the target model based on the training data.

[0084] In other implementations, the acquisition method may include training the target model by a device that deploys the target model. That is, in the embodiment of the present application, the device that deploys the target model and the device that trains the target model are the same device.

[0085] For ease of understanding, the following describes, in conjunction with FIG4 , a method for obtaining the first model and a method for obtaining the auxiliary test model in a communication scenario between a terminal device and a network device.

[0086] As shown in FIG4 , for the first model of the device under test 410, the first model may be obtained by being provided by the node under test, where the node under test may be a terminal device or a network device. The first model may be obtained by being provided by a peer node of the node under test, where, for example, the node under test is a terminal device, the peer node may be a network device. For example, where the node under test is a network device, the peer node may be a terminal device. The first model may be obtained by being predefined by a protocol.

[0087] For test device 420, the auxiliary test model may be obtained by being provided by the node under test, where the node under test can be a terminal device or a network device. Alternatively, the auxiliary test model may be obtained by being provided by a peer node of the node under test, where, for example, the node under test is a terminal device, the peer node can be a network device. Alternatively, the auxiliary test model may be obtained by being predefined by a protocol.

[0088] Taking the first condition and whether the target model is available as an example, the target model may include one or more of the following: the first model, and the peer model of the first model.

[0089] In some implementations, whether the target model is obtainable may include whether the target model can be acquired by the test device. That is, the first condition may include whether the test device can acquire the target model.

[0090] For example, assume that the peer model of the first model can be deployed on device 1, the first model and the peer model of the first model are trained on device 2, and the test device obtains the peer model of the first model from device 1. In this case, if device 2 does not send the peer model of the first model to device 1, the peer model of the first model is not stored in device 1. In this case, the test device cannot obtain the peer model of the first model from device 1.

[0091] For another example, assume that the first model can be deployed on device 1, the first model and its peer model are trained on device 2, and the test device obtains the first model from device 1. In this case, if device 2 does not send the first model to device 1, the first model is not stored on device 1. In this case, the test device cannot obtain the first model from device 1.

[0092] Taking the first condition and whether the target model is successfully deployed as an example, the target model may include one or more of the following: a first model, a peer model of the first model; a reference model; and a peer model of the reference model.

[0093] In some implementations, whether the target model is successfully deployed may include one or more of the following: whether the target model is successfully transmitted; whether the indication information of the target model is successfully transmitted; whether the training data of the target model is successfully transmitted; whether the indication information of the training data of the target model is successfully transmitted.

[0094] In an embodiment of the present application, whether the target model is successfully deployed may include whether the target model providing device successfully deploys the target model in the target model device (hereinafter referred to as the "deployment device"), wherein the target model providing device may be a device that can provide the target model. For example, the providing device may be a training device for the target model. For another example, the providing device may also be a device that only obtains the target model but does not train the target model.

[0095] Accordingly, whether the target model is successfully deployed may include one or more of the following: whether the providing device transmits the target model to the deployment device, whether the providing device successfully transmits the indication information of the target model to the deployment device, whether the providing device transmits the training data of the target model to the deployment device, and whether the providing device successfully transmits the indication information of the training data of the target model to the deployment device.

[0096] Taking the target model as the first model as an example, the first condition is associated with whether the first model is successfully deployed, and whether the first model is successfully deployed is associated with one or more of the following: whether the first model is successfully transmitted; whether the first model is successfully indicated; whether the training data of the first model is successfully transmitted; whether the training data of the first model is successfully indicated.

[0097] Taking the target model as the peer model of the first model as an example, the first condition is associated with whether the peer model of the first model is successfully deployed, and whether the peer model of the first model is successfully deployed is associated with one or more of the following: whether the peer model of the first model is successfully transmitted; whether the peer model of the first model is successfully indicated; whether the training data of the peer model of the first model is successfully transmitted; whether the training data of the peer model of the first model is successfully indicated; and / or

[0098] Taking the target model as the reference model as an example, the first condition is associated with whether the reference model is successfully deployed, and whether the reference model is successfully deployed is associated with one or more of the following: whether the reference model is successfully transmitted; whether the reference model is successfully indicated; whether the training data of the reference model is successfully transmitted; whether the training data of the reference model is successfully indicated; and / or

[0099] Taking the target model as the peer model of the reference model as an example, the first condition is associated with whether the peer model of the reference model is successfully deployed, and whether the peer model of the reference model is successfully deployed is associated with one or more of the following: whether the peer model of the reference model is successfully transmitted; whether the peer model of the reference model is successfully indicated; whether the training data of the peer model of the reference model is successfully transmitted; whether the training data of the peer model of the reference model is successfully indicated.

[0100] Taking the first condition being associated with whether the test device supports the peer model of the first model as an example, alternatively, the first condition may include the test device supporting the peer model of the first model and the test device not supporting the peer model of the first model.

[0101] In some scenarios, the model has multiple compilation formats, for example, TensorFlow model format, open neural network exchange (ONNX) model format, Keras model format, Caffe (convolutional architecture for fast feature embedding) model format, PyTorch model format, and so on. The measuring device may not be able to recognize every compilation format. At this time, it is possible that the test device can obtain the peer model of the first model, but the test device cannot recognize the peer model of the first model, that is, the test device does not support the peer model of the first model. In an embodiment of the present application, the reason why the test device cannot recognize the peer model of the first model is not specifically limited. For example, the reason why the test device does not support the peer model of the first model may also include the fact that the test device cannot recognize the identifier of the peer model of the first model.

[0102] The first condition in the embodiment of the present application is described above. The above first condition can be used alone to select the auxiliary test model of the first model. Of course, in the embodiment of the present application, the above first conditions can also be used in combination with each other to select the auxiliary test model of the first model. The following is an introduction in conjunction with Implementation Methods 1 to 6. It should be understood that the combination of the first conditions in the embodiment of the present application is not limited to this.

[0103] Implementation method 1: If the first condition includes that the first model and the peer model of the first model are trained by the first device, and the peer model of the first model is deployed by the first device for the second device, then the auxiliary test model is the peer model of the first model.

[0104] In some implementations, the peer model of the first model is indicated by the first device to the testing device. For example, the first device may indicate an identifier of the peer model of the first model to the testing device. In other implementations, the peer model of the first model is transmitted by the first device to the testing device of the first model. For example, the first device may send model structure information and / or model parameter information of the peer model of the first model to the testing device.

[0105] In some implementations, the first device deploying the peer model of the first model for the second device may include the first device transmitting the peer model of the first model to the second device, for example, the first device transmitting model structure information and / or model parameter information of the peer model of the first model to the second device. In other implementations, the first device deploying the peer model of the first model for the second device may include the first device transmitting training data of the peer model of the first model to the second device. In other implementations, the first device deploying the peer model of the first model for the second device may include the first device indicating the peer model of the first model to the second device, for example, the first device indicating an identifier of the peer model of the first model to the second device. In other implementations, the first device deploying the peer model of the first model for the second device may include the first device indicating the training data of the peer model of the first model to the second device. For example, the first device may indicate an identifier of the training data of the peer model of the first model to the second device.

[0106] Assume that the first device is a network device and the first model is deployed in the network device. The first condition includes the network device training the first model and a peer model of the first model, and the network device deploying the peer model of the first model on the terminal device. In this case, the auxiliary test model of the first model can be the peer model of the first model provided by the network device.

[0107] For example, the first model is a CSI reconstruction model, and its counterpart is a CSI generation model. The CSI generation model is deployed in a terminal device, and the CSI reconstruction model is deployed in a network device. The first condition includes the network device training the CSI generation model and the CSI reconstruction model, sending training data for the CSI generation model to the terminal device, and / or sending the CSI generation model to the terminal device. In this case, the auxiliary test model for the CSI reconstruction model can be the CSI generation model provided by the network device.

[0108] For example, the first model is a CSI recovery model, and the counterpart model of the first model is a CSI compression model. The CSI recovery model is deployed in a network device, and the CSI compression model is deployed in a terminal device. The first condition includes the network device training the CSI compression model and the CSI recovery model, sending training data for the CSI compression model to the terminal device, and / or sending the CSI compression model to the terminal device. In this case, the auxiliary test model for the CSI recovery model can be a CSI reconstruction model provided by the network device.

[0109] Assume that the first device is a terminal device and the first model is deployed in the terminal device. The first condition includes that the terminal device trains the first model and a peer model of the first model, and that the terminal device deploys the peer model of the first model on a network device. In this case, the auxiliary test model of the first model can be the peer model of the first model provided by the terminal device.

[0110] For example, the first model is a CSI generation model, and its counterpart is a CSI reconstruction model. The CSI generation model is deployed in a terminal device, and the CSI reconstruction model is deployed in a network device. The first condition includes the terminal device training the CSI generation model and the CSI reconstruction model, sending training data for the CSI reconstruction model to the network device, and / or sending the CSI reconstruction model to the network device. In this case, the auxiliary test model for the CSI generation model can be the CSI reconstruction model provided by the terminal device.

[0111] For example, a first model is a CSI compression model, and the counterpart model of the first model is a CSI recovery model. The CSI recovery model is deployed in a network device, and the CSI compression model is deployed in a terminal device. The first condition includes the terminal device training the CSI compression model and the CSI recovery model, and the terminal device sending training data for the CSI recovery model to the network device, and / or the terminal device sending the CSI recovery model to the network device. In this case, the auxiliary test model of the CSI compression model can be a CSI recovery model provided by the terminal device.

[0112] Implementation method 2: If the first condition includes that the first model and the peer model of the first model are trained by the first device, and the first model is deployed by the first device to the second device, then the auxiliary test model is the peer model of the first model.

[0113] In some implementations, the peer model of the first model is indicated by the first device to the testing device. For example, the first device may indicate an identifier of the peer model of the first model to the testing device. In other implementations, the peer model of the first model is transmitted by the first device to the testing device of the first model. For example, the first device may send model structure information and / or model parameter information of the peer model of the first model to the testing device.

[0114] In some implementations, the first device deploying the first model for the second device may include the first device transmitting the first model to the second device, for example, the first device transmitting model structure information and / or model parameter information of the first model to the second device. In other implementations, the first device deploying the first model for the second device may include the first device transmitting training data of the first model to the second device. In other implementations, the first device deploying the first model for the second device may include the first device indicating the first model to the second device, for example, the first device indicating an identifier of the first model to the second device. In other implementations, the first device deploying the first model for the second device may include the first device indicating the training data of the first model to the second device. For example, the first device may indicate an identifier of the training data of the first model to the second device.

[0115] Assume that the first device is a network device, the first model is a model deployable on a terminal device, and the first condition includes the network device training the first model and a peer model of the first model, and the network device deploying the first model on the terminal device. In this case, the auxiliary test model of the first model can be a peer model of the first model provided by the network device.

[0116] For example, the first model is a CSI generation model, and its counterpart is a CSI reconstruction model. The CSI generation model is deployed in a terminal device, and the CSI reconstruction model is deployed in a network device. The first condition includes the network device training the CSI generation model and the CSI reconstruction model, sending training data for the CSI generation model to the terminal device, and / or sending the CSI generation model to the terminal device. In this case, the auxiliary test model for the CSI generation model can be a CSI reconstruction model provided by the network device.

[0117] For example, the first model is a CSI compression model, and the counterpart model of the first model is a CSI recovery model. The CSI compression model is deployed in a terminal device, and the CSI recovery model is deployed in a network device. The first condition includes the network device training the CSI compression model and the CSI recovery model, and the network device sending training data for the CSI compression model to the terminal device, and / or the network device sending the CSI compression model to the terminal device. In this case, the auxiliary test model for the CSI compression model can be a CSI recovery model provided by the network device.

[0118] Assume that the first device is a terminal device, the first model is a model deployable on a network device, and the first condition includes the terminal device training the first model and a peer model of the first model, and the terminal device deploying the first model on the network device. In this case, the auxiliary test model of the first model can be the peer model of the first model provided by the terminal device.

[0119] For example, the first model is a CSI reconstruction model, and its counterpart is a CSI generation model. The CSI reconstruction model is deployed in a network device, and the CSI generation model is deployed in a terminal device. The first condition includes the terminal device training the CSI generation model and the CSI reconstruction model, and the terminal device sending training data for the CSI reconstruction model to the network device, and / or the terminal device sending the CSI reconstruction model to the network device. In this case, the auxiliary test model for the CSI reconstruction model can be the CSI generation model provided by the terminal device.

[0120] For example, the first model is a CSI recovery model, and the counterpart model of the first model is a CSI compression model. The CSI compression model is deployed in the terminal device, and the CSI recovery model is deployed in the network device. The first condition includes the terminal device training the CSI compression model and the CSI recovery model, and the terminal device sending training data for the CSI recovery model to the network device, and / or the terminal device sending the CSI recovery model to the network device. In this case, the auxiliary test model of the CSI recovery model can be the CSI compression model provided by the terminal device.

[0121] Implementation method 3: If the first condition includes that the first model and the reference model are trained by the first device, the reference model can be deployed by the first device for the second device, and the second device has not deployed the reference model, then the auxiliary test model is the reference model.

[0122] In some implementations, a reference model may be deployed by a first device for a second device, and the second device may not have the reference model deployed. This can be understood as the second device being able to obtain the reference model but not yet having it. For example, the first device may be a network device, and the second device may be a terminal device. In other words, the first condition described above can be understood as the network device training the first model and the reference model, and then testing the first model after the terminal device leaves the factory and does not enter a cell within the coverage area of ​​the network device. At this point, the reference model has not yet been deployed to the terminal device, but the reference model can be deployed to the terminal device after the terminal device enters the cell.

[0123] For another example, the first device may be a terminal device, and the second device may be a network device. That is to say, the first condition mentioned above may be understood as the terminal device training the first model and the reference model, and when the terminal device does not enter a cell within the coverage of the network device after leaving the factory, the first model needs to be tested. At this time, the reference model has not yet been deployed to the network device, but the reference model can be deployed for the network device after the terminal device enters the cell.

[0124] In some implementations, the first device deploys a reference model for the second device, and the second device does not deploy the reference model, which may include that the first device can transmit the reference model to the second device, but the first device has not yet transmitted the reference model to the second device. For example, the first device can transmit model structure information and / or model parameter information of the reference model to the second device, but the first device has not yet transmitted the model structure information and / or model parameter information of the reference model to the second device. In other implementations, the first device deploys a reference model for the second device, and the second device does not deploy the reference model, which may include that the first device can transmit training data of the reference model to the second device, but the first device has not yet transmitted the training data of the reference model to the second device. In other implementations, the first device deploys a reference model for the second device, and the second device does not deploy the reference model, which may include that the first device can indicate the reference model to the second device, but the first device has not yet indicated the reference model to the second device. For example, the first device can indicate the identifier of the reference model to the second device, but the first device has not yet indicated the identifier of the reference model to the second device. In some other implementations, the first device deploying the reference model for the second device, and the second device not deploying the reference model, may include the first device being able to indicate training data of the reference model to the second device, but the first device not yet indicating the training data of the reference model to the second device. For example, the first device being able to indicate an identifier of the training data of the reference model to the second device, but the first device not yet indicating the identifier of the training data of the reference model to the second device.

[0125] In some implementations, the reference model is indicated by the first device to the test device. For example, the first device may indicate an identifier of the reference model to the test device. In other implementations, the reference model is transmitted by the first device to the test device. For example, the first device may send model structure information and / or model parameter information of the reference model to the test device.

[0126] Assume that the first device is a network device, and the first model is deployed on the network device. The first condition includes that the network device trains the first model and the reference model, that the reference model can be deployed by the network device for the terminal device, and that the terminal device does not have the reference model deployed. In this case, the auxiliary test model for the first model can be a reference model provided by the network device.

[0127] For example, the first model is a CSI reconstruction model, and the reference model is a CSI generation model. The CSI generation model is deployed in the terminal device, and the CSI reconstruction model is deployed in the network device. The first condition includes the network device training the CSI generation model and the CSI reconstruction model, and the network device can send CSI generation model training data to the terminal device, but the network device has not yet sent CSI generation model training data to the terminal device. In this case, the auxiliary test model for the CSI reconstruction model can be the CSI generation model provided by the network device.

[0128] For example, the first model is a CSI reconstruction model, and the reference model is a CSI generation model. The CSI generation model is deployed in the terminal device, and the CSI reconstruction model is deployed in the network device. The first condition includes the network device training the CSI generation model and the CSI reconstruction model, and the network device can send the CSI generation model to the terminal device, but the network device has not yet sent the CSI generation model to the terminal device. In this case, the auxiliary test model for the CSI reconstruction model can be the CSI generation model provided by the network device.

[0129] For example, the first model is a CSI recovery model, and the reference model is a CSI compression model. The CSI recovery model is deployed in a network device, and the CSI compression model is deployed in a terminal device. The first condition includes the network device training the CSI compression model and the CSI recovery model, and the network device can send CSI compression model training data to the terminal device, but the network device has not yet sent CSI compression model training data to the terminal device. In this case, the auxiliary test model for the CSI recovery model can be the CSI compression model provided by the network device.

[0130] For example, the first model is a CSI recovery model, and the reference model is a CSI compression model. The CSI recovery model is deployed in the network device, and the CSI compression model is deployed in the terminal device. The first condition includes the network device training the CSI compression model and the CSI recovery model, and the network device sending the CSI compression model to the terminal device, but the network device has not yet sent the CSI compression model to the terminal device. In this case, the auxiliary test model for the CSI recovery model can be the CSI compression model provided by the network device.

[0131] Assume that the first device is a terminal device, and the first model is deployed in the terminal device. The first condition includes that the terminal device has trained the first model and the reference model, and that the terminal device can deploy the reference model for the network device, but the network device has not yet deployed the reference model. In this case, the auxiliary test model of the first model can be the reference model provided by the terminal device.

[0132] For example, the first model is a CSI generation model, and the reference model is a CSI reconstruction model. The CSI generation model is deployed in the terminal device, and the CSI reconstruction model is deployed in the network device. The first condition includes the terminal device training the CSI generation model and the CSI reconstruction model, and the terminal device can send CSI reconstruction model training data to the network device, but the terminal device has not yet sent CSI reconstruction model training data to the network device. In this case, the auxiliary test model for the CSI generation model can be the CSI reconstruction model provided by the terminal device.

[0133] For example, the first model is a CSI generation model, and the reference model is a CSI reconstruction model. The CSI generation model is deployed in the terminal device, and the CSI reconstruction model is deployed in the network device. The first condition includes the terminal device training the CSI generation model and the CSI reconstruction model, and the terminal device sending the CSI reconstruction model to the network device, but the terminal device has not yet sent the CSI reconstruction model to the network device. In this case, the auxiliary test model for the CSI generation model can be the CSI reconstruction model provided by the terminal device.

[0134] For example, the first model is a CSI compression model, and the reference model is a CSI recovery model. The CSI recovery model is deployed in the network device, and the CSI compression model is deployed in the terminal device. The first condition includes that the terminal device has trained the CSI compression model and the CSI recovery model, and that the terminal device can send training data for the CSI recovery model to the network device, but the terminal device has not yet sent training data for the CSI recovery model to the network device. In this case, the auxiliary test model for the CSI compression model can be the CSI recovery model provided by the terminal device.

[0135] For example, the first model is a CSI compression model, and the reference model is a CSI recovery model. The CSI recovery model is deployed in the network device, and the CSI compression model is deployed in the terminal device. The first condition includes that the terminal device has trained the CSI compression model and the CSI recovery model, and that the terminal device can send the CSI recovery model to the network device, but the terminal device has not yet sent the CSI recovery model to the network device. In this case, the auxiliary test model of the CSI compression model can be the CSI recovery model provided by the terminal device.

[0136] Implementation method 4: If the first condition includes that the first model and the reference model are trained by the first device, and the first model can be deployed by the first device for the second device, and the second device has not deployed the first model, then the auxiliary test model is the reference model.

[0137] In some implementations, the first model can be deployed by the first device for the second device, and the second device has not deployed the first model. This can be understood as the second device having access to the first model but not yet having obtained it. For example, the first device can be a network device, and the second device can be a terminal device. That is, the first condition can be understood as the network device training the first model and the reference model, and when the terminal device does not enter a cell within the coverage area of ​​the network device after leaving the factory, the first model needs to be tested. At this time, the first model has not yet been deployed to the terminal device, but the first model can be deployed to the terminal device after the terminal device enters the cell.

[0138] For another example, the first device may be a terminal device, and the second device may be a network device. That is to say, the above-mentioned first condition may be understood as the terminal device training the first model and the reference model, and when the terminal device does not enter the cell within the coverage of the network device after leaving the factory, the first model needs to be tested. At this time, the first model has not yet been deployed to the network device, but the first model can be deployed to the network device after the terminal device enters the cell.

[0139] In some implementations, the first device deploys the first model for the second device, and the second device does not deploy the first model, which may include that the first device can transmit the first model to the second device, but the first device has not yet transmitted the first model to the second device. For example, the first device can transmit model structure information and / or model parameter information of the first model to the second device, but the first device has not yet transmitted the model structure information and / or model parameter information of the first model to the second device. In other implementations, the first device deploys the first model for the second device, and the second device does not deploy the first model, which may include that the first device can transmit training data of the first model to the second device, but the first device has not yet transmitted the training data of the first model to the second device. In other implementations, the first device deploys the first model for the second device, and the second device does not deploy the first model, which may include that the first device can indicate the first model to the second device, but the first device has not yet indicated the first model to the second device. For example, the first device can indicate the identifier of the first model to the second device, but the first device has not yet indicated the identifier of the first model to the second device. In some other implementations, the first device deploying the first model to the second device, and the second device not deploying the first model, may include the first device being able to indicate the training data of the first model to the second device, but the first device not yet indicating the training data of the first model to the second device. For example, the first device being able to indicate the identification of the training data of the first model to the second device, but the first device not yet indicating the identification of the training data of the first model to the second device.

[0140] In some implementations, the reference model is indicated by the first device to the test device. For example, the first device may indicate an identifier of the reference model to the test device. In other implementations, the reference model is transmitted by the first device to the test device. For example, the first device may send model structure information and / or model parameter information of the reference model to the test device.

[0141] It should be noted that in some scenarios, the second device can obtain a reference model. Accordingly, the second device can determine the model to be tested (also known as the peer model of the reference model) based on the reference model. In other words, the second device can obtain the model to be tested corresponding to the reference model based on the reference model. Alternatively, the second device can select the model to be tested corresponding to the reference model based on the reference model. Alternatively, the second device can perform testing using the model to be tested corresponding to the reference model based on the reference model.

[0142] Assume that the first device is a network device and the reference model is deployed on the network device. The first condition includes that the network device trains the first model and the reference model, that the first model can be deployed by the network device for the terminal device, and that the terminal device does not have the first model deployed. In this case, the auxiliary test model for the first model can be the reference model provided by the network device.

[0143] In some scenarios, if the terminal device can obtain the reference model provided by the network device, the terminal device can obtain, select, and use the corresponding model to be tested based on the reference model. Accordingly, the test device can use the above reference model as an auxiliary test model.

[0144] For example, the first model is a CSI generation model, and the reference model is a CSI reconstruction model. The CSI generation model is deployed in the terminal device, and the CSI reconstruction model is deployed in the network device. The first condition includes the network device training the CSI generation model and the CSI reconstruction model, and the network device can send CSI generation model training data to the terminal device, but the network device has not yet sent CSI generation model training data to the terminal device. In this case, the auxiliary test model for the CSI generation model can be the CSI reconstruction model provided by the network device.

[0145] For example, the first model is a CSI generation model, and the reference model is a CSI reconstruction model. The CSI generation model is deployed in the terminal device, and the CSI reconstruction model is deployed in the network device. The first condition includes the network device training the CSI generation model and the CSI reconstruction model, and the network device can send the CSI generation model to the terminal device, but the network device has not yet sent the CSI generation model to the terminal device. In this case, the auxiliary test model for the CSI generation model can be the CSI reconstruction model provided by the network device.

[0146] For example, the first model is a CSI compression model, and the reference model is a CSI recovery model. The CSI recovery model is deployed in the network device, and the CSI compression model is deployed in the terminal device. The first condition includes that the network device has trained the CSI compression model and the CSI recovery model, and that the network device can send CSI compression model training data to the terminal device, but the network device has not yet sent CSI compression model training data to the terminal device. In this case, the auxiliary test model for the CSI compression model can be the CSI recovery model provided by the network device.

[0147] For example, the first model is a CSI compression model, and the reference model is a CSI recovery model. The CSI recovery model is deployed in the network device, and the CSI compression model is deployed in the terminal device. The first condition includes the network device training the CSI compression model and the CSI recovery model, and the network device sending the CSI compression model to the terminal device, but the network device has not yet sent the CSI compression model to the terminal device. In this case, the auxiliary test model for the CSI compression model can be the CSI recovery model provided by the network device.

[0148] Assume that the first device is a terminal device and the reference model is deployed in the terminal device. The first condition includes that the terminal device has trained the first model and the reference model, and that the terminal device can deploy the first model for the network device, but the network device has not yet deployed the first model. In this case, the auxiliary test model of the first model can be the reference model provided by the terminal device.

[0149] In some scenarios, if the network device can obtain the reference model provided by the terminal device, the network device can obtain, select, and use the corresponding model to be tested based on the reference model for testing. Accordingly, the test device can use the above reference model as an auxiliary test model.

[0150] For example, the first model is a CSI reconstruction model, and the reference model is a CSI generation model. The CSI generation model is deployed in the terminal device, and the CSI reconstruction model is deployed in the network device. The first condition includes the terminal device training the CSI generation model and the CSI reconstruction model, and the terminal device can send CSI reconstruction model training data to the network device, but the terminal device has not yet sent CSI reconstruction model training data to the network device. In this case, the auxiliary test model for the CSI reconstruction model can be the CSI generation model provided by the terminal device.

[0151] For example, the first model is a CSI reconstruction model, and the reference model is a CSI generation model. The CSI generation model is deployed in the terminal device, and the CSI reconstruction model is deployed in the network device. The first condition includes the terminal device training the CSI generation model and the CSI reconstruction model, and the terminal device sending the CSI reconstruction model to the network device, but the terminal device has not yet sent the CSI reconstruction model to the network device. In this case, the auxiliary test model for the CSI reconstruction model can be the CSI generation model provided by the terminal device.

[0152] For example, the first model is a CSI recovery model, and the reference model is a CSI compression model. The CSI recovery model is deployed in a network device, and the CSI compression model is deployed in a terminal device. The first condition includes the terminal device training the CSI compression model and the CSI recovery model, and the terminal device can send CSI recovery model training data to the network device, but the terminal device has not yet sent CSI recovery model training data to the network device. In this case, the auxiliary test model for the CSI recovery model can be the CSI compression model provided by the terminal device.

[0153] For example, the first model is a CSI recovery model, and the reference model is a CSI compression model. The CSI recovery model is deployed in the network device, and the CSI compression model is deployed in the terminal device. The first condition includes that the terminal device has trained the CSI compression model and the CSI recovery model, and that the terminal device can send the CSI recovery model to the network device, but the terminal device has not yet sent the CSI recovery model to the network device. In this case, the auxiliary test model of the CSI recovery model can be the CSI compression model provided by the terminal device.

[0154] Implementation 5: If the first condition includes the test device not supporting the peer model of the first model, the auxiliary test model is a predefined model associated with the first model test. The explanation of the test device not supporting the peer model of the first model can be found above. Alternatively, the predefined model can be, for example, a public model, as described above. For the sake of brevity, this is not further explained here.

[0155] In some implementations, if the peer model of the first model is provided by a providing device, and the test device does not support the peer model of the first model, it can be replaced with a model provided by the providing device that the test device does not support, wherein the model provided by the providing device may include the peer model of the first model. In addition, in an embodiment of the present application, the providing device can be a device that can provide the peer model of the first model, for example, a training device for the peer model of the first model, and / or the providing device can be a deployment device for the first model; the providing device can be a deployment device for the peer model of the first model.

[0156] Taking the providing device as a terminal device as an example, if the test device does not support the opposite end model of the first model, it can be replaced with a model that the test device does not support and provided by the terminal device.

[0157] Assume that the first device is a terminal device, the first model is a model deployable on a network device, and the first condition includes that the terminal device trains the first model and its peer model, the terminal device deploys the first model on the network device, and the test device does not support the model provided by the terminal device. In this case, the auxiliary test model for the first model can be a predefined model associated with the first model test.

[0158] Assume that the first device is a terminal device, the first model is deployed in the terminal device, and the first condition includes that the terminal device trains the first model and a peer model of the first model, the terminal device deploys the peer model of the first model on a network device, and the test device does not support the model provided by the terminal device. In this case, the auxiliary test model of the first model can be a predefined model associated with the first model test.

[0159] Taking the providing device as a network device as an example, if the test device does not support the peer model of the first model, it can be replaced with a model that the test device does not support and provided by the network device.

[0160] Assume that the first device is a network device, the first model is a model deployable on a terminal device, and the first condition includes the network device training the first model and its peer model, deploying the first model on the terminal device, and the test device not supporting the model provided by the network device. In this case, the auxiliary test model for the first model can be a predefined model associated with the first model test.

[0161] Assume that the first device is a network device, the first model is deployed on the network device, and the first condition includes that the network device trains the first model and a peer model of the first model, the network device deploys the peer model of the first model on the terminal device, and the test device does not support the model provided by the network device. In this case, the auxiliary test model of the first model can be a predefined model associated with the test of the first model.

[0162] It should be noted that in the embodiment of the present application, the above-mentioned test device does not support the model provided by the providing device, which can also be understood as the test device does not support the reference model provided by the providing device. Accordingly, the implementation method of the first condition and the selection method of the auxiliary test model are the same as the above introduction, only the opposite end model of the first model is replaced by the reference model. For the sake of brevity, it will not be repeated here.

[0163] Implementation method 6: If the first condition includes that the second model and the peer model of the second model are trained by the first device, and the second model is not deployed to the second device, then the model to be tested is the first model deployed in the second device, and the auxiliary test model is a predefined model associated with the first model test.

[0164] That is to say, when the second model is not deployed to the second device, if the model in the second device is to be tested, the model to be tested can be the first model deployed in the second device. Accordingly, the auxiliary test model of the first model can be a predefined model associated with the first model test.

[0165] In some implementations, the above-mentioned second model not being deployed to the second device may include that the first device has not transmitted the second model to the second device. For example, the first device has not transmitted the model structure information and / or model parameter information of the second model to the second device. In other implementations, the second device not deploying the second model may include that the first device has not transmitted the training data of the second model to the second device. In other implementations, the second device not deploying the second model may include that the first device has not indicated the second model to the second device. For example, the first device has not indicated the identifier of the second model to the second device. In other implementations, the second device not deploying the second model may include that the first device has not indicated the training data of the second model to the second device. For example, the first device has not indicated the identifier of the training data of the second model to the second device.

[0166] Assuming that the first device is a terminal device and the second device is a network device, the first condition includes that the second model and the counterpart model of the second model are trained by the terminal device, and the second model is not deployed to the network device, then the model to be tested is the first model deployed in the network device, and the auxiliary test model is a predefined model associated with the first model test.

[0167] Assuming that the first device is a network device and the second device is a terminal device, the first condition includes that the second model and the counterpart model of the second model are trained by the network device, and the second model is not deployed to the terminal device, then the model to be tested is the first model deployed in the terminal device, and the auxiliary test model is a predefined model associated with the first model test.

[0168] Based on the first condition described above, it can be seen that the selection scheme of the auxiliary test model associated with different first conditions is introduced in the embodiment of the present application. The above-mentioned first condition introduces information such as the model training method, the method of obtaining the model (for example, the first model, the peer model of the first model), and whether the first model and / or the peer model are available during testing into the selection scheme of the auxiliary test model, which helps to avoid some unreasonable auxiliary test model selection schemes.

[0169] For ease of understanding, the following text combines Table 1 to take network devices and terminal devices as examples to introduce the auxiliary test model of the model to be tested in the embodiment of the present application. As shown in Table 1, Table 1 shows the acquisition scheme of the auxiliary test model of the model to be tested under the first condition. For ease of description, the model deployed in the network device is referred to as the "network model", and the model deployed in the terminal device is referred to as the "terminal model". In addition, for the reference model, the reference model that can be deployed but not deployed in the network device is referred to as the "network reference model", and the reference model that can be deployed but not deployed in the terminal device is referred to as the "terminal reference model".

[0170] Table 1

[0171] The above introduces the method of obtaining the auxiliary test model in the embodiment of the present application. The following introduces the information interaction scheme between the test device and the device to be tested (i.e., the device where the model to be tested is deployed, also known as the "device where the first model is deployed") in the embodiment of the present application.

[0172] In some implementations, if the target device is a test device, the method further includes: the device to be tested transmits first information to the test device (see step S510). In this embodiment of the present application, the transmission of the first information can be used in conjunction with the selection of the auxiliary test model described above. In this case, step S510 can be performed before step S310.

[0173] In some implementations, the first information includes one or more of the following: information for determining the first condition; information for indicating the auxiliary test model; and information for indicating the first model.

[0174] Taking the example that the first information includes information for determining the first condition, in some implementations, the information for determining the first condition may include indication information corresponding to the first condition (for example, the identifier of the first condition shown in Table 1). In other implementations, the information for determining the first condition may include the first condition. For example, the information for determining the first condition may include one or more of the following: a training method for the first model; a method for obtaining the first model; a training method for the peer model of the first model; a method for obtaining the peer model of the first model; whether the first model is available; whether the peer model of the first model is available; whether the first model is successfully deployed; whether the peer model of the first model is successfully deployed; whether the reference model is successfully deployed; whether the peer model of the reference model is successfully deployed.

[0175] Taking the scenario of a network device and a terminal device as an example, for example, the information used to determine the first condition may indicate that the model used by the terminal device was transmitted to the terminal device by the network device, or that the model used by the terminal device was obtained based on training data transmitted by the network device. For another example, the information used to determine the first condition may indicate that the model used by the terminal device will be transmitted to the terminal device by the network device, but the terminal device has not yet obtained the model transmitted by the network device. For another example, the information used to determine the first condition may indicate that the model used by the terminal device was generated by the terminal device itself. For another example, the information used to determine the first condition may indicate that the terminal device uses a predefined model. For another example, the information used to determine the first condition may indicate that the model used by the network device was transmitted to the terminal device, or that the model used by the network device was trained using data transmitted by the terminal device. For another example, the information used to determine the first condition may indicate that the model used by the network device will be transmitted to the terminal device by the terminal device, but the network device has not yet obtained the model transmitted by the terminal device. For another example, the information used to determine the first condition may indicate that the model used by the network device was generated by the network device itself. For another example, the information used to determine the first condition may indicate that the network device uses a model predefined by a protocol.

[0176] Taking the example of the first information including information indicating an auxiliary test model, in some implementations, the information indicating the auxiliary test model may include information indicating the auxiliary test model (e.g., a model ID). In other implementations, the information indicating the auxiliary test model may include a model type of the auxiliary test model.

[0177] Taking the scenario of a network device and a terminal device as an example, for example, the information for indicating the auxiliary test model may instruct the test device to use the model provided by the terminal device as the auxiliary test model. For another example, the information for indicating the auxiliary test model may instruct the test device to use the model provided by the network device as the auxiliary test model. For another example, the information for indicating the auxiliary test model may instruct the test device to use a predefined model as the auxiliary test model. For another example, the information for indicating the auxiliary test model may instruct the test device to use the model provided by the terminal device as the auxiliary test model. For another example, the information for indicating the auxiliary test model may instruct the test device to use the model provided by the network device as the auxiliary test model. For another example, the information for indicating the auxiliary test model may instruct the test device to use a reference model as the auxiliary test model.

[0178] Taking the example of the first information including information indicating the first model, in some implementations, the information indicating the first model may include information indicating the first model (e.g., a model ID). In other implementations, the information indicating the first model may include a model type of the first model.

[0179] Taking the scenario of a network device and a terminal device as an example, for example, the information used to indicate the first model may indicate that the model to be tested is generated by the terminal device. For another example, the information used to indicate the first model may indicate that the model to be tested is generated by the network device. For another example, the information used to indicate the first model may indicate that it is obtained by transmission from the network device. For another example, the information used to indicate the first model may indicate that the model to be tested is a reference model defined by a protocol. For another example, the information used to indicate the first model may indicate that the model to be tested is generated by the terminal device. For another example, the information used to indicate the first model may indicate that it is obtained by transmission from the terminal device. For another example, the information used to indicate the first model may indicate that the model to be tested is generated by the network device. For another example, the information used to indicate the first model may indicate that the model to be tested is a predefined model.

[0180] In an embodiment of the present application, the device under test can indicate first condition information to the testing device, thereby enabling the testing device to determine an auxiliary test model based on the first condition information. Alternatively, the device under test can indicate an auxiliary test model to be matched to the testing device, thereby enabling the testing device to determine the auxiliary test model. Furthermore, the device under test can indicate a model to be tested to the testing device, thereby enabling the testing device to determine an auxiliary test model for the model under test.

[0181] In some implementations, if the target device is a test device, the method further includes: the test device sending second information to the device under test (see step S610). In this embodiment of the present application, the transmission of the second information can be used in conjunction with the selection of the auxiliary test model described above. In this case, step S610 can be performed after step S310.

[0182] In some implementations, the second information includes one or more of the following: information indicating a method for obtaining the first model; information indicating an auxiliary test model; and information indicating the first model.

[0183] Taking the example where the second information includes information for indicating a method for obtaining the first model, the method for obtaining the first model may include being provided by a device for training the first model; and / or being provided by a device for deploying the first model.

[0184] Taking the example of the second information including information indicating an auxiliary test model, in some implementations, the information indicating the auxiliary test model may include information indicating the auxiliary test model (e.g., a model ID). In other implementations, the information indicating the auxiliary test model may include a model type of the auxiliary test model.

[0185] Taking the scenario of a network device and a terminal device as an example, for example, the information for indicating the auxiliary test model may instruct the test device to use the model provided by the terminal device as the auxiliary test model. For another example, the information for indicating the auxiliary test model may instruct the test device to use the model provided by the network device as the auxiliary test model. For another example, the information for indicating the auxiliary test model may instruct the test device to use a predefined model as the auxiliary test model. For another example, the information for indicating the auxiliary test model may instruct the test device to use the model provided by the terminal device as the auxiliary test model. For another example, the information for indicating the auxiliary test model may instruct the test device to use the model provided by the network device as the auxiliary test model. For another example, the information for indicating the auxiliary test model may instruct the test device to use a reference model as the auxiliary test model.

[0186] Taking the example of the second information including information indicating the first model, in some implementations, the information indicating the first model may include information indicating the first model (e.g., a model ID). In other implementations, the information indicating the first model may include a model type of the first model.

[0187] Taking the scenario of a network device and a terminal device as an example, for example, the information used to indicate the first model may indicate that the model to be tested is generated by the terminal device. For another example, the information used to indicate the first model may indicate that the model to be tested is generated by the network device. For another example, the information used to indicate the first model may indicate that it is obtained by transmission from the network device. For another example, the information used to indicate the first model may indicate that the model to be tested is a reference model defined by a protocol. For another example, the information used to indicate the first model may indicate that the model to be tested is generated by the terminal device. For another example, the information used to indicate the first model may indicate that it is obtained by transmission from the terminal device. For another example, the information used to indicate the first model may indicate that the model to be tested is generated by the network device. For another example, the information used to indicate the first model may indicate that the model to be tested is a predefined model.

[0188] As described above, the model providing device can be a model training device, or the model providing device can be a model deployment device. Accordingly, in some scenarios, the training device used to train the model and the deployment device used to deploy the model can be different devices, wherein the model can be one or more models mentioned above. For example, the training device can be a network device, and accordingly, the deployment device can be a terminal device. For another example, the training device can be a terminal device, and accordingly, the deployment device can be a network device. For another example, the training device can be a first terminal device, and accordingly, the deployment device can be a second terminal device. For another example, the training device can be a first network device, and accordingly, the deployment device can be a second network device.

[0189] In other scenarios, the training device used to train the model and the deployment device used to deploy the model can be the same device, where the model can be one or more of the models mentioned above. For example, the training device and the deployment device can both be network devices. In another example, the training device and the deployment device can both be terminal devices.

[0190] In the embodiments of the present application, there is no limitation on the devices on which the first model and the peer model of the first model are deployed. For example, the first model can be deployed on a network device, and accordingly, the peer model of the first model can be deployed on a terminal device. For another example, the peer model of the first model can be deployed on a network device, and accordingly, the first model can be deployed on a terminal device. For another example, the peer model of the first model can be deployed on a first terminal device, and accordingly, the first model can be deployed on a second terminal device. For another example, the peer model of the first model can be deployed on a first network device, and accordingly, the first model can be deployed on a second network device.

[0191] The method embodiment of the present application is described in detail above in conjunction with Figures 1 to 6 , and the device embodiment of the present application is described in detail below in conjunction with Figures 7 to 8 . It should be understood that the description of the method embodiment corresponds to the description of the device embodiment, and therefore, for portions not described in detail, reference can be made to the above method embodiment.

[0192] FIG7 is a device for model testing according to an embodiment of the present application. The device 700 shown in FIG7 includes: a processing unit 710 .

[0193] Processing unit 710 is used to determine an auxiliary test model for testing the first model, wherein the auxiliary test model is determined based on a first condition, and the first condition is associated with one or more of the following: a training method of the first model; a method for obtaining the first model; a training method of the peer model of the first model; a method for obtaining the peer model of the first model; whether the first model is available; whether the peer model of the first model is available; whether the first model is successfully deployed; whether the peer model of the first model is successfully deployed; whether the reference model is successfully deployed; whether the peer model of the reference model is successfully deployed; whether the test device supports the peer model of the first model.

[0194] In some implementations, if the first condition is associated with a training method, the target model associated with the training method includes the first model and / or the peer model of the first model, and the training method includes one or more of joint training, single-end training, and sequential training.

[0195] In some implementations, if the first condition is associated with an acquisition method, the target model associated with the acquisition method includes the first model and / or the peer model of the first model, and the acquisition method includes: the target model is transmitted by the target model providing device to the device where the target model is deployed; and the target model is trained by the device where the target model is deployed.

[0196] In some implementations, if the acquisition method includes the target model being trained by a device that deploys the target model, the training data of the target model is acquired from a providing device of the target model, and the providing device is used to train the first model and the peer model of the first model.

[0197] In some implementations, if the first condition is associated with whether the first model is successfully deployed, whether the first model is successfully deployed is associated with one or more of the following: whether the first model is successfully transmitted; whether the first model is successfully indicated; whether the training data of the first model is successfully transmitted; whether the training data of the first model is successfully indicated; and / or

[0198] If the first condition is associated with whether the peer model of the first model is successfully deployed, whether the peer model of the first model is successfully deployed is associated with one or more of the following: whether the peer model of the first model is successfully transmitted; whether the peer model of the first model is successfully indicated; whether the training data of the peer model of the first model is successfully transmitted; whether the training data of the peer model of the first model is successfully indicated; and / or

[0199] If the first condition is associated with whether the reference model is successfully deployed, whether the reference model is successfully deployed is associated with one or more of the following: whether the reference model is successfully transmitted; whether the reference model is successfully indicated; whether the training data of the reference model is successfully transmitted; whether the training data of the reference model is successfully indicated; and / or

[0200] If the first condition is associated with whether the peer model of the reference model is successfully deployed, whether the peer model of the reference model is successfully deployed is associated with one or more of the following: whether the peer model of the reference model is successfully transmitted; whether the peer model of the reference model is successfully indicated; whether the training data of the peer model of the reference model is successfully transmitted; whether the training data of the peer model of the reference model is successfully indicated.

[0201] In some implementations, if the first condition includes that the first model and the peer model of the first model are trained by a first device, and the peer model of the first model is deployed by the first device for a second device, then the auxiliary test model is the peer model of the first model; or

[0202] If the first condition includes that the first model and the peer model of the first model are trained by a first device, and the first model is deployed by the first device to a second device, then the auxiliary test model is the peer model of the first model.

[0203] In some implementations, the peer model of the first model is indicated by the first device to the testing device of the first model, or the peer model of the first model is transmitted by the first device to the testing device of the first model.

[0204] In some implementations, if the first condition includes that the first model and the reference model are trained by a first device, the reference model can be deployed by the first device for a second device, and the second device does not deploy the reference model, then the auxiliary test model is the reference model;

[0205] If the first condition includes that the first model and the reference model are trained by a first device, and the first model can be deployed by the first device for a second device, and the second device has not deployed the first model, then the auxiliary test model is the reference model.

[0206] In some implementations, the reference model is indicated by the first device to a test device for the first model, or the reference model is transmitted by the first device to a test device for the first model.

[0207] In some implementations, if the first condition is associated with whether the test device of the first model supports the peer model of the first model, and the first condition includes that the test device does not support the peer model of the first model, then the auxiliary test model is a predefined model associated with the first model test.

[0208] In some implementations, if the first condition includes that the second model and the peer model of the second model are trained by the first device, and the second model is not deployed to the second device, then the model to be tested is the first model deployed in the second device, and the auxiliary test model is a predefined model associated with the test of the first model.

[0209] In some implementations, the target device is a test device, or the target device is a device on which the first model is deployed.

[0210] In some implementations, if the target device is a test device, the apparatus further includes: a first receiving unit for receiving first information sent by a device deploying the first model, the first information including one or more of the following: information for determining the first condition; information for indicating the auxiliary test model; information for indicating the first model.

[0211] In some implementations, if the target device is a test device, the apparatus further includes: a first sending unit for sending second information to the device on which the first model is deployed, the second information including information for indicating one or more of the following: information for indicating a method for obtaining the first model; information for the auxiliary test model; information for indicating the first model.

[0212] In some implementations, if the target device is a device on which the first model is deployed, the apparatus further includes: a second sending unit for sending first information to the test device, the first information including one or more of the following: information for determining the first condition; information for indicating the auxiliary test model; information for indicating the first model.

[0213] In some implementations, if the target device is a device on which the first model is deployed, the apparatus further includes: a second receiving unit for receiving second information sent by the test device, the second information including information for indicating one or more of the following: information for indicating a method for obtaining the first model; information for the auxiliary test model; information for indicating the first model.

[0214] In an optional embodiment, the processing unit 710 may be a processor 810. The apparatus 700 for model testing may further include a transceiver 830 and a memory 820, as specifically shown in FIG8 .

[0215] FIG8 is a schematic block diagram of an apparatus according to an embodiment of the present application. The dashed lines in FIG8 indicate that the unit or module is optional. Apparatus 800 may be used to implement the method described in the above method embodiment. Apparatus 800 may be a chip, a terminal device, or a network device.

[0216] The device 800 may include one or more processors 810. The processor 810 may support the device 800 to implement the method described in the method embodiment above. The processor 810 may be a general-purpose processor or a special-purpose processor. For example, the processor may be a central processing unit (CPU). Alternatively, the processor may be another general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc. The general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0217] The apparatus 800 may further include one or more memories 820. The memories 820 store programs that can be executed by the processor 810, causing the processor 810 to perform the methods described in the above method embodiments. The memories 820 may be independent of the processor 810 or integrated into the processor 810.

[0218] The apparatus 800 may further include a transceiver 830. The processor 810 may communicate with other devices or chips via the transceiver 830. For example, the processor 810 may transmit and receive data with other devices or chips via the transceiver 830.

[0219] The present application also provides a computer-readable storage medium for storing a program. The computer-readable storage medium can be applied to a terminal or network device provided in the present application, and the program enables a computer to execute the method performed by the terminal or network device in each embodiment of the present application.

[0220] The present application also provides a computer program product. The computer program product includes a program. The computer program product can be applied to a terminal or network device provided in the present application, and the program causes a computer to execute the method performed by the terminal or network device in each embodiment of the present application.

[0221] The embodiments of the present application also provide a computer program. The computer program can be applied to the terminal or network device provided in the embodiments of the present application, and the computer program enables a computer to execute the method performed by the terminal or network device in each embodiment of the present application.

[0222] It should be understood that the terms "system" and "network" in this application can be used interchangeably. In addition, the terms used in this application are only used to explain the specific embodiments of this application and are not intended to limit this application. The terms "first", "second", "third", and "fourth" in the specification and claims of this application and the accompanying drawings are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0223] In the embodiments of this application, the term "indication" may refer to a direct indication, an indirect indication, or an indication of an association. For example, "A indicates B" may refer to a direct indication of B, e.g., B can obtain information through A; it may refer to an indirect indication of B, e.g., A indicates C, e.g., B can obtain information through C; or it may refer to an association between A and B.

[0224] In the embodiment of the present application, "B corresponding to A" means that B is associated with A and B can be determined based on A. However, it should be understood that determining B based on A does not mean determining B based solely on A, but B can also be determined based on A and / or other information.

[0225] In the embodiments of the present application, the term "corresponding" may indicate a direct or indirect correspondence between the two, or an association relationship between the two, or a relationship between indication and indication, configuration and configuration, etc.

[0226] In the embodiments of the present application, "pre-definition" or "pre-configuration" may be implemented by pre-storing corresponding codes, tables, or other methods that can be used to indicate relevant information in a device (e.g., a terminal device and a network device). The present application does not limit the specific implementation method. For example, pre-definition may refer to information defined in a protocol.

[0227] In the embodiments of the present application, the “protocol” may refer to a standard protocol in the communications field, for example, it may include an LTE protocol, an NR protocol, and related protocols used in future communication systems, and the present application does not limit this.

[0228] In the embodiments of this application, the term "and / or" is simply a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0229] In various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0230] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0231] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0232] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0233] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be read by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0234] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for model testing, characterized in that: include: The target device determines an auxiliary test model for testing the first model, wherein the auxiliary test model is determined based on a first condition, and the first condition is associated with one or more of the following: a training method of the first model; a method for obtaining the first model; a training method of a peer model of the first model; A method for obtaining a peer model of the first model; whether the first model is available; Whether the peer model of the first model is available; Whether the first model is successfully deployed; Whether the peer model of the first model is successfully deployed; Whether the reference model was successfully deployed; Whether the peer model of the reference model is successfully deployed; The test device supports the peer model of the first model.

2. The method according to claim 1, characterized in that If the first condition is associated with a training method, the target model associated with the training method includes the first model and / or the counterpart model of the first model, and the training method includes one or more of joint training, single-end training, and sequential training.

3. The method according to claim 1 or 2, characterized in that If the first condition is associated with an acquisition method, the target model associated with the acquisition method includes the first model and / or a peer model of the first model, and the acquisition method includes: The target model is transmitted from a device providing the target model to a device deploying the target model; The target model is trained by a device on which the target model is deployed.

4. The method according to claim 3, characterized in that If the acquisition method includes training the target model by a device that deploys the target model, the training data of the target model is acquired from a providing device of the target model, and the providing device is used to train the first model and a peer model of the first model.

5. The method according to any one of claims 1 to 4, characterized in that: If the first condition is associated with whether the first model is successfully deployed, whether the first model is successfully deployed is associated with one or more of the following: whether the first model is successfully transmitted; whether the first model is successfully indicated; whether the training data of the first model is successfully transmitted; whether the training data of the first model is successfully indicated; and / or If the first condition is associated with whether the peer model of the first model is successfully deployed, whether the peer model of the first model is successfully deployed is associated with one or more of the following: whether the peer model of the first model is successfully transmitted; whether the peer model of the first model is successfully indicated; whether the training data of the peer model of the first model is successfully transmitted; Whether the training data of the peer model of the first model is successfully indicated; and / or If the first condition is associated with whether the reference model is successfully deployed, whether the reference model is successfully deployed is associated with one or more of the following: whether the reference model is successfully transmitted; whether the reference model is successfully indicated; whether the training data of the reference model is successfully transmitted; whether the training data of the reference model is successfully indicated; and / or If the first condition is associated with whether the peer model of the reference model is successfully deployed, whether the peer model of the reference model is successfully deployed is associated with one or more of the following: whether the peer model of the reference model is successfully transmitted; Whether the peer model of the reference model is successfully indicated; Whether the training data of the peer model of the reference model is successfully transmitted; Whether the training data of the peer model of the reference model is successfully indicated.

6. The method according to any one of claims 1 to 5, characterized in that: If the first condition includes that the first model and the peer model of the first model are trained by a first device, and the peer model of the first model is deployed by the first device for a second device, then the auxiliary test model is the peer model of the first model; or If the first condition includes that the first model and the peer model of the first model are trained by a first device, and the first model is deployed by the first device to a second device, then the auxiliary test model is the peer model of the first model.

7. The method according to claim 6, characterized in that The peer model of the first model is indicated by the first device to the test device of the first model, or the peer model of the first model is transmitted by the first device to the test device of the first model.

8. The method according to any one of claims 1 to 5, characterized in that: If the first condition includes that the first model and the reference model are trained by a first device, the reference model can be deployed by the first device for a second device, and the second device does not deploy the reference model, then the auxiliary test model is the reference model; If the first condition includes that the first model and the reference model are trained by a first device, and the first model can be trained by the The first device is deployed by the second device, and the second device is not deployed with the first model, then the auxiliary test model is the reference model.

9. The method according to claim 8, characterized in that The reference model is indicated by the first device to a test device of the first model, or the reference model is transmitted by the first device to a test device of the first model.

10. The method according to any one of claims 1 to 5, characterized in that If the first condition is associated with whether the test device of the first model supports the peer model of the first model, and the first condition includes that the test device does not support the peer model of the first model, then the auxiliary test model is a predefined model associated with the first model test.

11. The method according to any one of claims 1 to 5, characterized in that If the first condition includes that the second model and the peer model of the second model are trained by the first device, and the second model is not deployed to the second device, then the model to be tested is the first model deployed in the second device, and the auxiliary test model is a predefined model associated with the first model test.

12. The method according to any one of claims 1 to 11, characterized in that The target device is a test device, or the target device is a device on which the first model is deployed.

13. The method according to claim 12, characterized in that If the target device is a test device, the method further includes: The test device receives first information sent by a device deploying the first model, where the first information includes one or more of the following: information for determining the first condition; Information for indicating the auxiliary test model; Information indicating the first model.

14. The method according to claim 12, characterized in that If the target device is a test device, the method further includes: The test device sends second information to the device where the first model is deployed, where the second information includes information indicating one or more of the following: Used to indicate a method for obtaining the first model; Information of the auxiliary test model; Information indicating the first model.

15. The method according to claim 12, characterized in that If the target device is a device on which the first model is deployed, the method further includes: The target device sends first information to the test device, where the first information includes one or more of the following: information for determining the first condition; Information for indicating the auxiliary test model; Information indicating the first model.

16. The method according to claim 12, characterized in that If the target device is a device on which the first model is deployed, the method further includes: The target device receives second information sent by the test device, where the second information includes information indicating one or more of the following: Used to indicate a method for obtaining the first model; Information of the auxiliary test model; Information indicating the first model.

17. A device for model testing, characterized in that: include: A processing unit is configured to determine an auxiliary test model for testing the first model, wherein the auxiliary test model is determined based on a first condition, and the first condition is associated with one or more of the following: a training method of the first model; a method for obtaining the first model; a training method of a peer model of the first model; A method for obtaining a peer model of the first model; whether the first model is available; Whether the peer model of the first model is available; Whether the first model is successfully deployed; Whether the peer model of the first model is successfully deployed; Whether the reference model was successfully deployed; Whether the peer model of the reference model is successfully deployed; The test device supports the peer model of the first model.

18. The device according to claim 17, characterized in that If the first condition is associated with a training method, the target model associated with the training method includes the first model and / or the counterpart model of the first model, and the training method includes one or more of joint training, single-end training, and sequential training.

19. The device according to claim 17 or 18, characterized in that If the first condition is associated with an acquisition method, the target model associated with the acquisition method includes the first model and / or a peer model of the first model, and the acquisition method includes: The target model is transmitted from a device providing the target model to a device deploying the target model; The target model is trained by a device on which the target model is deployed.

20. The device according to claim 19, characterized in that If the acquisition method includes training the target model by a device that deploys the target model, the training data of the target model is acquired from a providing device of the target model, and the providing device is used to train the first model and a peer model of the first model.

21. The device according to any one of claims 17 to 20, characterized in that: If the first condition is associated with whether the first model is successfully deployed, whether the first model is successfully deployed is associated with one or more of the following: whether the first model is successfully transmitted; whether the first model is successfully indicated; whether the training data of the first model is successfully transmitted; whether the training data of the first model is successfully indicated; and / or If the first condition is associated with whether the peer model of the first model is successfully deployed, whether the peer model of the first model is successfully deployed is associated with one or more of the following: whether the peer model of the first model is successfully transmitted; whether the peer model of the first model is successfully indicated; whether the training data of the peer model of the first model is successfully transmitted; Whether the training data of the peer model of the first model is successfully indicated; and / or If the first condition is associated with whether the reference model is successfully deployed, whether the reference model is successfully deployed is associated with one or more of the following: whether the reference model is successfully transmitted; whether the reference model is successfully indicated; whether the training data of the reference model is successfully transmitted; whether the training data of the reference model is successfully indicated; and / or If the first condition is associated with whether the peer model of the reference model is successfully deployed, whether the peer model of the reference model is successfully deployed is associated with one or more of the following: whether the peer model of the reference model is successfully transmitted; Whether the peer model of the reference model is successfully indicated; Whether the training data of the peer model of the reference model is successfully transmitted; Whether the training data of the peer model of the reference model is successfully indicated.

22. The device according to any one of claims 17 to 21, characterized in that: If the first condition includes that the first model and the peer model of the first model are trained by a first device, and the peer model of the first model is deployed by the first device for a second device, then the auxiliary test model is the peer model of the first model; or If the first condition includes that the first model and the peer model of the first model are trained by a first device, and the first model is deployed by the first device to a second device, then the auxiliary test model is the peer model of the first model.

23. The device according to claim 22, characterized in that The peer model of the first model is indicated by the first device to the test device of the first model, or the peer model of the first model is transmitted by the first device to the test device of the first model.

24. The device according to any one of claims 17 to 21, characterized in that: If the first condition includes that the first model and the reference model are trained by a first device, the reference model can be deployed by the first device for a second device, and the second device does not deploy the reference model, then the auxiliary test model is the reference model; If the first condition includes that the first model and the reference model are trained by a first device, and the first model can be deployed by the first device for a second device, and the second device has not deployed the first model, then the auxiliary test model is the reference model.

25. The device according to claim 24, characterized in that The reference model is indicated by the first device to a test device of the first model, or the reference model is transmitted by the first device to a test device of the first model.

26. The device according to any one of claims 17 to 21, characterized in that If the first condition is associated with whether the test device of the first model supports the peer model of the first model, and the first condition includes that the test device does not support the peer model of the first model, then the auxiliary test model is a predefined model associated with the first model test.

27. The device according to any one of claims 17 to 21, characterized in that If the first condition includes that the second model and the peer model of the second model are trained by the first device, and the second model is not deployed to the second device, then the model to be tested is the first model deployed in the second device, and the auxiliary test model is a predefined model associated with the first model test.

28. The device according to any one of claims 17 to 27, characterized in that The target device is a test device, or the target device is a device on which the first model is deployed.

29. The device according to claim 28, characterized in that If the target device is a test device, the apparatus further includes: A first receiving unit is configured to receive first information sent by a device that deploys the first model, where the first information includes one or more of the following: information for determining the first condition; Information for indicating the auxiliary test model; Information indicating the first model.

30. The device according to claim 28, characterized in that If the target device is a test device, the apparatus further includes: A first sending unit is configured to send second information to a device where the first model is deployed, where the second information includes information indicating one or more of the following: Used to indicate a method for obtaining the first model; Information of the auxiliary test model; Information indicating the first model.

31. The device according to claim 28, characterized in that If the target device is a device that deploys the first model, the apparatus further includes: The second sending unit is configured to send first information to the test device, where the first information includes one or more of the following: information for determining the first condition; Information for indicating the auxiliary test model; Information indicating the first model.

32. The device according to claim 28, characterized in that If the target device is a device that deploys the first model, the apparatus further includes: A second receiving unit is configured to receive second information sent by the test device, wherein the second information includes information indicating one or more of the following: Used to indicate a method for obtaining the first model; Information of the auxiliary test model; Information indicating the first model.

33. A device for model testing, characterized in that: It comprises a transceiver, a memory and a processor, wherein the memory is used to store programs, and the processor is used to call the programs in the memory and control the transceiver to receive or send signals so that the terminal device executes the method as described in any one of claims 1 to 16.

34. A device, characterized in that The device comprises a processor, configured to call a program from a memory so as to enable the device to execute the method according to any one of claims 1 to 16.

35. A chip, characterized in that: It comprises a processor, which is used to call a program from a memory, so that a device equipped with the chip executes the method according to any one of claims 1 to 16.

36. A computer-readable storage medium, characterized in that: A program is stored thereon, the program causing a computer to execute the method according to any one of claims 1 to 16.

37. A computer program product, characterized in that The method comprises a program which causes a computer to execute the method according to any one of claims 1 to 16.

38. A computer program, characterized in that The computer program enables a computer to execute the method according to any one of claims 1 to 16.

Images