Information determination method and apparatus, and terminal, network-side device, and medium

Abstract

The present application belongs to the technical field of communications. Disclosed are an information determination method and apparatus, and a terminal, a network-side device, and a medium. The information determination method in the embodiments of the present application comprises: a first device receiving first intermediate data from a second device; and the first device determining, on the basis of the first intermediate data, performance information of a VFL model, wherein the first intermediate data is intermediate data corresponding to validation data, and the validation data is used for validating the VFL model.

Description

Information determination methods, devices, terminals, network-side equipment and media

[0001] Cross-reference to related applications

[0002] This application claims priority to Chinese Patent Application No. 202411584995.0, filed in China on November 7, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application belongs to the field of communication technology, specifically relating to an information determination method, apparatus, terminal, network-side equipment, and medium. Background Technology

[0004] Currently, during model training, the model periodically calculates performance information, such as accuracy, which is the percentage of correct predictions out of the total number of predictions. Specifically, by comparing the predicted values ​​generated by the model with the labels of that training iteration, the correctness of the training can be determined, thus calculating the model's performance information. This performance information allows for an objective evaluation of the model's training performance, enabling continuous adjustment of model parameters to improve training effectiveness.

[0005] Vertical Federated Learning (VFL) is a model training method that essentially combines features. It is suitable for scenarios with high user overlap but low feature overlap. By combining the different data features of common samples from participating parties, VFL can increase the feature dimensionality of the training samples, thereby training a better model.

[0006] However, there is currently no solution for how the VFL server can determine the performance information of the VFL model. This makes it impossible to objectively evaluate the model's training performance, resulting in poor training results for the VFL model. Summary of the Invention

[0007] This application provides an information determination method, apparatus, terminal, network-side device, and medium that can determine the performance information of a VFL model, thereby objectively evaluating the model's training performance and improving the training effect of the VFL model.

[0008] In a first aspect, an information determination method is provided, executed by a first device, the method comprising: the first device receiving first intermediate data from a second device; the first device determining performance information of a VFL model based on the first intermediate data; wherein the first intermediate data is intermediate data corresponding to verification data, and the verification data is used to verify the VFL model.

[0009] Secondly, an information determination method is provided, executed by a second device, the method comprising: the second device sending first intermediate data to a first device; wherein the first intermediate data is used to determine the performance information of the VFL model, the first intermediate data is intermediate data corresponding to the verification data, and the verification data is used to verify the VFL model.

[0010] Thirdly, an information determination device is provided, the device including a first receiving module and a first processing module; the first receiving module is used to receive first intermediate data from a second device; the first processing module is used to determine the performance information of a VFL model based on the first intermediate data received by the first receiving module; wherein the first intermediate data is intermediate data corresponding to the verification data, and the verification data is used to verify the VFL model.

[0011] Fourthly, an information determination device is provided, the device including a second sending module; the second sending module is used to send first intermediate data to a first device; wherein the first intermediate data is used to determine the performance information of the VFL model, the first intermediate data is intermediate data corresponding to the verification data, and the verification data is used to verify the VFL model.

[0012] Fifthly, an information determining apparatus is provided, the apparatus being configured to perform the steps of the method described in the first aspect, or to implement the steps of the method described in the second aspect.

[0013] In a sixth aspect, a terminal is provided, the terminal including a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the method as described in the first aspect, or implementing the steps of the method as described in the second aspect.

[0014] In a seventh aspect, a terminal is provided, including a processor and a communication interface, wherein the communication interface is used to receive first intermediate data from a second device; the processor is used to determine performance information of a VFL model based on the first intermediate data received by a first receiving module; wherein the first intermediate data is intermediate data corresponding to verification data, and the verification data is used to verify the VFL model.

[0015] Alternatively, a communication interface is used to send first intermediate data to a first device; wherein the first intermediate data is used to determine the performance information of the VFL model, the first intermediate data is intermediate data corresponding to the verification data, and the verification data is used to verify the VFL model.

[0016] Eighthly, a network-side device is provided, the network-side device including a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the method as described in the second aspect, or implementing the steps of the method as described in the second aspect.

[0017] In a ninth aspect, a network-side device is provided, including a processor and a communication interface, wherein the communication interface is used to receive first intermediate data from a second device; the processor is used to determine performance information of a VFL model based on the first intermediate data received by a first receiving module; wherein the first intermediate data is intermediate data corresponding to verification data, and the verification data is used to verify the VFL model.

[0018] Alternatively, a communication interface is used to send first intermediate data to a first device; wherein the first intermediate data is used to determine the performance information of the VFL model, the first intermediate data is intermediate data corresponding to the verification data, and the verification data is used to verify the VFL model.

[0019] In a tenth aspect, a readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect, or implement the steps of the method described in the second aspect.

[0020] Eleventhly, a wireless communication system is provided, comprising: a terminal and a network-side device, wherein the terminal can be used to perform the steps of the method described in the first aspect, and the network-side device can be used to perform the steps of the method described in the second aspect; or, the terminal can be used to perform the steps of the method described in the second aspect, and the network-side device can be used to perform the steps of the method described in the first aspect.

[0021] In a twelfth aspect, a chip is provided, the chip including a processor and a communication interface coupled to the processor, the processor being configured to run programs or instructions to implement the method as described in the first aspect, or to implement the method as described in the second aspect.

[0022] In a thirteenth aspect, a computer program / program product is provided, which is stored in a storage medium and is executed by at least one processor to implement the steps of the method as described in the first aspect, or to implement the steps of the method as described in the second aspect.

[0023] In this embodiment, a first device receives first intermediate data from a second device; the first device determines the performance information of the VFL model based on the first intermediate data; wherein, the first intermediate data is the intermediate data corresponding to the validation data, and the validation data is used to validate the VFL model. Through this scheme, the first device can first receive the first intermediate data from the second device, i.e., the intermediate data corresponding to the validation data used to validate the VFL model, and then determine the performance information of the VFL model based on the first intermediate data. This performance information allows for an objective evaluation of the model's training performance, thereby improving the training effect of the VFL model. Attached Figure Description

[0024] Figure 1 is a block diagram of a wireless communication system applicable to an embodiment of this application;

[0025] Figure 2 is a flowchart of an information determination method provided in an embodiment of this application;

[0026] Figure 3 is one of the schematic diagrams of the signaling flow in an information determination method provided in an embodiment of this application;

[0027] Figure 4 is a second schematic diagram of the signaling flow in an information determination method provided in an embodiment of this application;

[0028] Figure 5 is a flowchart of another information determination method provided in an embodiment of this application;

[0029] Figure 6 is a schematic diagram of an information determination device provided in an embodiment of this application;

[0030] Figure 7 is a schematic diagram of another information determination device provided in an embodiment of this application;

[0031] Figure 8 is a schematic diagram of the communication device provided in an embodiment of this application;

[0032] Figure 9 is a schematic diagram of the hardware structure of the terminal provided in an embodiment of this application;

[0033] Figure 10 is a schematic diagram of the hardware structure of the network-side device provided in an embodiment of this application. Detailed Implementation

[0034] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0035] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, not limited in number; for example, the first object can be one or more. Furthermore, "or" in this application indicates at least one of the connected objects. For example, the scope of protection for "A or B" covers at least three scenarios: Scenario 1: including A but not B; Scenario 2: including B but not A; Scenario 3: including both A and B. In addition, the terms "A and / or B," "at least one of A and B," and "at least one of A or B" also cover at least the above three scenarios. The character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0036] The term "instruction" in this application can be either a direct instruction (or explicit instruction) or an indirect instruction (or implicit instruction). A direct instruction can be understood as one in which the sender explicitly informs the receiver of specific information, the operation to be performed, or the requested result, etc., in the instruction sent. An indirect instruction can be understood as one in which the receiver determines the corresponding information based on the instruction sent by the sender, or makes a judgment and determines the operation to be performed or the requested result, etc., based on the judgment result.

[0037] It is worth noting that the technologies described in this application are not limited to Long Term Evolution (LTE) / LTE-Advanced (LTE-A) systems, but can also be used in other wireless communication systems, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal Frequency Division Multiple Access (OFDMA), Single-carrier Frequency-Division Multiple Access (SC-FDMA), or other systems. The terms "system" and "network" in this application are often used interchangeably, and the described technologies can be used with the systems and radio technologies mentioned above, as well as with other systems and radio technologies. The following description describes New Radio (NR) systems for illustrative purposes, and the term NR is used in most of the following description; however, these technologies can also be applied to systems other than NR systems, such as 6th generation (6G) radio systems. th Generation 6G communication system.

[0038] Figure 1 shows a block diagram of a wireless communication system applicable to an embodiment of this application. The wireless communication system includes a terminal 11 and a network-side device 12. The terminal 11 can be a mobile phone, tablet computer, laptop computer, notebook computer, personal digital assistant (PDA), handheld computer, netbook, ultra-mobile personal computer (UMPC), mobile internet device (MID), augmented reality (AR), virtual reality (VR) device, robot, wearable device, flight vehicle, vehicle user equipment (VUE), shipboard equipment, pedestrian user equipment (PUE), smart home (home devices with wireless communication capabilities, such as refrigerators, televisions, washing machines, or furniture), game console, personal computer (PC), ATM, or self-service machine, etc. Wearable devices include: smartwatches, smart bracelets, smart headphones, smart glasses, smart jewelry (smart bracelets, smart chains, smart rings, smart necklaces, smart anklets, smart anklets, etc.), smart wristbands, smart clothing, etc. Among these, in-vehicle devices can also be referred to as in-vehicle terminals, in-vehicle controllers, in-vehicle modules, in-vehicle components, in-vehicle chips, or in-vehicle units, etc. It should be noted that the specific type of terminal 11 is not limited in this application embodiment. Network-side equipment 12 may include access network equipment or core network equipment, wherein access network equipment may also be referred to as Radio Access Network (RAN) equipment, radio access network function, or radio access network unit. Access network equipment may include base stations, Wireless Local Area Network (WLAN) access points (AS), or Wireless Fidelity (WiFi) nodes, etc.The term "base station" can be referred to as Node B (NB), Evolved Node B (eNB), Next Generation Node B (gNB), New Radio Node B (NR Node B), Access Point, Relay Base Station (RBS), Serving Base Station (SBS), Base Transceiver Station (BTS), Radio Base Station, Radio Transceiver, Basic Service Set (BSS), Extended Service Set (ESS), Home Node B (HNB), Home Evolved Node B, Transmit / Receive Point (TRP), or any other suitable term in the relevant field, as long as the same technical effect is achieved. The term "base station" is not limited to any specific technical terminology. It should be noted that this application embodiment only uses a base station in an NR system as an example for description and does not limit the specific type of base station.

[0039] Core network equipment, also known as core network nodes, core network functions, or core network elements, includes, but is not limited to, at least one of the following: Mobility Management Entity (MME), Access and Mobility Management Function (AMF), Session Management Function (SMF), User Plane Function (UPF), Policy Control Function (PCF), Policy and Charging Rules Function (PCRF), Edge Application Server Discovery Function (EASDF), Unified Data Management (UDM), Unified Data Repository (UDR), Home Subscriber Server (HSS), Centralized network configuration (CNC), Network Repository Function (NRF), Network Exposure Function (NEF), Local NEF (or L-NEF), and Binding Support. The core network functions include: BSF (Block Network Function), Application Function (AF), Location Management Function (LMF), Gateway Mobile Location Centre (GMLC), and Network Data Analytics Function (NWDAF). It should be noted that this application embodiment only uses core network equipment in the NR system as an example and does not limit the specific type of core network equipment. If the name of the core network equipment mentioned in this application embodiment changes in subsequent protocol versions (e.g., 6G), it will still be within the scope of protection of this application.

[0040] Optionally, the core network equipment can be implemented by one or more functional modules in a single device, or by multiple devices working together; this application does not specifically limit this. It is understood that the aforementioned functional modules can be network elements in hardware devices, software functional modules running on dedicated hardware, or virtualized functional modules instantiated on a platform (e.g., a cloud platform).

[0041] The following explanations and descriptions cover some concepts and / or terms involved in the information determination methods, apparatus, terminals, network-side devices, and media provided in the embodiments of this application.

[0042] VFL (Vertical Federation): Essentially, it's a feature federation approach suitable for scenarios with significant user overlap but minimal feature overlap, such as different services offered to the same user by the CN and RAN domains within a communication network. By combining the different data features of common samples from participating parties, vertical federation increases the feature dimensions of the training samples, resulting in a better model.

[0043] Artificial intelligence models encompass various algorithmic implementations, such as neural networks, decision trees, support vector machines, and Bayesian classifiers. In practical applications, an artificial intelligence model refers to a file containing elements such as network structure and parameter information. A trained artificial intelligence model can be directly reused by its framework platform without repeated construction or learning, enabling intelligent functions such as judgment and recognition.

[0044] It should be noted that the embodiments in this application are illustrated using neural networks as an example, and in actual implementation, the specific type of artificial intelligence model is not limited.

[0045] Accuracy: During the training process of an artificial intelligence model, the model periodically calculates its accuracy. Accuracy is the percentage of correct predictions out of the total number of predictions. In the training phase, the validation dataset contains input data and labels (label data), which are correlated; one set of input data corresponds to one (or set of) labels. By comparing the predicted values ​​generated by the model with the labels used in that training iteration, the accuracy of the training is determined.

[0046] The information determination method, apparatus, terminal, network-side device, and medium provided in this application will be described in detail below with reference to the accompanying drawings and through some embodiments and application scenarios.

[0047] The information determination method provided in this application embodiment can be applied to VFL scenarios, for example, it can be applied to VFL model training scenarios.

[0048] This application provides an information determination method, and Figure 2 shows a flowchart of the information determination method provided by this application. As shown in Figure 2, the information determination method provided by this application may include the following steps 201 and 202.

[0049] Step 201: The first device receives the first intermediate data from the second device.

[0050] The first intermediate data is the intermediate data corresponding to the validation data, which is used to validate the VFL model.

[0051] In this embodiment of the application, the VFL model is the model trained using VFL.

[0052] Optionally, in this embodiment, both the first device and the second device are communication devices with VFL-related capabilities. For example, the first device can be a communication device with VFL service capabilities, and the second device can be a communication device with VFL client capabilities.

[0053] Optionally, in the embodiments of this application, both the first device and the second device can be intelligent network elements in the network. For example, they can be NWDAF or AF, etc.; wherein, AF can be a communication device outside the core network, such as a third-party server, etc., and NWDAF can be a Model Training logical function (MTLF) or an Analytics logical function (AnLF), etc.

[0054] Optionally, in this embodiment of the application, the first device can be a VFL server corresponding to the VFL model, and the second device can be a VFL client corresponding to the VFL model.

[0055] Optionally, in this embodiment, the first device may be a terminal or a network-side device.

[0056] Optionally, in this embodiment, the second device may be a terminal or a network-side device.

[0057] Optionally, in this embodiment of the application, the first intermediate data may be calculated based on the verification data.

[0058] It should be noted that intermediate data can refer to intermediate results obtained by the second device based on its own model and data. For example, the intermediate results obtained by the second device based on its own model and validation data can specifically include: the second device inputting validation data into the model to obtain the intermediate result; or, the second device determining the input data based on the identification information of the validation data, and then inputting the input data into the model to obtain the intermediate result.

[0059] Optionally, in the embodiments of this application, the intermediate data mentioned above may include one set of data or multiple sets of data.

[0060] Optionally, in the embodiments of this application, a set of intermediate data may include multiple intermediate data themselves, or may include identification information of multiple intermediate data.

[0061] Optionally, in this embodiment, the aforementioned multiple sets of data may be intermediate data received multiple times. Optionally, in this embodiment, the aforementioned verification data may include at least one of the following:

[0062] Data used in the verification phase;

[0063] Actual usage data;

[0064] Model inference data;

[0065] Model test data.

[0066] Optionally, in this embodiment, the validation data can be data obtained by dividing the aligned sample data. The first device can divide the sample data related to the VFL model to obtain a validation dataset and a training dataset. The data in the validation dataset can be used to validate the VFL model, and the data in the training dataset can be used to train the VFL model.

[0067] It is understandable that the validation data and the actual data used are inconsistent. The validation data can also be data from the training phase. Therefore, the performance information generated based on the validation data is the performance information during model training.

[0068] Optionally, in the embodiments of this application, the actual data used refers to the data used in actual use.

[0069] Optionally, in this embodiment, the model inference data refers to the data used in the model inference stage.

[0070] Optionally, in this embodiment of the application, the model test data is the data used in the model testing phase.

[0071] In this embodiment of the application, since the above-mentioned verification data may include at least one of the data in the verification dataset, the actual usage data, the model inference data, and the model test data, the second device can determine and feed back the corresponding intermediate data based on different verification data, so as to improve the accuracy of determining the performance information of the VFL model.

[0072] Optionally, in the embodiments of this application, the verification data may include one set of data or multiple sets of data.

[0073] Optionally, in the embodiments of this application, a set of data in the above-mentioned verification data may include multiple verification data themselves, or may include identification information of multiple verification data.

[0074] In this embodiment of the application, since the above verification data may include one set of data or multiple sets of data, the second device can determine and feed back the corresponding intermediate data based on different amounts of verification data, so as to improve the accuracy of determining the performance information of the VFL model.

[0075] Step 202: The first device determines the performance information of the VFL model based on the first intermediate data.

[0076] Optionally, in the embodiments of this application, the above-mentioned performance information may also be referred to as performance information or performance performance information, etc.

[0077] Optionally, in the embodiments of this application, the above-mentioned performance information may include at least one of the following: the accuracy of VFL model training, the performance information of VFL model training, the performance information during actual use, the performance information of inference, and the performance information of testing.

[0078] It should be noted that performance information can be expressed as a specific percentage, such as 90%; it can also be expressed in a categorical form, such as high, medium, low, etc.; or it can be normalized data, such as 0.9. This application does not specifically limit the form of performance information. Performance information can indicate the accuracy or error level of the model's inference results for a task, either positively or negatively. For example, the inference accuracy can be indicated negatively by calculating the model's inference error or inference error rate. The calculation methods for inference error or inference error rate are diverse, such as Mean Absolute Error (MAE) and Mean Square Error (MSE).

[0079] Optionally, in the embodiments of this application, step 202 can be implemented by steps 202a and 202b as described below.

[0080] Step 202a: The first device determines the output data of the VFL model based on the first intermediate data.

[0081] Optionally, in the embodiments of this application, the above-mentioned output data may also be referred to as output results or inference results, etc.

[0082] Optionally, in this embodiment of the application, the first device can input the first intermediate data into the VFL model of the first device to generate the output data of the VFL model.

[0083] It should be noted that the first device can also determine the output data of the VFL model based on the first intermediate data using other methods, such as a weighted average method based on multiple intermediate data.

[0084] Step 202b: The first device determines the performance information of the VFL model based on the output data and tag data of the VFL model.

[0085] The aforementioned label data corresponds to the aforementioned verification data.

[0086] It can be understood that the aforementioned label data is the ground truth output data corresponding to the aforementioned verification data, such as the data ground truth, which is the actual facts and data that actually occurred.

[0087] Optionally, in this embodiment of the application, the above-mentioned label data corresponds to the sample data identifier corresponding to the verification data, that is, a set of verification data and label data has the same sample identifier.

[0088] Optionally, in the embodiments of this application, the above-mentioned label data and the above-mentioned verification data can correspond one-to-one, or one label can correspond to multiple verification data.

[0089] Optionally, in this embodiment of the application, the first device can compare the output data of the VFL model with the label data, and determine the performance information of the VFL model based on the comparison result.

[0090] Specifically, the first device can determine the performance information of the VFL model based on the model results corresponding to one or more sets of validation data. That is, a set of validation data can contain multiple data sets. After the first device sends a set of validation data to the second device, the second device can provide feedback on the intermediate data corresponding to that set of data, i.e., multiple intermediate data corresponding to multiple data identifiers. After receiving this intermediate data, the first device generates multiple output data sets for the VFL model and compares the generated output data with the corresponding label data, thereby determining the performance information of the VFL model based on the comparison results. For example, by determining whether the output data and label data are the same (or consistent) or within an allowable difference range, the first device determines the performance information of the VFL model. The first device can also generate the performance information of a VFL model based on multiple sets of validation data, for example, when a set of validation data is insufficient, or when the first device bases its decisions on internal logic or implementation details.

[0091] In this embodiment of the application, since the first device can first determine the output data of the VFL model based on the first intermediate data, and then determine the performance information of the VFL model based on the output data and label data of the VFL model, a more accurate performance information of the VFL model can be determined by the difference between the output data and the true value.

[0092] It should be noted that step 202 can also be performed after repeating step 201 multiple times, that is, the first device determines the performance information of the VFL model based on the first intermediate data repeatedly fed back by the second device. This can improve the accuracy of determining the performance information of the VFL model.

[0093] In the information determination method provided in the embodiments of this application, the first device can first receive the first intermediate data from the second device, that is, the intermediate data corresponding to the verification data used to verify the VFL model, and then determine the performance information of the VFL model based on the first intermediate data. Thus, the model training performance can be objectively evaluated through the performance information, thereby improving the training effect of the VFL model.

[0094] Optionally, the information determination method provided in this application embodiment may further include the following step 203.

[0095] Step 203: The first device sends the first information to the second device.

[0096] The first piece of information includes at least one of the following:

[0097] Validate the data;

[0098] Verify the identification information of the data;

[0099] The data range for verification;

[0100] The first instruction information is used to indicate relevant information including verification data;

[0101] The second instruction information is used to indicate relevant information for verification.

[0102] Optionally, in this embodiment of the application, when the first information includes the identification information of the verification data, the first device can send the identification information of the verification data to the second device to inform the second device to use the data corresponding to the identification information.

[0103] It should be noted that the identification information of the verification data can be understood as: the sample identifier corresponding to the verification data, that is, the identification information corresponding to the sample data.

[0104] Optionally, in this embodiment, the range of validation data can indicate the range of validation data in the final samples. For example, the entire dataset is sent, and then a separate IE (message or parameter) indicates which data in the dataset is used for validation.

[0105] Optionally, in the embodiments of this application, the relevant information of the above-mentioned verification data may include at least one of the following: verification data, identification information of verification data, data range of verification data, information indicating that the first information is used to transmit verification data, and information indicating that the data sent to the second device includes verification data.

[0106] Optionally, in the embodiments of this application, the relevant information for the above verification may include information characterizing the start of the verification phase and the use of verification data for verification, that is, the second indication information may specifically indicate: 1. the start of the verification phase; 2. the use of verification data for verification.

[0107] Optionally, in this embodiment of the application, the first device sends first information to the second device to request verification, that is, to request the second device to determine the first intermediate data based on the first information.

[0108] Optionally, in this embodiment of the application, the first device can send first information to the second device via NEF.

[0109] It should be noted that NEF can be a network element exposed in the network, that is, a communication device that provides exposed core network internal services to (external) communication devices; it can also be other network elements with similar functions.

[0110] In this embodiment of the application, since the first device can send first information related to verification to the second device, the second device can determine first intermediate data based on the first information, so as to use the first device to determine the performance information of the VFL model.

[0111] Optionally, in this embodiment of the application, before step 203 above, the information determination method provided in this embodiment of the application may further include step 204 below.

[0112] Step 204: Based on the second information, the first device determines to verify the VFL model.

[0113] The second piece of information includes at least one of the following: the number of training iterations of the VFL model, and the loss function of the VFL model.

[0114] Specifically, the first device can determine whether to validate the VFL model based on the change information of the loss function of the VFL model.

[0115] It should be noted that the second information can be determined by the first device itself, or it can be pre-configured, or it can be sent by other communication devices.

[0116] For example, when the number of training iterations of the VFL model is greater than or equal to a threshold for the number of training iterations, the first device may determine to validate the VFL model; or, when the change in the loss function of the VFL model is greater than or equal to a threshold for the change in the loss function, the first device may determine to validate the VFL model.

[0117] In this embodiment of the application, since the first device can determine the need to verify the VFL model based on the second information, it can first determine that verification is required based on at least one of the training iterations of the VFL model and the loss function of the VFL model, and then send the first information to the second device. Thus, the verification process can be triggered only when it is determined that verification is required, thereby avoiding unnecessary verification processes and saving signaling overhead.

[0118] Optionally, the information determination method provided in this application embodiment may further include the following step 205.

[0119] Step 205: The first device sends the third information to the second device.

[0120] The third information includes at least one of the following:

[0121] Training data;

[0122] Identification information for training data;

[0123] The range of training data;

[0124] The third instruction information is used to indicate relevant information including training data.

[0125] The fourth instruction information is used to indicate relevant training information.

[0126] The training data mentioned above was used to train the VFL model.

[0127] Optionally, in this embodiment of the application, when the third information includes the identification information of the training data, the first device can send the identification information of the training data to the second device to inform the second device to use the data corresponding to the identification information.

[0128] It should be noted that the identification information of training data can be understood as: the sample identifier corresponding to the training data, that is, the identification information corresponding to the sample data.

[0129] Optionally, in this embodiment, the range of training data can indicate the range of training data in the final samples. For example, the entire dataset is sent, and then a separate IE (message or parameter) indicates which data in the dataset is used for training.

[0130] Optionally, in the embodiments of this application, the relevant information of the training data mentioned above may include at least one of the following: training data, identification information of training data, data range of training data, information representing the third information used to transmit training data, and information representing that the data sent to the second device includes training data.

[0131] Optionally, in the embodiments of this application, the training-related information mentioned above may include information representing the start of the training phase and the use of training data for training, that is, the fourth indication information may specifically indicate: 1. the start of the training phase; 2. the use of training data for training.

[0132] Optionally, in this embodiment of the application, the first device may send the third information and the first information in the same message, or it may send the third information and the first information separately.

[0133] Optionally, in this embodiment of the application, the first device sends third information to the second device to trigger the training process, that is, to request the second device to generate intermediate data corresponding to the training data based on the third information.

[0134] Optionally, in this embodiment of the application, the first device can send third information to the second device via NEF.

[0135] In this embodiment of the application, since the first device can send third information related to training to the second device, the training process can be triggered to update the VFL model, thereby continuously iterating and updating the model and determining the performance information of the VFL model, which can improve the training effect of the VFL model.

[0136] Optionally, the information determination method provided in this application embodiment may further include the following step 206.

[0137] Step 206: The first device determines the training data and validation data based on the aligned sample data.

[0138] The sample data mentioned above is related to the VFL model.

[0139] Optionally, in the embodiments of this application, the training data can be in the form of a set or a non-set.

[0140] Optionally, in the embodiments of this application, the verification data can be in a set form or a non-set form.

[0141] For example, the first device may determine the training dataset and the validation dataset based on the aligned sample data, wherein the training dataset includes training data and the validation dataset includes validation data; or, the first device may determine the training data and the validation data based on the identification information of the sample data, for example, determining the first 80% of the sample data as training data and the last 20% of the sample data as validation data.

[0142] Optionally, in this embodiment of the application, the training data and the validation data can be different data.

[0143] In this embodiment, since the first device can determine the training data and validation data based on the aligned sample data, it can send the training data and validation data to the second device, or indicate the relevant information of the training data and validation data, so as to clarify the data used for training and the data used for validation, thereby improving the efficiency of VFL model training.

[0144] The information determination method provided in the embodiments of this application will be described exemplarily below with reference to the accompanying drawings.

[0145] For example, assuming the first device is a VFL server and the second device is a VFL client, then:

[0146] As shown in Figure 3, the information determination method provided in this application embodiment may include the following process:

[0147] Step 0: The VFL server and VFL client interact to perform sample alignment and generate shared sample data. Sample alignment refers to identifying shared samples based on the sample identification information of the data, ensuring that the same samples are used during training and other processes, preventing missing or inconsistent samples.

[0148] Step 1: The VFL server sends the aligned sample data to the VFL client.

[0149] It should be noted that when the VFL server or VFL client is an untrusted AF, this step of the interaction may be completed through NEF. The VFL server first sends the aligned sample data to NEF, and then NEF sends the aligned sample data to the VFL client.

[0150] Step 2 (optional): The VFL server determines the training data and validation data, that is, the VFL server divides the aligned samples into training data (set) and validation data (set).

[0151] It should be noted that this distinction can be between two new datasets, or it can be based on the identification information of the sample data, such as dividing the first 80% of the data into training data and the last 20% into test data.

[0152] Step 3: The VFL server sends training data to the VFL client and triggers the VFL training process. The VFL server sends training data directly or indirectly to the VFL client, thereby requesting the VFL client to generate intermediate data based on the data and provide feedback.

[0153] Specifically, this step may include at least one of the following: training data; identification information of the training data; data range of the training data; third instruction information; and fourth instruction information.

[0154] It should be noted that when the VFL server or VFL client is an untrusted AF, this step of the interaction may be completed through NEF.

[0155] Step 4: The VFL client sends intermediate data back to the VFL server. This intermediate data can be the intermediate data corresponding to the training data in Step 3, that is, the intermediate data is obtained based on the above training data. For example, the input data corresponding to the identification information of the training data is input into the local model of the VFL client, and the generated model outputs the results.

[0156] It should be noted that when the VFL server or VFL client is an untrusted AF, this step of the interaction may be completed through NEF.

[0157] Step 5: The VFL server and VFL client train the model based on the training data, thereby updating the model and completing the training. Specifically, steps 3 and 4 can be repeated between the VFL server and VFL client: the VFL server sends training data to the VFL client, the VFL client provides intermediate data based on the training data, and the VFL server uses the intermediate data provided by the VFL client as inverse intermediate results such as loss or gradient for model updates, thus helping the VFL client update the model. The model training is completed after multiple iterations.

[0158] It should be noted that the training data mentioned above can be the same or different each time it is sent to the VFL client. For example, the VFL server can use all the training data in one training session, or it can use different training data, such as a portion of the training data, in each training session.

[0159] Step 6: The VFL server sends verification data to the VFL client and requests verification. The VFL server can send verification data to the VFL client, requesting the VFL client to perform calculations based on the data to generate corresponding intermediate data, thereby determining the model's performance information based on the intermediate data.

[0160] Before sending validation data to the VFL client, the VFL server decides whether to proceed with the validation phase. Specifically, the VFL server determines whether to proceed with the validation phase based on factors such as the number of training epochs or changes in the loss function during training.

[0161] It should be noted that when the VFL server or VFL client is an untrusted AF, this step of the interaction may be completed through NEF.

[0162] When the VFL server sends verification data to the VFL client, it may carry at least one of the following: verification data; identification information of the verification data; data range of the verification data; first indication information; second indication information.

[0163] It should be noted that the verification data mentioned above can also be actual usage data, inference data, or test data, etc. That is, data used in the actual use, inference, or testing phases. In this case, the indication information can also be the corresponding inference indication information or test indication information, etc.

[0164] Step 7: The VFL client sends intermediate data back to the VFL server. This intermediate data can be the intermediate data corresponding to the verification data in Step 6, that is, the intermediate data is obtained based on the above verification data. For example, the input data corresponding to the identification information of the verification data is input into the local model of the VFL client, and the generated model output results are obtained.

[0165] It should be noted that when the VFL server or VFL client is an untrusted AF, this step may be completed through NEF. The intermediate data mentioned above can also be intermediate data corresponding to actual data used, inference data, or test data.

[0166] Step 8: The VFL server determines model performance information. Specifically, the VFL server can generate model output data based on the intermediate data fed back by the VFL client in Step 7, thereby determining the model's performance information. Specifically, it compares the model's output data with the label data (true values) corresponding to the validation data to generate model performance information. Specifically, the VFL server can determine model performance information based on the model results corresponding to one or more validation data sets.

[0167] It should be noted that when the data carried in step 6 is actual data, inference data, or test data, the performance information determined by the VFL server is the performance information of the model when it is actually used, the performance information of inference, or the performance information of testing, or the accuracy of the model.

[0168] It should also be noted that step 8 may be executed after steps 6-7 have been repeated multiple times; that is, step 8 will not be executed immediately after each execution of step 7.

[0169] In this way, the VFL server can control the different data identifiers sent to the VFL client, thereby causing the VFL client to provide intermediate data based on different data, thus generating more accurate performance information of the VFL model.

[0170] As shown in Figure 4, the difference between the information determination method illustrated in Figure 4 and the method in the example in Figure 3 is that in the method shown in Figure 4, the VFL server first sends training data and / or validation data to the VFL client, and then in subsequent training and validation phases, it can send only the instruction information for training and / or validation. The specific differences are as follows:

[0171] The parameters that can be carried in steps 4a, 4b, and 5 include at least one of the following:

[0172] Indication information for training data, which indicates that the transmitted data includes training data;

[0173] Training data, or identification information indicating the training data, such as 1-100, or indicating the range of training data in the final sample data;

[0174] The indication information for verifying data is used to indicate that the transmitted data includes verification data.

[0175] Validation data, or identification information indicating validation data, such as 101-200, or an indication of the range of validation data in the final sample data.

[0176] It should be noted that the above verification data can also be actual data used, inference data, or test data.

[0177] Thus, the VFL server can determine the performance information of the VFL model through the information determination method provided in the embodiments of this application.

[0178] This application provides another information determination method, and Figure 5 shows a flowchart of the information determination method provided in this application embodiment. As shown in Figure 5, the information determination method provided in this application embodiment may include the following step 501.

[0179] Step 501: The second device sends the first intermediate data to the first device.

[0180] The first intermediate data is used to determine the performance information of the VFL model. The first intermediate data is the intermediate data corresponding to the validation data, and the validation data is used to validate the VFL model.

[0181] Optionally, in this embodiment of the application, the verification data may include at least one of the following:

[0182] Data used in the verification phase;

[0183] Actual usage data;

[0184] Model inference data;

[0185] Model test data.

[0186] Optionally, in the embodiments of this application, the verification data described above may include one set of data or multiple sets of data.

[0187] In the information determination method provided in the embodiments of this application, the second device can send first intermediate data to the first device, that is, intermediate data corresponding to the verification data used to verify the VFL model, so that the first device can determine the performance information of the VFL model based on the first intermediate data, and then objectively evaluate the training performance of the model through the performance information, thereby improving the training effect of the VFL model.

[0188] Optionally, the information determination method provided in this application embodiment may further include the following steps 502 and 503.

[0189] Step 502: The second device receives the first information from the first device.

[0190] The first piece of information includes at least one of the following:

[0191] Validate the data;

[0192] Verify the identification information of the data;

[0193] The data range for verification;

[0194] The first instruction information is used to indicate relevant information including verification data;

[0195] The second instruction information is used to indicate relevant information for verification.

[0196] Step 503: The second device determines the first intermediate data based on the first information.

[0197] Optionally, the information determination method provided in the embodiments of this application may further include the following steps 504 and 505.

[0198] Step 504: The second device receives third information from the first device.

[0199] The third information includes at least one of the following:

[0200] Training data;

[0201] Identification information for training data;

[0202] The range of training data;

[0203] The third instruction information is used to indicate relevant information including training data.

[0204] The fourth instruction information is used to indicate relevant training information.

[0205] The training data mentioned above was used to train the VFL model.

[0206] Step 505: The second device determines the second intermediate data based on the third information.

[0207] The second intermediate data is the intermediate data corresponding to the training data mentioned above.

[0208] For further descriptions of the information determination method provided in the embodiments of this application, please refer to the relevant descriptions in the first device-side method embodiments described above. To avoid repetition, they will not be repeated here.

[0209] The above-described method embodiments, or various possible implementations of the method embodiments, can be executed individually, or, provided there are no contradictions, they can be combined with each other. The specific implementation can be determined according to actual usage requirements, and this application embodiment does not impose any restrictions on this.

[0210] The information determination method provided in this application can be executed by an information determination device. This application uses an information determination device executing the information determination method as an example to illustrate the information determination device provided in this application.

[0211] This application provides an information determination device. As an example, the information determination device may be a communication device or a component within a communication device, such as a chip. The communication device may be a terminal, a network-side device, or a server, etc. Exemplarily, the terminal may include, but is not limited to, the type of terminal 11 listed above, and the network-side device may include, but is not limited to, the type of network-side device 12 listed above. This application does not impose specific limitations.

[0212] The information determination device includes a receiving module, a transmitting module, and a processing module. These modules can be implemented in software or hardware. When implemented in hardware, the processing module can be implemented by a processor. For example, the processor can include general-purpose processors, special-purpose processors, such as a Central Processing Unit (CPU), microprocessor, Digital Signal Processor (DSP), Artificial Intelligence (AI) processor, Graphics Processing Unit (GPU), Application Specific Integrated Circuit (ASIC), Network Processor (NP), Field Programmable Gate Array (FPGA), or other programmable logic devices, gate circuits, transistors, discrete hardware components, etc. The receiving and transmitting modules can be implemented by a communication interface, which can include one or more of the following: transceiver, pins, circuits, bus, radio frequency unit, etc.

[0213] Specifically, referring to Figure 6, the information determination device 60 includes a first receiving module 61 and a first processing module 62.

[0214] The first receiving module 61 can be used to receive first intermediate data from the second device. The first processing module 62 can be used to determine the performance information of the VFL model based on the first intermediate data received by the first receiving module 61. The first intermediate data is the intermediate data corresponding to the verification data, which is used to verify the VFL model.

[0215] In one possible implementation, the first processing module 62 can be used to determine the output data of the VFL model based on the first intermediate data; and to determine performance information based on the output data and the label data. The label data corresponds to the verification data.

[0216] In one possible implementation, the information determining device 60 may further include a first sending module. The first sending module can be used to send first information to the second device. The first information includes at least one of the following: verification data; identification information of the verification data; data range of the verification data; first indication information, used to indicate that the first information includes relevant information related to the verification data; and second indication information, used to indicate relevant information related to the verification.

[0217] In one possible implementation, the first processing module 62 can also be used to determine, based on the second information, whether to validate the VFL model before the first sending module sends the first information to the second device. The second information includes at least one of the following: the number of training iterations of the VFL model, and the loss function of the VFL model.

[0218] In one possible implementation, the information determining device 60 may further include a first transmitting module. The first transmitting module can be used to transmit third information to the second device. The third information includes at least one of the following: training data; identification information of the training data; data range of the training data; third indication information, used to indicate that the third information includes relevant information about the training data; and fourth indication information, used to indicate relevant information about training; wherein the training data is used to train the VFL model.

[0219] In one possible implementation, the first processing module 62 can also be used to determine training and validation data based on the aligned sample data. The sample data is related to the VFL model.

[0220] In one possible implementation, the verification data may include at least one of the following: data used in the verification phase; actual usage data; model inference data; and model test data.

[0221] In one possible implementation, the verification data may include one set of data or multiple sets of data.

[0222] In the information determination device provided in the embodiments of this application, the information determination device can first receive first intermediate data from the second device, that is, intermediate data corresponding to the verification data used to verify the VFL model, and then determine the performance information of the VFL model based on the first intermediate data. Thus, the model training performance can be objectively evaluated through the performance information, thereby improving the training effect of the VFL model.

[0223] The information determination device provided in this application embodiment can implement the various processes implemented in the above-described first device-side method embodiment and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0224] Referring to Figure 7, the information determination device 70 includes: a second transmission module 71.

[0225] The second sending module 71 can be used to send first intermediate data to the first device; wherein the first intermediate data is used to determine the performance information of the VFL model, the first intermediate data is the intermediate data corresponding to the verification data, and the verification data is used to verify the VFL model.

[0226] In one possible implementation, the information determining device 70 may further include a second receiving module and a second processing module. The second receiving module is used to receive first information from the first device. The second processing module is used to determine first intermediate data based on the first information received by the second receiving module. The first information includes at least one of the following: verification data; identification information of the verification data; data range of the verification data; first indication information indicating that the first information includes relevant information related to the verification data; and second indication information indicating relevant information related to the verification.

[0227] In one possible implementation, the information determining device 70 may further include a second receiving module and a second processing module. The second receiving module can be used to receive third information from the first device. The second processing module can be used to determine second intermediate data based on the third information received by the second receiving module. The second intermediate data is intermediate data corresponding to the training data, which is used to train the VFL model. The third information includes at least one of the following: training data; identification information of the training data; the data range of the training data; third indication information, used to indicate that the third information includes relevant information about the training data; and fourth indication information, used to indicate relevant information about the training.

[0228] In one possible implementation, the verification data may include at least one of the following: data used in the verification phase; actual usage data; model inference data; and model test data.

[0229] In one possible implementation, the verification data may include one set of data or multiple sets of data.

[0230] In the information determination device provided in the embodiments of this application, the information determination device can send first intermediate data to the first device, that is, intermediate data corresponding to the verification data used to verify the VFL model, so that the first device can determine the performance information of the VFL model based on the first intermediate data, and then objectively evaluate the training performance of the model through the performance information, thereby improving the training effect of the VFL model.

[0231] The information determination device provided in this application embodiment can implement the various processes implemented in the above-described second device-side method embodiment and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0232] As shown in Figure 8, this application embodiment also provides a communication device 100, including a processor 101 and a memory 102. The memory 102 stores a program or instructions that can run on the processor 101. When the program or instructions are executed by the processor 101, they can implement the steps of the first device-side method embodiment described above, or implement the steps of the second device-side method embodiment described above. And they can achieve the same technical effects; to avoid repetition, they will not be described again here.

[0233] This application also provides a terminal, including a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the steps in the above-described first device-side or second device-side method embodiments. This terminal embodiment corresponds to the above-described first device-side or second device-side method embodiments; all implementation processes and methods of the above method embodiments can be applied to this terminal embodiment and achieve the same technical effect. The terminal may be the information determination device shown in FIG. 6 or FIG. 7. Specifically, FIG. 9 is a schematic diagram of the hardware structure of a terminal implementing an embodiment of this application.

[0234] The terminal 1000 includes, but is not limited to, at least some of the following components: radio frequency unit 1001, network module 1002, audio output unit 1003, input unit 1004, sensor 1005, display unit 1006, user input unit 1007, interface unit 1008, memory 1009, and processor 1010.

[0235] Those skilled in the art will understand that the terminal 1000 may also include a power supply (such as a battery) for powering various components. The power supply can be logically connected to the processor 1010 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The terminal structure shown in Figure 9 does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0236] It should be understood that, in this embodiment, the input unit 1004 may include a graphics processor 10041 and a microphone 10042. The graphics processor 10041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 1006 may include a display panel 10061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 1007 includes a touch panel 10071 and at least one of other input devices 10072. The touch panel 10071 is also called a touch screen. The touch panel 10071 may include a touch detection device and a touch controller. Other input devices 10072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0237] In this embodiment, after receiving downlink data from the network-side device, the radio frequency unit 1001 can transmit it to the processor 1010 for processing; in addition, the radio frequency unit 1001 can send uplink data to the network-side device. Typically, the radio frequency unit 1001 includes, but is not limited to, antennas, amplifiers, transceivers, couplers, low-noise amplifiers, duplexers, etc.

[0238] The memory 1009 can be used to store software programs or instructions, as well as various data. The memory 1009 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1009 may include volatile memory or non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1009 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0239] The processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into the processor 1010.

[0240] The radio frequency unit 1001 can be used to receive first intermediate data from the second device. The processor 1010 can be used to determine the performance information of the VFL model based on the first intermediate data received by the radio frequency unit 1001. The first intermediate data is intermediate data corresponding to the verification data, which is used to verify the VFL model.

[0241] In one possible implementation, the processor 1010 can be used to determine the output data of the VFL model based on the first intermediate data; and to determine performance information based on the output data and the tag data. The tag data corresponds to the verification data.

[0242] In one possible implementation, the radio frequency unit 1001 can also be used to send first information to the second device. The first information includes at least one of the following: verification data; identification information of the verification data; data range of the verification data; first indication information, used to indicate that the first information includes relevant information related to the verification data; and second indication information, used to indicate relevant information related to the verification.

[0243] In one possible implementation, the processor 1010 can also be used to determine, based on the second information, whether to validate the VFL model before the radio frequency unit 1001 sends the first information to the second device. The second information includes at least one of the following: the number of training iterations of the VFL model, and the loss function of the VFL model.

[0244] In one possible implementation, the radio frequency unit 1001 can also be used to send third information to the second device. The third information includes at least one of the following: training data; identification information of the training data; data range of the training data; third indication information, used to indicate that the third information includes relevant information about the training data; and fourth indication information, used to indicate relevant information about training; wherein the training data is used to train the VFL model.

[0245] In one possible implementation, the processor 1010 can also be used to determine training and validation data based on the aligned sample data. The sample data is related to the VFL model.

[0246] In one possible implementation, the verification data may include at least one of the following: data used in the verification phase; actual usage data; model inference data; and model test data.

[0247] In one possible implementation, the verification data may include one set of data or multiple sets of data.

[0248] In the terminal provided in this application embodiment, the terminal can first receive first intermediate data from the second device, that is, intermediate data corresponding to the verification data used to verify the VFL model, and then determine the performance information of the VFL model based on the first intermediate data. Thus, the model training performance can be objectively evaluated through the performance information, thereby improving the training effect of the VFL model.

[0249] The terminal provided in this application embodiment can implement all the processes implemented in the first device-side method embodiment described above and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0250] or,

[0251] The radio frequency unit 1001 can be used to send first intermediate data to the first device; wherein the first intermediate data is used to determine the performance information of the VFL model, the first intermediate data is the intermediate data corresponding to the verification data, and the verification data is used to verify the VFL model.

[0252] In one possible implementation, the radio frequency unit 1001 can also be used to receive first information from the first device. The processor 1010 can be used to determine first intermediate data based on the first information received by the radio frequency unit 1001. The first information includes at least one of the following: verification data; identification information of the verification data; data range of the verification data; first indication information, used to indicate that the first information includes relevant information related to the verification data; and second indication information, used to indicate relevant information related to the verification.

[0253] In one possible implementation, the radio frequency unit 1001 can also be used to receive third information from the first device. The processor 1010 can be used to determine second intermediate data based on the third information received by the radio frequency unit 1001. The second intermediate data is intermediate data corresponding to the training data, which is used to train the VFL model. The third information includes at least one of the following: training data; identification information of the training data; data range of the training data; third indication information, used to indicate that the third information includes relevant information about the training data; and fourth indication information, used to indicate relevant information about the training.

[0254] In one possible implementation, the verification data may include at least one of the following: data used in the verification phase; actual usage data; model inference data; and model test data.

[0255] In one possible implementation, the verification data may include one set of data or multiple sets of data.

[0256] In the terminal provided in this application embodiment, the terminal can send first intermediate data to the first device, that is, intermediate data corresponding to the verification data used to verify the VFL model, so that the first device can determine the performance information of the VFL model based on the first intermediate data, and then objectively evaluate the training performance of the model through the performance information, thereby improving the training effect of the VFL model.

[0257] The terminal provided in this application embodiment can implement all the processes implemented in the above-described second device-side method embodiment and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0258] This application also provides a network-side device, including a processor and a communication interface. The communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the steps of the first device-side or second device-side method embodiments described above. This network-side device embodiment corresponds to the first device-side or second device-side method embodiments described above. All implementation processes and methods of the above method embodiments can be applied to this network-side device embodiment and achieve the same technical effects.

[0259] Specifically, this application embodiment also provides a network-side device, which may be the information determination device shown in FIG. 6 or FIG. 7. As shown in FIG. 10, the network-side device 1100 includes: an antenna 111, a radio frequency device 112, a baseband device 113, a processor 114, and a memory 115. The antenna 111 is connected to the radio frequency device 112. In the uplink direction, the radio frequency device 112 receives information through the antenna 111 and sends the received information to the baseband device 113 for processing. In the downlink direction, the baseband device 113 processes the information to be transmitted and sends it to the radio frequency device 112. The radio frequency device 112 processes the received information and transmits it through the antenna 111.

[0260] The method executed by the first or second device in the above embodiments can be implemented in the baseband device 113, which includes a baseband processor.

[0261] The baseband device 113 may include at least one baseband board, on which multiple chips are disposed, as shown in FIG10. One of the chips is, for example, a baseband processor, which is connected to the memory 115 via a bus interface to call the program in the memory 115 and execute the network device operation shown in the above method embodiment.

[0262] The network-side device may also include a network interface 116, such as a Common Public Radio Interface (CPRI).

[0263] Specifically, the network-side device 1100 in this application embodiment further includes: instructions or programs stored in memory 115 and executable on processor 114. The processor 114 calls the instructions or programs in memory 115 to execute the steps of the first device-side or second device-side method embodiments described above, and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0264] The radio frequency device 112 can be used to receive first intermediate data from the second device. The processor 114 can be used to determine the performance information of the VFL model based on the first intermediate data received by the radio frequency device 112. The first intermediate data is intermediate data corresponding to the verification data, which is used to verify the VFL model.

[0265] In one possible implementation, the processor 114 can be used to determine the output data of the VFL model based on the first intermediate data; and to determine performance information based on the output data and the label data. The label data corresponds to the verification data.

[0266] In one possible implementation, the radio frequency device 112 can also be used to send first information to the second device. The first information includes at least one of the following: verification data; identification information of the verification data; data range of the verification data; first indication information, used to indicate that the first information includes relevant information related to the verification data; and second indication information, used to indicate relevant information related to the verification.

[0267] In one possible implementation, the processor 114 can also be used to determine, based on second information, whether to validate the VFL model before the radio frequency device 112 sends the first information to the second device. The second information includes at least one of the following: the number of training iterations of the VFL model, and the loss function of the VFL model.

[0268] In one possible implementation, the radio frequency device 112 can also be used to send third information to the second device. The third information includes at least one of the following: training data; identification information of the training data; data range of the training data; third indication information, used to indicate that the third information includes relevant information about the training data; and fourth indication information, used to indicate relevant information about training; wherein the training data is used to train the VFL model.

[0269] In one possible implementation, the processor 114 can also be used to determine training and validation data based on the aligned sample data. The sample data is related to the VFL model.

[0270] In one possible implementation, the verification data may include at least one of the following: data used in the verification phase; actual usage data; model inference data; and model test data.

[0271] In one possible implementation, the verification data may include one set of data or multiple sets of data.

[0272] In the network-side device provided in this application embodiment, the network-side device can first receive first intermediate data from the second device, that is, intermediate data corresponding to the verification data used to verify the VFL model, and then determine the performance information of the VFL model based on the first intermediate data. Thus, the model training performance can be objectively evaluated through the performance information, thereby improving the training effect of the VFL model.

[0273] The network-side device provided in this application embodiment can implement all the processes implemented in the first device-side method embodiment described above and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0274] or,

[0275] The radio frequency device 112 can be used to send first intermediate data to the first device; wherein the first intermediate data is used to determine the performance information of the VFL model, the first intermediate data is the intermediate data corresponding to the verification data, and the verification data is used to verify the VFL model.

[0276] In one possible implementation, the radio frequency device 112 can also be used to receive first information from the first device. The processor 114 can be used to determine first intermediate data based on the first information received by the radio frequency device 112. The first information includes at least one of the following: verification data; identification information of the verification data; data range of the verification data; first indication information, used to indicate that the first information includes relevant information related to the verification data; and second indication information, used to indicate relevant information related to the verification.

[0277] In one possible implementation, the radio frequency device 112 can also be used to receive third information from the first device. The processor 114 can be used to determine second intermediate data based on the third information received by the radio frequency device 112. The second intermediate data is intermediate data corresponding to the training data, which is used to train the VFL model. The third information includes at least one of the following: training data; identification information of the training data; data range of the training data; third indication information, used to indicate that the third information includes relevant information about the training data; and fourth indication information, used to indicate relevant information about the training.

[0278] In one possible implementation, the verification data may include at least one of the following: data used in the verification phase; actual usage data; model inference data; and model test data.

[0279] In one possible implementation, the verification data may include one set of data or multiple sets of data.

[0280] In the network-side device provided in this application embodiment, the network-side device can send first intermediate data to the first device, that is, intermediate data corresponding to the verification data used to verify the VFL model, so that the first device can determine the performance information of the VFL model based on the first intermediate data, and then objectively evaluate the training performance of the model through the performance information, thereby improving the training effect of the VFL model.

[0281] The network-side device provided in this application embodiment can implement all the processes implemented in the above-described second device-side method embodiment and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0282] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described information determination method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0283] The processor mentioned above is the processor in the terminal described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk. In some examples, the readable storage medium may be a non-transient readable storage medium.

[0284] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described information determination method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0285] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0286] This application also provides a computer program / program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described information determination method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0287] This application also provides a communication system, including: a terminal and a network-side device. The terminal can be used to perform the steps of the first device-side method described above, and the network-side device can be used to perform the steps of the second device-side method described above. Alternatively, the terminal can be used to perform the steps of the second device-side method described above, and the network-side device can be used to perform the steps of the first device-side method described above.

[0288] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0289] From the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of computer software products plus necessary general-purpose hardware platforms, and of course, they can also be implemented by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, magnetic disk, optical disk, etc.) and includes several instructions to cause the terminal or network-side device to execute the methods described in the various embodiments of this application.

[0290] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other implementations under the guidance of this application without departing from the spirit and scope of the claims. All of these implementations are within the protection scope of this application.

Claims

1. An information determination method, the method comprising: The first device receives first intermediate data from the second device; The first device determines the performance information of the longitudinal federated learning VFL model based on the first intermediate data; Wherein, the first intermediate data is the intermediate data corresponding to the verification data, and the verification data is used to verify the VFL model.

2. The method according to claim 1, wherein, The first device determines the performance information of the VFL model based on the first intermediate data, including: The first device determines the output data of the VFL model based on the first intermediate data; The first device determines the performance information based on the output data and the tag data, wherein the tag data corresponds to the verification data.

3. The method according to claim 1 or 2, wherein, The method further includes: The first device sends first information to the second device; The first information includes at least one of the following: The verification data; The identification information of the verification data; The data range of the verification data; First indication information, the first indication information is used to indicate that the first information includes relevant information of the verification data; The second indication information is used to indicate relevant information for verification.

4. The method according to claim 3, wherein, Before the first device sends the first information to the second device, the method further includes: The first device determines to verify the VFL model based on the second information; The second information includes at least one of the following: the number of training iterations of the VFL model, and the loss function of the VFL model.

5. The method according to any one of claims 1 to 4, wherein, The method further includes: The first device sends a third message to the second device; The third information includes at least one of the following: Training data; Identification information for training data; The range of training data; The third indication information is used to indicate that the third information includes relevant information about the training data; The fourth indication information is used to indicate relevant training information; The training data is used to train the VFL model.

6. The method according to any one of claims 1 to 5, wherein, The method further includes: The first device determines training and validation data based on aligned sample data, which are related to the VFL model.

7. The method according to any one of claims 1 to 6, wherein, The verification data includes at least one of the following: Data used in the verification phase; Actual usage data; Model inference data; Model test data.

8. The method according to any one of claims 1 to 7, wherein, The verification data includes one set of data or multiple sets of data.

9. An information determination method, the method comprising: The second device sends the first intermediate data to the first device; The first intermediate data is used to determine the performance information of the VFL model, and the first intermediate data is the intermediate data corresponding to the verification data, which is used to verify the VFL model.

10. The method according to claim 9, wherein, The method further includes: The second device receives the first information from the first device; The second device determines the first intermediate data based on the first information; The first information includes at least one of the following: The verification data; The identification information of the verification data; The data range of the verification data; First indication information, the first indication information is used to indicate that the first information includes relevant information of the verification data; The second indication information is used to indicate relevant information for verification.

11. The method according to claim 9 or 10, wherein, The method further includes: The second device receives third information from the first device; The second device determines the second intermediate data based on the third information; Wherein, the second intermediate data is the intermediate data corresponding to the training data, the training data being used to train the VFL model, and the third information includes at least one of the following: The training data; The identification information of the training data; The data range of the training data; The third indication information is used to indicate that the third information includes relevant information about the training data; The fourth indication information is used to indicate relevant training information.

12. The method according to any one of claims 9 to 11, wherein, The verification data includes at least one of the following: Data used in the verification phase; Actual usage data; Model inference data; Model test data.

13. The method according to any one of claims 9 to 12, wherein, The verification data includes one set of data or multiple sets of data.

14. An information determining device, the device comprising a first receiving module and a first processing module; The first receiving module is used to receive first intermediate data from the second device; The first processing module is used to determine the performance information of the VFL model based on the first intermediate data received by the first receiving module; in, The first intermediate data is the intermediate data corresponding to the verification data, which is used to verify the VFL model.

15. The apparatus according to claim 14, wherein, The first processing module is specifically used to determine the output data of the VFL model based on the first intermediate data; and to determine the performance information based on the output data and the label data, wherein the label data corresponds to the verification data.

16. The apparatus according to claim 14 or 15, wherein, The device also includes a first transmitting module; The first sending module is used to send first information to the second device; The first information includes at least one of the following: The verification data; The identification information of the verification data; The data range of the verification data; First indication information, the first indication information is used to indicate that the first information includes relevant information of the verification data; The second indication information is used to indicate relevant information for verification.

17. The apparatus according to claim 16, wherein, The first processing module is further configured to determine, based on the second information, to verify the VFL model before the first sending module sends the first information to the second device; The second information includes at least one of the following: the number of training iterations of the VFL model, and the loss function of the VFL model.

18. The apparatus according to any one of claims 14 to 17, wherein, The device also includes a first transmitting module; The first sending module is used to send third information to the second device; The third information includes at least one of the following: Training data; Identification information for training data; The range of training data; The third indication information is used to indicate that the third information includes relevant information about the training data; The fourth indication information is used to indicate relevant training information; The training data is used to train the VFL model.

19. The apparatus according to any one of claims 14 to 18, wherein, The first processing module is further configured to determine training data and validation data based on the aligned sample data, wherein the sample data is related to the VFL model.

20. The apparatus according to any one of claims 14 to 19, wherein, The verification data includes at least one of the following: Data used in the verification phase; Actual usage data; Model inference data; Model test data.

21. The apparatus according to any one of claims 14 to 20, wherein, The verification data includes one set of data or multiple sets of data.

22. An information determining device, the device comprising a second transmitting module; The second sending module is used to send first intermediate data to the first device; in, The first intermediate data is used to determine the performance information of the VFL model. The first intermediate data is the intermediate data corresponding to the verification data, and the verification data is used to verify the VFL model.

23. The apparatus according to claim 22, wherein, The device further includes a second receiving module and a second processing module; The second receiving module is used to receive first information from the first device; The second processing module is used to determine the first intermediate data based on the first information received by the second receiving module; The first information includes at least one of the following: The verification data; The identification information of the verification data; The data range of the verification data; First indication information, the first indication information is used to indicate that the first information includes relevant information of the verification data; The second indication information is used to indicate relevant information for verification.

24. The apparatus according to claim 22 or 23, wherein, The device further includes a second receiving module and a second processing module; The second receiving module is used to receive third information from the first device; The second processing module is used to determine the second intermediate data based on the third information received by the second receiving module; Wherein, the second intermediate data is the intermediate data corresponding to the training data, the training data being used to train the VFL model, and the third information includes at least one of the following: The training data; The identification information of the training data; The data range of the training data; The third indication information is used to indicate that the third information includes relevant information about the training data; The fourth indication information is used to indicate relevant training information.

25. The apparatus according to any one of claims 22 to 24, wherein, The verification data includes at least one of the following: Data used in the verification phase; Actual usage data; Model inference data; Model test data.

26. The apparatus according to any one of claims 22 to 25, wherein, The verification data includes one set of data or multiple sets of data.

27. A terminal comprising a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the information determination method as claimed in any one of claims 1 to 8, or implementing the steps of the information determination method as claimed in any one of claims 9 to 13.

28. A network-side device, comprising a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions, when executed by the processor, implementing the steps of the information determination method as claimed in any one of claims 1 to 8, or implementing the steps of the information determination method as claimed in any one of claims 9 to 13.

29. A readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the information determination method as claimed in any one of claims 1 to 8, or implement the steps of the information determination method as claimed in any one of claims 9 to 13.

30. A computer program product, said computer program product being executed by at least one processor to implement the steps of the information determination method as claimed in any one of claims 1 to 8, or to implement the steps of the information determination method as claimed in any one of claims 9 to 13.

31. An electronic device comprising being configured to perform the steps of the information determination method as claimed in any one of claims 1 to 8, or to perform the steps of the information determination method as claimed in any one of claims 9 to 13.

32. A chip comprising a processor and a communication interface coupled to the processor, the processor being configured to run a program or instructions to implement the steps of the information determination method as described in any one of claims 1 to 8, or to implement the steps of the information determination method as described in any one of claims 9 to 13.

Images