Network performance index determination method and device, data screening method and device, related equipment, storage medium and computer program product

Abstract

The invention discloses a network performance index determination method and device, a data screening method and device, related equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring first data; determining a first model matched with first data, wherein the first data is related to a real-time communication network condition; determining a network performance index corresponding to the first data by using the matched first model; wherein the first data set is utilized to optimize the first model, the first data set comprises data obtained through a second model, the second model comprises a model determined based on related information of the first model, and the second model is used for screening data associated with the first scene from the second data set; the second data set at least comprises first data, the second data set further comprises second data and / or third data, the second data is associated with historical communication network conditions, the third data is associated with analog communication network conditions, and the first scene comprises a scene associated with the first data.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a method for determining network performance indicators, a data filtering method, an apparatus, related equipment, a storage medium, and a computer program product. Background Technology

[0002] Among related technologies, the application of artificial intelligence (AI) technology in the construction of digital twin networks (DTN) can realize the simulation of complex network environments and improve the simulation speed.

[0003] However, in scenarios where AI technology is used to build DTN, there is a problem of low model performance. Summary of the Invention

[0004] To address related technical issues, embodiments of this application provide a method for determining network performance indicators, a data filtering method, an apparatus, related devices, a storage medium, and a computer program product.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a method for determining network performance indicators, applied to a digital twin network, the method comprising:

[0007] Acquire first data; determine a first model that matches the first data, wherein the first data is related to the real-time communication network situation;

[0008] Using the first matching model, the network performance metrics corresponding to the first data are determined; wherein,

[0009] The first model is optimized using a first dataset, which contains data obtained through a second model. The second model includes a model determined based on relevant information from the first model. The second model is used to filter data associated with a first scenario from the second dataset. The second dataset contains at least the first data and also includes second data and / or third data. The second data is associated with historical communication network conditions, and the third data is associated with simulated communication network conditions. The first scenario includes scenarios associated with the first data.

[0010] In the above scheme, determining the first model that matches the first dataset includes:

[0011] Using the first data, determine the relevant information for the first scenario;

[0012] Based on the relevant information of the first scenario, the first model is determined.

[0013] In the above scheme, determining the first model based on relevant information from the first scenario includes:

[0014] The relevant information of the first scenario is matched with the relevant information of the known scenario. If the relevant information of the known scenario is matched, the model corresponding to the relevant information of the known scenario is used as the first model.

[0015] or,

[0016] The relevant information of the first scenario is matched with the relevant information of known scenarios. If no relevant information of known scenarios is matched, the relevant information of the first scenario is used to determine the features of the first scenario. The first model is constructed using the features of the first scenario.

[0017] In the above scheme, constructing the first model using the features of the first scenario includes:

[0018] Based on the preset model architecture and combined with the characteristics of the first scenario, the first model is constructed.

[0019] The method in the above scheme further includes:

[0020] If no relevant information for the known scenario is found, the relevant information for the known scenario is updated using the first scenario and the constructed first model.

[0021] The method in the above scheme further includes:

[0022] Obtain the second dataset;

[0023] The second model is determined using relevant information from the first model;

[0024] The second model is used to filter out data related to the first scenario from the second dataset to obtain the first dataset.

[0025] In the above scheme, the second model is used to filter data related to the first scenario from the second dataset to obtain the first dataset, which includes:

[0026] For each data point in the second dataset, the second model is used to determine the loss value for that data.

[0027] The data in the second dataset whose loss values ​​satisfy the first condition are identified as the target data;

[0028] If the number of determined target data is less than a preset threshold, the second model is optimized using the determined target data; for data in the second dataset whose loss value does not meet the first condition, the loss value of the data is determined again using the optimized second model; data whose loss value meets the first condition again is also determined as target data; until the number of determined target data reaches the preset threshold; the first dataset is obtained using all target data.

[0029] In the above scheme, determining the data whose loss value in the second dataset meets the first condition as the target data includes:

[0030] For each data point in the second dataset, determine the validation loss.

[0031] The data in the second dataset whose loss values ​​satisfy the first condition and whose loss satisfies the second condition are identified as the target data.

[0032] In the above scheme, before using the second model to filter out data related to the first scene from the second dataset, the method further includes:

[0033] The second model is trained using the first data.

[0034] This application also provides a data filtering method applied to a digital twin network, including:

[0035] Obtain relevant information about a second dataset and a first model. The second dataset contains at least the first data and also includes the second data and / or the third data. The first data is related to the real-time communication network situation, the second data is associated with the historical communication network situation, and the third data is associated with the simulated communication network situation. The first model is used to determine the network performance index corresponding to the first data.

[0036] The second model is determined using the relevant information from the first model;

[0037] The second model is used to filter data associated with the first scenario from the second dataset to obtain the first dataset. The first scenario includes scenarios associated with the first data. The first dataset is used at least to optimize the first model.

[0038] In the above scheme, the second model is used to filter data related to the first scenario from the second dataset to obtain the first dataset, which includes:

[0039] For each data point in the second dataset, the second model is used to determine the loss value for that data.

[0040] The data in the second dataset whose loss values ​​satisfy the first condition are identified as the target data;

[0041] If the number of determined target data is less than a preset threshold, the second model is optimized using the determined target data; for data in the second dataset whose loss value does not meet the first condition, the loss value of the data is determined again using the optimized second model; data whose loss value meets the first condition again is also determined as target data; until the number of determined target data reaches the preset threshold; the first dataset is obtained using all target data.

[0042] In the above scheme, determining the data whose loss value in the second dataset meets the first condition as the target data includes:

[0043] For each data point in the second dataset, determine the validation loss.

[0044] The data in the second dataset whose loss values ​​satisfy the first condition and whose loss satisfies the second condition are identified as the target data.

[0045] In the above scheme, before using the second model to filter out data related to the first scene from the second dataset, the method further includes:

[0046] Obtain the first data;

[0047] The second model is trained using the first data.

[0048] This application also provides a network performance index determination device, applied to a digital twin network, comprising:

[0049] The first acquisition unit is used to acquire first data, which is related to the real-time communication network situation.

[0050] The first determining unit is configured to determine a first model that matches the first data; and to determine the network performance index corresponding to the first data using the matched first model.

[0051] An optimization unit is configured to optimize the first model using a first dataset, the first dataset containing data obtained through a second model, the second model including a model determined based on relevant information of the first model, the second model being used to filter data associated with a first scenario from the second dataset; the second dataset at least contains the first data, the second dataset also includes second data and / or third data, the second data being associated with historical communication network conditions, the third data being associated with simulated communication network conditions, and the first scenario including a scenario associated with the first data.

[0052] This application also provides a data filtering device applied to a digital twin network, comprising:

[0053] The second acquisition unit is used to acquire relevant information of the second dataset and the first model. The second dataset contains at least the first data and also includes the second data and / or the third data. The first data is related to the real-time communication network situation, the second data is associated with the historical communication network situation, and the third data is associated with the simulated communication network situation. The first model is used to determine the network performance index corresponding to the first data.

[0054] The second determining unit is used to determine the second model using the relevant information of the first model;

[0055] A filtering unit is used to filter data associated with a first scenario from the second dataset using the second model to obtain a first dataset, wherein the first scenario includes scenarios associated with the first data, and the first dataset is used at least to optimize the first model.

[0056] This application also provides an electronic device applied to a digital twin network, comprising:

[0057] A first communication interface is used to acquire first data, which is related to the real-time communication network status.

[0058] A first processor is configured to: determine a first model matching the first data; determine network performance metrics corresponding to the first data using the matched first model; and optimize the first model using a first dataset, wherein the first dataset contains data obtained through a second model, the second model includes a model determined based on relevant information of the first model, the second model being used to filter data associated with a first scenario from the second dataset; the second dataset contains at least the first data, the second dataset also includes second data and / or third data, the second data is associated with historical communication network conditions, the third data is associated with simulated communication network conditions, and the first scenario includes a scenario associated with the first data;

[0059] This application also provides an electronic device applied to a digital twin network, comprising:

[0060] The second communication interface is used to obtain relevant information about the second dataset and the first model. The second dataset contains at least the first data and also includes the second data and / or the third data. The first data is related to the real-time communication network situation, the second data is related to the historical communication network situation, and the third data is related to the simulated communication network situation. The first model is used to determine the network performance index corresponding to the first data.

[0061] A second processor is configured to determine a second model using relevant information from the first model; and to filter data associated with a first scenario from the second dataset using the second model to obtain a first dataset, wherein the first scenario includes scenarios associated with the first data, and the first dataset is used at least to optimize the first model.

[0062] This application also provides a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of any of the above methods.

[0063] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0064] The network performance index determination method, data filtering method, apparatus, related equipment, storage medium, and computer program products provided in this application embodiment obtain first data; determine a first model matching the first data, wherein the first data is related to real-time communication network conditions; and determine the network performance index corresponding to the first data using the matched first model. The first model is optimized using a first dataset, which includes data obtained through a second model. The second model includes a model determined based on relevant information from the first model, and is used to filter data associated with a first scenario from the second dataset. The second dataset includes at least the first data, and also includes second data and / or third data, wherein the second data is associated with historical communication network conditions, the third data is associated with simulated communication network conditions, and the first scenario includes scenarios associated with the first data. The solution provided in this application embodiment determines a first model based on data related to real-time communication network conditions, and uses filtered data related to scenarios corresponding to real-time communication network conditions to train the first model, effectively improving the performance of the first model and thus more accurately predicting network performance indexes of the communication network using the first model. Attached Figure Description

[0065] Figure 1 This is a flowchart illustrating the method for determining network performance indicators according to an embodiment of this application.

[0066] Figure 2 This is a flowchart illustrating the data filtering method in an embodiment of this application;

[0067] Figure 3 This application provides an example of a schematic diagram of the structure of a DTN system;

[0068] Figure 4This is a flowchart illustrating a model and data joint optimization method for DTN, serving as an application example of this application.

[0069] Figure 5 This is a schematic diagram of the network performance index determination device according to an embodiment of this application;

[0070] Figure 6 This is a schematic diagram of the data filtering device structure according to an embodiment of this application;

[0071] Figure 7 This is a schematic diagram of a related device structure according to an embodiment of this application;

[0072] Figure 8 This is a schematic diagram of another related device structure according to an embodiment of this application. Detailed Implementation

[0073] The present application will now be described in further detail with reference to the accompanying drawings and embodiments.

[0074] In related technologies, a real-time mirror of the physical network can be obtained by constructing a DTN. Therefore, the DTN can serve as a secure and cost-effective network performance evaluation environment, allowing network operators to assess network performance in various hypothetical scenarios. In other words, the DTN can enhance the systematic simulation, optimization, verification, and control capabilities lacking in physical networks. Here, constructing a DTN can also be understood as simulating a network environment.

[0075] To construct a real-time, lightweight (i.e., low resource overhead), and high-precision (i.e., high simulation accuracy) DTN, accurate network modeling is crucial. Currently, network modeling is typically performed using network simulators based on discrete event simulation. These simulators offer high accuracy; however, when the network size is large, the simulation process becomes very time-consuming. Furthermore, these simulators support limited scenarios, and the simulated network model differs somewhat from the real-world network environment.

[0076] With the development of AI technology, AI models can be used to simulate complex network environments, possessing capabilities such as analysis, prediction, and decision-making. Therefore, applying AI technology (i.e., AI-based network modeling methods) has become a promising option for network modeling. For example, AI-based network modeling methods can be applied to scenarios such as performance prediction, resource allocation, and task scheduling. Compared to network modeling methods using network simulators based on discrete event simulations, AI-based network modeling methods can significantly improve simulation speed; however, the performance (or accuracy or quality) of the resulting AI model may experience a certain degree of degradation.

[0077] The performance of an AI model depends primarily on two aspects: model architecture and data quality (i.e., the quality of training data). Based on this, some technologies improve the performance of AI models by designing novel neural network architectures (i.e., model architectures), which can also be understood as improving the accuracy of network modeling; while other technologies improve the performance of AI models (such as accuracy and generalization ability) by improving the quality of the data used to train the AI ​​model.

[0078] In practical applications, both the model architecture and training data in a Data Network (DTN) are crucial components, interconnected and interacting to form an organic whole. Therefore, improving the performance of DTN-related AI models by simultaneously addressing both model architecture and data quality is a pressing issue that needs to be addressed.

[0079] Based on this, in various embodiments of this application, a first model (or the model architecture of the first model) is determined based on data related to the real-time communication network situation, and the first model is trained using data related to the scenario corresponding to the real-time communication network situation obtained through screening. This can effectively improve the performance of the first model, thereby more accurately using the first model to predict the network performance indicators of the communication network.

[0080] This application provides a method for determining network performance indicators, applied to DTN, such as... Figure 1 As shown, the method includes:

[0081] Step 101: Obtain first data; determine a first model that matches the first data, wherein the first data is related to the real-time communication network situation;

[0082] Step 102: Using the first matching model, determine the network performance metrics corresponding to the first data; wherein,

[0083] The first model is optimized using a first dataset, which contains data obtained through a second model. The second model includes a model determined based on relevant information from the first model. The second model is used to filter data associated with a first scenario from the second dataset. The second dataset contains at least the first data and also includes second data and / or third data. The second data is associated with historical communication network conditions, and the third data is associated with simulated communication network conditions. The first scenario includes scenarios associated with the first data.

[0084] In practical applications, this method is applied to the DTN. Specifically, the method can be executed by a DTN-related server or an electronic device with corresponding functions. In this application embodiment, the name of the device executing the method is not limited, as long as its function is implemented. The device executing the method will be referred to as an electronic device.

[0085] In practical applications, to accurately determine network performance metrics (or accurately predict network performance), it is necessary to improve the performance of the model used for determining network performance metrics (i.e., the first model). Specifically, the model's performance can be improved in the following two ways:

[0086] Firstly, a first model that is more in line with the real-time communication network scenario (i.e., the first scenario) is constructed so that when the constructed first model analyzes the data in the real-time communication network scenario (which can also be understood as scenario data), it can more accurately determine the network performance indicators.

[0087] Secondly, the first model is trained using high-quality scene data to improve its ability to process scene data, thereby more accurately determining network performance metrics.

[0088] The first model can also be referred to as a DTN network performance model or a scenario model, and the name of the first model is not limited in this embodiment. High-quality scenario data can also be understood as data that is more relevant to real-time communication network scenarios (i.e., more closely fits real-time communication network scenarios) and can effectively improve the performance of the first model (e.g., accurate and diverse).

[0089] Regarding the first aspect mentioned above, in order to construct a first model that better fits the first scenario, in step 101, the electronic device needs to acquire data related to the real-time communication network situation (i.e., first data) to determine the first scenario. Specifically, the electronic device can acquire the first data through relevant data interfaces. This application embodiment does not limit the specific implementation method for acquiring the first data. The first data can also be called real-time network data; this application embodiment does not limit the name of the first data.

[0090] After acquiring the first data, the electronic device can perform scenario analysis on the first data to determine the first scenario (i.e., the real-time communication network scenario) corresponding to the first data, thereby determining the first model that conforms to the first scenario.

[0091] Based on this, in one embodiment, determining the first model that matches the first dataset includes:

[0092] Using the first data, determine the relevant information for the first scenario;

[0093] Based on the relevant information of the first scenario, the first model is determined.

[0094] In practical applications, the electronic device can utilize traffic classification technology and process the first data through a neural network to determine relevant information about the first scenario. Specifically, the relevant information about the first scenario may include scenario characteristics, traffic patterns, and service types.

[0095] For example, the electronic device can use traffic classification technology to divide the first data into one or more (one or more can also be understood as at least one) traffic patterns, and determine the first scenario based on the one or more traffic patterns obtained from the division. Here, multiple traffic patterns can also be referred to as a combination of multiple traffic patterns, and traffic patterns can also be understood as network traffic patterns. Specific traffic patterns may include: Constant Bit Rate (CBR), Multi-Burst (MB), Poisson distribution, etc.

[0096] After determining the relevant information of the first scenario, the electronic device can directly construct a first model that conforms to the first scenario.

[0097] Of course, when the electronic device stores one or more (or at least one) known scenarios and a model corresponding to each known scenario, the electronic device can first determine whether the first scenario belongs to a known scenario. If the first scenario belongs to a known scenario (i.e., matches a known scenario), the electronic device can directly use the model corresponding to the matched known scenario as the first model, thereby reducing the time required to build the model and improving processing efficiency. The known scenarios may include one or more pre-set scenarios, or scenarios determined using historical communication network data during the determination of network performance indicators. This application embodiment does not limit this.

[0098] Based on this, in one embodiment, determining the first model based on relevant information from the first scenario includes:

[0099] The relevant information of the first scenario is matched with the relevant information of a known scenario. If the relevant information of a known scenario is matched, the model corresponding to the relevant information of the known scenario is used as the first model.

[0100] In cases where the first scenario does not belong to a known scenario (i.e., does not match a known scenario), the DTN can construct a first model that conforms to the first scenario based on the features of the first scenario.

[0101] Based on this, in one embodiment, determining the first model based on relevant information from the first scenario includes:

[0102] The relevant information of the first scenario is matched with the relevant information of known scenarios. If no relevant information of known scenarios is matched, the relevant information of the first scenario is used to determine the features of the first scenario. The first model is constructed using the features of the first scenario.

[0103] In practical applications, the electronic device can design, extract, and select features for the first scenario through feature engineering. Specifically, the electronic device can utilize feature selection techniques in feature engineering to select a suitable feature set for constructing the first model. This avoids an excessive number of features in the first model, which would increase its complexity and potentially lead to overfitting; conversely, it avoids an insufficient number of features, which would result in underfitting. In other words, by selecting a suitable feature set through feature selection techniques, the first model constructed using this selected feature set has appropriate complexity, strong transferability, and is easy to train and interpret. Specific feature selection methods within the feature selection techniques can include filtering, wrapping, or embedding methods, among others.

[0104] For example, assuming the first scenario corresponds to two traffic modes, CBR and MB, the electronic device can determine the characteristics of the first scenario (which can also be understood as scenario-specific characteristics) including the mean of packet transmission interval (which can be expressed as Flow Inter-Packet Gap mean, or simply Flow-IPG-mean), the variance of packet transmission interval (which can be expressed as Flow-IPG-var or Flow-IPG-variance), and the average packet transmission rate (which can be expressed as Flow-on-rate).

[0105] After determining the features of the first scene, the electronic device can add the features of the first scene to a preset model architecture to obtain the first model.

[0106] Based on this, in one embodiment, constructing the first model using the features of the first scene includes:

[0107] Based on the preset model architecture and combined with the characteristics of the first scenario, the first model is constructed.

[0108] In practical applications, the preset model can also be called a general model. The process of constructing the first model using the preset model architecture can be understood as deriving the first model from the general model. The preset model includes models trained based on a large amount of historical data associated with real communication network conditions and simulated data generated from data simulation. This application does not limit the name of the preset model.

[0109] For example, assuming the preset model is a network performance model based on a graph neural network (GNN), with network state as input and flow-level network performance metrics (such as latency, jitter, packet loss rate, etc.) as output, the structure of the first model can adopt the same model architecture (i.e., GNN-based network performance model architecture) and input and output types as the preset model. Meanwhile, assuming the first scenario can be described using the states of three feature vectors—flow F, queue Q, and link L—then a circular dependency relationship can be defined between the three feature vectors in the first model, specifically expressed as formula (1):

[0110] h f =G f (h q ,h l ),h q =G q (h f ),h l =G l (h q (1)

[0111] Among them, h f The implicit features representing the state of flow F (which can also be understood as scene features), h q The implicit feature representing the state of queue Q, h l The implicit features representing the state of link L, G f G q G l This represents the unknown function that needs to be determined (i.e., the parameters related to the features of the first scene that need to be determined).

[0112] After determining the first model, if the first scene does not belong to a known scene (or can be understood as the first scene being an unknown scene), the electronic device can treat the first scene as a known scene to improve the efficiency of subsequently determining the first model.

[0113] Based on this, in one embodiment, the method may further include:

[0114] If no relevant information for the known scenario is found, the relevant information for the known scenario is updated using the first scenario and the constructed first model.

[0115] As can be seen from the above description, regarding the first aspect, the electronic device can construct a first model that matches (or conforms to) the first scenario, thereby improving the performance of the first model in processing the first data corresponding to the first scenario.

[0116] Regarding the second aspect mentioned above, in order to obtain high-quality scene data (i.e., the first dataset), the electronic device needs to acquire a second dataset, and then filter out high-quality scene data associated with the first scene from the second dataset. Specifically, the electronic device can acquire the second dataset through a relevant data interface. This application embodiment does not limit the specific implementation method for acquiring the second dataset. The second dataset can also be called a candidate dataset or a multi-source candidate dataset; the second dataset includes first data, second data, and / or third data; the second data can also be called historical data, specifically including historical data associated with real communication network conditions; the third data can also be called simulated data, specifically including network condition data generated through a network simulator and / or a generative AI model. This application embodiment does not limit the names of the second data, the third data, and the second dataset.

[0117] After obtaining the second dataset, the electronic device needs to determine a second model, and then use the second model to filter the second dataset to obtain the first dataset. Specifically, the electronic device can determine the second model based on the first model, so that when filtering data using the second model, data that is more consistent with the first scenario can be filtered out. The second model can also be called a filtering model; this application embodiment does not limit the name of the second model.

[0118] Based on this, in one embodiment, the method may further include:

[0119] Obtain the second dataset;

[0120] The second model is determined using relevant information from the first model;

[0121] The second model is used to filter out data related to the first scenario from the second dataset to obtain the first dataset.

[0122] In practical applications, the relevant information of the first model may specifically include the feature information of the first model, such as feature information related to the first scene. The electronic device can use the feature information of the first model to determine the second model so that the data obtained after being filtered by the second model (i.e., the first dataset) can be better used to determine the feature parameters of the first model (i.e., optimize (or train) the first model).

[0123] Specifically, the second model can be obtained by adjusting the output layer of the first model. In other words, the architecture and parameters of the second model can be the same as the basic architecture (or architecture other than the output layer) and parameters of the first model, with the only difference being the output layer.

[0124] After determining the second model, the electronic device can use the first data as a seed sample and use the seed sample to train the initial second model.

[0125] Based on this, in one embodiment, before filtering out the data associated with the first scene from the second dataset using the second model, the method may further include:

[0126] The second model is trained using the first data.

[0127] In practical applications, the electronic device can use the second model to calculate the loss value (which can be expressed as loss) for each data in the second dataset, and use the data whose loss value meets the conditions as the data in the first dataset.

[0128] Based on this, in one embodiment, the second model is used to filter data associated with the first scene from the second dataset to obtain a first dataset, including:

[0129] For each data point in the second dataset, the second model is used to determine the loss value for that data.

[0130] The data in the second dataset whose loss values ​​satisfy the first condition are identified as the target data;

[0131] If the number of determined target data is less than a preset threshold, the second model is optimized using the determined target data; for data in the second dataset whose loss value does not meet the first condition, the loss value of the data is determined again using the optimized second model; data whose loss value meets the first condition again is also determined as target data; until the number of determined target data reaches the preset threshold; the first dataset is obtained using all target data.

[0132] Specifically, the electronic device infers each data point in the second dataset using the second model to obtain a loss value for each data point. A lower loss value indicates higher accuracy of the second model's inference of that data. After obtaining the loss value for each data point, the electronic device can sort the data in the second dataset according to the loss value and use formula (2) to process the top N data points with the lowest loss values. j Data, i.e., calculating N j The probability that each data point in N data points is selected as the target data, N j Integers greater than or equal to 1:

[0133]

[0134] Among them, P i Represents the first N jThe probability that the i-th sample out of a set of samples is identified as the target data, where P1 represents the probability that the data with the lowest loss value is selected as the target data. This indicates the Nth loss value after sorting from low to high. j The probability that a data point is selected as the target data, γ iter The selection ratio represents the ratio between P1 and P2. The ratio can be set according to actual needs.

[0135] Determine the top N with the lowest loss values j After determining the probability of each data point being selected as the target data from the given data, the electronic device can select the top N data points with the lowest loss values ​​based on a determined probability. j J target data points are selected from a set of data. These J data points can also be referred to as simple data or simple samples, where J is greater than or equal to 1 and less than or equal to N. j Integers.

[0136] Meanwhile, the electronic device can use formula (3) to calculate the top N with the highest loss values. k The probability that N data points are selected as target data. k Integers greater than or equal to 1:

[0137]

[0138] Among them, P Nk This indicates the Nth loss value after sorting from highest to lowest. k The probability that a data point is selected as the target data is also used to characterize the relationship between P1 and P2. Nk The ratio of .

[0139] Determine the top N with the highest loss values k After determining the probability of each data point being selected as the target data from the given data, the electronic device can select the top N data points with the highest loss values ​​based on a determined probability. k Selecting K target data points from a given set of data ensures sample diversity. K is an integer greater than or equal to 1 and less than or equal to N. k Integers.

[0140] In practical applications, the K data points determined using formula (3) may include the following two types of data:

[0141] First type of data: The data loss value is high due to the insufficient performance of the second model;

[0142] The second type of data: data with high loss values ​​due to low data quality.

[0143] The first type of data can also be called difficult data or difficult samples, and the second type of data can also be called out-of-distribution (OOD) data or OOD samples.

[0144] In practical applications, low-quality OOD data should be filtered out from the target data, and the remaining target data should be used as the data in the first dataset to ensure the data quality in the first dataset.

[0145] Based on this, in the embodiments of this application, an impact-based approach is used to determine whether the data is OOD data. That is, OOD data is identified by evaluating the impact of the data on the first model.

[0146] Specifically, in one embodiment, determining the data in the second dataset whose loss value satisfies the first condition as target data includes:

[0147] For each data point in the second dataset, determine the validation loss.

[0148] The data in the second dataset whose loss values ​​satisfy the first condition and whose loss satisfies the second condition are identified as the target data.

[0149] Specifically, determining the validation loss of the data may include: calculating the loss value obtained by the first model processing the dataset containing the data, and calculating the loss value obtained by the first model processing the dataset not containing the data, calculating the difference between the two loss values, and taking the ratio between the difference and the loss value obtained by the first model processing the dataset not containing the data as the validation loss of the data. This can be specifically expressed as formula (4):

[0150]

[0151] Among them, l val () represents the function that calculates the loss value of the first model, and is associated with the first model, M. w M represents the dataset containing this data. w / o θ represents a dataset that does not contain this data. iter The ratio represents a preset threshold. If the ratio is greater than or equal to the preset threshold, it indicates that using this data as training data for the first model will have a significant error impact, i.e., the data is OOD data; if the ratio is less than the preset threshold, the data is not OOD data.

[0152] In practical applications, the electronic device can identify target data from the K data points determined by formula (3) whose corresponding ratio is less than the preset threshold. In this case, the data's verification loss satisfying the second condition can also be understood as the data's verification loss not satisfying formula (4), or the data's verification loss being less than the preset threshold. The above process can also be understood as selecting difficult data from the K data points and removing OOD data.

[0153] In practical applications, the electronic device uses formulas (2), (3), and (4) to determine simple and difficult data, and then uses the simple and difficult data as target data. If the number of target data is greater than or equal to a preset threshold (which can also be understood as meeting the scale requirements), the electronic device can directly use all target data as the first dataset; if the number of data in the first dataset is less than the preset threshold, the electronic device can first use all the currently determined target data to train the second model (i.e. optimize the second model) to improve the filtering performance of the second model, and then use the trained second model to determine the loss value of other data in the second dataset that have not been determined as target data (i.e., data whose loss value does not meet the first condition), and determine the target data according to the above process (which can also be understood as iterating) until the number of determined target data is greater than or equal to the preset threshold, and use all target data as the first dataset.

[0154] After determining the first dataset, the electronic device can use the first dataset to optimize the first model, such as by tuning the parameters of the first model. This application embodiment does not limit the specific implementation of optimizing the first model using the first dataset.

[0155] As can be seen from the above description, regarding the second aspect, the electronic device can filter high-quality scene data from the second dataset and use the filtered data to optimize the first model, thereby improving the performance of the first model. In practical applications, the electronic device can also utilize the first data to optimize the first model, further improving its performance.

[0156] In practical applications, after improving the performance of the first model through the above two aspects, in step 102, the electronic device can use the first model to infer network performance indicators.

[0157] Specifically, the electronic device can represent the first data as a network state (such as network topology, traffic matrix, routing configuration, etc.), and use the network state as input to the first model to obtain the predicted results of the flow-level network performance indicators output by the first model. Before inputting the network state into the first model, the electronic device can also perform feature extraction on the network state and input the extracted feature vector into the first model.

[0158] In practical applications, the processes corresponding to the first and second aspects described above can be executed by the same electronic device in the DTN, or the processes corresponding to the first and second aspects described above can be executed by two electronic devices in the DTN respectively.

[0159] Accordingly, when the processes corresponding to the first and second aspects described above are executed by two electronic devices in the DTN respectively, embodiments of this application also provide a network element function configuration method applied to the DTN, such as... Figure 2 As shown, the method includes:

[0160] Step 201: Obtain relevant information about the second dataset and the first model. The second dataset contains at least the first data and also includes the second data and / or the third data. The first data is related to the real-time communication network situation, the second data is related to the historical communication network situation, and the third data is related to the simulated communication network situation. The first model is used to determine the network performance index corresponding to the first data.

[0161] Step 202: Determine the second model using the relevant information from the first model;

[0162] Step 203: Use the second model to filter out data associated with the first scenario from the second dataset to obtain the first dataset. The first scenario includes scenarios associated with the first data. The first dataset is used at least to optimize the first model.

[0163] In practical applications, when two electronic devices in the DTN respectively execute the processes corresponding to the first and second aspects described above, this method is used to implement the process corresponding to the second aspect. Specifically, the method can be executed by a DTN-related server or an electronic device with corresponding functions. This application embodiment does not limit the name of the device executing the method, as long as its function is implemented. Hereinafter, the electronic device implementing the process corresponding to the first aspect will be referred to as the first device, and the electronic device implementing the process corresponding to the second aspect will be referred to as the second device.

[0164] In practical applications, the second device can receive relevant information about the first model sent by the first device, and thus use this information to determine the second model. Simultaneously, after obtaining the first dataset, the second device can send the first dataset back to the first device, allowing the first device to optimize the first model using the dataset.

[0165] In practical applications, the specific processing procedures for steps 201 to 203 can be referred to the relevant description in the second aspect above, and will not be repeated here.

[0166] The network performance index determination method provided in this application embodiment obtains first data; determines a first model matching the first data, wherein the first data is related to real-time communication network conditions; and determines the network performance index corresponding to the first data using the matched first model. The method optimizes the first model using a first dataset, which includes data obtained through a second model. The second model includes a model determined based on relevant information from the first model, and is used to filter data associated with a first scenario from the second dataset. The second dataset includes at least the first data, and also includes second data and / or third data, wherein the second data is associated with historical communication network conditions, the third data is associated with simulated communication network conditions, and the first scenario includes scenarios associated with the first data. The solution provided in this application embodiment determines a first model based on data related to real-time communication network conditions, and uses filtered data related to scenarios corresponding to real-time communication network conditions to train the first model. This effectively improves the performance of the first model, thereby more accurately predicting network performance indexes of the communication network using the first model.

[0167] The following section provides a more detailed description of this application with reference to application examples.

[0168] This application example provides a DTN system, such as Figure 3 As shown, this includes model optimization flow and data optimization flow. The goal of the model optimization flow is to continuously improve the DTN network performance model (i.e., based on the feature changes in real-time data) Figure 3 The scene model in the DTN (Digital Network Technology) can also be understood as the performance of the AI ​​model (e.g., prediction accuracy). The model optimization stream optimizes the model architecture of the DTN network performance model. Simultaneously, the data optimization stream aims to select high-quality scene data from multi-source candidate data; that is, it selects training data that can improve the performance of the DTN network performance model. Therefore, in practical applications, using the aforementioned DTN system, we can start from both network model architecture design and data quality, allowing these two aspects to complement each other and jointly improve the performance of the DTN network performance model.

[0169] Based on the above system, this application example provides a joint optimization method for models and data in DTN, such as... Figure 4 As shown, the specific steps include:

[0170] Step 401: Real-time network data scenario analysis;

[0171] In practical applications, a neural network (specifically, traffic classification technology can be applied) is used to perform scenario analysis on real-time network data (i.e., the first data mentioned above, or real-time network data). If the scenario corresponding to the real-time network data (i.e., the first scenario mentioned above) is a known scenario, the model of the known scenario is used as the scenario model, and step 403 is executed. If the scenario corresponding to the real-time network data is an unknown scenario (which can also be understood as a new scenario), step 402 is executed to determine the scenario model.

[0172] Step 402: When the scenario corresponding to the real-time network data is an unknown scenario, feature selection is performed and a new scenario model is derived;

[0173] In practical applications, feature engineering techniques are used to design and select scene features (i.e., the features of the first scene mentioned above) for scenarios corresponding to real-time network data. After obtaining the scene features, they are added to a general model to derive a scene model, which has a neural network architecture that is basically the same as the general model. Specifically, the general model can include a GNN-based network performance model. In this way, different features can be designed for different scenarios, resulting in scene models that can better adapt to complex real-world networks.

[0174] Step 403: Train the scene model;

[0175] In practical applications, real-time network data and high-quality scene data obtained through data optimization and filtering can be used to train and fine-tune the scene model. Specifically, real-time network data and high-quality scene data can be used as training data, represented as network topology, routing configuration, and traffic matrix. Then, feature vectors can be extracted to obtain feature vectors, such as feature vectors representing flow F, queue Q, and link L. The extracted feature vectors can then be used to train or fine-tune the scene model.

[0176] In practical applications, to obtain high-quality scene data, steps 404 to 406 are executed. After the scene model is trained, step 407 is executed.

[0177] Step 404: Determine the initial screening model using the scenario model;

[0178] In practical applications, relevant information about the scenario model (such as feature dimensions) can be used to determine the selection model, so that the selection model can accurately select data suitable for training the scenario model.

[0179] Step 405: Use real-time network data as seed samples and train the initial screening model using the seed samples to obtain the trained screening model;

[0180] Step 406: Obtain multi-source candidate data (i.e., the second dataset mentioned above), and use the trained screening model to automatically select high-quality scene data (i.e., the first dataset mentioned above) from the multi-source candidate data.

[0181] Among them, the multi-source candidate data includes: real-time network data, historical data (which can also be understood as historical real data, i.e., the second data mentioned above), and simulated data (which can also be understood as data generated by network simulators and generative AI models, i.e., the third data mentioned above).

[0182] For example, assuming Dc represents multi-source candidate data, Ds represents seed samples (i.e., real-time network data), M represents the screening model, and Dt represents the selected data (i.e., the target data mentioned above), the algorithmic process for screening (or selecting) high-quality scene data can include:

[0183] Using Ds as the initial Dt, the following loop is then performed:

[0184] M is trained using Dt;

[0185] Remove Dt from Dc to obtain the data to be filtered, Dr, and calculate the number of data (i.e. the number of samples) Nr contained in Dr;

[0186] Using M to perform inference on Dr, we obtain the loss value for each data point (which can also be understood as a list of loss values, specifically represented as...). Sort the data in Dr according to the loss value;

[0187] Use the above formula (2) to select simple samples from Dr;

[0188] Use the above formula (3) to select difficult samples and OOD samples from Dr, and use the above formula (4) to remove OOD samples;

[0189] Add simple and difficult samples to Dt;

[0190] Determine if the size of Dt (i.e. the number of data in Dt) meets the requirements; if it does, end the loop and use Dt as high-quality scene data for training the scene model; if it does not meet the requirements, proceed to the next loop.

[0191] In this way, by continuously iterating and optimizing M, high-precision, diverse, and high-quality scene data that fits the actual situation of real data can be automatically selected from Dc, so that the DTN network performance scene model can be effectively trained using high-quality scene data.

[0192] Step 407: Use the scenario model for reasoning.

[0193] In practical applications, the trained scene model can be used to process the real-time network data after feature extraction and output prediction results of network performance indicators at the flow level (such as latency, jitter, packet loss rate, etc.).

[0194] The solution provided in this application example can address the issues of insufficient accuracy, low performance, and limited scenarios in existing network modeling methods by focusing on both neural network architecture design and data quality.

[0195] Meanwhile, the model optimization stream can identify specific scenarios based on the feature changes of real-time data, and derive different scenario models based on the general model for different scenarios, continuously improving the prediction accuracy of the scenario model;

[0196] Meanwhile, the data optimization stream can filter out high-quality scene data from multi-source candidate data, helping scene models to be effectively trained with high-quality data, thereby improving the prediction accuracy of scene models.

[0197] To implement the network performance index determination method of this application embodiment, this application embodiment also provides a network performance index determination device, applied to a digital twin network, and installed on an electronic device, such as... Figure 5 As shown, the device includes:

[0198] The first acquisition unit 501 is used to acquire first data, which is related to the real-time communication network situation;

[0199] The first determining unit 502 is used to determine a first model that matches the first data; and to determine the network performance index corresponding to the first data using the matched first model.

[0200] The optimization unit 503 is used to optimize the first model using a first dataset, the first dataset containing data obtained through a second model, the second model including a model determined based on relevant information of the first model, the second model being used to filter data associated with a first scenario from the second dataset; the second dataset at least contains the first data, the second dataset also includes second data and / or third data, the second data being associated with historical communication network conditions, the third data being associated with simulated communication network conditions, and the first scenario including a scenario associated with the first data.

[0201] In one embodiment, the first determining unit 502 is specifically used for:

[0202] Using the first data, determine the relevant information for the first scenario;

[0203] Based on the relevant information of the first scenario, the first model is determined.

[0204] In one embodiment, the first determining unit 502 is specifically used for:

[0205] The relevant information of the first scenario is matched with the relevant information of the known scenario. If the relevant information of the known scenario is matched, the model corresponding to the relevant information of the known scenario is used as the first model.

[0206] or,

[0207] The relevant information of the first scenario is matched with the relevant information of known scenarios. If no relevant information of known scenarios is matched, the relevant information of the first scenario is used to determine the features of the first scenario. The first model is constructed using the features of the first scenario.

[0208] In one embodiment, the first determining unit 502 is specifically used for:

[0209] Based on the preset model architecture and combined with the characteristics of the first scenario, the first model is constructed.

[0210] In one embodiment, the first determining unit 502 is further configured to:

[0211] If no relevant information for the known scenario is found, the relevant information for the known scenario is updated using the first scenario and the constructed first model.

[0212] In one embodiment, the first acquisition unit 501 is further configured to:

[0213] Obtain the second dataset;

[0214] The first determining unit 502 is further configured to:

[0215] The second model is determined using relevant information from the first model;

[0216] The second model is used to filter out data related to the first scenario from the second dataset to obtain the first dataset.

[0217] In one embodiment, the first determining unit 502 is specifically used for:

[0218] For each data point in the second dataset, the second model is used to determine the loss value for that data.

[0219] The data in the second dataset whose loss values ​​satisfy the first condition are identified as the target data;

[0220] If the number of determined target data is less than a preset threshold, the second model is optimized using the determined target data; for data in the second dataset whose loss value does not meet the first condition, the loss value of the data is determined again using the optimized second model; data whose loss value meets the first condition again is also determined as target data; until the number of determined target data reaches the preset threshold; the first dataset is obtained using all target data.

[0221] In one embodiment, the first determining unit 502 is specifically used for:

[0222] For each data point in the second dataset, determine the validation loss.

[0223] The data in the second dataset whose loss values ​​satisfy the first condition and whose loss satisfies the second condition are identified as the target data.

[0224] In one embodiment, before using the second model to filter out data associated with the first scene from the second dataset, the optimization unit 503 is further configured to:

[0225] The second model is trained using the first data.

[0226] In practical applications, the first acquisition unit 501 can be implemented by the processor in the network performance index determination device in combination with the communication interface, and the first determination unit 502 and the optimization unit 503 can be implemented by the processor in the network performance index determination device.

[0227] It should be noted that the network performance index determination device provided in the above embodiments is only illustrated by the division of the above-described program units when determining network performance indexes. In practical applications, the above processing can be assigned to different program units as needed, that is, the internal structure of the device can be divided into different program units to complete all or part of the processing described above. In addition, the network performance index determination device and the network performance index determination method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0228] To implement the data filtering method of this application embodiment, this application embodiment also provides a data filtering device, applied to a digital twin network, and installed on an electronic device, such as... Figure 6 As shown, the device includes:

[0229] The second acquisition unit 601 is used to acquire relevant information of the second dataset and the first model. The second dataset contains at least the first data and also includes the second data and / or the third data. The first data is related to the real-time communication network situation, the second data is associated with the historical communication network situation, and the third data is associated with the simulated communication network situation. The first model is used to determine the network performance index corresponding to the first data.

[0230] The second determining unit 602 is used to determine the second model using the relevant information of the first model;

[0231] The filtering unit 603 is used to filter data associated with the first scenario from the second dataset using the second model to obtain a first dataset, wherein the first scenario includes scenarios associated with the first data, and the first dataset is used at least to optimize the first model.

[0232] In one embodiment, the filtering unit 603 is specifically used for:

[0233] For each data point in the second dataset, the second model is used to determine the loss value for that data.

[0234] The data in the second dataset whose loss values ​​satisfy the first condition are identified as the target data;

[0235] If the number of determined target data is less than a preset threshold, the second model is optimized using the determined target data; for data in the second dataset whose loss value does not meet the first condition, the loss value of the data is determined again using the optimized second model; data whose loss value meets the first condition again is also determined as target data; until the number of determined target data reaches the preset threshold; the first dataset is obtained using all target data.

[0236] In one embodiment, the filtering unit 603 is specifically used for:

[0237] For each data point in the second dataset, determine the validation loss.

[0238] The data in the second dataset whose loss values ​​satisfy the first condition and whose loss satisfies the second condition are identified as the target data.

[0239] In one embodiment, before filtering out data associated with the first scene from the second dataset using the second model, the second acquisition unit 601 is further configured to:

[0240] Obtain the first data;

[0241] The second determining unit 602 is further configured to:

[0242] The second model is trained using the first data.

[0243] In practical applications, the second acquisition unit 601 can be implemented by the processor in the data filtering device in conjunction with the communication interface, and the second determination unit 602 and the filtering unit 603 can be implemented by the processor in the data filtering device.

[0244] It should be noted that the data filtering device provided in the above embodiments is only illustrated by the division of the above-described program units. In practical applications, the above processing can be assigned to different program units as needed, that is, the internal structure of the device can be divided into different program units to complete all or part of the processing described above. In addition, the data filtering device and the data filtering method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0245] Based on the hardware implementation of the above program modules, and in order to implement the network performance index determination method of this application embodiment, this application embodiment also provides an electronic device, such as... Figure 7 As shown, the electronic device 700 includes:

[0246] The first communication interface 701 is capable of exchanging information with other devices;

[0247] The first processor 702 is connected to the first communication interface 701 to enable information interaction with other devices and to execute the methods provided by one or more of the above-mentioned technical solutions when running a computer program;

[0248] The computer program is stored in the first memory 703.

[0249] Specifically, the first communication interface 701 is used to acquire first data, which is related to the real-time communication network situation;

[0250] The first processor 702 is configured to: determine a first model matching the first data; determine network performance metrics corresponding to the first data using the matched first model; and optimize the first model using a first dataset, wherein the first dataset contains data obtained through a second model, the second model includes a model determined based on relevant information of the first model, the second model being used to filter data associated with a first scenario from the second dataset; the second dataset contains at least the first data, the second dataset also includes second data and / or third data, the second data is associated with historical communication network conditions, the third data is associated with simulated communication network conditions, and the first scenario includes a scenario associated with the first data.

[0251] In one embodiment, the first processor 702 is specifically used for:

[0252] Using the first data, determine the relevant information for the first scenario;

[0253] Based on the relevant information of the first scenario, the first model is determined.

[0254] In one embodiment, the first processor 702 is specifically used for:

[0255] The relevant information of the first scenario is matched with the relevant information of the known scenario. If the relevant information of the known scenario is matched, the model corresponding to the relevant information of the known scenario is used as the first model.

[0256] or,

[0257] The relevant information of the first scenario is matched with the relevant information of known scenarios. If no relevant information of known scenarios is matched, the relevant information of the first scenario is used to determine the features of the first scenario. The first model is constructed using the features of the first scenario.

[0258] In one embodiment, the first processor 702 is specifically used for:

[0259] Based on the preset model architecture and combined with the characteristics of the first scenario, the first model is constructed.

[0260] In one embodiment, the first processor 702 is further configured to:

[0261] If no relevant information for the known scenario is found, the relevant information for the known scenario is updated using the first scenario and the constructed first model.

[0262] In one embodiment, the first communication interface 701 is further configured to:

[0263] Obtain the second dataset;

[0264] The first processor 702 is further configured to:

[0265] The second model is determined using relevant information from the first model;

[0266] The second model is used to filter out data related to the first scenario from the second dataset to obtain the first dataset.

[0267] In one embodiment, the first processor 702 is specifically used for:

[0268] For each data point in the second dataset, the second model is used to determine the loss value for that data.

[0269] The data in the second dataset whose loss values ​​satisfy the first condition are identified as the target data;

[0270] If the number of determined target data is less than a preset threshold, the second model is optimized using the determined target data; for data in the second dataset whose loss value does not meet the first condition, the loss value of the data is determined again using the optimized second model; data whose loss value meets the first condition again is also determined as target data; until the number of determined target data reaches the preset threshold; the first dataset is obtained using all target data.

[0271] In one embodiment, the first processor 702 is specifically used for:

[0272] For each data point in the second dataset, determine the validation loss.

[0273] The data in the second dataset whose loss values ​​satisfy the first condition and whose loss satisfies the second condition are identified as the target data.

[0274] In one embodiment, before filtering data associated with the first scene from the second dataset using the second model, the first processor 702 is further configured to:

[0275] The second model is trained using the first data.

[0276] It should be noted that the specific processing procedures of the first processor 702 and the first communication interface 701 can be understood by referring to the above method.

[0277] Of course, in practical applications, the various components in electronic device 700 are coupled together through bus system 704. It can be understood that bus system 704 is used to realize the connection and communication between these components. In addition to a data bus, bus system 704 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in... Figure 7 The general designated all buses as Bus System 704.

[0278] The first memory 703 in this embodiment is used to store various types of data to support the operation of the electronic device 700. Examples of such data include any computer program used to operate on the electronic device 700.

[0279] The methods disclosed in the embodiments of this application can be applied to the first processor 702, or implemented by the first processor 702. The first processor 702 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware or by instructions in the form of software in the first processor 702. The first processor 702 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The first processor 702 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in the first memory 703. The first processor 702 reads the information in the first memory 703 and completes the steps of the aforementioned method in combination with its hardware.

[0280] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0281] Based on the hardware implementation of the above program modules, and in order to implement the data filtering method of this application embodiment, this application embodiment also provides an electronic device, such as... Figure 8 As shown, the electronic device 800 includes:

[0282] The second communication interface 801 is capable of exchanging information with other devices;

[0283] The second processor 802 is connected to the second communication interface 801 to enable information interaction with the electronic device and to execute the methods provided by one or more technical solutions on the electronic device side when running a computer program.

[0284] The computer program is stored in the second memory 803.

[0285] Specifically, the second communication interface 801 is used for:

[0286] Obtain relevant information about a second dataset and a first model. The second dataset contains at least the first data and also includes the second data and / or the third data. The first data is related to the real-time communication network situation, the second data is associated with the historical communication network situation, and the third data is associated with the simulated communication network situation. The first model is used to determine the network performance index corresponding to the first data.

[0287] The second processor 802 is used for:

[0288] A second model is determined using relevant information from the first model; and a first dataset is obtained by filtering data associated with a first scenario from the second dataset using the second model, wherein the first scenario includes scenarios associated with the first data, and the first dataset is used at least to optimize the first model.

[0289] In one embodiment, the second processor 802 is specifically used for:

[0290] For each data point in the second dataset, the second model is used to determine the loss value for that data.

[0291] The data in the second dataset whose loss values ​​satisfy the first condition are identified as the target data;

[0292] If the number of determined target data is less than a preset threshold, the second model is optimized using the determined target data; for data in the second dataset whose loss value does not meet the first condition, the loss value of the data is determined again using the optimized second model; data whose loss value meets the first condition again is also determined as target data; until the number of determined target data reaches the preset threshold; the first dataset is obtained using all target data.

[0293] In one embodiment, the second processor 802 is specifically used for:

[0294] For each data point in the second dataset, determine the validation loss.

[0295] The data in the second dataset whose loss values ​​satisfy the first condition and whose loss satisfies the second condition are identified as the target data.

[0296] In one embodiment, before filtering data associated with the first scene from the second dataset using the second model, the second communication interface 801 is further configured to:

[0297] Obtain the first data;

[0298] The second processor 802 is also used for:

[0299] The second model is trained using the first data.

[0300] It should be noted that the specific processing procedures of the second processor 802 and the second communication interface 801 can be understood by referring to the above method.

[0301] Of course, in practical applications, the various components in electronic device 800 are coupled together through bus system 804. It can be understood that bus system 804 is used to realize the connection and communication between these components. In addition to a data bus, bus system 804 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in... Figure 8 The general labeled all buses as Bus System 804.

[0302] The second memory 803 in this embodiment is used to store various types of data to support the operation of the electronic device 800. Examples of such data include any computer program used to operate on the electronic device 800.

[0303] The methods disclosed in the embodiments of this application can be applied to the second processor 802, or implemented by the second processor 802. The second processor 802 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware or by instructions in the form of software in the second processor 802. The second processor 802 may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The second processor 802 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in the second memory 803. The second processor 802 reads the information in the second memory 803 and completes the steps of the aforementioned method in combination with its hardware.

[0304] In an exemplary embodiment, the electronic device 800 may be implemented by one or more ASICs, DSPs, PLDs, CPLDs, FPGAs, general-purpose processors, controllers, MCUs, microprocessors, or other electronic components to perform the aforementioned method.

[0305] It is understood that the memories (first memory 703, second memory 803) in the embodiments of this application can be volatile memories or non-volatile memories, or both. Non-volatile memories can be read-only memories (ROM), programmable read-only memories (PROM), erasable programmable read-only memories (EPROM), electrically erasable programmable read-only memories (EEPROM), magnetic random access memories (FRAM), flash memories, magnetic surface memories, optical discs, or compact disc read-only memories (CD-ROM); magnetic surface memories can be disk storage or magnetic tape storage. Volatile memories can be random access memories (RAM), which are used as external caches. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.

[0306] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium. For example, it may include a first memory 703 storing a computer program, which can be executed by a first processor 702 of an electronic device 700 to complete the steps described in the method on the electronic device 700 side. Another example is a second memory 803 storing a computer program, which can be executed by a second processor 802 of an electronic device 800 to complete the steps described in the method on the electronic device 800 side. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0307] In an exemplary embodiment, this application also provides a computer program product, including a computer program that can be executed by a first processor 702 of an electronic device 700 to complete the steps of the aforementioned transmitting end-side method, or the computer program can be executed by a second processor 802 of an electronic device 800 to complete the steps of the aforementioned electronic device-side method.

[0308] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0309] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0310] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. A method of determining a network performance indicator, characterized by, Applied to digital twin networks, the method includes: Acquire first data; determine a first model that matches the first data, wherein the first data is related to the real-time communication network situation; Using the first matching model, the network performance metrics corresponding to the first data are determined; wherein, The first model is optimized using a first dataset, which contains data obtained through a second model. The second model includes a model determined based on relevant information from the first model. The second model is used to filter data associated with a first scenario from the second dataset. The second dataset contains at least the first data and also includes second data and / or third data. The second data is associated with historical communication network conditions, and the third data is associated with simulated communication network conditions. The first scenario includes scenarios associated with the first data.

2. The method of claim 1, wherein, The determination of the first model matching the first dataset includes: Using the first data, determine the relevant information for the first scenario; Based on the relevant information of the first scenario, the first model is determined.

3. The method of claim 2, wherein, Determining the first model based on relevant information from the first scenario includes: The relevant information of the first scenario is matched with the relevant information of the known scenario. If the relevant information of the known scenario is matched, the model corresponding to the relevant information of the known scenario is used as the first model. or, The relevant information of the first scenario is matched with the relevant information of known scenarios. If no relevant information of known scenarios is matched, the relevant information of the first scenario is used to determine the features of the first scenario. The first model is constructed using the features of the first scenario.

4. The method of claim 3, wherein, The step of constructing the first model using the features of the first scenario includes: Based on the preset model architecture and combined with the characteristics of the first scenario, the first model is constructed.

5. The method of claim 3, wherein, The method further includes: If no relevant information for the known scenario is found, the relevant information for the known scenario is updated using the first scenario and the constructed first model.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Obtain the second dataset; The second model is determined using relevant information from the first model; The second model is used to filter out data related to the first scenario from the second dataset to obtain the first dataset.

7. The method of claim 6, wherein, The second model is used to filter data related to the first scenario from the second dataset to obtain the first dataset, which includes: For each data point in the second dataset, the second model is used to determine the loss value for that data. The data in the second dataset whose loss values ​​satisfy the first condition are identified as the target data; If the number of determined target data is less than a preset threshold, the second model is optimized using the determined target data; for data in the second dataset whose loss value does not meet the first condition, the loss value of the data is determined again using the optimized second model; data whose loss value meets the first condition again is also determined as target data; until the number of determined target data reaches the preset threshold; the first dataset is obtained using all target data.

8. The method of claim 7, wherein, The step of determining the data in the second dataset whose loss value meets the first condition as target data includes: For each data point in the second dataset, determine the validation loss. The data in the second dataset whose loss values ​​satisfy the first condition and whose loss satisfies the second condition are identified as the target data.

9. The method of claim 6, wherein, Before using the second model to filter out data related to the first scene from the second dataset, the method further includes: The second model is trained using the first data.

10. A method of data screening, characterized by, Applications in digital twin networks include: Obtain relevant information about a second dataset and a first model. The second dataset contains at least the first data and also includes the second data and / or the third data. The first data is related to the real-time communication network situation, the second data is associated with the historical communication network situation, and the third data is associated with the simulated communication network situation. The first model is used to determine the network performance index corresponding to the first data. The second model is determined using the relevant information from the first model; The second model is used to filter data associated with the first scenario from the second dataset to obtain the first dataset. The first scenario includes scenarios associated with the first data. The first dataset is used at least to optimize the first model.

11. The method of claim 10, wherein, The second model is used to filter data related to the first scenario from the second dataset to obtain the first dataset, which includes: For each data point in the second dataset, the second model is used to determine the loss value for that data. The data in the second dataset whose loss values ​​satisfy the first condition are identified as the target data; If the number of determined target data is less than a preset threshold, the second model is optimized using the determined target data; for data in the second dataset whose loss value does not meet the first condition, the loss value of the data is determined again using the optimized second model; data whose loss value meets the first condition again is also determined as target data; until the number of determined target data reaches the preset threshold; the first dataset is obtained using all target data.

12. The method of claim 11, wherein, The step of determining the data in the second dataset whose loss value meets the first condition as target data includes: For each data point in the second dataset, determine the validation loss. The data in the second dataset whose loss values ​​satisfy the first condition and whose loss satisfies the second condition are identified as the target data.

13. The method of claim 10, wherein, Before using the second model to filter out data related to the first scene from the second dataset, the method further includes: Obtain the first data; The second model is trained using the first data.

14. A network performance indicator determination apparatus characterized by comprising: Applications in digital twin networks include: The first acquisition unit is used to acquire first data, which is related to the real-time communication network situation. The first determining unit is configured to determine a first model that matches the first data; and to determine the network performance index corresponding to the first data using the matched first model. An optimization unit is configured to optimize the first model using a first dataset, the first dataset containing data obtained through a second model, the second model including a model determined based on relevant information of the first model, the second model being used to filter data associated with a first scenario from the second dataset; the second dataset at least contains the first data, the second dataset also includes second data and / or third data, the second data being associated with historical communication network conditions, the third data being associated with simulated communication network conditions, and the first scenario including a scenario associated with the first data.

15. A data filtering device, characterized in that, Applications in digital twin networks include: The second acquisition unit is used to acquire relevant information of the second dataset and the first model. The second dataset contains at least the first data and also includes the second data and / or the third data. The first data is related to the real-time communication network situation, the second data is associated with the historical communication network situation, and the third data is associated with the simulated communication network situation. The first model is used to determine the network performance index corresponding to the first data. The second determining unit is used to determine the second model using the relevant information of the first model; A filtering unit is used to filter data associated with a first scenario from the second dataset using the second model to obtain a first dataset, wherein the first scenario includes scenarios associated with the first data, and the first dataset is used at least to optimize the first model.

16. An electronic device, characterized in that, Applications in digital twin networks include: A first communication interface is used to acquire first data, which is related to the real-time communication network status. A first processor is configured to: determine a first model matching the first data; determine network performance metrics corresponding to the first data using the matched first model; and optimize the first model using a first dataset, wherein the first dataset contains data obtained through a second model, the second model includes a model determined based on relevant information of the first model, the second model being used to filter data associated with a first scenario from the second dataset; the second dataset contains at least the first data, the second dataset also includes second data and / or third data, the second data being associated with historical communication network conditions, the third data being associated with simulated communication network conditions, and the first scenario including a scenario associated with the first data.

17. An electronic device, characterized in that, Applications in digital twin networks include: The second communication interface is used to obtain relevant information about the second dataset and the first model. The second dataset contains at least the first data and also includes the second data and / or the third data. The first data is related to the real-time communication network situation, the second data is related to the historical communication network situation, and the third data is related to the simulated communication network situation. The first model is used to determine the network performance index corresponding to the first data. A second processor is configured to determine a second model using relevant information from the first model; and to filter data associated with a first scenario from the second dataset using the second model to obtain a first dataset, wherein the first scenario includes scenarios associated with the first data, and the first dataset is used at least to optimize the first model.

18. An electronic device, characterized in that, include: A first processor and a first memory for storing computer programs capable of running on the processor. Wherein, when the first processor is used to run the computer program, it performs the steps of the method according to any one of claims 1 to 9.

19. An electronic device, characterized in that, include: A second processor and a second memory for storing computer programs that can run on the processor. Wherein, when the second processor is used to run the computer program, it performs the steps of the method according to any one of claims 10 to 13.

20. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9, or the steps of the method according to any one of claims 10 to 13.

21. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9, or the steps of the method according to any one of claims 10 to 13.

Images