Data selection method and device, electronic equipment and storage medium

Abstract

The invention discloses a data selection method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out the clustering of a plurality of pieces of first data based on the feature information of each piece of first data in the plurality of pieces of first data, and obtaining a plurality of types of first data; the first data is source data of the DTN; determining a first data set corresponding to the multiple types of first data; wherein the first data set is used for training a DTN model, the first data set at least comprises first data located in a clustering center and K nearest neighbor first data serving as the clustering center in each type of first data in the multiple types of first data, and K is a positive integer.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a data selection method, device, electronic device and storage medium. Background Art

[0002] In the related technology, a digital twin network (DTN) performance model can be constructed based on a data-driven artificial intelligence (AI) model. However, the accuracy of the DTN performance model is low. Summary of the invention

[0003] To solve related technical problems, embodiments of the present application provide a data selection method, device, electronic device and storage medium.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application embodiment provides a data selection method, including:

[0006] Based on feature information of each of the plurality of first data, clustering the plurality of first data to obtain a plurality of categories of first data; the first data is source data of the DTN;

[0007] Determine a first data set corresponding to the multiple types of first data; wherein,

[0008] The first data set is used to train the DTN model, and the first data set includes at least the first data located at the cluster center and the first data of K nearest neighbors of the cluster center in each category of the multiple categories of first data, where K is a positive integer.

[0009] In the above solution, the first data includes data related to the link of the DTN network, and the characteristic information of the first data includes link status information; and / or

[0010] The first data includes relevant data of a queue of the DTN network, and the characteristic information of the first data includes queue status information; and / or

[0011] The first data includes data related to a data flow of a DTN network, and the characteristic information of the first data includes flow state information.

[0012] In the above scheme, the method further comprises:

[0013] The first data set is validated by using expert knowledge to obtain a second data set corresponding to the first data set and having passed the validation.

[0014] In the above solution, the expert knowledge includes a first set range corresponding to each feature information; and the method of using the expert knowledge to verify the validity of the first data set to obtain a verified second data set corresponding to the first data set includes:

[0015] Determine a numerical range of each type of characteristic information based on the value of each type of characteristic information in the first data set;

[0016] When the value range of each type of characteristic information belongs to the corresponding first set range, the first data set is determined as the second data set.

[0017] In the above solution, after determining the value range of each type of feature information, the method further includes:

[0018] When the value range of any characteristic information of the first data in the first data set does not belong to the corresponding first set range, the M first data in the first data set are deleted, and the first data of the nearest neighbors corresponding to the M first data are added to the first data set; wherein,

[0019] M is a positive integer, and M is less than the total number of first data included in the first data set.

[0020] In the above solution, before clustering the plurality of first data based on the feature information of each first data in the plurality of first data, the method further includes at least one of the following:

[0021] Selecting a plurality of real data from the first candidate data to obtain a plurality of first data;

[0022] A plurality of real data and / or simulated data of set scenarios are selected from the first candidate data to obtain a plurality of first data.

[0023] In the above solution, after the validity of the first data set is verified by using expert knowledge to obtain a second data set corresponding to the first data set and having passed the verification, the method further includes:

[0024] Determine the second data that meets the set condition in the second candidate data; the second candidate data is the data in the first candidate data other than the plurality of first data;

[0025] Add the determined part or all of the second data to the second data set.

[0026] In the above scheme, the setting condition includes at least one of the following:

[0027] The candidate data with the top X accuracy rankings, where X is a positive integer;

[0028] The candidate data with accuracy ranking Y behind, Y is a positive integer;

[0029] The value of the network performance indicator corresponding to the candidate data belongs to the corresponding second set range.

[0030] In the above solution, before clustering the plurality of first data based on the feature information of each first data in the plurality of first data, the method further includes:

[0031] Calling a set first model to process each of the multiple first data to obtain feature information of each first data; wherein the set first model is trained based on the first candidate data and is used to determine the feature information and / or the value of the feature information of the input data.

[0032] The embodiment of the present application also provides a data selection device, including:

[0033] The clustering unit is used to cluster the multiple first data based on the feature information of each first data in the multiple first data to obtain multiple types of first data; the first data is the source data of the DTN.

[0034] A determination unit is used to determine a first data set corresponding to the multiple types of first data; wherein,

[0035] The first data set is used to train the DTN model, and the first data set includes at least the first data located at the cluster center and the first data of K nearest neighbors of the cluster center in each category of the multiple categories of first data, where K is a positive integer.

[0036] The embodiment of the present application also provides an electronic device, including a processor and a communication interface, wherein:

[0037] The processor is used to cluster the plurality of first data based on the feature information of each first data in the plurality of first data to obtain multiple categories of first data, and to determine a first data set corresponding to the multiple categories of first data; wherein,

[0038] The first data is the source data of DTN; the first data set is used to train the DTN model, and the first data set at least includes the first data located at the cluster center of each type of first data in the multiple types of first data and the first data of K nearest neighbors of the cluster center, where K is a positive integer.

[0039] The embodiment of the present application further provides an electronic device, comprising a processor and a memory for storing a computer program that can be run on the processor.

[0040] Wherein, the processor is used to execute the steps of any of the above methods when running the computer program.

[0041] An embodiment of the present application further provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0042] In the data selection method, device, electronic device and storage medium provided in the embodiments of the present application, based on the feature information of each of the multiple first data, the multiple first data are clustered to obtain multiple categories of first data; the first data is the source data of DTN; a first data set corresponding to the multiple categories of first data is determined; wherein the first data set is used to train the DTN model, and the first data set at least includes the first data located at the cluster center in each category of the multiple categories of first data and the first data of the K nearest neighbors of the cluster center, where K is a positive integer. In the above scheme, since multiple categories of first data are obtained by clustering based on the feature information of the first data, the same category of first data has similarities, and different categories of first data have different characteristics. In addition, the feature similarity between the first data located at the cluster center and the corresponding K nearest neighbor first data in each category of first data is the highest, and the data quality is the same or approximately the same. By adding the first data located at the cluster center and the first data of the K nearest neighbors serving as the cluster center in each category of first data to the first data set, the first data with the same or approximately the same data quality in each category of first data can be screened out, and the screened out first data of different categories have different characteristics, thereby improving the data quality and data diversity of the first data set. When the DTN model is trained by the first data set, high-quality training data can improve the accuracy of the DTN model, and diversified training data can improve the generalization ability of the DTN model. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 An example diagram of data-driven DTN performance modeling in related technologies;

[0044] Figure 2 A schematic diagram of a data selection method implementation flow in an embodiment of the present application;

[0045] Figure 3 This is an example diagram of data selection in an embodiment of the present application;

[0046] Figure 4 This is an example diagram of performing incremental optimization processing on the second data set in an embodiment of the present application;

[0047] Figure 5 Example diagrams for selecting data for application examples of this application;

[0048] Figure 6 This is a schematic diagram of the structure of a data selection device according to an embodiment of the present application;

[0049] Figure 7This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0050] By building a real-time image of the physical network through DTN, the systematic simulation, optimization, verification and control capabilities that are lacking in the physical network can be enhanced. DTN can be used as a safe and cost-effective performance evaluation environment for network operation and maintenance personnel to evaluate network performance in various hypothetical scenarios. Among them, performance modeling is one of the key technologies of DTN and plays an important role in typical network management scenarios such as planning, operation, optimization and upgrade.

[0051] At present, AI models can be used to simulate complex network environments, for example, to analyze, predict, and make decisions on network environments, in order to help build real-time, lightweight, and high-precision DTNs. Data-driven AI models, especially neural networks, can be used to build DTN performance models. The quality of AI models mainly depends on the model architecture and data. However, the prior art mainly focuses on the model architecture and pays less attention to data quality, while data quality (especially training data) directly affects the accuracy and generalization ability of AI models. For example, diverse data sources with different qualities are difficult to use directly for DTN performance modeling, and diverse data sources with different qualities need to be processed, but the accuracy of the DTN performance model trained based on the processed data source is reduced.

[0052] Figure 1 An example diagram of data-driven DTN performance modeling in related technologies is shown, such as Figure 1 As shown in Figure 1, the data sources used for DTN performance modeling include real data from production networks, data generated by network simulators, and data generated by AI models. The characteristics of these three data sources are as follows:

[0053] Real data from production networks: It is of high value, but difficult to obtain, and has limited quantity, type, and accuracy.

[0054] Data generated by network simulators: large, diverse, and accurate, but time-consuming to generate and different from real data on production networks;

[0055] Data generated by AI models: large in volume, diverse, and generated quickly, but with limited accuracy and different from real data in the production network.

[0056] To sum up, in the related art, the data sources of the training data of the DTN performance model are diverse, the data accuracy is limited, and there are deviations, which makes the quality of the training data of the DTN performance model poor, and thus leads to low accuracy of the DTN performance model.

[0057] Based on this, in various embodiments of the present application, based on the feature information of each first data in a plurality of first data, the plurality of first data are clustered to obtain multiple categories of first data; the first data is the source data of DTN; a first data set corresponding to the multiple categories of first data is determined; wherein the first data set is used to train the DTN model, and the first data set includes at least the first data located at the cluster center in each category of first data in the multiple categories of first data and the first data of the K nearest neighbors of the cluster center, where K is a positive integer. In the above scheme, since multiple categories of first data are obtained by clustering based on the feature information of the first data, the first data of the same category have similarities, and the first data of different categories have different characteristics. In addition, the first data located at the cluster center and the first data of the corresponding K nearest neighbors in each category of first data have the highest feature similarity and the data quality is the same or approximately the same. The first data located at the cluster center and the first data of the K nearest neighbors as the cluster center in each category of first data are added to the first data set, so that the first data with the same or approximately the same data quality in each category of first data can be screened out, and the screened first data of different categories have different characteristics, thereby improving the data quality and data diversity of the first data set. When the DTN model is trained by the first data set, high-quality training data can improve the accuracy of the DTN model, and diversified training data can improve the generalization ability of the DTN model. .

[0058] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0059] The present application embodiment provides a data selection method, which is applied to an electronic device, wherein the electronic device includes a terminal, a server, and a controller. Figure 2 As shown, the method includes:

[0060] Step 201: clustering the plurality of first data based on feature information of each first data in the plurality of first data to obtain a plurality of categories of first data.

[0061] The first data is the source data of DTN.

[0062] Here, the electronic device determines the characteristic information of each first data among the plurality of first data, and based on the characteristic information of each first data, clusters all the first data using a set clustering algorithm to obtain a plurality of categories of first data; the first data of the same category have similarities, and the first data of different categories have different characteristics. The set clustering algorithm may be a K-means clustering algorithm, or a density-based spatial clustering algorithm with noise (DBSCAN, Density-Based Spatial Clustering of Applications with Noise).

[0063] Since the source data of DTN is divided into two categories: real data and simulated data, multiple first data may include real data and / or simulated data. Among them, the real data comes from a physical network such as a production network, and the simulated data comes from a network simulator and / or a generative AI model, etc. Simulation data is also called simulated network data. Simulated network data can be understood as network data of a simulated network, and a simulated network can be understood as a simulated network generated by simulating a physical network. The network simulator is used to generate simulation data, and the specific structure of the network simulator is not limited here. In one embodiment, the network simulator can be a network simulator in the related technology (such as NS-2, NS-3 or OMNet++, etc.).

[0064] In order to obtain diversified training data (first data set), in one embodiment, the first data includes relevant data of the link of the DTN network, and the characteristic information of the first data includes link status information; and / or

[0065] The first data includes relevant data of a queue of the DTN network, and the characteristic information of the first data includes queue status information; and / or

[0066] The first data includes data related to a data flow of a DTN network, and the characteristic information of the first data includes flow state information.

[0067] Here, since a network (physical network, simulation network, DTN network) can be defined as a collection of data flow F, queue Q and link L from a source address to a destination address, the first data may include relevant data of the link and / or queue and / or data flow of the DTN network, and the data flow may be referred to as flow. Link state information may include link utilization; queue state information may include port utilization and / or queue utilization; flow state information includes at least one of latency, throughput, jitter and packet loss rate.

[0068] In practical applications, the feature information can be converted into feature vectors for clustering to improve clustering efficiency. Based on this, in one embodiment, clustering the plurality of first data based on the feature information of each first data in the plurality of first data includes:

[0069] Determine a feature vector of each first data based on all feature information of each first data;

[0070] All the first data are clustered based on the feature vector of each first data.

[0071] Here, the electronic device determines the feature vector corresponding to the first data based on all feature information of each first data; based on the feature vector of each first data, all the first data are clustered using a set clustering algorithm to obtain multiple categories of first data. In actual application, the feature vectors of all the first data can be clustered to obtain the classification result of the feature vector, thereby obtaining the category to which the first data corresponding to the feature vector belongs.

[0072] The dimension of the feature vector corresponds to the number of categories of the feature information of the first data. For example, the feature information of the first data includes three types of feature information: link state information, queue state information, and flow state information. The feature vector of the first data is [S F ,S Q ,S L ],S F is the feature vector corresponding to the flow state information, S Q is the feature vector corresponding to the queue status information, S L is the feature vector corresponding to the link state information.

[0073] In order to improve the accuracy of the feature information of the first data and obtain high-quality training data (first data set), the feature information of the first data can be extracted through an AI model. Based on this, in one embodiment, the method further includes:

[0074] Calling a set first model to process each of the multiple first data to obtain feature information of each first data; wherein the set first model is trained based on the alternative data and is used to determine the feature information and / or the value of the feature information of the input data.

[0075] Here, the electronic device calls the set first model to extract features from each of the multiple first data to obtain feature information of each of the first data.

[0076] The set first model can be trained based on multiple first data, or based on DTN source data other than the first data. The set first model is used to process the input data to obtain feature information and / or feature information value of each input data.

[0077] The set first model can be called a network performance evaluation model, a performance evaluation model or a performance prediction model. The structure of the set first model is not limited here, and the structure of the set first model can be the same as the structure of the network performance evaluation model, the performance evaluation model or the performance prediction model in the related art. It should be noted that in the process of training the first model, the loss value can be determined based on at least one of the delay, throughput, jitter and packet loss rate, and the first model can be trained based on the loss value until the set convergence condition is reached. The set convergence condition can be set according to the actual situation. For example, the set convergence condition can be that the loss value is less than or equal to the set value, or that the number of training times reaches the set number of times.

[0078] In one embodiment, before clustering the plurality of first data based on the feature information of each first data in the plurality of first data, the method further includes at least one of the following:

[0079] Selecting a plurality of real data from the first candidate data to obtain a plurality of first data;

[0080] A plurality of real data and / or simulated data of set scenarios are selected from the first candidate data to obtain a plurality of first data.

[0081] Here, the electronic device can select multiple first data from the first candidate data based on the data selection strategy, and the first candidate data can be understood as a data set containing multiple source data. The data selection strategy may include data selection rules, and / or set conditions that need to be followed when selecting data from the candidate data. The data selection strategy includes a high-fidelity strategy and / or a high-coverage strategy. Specifically, the electronic device can use a high-fidelity strategy to select part or all of the real data in the first candidate data to obtain multiple first data; the electronic device can also use a high-coverage strategy to select multiple real data and / or simulated data of set scenarios in the first candidate data to obtain multiple first data.

[0082] The high-fidelity strategy is used to select real data and / or to select data that fits the real data. That is, when the electronic device uses the high-fidelity strategy to select data from the first candidate data, the selected data can be as close to the real data as possible. For example, the selected data contains similar topology and / or similar routing strategy and / or similar traffic model. The high-coverage strategy is used to enable the selected data to cover as many scenarios as possible, which can be business scenarios or application scenarios.

[0083] It should be noted that the multiple first data selected by the electronic device using the high-fidelity strategy are all real data, and the multiple real data do not distinguish between business scenarios or application scenarios.

[0084] In an embodiment of the present application, a corresponding data selection strategy can be selected according to actual needs, and a plurality of first data can be selected from the first candidate data using the selected data selection strategy, thereby improving the flexibility of data selection; the first data may include simulated data and / or real data, and the simulated data has a diverse characteristic, so that the first data can contain different types of data, thereby improving the diversity of data.

[0085] Step 202: Determine a first data set corresponding to the multiple types of first data.

[0086] The first data set is used to train the DTN model, and the first data set includes at least the first data located at the cluster center and the first data of K nearest neighbors of the cluster center in each category of the multiple categories of first data, where K is a positive integer.

[0087] Here, the electronic device first creates a first data set, which is an empty set at this time, and then performs the following operations on each type of first data in the multiple types of first data:

[0088] Determine the cluster center and K nearest neighbors of the cluster center of the same class of first data; add the first data at the cluster center and the first data of the K nearest neighbors of the cluster center to the first data set, one nearest neighbor corresponds to one first data. The cluster center is also called the class center, and the first data set can be called a seed data set or a seed sample set.

[0089] That is, the electronic device adds the first data located at the cluster center and the first data of K nearest neighbors of the cluster center in each category of the multiple categories of first data to the first data set.

[0090] In actual application, the electronic device can determine the cluster center of each category of first data based on the set clustering algorithm. For example, when the set clustering algorithm is the K-means clustering algorithm, the average value of the feature information or feature vector of the same category of first data is calculated, and the first data corresponding to the average value is determined as the cluster center; based on the distance between the cluster center and other first data in the corresponding class, the K nearest neighbors of the cluster center are determined, and the distance can be the Euclidean distance or the cosine distance. The K nearest neighbors of the cluster center are the K first data in the same category of first data that are closest to the cluster center.

[0091] In order to improve the accuracy and credibility of the data in the first data set and improve the quality of the training data, after obtaining the first data set, the validity or accuracy of the first data set can also be verified. Based on this, in one embodiment, the method further includes:

[0092] The first data set is validated by using expert knowledge to obtain a second data set corresponding to the first data set and having passed the validation.

[0093] Here, when the electronic device obtains the first data set, it takes the first data set as a whole and uses expert knowledge to verify the validity of the first data set to verify the validity or accuracy of the first data set. The expert knowledge may include prior knowledge and / or set domain knowledge. When all data in the first data set are verified, the first data set is verified and a verified second data set is obtained. At this time, the second data set is the same as the first data set.

[0094] When any data in the first data set fails the verification, the first data set is optimized or updated, for example, data that fails the verification is deleted, multiple data are randomly deleted, and data that meets the set conditions in the clustering results are added, and the set conditions include the nearest neighbors of the deleted data; the optimized or updated data set is verified again until all data in the optimized or updated data set are verified, so as to obtain a verified second data set. At this time, the second data set is the optimized or updated first data set, which is different from the initial first data set.

[0095] In the embodiment of the present application, by verifying the validity or accuracy of the data in the first data set, the accuracy and credibility of the second data set can be improved, and when the DTN model is trained using the second data set, the accuracy of the DTN model can be further improved.

[0096] In order to improve the verification efficiency of the first data set and improve the accuracy or credibility of the verification result, thereby improving the precision of the data in the first data set, in one embodiment, the expert knowledge includes a first set range corresponding to each feature information; the validation of the first data set by using the expert knowledge to obtain a second data set corresponding to the first data set that has passed the validation includes:

[0097] Determine a numerical range of each type of characteristic information based on the value of each type of characteristic information in the first data set;

[0098] When the value range of each type of characteristic information belongs to the corresponding first set range, the first data set is determined as the second data set.

[0099] Here, based on the value of the characteristic information of each first data in the first data set, the numerical range of each characteristic information is determined, for example, based on the maximum and minimum values ​​of the characteristic information, the corresponding numerical range is determined; by comparing the numerical range of each characteristic information with the corresponding first setting range, it is determined whether the numerical range of the characteristic information belongs to the corresponding first setting range; in this way, it is determined whether the numerical range of each characteristic information belongs to the corresponding first setting range; when the numerical range of each characteristic information of all first data belongs to the corresponding first setting range, the first data set is verified and the first data set is determined as the second data set. Wherein, in the case where the characteristic information is link state information, the value of the characteristic information can be the value of link utilization; in the case where the characteristic information is queue state information, the value of the characteristic information can be the value of port occupancy or the value of queue utilization; in the case where the characteristic information is flow state information, the value of the characteristic information can be at least one of delay, throughput, jitter and packet loss rate.

[0100] In the case where the numerical range of any type of characteristic information does not belong to the corresponding first set range, the first data set verification fails, or the first data set verification fails; the first data set is optimized or updated, and based on the characteristic information of each first data in the optimized or updated first data set, the numerical range of each type of characteristic information is re-determined; in the case where the numerical range of each type of characteristic information belongs to the corresponding first set range, the optimized or updated first data set is verified to obtain a verified second data set; in the case where the numerical range of any type of characteristic information does not belong to the corresponding first set range, the first data set is optimized or updated again, and subsequent related steps are performed until the optimized or updated first data set is verified.

[0101] In order to obtain high-quality training data, if the first data set fails to be verified, it is necessary to update the first data set and then verify it again. Based on this, in one embodiment, after determining the numerical range of each feature information, the method further includes:

[0102] When the value range of any characteristic information of the first data in the first data set does not belong to the corresponding first set range, the M first data in the first data set are deleted, and the first data of the nearest neighbors corresponding to the M first data are added to the first data set; wherein,

[0103] M is a positive integer, and M is less than the total number of first data included in the first data set.

[0104] Here, if the numerical range of any characteristic information of any first data in the first data set does not belong to the corresponding first set range, the first data set verification fails, and M first data are randomly selected in the first data set. Among the multiple types of first data, the nearest neighbors corresponding to the M first data are determined, and each of the M first data corresponds to one nearest neighbor; the randomly selected M first data are deleted from the first data set, and the first data of the nearest neighbors corresponding to the M first data are added to the first data set to re-determine the numerical range of each characteristic information; based on the numerical range of each characteristic information and the corresponding first set range, the validity of the updated first data set is verified again. It should be noted that the first set range corresponding to each characteristic information can be determined based on the expert knowledge mentioned above, or it can be pre-configured. The first data of the nearest neighbors corresponding to the M first data can be understood as: M first data as the nearest neighbors of the M first data, and each of the M first data corresponds to one nearest neighbor first data.

[0105] The first data of the nearest neighbors corresponding to the M first data can be determined according to the following method:

[0106] For the convenience of description, each of the M randomly selected first data is recorded as the third data, the category to which each of the M third data belongs is determined among the multiple categories of first data, and the first data as the nearest neighbor of the third data is determined in the corresponding category to obtain the first data of the nearest neighbor corresponding to each of the M first data.

[0107] In actual application, electronic equipment is Figure 3 The data selection process shown is used to obtain the second data set.

[0108] In one embodiment, after the validity of the first data set is verified by using expert knowledge to obtain a second data set corresponding to the first data set and passing the verification, the method further includes:

[0109] Determine the second data that meets the set condition in the second candidate data; the second candidate data is the data in the first candidate data other than the plurality of first data;

[0110] Add the determined part or all of the second data to the second data set.

[0111] Here, after obtaining the second data set, the electronic device deletes all first data contained in the second data set in the first candidate data to obtain the second candidate data; in the second candidate data, the electronic device determines the second data that meets the set conditions, and adds part or all of the determined second data to the second data set.

[0112] It should be noted that adding part or all of the second data in the second candidate data that meets the set conditions to the second data set can be understood as performing incremental optimization processing on the second data set.

[0113] In an embodiment of the present application, adding part or all of the second data in the second candidate data that meets the set conditions to the second data set can enrich the data contained in the second data set and improve the data diversity of the second data set, so as to further improve the generalization ability of the DTN model trained based on the second data set.

[0114] In one embodiment, the setting condition includes at least one of the following:

[0115] The candidate data with the top X accuracy rankings, where X is a positive integer;

[0116] The candidate data with accuracy ranking Y behind, Y is a positive integer;

[0117] The value of the network performance indicator corresponding to the candidate data belongs to the corresponding second set range.

[0118] Here, the electronic device can determine the accuracy of each alternative data in the second alternative data, sort all the alternative data in the second alternative data based on the accuracy of the alternative data, and obtain an alternative data sequence; select the alternative data with the top X accuracy rankings in the alternative data sequence, and select the alternative data with the bottom Y accuracy rankings in the alternative data sequence.

[0119] Among them, the alternative data in the alternative data sequence can be arranged in order from high to low accuracy, or in order from low to high accuracy. If the alternative data in the alternative data sequence is arranged in order from high to low accuracy, then the accuracy of the alternative data ranked in the top X is higher, and the accuracy of the alternative data ranked in the bottom Y is lower; the alternative data ranked in the top X in accuracy can be understood as a valid sample or a simple sample; the alternative data ranked in the bottom Y in accuracy can include difficult examples (difficult samples) or out-of-distribution (OOD) samples, or may not include OOD samples, and difficult examples can be understood as difficult negative samples (hard negative). X and Y can be set according to actual conditions. It should be noted that adding difficult examples or OOD samples to the second data set can improve the diversity of data in the second data set; when the second data set is used as training data, difficult examples and OOD samples can improve the learning ability of the model and bring better training results. The network performance indicators include at least one of latency, throughput, jitter and packet loss rate.

[0120] It should be noted that accuracy can also be understood as accuracy rate or performance prediction accuracy. When the candidate data is real data, the accuracy of the candidate data can be 100%; when the candidate data is simulated data, the accuracy of the candidate data can be determined based on the difference between the simulated data and the real data. Of course, the accuracy can be for the network performance indicators of the candidate data; in actual application, the predicted value of the network performance indicator of each candidate data can be predicted through the performance evaluation model, and the performance prediction accuracy of the candidate data can be determined based on the predicted value of the network performance indicator and the corresponding label value or calibration value.

[0121] Figure 4 An example diagram of incremental optimization processing of the second data set is given, such as Figure 4 As shown, the implementation process of performing incremental optimization processing on the second data includes:

[0122] Step 1: Use the second dataset as the training set R.

[0123] Step 2: Delete the training set R from the first candidate data to obtain the candidate data set D.

[0124] The candidate data set D is the second candidate data mentioned above.

[0125] Step 3: Use the training set R to train the second model to obtain the trained second model.

[0126] The second model is a network performance evaluation model, a performance evaluation model or a performance prediction model. The second model can be understood as a filter.

[0127] Step 4: Call the second model to process the candidate data in the candidate data set D to obtain the predicted value of the network performance indicator of each candidate data in the candidate data set D.

[0128] The network performance index includes at least one of the following: link utilization, queue utilization, port utilization, delay, throughput, jitter and packet loss rate.

[0129] Step 5: Sort all candidate data in the candidate data set D based on their performance prediction accuracy to obtain a candidate data sequence; in the candidate data sequence, select the candidate data with the top X accuracy rankings and the candidate data with the bottom Y accuracy rankings in turn to obtain a data set S.

[0130] The data set S may include candidate data with an accuracy ranking of X and candidate data with an accuracy ranking of Y. The candidate data with an accuracy ranking of X can be understood as X candidate data with higher accuracy; the candidate data with an accuracy ranking of Y can be understood as Y candidate data with lower accuracy.

[0131] Step 6: Use the OOD detection algorithm to identify the OOD samples from the candidate data with the accuracy ranking at the bottom Y, and delete the identified OOD samples from the data set S.

[0132] Among them, step 6 is an optional step. If the OOD sample is not helpful for model training, the OOD sample in the data set S can be removed.

[0133] Step 7: Evaluate the data quality of each candidate data in the data set S based on expert knowledge, determine the second data with qualified data quality in the data set S, and add the second data to the training set R.

[0134] Here, the normal range of the network performance indicator can be determined based on expert knowledge, and the normal range is also called the set range. Based on the normal range of the network performance indicator and the value (real value or predicted value) of the network performance indicator of each candidate data in the data set S, the second data with qualified data quality is determined in the data set S, and the second data is added to the training set R. If the value of the network performance indicator of any candidate data in the data set S belongs to the corresponding normal range, then the data quality of the candidate data is qualified; if the value of the network performance indicator of any candidate data in the data set S does not belong to the corresponding normal range, then the data quality of the candidate data is unqualified.

[0135] Repeat steps 2 to 7 until the training set R reaches a preset size, that is, the number of candidate data in the training set R reaches a preset number, to obtain a high-quality training set R (second data set).

[0136] The present application is described in further detail below in conjunction with application examples.

[0137] like Figure 5 As shown, the data selection methods include:

[0138] Step 1: Based on a data selection strategy, a plurality of first data are determined from the first candidate data.

[0139] Specifically, the data selection strategy includes a high-fidelity strategy and / or a high-coverage strategy. The electronic device can select multiple real data from the first alternative data based on the high-fidelity strategy to obtain multiple first data; and / or, based on the high-coverage strategy, select multiple real data and / or simulated data of set scenarios from the first alternative data to obtain multiple first data.

[0140] Step 2: Training a network performance evaluation model based on the multiple first data until a set convergence condition is reached to obtain a set first model.

[0141] The first model is set to process the input data to obtain the characteristic information and / or the value of the characteristic information of each input data. In the process of training the first model, the loss value can be determined based on at least one of the delay, throughput, jitter and packet loss rate, and the first model can be trained based on the loss value until the set convergence condition is reached. The set convergence condition can be set according to the actual situation. For example, the set convergence condition can be that the loss value is less than or equal to the set value, or that the number of training times reaches the set number of times.

[0142] Step 3: Call the set first model to process each of the multiple first data to obtain feature information of each first data.

[0143] In one embodiment, the first data includes data related to a link of the DTN network, and the characteristic information of the first data includes link status information; and / or

[0144] The first data includes relevant data of a queue of the DTN network, and the characteristic information of the first data includes queue status information; and / or

[0145] The first data includes data related to a data flow of a DTN network, and the characteristic information of the first data includes flow state information.

[0146] Step 4: Clustering the multiple first data based on the feature information of each first data in the multiple first data to obtain multiple categories of first data.

[0147] Step 5: Determine a first data set corresponding to the multiple types of first data.

[0148] The first data set at least includes the first data located at the cluster center and the first data of K nearest neighbors of the cluster center in each category of the multiple categories of first data, where K is a positive integer.

[0149] Step 6: Use expert knowledge to verify the validity of the first data set to obtain a verified second data set corresponding to the first data set.

[0150] In one embodiment, the expert knowledge includes a first set range corresponding to each type of feature information; step 6 specifically includes:

[0151] Determine a numerical range of each type of characteristic information based on the value of each type of characteristic information in the first data set;

[0152] When the value range of each type of characteristic information belongs to the corresponding first set range, the first data set is determined as the second data set.

[0153] In one embodiment, after determining the value range of each type of feature information, the method further includes:

[0154] When the value range of any characteristic information of the first data in the first data set does not belong to the corresponding first set range, the M first data in the first data set are deleted, and the first data of the nearest neighbors corresponding to the M first data are added to the first data set; wherein,

[0155] M is a positive integer, and M is less than the total number of first data included in the first data set.

[0156] Step 7: Determine the second data that meets the set conditions from among the second candidate data.

[0157] The second candidate data is data other than the first data in the first candidate data. The setting condition includes at least one of the following:

[0158] The candidate data with the top X accuracy rankings, where X is a positive integer;

[0159] The candidate data with accuracy ranking Y behind, Y is a positive integer;

[0160] The value of the network performance indicator corresponding to the candidate data belongs to the corresponding second set range.

[0161] Step 8: Add part or all of the determined second data to the second data set to train the DTN model using the second data set.

[0162] Among them, step 7 and step 8 are to perform incremental optimization processing on the second data set. In actual application, it can be based on Figure 4 The process shown is incrementally optimized.

[0163] Table 1 shows the simulation data of the generalization ability of the DTN performance model obtained by training based on the second data set containing the second data. Among them, the DTN performance model is the RouteNet network performance model, and RouteNet is a new graph neural network (GNN) architecture. The evaluation indicators of the DTN performance model are the mean absolute percentage error (MAPE) of the predicted value and the MAPE of the true value.

[0164] Table 1

[0165]

[0166]

[0167] It can be seen from Table 1 that the DTN performance model trained using the second data set containing the second data can obtain a lower MAPE in DTN networks of various sizes, that is, the DTN performance model has a strong generalization ability.

[0168] Table 2 gives examples of the impact of different stages on the accuracy of the DTN performance model.

[0169] Table 2

[0170] Iteration Number of training samples MAPE of the true value MAPE of predicted values Seed sample selection stage 20(seed sample) 10.96％ 12.38％ Incremental Optimization Phase 1 38 9.46％ 11.34％ Incremental Optimization Phase 2 57 8.43％ 10.76％ Incremental Optimization Phase 3 79 7.13％ 9.43％ Incremental Optimization Phase 4 100 6.63％ 8.55％

[0171] It can be seen from Table 2 that the incremental optimization stage further improves the performance of the DTN performance model, indicating that the incremental optimization stage further improves the data quality of the training samples.

[0172] The above scheme, through data selection, seed sample selection and incremental optimization and other processes, can screen out a high-precision, diversified, and high-quality second data set that fits the real data from the source data source. When using the second data set to train DTN-related models, the quality of the training data is improved, thereby improving the accuracy and generalization ability of the DTN-related models, so that a high-precision DTN can be constructed using the DTN-related models.

[0173] In order to implement the data selection method of the embodiment of the present application, the embodiment of the present application also provides a data selection device, which is arranged on an electronic device, such as Figure 6 As shown, the device comprises:

[0174] A clustering unit 601 is used to cluster the multiple first data based on feature information of each first data in the multiple first data to obtain multiple types of first data; the first data is source data of the DTN;

[0175] The determining unit 602 is used to determine a first data set corresponding to the multiple types of first data; wherein:

[0176] The first data set is used to train the DTN model, and the first data set includes at least the first data located at the cluster center and the first data of K nearest neighbors of the cluster center in each category of the multiple categories of first data, where K is a positive integer.

[0177] In one embodiment, the first data includes data related to a link of the DTN network, and the characteristic information of the first data includes link status information; and / or

[0178] The first data includes relevant data of a queue of the DTN network, and the characteristic information of the first data includes queue status information; and / or

[0179] The first data includes data related to a data flow of a DTN network, and the characteristic information of the first data includes flow state information.

[0180] In one embodiment, the device further comprises:

[0181] The verification unit is used to verify the validity of the first data set by using expert knowledge to obtain a verified second data set corresponding to the first data set.

[0182] In one embodiment, the expert knowledge includes a first set range corresponding to each type of characteristic information; the verification unit is specifically used to determine the numerical range of each type of characteristic information based on the value of the characteristic information of each first data in the first data set; when the numerical range of each type of characteristic information belongs to the corresponding first set range, the first data set is determined as the second data set.

[0183] In one embodiment, the device further comprises:

[0184] The first updating unit is used to delete M first data in the first data set when the value range of any characteristic information of the first data in the first data set does not belong to the corresponding first set range, and add the nearest neighbor first data corresponding to the M first data to the first data set; wherein

[0185] M is a positive integer, and M is less than the total number of first data included in the first data set.

[0186] In one embodiment, the device further comprises:

[0187] A selection unit is used to select a plurality of real data from the first candidate data to obtain a plurality of first data; and / or,

[0188] A plurality of real data and / or simulated data of set scenarios are selected from the first candidate data to obtain a plurality of first data.

[0189] In one embodiment, the device further comprises:

[0190] The second updating unit is used to determine the second data that meets the set conditions in the second candidate data; add part or all of the determined second data to the second data set; the second candidate data is the data in the first candidate data except the multiple first data.

[0191] In one embodiment, the setting condition includes at least one of the following:

[0192] The candidate data with the top X accuracy rankings, where X is a positive integer;

[0193] The candidate data with accuracy ranking Y behind, Y is a positive integer;

[0194] The value of the network performance indicator corresponding to the candidate data belongs to the corresponding second set range.

[0195] In one embodiment, the device further comprises:

[0196] A feature extraction unit is used to call a set first model to process each of the multiple first data to obtain feature information of each first data; wherein the set first model is trained based on the first candidate data and is used to determine the feature information and / or the value of the feature information of the input data.

[0197] In actual application, the clustering unit 601, the determination unit 602, the verification unit, the first updating unit, the selection unit, the second updating unit and the feature extraction unit can be implemented by a processor in a data selection device.

[0198] It should be noted that: the data selection device provided in the above embodiment only uses the division of the above program modules as an example to illustrate when performing data selection. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the data selection device provided in the above embodiment and the data selection method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0199] Based on the hardware implementation of the above program modules, and in order to implement the data selection method of the embodiment of the present application, the embodiment of the present application also provides an electronic device, such as Figure 7 As shown, the electronic device 700 includes:

[0200] Communication interface 701, capable of exchanging information with other network nodes;

[0201] The processor 702 is connected to the communication interface 701 to implement information exchange with other network nodes, and is used to execute the method provided by one or more technical solutions when running a computer program. The computer program is stored in the memory 703.

[0202] Specifically, the processor 702 is used to cluster the multiple first data based on the feature information of each first data in the multiple first data to obtain multiple categories of first data; determine a first data set corresponding to the multiple categories of first data; wherein the first data is the source data of the DTN; the first data set is used to train the DTN model, and the first data set at least includes the first data located at the cluster center in each category of the multiple categories of first data and the first data of the K nearest neighbors of the cluster center, where K is a positive integer.

[0203] In one embodiment, the first data includes data related to a link of the DTN network, and the characteristic information of the first data includes link status information; and / or

[0204] The first data includes relevant data of a queue of the DTN network, and the characteristic information of the first data includes queue status information; and / or

[0205] The first data includes data related to a data flow of a DTN network, and the characteristic information of the first data includes flow state information.

[0206] In one embodiment, the processor 702 is further configured to use expert knowledge to verify the validity of the first data set, and obtain a verified second data set corresponding to the first data set.

[0207] In one embodiment, the expert knowledge includes a first set range corresponding to each type of feature information; the processor 702 is specifically used to determine the numerical range of each type of feature information based on the value of the feature information of each first data in the first data set; when the numerical range of each type of feature information belongs to the corresponding first set range, the first data set is determined as the second data set.

[0208] In one embodiment, the processor 702 is further configured to delete M first data in the first data set when the value range of any characteristic information of the first data in the first data set does not belong to the corresponding first set range, and add the nearest neighbor first data corresponding to the M first data to the first data set; wherein,

[0209] M is a positive integer, and M is less than the total number of first data included in the first data set.

[0210] In one embodiment, the processor 702 is further configured to select a plurality of real data from the first candidate data to obtain a plurality of first data; and / or,

[0211] A plurality of real data and / or simulated data of set scenarios are selected from the first candidate data to obtain a plurality of first data.

[0212] In one embodiment, the processor 702 is further used to determine second data that meets set conditions in the second candidate data; add part or all of the determined second data to the second data set; the second candidate data is data in the first candidate data other than the multiple first data.

[0213] In one embodiment, the setting condition includes at least one of the following:

[0214] The candidate data with the top X accuracy rankings, where X is a positive integer;

[0215] The candidate data with accuracy ranking Y behind, Y is a positive integer;

[0216] The value of the network performance indicator corresponding to the candidate data belongs to the corresponding second set range.

[0217] In one embodiment, the processor 702 is also used to call a set first model to process each of the multiple first data to obtain feature information of each first data; wherein the set first model is trained based on the first candidate data and is used to determine the feature information and / or the value of the feature information of the input data.

[0218] It should be noted that the specific processing process of the processor 702 and the communication interface 701 can be understood by referring to the above method.

[0219] Of course, in actual application, the various components in the electronic device 700 are coupled together through the bus system 704. It can be understood that the bus system 704 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 704 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 7 Various buses are labeled as bus system 704 .

[0220] The memory 703 in the embodiment of the present application is used to store various types of data to support the operation of the electronic device 700. Examples of such data include: any computer program used to operate on the electronic device 700.

[0221] The method disclosed in the above embodiment of the present application can be applied to the processor 702, or implemented by the processor 702. The processor 702 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of the hardware in the processor 702 or an instruction in the form of software. The above-mentioned processor 702 may be a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 702 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present application, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in the memory 703, and the processor 702 reads the information in the memory 703 and completes the steps of the above method in combination with its hardware.

[0222] In an exemplary embodiment, the electronic device 700 can be implemented by one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to execute the aforementioned method.

[0223] It can be understood that the memory (memory 703) of the embodiment of the present application can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a ferromagnetic random access memory, a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAMbus random access memory (DRRAM, Direct Rambus Random Access Memory).The memories described in the embodiments of the present application are intended to include, but are not limited to, these and any other suitable types of memories.

[0224] In an exemplary embodiment, the present application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, for example, a memory 703 storing a computer program, and the computer program can be executed by a processor 702 of an electronic device 700 to complete the steps of the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface storage, optical disk, or CD-ROM.

[0225] It should be noted that: "first", "second", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0226] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0227] In addition, the technical solutions described in the embodiments of the present application can be combined arbitrarily without conflict.

[0228] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. A data selection method, characterized in that: include: Clustering the plurality of first data based on feature information of each first data to obtain a plurality of categories of first data; The first data is the source data of the digital twin network DTN; Determine a first data set corresponding to the multiple types of first data; wherein, The first data set is used to train the DTN model, and the first data set includes at least the first data located at the cluster center and the first data of K nearest neighbors of the cluster center in each category of the multiple categories of first data, where K is a positive integer.

2. The method according to claim 1, characterized in that The first data includes data related to a link of the DTN network, and the characteristic information of the first data includes link status information; and / or The first data includes relevant data of a queue of the DTN network, and the characteristic information of the first data includes queue status information; and / or The first data includes data related to a data flow of a DTN network, and the characteristic information of the first data includes flow state information.

3. The method according to claim 1, characterized in that The method further comprises: The first data set is validated using expert knowledge to obtain a validated second data set corresponding to the first data set.

4. The method according to claim 3, characterized in that The expert knowledge includes a first set range corresponding to each feature information; and the validation of the first data set by using the expert knowledge to obtain a second data set corresponding to the first data set that has passed the validation includes: Determine a value range of each type of feature information based on the value of the feature information of each first data in the first data set; When the value range of each type of characteristic information belongs to the corresponding first set range, the first data set is determined as the second data set.

5. The method according to claim 4, characterized in that After determining the value range of each type of characteristic information, the method further includes: When the value range of any characteristic information of the first data in the first data set does not belong to the corresponding first set range, the M first data in the first data set are deleted, and the first data of the nearest neighbors corresponding to the M first data are added to the first data set; wherein, M is a positive integer, and M is less than the total number of first data included in the first data set.

6. The method according to any one of claims 1 to 5, characterized in that: Before clustering the plurality of first data based on the feature information of each first data in the plurality of first data, the method further includes at least one of the following: Selecting a plurality of real data from the first candidate data to obtain a plurality of first data; A plurality of real data and / or simulated data of set scenarios are selected from the first candidate data to obtain a plurality of first data.

7. The method according to any one of claims 3 to 5, characterized in that: After the validity of the first data set is verified by using expert knowledge to obtain a verified second data set corresponding to the first data set, the method further includes: Determine the second data satisfying the set condition in the second candidate data; the second candidate data is the data in the first candidate data other than the plurality of first data; Add the determined part or all of the second data to the second data set.

8. The method according to claim 7, characterized in that The setting condition includes at least one of the following: The candidate data with the top X accuracy rankings, where X is a positive integer; The candidate data with accuracy ranking Y behind, Y is a positive integer; The value of the network performance indicator corresponding to the candidate data belongs to the corresponding second set range.

9. The method according to claim 6, characterized in that Before clustering the plurality of first data based on the feature information of each first data in the plurality of first data, the method further includes: Calling a set first model to process each of the multiple first data to obtain feature information of each first data; wherein the set first model is trained based on the first candidate data and is used to determine the feature information and / or the value of the feature information of the input data.

10. A data selection device, characterized in that: include: A clustering unit, configured to cluster the plurality of first data based on feature information of each first data in the plurality of first data to obtain a plurality of categories of first data; The first data is source data of DTN; A determination unit is used to determine a first data set corresponding to the multiple types of first data; wherein, The first data set is used to train the DTN model, and the first data set includes at least the first data located at the cluster center and the first data of K nearest neighbors of the cluster center in each category of the multiple categories of first data, where K is a positive integer.

11. An electronic device, characterized in that: comprising a processor and a communication interface, wherein: The processor is used to cluster the multiple first data based on the feature information of each first data in the multiple first data to obtain multiple categories of first data; determine the first data set corresponding to the multiple categories of first data; wherein, The first data is the source data of DTN; the first data set is used to train the DTN model, and the first data set at least includes the first data located at the cluster center of each type of first data in the multiple types of first data and the first data of K nearest neighbors of the cluster center, where K is a positive integer.

12. An electronic device, characterized in that: comprising a processor and a memory for storing a computer program capable of being executed on the processor, Wherein, when the processor is used to run the computer program, it executes the steps of the method described in any one of claims 1 to 9.

13. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images