Calculation method and system for data feature classification path

Abstract

The invention relates to a calculation method and system for a data feature classification path, and belongs to the technical field of data processing, and the method comprises the steps: obtaining 5G business data; performing feature extraction on the 5G business data based on the target machine learning model to obtain business features; determining cluster features of clusters of the service features, determining cluster centroids, cluster radiuses and cluster diameters according to the cluster features of the clusters, calculating distances among the clusters based on the cluster centroids, the cluster radiuses and the cluster diameters, constructing a clustering feature tree based on the cluster features of the clusters and the construction parameters, and determining the clustering features of the service features based on the clustering feature tree; and constructing a random forest according to the service features and the feature clusters, calculating feature importance scores of the service features based on the random forest and an XGBoost model, classifying the service features of which the feature importance scores are greater than a preset score in the feature importance scores based on a decision classification tree model, and taking a classification result as a service path corresponding to the service features.

Description

Technical Field

[0001] The present invention belongs to the field of data processing technology, and in particular relates to a method and system for calculating a data feature classification path. Background Art

[0002] Among the related technologies, 5G technology is mainly used to meet the needs of the rapidly developing mobile Internet and Internet of Things services. It has the characteristics of low cost, low energy consumption, safety and reliability. The traditional data feature classification path calculation technology cannot provide this new user experience and high-quality service. In addition, when the network reaches a certain scale, the traditional path calculation algorithm will have serious performance bottlenecks due to the constraints of the algorithm itself and machine performance. Therefore, how to better realize the calculation of data feature classification paths has become an urgent problem to be solved. Summary of the invention

[0003] In view of the above deficiencies in the prior art, an object of the present invention is to provide a method and system for calculating a data feature classification path.

[0004] In the first aspect of the present invention, a method for calculating a data feature classification path is proposed, including: S1, obtaining 5G service data; S2, extracting features of the 5G service data based on a target machine learning model to obtain service features; S3, determining cluster features of the clustering of the service features, determining the cluster centroid, cluster radius and cluster diameter according to the cluster features of the clustering, calculating the distance between each cluster based on the cluster centroid, the cluster radius and the cluster diameter, constructing a cluster feature tree based on the cluster features of the clustering and construction parameters, and determining the cluster features of the service features based on the cluster feature tree; S4, constructing a random forest based on the service features and the feature clustering, and calculating the feature importance scores of the service features based on the random forest and XGBoost models, classifying the service features whose feature importance scores are greater than a preset score based on a decision classification tree model, and using the classification results as the service paths corresponding to the service features.

[0005] Furthermore, the target machine learning model is trained in the following manner: obtaining a target data set of the 5G service data, the target data set consisting of a basic feature set T and a generated feature set S; dividing the target data set into a training set D tr and validation set D vld ; Based on the training set D tr Train the first machine learning model L and based on ε(L(D tr , T+S), D vld , T+S) evaluates the first machine learning model L; ε(L(D tr , T+S), D vld , T+S) means using the training set Dtr The first machine learning model L obtained by training is used in the validation set D vld The evaluation index calculated above, L(D tr , T+S) means using the training set D tr Training model: a first machine learning model L; based on the first data added to the basic feature set T, retraining the first machine learning model L, and using the retrained first machine learning model L as the target machine learning model.

[0006] Further, feature extraction is performed on the 5G service data based on the target machine learning model to obtain service features, including: Filter out the optimal feature set from the candidate feature set A(T) generated by the basic feature set T Based on the optimal feature set The service characteristics are determined.

[0007] Further, it is characterized in that determining the cluster features of the clustering of the business features includes: defining each of the business features as a triplet CF, the triplet CF includes the number of data points N in the cluster, a vector LS and a scalar SS, wherein the vector LS represents the linear sum of each data point, and the scalar SS represents the square sum of each data point; wherein the vector n represents the nth data point, M represents the dimension of the data point, and the scalar X ni Represents the i-th dimension data of the n-th data point; obtains the triples of the multiple business features, according to CF 1 +CF 2 +...+CF b =(N 1 +N 2 +...+N b ,LS 1 +LS 2 +...+LS b ,SS 1 +SS 2 +...+SS b ), superimposing the triples of the multiple business features to obtain cluster features of the business features; wherein b represents the number of business features, the triple CF 1 =(N 1 ,LS 1 ,SS 1 ), triple CF 2 =(N 2 ,LS 2 ,SS 2 ), triple CF b =(Nb ,LS b ,SS b );N 1 To N b Respectively represent the number of data points in the clusters of the 1st to bth business features; LS 1 To LS b Respectively represent the vectors of the 1st to bth service characteristics; SS 1 To SS b Respectively represent the scalars of the 1st to bth service features; wherein the cluster centroid, cluster radius and cluster diameter are determined according to the cluster features of the clustering, including: based on Determine the cluster centroid; based on Determine the cluster radius, where the cluster radius represents the distance from the data point to the cluster centroid; based on Determine the cluster diameter, the cluster diameter represents the average distance between every two data points; wherein the distance between each cluster is calculated based on the cluster centroid, the cluster radius and the cluster diameter, including: based on Calculate the distance between clusters.

[0008] Further, the construction parameters include a branch balance factor β, a leaf balance factor λ and a spatial threshold τ, wherein a clustering feature tree is constructed based on the cluster features and construction parameters of the clustering, including: initializing the branch balance factor β, the leaf balance factor λ and the spatial threshold τ; determining a target leaf node closest to a target sample and a target triplet CF of the target leaf node from the leaf nodes under the root node, wherein the target triplet CF is a triplet CF closest to the target sample among multiple triples CF in the target leaf node; judging whether a hypersphere radius corresponding to the target triplet CF is less than the spatial threshold τ when the target sample is added to the target triplet CF; when it is determined that the hypersphere radius corresponding to the target triplet CF is less than the spatial threshold τ, updating all the triples CF on the path; when it is determined that the hypersphere radius corresponding to the target triplet CF is not less than the spatial threshold τ, updating all the triples on the path according to the number of triples CF of the target leaf node.

[0009] Further, all the triplets on the path are updated according to the number of triples CF of the target leaf node, including: judging whether the number of triples CF of the target leaf node is less than the leaf balance factor λ; when it is determined that the number of triples CF of the target leaf node is less than the leaf balance factor λ, creating a first triple under the target leaf node, storing the target sample in the first triple, and updating all the triplets on the path; when it is determined that the number of triples CF of the target leaf node is not less than the leaf balance factor λ, obtaining all triples CF corresponding to the target leaf node, and based on the hypersphere distance, obtaining the second triple CF and the third triple CF in all triples CF; dividing the target leaf node into a first leaf node and a second leaf node; using the second triple CF or the third triple CF as the first triple CF of the first leaf node or the second leaf node, and placing the remaining triples CF in all triples CF except the second triple CF and the third triple CF and the target sample into the first leaf node or the second leaf node according to the principle of distance.

[0010] Furthermore, the feature importance score of the business feature is calculated based on the random forest and XGBoost models, the business features with a feature importance score greater than a preset score are classified based on the decision classification tree model, and the classification result is used as the business path corresponding to the business feature, including: calculating the first feature importance score of the business feature based on the random forest, and obtaining the first business feature with a first feature importance score greater than the preset score; calculating the second feature importance score of the business feature based on the XGBoost model, and obtaining the second business feature with a second feature importance score greater than the preset score; using the intersection business feature of the first business feature and the second business feature as the target business feature; and classifying the target business feature based on the decision classification tree model, and using the classification result as the business path corresponding to the target business feature.

[0011] In the second aspect of the present invention, a calculation system for a data feature classification path is proposed, including: an acquisition module for acquiring 5G service data; an extraction module for performing feature extraction on the 5G service data based on a target machine learning model to obtain service features; a construction module for determining cluster features of the clustering of the service features, determining the cluster centroid, cluster radius and cluster diameter according to the cluster features of the clustering, calculating the distance between each cluster based on the cluster centroid, the cluster radius and the cluster diameter, constructing a cluster feature tree based on the cluster features of the clustering and construction parameters, and determining the cluster features of the service features based on the cluster feature tree; a determination module for constructing a random forest based on the service features and the feature clustering, and calculating the feature importance score of the service features based on the random forest and XGBoost model, classifying the service features whose feature importance scores are greater than a preset score based on a decision classification tree model, and using the classification results as the service path corresponding to the service features.

[0012] According to a third aspect of the present invention, an electronic device is proposed, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute any of the methods described in the first aspect of the present invention.

[0013] According to a fourth aspect of the present invention, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any one of the methods according to the first aspect of the present invention.

[0014] The beneficial effects of the present invention are as follows:

[0015] The calculation method and system of the data feature classification path of the present invention obtain 5G business data; extract features of 5G business data based on the target machine learning model to obtain business features; determine the cluster features of the clustering of business features, determine the cluster centroid, cluster radius and cluster diameter according to the cluster features of the clustering, calculate the distance between each cluster based on the cluster centroid, cluster radius and cluster diameter, build a cluster feature tree based on the cluster features of the clustering and construction parameters, and determine the cluster features of the business features based on the cluster feature tree; build a random forest according to the business features and feature clustering, and calculate the feature importance score of the business features based on the random forest and XGBoost models, classify the business features with a feature importance score greater than a preset score based on the decision classification tree model, and use the classification result as the business path corresponding to the business feature. The method can quickly and accurately calculate the path of a large-scale network, and effectively solve the bottleneck of the performance of the traditional path calculation algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are only used to illustrate specific embodiments and are not considered to limit the present invention. In the entire drawings, the same reference symbols represent the same components. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings.

[0017] Figure 1 is a flow chart of a method for calculating a data feature classification path according to an embodiment of the present invention;

[0018] Figure 2 is a flow chart of a method for calculating a data feature classification path according to a specific embodiment of the present invention;

[0019] Figure 3 is a schematic diagram of a computing system for a data feature classification path according to an embodiment of the present invention;

[0020] Figure 4 It is a structural block diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making creative work should fall within the scope of protection of the present invention.

[0022] Furthermore, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concepts disclosed in the present invention.

[0023] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the orientation or position relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. is based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, the terms "first", "second", and "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. The terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0024] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of methods and systems consistent with some aspects of the present invention as detailed in the appended claims.

[0025] The present invention proposes a method, system and related equipment for calculating a data feature classification path. Specifically, the method, system and related equipment for calculating a data feature classification path according to an embodiment of the present invention are described below with reference to the accompanying drawings.

[0026] Figure 1 It is a flow chart of a method for calculating a data feature classification path according to an embodiment of the present invention. It should be noted that the method for calculating a data feature classification path according to an embodiment of the present invention can be applied to a calculation system for a data feature classification path according to an embodiment of the present invention. The calculation system for the data feature classification path can be configured on an electronic device or in a server. This embodiment of the present application does not limit this.

[0027] like Figure 1 As shown, the calculation method of the data feature classification path includes:

[0028] S110, obtaining 5G service data.

[0029] S120, extract features of 5G service data based on the target machine learning model to obtain service features.

[0030] In an embodiment of the present invention, the target machine learning model is trained in the following manner: obtaining a target data set of 5G service data, the target data set consisting of a basic feature set T and a generated feature set S; dividing the target data set into a training set D tr and validation set D vld ; Based on the training set D tr Train the first machine learning model L and based on ε(L(D tr , T+S), D vld , T+S) evaluates the first machine learning model L; ε(L(D tr , T+S), D vld , T+S) means using the training set D tr The first machine learning model L obtained by training is used in the validation set D vld The evaluation index calculated on the validation set D vld If the validation set model does not perform well, the model needs to be optimized. tr , T+S) means using the training set D tr Training model: a first machine learning model L; based on the first data added to the basic feature set T, the first machine learning model L is retrained, and the retrained first machine learning model L is used as the target machine learning model.

[0031] The first data can be understood as newly added features. When new features are added to the basic feature set T, it is necessary to evaluate how much performance improvement these new features can bring. The evaluation procedure includes including the new features in the basic feature set T, retraining the first machine learning model L, and observing the change in validation loss; the input of feature enhancement includes the target data set, the new features T' and the basic prediction The reduction in the feature boosting output loss Δ is used as an evaluation of the incremental performance of the newly added features, and this evaluation drives the feature enhancement algorithm.

[0032] For example, there is a data set D with v samples = {(x i ,y i ), i = 1, 2, ... v}. F represents a set of models defined on basic features. There exists a model f∈F that has been trained and converged on the dataset D. Basic prediction loss Let F' denote a set of models defined on the newly added features T'. Given a new model f i ∈F', to improve the performance of f and minimize Penalize f and optimize f through gradient descent iInitialized with base predictions, feature augmentation can estimate incremental performance by training new features instead of the full feature set, thus converging faster to improve efficiency.

[0033] A 5G big data service feature extraction solution based on feature self-representation is constructed by continuous feature pruning and combining feature importance attribution. This means that in the feature selection process, unimportant features are continuously removed and the importance of features to the results is considered, thereby constructing an efficient feature extraction solution.

[0034] In an embodiment of the present invention, according to Filter out the optimal feature set from the candidate feature set A(T) generated from the basic feature set T Based on the optimal feature set Determine business characteristics.

[0035] Among them, the candidate feature set A(T) is determined by the operator set ο used to transform basic features, and the operator ο includes unary operations (such as log, sigmoid, square) and binary operations (such as *, / , min, max, GroupByThenMean).

[0036] S130, determining cluster characteristics of clustering of business characteristics, determining cluster centroid, cluster radius and cluster diameter according to the cluster characteristics of clustering, calculating the distance between each cluster based on the cluster centroid, cluster radius and cluster diameter, constructing a clustering feature tree based on the cluster characteristics of clustering and construction parameters, and determining cluster characteristics of business characteristics based on the clustering feature tree.

[0037] S140, construct a random forest based on business features and feature clustering, calculate feature importance scores of business features based on random forest and XGBoost models, classify business features whose feature importance scores are greater than preset scores based on a decision classification tree model, and use the classification results as business paths corresponding to the business features.

[0038] According to the calculation method of the data feature classification path of the embodiment of the present invention, 5G business data is obtained; based on the target machine learning model, feature extraction is performed on the 5G business data to obtain business features; cluster features of the clustering of business features are determined, cluster centroids, cluster radius and cluster diameter are determined according to the cluster features of the clustering, the distances between clusters are calculated based on the cluster centroids, cluster radius and cluster diameter, cluster feature trees are constructed based on the cluster features of the clustering and construction parameters, and cluster features of business features are determined based on the cluster feature tree; random forests are constructed according to business features and feature clustering, and feature importance scores of business features are calculated based on random forests and XGBoost models, business features with feature importance scores greater than preset scores are classified based on the decision classification tree model, and the classification results are used as the business paths corresponding to the business features. This method can quickly and accurately calculate the paths of large-scale networks, effectively solving the performance bottleneck of traditional path calculation algorithms.

[0039] In order to make it easier for those skilled in the art to understand the present invention, Figure 2 is a method for calculating a data feature classification path according to a specific embodiment of the present invention, such as Figure 2 As shown, the calculation method of the data feature classification path includes:

[0040] S210, obtaining 5G service data.

[0041] S220, extract features of 5G service data based on the target machine learning model to obtain service features.

[0042] S230: Determine cluster features of the clustering of service features.

[0043] In the embodiment of the present invention, each service feature is defined as a triplet CF, which includes the number of data points N in the cluster, a vector LS and a scalar SS, wherein the vector LS represents the linear sum of each data point, and the scalar SS represents the square sum of each data point; wherein the vector n represents the nth data point, M represents the dimension of the data point, and scalar X ni Represents the i-th dimension data of the n-th data point; obtains the triples of multiple business features, according to CF 1 +CF 2 +...+CF b =(N 1 +N 2 +...+N b ,LS 1 +LS 2 +...+LS b ,SS 1 +SS 2 +...+SS b), superimpose the triplets of multiple business features to obtain the cluster features of the business features; where b represents the number of business features, CF 1 =(N 1 ,LS 1 ,SS 1 ), CF 2 =(N 2 ,LS 2 ,SS 2 ), CF b =(N b ,LS b ,SS b );N 1 To N b Respectively represent the number of data points in the clusters of the 1st to bth business features; LS 1 To LS b Respectively represent the vectors of the 1st to bth service characteristics; SS 1 To SS b Scalars representing the 1st to bth service characteristics respectively.

[0044] S240, determining a cluster centroid, a cluster radius, and a cluster diameter according to cluster characteristics of clustering, and calculating the distance between each cluster based on the cluster centroid, the cluster radius, and the cluster diameter.

[0045] Among them, based on Determine the cluster centroid; based on Determine the cluster radius, which represents the distance from the data point to the cluster centroid; based on Determine the cluster diameter, which represents the average distance between every two data points.

[0046] Among them, based on Calculate the distance between clusters.

[0047] S250: construct a clustering feature tree based on cluster features and construction parameters of clustering, and determine clustering features of business features based on the clustering feature tree.

[0048] In an embodiment of the present invention, the construction parameters include a branch balance factor β, a leaf balance factor λ, and a space threshold τ.

[0049] In an embodiment of the present invention, a branch balance factor β, a leaf balance factor λ and a spatial threshold τ are initialized; a target leaf node closest to a target sample and a target triple CF of the target leaf node are determined from the leaf nodes under the root node, the target triple CF being a triple CF closest to the target sample among multiple triples CF in the target leaf node; it is determined whether, when the target sample is added to the target triple CF, the radius of a hypersphere corresponding to the target triple CF is less than the spatial threshold τ; when it is determined that the radius of a hypersphere corresponding to the target triple CF is less than the spatial threshold τ, all triples CF on the path are updated; when it is determined that the radius of a hypersphere corresponding to the target triple CF is not less than the spatial threshold τ, all triples on the path are updated according to the number of triples CF of the target leaf node.

[0050] Among them, all the triplets on the path are updated according to the number of triples CF of the target leaf node, and the implementation method is as follows: determine whether the number of triples CF of the target leaf node is less than the leaf balance factor λ; when it is determined that the number of triples CF of the target leaf node is less than the leaf balance factor λ, create a first triple under the target leaf node, store the target sample in the first triple, and update all the triplets on the path; when it is determined that the number of triples CF of the target leaf node is not less than the leaf balance factor λ, obtain all triples CF corresponding to the target leaf node, and based on the hypersphere distance, obtain the second triple CF and the third triple CF in all triples CF; divide the target leaf node into a first leaf node and a second leaf node; use the second triple CF or the third triple CF as the first triple CF of the first leaf node or the second leaf node, and put the remaining triples CF and the target sample in all triples CF except the second triple CF and the third triple CF into the first leaf node or the second leaf node according to the distance principle.

[0051] That is to say, the initialization parameters are: branch balance factor β, leaf balance factor λ and spatial threshold τ, among which these parameters play an important control role in the process of constructing the cluster feature tree. β is used to control the balance of the branch structure of the tree, λ is used to control the balance of the number of triple CF nodes in the leaf node, and τ is a spatial metric used to determine whether the radius of the hypersphere meets the requirements; Find the leaf node and triple CF closest to the target sample: search from the root node downward, that is, starting from the root node of the tree, through the distance measurement method, find the leaf node closest to the target sample, and the triple CF closest to the target sample in the leaf node. This step is the basis for subsequent operations, and the purpose is to determine the most likely location of the new sample; Determine whether to update the triple: If the target sample is added to the found triple CF, the hypersphere radius corresponding to the triple CF is still less than the spatial threshold τ, indicating that the addition of the target sample has not caused the clustering hypersphere represented by the triple CF to "expand" excessively. At this point, all triples CF on the path from the root node to the triple CF node can be directly updated, and the insertion operation is completed; judgment of the number of leaf nodes: if the number of triples CF in the current leaf node is less than the leaf balance factor λ, it means that the leaf node still has enough "space" to accommodate the target sample. At this time, create a new triple CF, put the target sample into this new triple CF, and then put the new triple CF into the current leaf node, and then update all triples CF, and the insertion operation ends; leaf node splitting: if the number of triples CF in the current leaf node is not less than the leaf balance factor λ, it means that the leaf node is already "crowded". At this time, divide the current leaf node into two new leaf nodes. Select the two triples CF with the farthest hypersphere distance among all triples CF in the old leaf node, and use them as the first triple CF of the two new leaf nodes. Then, according to the principle of distance, put the other triples CF and the target sample tuple into the corresponding new leaf node.

[0052] It also includes: Parent node split check: that is, check the parent node upwards. After the leaf node splits, it is necessary to check whether the parent node also needs to be split. If the parent node needs to be split, its splitting method is the same as the leaf node splitting method, that is, by judging the number of triple CFs and the radius of the hypersphere, it is decided whether to update the triple CF, create a new triple CF node, or further split.

[0053] For example, the generation example of the clustering feature tree CF-Tree: (1) At the beginning, the clustering feature tree CF-Tree is empty, there is no task sample, and the first sample is read from the target data set D Put it into a new CF triple A, and put this new CF into the root node root; (2) Continue to read the second sample point The sample point and the first sample point are within the range of a hypersphere with a radius of the spatial threshold τ. These two sample points belong to the same CF. The second sample point is added to the CF triple A. (3) Read the third sample point The mattress sample point cannot be integrated into the hypersphere formed in step (2), so a new CF triple B needs to be created to accommodate the new sample point. At this time, the root node has two CF triples A and B;

[0054] Among them, the construction process of the clustering feature tree CF-Tree involves node splitting. Assume that the leaf node LN1 has three triples CF, LN2 and LN3 each have two triples CF, and the leaf balance factor of the leaf node λ=3. At this time, a new sample (i.e., the target sample) is added, which is closest to LN1 and is not in the three hyperspheres of CF1, CF2, and CF3. Then a new CF8 needs to be established to store it, but the current leaf balance factor λ=3, LN1 cannot add a new CF, and LN1 can only be divided into two.

[0055] S260, construct a random forest based on business features and feature clustering.

[0056] In an embodiment of the present invention, random forest is an integrated learning algorithm based on decision trees (decision tree algorithms can use ID3, C4.5 and CART), which is trained by randomly selecting features and samples, each decision tree grows independently, and finally the final result is obtained by voting or averaging.

[0057] S270, calculating the feature importance scores of the business features based on the random forest and XGBoost models, classifying the business features whose feature importance scores are greater than a preset score based on the decision classification tree model, and using the classification results as the business paths corresponding to the business features.

[0058] In an embodiment of the present invention, a first feature importance score of a business feature is calculated based on a random forest, and a first business feature whose first feature importance score is greater than a preset score is obtained; a second feature importance score of the business feature is calculated based on an XGBoost model, and a second business feature whose second feature importance score is greater than a preset score is obtained; a business feature that is the intersection of the first business feature and the second business feature is used as a target business feature; the target business feature is classified based on a decision classification tree model, and the classification result is used as a business path corresponding to the target business feature.

[0059] In an embodiment of the present invention, the implementation method of calculating the first feature importance score of the business feature based on random forest is: input the business feature into the random forest to obtain an error e; add random noise to all business features in turn, and input them into the random forest to obtain an error set E; compare the error e with the elements in the error set E in turn, and the larger the comparison difference, the higher the feature importance score (the higher the score, the greater the impact of the feature on the classification result).

[0060] In an embodiment of the present invention, the implementation method of calculating the second feature importance score of the business feature based on the XGBoost model is: initialize and set the parameters of the XGBoost model, such as learning rate, maximum depth, and number of iterations; train the XGBoost model using the business features; extract feature importance scores from the trained XGBoost model; sort according to the importance scores and select features with higher importance.

[0061] That is to say, select business features with importance scores greater than the threshold θr from the random forest, select business features with feature importance scores greater than the threshold θx from the XGBoost algorithm, and then find the intersection of these two sets of features to obtain the business features selected by both (i.e., the target business features). Then, use the selected business features (i.e., the target business features) to train with other decision classification tree models, such as the Adaboost model, to obtain the final decision classification tree model. Use the test samples in the target business features to perform classification tests on the decision classification tree model, and the classification results obtained are the business paths corresponding to the current business features. In this way, the advantages of random forest and Xgboost can be comprehensively utilized to reduce the limitations of a single algorithm in feature selection, improve the accuracy of feature selection and the performance of the final decision classification tree model.

[0062] According to the calculation method of the data feature classification path of the embodiment of the present invention, the feature extraction of 5G service data by the target machine learning model can mine valuable service features, lay the foundation for subsequent operations such as accurate classification, help focus on key data information, reduce data dimensions and redundancy, and improve subsequent processing efficiency. Use triples to characterize service features and cluster features: define each service feature as a triple containing the number of data points, vectors and scalars in the cluster, and obtain cluster features of clustering by superimposing multiple triples of service features through reasonable mathematical operations. This method can comprehensively and quantitatively describe the relevant characteristics of data after clustering, such as the distribution and concentration of data. According to the cluster features, the cluster centroid, cluster radius and cluster diameter are determined, and the distance between each cluster is calculated based on these parameters. The clustering morphology and mutual relationship are characterized from multiple dimensions, which helps to understand the characteristics of data at the clustering level in more detail, and provide a more accurate basis for the subsequent construction of clustering feature trees and classification operations. By setting construction parameters such as branch balance factor, leaf balance factor and space threshold, the branch structure balance of cluster feature tree (CF-Tree), the balance of triples in leaf nodes and whether the radius of the hypersphere meets the requirements can be effectively controlled, so that the constructed cluster feature tree structure is more reasonable and compact, and can better adapt to different data distribution situations, avoiding problems such as excessive imbalance of tree structure or excessive node crowding. When new samples are inserted, through a series of judgment conditions and corresponding update and split operations, such as judging the relationship between the hypersphere radius and the space threshold after the target sample is added, the number of triples in the leaf node and the leaf balance factor are compared, so that the cluster feature tree can dynamically accept new samples, continuously optimize and update the tree structure, and ensure that the cluster feature tree can always accurately reflect the clustering state of business features. Constructing random forests, with the characteristics of being based on decision trees and training by randomly selecting features and samples, each decision tree grows independently and finally obtains the results by comprehensive voting or averaging, enhances the generalization ability and stability of the model, reduces the risk of overfitting, and improves the accuracy of business feature classification. Combining the random forest and XGBoost models to calculate the feature importance scores of business features and taking the target business features of the intersection of the two can comprehensively utilize the advantages of these two different models in feature selection, make up for the limitations of a single algorithm in feature selection, screen out business features that have a greater impact on the classification results and are more critical, and then classify based on the decision classification tree model to obtain the business path, improve the accuracy and reliability of the final classification, and optimize the overall classification effect. Through the above multiple steps and corresponding mechanisms, this solution can effectively cope with such complex data situations, accurately classify business features and determine business paths, and provide strong support for subsequent further analysis, decision-making, and application development based on business paths.

[0063] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0064] According to one aspect of an embodiment of the present invention, a computing system for a data feature classification path is also proposed. Figure 3 is a schematic diagram of a computing system for a data feature classification path according to an embodiment of the present invention; Figure 3 As shown, including:

[0065] An acquisition module 310 is used to acquire 5G service data;

[0066] An extraction module 320 is used to extract features from the 5G service data based on a target machine learning model to obtain service features;

[0067] A construction module 330, configured to determine cluster features of the clustering of the service features, determine a cluster centroid, a cluster radius and a cluster diameter according to the cluster features of the clustering, calculate the distance between each cluster based on the cluster centroid, the cluster radius and the cluster diameter, construct a cluster feature tree based on the cluster features of the clustering and construction parameters, and determine the cluster features of the service features based on the cluster feature tree;

[0068] The determination module 340 is used to construct a random forest according to the business features and the feature clustering, and calculate the feature importance score of the business features based on the random forest and XGBoost model, classify the business features whose feature importance score is greater than a preset score based on the decision classification tree model, and use the classification result as the business path corresponding to the business feature.

[0069] According to the calculation system of the data feature classification path of the embodiment of the present invention, 5G business data is obtained; based on the target machine learning model, feature extraction is performed on the 5G business data to obtain business features; cluster features of the clustering of business features are determined, cluster centroids, cluster radius and cluster diameter are determined according to the cluster features of the clustering, the distances between clusters are calculated based on the cluster centroids, cluster radius and cluster diameter, cluster feature trees are constructed based on the cluster features of the clustering and construction parameters, and cluster features of business features are determined based on the cluster feature tree; random forests are constructed according to business features and feature clustering, and feature importance scores of business features are calculated based on random forests and XGBoost models, business features with feature importance scores greater than preset scores are classified based on the decision classification tree model, and the classification results are used as the business paths corresponding to the business features. In this way, the paths of large-scale networks can be calculated quickly and accurately, effectively solving the performance bottleneck of traditional path calculation algorithms.

[0070] Optionally, the target machine learning model is trained in the following manner: obtaining a target data set of the 5G service data, wherein the target data set consists of a basic feature set T and a generated feature set S;

[0071] The target data set is divided into a training set D tr and validation set D vld ; Based on the training set D tr Train the first machine learning model L and based on ε(L(D tr , T+S), D vld , T+S) evaluates the first machine learning model L; ε(L(D tr , T+S), D vld , T+S) means using the training set D tr The first machine learning model L obtained by training is used in the validation set D vld The evaluation index calculated above, L(D tr , T+S) means using the training set D tr Training model: a first machine learning model L; based on the first data added to the basic feature set T, retraining the first machine learning model L, and using the retrained first machine learning model L as the target machine learning model.

[0072] Optionally, the extraction module 320 is specifically configured to: Filter out the optimal feature set from the candidate feature set A(T) generated by the basic feature set T Based on the optimal feature set The service characteristics are determined.

[0073] Optionally, the construction module 330 is specifically configured to define each of the service features as a triplet CF, wherein the triplet CF includes the number N of data points in the cluster, a vector LS and a scalar SS, wherein the vector LS represents the linear sum of each data point, and the scalar SS represents the square sum of each data point; wherein the vector n represents the nth data point, M represents the dimension of the data point, and the scalar Xni represents the i-th dimension data of the n-th data point; obtain the triples of the multiple business features, according to CF 1 +CF 2 +...+CF b =(N 1 +N 2 +...+N b ,LS 1 +LS 2 +...+LS b ,SS 1 +SS 2 +...+SS b ), superimposing the triples of the multiple service features to obtain cluster features of the service features, where b represents the number of service features, and the triple CF 1 =(N 1 ,LS 1 ,SS 1 ), triple CF 2 =(N 2 ,LS 2 ,SS 2 ), triple CF b =(N b ,LS b ,SS b );N 1 To N b Respectively represent the number of data points in the clusters of the 1st to bth business features; LS 1 To LS b Respectively represent the vectors of the 1st to bth service characteristics; SS 1 To SS b Respectively represent the scalars of the 1st to bth service characteristics; based on Determine the cluster centroid; based on Determine the cluster radius, where the cluster radius represents the distance from the data point to the cluster centroid; based on Determine the cluster diameter, which represents the average distance between every two data points; based on Calculate the distance between clusters.

[0074] Optionally, the construction parameters include a branch balance factor β, a leaf balance factor λ and a spatial threshold τ, wherein the construction module 330 is specifically used to initialize the branch balance factor β, the leaf balance factor λ and the spatial threshold τ; determine the target leaf node closest to the target sample and the target triplet CF of the target leaf node from the leaf nodes under the root node, the target triplet CF being the triplet CF closest to the target sample among multiple triples CF in the target leaf node; determine whether the radius of the hypersphere corresponding to the target triplet CF is less than the spatial threshold τ when the target sample is added to the target triplet CF; if it is determined that the radius of the hypersphere corresponding to the target triplet CF is less than the spatial threshold τ, update all the triples CF on the path; if it is determined that the radius of the hypersphere corresponding to the target triplet CF is not less than the spatial threshold τ, update all the triples on the path according to the number of triples CF of the target leaf node.

[0075] Optionally, a construction module 330 is specifically used to determine whether the number of triples CF of the target leaf node is less than the leaf balance factor λ; when it is determined that the number of triples CF of the target leaf node is less than the leaf balance factor λ, a first triple is created under the target leaf node, and the target sample is stored in the first triple, and all the triples on the path are updated; when it is determined that the number of triples CF of the target leaf node is not less than the leaf balance factor λ, all triples CF corresponding to the target leaf node are obtained, and based on the hypersphere distance, the second triple CF and the third triple CF in all triples CF are obtained; the target leaf node is divided into a first leaf node and a second leaf node; the second triple CF or the third triple CF is used as the first triple CF of the first leaf node or the second leaf node, and the remaining triples CF in all triples CF except the second triple CF and the third triple CF and the target sample are placed in the first leaf node or the second leaf node according to the principle of distance.

[0076] Optionally, a determination module 340 is used to calculate a first feature importance score of the business feature based on the random forest, and obtain a first business feature whose first feature importance score is greater than the preset score; calculate a second feature importance score of the business feature based on the XGBoost model, and obtain a second business feature whose second feature importance score is greater than the preset score; use the intersection business feature of the first business feature and the second business feature as the target business feature; classify the target business feature based on the decision classification tree model, and use the classification result as the business path corresponding to the target business feature.

[0077] According to one aspect of an embodiment of the present invention, an electronic device is provided.

[0078] Figure 4 Schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Figure 4 As shown, the electronic device may include one or more ( Figure 4 Only one is shown in the figure) processor 102 (processor 102 may include but is not limited to a microprocessor (Microprocessor Unit, referred to as MPU) or a programmable logic device (Programmable logic device, referred to as PLD)) and a memory 104 for storing data. In an exemplary embodiment, the electronic device may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 4 The structure shown is for illustration only and does not limit the structure of the above terminal device. Figure 4 More or fewer components as shown, or with Figure 4 Equivalent functions or comparisons shown Figure 4 A different configuration with more features is shown.

[0079] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the calculation method of the data feature classification path in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0080] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the switching device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0081] The present invention provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute a method for calculating a data feature classification path.

[0082] The applicant of the present invention has made a detailed explanation and description of the implementation examples of the present invention in conjunction with the drawings in the specification. However, those skilled in the art should understand that the above implementation examples are only preferred implementation schemes of the present invention, and the detailed description is only to help readers better understand the spirit of the present invention, but not to limit the scope of protection of the present invention. On the contrary, any improvements or modifications based on the inventive spirit of the present invention should fall within the scope of protection of the present invention.

[0083] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0084] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the above embodiments, or replace some of the technical features therein by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or replacements that can be easily thought of by those skilled in the art within the technical scope disclosed in the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for calculating a data feature classification path, characterized in that: include: S1, obtain 5G service data; S2, extracting features from the 5G service data based on a target machine learning model to obtain service features; S3, determining cluster features of the clustering of the service features, determining a cluster centroid, a cluster radius, and a cluster diameter according to the cluster features of the clusters, calculating distances between clusters based on the cluster centroid, the cluster radius, and the cluster diameter, constructing a cluster feature tree based on the cluster features of the clusters and construction parameters, and determining cluster features of the service features based on the cluster feature tree; S4, constructing a random forest according to the business features and the feature clustering, and calculating the feature importance scores of the business features based on the random forest and XGBoost models, classifying the business features whose feature importance scores are greater than a preset score based on a decision classification tree model, and using the classification results as the business paths corresponding to the business features.

2. The method for calculating the data feature classification path according to claim 1, characterized in that: The target machine learning model is trained in the following way: Obtain a target data set of the 5G service data, where the target data set consists of a basic feature set T and a generated feature set S; The target data set is divided into a training set D tr and validation set D vld ; Based on the training set D tr Train the first machine learning model L and based on ε(L(D tr , T+S), D vld , T+S) evaluates the first machine learning model L; ε(L(D tr , T+S), D vld , T+S) means using the training set D tr The first machine learning model L obtained by training is used in the validation set D vld The evaluation index calculated above, L(D tr , T+S) means using the training set D tr Training model first machine learning model L; Based on the first data added to the basic feature set T, the first machine learning model L is retrained, and the retrained first machine learning model L is used as the target machine learning model.

3. The method for calculating the data feature classification path according to claim 2, characterized in that: Feature extraction is performed on the 5G service data based on the target machine learning model to obtain service features, including: according to Filter out the optimal feature set from the candidate feature set A(T) generated by the basic feature set T Based on the optimal feature set The service characteristics are determined.

4. The method for calculating the data feature classification path according to claim 2, characterized in that: Determining cluster features of the clustering of the service features includes: Each of the business features is defined as a triplet CF, wherein the triplet CF includes the number of data points N in the cluster, a vector LS and a scalar SS, wherein the vector LS represents the linear sum of each data point, and the scalar SS represents the square sum of each data point; wherein the vector n represents the nth data point, M represents the dimension of the data point, and the scalar X ni Represents the i-th dimension data of the n-th data point; Obtain the triplet of the plurality of service features, according to CF1+CF2+...+CF b =(N1+N2+...+N b ,LS1+LS2+...+LS b ,SS1+SS2+...+SS b ), superimposing the triplets of the plurality of the service features to obtain cluster features of the clustering of the service features; wherein b represents the number of service features, triple CF1 = (N1, LS1, SS1), triple CF2 = (N2, LS2, SS2), triple CF b =(N b ,LS b ,SS b ); N1 to N b Respectively represent the number of data points in the clusters of the 1st to bth business features; LS1 to LS b Respectively represent the vectors of the 1st to bth service characteristics; SS1 to SS b scalars representing the 1st to bth service characteristics respectively; Wherein, determining the cluster centroid, cluster radius and cluster diameter according to the cluster characteristics of the clustering includes: based on determining the cluster centroid; based on Determine the cluster radius, where the cluster radius represents the distance from the data point to the cluster centroid; based on Determine the cluster diameter, wherein the cluster diameter represents the average distance between every two data points; The distance between each cluster is calculated based on the cluster centroid, the cluster radius and the cluster diameter, including: based on Calculate the distance between clusters.

5. The method for calculating the data feature classification path according to claim 4, characterized in that: The construction parameters include a branch balance factor β, a leaf balance factor λ and a space threshold τ, wherein a cluster feature tree is constructed based on the cluster features and the construction parameters of the cluster, including: Initialize the branch balance factor β, the leaf balance factor λ and the space threshold τ; Determine a target leaf node closest to the target sample and a target triple CF of the target leaf node from the leaf nodes under the root node, wherein the target triple CF is a triple CF closest to the target sample among multiple triples CF in the target leaf node; Determine whether, when the target sample is added to the target triplet CF, the radius of the hypersphere corresponding to the target triplet CF is less than the spatial threshold τ; When it is determined that the radius of the hypersphere corresponding to the target triplet CF is smaller than the spatial threshold τ, updating all the triplet CFs on the path; When it is determined that the radius of the hypersphere corresponding to the target triple CF is not less than the spatial threshold τ, all the triples on the path are updated according to the number of triple CFs of the target leaf node.

6. The method for calculating the data feature classification path according to claim 5, characterized in that: Updating all the triplets on the path according to the number of triplet CFs of the target leaf node includes: Determine whether the number of triples CF of the target leaf node is less than the leaf balance factor λ; When it is determined that the number of triples CF of the target leaf node is less than the leaf balance factor λ, a first triple is created under the target leaf node, the target sample is stored in the first triple, and all the triples on the path are updated; When it is determined that the number of triplet CFs of the target leaf node is not less than the leaf balance factor λ, all triplet CFs corresponding to the target leaf node are obtained, and based on the hypersphere distance, the second triplet CF and the third triplet CF in all the triplet CFs are obtained; Dividing the target leaf node into a first leaf node and a second leaf node; Use the second triplet CF or the third triplet CF as the first triplet CF of the first leaf node or the second leaf node, and place the remaining triplet CF of all triples CF except the second triplet CF and the third triplet CF and the target sample into the first leaf node or the second leaf node according to the principle of distance.

7. The method for calculating the data feature classification path according to claim 1, characterized in that: Calculating the feature importance score of the business feature based on the random forest and XGBoost models, classifying the business features with a feature importance score greater than a preset score based on a decision classification tree model, and using the classification result as the business path corresponding to the business feature, including: Calculate a first feature importance score of the business feature based on the random forest, and obtain a first business feature whose first feature importance score is greater than the preset score; Calculate the second feature importance score of the business feature based on the XGBoost model, and obtain the second business feature whose second feature importance score is greater than the preset score; Taking the intersection of the first service feature and the second service feature as the target service feature; The target service feature is classified based on the decision classification tree model, and the classification result is used as the service path corresponding to the target service feature.

8. A data feature classification path computing system, characterized in that: include: An acquisition module, used to acquire 5G service data; An extraction module, used to extract features from the 5G service data based on a target machine learning model to obtain service features; A construction module, used to determine the cluster characteristics of the clustering of the service characteristics, determine the cluster centroid, cluster radius and cluster diameter according to the cluster characteristics of the cluster, calculate the distance between each cluster based on the cluster centroid, the cluster radius and the cluster diameter, construct a cluster characteristic tree based on the cluster characteristics of the cluster and construction parameters, and determine the cluster characteristics of the service characteristics based on the cluster characteristic tree; A determination module is used to construct a random forest based on the business features and the feature clustering, and calculate the feature importance scores of the business features based on the random forest and XGBoost models, classify the business features whose feature importance scores are greater than a preset score based on a decision classification tree model, and use the classification results as the business paths corresponding to the business features.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

Images