Prediction model visual comparison and iterative optimization method and system based on feature subsets

Abstract

The invention provides a feature subset-based prediction model visual comparison and iterative optimization method and system, and belongs to the technical field of machine learning and data visualization. The method comprises the following steps: acquiring multivariable time sequence data and constructing a feature subset; constructing a multivariable time series prediction model by using the selected feature subset and adopting an integrated learning method; repeating and constructing different models; performing model correlation calculation from multiple dimensions of feature subset types, feature importance, algorithm weights and model evaluation indexes; recording an exploration process of a user by utilizing a hierarchical node link graph, performing model comparison visual analysis and performing layout optimization; the optimal feature subset is derived after iteratively optimizing the feature subsets. A visual analysis method is applied to the selection of the optimal feature subset, and a user is assisted to iteratively select the feature subset to construct a model in combination with domain knowledge, so that different feature subsets and models are better understood, diagnosed and compared, and the accuracy and credibility of a prediction result are enhanced.

Description

Technical Field

[0001] This invention relates to the field of machine learning and data visualization technology, specifically to a method and system for iteratively selecting feature subsets, comparing model performance, and making optimization decisions on a computer device through an interactive visualization interface during the construction of a time series prediction model. Background Technology

[0002] Cigarette sales data is influenced by a variety of complex factors, including seasonal fluctuations, holiday effects, macroeconomic policies, price changes, and inventory levels, exhibiting significant nonlinearity, volatility, and lag. In the digital transformation and lean supply chain management of the tobacco industry, accurate sales demand forecasting is crucial for guiding cigarette production scheduling, logistics, and retailer ordering.

[0003] In the fields of machine learning and data mining, feature engineering is a crucial step in building high-performance predictive models. However, when faced with high-dimensional, multivariate time-series data generated by tobacco sales (including historical sales figures, social inventory, price structure, and holiday factors), traditional feature selection methods often struggle to effectively capture the complex coupling relationships between variables. Existing technologies largely rely on manual trial and error to find the optimal feature combination, resulting in a blind, inefficient, and difficult-to-track model iteration process.

[0004] Furthermore, during the iterative optimization of models, a large number of candidate model versions are generated. Existing analytical methods lack quantitative calculation methods for the evolutionary relationships between different models, and cannot clearly present the specific impact path of feature subset changes on model accuracy (RMSE). When the number of models surges, the topological structure of model evolutionary relationships becomes extremely complex, leading to wasted computational resources and a loss of optimization direction. Therefore, there is an urgent need for a predictive model construction method that can combine the characteristics of tobacco market sales, quantify model similarity, and automatically optimize the evolutionary path topology to quickly converge to the optimal feature subset. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for visual comparison and iterative optimization of prediction models based on feature subsets, which solves the technical problems of difficult feature selection and low iteration efficiency in tobacco sales prediction models, and greatly improves prediction accuracy as well as the efficiency and effectiveness of feature engineering and model optimization.

[0006] To achieve the above objectives, embodiments of the present invention provide a method for visual comparison and iterative optimization of prediction models based on feature subsets. This method is executed on a computer device, which includes a processor, a memory, and a display connected to the processor. Acquire multivariate time-series data of tobacco sales at the target terminal. The multivariate time-series data includes at least historical sales flow characteristic data, market status characteristic data, and external environment characteristic data. Perform preprocessing and feature reconstruction to construct an initial feature subset. Based on the initially selected feature subset, construct the fusion XGBoost , LightGBM and Pathformer The algorithm's time-series prediction model is proposed, and the model parameters are optimized using a batch gradient descent iterative algorithm, with the root mean square error value used. RMSE As a model evaluation metric; The system receives user instructions on adjusting feature subsets and algorithm weights via a graphical user interface. Based on these instructions, it reconstructs or reselects features to generate new feature subsets. Then, based on these new feature subsets and the adjusted algorithm weight parameters, it trains and generates new prediction models. Each training iteration generates a prediction model along with its corresponding feature subset, algorithm weights, feature importance, and... RMSE Indicators form an exploration history and are stored in memory; For any two models in the exploration history, the similarity between different models is calculated by weighted fusion from four dimensions: feature subset type, feature importance, algorithm weight, and model evaluation index, and the similarity values ​​between different models are obtained. A node link graph is constructed based on similarity values ​​and model generation order to represent the model evolution path. Each node corresponds to a prediction model, and the edges represent the order in which the models are generated. The node link graph guides the iterative selection of subsequent feature subsets. Automatic layout optimization is performed on the node link graph. An integer linear programming (ILP) model is established with the objective function of minimizing the number of edge intersections in the node link graph. The solution is obtained by combining node level, relative position variables, edge intersection variables, and transitivity constraints to obtain the layout coordinates of node positions and connections. The layout coordinates are output as a renderable data structure and rendered on the monitor. Based on the exploration history and interactive operations of the node link graph, the feature subset is iteratively optimized and the best feature subset is output; wherein, feature reconstruction includes feature transformation and feature combination, and, XGBoost , LightGBM and Pathformer The algorithm's output weights are user-defined.

[0007] Preferably, the fusion is constructed based on the initially selected feature subset. XGBoost , LightGBM and Pathformer The algorithm's time-series prediction model includes: For the processed feature set use XGBoost Algorithm modeling, the model's output as the first Each feature is added to the feature set to obtain... This feature set serves as LightGBM The prediction result obtained from the input ; use XGBoost For feature set Modeling yields prediction results ; use Pathformer For feature set Modeling yields prediction results ; right , , The three results are weighted and fused to form the final output of the model.

[0008] Preferably, in XGBoost and LightGBM When training a single model, 5-fold cross-validation is used and the average value is taken to avoid overfitting.

[0009] Preferably, for any two models in the exploration history, the similarity between different models is calculated by weighted fusion from four dimensions: feature subset types, feature importance, algorithm weights, and model evaluation metrics, including data reconstruction: use json The feature subset used by the file storage model, model weights, and the final RMSE index; The feature subset used in the final model is For each feature, the feature importance to the model is calculated, resulting in a feature importance dictionary. ,in, Representation of features The importance of features is ; Model weights use a set R express, ,in, In the fusion model XGBoost The weight it accounts for In the fusion model LightGBM The weight it accounts for In the fusion model LSTM The weight it occupies; Model evaluation metrics adopted M express, Where m is the model's RMSE The value; According to formula (1), the feature subset, feature importance, model weights, and model metrics are integrated into a set. G And adopt jsonFormat representation,

[0010] The model is represented by formulas (2) and (3) respectively. A and model B ,

[0011] .

[0012] Preferably, the step of calculating the similarity between different models by weighted fusion from four dimensions—feature subset types, feature importance, algorithm weights, and model evaluation metrics—for any two models in the exploration history further includes: Calculate the similarity of feature subsets of the model: According to formula (4), Jaccard Similarity set and set The similarity between them

[0013] in, Represents a set and The size of the intersection, Represents a set and The size of the union, The larger the value, the higher the similarity between the two sets; Compute the similarity of feature importance in the models: According to formula (5), cosine similarity is used to calculate the set. and set Similarity between them

[0014] in, This represents the result of multiplying the elements of two vectors one by one. and Represents the length of the vector; if and If the features in the set are not completely identical, then a unified feature space is used to map the elements in each set to a common feature space, and missing features are padded with 0. The larger the value, the higher the similarity. Calculate the similarity of the weights in the model algorithms: Based on formulas (6) and (7), the Euclidean distance is used to calculate the set. and Similarity,

[0015]

[0016] Formula (7) is used to quantize the result to a value between 0 and 1. Compute the similarity of evaluation metrics for the models: Based on formulas (8) and (9), the absolute value of the difference is used to measure... and Similarity,

[0017]

[0018] Formula (9) is used to quantize the result to a range of 0 to 1.

[0019] Preferably, the method of calculating the similarity between different models by weighted fusion from four dimensions—feature subset types, feature importance, algorithm weights, and model evaluation metrics—for any two models in the exploration history, further includes calculating multi-dimensional similarity values ​​for different models: According to formula (10), the four similarity measures of different dimensions are weighted and summed, and the similarity is quantified into the interval (0,1).

[0020] Among them, the weighting coefficient , , , Constraints must be met: .

[0021] Preferably, the results of each feature subset selection used by the user to build the model are recorded, forming a historical exploration record. Comparing the results of historical models helps the user optimize subsequent feature selection processes, including: Designed to visualize composite primitives of a single model, it includes at least a first annular plot representing the weight distribution of each algorithm within the model, and a plot representing the model itself. RMSE A second annular plot of index values, and a surrounding radial histogram to represent the importance of each feature; The entire exploration process of the user is visualized in the form of a node link graph, where each node is a composite primitive of the model. The connecting edges between nodes include parent-child relationship edges that indicate the order in which the models are generated, and association relationship edges that indicate that the comprehensive similarity between non-parent-child models exceeds a preset threshold. The node connection graph is automatically optimized by establishing an integer linear programming model. ILPThe model uses minimizing the number of edge intersections in the graph as the objective function. It combines node level, relative position variables, edge intersection variables, and transitivity constraints to solve the problem and obtain a clear layout with the fewest edge intersections.

[0022] Preferably, the entire user exploration process is visualized in the form of a node-linked graph, wherein each node is a composite primitive of the model, and the connecting edges between nodes include parent-child relationship edges representing the order in which models are generated, and association relationship edges representing the overall similarity between non-parent-child models exceeding a preset threshold; including: Define a graph: Given a suitable hierarchical node link graph G , containing node set V and edge set E Each node There exists a hierarchical allocation function. ,in K It is the total number of layers, ensuring all edges Connect nodes at different levels; Define constraints: Given a graph G The standard model is defined using position variables, cross variables, cross constraints, and transitive constraints. ,in, In position variables, use Represents nodes in the same layer i and j The position of the node i At the node j Above, Conversely ,and, ; In the cross variable, use Represents the edges in the graph and Whether they intersect, if the edges and edge Intersection, then ,otherwise ,and, ; In the cross constraint, the cross variable is defined according to formula (12) and formula (13). And ensure that if the side and Crossing ,

[0023]

[0024] Formula (12) is used to ensure that when exist Above and exist Below, cross variables Formula (13) is used to ensure that when exist Below and exist When above, cross variables ; In transitivity constraints, transitivity constraints are directly applied to all node triples at the same level according to formulas (14) and (15).

[0025]

[0026] Among them, if and =1, then ;like and =0, then .

[0027] Preferably, the step of establishing an integer linear programming problem... ILP The model includes: Construct position variables, cross variables, cross constraints, and transitive constraints into ILP The model, in which the effective replication of position variables and cross variables corresponds to the layout of the graph, and the sum of all cross variables is the number of edge crosses in the graph; the replication of cross variables with the minimum sum represents the layout with the fewest crosses, and the model minimizes all cross variables according to the objective function formula (16). The sum of,

[0028] in, Indicates the first Layer and first Edge sets between layers; The cross variable represents the edge. and edge Whether it crosses or not, if it crosses, it is 1, otherwise it is 0.

[0029] Preferably, the step of combining node hierarchy, relative position variables, edge intersection variables, and transitivity constraints to obtain a clear layout with the fewest edge intersections includes: For a given input graph, encode it using cross constraints and transitivity constraints as Model, and pass it to Solver, The solver finds the assignment that minimizes the objective function, processes the assignment, and generates an optimized hierarchical node link graph; wherein, Select the first node pair in the same layer. and corresponding Assuming node Located at node Below this, a symmetry breaking method is introduced to accelerate the solver's efficiency; simultaneously, based on this, the method that appears most frequently in the cross constraints is selected. Set the variable to 0.

[0030] Another aspect of the present invention provides a visual comparison and iterative optimization system for prediction models based on feature subsets, the system comprising: The data acquisition module is used to acquire multivariate time-series data on tobacco sales, including historical sales, inventory status, and market environment data. The model building module is used to construct fusion models based on feature subsets. XGBoost , LightGBM and Pathformer The algorithm's time-series prediction model; The similarity calculation module is used to calculate the multi-dimensional similarity between different iterative versions of the model; The layout optimization module is used to calculate the topological layout of the model evolution graph using the ILP model described above. It also includes a feedback update module, which responds to feature adjustment instructions and triggers model retraining until the optimal feature subset and model are obtained to output the predicted tobacco sales volume.

[0031] Through the above technical solution, the present invention stores the model exploration process in a structured way as an exploration history, and generates a node link graph based on the record, so that the model relationship can be uniformly managed, called and visualized and reconstructed in the computer system; by establishing and solving the ILP layout optimization model, with minimizing the number of edge intersections as the objective function, the edge intersections and occlusions of the node link graph on the display can be significantly reduced, and the recognizability and stability of the graphic display can be improved.

[0032] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0033] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating the overall architecture of the visual comparison and iterative optimization method for prediction models based on feature subsets provided by the present invention. Figure 2 These are the primitives representing the model in the visual comparison and iterative optimization method for prediction models based on feature subsets provided by this invention. Figure 3 This is a schematic diagram of the cross constraint in the minimization cross point optimization of the prediction model based on feature subsets provided by the present invention; Figure 4 This is a node link graph with symmetry in the minimization of crosspoint optimization in the visual comparison and iterative optimization method of the prediction model based on feature subsets provided by the present invention. Figure 5 This is a node link diagram of different exploration stages of the visual comparison and iterative optimization method for prediction models based on feature subsets provided by the present invention; Figure 6 This is a visual analysis system diagram illustrating the visual comparison and iterative optimization method for prediction models based on feature subsets provided by the present invention. Detailed Implementation

[0034] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0035] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application all comply with relevant laws and regulations. In the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0036] This invention provides a method for visual comparison and iterative optimization of prediction models based on feature subsets. This method is executed on a computer device, which includes a processor, memory, and a display connected to the processor. Acquire multivariate time-series data of tobacco sales at the target terminal. The multivariate time-series data includes at least historical sales flow characteristic data, market status characteristic data, and external environment characteristic data. Perform preprocessing and feature reconstruction to construct an initial feature subset. Based on the initially selected feature subset, construct the fusion XGBoost , LightGBM and Pathformer The algorithm's time-series prediction model is proposed, and the model parameters are optimized using a batch gradient descent iterative algorithm, with the root mean square error value used. RMSE As a model evaluation metric; The system receives user instructions on adjusting feature subsets and algorithm weights via a graphical user interface. Based on these instructions, it reconstructs or reselects features to generate new feature subsets. Then, based on these new feature subsets and the adjusted algorithm weight parameters, it trains and generates new prediction models. Each training iteration generates a prediction model along with its corresponding feature subset, algorithm weights, feature importance, and... RMSE Indicators form an exploration history and are stored in memory; For any two models in the exploration history, the similarity between different models is calculated by weighted fusion from four dimensions: feature subset type, feature importance, algorithm weight, and model evaluation index, and the similarity values ​​between different models are obtained. A node link graph is constructed based on similarity values ​​and model generation order to represent the model evolution path. Each node corresponds to a prediction model, and the edges represent the order in which the models are generated. The node link graph guides the iterative selection of subsequent feature subsets. Automatic layout optimization is performed on the node link graph. An integer linear programming (ILP) model is established with the objective function of minimizing the number of edge intersections in the node link graph. The solution is obtained by combining node level, relative position variables, edge intersection variables, and transitivity constraints to obtain the layout coordinates of node positions and connections. The layout coordinates are output as a renderable data structure and rendered on the monitor. Based on the exploration history and interactive operations of the node link graph, the feature subset is iteratively optimized and the best feature subset is output; wherein, feature reconstruction includes feature transformation and feature combination, and, XGBoost , LightGBM and Pathformer The algorithm's output weights are user-defined.

[0037] The above-mentioned feature subset is used to construct a fusion. XGBoost , LightGBM and Pathformer The algorithm's time-series prediction model includes: For the processed feature set use XGBoost Algorithm modeling, the model's output as the first Each feature is added to the feature set to obtain... This feature set serves as LightGBM The prediction result obtained from the input ; use XGBoost For feature set Modeling yields prediction results ; use Pathformer For feature set Modeling yields prediction results ; right , , The three results are weighted and fused to form the final output of the model.

[0038] Furthermore, in this embodiment, in XGBoost and LightGBM When training a single model, 5-fold cross-validation is used and the average value is taken to avoid overfitting.

[0039] The above method calculates the similarity between any two models in the exploration history by weighted fusion across four dimensions: feature subset types, feature importance, algorithm weights, and model evaluation metrics, including data reconstruction. use json The feature subset used by the file storage model, model weights, and the final RMSE index; The feature subset used in the final model is For each feature, the feature importance to the model is calculated, resulting in a feature importance dictionary. ,in, Representation of features The importance of features is ; Model weights use a set R express, ,in, In the fusion model XGBoost The weight it accounts for In the fusion model LightGBM The weight it accounts for In the fusion model LSTM The weight it occupies; Model evaluation metrics adopted M express, Where m is the model's RMSE The value; According to formula (1), the feature subset, feature importance, model weights, and model metrics are integrated into a set. G And adopt json Format representation,

[0040] The model is represented by formulas (2) and (3) respectively. A and model B ,

[0041] .

[0042] In addition, it also includes: Calculate the similarity of feature subsets of the model: According to formula (4), Jaccard Similarity set and set The similarity between them

[0043] in, Represents a set and The size of the intersection, Represents a set and The size of the union, The larger the value, the higher the similarity between the two sets; Compute the similarity of feature importance in the models: According to formula (5), cosine similarity is used to calculate the set. and set Similarity between them

[0044] in, This represents the result of multiplying the elements of two vectors one by one. and Represents the length of the vector; if and If the features in the set are not completely identical, then a unified feature space is used to map the elements in each set to a common feature space, and missing features are padded with 0. The larger the value, the higher the similarity. Calculate the similarity of the weights in the model algorithms: Based on formulas (6) and (7), the Euclidean distance is used to calculate the set. and Similarity,

[0045]

[0046] Formula (7) is used to quantize the result to a value between 0 and 1. Compute the similarity of evaluation metrics for the models: Based on formulas (8) and (9), the absolute value of the difference is used to measure... and Similarity,

[0047]

[0048] Formula (9) is used to quantize the result to a range of 0 to 1.

[0049] Furthermore, the above-mentioned method calculates the similarity between any two models in the exploration history by weighted fusion of four dimensions: feature subset types, feature importance, algorithm weights, and model evaluation metrics. It also includes calculating multi-dimensional similarity values ​​between different models. According to formula (10), the four similarity measures of different dimensions are weighted and summed, and the similarity is quantified into the interval (0,1).

[0050] Among them, the weighting coefficient , , , Constraints must be met: .

[0051] In this implementation, the results of each feature subset selection used by the user to build the model are recorded, forming a historical exploration record. By comparing the results of the historical models, the user can optimize the subsequent feature selection process, including: Designed to visualize composite primitives of a single model, it includes at least a first annular plot representing the weight distribution of each algorithm within the model, and a plot representing the model itself. RMSE A second annular plot of index values, and a surrounding radial histogram to represent the importance of each feature; The entire exploration process of the user is visualized in the form of a node link graph, where each node is a composite primitive of the model. The connecting edges between nodes include parent-child relationship edges that indicate the order in which the models are generated, and association relationship edges that indicate that the comprehensive similarity between non-parent-child models exceeds a preset threshold. The node connection graph is automatically optimized by establishing an integer linear programming model. ILP The model uses minimizing the number of edge intersections in the graph as the objective function. It combines node level, relative position variables, edge intersection variables, and transitivity constraints to solve the problem and obtain a clear layout with the fewest edge intersections.

[0052] Specifically, the above-mentioned visualization of the user's entire exploration process in the form of a node-linked graph, wherein each node is a composite primitive of the model, and the connecting edges between nodes include parent-child relationship edges indicating the order in which models are generated, and association relationship edges indicating that the comprehensive similarity between non-parent-child models exceeds a preset threshold; including: Define a graph: Given a suitable hierarchical node link graph G , containing node set V and edge set E Each node There exists a hierarchical allocation function. ,in K It is the total number of layers, ensuring all edges Connect nodes at different levels; Define constraints: Given a graph G The standard model is defined using position variables, cross variables, cross constraints, and transitive constraints. ,in, In position variables, use Represents nodes in the same layer i and j The position of the node i At the node j Above, Conversely ,and, ; In the cross variable, use Represents the edges in the graph and Whether they intersect, if the edges and edge Intersection, then ,otherwise ,and, ; In the cross constraint, the cross variable is defined according to formula (12) and formula (13). And ensure that if the side and Crossing ,

[0053]

[0054] Formula (12) is used to ensure that when exist Above and exist Below, cross variables Formula (13) is used to ensure that when exist Below and exist When above, cross variables ; In transitivity constraints, transitivity constraints are directly applied to all node triples at the same level according to formulas (14) and (15).

[0055]

[0056] Among them, if and =1, then ;like and =0, then .

[0057] In this embodiment, the establishment of integer linear programming ILP The model includes: Construct position variables, cross variables, cross constraints, and transitive constraints into ILP The model, in which the effective replication of position variables and cross variables corresponds to the layout of the graph, and the sum of all cross variables is the number of edge crosses in the graph; the replication of cross variables with the minimum sum represents the layout with the fewest crosses, and the model minimizes all cross variables according to the objective function formula (16). The sum of,

[0058] in, Indicates the first Layer and first Edge sets between layers; The cross variable represents the edge. and edge Whether it crosses or not, if it crosses, it is 1, otherwise it is 0.

[0059] In this embodiment, the step of combining node hierarchy, relative position variables, edge intersection variables, and transitivity constraints to obtain a clear layout with the fewest edge intersections includes: For a given input graph, encode it using cross constraints and transitivity constraints as Model, and pass it to Solver, The solver finds the assignment that minimizes the objective function, processes the assignment, and generates an optimized hierarchical node link graph; wherein, Select the first node pair in the same layer. and corresponding Assuming node Located at node Below this, a symmetry breaking method is introduced to accelerate the solver's efficiency; simultaneously, based on this, the method that appears most frequently in the cross constraints is selected. Set the variable to 0.

[0060] Furthermore, this invention also provides a visual comparison and iterative optimization system for prediction models based on feature subsets, the system comprising: The data acquisition module is used to acquire multivariate time-series data on tobacco sales, including historical sales, inventory status, and market environment data. The model building module is used to construct fusion models based on feature subsets. XGBoost , LightGBM and Pathformer The algorithm's time-series prediction model; The similarity calculation module is used to calculate the multi-dimensional similarity between different iterative versions of the model; The layout optimization module is used to calculate the topological layout of the model evolution graph using the ILP model described above. It also includes a feedback update module, which responds to feature adjustment instructions and triggers model retraining until the optimal feature subset and model are obtained to output the predicted tobacco sales volume.

[0061] The following provides a specific embodiment to illustrate the present invention: Step 1: The user selects a cigarette sales dataset from a tobacco company in a specific region. This dataset covers sales records from the past three years. The original data contains 19 features, including sales characteristics, market status characteristics, and external environment characteristics. After cleaning (removing outliers and imputing missing values) and normalizing the data, an initial feature subset is created using the original feature data.

[0062] Step 2: Construction of a multivariate time series prediction model. A model is created from the constructed feature subset. The model consists of the algorithms XGBoost, LightGBM, and Pathformer, where the weights of the results from these three algorithms can be customized. Specifically, the time series prediction model of this invention integrates the traditional tree model XGBoost and LightGBM with the deep learning model Pathformer.

[0063] First, the processed feature set The XGBoost algorithm is used for modeling, and the model's output is used as the (a+1)th feature, which is then added to the feature set to obtain... The prediction result obtained by using this feature set as input to LightGBM ; Subsequently, XGBoost is used to process the feature set. Modeling yields prediction results The Pathformer is used to process the feature set. Modeling yields prediction results ; Ultimately, adopted , , The weighted fusion of the three results is used as the final output of the model.

[0064] Here, to avoid overfitting, 5-fold cross-validation and averaging are used during the training of both XGBoost and LightGBM single models. Optimal parameters are obtained using a batch gradient descent iterative algorithm during the training of the hybrid model.

[0065] Step 3: Repeat Step 2. Based on user domain knowledge, reconstruct the original features, selecting different feature subsets and different algorithm weights to build different models. The resulting node connection graph is shown below. Figure 5 As shown in (a).

[0066] Step 4: Method for calculating the correlation between prediction models. This invention calculates the similarity between different models by fusing four dimensions: feature subset types, feature importance, algorithm weights, and model evaluation metrics. The main steps are as follows: Step 4.1, Data Reconstruction. Store the feature subset used by the model, model weights, and the final RMSE metric in a JSON file. Assume the final feature subset used by the model is... For each feature, its importance to the model can be calculated, resulting in a feature importance dictionary, denoted as . , Representative characteristics The importance of features is The model weights are represented by a set R. , This represents the weight of XGBoost in the fusion model. This represents the weight of LightGBM in the fusion model. This represents the weight of the LSTM in the fusion model. The model evaluation metric is denoted by M. , where m represents the RMSE value of the model. Then, the feature subset, feature importance, model weights, and model metrics are merged into a set G, represented in JSON format, resulting in the following dataset, as shown in formula (1). Model A and Model B can be represented as formulas (2) and (3):

[0067]

[0068]

[0069] Step 4.2: Calculate the similarity of feature subsets of the model. Use the Jaccard similarity set. and set The similarity between them is shown in formula (4):

[0070] in, Represents a set and The size of the intersection of the two sets is the number of features that exist in both sets. Represents a set and The size of the union of the two sets is the number of all unique features in the two sets. The larger the value, the higher the similarity between the two sets.

[0071] Step 4.3: Calculate the similarity of model feature importance. Cosine similarity is used to calculate the similarity of the set. and set The similarity between them is shown in formula (5):

[0072] in, Represents two vectors (i.e., sets) and The result of multiplying the elements of the value in the table one by one; and Indicates the length of the vector. When and The features in each set are not completely identical. We can unify the feature space for them, map the elements in each set to a common feature space, and then fill in the missing features with 0. A higher value indicates a higher similarity.

[0073] Step 4.4: Calculate the similarity of the model algorithm weights. Use Euclidean distance to calculate the similarity of the set. and The similarity is shown in formulas (6) and (7):

[0074]

[0075] Formula (7) is used to quantize the result to a range of 0 to 1.

[0076] Step 4.5: Calculate the similarity of the model evaluation indicators. The absolute value of the difference is used to measure this. and The similarity is shown in Equations (8) and (9):

[0077]

[0078] Formula (9) is used to quantify the result to a range of 0 to 1.

[0079] Step 4.6: Calculate the multi-dimensional similarity values ​​of different models. Finally, to evaluate the similarity between Model A and Model B, a weighted comprehensive similarity method is used, summing the four different similarity measures in a weighted manner. The magnitude of the similarity is quantified into intervals. As shown in formulas (10) and (11):

[0080] Among them, the weighting coefficient , , , The following constraints must be met:

[0081] The weights mentioned above are used to balance the influence of each similarity metric in the overall similarity calculation. By setting the weights appropriately, the similarity of certain dimensions can be emphasized according to the needs of specific application scenarios. For example, in some cases, more attention may be paid to the similarity of model performance metrics, in which case the weights can be appropriately increased. The value of . In practical applications, it may be necessary to determine the most suitable combination of weights through experiments or expert experience to ensure the effectiveness and reliability of the overall similarity measure.

[0082] Example Application Analysis: The user sets the feature category weight to 0.5 and the performance index to 0.5, then calculates their correlation. The system connects primitives with a correlation threshold greater than 0.75, such as... Figure 5 As shown in (b), primitives T-301 and T-307 exhibit high similarity in feature set types. However, examining the innermost ring reveals a significant difference in their RMSE values. Clicking to view the matrix views corresponding to T-301 and T-307 reveals that T-301 includes the feature "Social Inventory_Month-on-Month Growth Rate," while T-307 does not. This single feature difference leads to a substantial gap in evaluation metrics (T-301 demonstrates significantly higher accuracy). This finding validates the inhibitory or stimulating effect of "inventory" on subsequent order volume in tobacco sales. Therefore, users consider "social inventory" features to be key features in tobacco sales forecasting scenarios. Consequently, the feature subsets used in subsequent exploration processes will include this type of feature.

[0083] Step 5: Visual Analysis of Model Comparison. Design a visual analysis method to showcase the user's workflow in exploring the optimal feature subset and assist users in comparing model differences. Specifically, record the results of each feature subset selection and model building by the user, creating a historical exploration record. By comparing historical model results, help users optimize subsequent feature selection processes. This design method will significantly improve the efficiency of iteration and optimization during feature selection, driving feature engineering towards a more refined and efficient direction. The main steps are as follows: Step 5.1, Visual design method for a single model. For example... Figure 2 As shown, each model is represented by a primitive, the content of which includes the feature types, feature importance, algorithm weights, and model evaluation metrics. Specifically, the primitive design of a single model combines pie charts and bar charts, divided into three layers, each carrying different information display functions. The first layer uses a pie chart to show the proportion of different algorithms in the model; the larger the arc of the pie chart, the greater the weight of the algorithm, with a total weight sum of 1. The second layer uses a single pie chart to represent the magnitude of the model's validation metric. The third layer presents the importance of different features in the form of a bar chart, which is a traditional bar chart bent into a circle.

[0084] Step 5.2: Visual Design Method and Layout Optimization of the Exploration Process. A node link graph is used to record the user's exploration process. The entire node link graph represents the user's exploration of a feature subset, with each node representing a model, represented by primitives. When a user creates a model using the selected feature subset, a node is generated; the current node is generated based on the previous node. There are two types of node relationships in the node link graph: when a node is generated based on the previous node, it is called a parent-child node, connected by a light gray line; when the correlation between non-parent-child nodes (implemented according to the correlation calculation method in Step 4) is greater than a set threshold, they are called related nodes, connected by a red line. To avoid a chaotic layout in the node link graph, a hierarchical node link graph layout optimization algorithm is introduced. The specific algorithm steps are as follows: Step 5.2.1, Define the graph. Given a suitable hierarchical node-linked graph G, containing a node set V and an edge set E, where each node... There exists a hierarchical allocation function. Where K is the total number of layers, ensuring all edges Connect nodes at different levels.

[0085] Step 5.2.2: Define Constraints. Given graph G, define the standard model using the following variables and constraints. : (1) Position variables: using This indicates the positions of nodes i and j on the same level. If node i is above node j, then... ,on the contrary .have .

[0086] (2) Cross variables: using Represents the edges in the graph and Whether they intersect. If the edges... and edge Intersection, then ,otherwise And there is here. .

[0087] (3) Cross constraints:

[0088]

[0089] These constraints apply to each cross variable. Define and ensure that if the edge and Crossing Formula (12) ensures that when exist Above and exist Below, cross variables Formula (13) ensures that when exist Below and exist When above, cross variables .

[0090] (4) Transitivity constraints: Transitivity is ensured by directly applying transitivity constraints to all triples of nodes at the same level. The constraint formulas are as follows:

[0091]

[0092] Formulas (14) and (15) guarantee the following relationship: if and =1, then ;if and =0, then In other words, if exist Above, and exist Above that, then the requirements are... It is also necessary Above. Conversely, if exist Below, and exist Under these circumstances, then the requirements are... It is also necessary Below, these constraints are added to the ILP model, applying to all node triples at the same level. , , This ensures that the relative positions of all nodes within a layer follow transitivity.

[0093] Step 5.2.3: Construct the ILP model. Construct an ILP model using the above variables and constraints. Effective replication of position and intersection variables corresponds to the layout of the graph. The sum of all intersection variables represents the number of edge intersections in the graph. Therefore, the replication of intersection variables with the minimum sum represents the layout with the fewest intersections. The objective function is as follows:

[0094] in, Represents the r-th layer and the r-th layer Edge sets between layers; It is a cross variable, representing an edge. and edge Whether there is a crossover (1 for crossover, 0 for otherwise). In short, this objective function minimizes all crossover variables. The sum of.

[0095] Step 5.2.4: Solver Solution. For a given input graph, the constraints described above are used to encode it into an ILP model, which is then passed to the ILP solver. The ILP solver attempts to find the assignment that minimizes the objective function, and ultimately processes these assignments to generate an optimized hierarchical node link graph. To accelerate the solver's efficiency, a symmetry breaking method is introduced here. The symmetry diagram is shown below. Figure 4 As shown. Select the first pair of nodes in the same layer (let's assume it's node 1). and ) corresponding That is, assuming nodes Located at node Below. Based on this, the present invention selects the element that appears most frequently in the cross constraint. The variable fixes it to This approach provides the solver with a more favorable starting point, helping to find the optimal values ​​for other variables more efficiently and simplifying the model as much as possible.

[0096] When the user clicks "Layout Optimization," the node connection graph is rearranged, resulting in... Figure 5(c) The hierarchical node link graph has been rearranged, reducing the number of intersections from 10 to 0, making it visually clearer and easier to compare models. As shown in the graph, highly related primitives are placed on the same layer. This design strategy effectively reduces the user's visual cognitive load and makes the relationship features between models clearer and more intuitive.

[0097] Step 6: Iteratively optimize the feature subset to obtain the optimal feature subset. After user interaction, a new round of feature selection can be performed based on detailed information of historical or previously selected feature subsets and the performance metrics of the constructed model. This iterative optimization of the feature subset will then be displayed in the visual analytics system. For example... Figure 6 As shown, in addition to the node link diagram, the view combines bar charts, heat map matrices, stripe charts, and line charts to create a visual interactive form that supports human-in-the-loop participation in the entire feature subset construction process. This forms a clear, effective, and novel multi-view, hierarchical visual analysis system that ranges from overview to detail, helping users dynamically select feature subsets to iteratively improve the model.

[0098] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0099] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0100] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The function specified in one or more boxes.

[0101] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0102] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0103] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0104] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0105] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0106] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for visual comparison and iterative optimization of prediction models based on feature subsets, characterized in that, The method is executed on a computer device, the computer device including a processor, a memory, and a display connected to the processor, including: Acquire multivariate time-series data of tobacco sales at the target terminal. The multivariate time-series data includes at least historical sales flow characteristic data, market status characteristic data, and external environment characteristic data. Perform preprocessing and feature reconstruction to construct an initial feature subset. Based on the initially selected feature subset, construct the fusion XGBoost , LightGBM and Pathformer The algorithm's time-series prediction model is proposed, and the model parameters are optimized using a batch gradient descent iterative algorithm, with the root mean square error value used. RMSE As a model evaluation metric; The system receives user instructions regarding feature subsets and algorithm weights in a graphical user interface. Based on these instructions, it reconstructs or reselects features to generate new feature subsets. Then, based on these new feature subsets and the adjusted algorithm weight parameters, it trains and generates a new prediction model. Each training-generated prediction model, along with its corresponding feature subsets, algorithm weights, feature importance, and... RMSE Indicators are used to form an exploration history and stored in the memory; For any two models in the exploration history, the similarity between different models is calculated by weighted fusion from four dimensions: feature subset type, feature importance, algorithm weight, and model evaluation index, and the similarity values ​​between different models are obtained. A node link graph is constructed based on the similarity values ​​and the model generation order to represent the model evolution path; wherein, each node corresponds to a prediction model, the edges represent the order in which the models are generated, and the node link graph guides the iterative selection of subsequent feature subsets. Automatic layout optimization is performed on the node link graph. An integer linear programming (ILP) model is established with the objective function of minimizing the number of edge intersections in the node link graph. The solution is obtained by combining node level, relative position variables, edge intersection variables, and transitivity constraints to obtain the layout coordinates of node positions and connections. The layout coordinates are output as a renderable data structure and rendered and displayed on the display. Based on the exploration history and the interactive operations of the node link graph, the feature subset is iteratively optimized and the optimal feature subset is output; wherein, the feature reconstruction includes feature transformation and feature combination, and the XGBoost , LightGBM and Pathformer The algorithm's output weights are user-defined.

2. The method for visual comparison and iterative optimization of prediction models based on feature subsets according to claim 1, characterized in that, The fusion is constructed based on the initially selected feature subset. XGBoost , LightGBM and Pathformer The algorithm's time-series prediction model includes: For the processed feature set use XGBoost Algorithm modeling, the model's output as the first Each feature is added to the feature set to obtain... This feature set serves as LightGBM The prediction result obtained from the input ; use XGBoost For feature set Modeling yields prediction results ; use Pathformer For feature set Modeling yields prediction results ; right , , The three results are weighted and fused to form the final output of the model.

3. The method for visual comparison and iterative optimization of prediction models based on feature subsets according to claim 2, characterized in that, exist XGBoost and LightGBM When training a single model, 5-fold cross-validation is used and the average value is taken to avoid overfitting.

4. The method for visual comparison and iterative optimization of prediction models based on feature subsets according to claim 1, characterized in that, The method calculates the similarity between any two models in the exploration history by weighted fusion from four dimensions: feature subset types, feature importance, algorithm weights, and model evaluation metrics, including data reconstruction. use json The feature subset used by the file storage model, model weights, and the final RMSE index; The feature subset used in the final model is For each feature, the feature importance to the model is calculated, resulting in a feature importance dictionary. ,in, Representation of features The importance of features is ; Model weights use a set R express, ,in, In the fusion model XGBoost The weight it accounts for In the fusion model LightGBM The weight it accounts for In the fusion model LSTM The weight it occupies; Model evaluation metrics adopted M express, Where m is the model's RMSE The value; According to formula (1), the feature subset, feature importance, model weights, and model metrics are integrated into a set. G And adopt json Format representation, The model is represented by formulas (2) and (3) respectively. A and model B , 。 5. The method for visual comparison and iterative optimization of prediction models based on feature subsets according to claim 4, characterized in that, The method of calculating the similarity between any two models in the exploration history by weighted fusion from four dimensions—feature subset types, feature importance, algorithm weights, and model evaluation metrics—also includes: Calculate the similarity of feature subsets of the model: According to formula (4), Jaccard Similarity set and set The similarity between them in, Represents a set and The size of the intersection, Represents a set and The size of the union, The larger the value, the higher the similarity between the two sets; Compute the similarity of feature importance in the models: According to formula (5), cosine similarity is used to calculate the set. and set Similarity between them in, This represents the result of multiplying the elements of two vectors one by one. and Represents the length of the vector; if and If the features in the set are not completely identical, then a unified feature space is used to map the elements in each set to a common feature space, and missing features are padded with 0. The larger the value, the higher the similarity. Calculate the similarity of the weights in the model algorithms: Based on formulas (6) and (7), the Euclidean distance is used to calculate the set. and Similarity, Formula (7) is used to quantize the result to a value between 0 and 1. Compute the similarity of evaluation metrics for the models: Based on formulas (8) and (9), the absolute value of the difference is used to measure... and Similarity, Formula (9) is used to quantize the result to a range of 0 to 1.

6. The method for visual comparison and iterative optimization of prediction models based on feature subsets according to claim 5, characterized in that, The method calculates the similarity between any two models in the exploration history by weighted fusion of four dimensions: feature subset types, feature importance, algorithm weights, and model evaluation metrics. It also includes calculating multi-dimensional similarity values ​​for different models. According to formula (10), the four similarity measures of different dimensions are weighted and summed, and the similarity is quantified into the interval (0,1). Among them, the weighting coefficient , , , Constraints must be met: .

7. The method for visual comparison and iterative optimization of prediction models based on feature subsets according to claim 1, characterized in that, Record the results of each feature subset selection used by the user to build the model, creating a historical exploration record. By comparing the results of historical models, users can optimize subsequent feature selection processes, including: Designed to visualize composite primitives of a single model, it includes at least a first annular plot representing the weight distribution of each algorithm within the model, and a plot representing the model itself. RMSE A second annular plot of index values, and a surrounding radial histogram to represent the importance of each feature; The entire exploration process of the user is visualized in the form of a node link graph, where each node is a composite primitive of the model. The connecting edges between nodes include parent-child relationship edges that indicate the order in which the models are generated, and association relationship edges that indicate that the comprehensive similarity between non-parent-child models exceeds a preset threshold. The node connection graph is automatically optimized by establishing an integer linear programming model. ILP The model uses minimizing the number of edge intersections in the graph as the objective function. It combines node hierarchy, relative position variables, edge intersection variables, and transitivity constraints to solve the problem and obtain a clear layout with the fewest edge intersections.

8. The method for visual comparison and iterative optimization of prediction models based on feature subsets according to claim 7, characterized in that, The entire user exploration process is visualized in the form of a node-linked graph, where each node is a composite primitive of the model. The connecting edges between nodes include parent-child relationship edges indicating the order in which models are generated, and association relationship edges indicating that the comprehensive similarity between non-parent-child models exceeds a preset threshold; including: Define a graph: Given a suitable hierarchical node link graph G , containing node set V and edge set E Each node There exists a hierarchical allocation function. ,in K It is the total number of layers, ensuring all edges Connect nodes at different levels; Define constraints: Given a graph G The standard model is defined using position variables, cross variables, cross constraints, and transitive constraints. ,in, In position variables, use Represents nodes in the same layer i and j The position of the node i At the node j Above, Conversely ,and, ; In the cross variable, use Represents the edges in the graph and Whether they intersect, if the edges and edge Intersection, then ,otherwise ,and, ; In the cross constraint, the cross variable is defined according to formula (12) and formula (13). And ensure that if the side and Crossing , Formula (12) is used to ensure that when exist Above and exist Below, cross variables Formula (13) is used to ensure that when exist Below and exist When above, cross variables ; In transitivity constraints, transitivity constraints are directly applied to all node triples at the same level according to formulas (14) and (15). Among them, if and =1, then ;like and =0, then .

9. The method for visual comparison and iterative optimization of prediction models based on feature subsets according to claim 8, characterized in that, The establishment of integer linear programming ILP The model includes: Construct position variables, cross variables, cross constraints, and transitive constraints into ILP The model, in which the effective replication of position variables and cross variables corresponds to the layout of the graph, and the sum of all cross variables is the number of edge crosses in the graph; the replication of cross variables with the minimum sum represents the layout with the fewest crosses, and the model minimizes all cross variables according to the objective function formula (16). The sum of, in, Indicates the first Layer and first Edge sets between layers; The cross variable represents the edge. and edge Whether it crosses or not, if it crosses, it is 1, otherwise it is 0.

10. The method for visual comparison and iterative optimization of prediction models based on feature subsets according to claim 9, characterized in that, The method combines node hierarchy, relative position variables, edge intersection variables, and transitivity constraints to obtain a clear layout with minimal edge intersections, including: For a given input graph, encode it using cross constraints and transitivity constraints as Model, and pass it to Solver, The solver finds the assignment that minimizes the objective function, processes the assignment, and generates an optimized hierarchical node link graph; wherein, Select the first node pair in the same layer. and corresponding Assuming node Located at node Below this, a symmetry breaking method is introduced to accelerate the solver's efficiency; simultaneously, based on this, the method that appears most frequently in the cross constraints is selected. Set the variable to 0.

11. A visual comparison and iterative optimization system for prediction models based on feature subsets, characterized in that, The system includes: The data acquisition module is used to acquire multivariate time-series data on tobacco sales, including historical sales, inventory status, and market environment data. The model building module is used to construct fusion models based on feature subsets. XGBoost , LightGBM and Pathformer The algorithm's time-series prediction model; The similarity calculation module is used to calculate the multi-dimensional similarity between different iterative versions of the model; A layout optimization module is used to calculate the topological layout of the model evolution graph using the ILP model as described in any one of claims 7-10; It also includes a feedback update module, which responds to feature adjustment instructions and triggers model retraining until the optimal feature subset and model are obtained to output the predicted tobacco sales volume.

Images