Rapid data index query method based on AI model

Abstract

The invention relates to the technical field of data query, and discloses a data index quick query method based on an AI model, which comprises the steps of data collection, data preprocessing, feature selection, index construction and data query. According to the method, multi-data-source numerical data can be integrated to construct a standard data matrix, the number of samples is sorted, abnormal values are cleaned, and the data quality is improved; through the feature selection step, redundant features are removed, key features are focused, and subsequent analysis accuracy is guaranteed; in terms of query efficiency, index construction divides data blocks according to time and constructs characteristic value range indexes, candidate blocks can be quickly positioned, and the query range is narrowed; the hierarchical query mechanism firstly performs coarse screening and then performs accurate search, so that efficient and accurate query is ensured; the whole process suitable for numerical data can cope with the condition of multiple data sources, is suitable for multiple fields such as data analysis and intelligent decision making, and assists the efficient development of services.

Description

Technical Field

[0001] The present invention relates to the field of data query technology, and in particular to a method for quickly querying data indicators based on an AI model. Background Art

[0002] Traditional data query methods often face many challenges when faced with large-scale, complex data sets. First, traditional methods usually require a comprehensive scan of the entire data set, which can lead to extremely low query efficiency when the data volume is huge, and consume a lot of time and computing resources. For example, in the databases of some large enterprises, hundreds of millions of transaction records, customer information and other data are stored. When it is necessary to query data with specific conditions, such as finding customer information whose consumption amount is greater than a certain value within a specific time period and whose geographical location is in a specific area, traditional query methods may need to traverse the entire database, which makes the query response time may be as long as several hours or even days, seriously affecting the timeliness of business and the efficiency of decision-making.

[0003] Secondly, traditional data query lacks an effective data feature screening mechanism. Without in-depth analysis and screening of data features, querying directly in the original data set will introduce a large number of irrelevant data features, further increasing the query burden and reducing the query accuracy. For example, in a medical data set, when querying data related to a certain disease, data features such as the patient's name and height may have a low correlation with the disease, but traditional query methods cannot automatically distinguish and exclude these irrelevant features, thus affecting the pertinence and effectiveness of the query results.

[0004] In order to solve these problems, a fast query method for data indicators based on AI models came into being. Summary of the invention

[0005] The purpose of the present invention is to provide a method for quickly querying data indicators based on an AI model, which solves the technical problems raised in the background technology.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] The method for quickly querying data indicators based on the AI ​​model includes the following steps:

[0008] Step 1: Data collection:

[0009] Collect numerical target datasets from multiple data sources;

[0010] Step 2: Data preprocessing:

[0011] Construct the collected target data set into a data matrix;

[0012] Step 3: Feature selection:

[0013] First, the Pearson correlation coefficient between each data feature column of the data matrix is ​​calculated, and highly correlated feature pairs are formed based on the Pearson correlation coefficient. Then, among the highly correlated feature pairs, one is retained based on the data feature information entropy. Then, among the remaining features, the total entropy of the target data set is calculated, and then the conditional entropy of each data feature is calculated. Then, the information gain of the data feature is calculated based on the total entropy and conditional entropy, and the data features that constitute the data feature subset are determined based on the information gain.

[0014] Step 4: Index construction:

[0015] Divide the data feature subset into multiple data blocks according to the time sequence of the data, wherein each data block contains a number of data samples, and then construct an index based on the data feature value range for each data block;

[0016] Step 5: Data query:

[0017] Receive a search request from the user containing a numerical data query index condition, then traverse the data blocks in the index table according to the query index condition, determine the candidate block according to the comparison operator, then traverse the data samples in the candidate block, and output the data samples that meet all the search conditions as the search results.

[0018] As a further solution of the present invention: when constructing the data matrix, each row represents a data sample and each column represents a data feature;

[0019] And mark the data matrix corresponding to the target data set as X ij , whose dimension is m×n, that is, i=1, 2, ...m, j=1, 2, ...n, where m is the number of data samples in the data matrix and n is the number of data features in the data matrix.

[0020] As a further solution of the present invention: the data matrix is ​​constructed as follows:

[0021] StepY1. In the data matrix, traverse the number of data samples corresponding to each data feature column and mark it as m0 i ;

[0022] Then m0 i Compare with m, and organize the data matrix based on the comparison results:

[0023] If m0 i > m, it means that the number of data samples in the jth data feature column is redundant, and then the data sample with the largest absolute value in the jth data feature column is removed until m0 i =m;

[0024] If m0i <m, it means that the number of data samples in the j-th data feature column is missing, then the average value of all data samples in the j-th data feature column is calculated, marked as a supplementary value, and supplemented with m-m0 i Add the supplementary value to the jth data feature column so that m0 i =m;

[0025] If m0 i =m, it means that the number of data samples in the jth data feature column is qualified;

[0026] StepY2: Clean and replace outliers on the data sample values ​​corresponding to each data feature column in the data matrix.

[0027] As a further solution of the present invention: in Step Y2, the outlier cleaning and replacement method is as follows:

[0028] Let the value of j be 1;

[0029] Then through:

[0030] Calculate the average value XP of the data sample value of the jth data feature column j and standard deviation XB j ;

[0031] Then, each X ij Outlier cleaning is performed based on the corresponding mean and standard deviation;

[0032] When |X ij -XP j |≤3×XB j When X ij The value is normal, and then no outlier replacement is performed;

[0033] When |X ij -XP j |>3×XB j When X ij The value is abnormal, and then the abnormal value replacement is performed as follows:

[0034] If X ij >XP j -3×XB j When XP j -3×XB j The result is X ij 's replacement value;

[0035] If X ij >XP j +3×XB j When XPj +3×XB j The result is X ij 's replacement value;

[0036] By analogy, let the value of j be 2, 3...n, and then the outlier values ​​of each data feature column can be cleaned and replaced.

[0037] As a further solution of the present invention: the method of feature selection is as follows:

[0038] StepX1, calculate the Pearson correlation coefficient between each data feature column in the data matrix;

[0039] The method is as follows:

[0040] In the data matrix, two different data feature columns are marked as X i，j1 and X i，j2 ;

[0041] Among them, j1∈j, j2∈j, and j1≠j2;

[0042] Then pass:

[0043] Calculate the correlation coefficient R between two different data feature columns j1，j2 ;

[0044] In the formula, XP j1 and XP j2 X i，j1 and X i，j2 The average value of

[0045] StepX2: The correlation coefficient R between two different data feature columns j1，j2 Compare with the pre-set correlation threshold RY:

[0046] If | R j1，j2 |>RY, then determine X i，j1 and X i，j2 Highly correlated, and group them into highly correlated feature pairs;

[0047] If | R j1，j2 |≤RY, then determine X i，j1 and X i，j2 The correlation is low and they do not form feature pairs;

[0048] StepX3, among the highly correlated feature pairs, select one of the data features to retain based on the information entropy of the data feature;

[0049] StepX4. Among the remaining features, calculate the information gain of each feature, and then determine the data features that constitute the data feature subset based on the information gain of each data feature.

[0050] As a further solution of the present invention: wherein, the remaining features refer to the data features retained after the highly correlated feature pairs, and the data features that do not constitute the highly correlated feature pairs.

[0051] As a further solution of the present invention: in Step X3, the data features in the feature pair are retained as follows:

[0052] Based on the data matrix, in the j1th data feature corresponding to the highly correlated feature pair, the number of different values ​​of the data sample is obtained and marked as k;

[0053] Then obtain the probability of the same data sample value appearing in the j1th data feature and mark it as PX t , t = 1, 2, ... k;

[0054] Then pass:

[0055] Calculate the information entropy H of the j1th data feature j1 ;

[0056] Similarly, the information entropy H of the j2th data feature corresponding to the highly correlated feature pair is calculated j2 ;

[0057] Then the information entropy H corresponding to the highly correlated feature pairs j1 and j2 is j1 and H j2 For comparison:

[0058] If H j1 >H j2 , then retain the j1th data feature;

[0059] If H j1 <H j2 , then retain the j2th data feature.

[0060] As a further solution of the present invention: in Step X4, the data feature subset is composed as follows:

[0061] Step X4.1, divide the target data set into multiple feature categories, and then classify each data feature into each corresponding feature category;

[0062] Then, among the multiple data features in the feature category, the number of values ​​of data samples corresponding to all data features in the feature category is obtained and marked as Yr, where r = 1, 2, ... e, and e represents the number of feature categories;

[0063] Then through:

[0064] Calculate the proportion of all data features in each feature category to the sum of all data features in the feature category in the target data set. r ;

[0065] StepX4.2, through:

[0066] Calculate the total entropy HY of the target data set;

[0067] StepX4.3. Among the multiple data features corresponding to the target data set, obtain the number of data samples corresponding to each data feature and mark it as B j ;

[0068] Then through:

[0069] Calculate the proportion of data samples corresponding to each data feature in the sum of the number of data samples corresponding to all data features in the target data set. j ;

[0070] Similarly, among the multiple data features corresponding to each feature category, obtain the proportion of the sum of the number of data samples corresponding to all data features in the feature category corresponding to each data feature, and mark it as PB j，r ;

[0071] StepX4.4, through:

[0072] Calculate the conditional entropy HU after each data feature divides the target data set into feature categories j ;

[0073] Then passed: HG j =HY-HU j

[0074] Calculate the information gain HG of each data feature j ;

[0075] StepX4.5, then the information gain HG of each data feature j Compare with the preset information gain threshold HGy respectively:

[0076] Then HG j >The data features corresponding to HGy constitute the data feature subset.

[0077] As a further solution of the present invention: the index is constructed as follows:

[0078] Count the minimum and maximum values ​​of each data feature in the statistical feature subset in the corresponding data block and mark it as X min，s and X max，s , where s = 1, 2, ... v, v is the number of data blocks;

[0079] Then an index table is constructed, where each row corresponds to a data block, each column corresponds to a data feature, and the elements in the index table are (X min，s , X max，s ).

[0080] As a further solution of the present invention: the data query method is as follows:

[0081] Step C1, receiving a search request input by a user, the search request includes an index condition for a numerical data query, wherein the index condition is M=(J, CP, Z), and J is a data feature of the query, CP is a comparison operator of the query, the comparison operator includes ">", "<", and Z is a data comparison value of the query;

[0082] Step C2, query the index conditions and index table, and filter out the data blocks that meet the index conditions; the method is as follows:

[0083] Traverse each row in the index table corresponding to the data block;

[0084] Make corresponding judgments based on the comparison operator CP:

[0085] If (X min，s , CP, Z) and (Z, CP, X max，s ), then mark the corresponding data block as a candidate block;

[0086] Step C3: For the selected candidate blocks, perform precise query in each data block in the following way:

[0087] Traverse each data sample in the candidate block and compare the data features and values ​​of the data sample one by one according to the index condition M=(J, CP, Z):

[0088] If the data sample meets all the search conditions, the data sample is output as the search result.

[0089] As a further solution of the present invention: the candidate blocks are marked as follows:

[0090] If CP is “>”, then judge X min，s >Whether Z is true, if it is true, the corresponding data block is marked as a candidate block, if it is not true, the corresponding data block is not marked as a candidate block;

[0091] If CP is "<", then Z<X max，s Whether it is true, if it is true, the corresponding data block is marked as a candidate block, and if it is not true, the corresponding data block is not marked as a candidate block.

[0092] Beneficial effects of the present invention:

[0093] Data integration and normalization: By collecting numerical target data sets from multiple data sources and constructing them into standardized data matrices, the number of data samples in different data feature columns is sorted and outliers are cleaned and replaced, making the data more regular and accurate before entering the subsequent analysis process, reducing the interference of outliers and data inconsistencies on subsequent queries and analysis results, and improving the overall data quality.

[0094] Feature selection optimization: The Pearson correlation coefficient is calculated to determine highly correlated feature pairs. One of the highly correlated feature pairs is retained based on information entropy, and then the data features that make up the data feature subset are determined from the remaining features based on information gain. This multi-step feature selection method can effectively remove redundant features and screen out data features that are more representative and critical to the target data set. While reducing the data dimension, it focuses on the most valuable information, which helps to improve the efficiency of subsequent queries and the accuracy of analysis results.

[0095] Index construction facilitates rapid positioning: Data feature subsets are divided into multiple data blocks according to the time sequence of the data, and an index based on the data feature value range is constructed for each data block. When facing a user's data query request, the candidate blocks that may contain the target data can be quickly screened out based on the index table, avoiding indiscriminate traversal of all data, greatly narrowing the query scope, and significantly improving the speed of data query.

[0096] The hierarchical query mechanism improves accuracy: first, the candidate blocks are screened out through the index, and then the data samples are traversed in the candidate blocks for precise query, and the data samples that meet all the search conditions are output as the search results. This hierarchical query mechanism not only uses the index to quickly locate the approximate range, but also ensures the accuracy of the results through precise query, making the entire query process efficient and accurate, and can better meet the user's needs for fast and accurate query of numerical data indicators.

[0097] Overall applicability: Through a complete set of process methods from data collection to query applicable to numerical data, it can cope with data from multiple data sources and has a wide range of applicability. It can be applied to a variety of scenarios that require indicator queries on large amounts of numerical data, such as data analysis, intelligent decision-making, etc., to assist related businesses to be carried out more efficiently. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] The present invention will be further described below in conjunction with the accompanying drawings.

[0099] Figure 1 It is a flow chart of the method for rapid query of data indicators based on the AI ​​model of the present invention.

[0100] Figure 2 It is a flow chart of the feature selection step in the data indicator rapid query method based on the AI ​​model of the present invention. DETAILED DESCRIPTION

[0101] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0102] Embodiment 1

[0103] See also Figure 1 and Figure 2 As shown, the present invention is a data indicator quick query method based on AI model, comprising the following steps:

[0104] Step 1: Data collection:

[0105] Collect numerical target datasets from multiple data sources;

[0106] In this embodiment, the data source includes a database, a file system, etc.; the collected target data set covers various indicator data related to the field to be analyzed;

[0107] For example, in the financial field, it may include stock prices, trading volume, financial indicators, etc.; in the industrial production field, it may include temperature, pressure, output and other data;

[0108] Step 2: Data preprocessing:

[0109] The collected target data set is constructed into a data matrix according to a preset format. When constructing the data matrix, each row represents a data sample and each column represents a data feature.

[0110] And mark the data matrix corresponding to the target data set as X ij , whose dimension is m×n, i=1, 2, ...m, j=1, 2, ...n, where m is the number of data samples in the data matrix, and n is the number of data features in the data matrix;

[0111] The data matrix is ​​constructed as follows:

[0112] In the data matrix, traverse the number of data samples corresponding to each data feature column and mark it as m0 i ;

[0113] Then m0 i Compare with m, and organize the data matrix based on the comparison results:

[0114] If m0 i > m, it means that the number of data samples in the jth data feature column is redundant, and then the data sample with the largest absolute value in the jth data feature column is removed until m0 i =m;

[0115] If m0 i <m, it means that the number of data samples in the j-th data feature column is missing, then the average value of all data samples in the j-th data feature column is calculated, marked as a supplementary value, and supplemented with m-m0 i Add the supplementary value to the jth data feature column so that m0 i =m;

[0116] If m0 i =m, it means that the number of data samples in the jth data feature column is qualified;

[0117] Step 3: Feature selection:

[0118] StepX1, calculate the Pearson correlation coefficient:

[0119] Calculate the Pearson correlation coefficient between each data feature column in the data matrix;

[0120] The method is as follows:

[0121] In the data matrix, two different data feature columns are marked as X i，j1 and X i，j2 ;

[0122] Among them, j1∈j, j2∈j, and j1≠j2;

[0123] Then pass:

[0124] Calculate the correlation coefficient R between two different data feature columns j1，j2 ;

[0125] In the formula, XP j1 and XP j2 X i，j1 and X i，j2 The average value of

[0126] StepX2, comparison of correlation coefficient and threshold:

[0127] The correlation coefficient R between two different data feature columns j1，j2 Compare with the pre-set correlation threshold RY:

[0128] If | R j1，j2 |>RY, then determine X i，j1 and X i，j2 Highly correlated, and group them into highly correlated feature pairs;

[0129] If | R j1，j2 |≤RY, then determine X i，j1 and X i，j2 The correlation is low and they do not form feature pairs;

[0130] StepX3, select features in feature pairs based on information entropy:

[0131] Among highly correlated feature pairs, one of the data features is selected to be retained based on the information entropy of the data features;

[0132] The method is as follows:

[0133] Based on the data matrix, in the j1th data feature corresponding to the highly correlated feature pair, the number of different values ​​of the data sample is obtained and marked as k;

[0134] Then obtain the probability of the same data sample value appearing in the j1th data feature and mark it as PX t , t = 1, 2, ... k;

[0135] Then pass:

[0136] Calculate the information entropy H of the j1th data feature j1 ;

[0137] Similarly, the information entropy H of the j2th data feature corresponding to the highly correlated feature pair is calculated j2 ;

[0138] Then the information entropy H corresponding to the highly correlated feature pairs j1 and j2 is j1 and H j2 For comparison:

[0139] If H j1 >H j2 , then retain the j1th data feature;

[0140] If H j1 <H j2 , then retain the j2th data feature;

[0141] StepX4, calculate the remaining feature information gain and determine the feature subset:

[0142] Among the remaining features, the information gain of each feature is calculated, and then the data features constituting the data feature subset are determined according to the information gain of each data feature;

[0143] Among them, the remaining features refer to the data features retained after the highly correlated feature pairs, and the data features that do not form highly correlated feature pairs;

[0144] The method is as follows:

[0145] Step X4.1, divide the target data set into multiple feature categories, and then classify each data feature into each corresponding feature category;

[0146] Then, among the multiple data features in the feature category, the number of values ​​of data samples corresponding to all data features in the feature category is obtained and marked as Yr, where r = 1, 2, ... e, and e represents the number of feature categories;

[0147] Then through:

[0148] Calculate the proportion of all data features in each feature category to the sum of all data features in the feature category in the target data set. r ;

[0149] StepX4.2, through:

[0150] Calculate the total entropy HY of the target data set;

[0151] StepX4.3. Among the multiple data features corresponding to the target data set, obtain the number of data samples corresponding to each data feature and mark it as B j ;

[0152] Then through:

[0153] Calculate the proportion of data samples corresponding to each data feature in the sum of the number of data samples corresponding to all data features in the target data set. j ;

[0154] Similarly, among the multiple data features corresponding to each feature category, obtain the proportion of the sum of the number of data samples corresponding to all data features in the feature category corresponding to each data feature, and mark it as PB j，r ;

[0155] StepX4.4, through:

[0156] Calculate the conditional entropy HU after each data feature divides the target data set into feature categories j ;

[0157] Then passed: HG j =HY-HU j

[0158] Calculate the information gain HG of each data feature j ;

[0159] StepX4.5, then the information gain HG of each data feature j Compare with the preset information gain threshold HGy respectively:

[0160] Then HG j >The data features corresponding to HGy constitute the data feature subset;

[0161] Step 4: Index construction:

[0162] Divide the data feature subset into v data blocks according to the time sequence of the data, where each data block contains a number of data samples, and then construct an index based on the data feature value range for each data block;

[0163] The method is as follows:

[0164] Count the minimum and maximum values ​​of each data feature in the statistical feature subset in the corresponding data block and mark it as X min，s and X max，s , where s = 1, 2, ..., v;

[0165] Then an index table is constructed, where each row corresponds to a data block, each column corresponds to a data feature, and the elements in the index table are (X min，s , X max，s );

[0166] Step 5: Data query:

[0167] Step C1, receiving a search request input by a user, the search request includes an index condition for a numerical data query, wherein the index condition is M=(J, CP, Z), and J is a data feature of the query, CP is a comparison operator of the query, the comparison operator includes ">", "<", and Z is a data comparison value of the query;

[0168] Step C2, query the index conditions and index table, and filter out the data blocks that meet the index conditions; the method is as follows:

[0169] Traverse each row in the index table corresponding to the data block;

[0170] Make corresponding judgments based on the comparison operator CP:

[0171] If (X min，s , CP, Z) and (Z, CP, X max，s ), then mark the corresponding data block as a candidate block;

[0172] In this embodiment, for example:

[0173] If CP is “>”, then judge X min，s >Whether Z holds true, if it does, the corresponding data block is marked as a candidate block;

[0174] If CP is "<", then Z<X max，s Is it true? If it is true, the corresponding data block is marked as a candidate block;

[0175] Step C3: For the selected candidate blocks, perform precise query in each data block in the following way:

[0176] Traverse each data sample in the candidate block and compare the data features and values ​​of the data sample one by one according to the index condition M=(J, CP, Z):

[0177] If the data sample meets all the search conditions, the data sample is output as the search result.

[0178] This embodiment collects numerical target data sets from multiple data sources and constructs a data matrix, organizes the number of data samples, standardizes and unifies the data structure, facilitates subsequent processing and analysis, and lays the foundation for efficient query; screens highly correlated feature pairs through the Pearson correlation coefficient, retains one of them based on information entropy, and then uses information gain to determine the data feature subset, effectively reducing data dimensions, highlighting key features, improving query accuracy and efficiency, and reducing the interference of redundant data on query results; divides data blocks based on time sequence and constructs a feature value range index, combined with a hierarchical query mechanism, first quickly locates candidate blocks based on the index, and then accurately queries within the candidate blocks, which greatly improves the data query speed, can quickly respond to numerical data indicator query needs, and is suitable for multi-field data query scenarios.

[0179] Embodiment 2

[0180] As the second embodiment of the present invention, when the present application is specifically implemented, compared with the first embodiment, the difference between the technical solution of the present embodiment and the first embodiment is that in the present embodiment, in the data preprocessing step, the data sample values ​​corresponding to each data feature column in the data matrix are also cleaned and replaced with abnormal values, and the specific method is as follows:

[0181] Let the value of j be 1;

[0182] Then through:

[0183] Calculate the average value XP of the data sample value of the jth data feature column j and standard deviation XB j ;

[0184] Then, each X ij Outlier cleaning is performed based on the corresponding mean and standard deviation;

[0185] When |X ij -XP j |≤3×XB j When X ij The value is normal, and then no outlier replacement is performed;

[0186] When |X ij -XP j |>3×XB j When X ij The value is abnormal, and then the abnormal value replacement is performed as follows:

[0187] If X ij >XP j -3×XB j When XP j -3×XB j The result is X ij 's replacement value;

[0188] If X ij >XP j +3×XB j When XP j +3×XB j The result is X ij 's replacement value;

[0189] By analogy, let the value of j be 2, 3...n, and then the outlier values ​​of each data feature column can be cleaned and replaced.

[0190] This embodiment adds an outlier cleaning and replacement step based on the first embodiment. This helps to further improve the data quality, remove abnormal fluctuations or erroneous data points in the data, and make the data more reflective of the real situation. In the subsequent feature selection, index construction and data query process, the adverse effects of abnormal data can be reduced, thereby improving the reliability and stability of the overall query results, which is of great significance in application scenarios with high requirements for data accuracy.

[0191] Embodiment 3

[0192] As the third embodiment of the present invention, when the present application is specifically implemented, compared with the first and second embodiments, the technical solution of this embodiment is to combine the solutions of the first and second embodiments mentioned above for implementation.

[0193] This embodiment integrates the advantages of embodiment one and embodiment two. It has the efficiency and accuracy of data organization and integration, feature optimization, indexing and query optimization, and further purifies the data through outlier cleaning. In summary, it can provide high-quality, high-precision and high-efficiency query services in the query tasks of numerical data indicators from multiple data sources, and can be widely used in data analysis and decision support work in many fields such as finance and industrial production, giving full play to the value of data and helping the refined operation and accurate decision-making of related businesses.

[0194] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters and thresholds in the formula are set by technicians in this field according to actual conditions.

[0195] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data index quick query method based on AI model, characterized in that: The following steps are involved: Step 1: Data collection: Collect numerical target datasets from multiple data sources; Step 2: Data preprocessing: Construct the collected target data set into a data matrix; Step 3: Feature selection: First, the Pearson correlation coefficient between each data feature column of the data matrix is ​​calculated, and highly correlated feature pairs are formed based on the Pearson correlation coefficient. Then, among the highly correlated feature pairs, one is retained based on the data feature information entropy. Then, among the remaining features, the total entropy of the target data set is calculated, and then the conditional entropy of each data feature is calculated. Then, the information gain of the data feature is calculated based on the total entropy and conditional entropy, and the data features that constitute the data feature subset are determined based on the information gain. Step 4: Index construction: Divide the data feature subset into multiple data blocks according to the time sequence of the data, wherein each data block contains a number of data samples, and then construct an index based on the data feature value range for each data block; Step 5: Data query: Receive a search request from the user containing a numerical data query index condition, then traverse the data blocks in the index table according to the query index condition, determine the candidate block according to the comparison operator, then traverse the data samples in the candidate block, and output the data samples that meet all the search conditions as the search results.

2. The method for rapid query of data indicators based on AI model according to claim 1 is characterized in that: When constructing a data matrix, each row represents a data sample and each column represents a data feature; And mark the data matrix corresponding to the target data set as X ij , whose dimension is m×n, that is, i=1, 2, ...m, j=1, 2, ...n, where m is the number of data samples in the data matrix and n is the number of data features in the data matrix.

3. The method for rapid query of data indicators based on AI model according to claim 2 is characterized in that: The data matrix is ​​constructed as follows: StepY1. In the data matrix, traverse the number of data samples corresponding to each data feature column and mark it as m0 i ; Then m0 i Compare with m, and organize the data matrix based on the comparison results: If m0 i > m, it means that the number of data samples in the jth data feature column is redundant, and then the data sample with the largest absolute value in the jth data feature column is removed until m0 i =m; If m0 i <m, it means that the number of data samples in the j-th data feature column is missing, then the average value of all data samples in the j-th data feature column is calculated, marked as a supplementary value, and supplemented with m-m0 i Add the supplementary value to the jth data feature column so that m0 i =m; If m0 i =m, it means that the number of data samples in the jth data feature column is qualified; StepY2: Clean and replace outliers on the data sample values ​​corresponding to each data feature column in the data matrix.

4. The method for rapid query of data indicators based on AI model according to claim 3 is characterized in that: In StepY2, the outlier cleaning and replacement method is as follows: Let the value of j be 1; Then through: Calculate the average value XP of the data sample value of the jth data feature column j and standard deviation XB j ; Then, each X ij Outlier cleaning is performed based on the corresponding mean and standard deviation; When |X ij -XP j |≤3×XB j When X ij The value is normal, and then no outlier replacement is performed; When |X ij -XP j |>3×XB j When X ij The value is abnormal, and then the abnormal value replacement is performed as follows: If X ij >XP j -3×XB j When XP j -3×XB j The result is X ij 's replacement value; If X ij >XP j +3×XB j When XP j +3×XB j The result is X ij 's replacement value; By analogy, let the value of j be 2, 3...n, and then the outlier values ​​of each data feature column can be cleaned and replaced.

5. The method for rapid query of data indicators based on AI model according to claim 2 is characterized in that: The feature selection method is as follows: StepX1, calculate the Pearson correlation coefficient between each data feature column in the data matrix; The method is as follows: In the data matrix, two different data feature columns are marked as X i，j1 and X i，j2 ; Among them, j1∈j, j2∈j, and j1≠j2; Then pass: Calculate the correlation coefficient R between two different data feature columns j1，j2 ; In the formula, XP j1 and XP j2 X i，j1 and X i，j2 The average value of StepX2: The correlation coefficient R between two different data feature columns j1，j2 Compare with the pre-set correlation threshold RY: If | R j1，j2 |>RY, then determine X i，j1 and X i，j2 Highly correlated, and group them into highly correlated feature pairs; If | R j1，j2 |≤RY, then determine X i，j1 and X i，j2 The correlation is low and they do not form feature pairs; StepX3, among the highly correlated feature pairs, select one of the data features to retain based on the information entropy of the data feature; StepX4. Among the remaining features, calculate the information gain of each feature, and then determine the data features that constitute the data feature subset based on the information gain of each data feature.

6. The method for rapid query of data indicators based on AI model according to claim 5 is characterized in that: in, The remaining features refer to the data features retained after the highly correlated feature pairs and the data features that do not form highly correlated feature pairs.

7. The method for rapid query of data indicators based on AI model according to claim 5 is characterized in that: In StepX3, the data features in the feature pair are preserved as follows: Based on the data matrix, in the j1th data feature corresponding to the highly correlated feature pair, the number of different values ​​of the data sample is obtained and marked as k; Then obtain the probability of the same data sample value appearing in the j1th data feature and mark it as PX t , t = 1, 2, ... k; Then pass: Calculate the information entropy H of the j1th data feature j1 ; Similarly, the information entropy H of the j2th data feature corresponding to the highly correlated feature pair is calculated j2 ; Then the information entropy H corresponding to the highly correlated feature pairs j1 and j2 is j1 and H j2 For comparison: If H j1 >H j2 , then retain the j1th data feature; If H j1 <H j2 , then retain the j2th data feature.

8. The method for rapid query of data indicators based on AI model according to claim 5 is characterized in that: In StepX4, the data feature subsets are composed as follows: Step X4.1, divide the target data set into multiple feature categories, and then classify each data feature into each corresponding feature category; Then, among the multiple data features in the feature category, the number of values ​​of data samples corresponding to all data features in the feature category is obtained and marked as Yr, where r = 1, 2, ... e, and e represents the number of feature categories; Then through: Calculate the proportion of all data features in each feature category to the sum of all data features in the feature category in the target data set. r ; StepX4.2, through: Calculate the total entropy HY of the target data set; StepX4.

3. Among the multiple data features corresponding to the target data set, obtain the number of data samples corresponding to each data feature and mark it as B j ; Then through: Calculate the proportion of data samples corresponding to each data feature in the sum of the number of data samples corresponding to all data features in the target data set. j ; Similarly, among the multiple data features corresponding to each feature category, obtain the proportion of the sum of the number of data samples corresponding to all data features in the feature category corresponding to each data feature, and mark it as PB j，r ; StepX4.4, through: Calculate the conditional entropy HU after each data feature divides the target data set into feature categories j ; Then passed: HG j =HY-HU j Calculate the information gain HG of each data feature j ; StepX4.5, then the information gain HG of each data feature j Compare with the preset information gain threshold HGy respectively: Then HG j >The data features corresponding to HGy constitute the data feature subset.

9. The method for rapid query of data indicators based on AI model according to claim 1 is characterized in that: The index is constructed as follows: Count the minimum and maximum values ​​of each data feature in the statistical feature subset in the corresponding data block and mark it as X min，s and X max，s , where s = 1, 2, ... v, v is the number of data blocks; Then an index table is constructed, where each row corresponds to a data block, each column corresponds to a data feature, and the elements in the index table are (X min，s , X max，s ).

10. The method for rapid query of data indicators based on AI model according to claim 9 is characterized in that: The data query method is as follows: Step C1, receiving a search request input by a user, the search request includes an index condition for a numerical data query, wherein the index condition is M=(J, CP, Z), and J is a data feature of the query, CP is a comparison operator of the query, the comparison operator includes ">" and "<", and Z is a data comparison value of the query; Step C2, query the index conditions and index table, and filter out the data blocks that meet the index conditions; the method is as follows: Traverse each row in the index table corresponding to the data block; Make corresponding judgments based on the comparison operator CP: If (X min，s , CP, Z) and (Z, CP, X max，s ), the corresponding data block is marked as a candidate block, and the marking method of the candidate block is as follows: If CP is ">", then judge X min，s >Whether Z is true, if it is true, the corresponding data block is marked as a candidate block, if it is not true, the corresponding data block is not marked as a candidate block; If CP is "<", then Z<X max，s Whether it is true, if it is true, the corresponding data block is marked as a candidate block, if it is not true, the corresponding data block is not marked as a candidate block; Step C3: For the selected candidate blocks, perform precise query in each data block in the following way: Traverse each data sample in the candidate block and compare the data features and values ​​of the data sample one by one according to the index condition M=(J, CP, Z): If the data sample meets all the search conditions, the data sample is output as the search result.

Images