A data search, classification method and device based on eigenvalue index

Abstract

The application discloses a data search and classification method based on eigenvalue index, comprising the following steps: obtaining a set based on original data through a regular semantic hybrid parser, wherein the set comprises feature names and feature values; obtaining a classification vector based on the set through mapping; obtaining classified index data based on the classification vector through two-level index search of multidimensional reverse arrangement and bitmap; obtaining a query data subset based on user query conditions through query rewriting, feature sorting and bitmap intersection; obtaining an updated query data subset based on the query data subset through sequential scanning and sorting; and obtaining updated classified index data based on the query data subset and new data through a bitmap merging strategy.

Description

Technical Field

[0001] This invention belongs to the field of cloud computing technology, and in particular relates to a data search and classification method and apparatus based on feature value indexing. Background Technology

[0002] In data-intensive application scenarios, as business scale continues to expand, the growth pattern of data assets (such as charts and models) has shifted from linear, slow growth to an exponential surge, with the volume often rapidly exceeding several thousand and climbing to even higher scales. These data assets include not only charts and model data. Against this backdrop, the limitations of traditional linear search models are dramatically amplified. On the one hand, this model requires traversing and comparing massive amounts of exponentially growing data, significantly increasing time complexity with the surge in data volume. On the other hand, when users need to filter data based on multiple feature combinations, the core logic of traversing and comparing further prolongs the retrieval time. This not only makes it difficult to meet users' needs for quickly locating data at data scales of several thousand, but also consumes a large amount of system resources, ultimately resulting in a significant decline in overall data processing capabilities. Summary of the Invention

[0003] To address the aforementioned issues, this invention proposes a data search and classification method and apparatus based on feature value indexing. Through regularized semantic hybrid parsing and collaboration with a teacher-student model, it achieves accurate feature extraction and domain label prediction from raw data. Dynamic threshold calculation and marginal suppression conflict resolution enable adaptive optimization and confidence enhancement of the candidate label set. A two-tier architecture combining a compressed bitmap inverted index and a dynamically pre-computed decision unit achieves efficient classification and rapid retrieval of multi-dimensional features. Query rewriting, feature sorting, and bitmap intersection operations enable precise matching of user query conditions and rapid result set filtering. Hierarchical bitmap merging and incremental cache invalidation reconstruction achieve seamless integration of new data and dynamic updates to the index structure.

[0004] The first aspect of the present invention provides a data search and classification method based on feature value indexing, comprising: A set is obtained from the original data using a regular semantic hybrid parser, and the set includes feature names and feature values. Based on the set, a classification vector is obtained through mapping; Based on the classification vector, classification index data is obtained through multi-dimensional reverse sorting and two-level index search of bitmap; Based on user query conditions, a subset of query data is obtained through query rewriting, feature sorting, and bitmap intersection. The updated subset of query data is obtained by sequential scanning and sorting based on the subset of query data. The updated categorization index data is obtained based on the queried data subset and the newly added data using a bitmap merging strategy.

[0005] Preferably, the step of obtaining the set based on the original data using a regular expression semantic hybrid parser further includes: The student model is trained based on the original data and the teacher model, and the domain labels and label prediction probabilities corresponding to the original data are obtained. Based on the domain labels and predicted label probabilities, a candidate label set is obtained through dynamic threshold calculation. The calculation process is as follows: A dynamic threshold is obtained based on the original data, and the calculation expression is: In the formula The following parameters are used: a baseline threshold, the number of samples with feature name f in the original data, and a scaling factor. The label set is obtained based on the dynamic threshold, and the calculation expression is: In the formula Predict the probability for the label with feature name f. The data with index i in the tag set; Based on the candidate label set, an optimized label set is obtained through marginal suppression conflict resolution calculation; The set is obtained by calculating confidence based on the optimized label set.

[0006] Preferably, the step of obtaining the optimized label set based on the candidate label set through marginal suppression conflict resolution calculation further includes: For each feature name, the marginal confidence gap is obtained through confidence level and marginal suppression. The calculation expression is as follows: In the formula, These are the maximum probability value corresponding to the feature name, the second maximum probability value corresponding to the feature name, and a preset probability threshold, respectively. The optimized label set is obtained based on the marginal confidence gap. Specifically, if the marginal confidence gap is less than the preset probability threshold, the label data corresponding to the feature name is discarded; otherwise, the optimized label set is obtained. The calculation expression is: In the formula These are, respectively, the optimized label set, the label data in the candidate label set, the feature name, and the predicted probability of the label data.

[0007] Preferably, the step of obtaining the set based on the tuning label set through confidence calculation further includes: obtaining the set based on the tuning label set, and the calculation expression is: In the formula, c is the confidence value, f is the feature name, and v is the feature value corresponding to the feature name.

[0008] Preferably, the step of obtaining classification index data based on the classification vector through a two-level index search using multi-dimensional reverse order and bitmap further includes: Based on the classification vector, a first-level index is obtained through a compressed bitmap inverted index builder, and an updated first-level index is obtained through a sorted index optimizer. Based on historical query logs and the first-level index, a pre-computation task queue is obtained through a dynamic pre-computation decision-maker, and a second-level index is obtained based on the pre-computation task queue and the first-level index; The final categorized index data is obtained through an index aggregator based on the primary index and the secondary index.

[0009] Preferably, the step of obtaining the first-level index based on the classification vector using a compressed bitmap inverted index constructor further includes: Initialize a global inverted index dictionary, where the keys of the index dictionary are feature classification data and the values ​​are bitmap data corresponding to the feature classification data; Obtain each data item of the classification vector and update the index of the data item to the bitmap data; The first-level index is obtained by sorting the index dictionary using the cardinality of the bitmap data.

[0010] Preferably, the step of obtaining a subset of query data based on user query conditions through query rewriting, feature sorting, and bitmap intersection further includes: Based on the user query conditions, a standardized query vector is obtained by identifying feature names or feature values, expanding domain synonyms, and performing normalization processing of the value range; Based on the standardized query vector and the first-level index, query tasks are obtained by sorting according to feature name or feature value. Based on the query task, the first-level index, and the second-level index, a preliminary result bitmap is obtained through the bitmap intersection operator; Based on the preliminary result bitmap and historical stored data, a subset of query data is obtained through assembly and sorting.

[0011] Preferably, the rules for obtaining the preliminary result bitmap based on the query task, the first-level index, and the second-level index through the bitmap intersection operator further include: Based on the query task, the secondary index is queried to obtain the cross bitmap, and the value of the preliminary result bitmap is set as the cross bitmap; if the cross bitmap is empty, the primary expected bitmap corresponding to each feature in the query task execution task is sequentially obtained from the primary index, and the preliminary result bitmap is generated based on the primary expected bitmap through bitmap intersection operation.

[0012] Preferably, the step of obtaining the updated categorization index data based on the queried data subset and the newly added data through a bitmap merging strategy further includes: Based on the newly added data, a set of bitmaps for the newly added data is obtained through continuous value density binning, discrete value normalization mapping, and bitmap allocation. Based on the newly added data bitmap set and the query data subset, a temporary merged index is obtained through hierarchical or merging process; The updated secondary index is obtained through incremental cache invalidation and rebuilding based on the temporary merged index and the secondary index; The categorized index data is obtained through the index metadata updater based on the temporary merged index and the secondary index.

[0013] A second aspect of the present invention provides a data search and classification apparatus based on feature value indexing, comprising: Data feature extraction module: used to obtain a set based on the original data through a regular semantic hybrid parser, the set including feature name and feature value; The classification module is used to obtain classification vectors based on the set through mapping and to obtain classification index data based on the classification vectors through a two-level index search using multi-dimensional reverse order and cross-bucket bitmap. The query processing module is used to obtain a subset of query data based on user query conditions through query rewriting, feature sorting, and bitmap intersection; to obtain an updated subset of query data based on the subset of query data through sequential scanning and sorting; and to obtain updated categorization index data based on the subset of query data and newly added data through a bitmap merging strategy.

[0014] This invention, by employing the above technical solutions, possesses the following advantages and positive effects compared to existing technologies: Through regular semantic hybrid parsing and teacher-student model collaboration, it achieves accurate feature extraction and domain label prediction from raw data; through dynamic threshold calculation and marginal suppression conflict resolution, it achieves adaptive optimization and confidence enhancement of the candidate label set; through a two-level architecture of compressed bitmap inverted index and dynamic pre-computation decision unit, it achieves efficient classification and rapid retrieval of multi-dimensional features; through query rewriting, feature sorting, and bitmap intersection operations, it achieves accurate matching of user query conditions and rapid filtering of result sets; and through hierarchical bitmap merging and incremental buffering… The system reconstructs and rebuilds from failures, enabling seamless integration of new data and dynamic updates to the index structure. Through continuous value density binning and discrete value normalization mapping, it standardizes the features of new data and efficiently allocates bitmaps. By merging primary and secondary indexes through an index aggregator, it achieves performance balance and acceleration for general queries and hot queries. Through a pre-computation task queue driven by historical query logs, it achieves proactive caching and response acceleration for high-frequency query patterns. Through a post-processing mechanism of sequential scanning and secondary sorting, it optimizes the order of query result sets and adapts the final display. By synchronizing index statistics through a metadata updater, it achieves real-time evolution of the index structure and query optimization. Attached Figure Description

[0015] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a flowchart illustrating the data search and classification method based on feature value indexing in this invention. Figure 2 This is an embodiment of the data search and classification method based on feature value indexing in this invention. Detailed Implementation

[0016] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The advantages and features of the present invention will become clearer from the following description and claims. It should be noted that the drawings are all in a very simplified form and use non-precise ratios, and are only used to facilitate and clarify the illustration of the embodiments of the present invention.

[0017] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0018] First Embodiment See Figure 1 and Figure 2 The first aspect of the present invention provides a data search and classification method based on feature value indexing, comprising: A set is obtained from the original data using a regular semantic hybrid parser. The set includes feature names and feature values. Classification vectors are obtained through mapping based on sets; Based on classification vectors, classification index data is obtained through multi-dimensional reverse sorting and two-level index search of bitmaps; Based on user query conditions, a subset of query data is obtained through query rewriting, feature sorting, and bitmap intersection. The updated subset of query data is obtained by sequential scanning and sorting based on the subset of query data. Updated categorization index data is obtained using a bitmap merging strategy based on a subset of queried data and newly added data.

[0019] By employing a regular semantic hybrid parser to perform deep semantic decomposition of the original data, millisecond-level accurate extraction of feature names and values ​​is achieved. Through an intelligent mapping mechanism from feature sets to classification vectors, a compact and structured representation of high-dimensional sparse data is realized. Through the collaborative design of multi-dimensional reverse order arrangement and two-level bitmap index, real-time classification and retrieval of massive data is achieved. Through query condition semantic reconstruction and dynamic feature sorting strategies, accurate matching and priority optimization of query conditions are achieved. Through a hybrid execution engine of bitmap intersection operation and sequential scanning, the query result set is pruned and sorted in real time. Through dynamic weight allocation of classification vectors, adaptive index structure adjustment driven by feature importance is achieved.

[0020] Taking the extraction of feature information from a storage box as an example, suppose the current storage box information is: 24 cm long, 6 cm wide, 12 cm high, corrugated cardboard, bright yellow exterior, no pattern; the keys and values ​​obtained through feature extraction are: length: 24 cm, width: 6 cm, height: 12 cm, material: corrugated cardboard, color: bright yellow, style: no pattern. For sorting the feature values ​​of the storage box, the above process can be used to progressively add the storage box information into the initialized set variable according to the level of feature values. Further examples illustrate this process. It is important to emphasize that the process will change depending on the needs. Based on the feature name and feature value of interest, the program matches the corresponding storage box information according to the feature name list. For example, if the requirement is to find a storage box with a width of 6 cm made of corrugated cardboard, then the program first sorts by feature value, see [reference needed]. Figure 2As shown, our search features are material and width, and we extract the corresponding feature values: corrugated cardboard and 6cm, respectively. During the search process, we first find a list of matching storage boxes based on the first feature name and feature value. Then, we filter this list to find storage boxes matching the second feature name, continuing this process until all feature names are extracted. All corrugated cardboard storage boxes matching our 6cm width are then selected. Data preprocessing and algorithm improvements to extract feature values ​​from all feature names accelerate search results while avoiding a global search on each query. This allows for rapid classification and retrieval of items with highly overlapping semantics or feature names. Use cases include, for example, the rapid retrieval of simulation values ​​containing multiple scan items. For example, in simulating the current output of a three-terminal device, voltages of [1.0, 1.2, 1.4, 1.6, 1.8, 2.0], [2.0, 2.2, 2.4, 2.6, 2.8, 3.0], and [0.0] are applied to ports a, b, and c, respectively. Simultaneously, it's necessary to understand the device's performance at temperatures of -40°C, 25°C, and 125°C. In processing the simulation results, the relevant information can be quickly located, such as all simulation results for a temperature of -40°C and a voltage of 1.5 at port a.

[0021] Preferably, the step of obtaining the set based on the original data using a regular expression semantic hybrid parser further includes: The student model is trained based on the original data and the teacher model, and the domain labels and label prediction probabilities corresponding to the original data are obtained. A candidate label set is obtained by calculating a dynamic threshold based on domain labels and predicted label probabilities. The calculation process is as follows: A dynamic threshold is obtained based on the original data, and the calculation expression is: In the formula These are the baseline threshold, the number of samples with feature name f in the original data, and the scaling factor, respectively. The label set is obtained based on the dynamic threshold, and the calculation expression is: In the formula Predict the probability for the label with feature name f. This refers to the data at index i in the tag set; The optimized label set is obtained by calculating the marginal suppression conflict resolution based on the candidate label set; The set is obtained by calculating confidence based on the optimized label set.

[0022] A hybrid regular expression semantic parser performs multimodal parsing on raw data, enabling accurate extraction of feature names and values. This addresses the limitations of traditional regular expressions in handling complex semantics and supports free text scenarios such as logs and medical records. A collaborative training mechanism between teacher and student models facilitates rapid cross-domain knowledge transfer and automatic labeling of raw data, improving labeling efficiency during the cold start phase. A dynamic threshold formula enables feature-granular adaptive label selection, avoiding the loss of long-tail information due to high-frequency monopolies and discovering more potential association patterns. A marginal suppression conflict resolution algorithm effectively filters multi-label ambiguity, and a secondary confidence check of the label set ensures a quality safety net for the final output labels, reducing the error label rate. A mapping transformation from classification vectors to a two-level index enables compact storage and rapid retrieval of high-dimensional sparse data, reducing memory usage and achieving zero-latency response for classification retrieval of massive amounts of data.

[0023] Preferably, the step of obtaining the optimized label set based on the candidate label set through marginal suppression conflict resolution calculation further includes: For each feature name, the marginal confidence gap is obtained through confidence level and marginal suppression. The calculation expression is as follows: In the formula, These are the maximum probability value corresponding to the feature name, the second largest probability value corresponding to the feature name, and the preset probability threshold, respectively. The optimized label set is obtained based on the marginal confidence gap. The specific rule is: if the marginal confidence gap is less than a preset probability threshold, the label data corresponding to the feature name is discarded; otherwise, the optimized label set is obtained. The calculation expression is: In the formula These are the optimized label set, the label data in the candidate label set, the feature name, and the predicted probability of the label data, respectively.

[0024] By quantitatively assessing the marginal confidence gap, the system effectively identifies and filters multi-label ambiguities, accurately captures uncertainties in label competition, and automatically eliminates controversial labels with low confidence. Dynamic threshold control using preset probability thresholds achieves a dynamic balance in label selection. A comparison mechanism between the highest and second-highest probabilities explicitly verifies label authority, avoiding misjudgments caused by a single probability factor; only labels with a significant lead are adopted, reducing overfitting risk. Secondary verification of the optimized label set constructs a dual-protection mechanism, further reducing the error label rate to meet the requirements of highly regulated scenarios such as finance and law. Independent judgment at the feature granularity level ensures that local errors do not affect the global index, and the discarding of labels for a single feature does not spread to other features, enhancing system fault tolerance and reducing maintenance costs.

[0025] Preferably, the step of obtaining the set based on the optimized label set through confidence calculation further includes: obtaining the set based on the optimized label set, and the calculation expression is: In the formula, c is the confidence level, f is the feature name, and v is the feature value corresponding to the feature name.

[0026] By explicitly labeling confidence scores, the credibility of tags is quantitatively represented. Each feature-value pair is assigned an interpretable confidence score, aiding in the rapid identification of high-risk samples during manual review. Quality control down to the individual feature instance level is achieved through the binding calculation of feature names and feature values. A dynamic confidence score calculation model enables continuous evaluation and iteration of feature value, automatically adjusting feature weights based on new data, identifying incrementally important features, and enhancing the model's self-learning capability.

[0027] Preferably, the step of obtaining classification index data based on classification vectors through a two-level index search using multi-dimensional reverse ordering and bitmaps further includes: The first-level index is obtained by using a compressed bitmap inverted index builder based on the classification vector, and the updated first-level index is obtained by using a sorted index optimizer based on the first-level index. Based on historical query logs and first-level indexes, a pre-computation task queue is obtained through a dynamic pre-computation decision-maker, and a second-level index is obtained based on the pre-computation task queue and the first-level index; The final categorized index data is obtained through an index aggregator based on the primary and secondary indexes.

[0028] By employing a compressed bitmap inverted index builder, compact storage and rapid location of high-dimensional sparse data are achieved. Bitmap compression technology further reduces memory consumption, enhancing retrieval capabilities and real-time response times for massive datasets. A sorting index optimizer reorganizes the primary index, achieving descending order of feature popularity. Historical query log mining and a dynamic pre-computation decision-maker enable proactive caching of high-frequency query patterns. An index aggregator merges primary and secondary indexes, achieving a performance balance between general queries and hot queries.

[0029] Preferably, the step of obtaining the first-level index based on the classification vector using a compressed bitmap inverted index constructor further includes: Initialize the global inverted index dictionary, where the keys are the feature classification data and the values ​​are the bitmap data corresponding to the feature classification data; Retrieve each data item from the classification vector and update the index of the data item to the bitmap data; The first-level index is obtained by sorting the index dictionary using the cardinality of the bitmap data.

[0030] By initializing a global inverted index dictionary, centralized management and fast access to feature-document mapping relationships are achieved. Bitmap data for all features is maintained uniformly, avoiding fragmentation issues caused by scattered storage, and reducing the feature retrieval time complexity to O(1). The use of a bitmap data structure enables extreme compression and high-speed computation of feature existence detection, leveraging the compactness of bitmaps. Real-time reflection and synchronization of incremental data are achieved through item-by-item index updates of classification vector data items.

[0031] Preferably, the step of obtaining a subset of query data based on user query conditions through query rewriting, feature sorting, and bitmap intersection further includes: Based on user query conditions, a standardized query vector is obtained by identifying feature names or feature values, expanding domain synonyms, and performing normalization processing of value ranges. Query tasks are obtained by sorting by feature name or feature value based on standardized query vectors and first-level indexes. Based on the query task, the first-level index, and the second-level index, the bitmap intersection operator obtains the initial result bitmap; Based on the initial result bitmap and historical stored data, a subset of query data is obtained through assembly and sorting.

[0032] By expanding domain synonyms and processing value range paradigms, standardized transformation of user query conditions is achieved, eliminating missed detections caused by differences in expression. A dual recognition mechanism of feature names and values ​​enables structured parsing of complex query conditions, avoiding ambiguity in natural language. Standardized query vectors and feature ranking in the first-level index enable priority filtering of high-discrimination features. Hardware acceleration of the bitmap intersection operator enables extremely fast response for multi-feature combination queries. The first-level index quickly locates single-feature matching data, while the second-level index pre-stores high-frequency combination results, enabling intelligent selection of query paths and improving overall query performance.

[0033] By dynamically assembling and sorting historical stored data, the final result set can be precisely pruned and output in an orderly manner.

[0034] Preferably, the rules for obtaining a preliminary result bitmap based on the query task, the first-level index, and the second-level index through the bitmap intersection operator further include: The query task retrieves the cross bitmap from the secondary index, and the value of the preliminary result bitmap is set as the cross bitmap. If the cross bitmap is empty, the primary expected bitmap corresponding to each feature in the query task execution task is retrieved sequentially from the primary index, and the preliminary result bitmap is generated by bitmap intersection operation based on the primary expected bitmap.

[0035] By directly returning the pre-stored cross-bitmap using a secondary index, zero-latency hits are achieved for high-frequency query patterns. For high-frequency feature combinations mined from historical query logs, their cross-bitmaps are pre-calculated and stored, ensuring direct hits for subsequent identical queries, reducing response time to microseconds. Priority scheduling using the secondary index maximizes the efficiency of limited resources, allowing computationally intensive pre-stored cross-bitmap tasks to be executed off-peak.

[0036] Preferably, the step of obtaining updated categorization index data based on a subset of queried data and newly added data using a bitmap merging strategy further includes: Based on the newly added data, a set of bitmaps for the newly added data is obtained through continuous value density binning, discrete value normalization mapping, and bitmap allocation; A temporary merged index is obtained by hierarchically or by merging based on the newly added data bitmap set and the queried data subset; Based on temporary merged indexes and secondary indexes, updated secondary indexes are obtained through incremental cache invalidation and rebuilding. Categorized index data is obtained through the index metadata updater based on temporary merged indexes and secondary indexes.

[0037] By using continuous value density binning and discrete value normalization mapping, standardized encoding and feature space alignment of new data are achieved, mapping discrete values ​​to a unified encoding space and eliminating index bias caused by differences in data distribution. A bitmap allocation mechanism models the mapping between feature names and feature values ​​of new data, dynamically generating corresponding bitmap segments for each new data item, allowing index building to begin without waiting for the full dataset to arrive. Temporary merged indexes are generated through hierarchical or merging strategies, achieving seamless integration of old and new data. Incremental interaction between temporary and secondary indexes enables the inheritance and evolution of historically accumulated knowledge, preserving high-frequency combination patterns stored in the secondary indexes while incorporating the statistical characteristics of the new data.

[0038] Second Embodiment A second aspect of the present invention provides a data search and classification apparatus based on feature value indexing, comprising: Data feature extraction module: used to obtain a set based on the original data using a regular semantic hybrid parser. The set includes feature names and feature values. The classification module is used to obtain classification vectors based on sets through mapping and to obtain classification index data based on classification vectors through a two-level index search using multi-dimensional reverse order and cross-bucket bitmap. The query processing module is used to obtain a subset of query data based on user query conditions through query rewriting, feature sorting, and bitmap intersection; to obtain an updated subset of query data based on the subset of query data through sequential scanning and sorting; and to obtain updated categorized index data based on the subset of query data and newly added data through a bitmap merging strategy.

[0039] A regular semantic hybrid parser extracts multimodal features from raw data, fully capturing feature names and values, laying a high-quality data foundation for subsequent processing. Classification vectors are generated based on the feature set, and a two-level index structure using multi-dimensional reverse order and cross-bucket bitmaps significantly improves data classification and retrieval efficiency. Query rewriting, feature sorting, and bitmap intersection operations enable precise matching and rapid filtering of user query conditions, reducing irrelevant computational overhead. Sequential scanning and sorting of query subsets ensure the accuracy and orderliness of the result set, meeting the refined needs of complex business scenarios. A bitmap merging strategy seamlessly integrates newly added data, supporting incremental updates to the classification index and ensuring system real-time performance and data timeliness.

[0040] In the description of this application, it should be noted that the terms "inner" and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product is in use. They are used only for the convenience of describing this application and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Furthermore, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0041] It should also be noted that, unless otherwise explicitly specified and limited, the terms "setup" and "connection" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0042] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific identification content executed by the system and device described above can be referred to the corresponding process in the foregoing method embodiments.

[0043] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, if these changes fall within the scope of the claims of the present invention and their equivalents, they shall still fall within the protection scope of the present invention.

Claims

1. A data search and classification method based on feature value indexing, characterized in that, include: A set is obtained from the original data using a regular semantic hybrid parser, and the set includes feature names and feature values. The steps of obtaining the set further include: training a student model based on the original data and the teacher model, and obtaining the domain labels and label prediction probabilities corresponding to the original data; obtaining a candidate label set by calculating a dynamic threshold based on the domain labels and the obtained label prediction probabilities; obtaining an optimized label set by calculating a marginal inhibition conflict resolution based on the candidate label set; and obtaining the set by calculating confidence based on the optimized label set. Based on the set, a classification vector is obtained through mapping; Based on the classification vector, classification index data is obtained through multi-dimensional reverse sorting and two-level index search of bitmap; Based on user query conditions, a subset of query data is obtained through query rewriting, feature sorting, and bitmap intersection. The step of obtaining an updated subset of query data based on the specified subset of query data through sequential scanning and sorting further includes: obtaining a subset of query data based on user query conditions through query rewriting, feature sorting, and bitmap intersection; obtaining a standardized query vector based on the user query conditions by identifying feature names or feature values, expanding domain synonyms, and performing normalization processing of value ranges; obtaining a query task based on the standardized query vector and the first-level index according to feature names or feature values; obtaining a preliminary result bitmap based on the query task, the first-level index, and the second-level index through a bitmap intersection operator; and obtaining a subset of query data based on the preliminary result bitmap and historical stored data through assembly and sorting. The updated categorization index data is obtained based on the queried data subset and the newly added data using a bitmap merging strategy.

2. The data search and classification method based on feature value indexing according to claim 1, characterized in that, The calculation process for obtaining the candidate label set based on the domain labels and the predicted probability of the labels using a dynamic threshold is as follows: A dynamic threshold is obtained based on the original data, and the calculation expression is: In the formula The following parameters are used: a baseline threshold, the number of samples with feature name f in the original data, and a scaling factor. The label set is obtained based on the dynamic threshold, and the calculation expression is: In the formula Predict the probability for the label with feature name f. The data with index i in the tag set.

3. The data search and classification method based on feature value indexing according to claim 2, characterized in that, The step of obtaining the optimized label set based on the candidate label set through marginal suppression conflict resolution calculation further includes: For each feature name, the marginal confidence gap is obtained through confidence level and marginal suppression. The calculation expression is as follows: In the formula, These are the maximum probability value corresponding to the feature name, the second maximum probability value corresponding to the feature name, and a preset probability threshold, respectively. The optimized label set is obtained based on the marginal confidence gap. Specifically, if the marginal confidence gap is less than the preset probability threshold, the label data corresponding to the feature name is discarded; otherwise, the optimized label set is obtained. The calculation expression is: In the formula These are, respectively, the optimized label set, the label data in the candidate label set, the feature name, and the predicted probability of the label data.

4. The data search and classification method based on feature value indexing according to claim 2, characterized in that, The step of obtaining the set based on the tuning label set through confidence calculation further includes: obtaining the set based on the tuning label set, and the calculation expression is: In the formula, c is the confidence value, f is the feature name, and v is the feature value corresponding to the feature name.

5. The data search and classification method based on feature value indexing according to claim 1, characterized in that, The step of obtaining classification index data based on the classification vector through a two-level index search using multi-dimensional reverse order and bitmaps further includes: Based on the classification vector, a first-level index is obtained through a compressed bitmap inverted index builder, and an updated first-level index is obtained through a sorted index optimizer. Based on historical query logs and the first-level index, a pre-computation task queue is obtained through a dynamic pre-computation decision-maker, and a second-level index is obtained based on the pre-computation task queue and the first-level index; The final categorized index data is obtained through an index aggregator based on the primary index and the secondary index.

6. The data search and classification method based on feature value indexing according to claim 5, characterized in that, The step of obtaining the first-level index based on the classification vector using the compressed bitmap inverted index builder further includes: Initialize a global inverted index dictionary, where the keys of the index dictionary are feature classification data and the values ​​are bitmap data corresponding to the feature classification data; Obtain each data item of the classification vector and update the index of the data item to the bitmap data; The first-level index is obtained by sorting the index dictionary using the cardinality of the bitmap data.

7. The data search and classification method based on feature value indexing according to claim 1, characterized in that, The rules for obtaining a preliminary result bitmap based on the query task, the first-level index, and the second-level index through the bitmap intersection operator further include: Based on the query task, the secondary index is queried to obtain the cross bitmap, and the value of the preliminary result bitmap is set as the cross bitmap; if the cross bitmap is empty, the primary expected bitmap corresponding to each feature in the query task execution task is sequentially obtained from the primary index, and the preliminary result bitmap is generated based on the primary expected bitmap through bitmap intersection operation.

8. The data search and classification method based on feature value indexing according to claim 1, characterized in that, The step of obtaining the updated categorization index data based on the queried data subset and the newly added data using a bitmap merging strategy further includes: Based on the newly added data, a set of bitmaps for the newly added data is obtained through continuous value density binning, discrete value normalization mapping, and bitmap allocation. Based on the newly added data bitmap set and the query data subset, a temporary merged index is obtained through hierarchical or merging process; The updated secondary index is obtained through incremental cache invalidation and rebuilding based on the temporary merged index and the secondary index; The categorized index data is obtained through the index metadata updater based on the temporary merged index and the secondary index.

9. A data search and classification device based on feature value indexing, characterized in that, include: Data feature extraction module: used to obtain a set based on the original data through a regular semantic hybrid parser, the set including feature name and feature value; The steps of obtaining the set further include: training a student model based on the original data and the teacher model, and obtaining the domain labels and label prediction probabilities corresponding to the original data; obtaining a candidate label set by calculating a dynamic threshold based on the domain labels and the obtained label prediction probabilities; obtaining an optimized label set by calculating a marginal inhibition conflict resolution based on the candidate label set; and obtaining the set by calculating confidence based on the optimized label set. The classification module is used to obtain classification vectors based on the set through mapping and to obtain categorization index data based on the classification vectors through a two-level index search using multi-dimensional reverse order and cross-bucket bitmaps. The query processing module is used to obtain a subset of query data based on user query conditions through query rewriting, feature sorting, and bitmap intersection; to obtain an updated subset of query data based on the subset of query data through sequential scanning and sorting; and to obtain updated categorization index data based on the subset of query data and newly added data through a bitmap merging strategy. The step of obtaining the updated subset of query data further includes: obtaining a subset of query data based on user query conditions through query rewriting, feature sorting, and bitmap intersection further includes: obtaining a standardized query vector based on the user query conditions by identifying feature names or feature values, domain synonym expansion, and value range paradigm processing; obtaining a query task based on the standardized query vector and the first-level index according to feature names or feature values; obtaining a preliminary result bitmap based on the query task, the first-level index, and the second-level index through a bitmap intersection operator; and obtaining a subset of query data based on the preliminary result bitmap and historical stored data through assembly and sorting.

Images