High-efficiency query acceleration and caching optimization device for data weaving

Abstract

This application provides a high-efficiency query acceleration and caching optimization device for data weaving, comprising: a semantic partitioning component: used to dynamically partition multi-source heterogeneous datasets based on multi-source heterogeneous semantic metadata, while embedding cross-domain association identifiers for data weaving to generate a semantic partitioning index table; a query intent mapping component: used to extract the execution intent of user-initiated query requests to generate a query intent representation, and to establish a mapping relationship between the query intent representation and the target data partition to be queried based on the semantic partitioning index table to generate a query-partition mapping table; a cache dynamic sorting component: used to generate a cache priority sorting table based on the query-partition mapping table and historical query frequency statistics; and a scheduling and query execution component: used to optimize the caching mechanism of the target data partition to be queried based on the cache priority sorting table and a preset proportion threshold. This application reduces the time spent on cache retrieval and repeated loading of source data in the later stages of the query.

Description

Technical Field

[0001] This application relates to the field of data management technology, and more specifically, to a device for efficient query acceleration and caching optimization for data weaving. Background Technology

[0002] As enterprises deepen their digital transformation, data weaving technology is widely used to integrate heterogeneous data from multiple sources, such as business systems, IoT devices, and log files, to achieve unified management and efficient querying of cross-domain data. In this scenario, the scale of data is growing exponentially and the relationships between data are becoming increasingly complex. Business departments are placing higher demands on the response speed of data queries and the efficiency of cache resource utilization—efficient query acceleration and cache optimization have become core requirements for ensuring the stable operation of data weaving systems and supporting the timeliness of business decisions.

[0003] In existing data weaving systems, most adopt a static partitioning method based on data storage paths to divide multi-source data without considering the semantic association attributes of the data itself. For user-initiated query requests, the data to be queried is usually located by simple keyword matching, lacking in-depth mining of the business intent behind the query. In the cache priority sorting stage, the caching strategy is mostly determined based on the historical query frequency of the data, without taking into account key characteristics such as data update frequency for comprehensive judgment.

[0004] In the existing solutions mentioned above, static partitioning is difficult to adapt to the dynamic semantic relationships of multi-source data, resulting in the need to traverse multiple unrelated partitions during cross-domain queries, leading to low data location efficiency; simple keyword matching of query intent can easily cause deviations in target data location, increasing the time and resource overhead of loading invalid data; single-dimensional caching and sorting strategies can easily lead to frequent replacement of high-business-value data with low update frequency, or low-value data occupying cache resources for a long time, ultimately resulting in high query response latency of the data weaving system and difficulty in meeting business needs in terms of cache resource utilization. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides an efficient query acceleration and caching optimization device for data weaving, thereby at least alleviating the aforementioned technical problems.

[0006] The technical solutions provided in this application are as follows: An efficient query acceleration and caching optimization device for data weaving, comprising: Semantic partitioning component: used to dynamically partition multi-source heterogeneous datasets based on multi-source heterogeneous semantic metadata, while embedding cross-domain association identifiers of data weaving to generate a semantic partitioning index table; Query intent mapping component: used to extract intent from user-initiated query requests to generate query intent representations, and to establish a mapping relationship between query intent representations and target data partitions to be queried based on the semantic partition index table, so as to generate a query-partition mapping table. Cache dynamic sorting component: used to generate a cache priority sorting table based on the query-partition mapping table and historical query frequency statistics; Scheduling and Query Execution Component: This component optimizes the caching mechanism of the target data partition to be queried based on a cache priority sorting table and a preset ratio threshold.

[0007] The technical solution in this application has the following technical advantages: 1. Regarding the problem in the existing solution that "static partitioning based on data storage paths does not consider semantic relationships, resulting in cross-domain queries requiring traversal of unrelated partitions and low location efficiency": In this solution, the semantic partitioning component dynamically partitions multi-source heterogeneous datasets based on multi-source heterogeneous semantic metadata. This allows for real-time adaptation to the dynamically changing semantic association attributes of multi-source data, rather than relying on fixed storage paths. Simultaneously, it embeds cross-domain association identifiers from data weaving and generates a semantic partitioning index table, directly providing semantic-level partition location criteria for subsequent queries. Compared to static partitioning, this component significantly reduces traversal operations on irrelevant partitions during cross-domain queries, substantially improving data location efficiency and fundamentally alleviating the bottleneck of "low data location efficiency."

[0008] 2. Regarding the problem that existing solutions "locate data through keyword matching, lack in-depth mining of query intent, easily leading to location errors and increased invalid loading overhead": In this solution, the query intent mapping component extracts the intent of user query requests to generate a query intent representation, capturing the underlying business needs (such as data domain preferences and related data requirements) rather than relying solely on surface keywords. Then, based on the semantic partitioning index table, it establishes a mapping relationship between the query intent representation and the target data partitions to be queried, generating a query-partition mapping table. This enables precise association between intent and partitions. Compared to keyword matching, this component reduces the probability of target data location deviation, reduces the time and resource overhead of loading invalid data, and solves the problem of "resource waste caused by query intent matching deviation."

[0009] 3. Regarding the problem that existing solutions "determine caching strategies solely based on historical query frequency, a single dimension can easily lead to the replacement of high-value, infrequently updated data and low utilization of cache resources": In this solution, the dynamic cache sorting component generates a cache priority sorting table based on the query-partition mapping table (reflecting the actual business association requirements of the partition to be queried) and historical query frequency statistics. This allows for a dual-dimensional approach to determining cache priority, combining "partition business association" and "historical access frequency," rather than solely relying on frequency. Subsequent scheduling and query execution components optimize the caching mechanism based on this sorting table and preset percentage thresholds, selectively retaining high-value partitions and releasing resources occupied by low-value partitions. Compared to single-dimensional caching strategies, this component combination reduces the frequent replacement of high-value but infrequently updated data, minimizes the ineffective use of cache resources by low-value data, and improves cache resource utilization.

[0010] 4. Regarding the issue that the existing solution has a "high overall query response latency, making it difficult to meet the timeliness requirements of business decision-making": The semantic partitioning component generates an index table through dynamic semantic partitioning and cross-domain association identifiers, significantly reducing data location time from the source and minimizing invalid traversal time in cross-domain queries. The query intent mapping component uses this index table to accurately associate intents with partitions, reducing the additional time spent loading invalid data. The cache dynamic sorting component generates a sorting table based on business relevance and historical frequency, significantly improving the cache hit rate of high-value data compared to a single-dimensional strategy and reducing reloading time during repeated queries. The scheduling and query execution component optimizes the caching mechanism based on the sorting table, allowing queries to prioritize high-value data in the cache, further compressing the data acquisition chain. This end-to-end collaborative design of "location-matching-caching-execution" amplifies the optimization effects of each stage, significantly shortening the overall query response time compared to the inefficient stacking of stages in existing solutions. This makes the query latency of the data weaving system more aligned with the timeliness requirements of business decisions, providing effective data support for rapid decision-making. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of the structure of an efficient query acceleration and caching optimization device for data weaving, according to an embodiment of this application.

[0012] Figure 2 This is a flowchart illustrating an efficient query acceleration and caching optimization method for data weaving, as described in an embodiment of this application. Detailed Implementation

[0013] Figure 1 This is a schematic diagram of a high-efficiency query acceleration and caching optimization device for data weaving, according to an embodiment of this application. Figure 1 As shown, a high-efficiency query acceleration and caching optimization device for data weaving includes: Semantic partitioning component: used to dynamically partition multi-source heterogeneous datasets based on multi-source heterogeneous semantic metadata, while embedding cross-domain association identifiers of data weaving to generate a semantic partitioning index table; Query intent mapping component: used to extract intent from user-initiated query requests to generate query intent representations, and to establish a mapping relationship between query intent representations and target data partitions to be queried based on the semantic partition index table, so as to generate a query-partition mapping table. Cache dynamic sorting component: used to generate a cache priority sorting table based on the query-partition mapping table and historical query frequency statistics; Scheduling and Query Execution Component: This component optimizes the caching mechanism of the target data partition to be queried based on a cache priority sorting table and a preset ratio threshold.

[0014] Optionally, the scheduling and query execution component is specifically used to: perform association preloading based on the cache priority sorting table and preset ratio threshold, and according to the association rule mining results of historical query sequences, to optimize the caching mechanism of the target data partition to be queried.

[0015] Optionally, the semantic partitioning component is used to dynamically partition multi-source heterogeneous datasets based on multi-source heterogeneous semantic metadata and embed cross-domain association identifiers to generate a semantic partitioning index table. Specifically, it includes the following two modules: Parsing Enhancement Module: Used to perform structured parsing of multi-source heterogeneous semantic metadata to extract the basic dimensions of data domain affiliation and data type, generate basic semantic dimensions to complete the hidden relationships between different data sources through cross-domain data association rules, and generate enhanced semantic metadata; Partitioning and Identifier Embedding Module: This module is used to determine the partitioning granularity based on enhanced semantic metadata, to perform dynamic partitioning of multi-source heterogeneous datasets to determine partitioning mapping relationships, and then embed cross-domain association identifiers woven from data into the partitioning mapping relationships to generate a semantic partitioning index table.

[0016] Optionally, the parsing enhancement module is used to perform structured parsing of multi-source heterogeneous semantic metadata to extract the basic dimensions of data domain affiliation and data type, generate basic semantic dimensions, and then use cross-domain data association rules to mine and complete the hidden relationships between different data sources to generate enhanced semantic metadata. Specifically, it includes the following three components: Basic Dimension Extraction Component: Used to perform structured parsing of multi-source heterogeneous semantic metadata to extract information on data domain affiliation, data type, and field meaning descriptions to generate basic semantic dimensions; Cross-domain association completion component: Based on the basic semantic dimension set, it performs association rule mining on multi-source heterogeneous semantic metadata through cross-domain data association rules to identify hidden association relationships between semantic metadata from different data sources; Enhanced metadata integration component: Used to align fields between basic semantic dimensions and hidden relationships to generate enhanced semantic metadata.

[0017] Preferably, the specific implementation process of the basic dimension extraction component is as follows: First, multi-source heterogeneous semantic metadata is used as the processing object, and data source type classification processing is performed on it—based on the source of metadata (such as business systems, IoT devices, log files) and format (structured, semi-structured, unstructured), it is divided into different type groups, generating data source type classification results; then, based on the data source type classification results, differentiated structured parsing rules are adopted for the metadata of different type groups: for the structured metadata (such as relational database table structure metadata) in the classification results, the data domain affiliation (such as "order data domain", "user count") is extracted through a preset field mapping table. The data includes the domain, data type (e.g., "integer" or "string"), and field meaning descriptions (e.g., "order amount: records the total transaction amount of the order"). For semi-structured metadata (e.g., JSON format log metadata) in the classification results, the same dimension information is extracted using a tag parsing algorithm. For unstructured metadata (e.g., document-type metadata) in the classification results, keyword extraction and semantic annotation techniques are used to supplement the data domain attribution and field meaning descriptions, generating multi-dimensional parsing information. Finally, the multi-dimensional parsing information is processed according to a fixed structure of "data source type - data domain attribution - data type - field meaning" to generate basic semantic dimensions.

[0018] Preferably, in the specific technical implementation of the cross-domain relationship completion component, the basic semantic dimension set and multi-source heterogeneous semantic metadata are used as common processing objects. First, a cross-domain relationship analysis model is constructed based on the basic semantic dimension set—extracting "data domain affiliation" and "field meaning description information" as core feature items from the basic semantic dimension set, and establishing a feature item index table in the format of "feature item-data source identifier"; then, based on the feature item index table, association rule mining is performed on the multi-source heterogeneous semantic metadata: the first step is to calculate the semantic similarity of feature items between metadata from different data sources (e.g., calculating the similarity between "user ID" and "data source identifier" in the feature item index table using a word vector model). The first step is to determine the semantic distance of the "customer identifier". The second step is to retrieve the metadata relationships in the historical cross-domain query records as an auxiliary verification basis for the semantic similarity calculation results. The third step combines the results of the above two steps to identify the co-occurrence relationships between semantic metadata of different data sources (such as the high-frequency co-occurrence of "order data field" in data source A and "payment data field" in data source B) and field dependency relationships (such as "order number" in data source A depending on "transaction serial number" in data source B), and generate the relationship identification results. Finally, the relationship identification results are used as the processing object and organized in a structured format of "source data source - target data source - associated field - association type" to generate a hidden relationship table.

[0019] Preferably, in a specific implementation of the enhanced metadata integration component in a scenario, the basic semantic dimension and the hidden relationship table are used as common processing objects. First, field alignment processing is performed: based on the "field meaning description information" in the basic semantic dimension, and comparing it with the "related field" information in the hidden relationship table, the field naming conventions of different data source metadata are unified (e.g., "User ID" in data source A and "Customer Identifier" in data source B are uniformly labeled as "User Unique Identifier"). Simultaneously, the "relationship type" information from the hidden relationship table is supplemented in the corresponding fields of the basic semantic dimension to generate the field alignment result. Then, using the field alignment result as the processing object, redundancy removal processing is performed: through field hash ratio... For duplicate field descriptions (such as "field meaning descriptions" with completely identical meanings in different data sources), invalid fields (such as null fields that cannot match any data field) are removed using null value detection rules, generating a deduplicated integration intermediate. Finally, using the integration intermediate as the processing object, a semantic association index is established—binding the hidden association relationship (derived from the hidden association relationship table) corresponding to each data field in the integration intermediate to the basic semantic dimension of that data field, forming a complete structure of "data field - basic semantic dimension - association relationship - association index", ultimately generating enhanced semantic metadata. This enhanced semantic metadata can be directly used as the core basis for determining the partition granularity of the subsequent partitioning and identifier embedding modules.

[0020] Optionally, the partitioning and identifier embedding module is used to determine the partitioning granularity based on enhanced semantic metadata, to perform dynamic partitioning of multi-source heterogeneous datasets to determine partitioning mapping relationships, and then embed cross-domain association identifiers woven from data into the partitioning mapping relationships to generate a semantic partitioning index table. Specifically, it includes the following three components: Partition granularity rule generation component: used to determine the density of data domain affiliation and association based on the enhanced semantic meta dataset, to calculate the optimal partition granularity threshold for different data domains, and to formulate partitioning rules according to the optimal partition granularity threshold to generate a dynamic partitioning rule table; Multi-source data partitioning execution component: Used to perform distributed dynamic partitioning processing on multi-source heterogeneous datasets based on a dynamic partitioning rule table, so as to allocate data of different data domains and different correlation strengths to corresponding partitions and generate partition mapping relationships; Cross-domain identifier embedding component: used to parse the cross-domain association requirement expression of data weaving to generate a unique cross-domain association identifier, and embed the unique cross-domain association identifier into the partition mapping relationship to generate a semantic partition index table.

[0021] Preferably, the specific implementation process of the partitioning granularity rule generation component is as follows: First, taking the enhanced semantic metadata as the processing object, perform data domain affiliation and association density extraction processing on it—extract the "data domain affiliation" information (such as "order domain", "user domain", "payment domain") from the enhanced semantic metadata, and at the same time count the association frequency between different data domains (such as the number of association queries between the order domain and the payment domain, and the number of field dependencies between the user domain and the order domain), calculate the association density (association frequency / total data volume), and organize the two in the format of "data domain-related data domain-association density" to generate a data domain affiliation-association density comparison table; then, based on the data domain affiliation-association density comparison table, calculate the optimal partitioning granularity threshold for different data domains: for association For data domains with high relational density (such as order and payment domains), set smaller thresholds (to ensure related data are in similar partitions and reduce cross-partition queries). For data domains with low relational density (such as log and user domains), set larger thresholds (to avoid excessively fine partitioning leading to increased management overhead). Generate a data domain-optimal threshold mapping table. Finally, based on the data domain-optimal threshold mapping table, formulate partitioning rules (including "data domain matching rules," "relationship strength partitioning rules," and "threshold adjustment trigger conditions," such as recalculating the threshold when the data domain relational density changes by more than 20%). Organize these rules according to the "rule ID-data domain-threshold-execution condition" structure to generate a dynamic partitioning rule table. This table differs from traditional static partitioning rules and can be adapted and adjusted as data relations change.

[0022] Preferably, in the specific technical implementation of the multi-source data partitioning execution component, the dynamic partitioning rule table and the multi-source heterogeneous dataset are used as common processing objects. First, data grouping preprocessing is performed: based on the "data domain matching rule" in the dynamic partitioning rule table, each piece of data in the multi-source heterogeneous dataset is mapped to its corresponding data domain (e.g., data containing the "order number" field is mapped to the order domain). Simultaneously, referring to the "association strength partitioning rule," cross-domain data with high association density (e.g., order data and corresponding payment data) are marked as "association group data," generating data domain-association group grouping results. Then, based on the data domain-association group grouping results... According to the method, distributed dynamic partitioning is performed on multi-source heterogeneous datasets: independent processing nodes are allocated according to different data domains, cross-node collaborative allocation is adopted for "related group data" (ensuring that data in the same related group falls into adjacent partitions, reducing cross-node query latency), and non-related data is split into corresponding partitions according to the optimal partition granularity threshold, and the partition identifier of each data is recorded (such as "order domain-001" and "user domain-003"). Finally, the correspondence between data and partition identifiers is organized in the format of "data ID-data domain-partition identifier-related group tag" to generate a partition mapping relationship, which can directly provide the partition location basis for subsequent cross-domain related queries.

[0023] Preferably, in a scenario, when the cross-domain identifier embedding component is specifically implemented, the cross-domain association requirement description woven from data (such as "support cross-domain query between order domain data and payment domain data" or "user domain data needs to be associated with log domain operation records") is taken as the processing object, and requirement parsing and identifier generation processing are performed: The source data domain, target data domain, and association type (such as "query," "dependency," and "traceability") in the requirement are decomposed, and a unique cross-domain association identifier (such as "ORD-PAY-JOIN-20240510," representing order domain, payment domain, query, and generation time, respectively) is generated using the format "source domain identifier-target domain identifier-association type code-timestamp" to ensure that each cross-domain association scenario corresponds to a unique identifier, and a cross-domain association identifier-requirement lookup table is generated; then, the cross-domain association identifier-requirement lookup table and the partition mapping relationship are used as... For common processing objects, perform identifier embedding processing: Filter out partition entries involving cross-domain associations from the partition mapping relationship (such as order domain partitions and payment domain partitions containing "association group tag"). Embed the corresponding identifier from the cross-domain association identifier-demand lookup table into the "association identifier" field of the partition entry (e.g., embed "ORD-PAY-JOIN-20240510" for the order domain-001 partition). Finally, using the partition mapping relationship after embedding the identifier as the processing object, supplement the partition index information (such as partition storage path, data update time, association identifier index), organize it according to the structure of "partition identifier-data domain-association identifier-storage path-update time", and generate a semantic partition index table. This table allows subsequent queries to directly locate cross-domain partitions through the association identifier, which has higher cross-domain query location efficiency than traditional unidentified partition tables.

[0024] Optionally, the query intent mapping component is used to extract the intent of a user's query request and establish a mapping relationship with the target data partition to be queried, so as to generate a query-partition mapping table. Specifically, it includes the following two modules: Intent extraction module: It is used to perform statement-by-state grammatical parsing on the query request initiated by the user to extract the data domains and filtering conditions involved in the query, and generate grammatical parsing results accordingly. Based on the grammatical parsing results, it integrates the user's historical query context to supplement implicit needs, so as to generate a structured query intent representation. Partition Mapping Module: Used to extract cross-domain association identifiers and partition attributes from the semantic partition index table, and then establish a matching relationship between the structured query intent representation and the target data partition to be queried, so as to generate a query-partition mapping table.

[0025] Optionally, the intent extraction module is used to perform statement-by-state syntactic parsing on the user's query request to extract the data domains and filtering conditions involved in the query, and generate syntactic parsing results accordingly. Based on the syntactic parsing results, the module integrates the user's historical query context to supplement implicit requirements, thereby generating a structured query intent representation. Specifically, it includes the following three components: Syntax parsing component: Used to perform statement-by-state syntax parsing on user-initiated query requests to identify data entities, operational logic, data fields, and filtering conditions in the query, and organize them into key-value pair structures according to a preset format to generate query syntax parsing results; Query Context Fusion Component: This component extracts data domain preferences and filtering logic related to the current query from the user's historical query records and integrates them into the query syntax parsing results to generate context-enhanced intent fragments. Structured Intent Component: Used to integrate context-enhanced intent fragments by “data domain - filtering conditions - aggregation requirements - related data identifiers” to generate a structured query intent representation.

[0026] Preferably, the specific implementation process of the syntax parsing component is as follows: First, taking the user-initiated query request (such as "querying user payment records of e-commerce orders with an amount greater than 1000 yuan in the second quarter of 2024") as the processing object, sentence-by-sentence syntax splitting is performed: the query is split into independent sentences according to punctuation marks and semantic pauses (if it is a single sentence, it is processed directly), and each sentence is segmented into words using a bidirectional LSTM word segmentation model (such as segmenting into "query / second quarter of 2024 / e-commerce orders / in / amount / greater than / 1000 yuan / of / user payment / records"), generating query word segmentation results; then, based on the query word segmentation results, key element identification is performed: through a pre-trained domain entity recognition model (optimized for data weaving scenarios, strengthening "data"...) Based on the ability to identify entities such as "domain" and "filter conditions", the system identifies data entities (e.g., "e-commerce orders" and "user payment records"), operational logic (e.g., "greater than"), data domains (e.g., "order domain" and "payment domain"), and filter conditions (e.g., "second quarter of 2024" and "amount greater than 1000 yuan"), generating an element identification list. Finally, using the element identification list as the processing object, the system organizes the data according to the preset key-value pair structure of "data entity: value, operational logic: value, data domain: value, filter condition: value" (e.g., "data entity: e-commerce order; data domain: order domain; filter condition: second quarter of 2024, amount > 1000 yuan"), generating query syntax parsing results. These results are more suitable for the element extraction needs of multi-source heterogeneous data queries than traditional general syntax parsing.

[0027] Preferably, in the specific technical implementation of the query context fusion component, the query syntax parsing results and user historical query records are used as common processing objects. First, historical query relevance filtering is performed: The "data field," "filtering conditions," and "query time" information of each record are extracted from the user's historical query records. The matching degree with the "data field" in the current query syntax parsing results is calculated (e.g., if the historical query contains an "order field" and the current query also involves an "order field," the matching degree is higher). Simultaneously, the query time is considered (historical queries within the last 30 days have higher weight), and the top 3 relevant historical query records are filtered out to generate a relevant historical query subset. Then, based on the relevant historical query subset and the query syntax parsing results, key information is extracted and fused: from the relevant historical query subset... The algorithm extracts data domain preferences (e.g., all of the user's last three queries contain the association of the "payment domain") and filtering logic preferences (e.g., the user often uses "quarter" as the time filtering granularity). Using a dynamic weighting algorithm (preferences from recent queries have higher weights than those from earlier queries), this preference information is added to the corresponding fields of the query syntax parsing results (e.g., adding "implicitly associated payment domain" to the "data domain" field and "default quarterly granularity" to the "filtering conditions" field), generating a preliminary fusion result. Finally, using this preliminary fusion result as the processing object, conflict detection and correction are performed (e.g., when the filtering conditions of historical preferences conflict with the explicit conditions of the current query, the current query takes precedence), generating a context-enhanced intent fragment. This fragment, compared to a simple syntax parsing result, better reflects the implicit needs that the user has not directly expressed.

[0028] Preferably, in a scenario, when the structured intent component is specifically implemented, the context-enhanced intent fragment is used as the processing object. First, dimension extraction processing is performed: the context-enhanced intent fragment is decomposed into the "data domain" dimension (including explicit data domains and implicit related data domains supplemented by context, such as "order domain + implicit payment domain"), the "filtering condition" dimension (including specific constraints such as time and value, such as "2024Q2, amount > 1000 yuan"), the "aggregation requirement" dimension (derived from the operation logic, such as "query" implying "list display", "average amount" implying "aggregation calculation"), and the "related data identifier" dimension (extracted from cross-domain related requirements, such as the "ORD-PAY-JOIN" identifier in the semantic partition index table corresponding to "user payment record"), generating a four-dimensional element set; then, based on the four-dimensional element set, structured integration is performed. The process involves filling in information from the four-dimensional element set according to a fixed format of "data domain:[], filter conditions:[], aggregation requirements:[], and associated data identifier:[]" (e.g., "data domain:['order domain', 'payment domain']; associated data identifier:['ORD-PAY-JOIN']"). Simultaneously, confidence scores for each dimension are added (e.g., the confidence score of the implicit data domain supplemented by the context is lower than that of the explicit data domain), generating a structured draft. Finally, using the structured draft as the processing object, format validation and standardization are performed (ensuring that the information format of each dimension matches the field requirements of the semantic partition index table, such as uniformly converting "filter conditions" to the format "field name:operator:value"), generating a structured query intent representation. This representation can be directly used as input for the query intent mapping component to establish partition mapping, exhibiting higher mapping efficiency than unstructured intent representations.

[0029] Optionally, the partition mapping module is used to extract cross-domain association identifiers and partition attributes from the semantic partition index table, and then establish a matching relationship between the structured query intent representation and the target data partition to be queried, so as to generate a query-partition mapping table. Specifically, it includes the following three components: Mapping rule generation component: used to extract partition attributes from semantic partition index table to obtain the data domain range, cross-domain association identifier and query adaptation conditions of each target data partition to be queried, and to formulate rules in the format of "intent element-partition attribute-matching weight" to generate intent-partition mapping rules; Mapping Relationship Matching Component: Based on the structured query intent representation, it calculates the matching degree between each intent element and the partition attribute by referring to the intent-partition mapping rule set, and filters the target data partitions whose matching degree meets the preset threshold to generate a mapping relationship table; Mapping relationship optimization component: used to remove redundancy and prioritize mapping entries in the mapping relationship table, remove duplicate partition mapping entries, sort by matching degree from high to low and mark the mapping basis to generate a query-partition mapping table.

[0030] Preferably, the specific implementation process of the mapping rule generation component is as follows: First, taking the semantic partition index table as the processing object, perform partition attribute extraction processing: extract the "data domain range" (such as "order domain - 2024Q2" and "payment domain - online channel"), "cross-domain association identifier" (such as "ORD-PAY-JOIN-20240510"), and "query adaptation conditions" (such as "supports amount filtering" and "adapts to time range query") of each partition from the semantic partition index table, and organize these attributes in the format of "partition identifier - data domain range - cross-domain association identifier - query adaptation conditions" to generate a partition attribute list; then, based on the partition attribute list, construct the correspondence between intent elements and partition attributes: match "data domain range" with "data domain" in intent elements, and match "cross-domain association identifier" with "data domain range" and "cross-domain association identifier" with "data domain range". The "Associated Data Identifier" and "Query Adaptation Condition" in the intent element correspond to the "Filter Condition" in the intent element, and a matching weight is assigned to each correspondence (e.g., the matching weight for cross-domain association identifier is set to 0.4, the matching weight for data domain range is set to 0.3, and the matching weight for query adaptation condition is set to 0.3, because association identifier can more accurately locate cross-domain partitions), generating a weight allocation result. Finally, the weight allocation result and the partition attribute list are used as common processing objects, and rules are formulated in the format of "intent element: data domain; partition attribute: data domain range; matching weight: 0.3" and "intent element: associated data identifier; partition attribute: cross-domain association identifier; matching weight: 0.4", etc. After summarizing, an intent-partition mapping rule set is generated. This rule set is different from the traditional single-dimensional matching rules, and improves the mapping accuracy through multi-dimensional weighting.

[0031] Preferably, in the specific technical implementation of the mapping relationship matching component, the structured query intent representation and the intent-partition mapping rule set are used as common processing objects. First, intent element decomposition is performed: intent elements such as "data domain," "related data identifier," and "filtering conditions" are extracted from the structured query intent representation (e.g., "data domain: ['order domain', 'payment domain']; related data identifier: ['ORD-PAY-JOIN']"). These are then categorized according to the "intent element" type in the intent-partition mapping rule set to generate a set of intent elements to be matched. Next, based on the set of intent elements to be matched and the intent-partition mapping rule set, the element matching degree is calculated: for each intent element, the corresponding partition attribute in the rule set is compared. A semantic similarity algorithm (such as editing distance to calculate the matching degree between "order domain" and "order domain-2024Q2") is used to obtain the single-feature matching degree, which is then multiplied by the matching weight in the rule set to obtain the weighted matching score. For the same partition, the weighted matching scores of all intent elements are summarized to obtain the total matching degree of the partition, and a partition-total matching degree lookup table is generated. Finally, using the partition-total matching degree lookup table as the processing object, target data partitions with a total matching degree ≥ a preset threshold (such as 0.6) are selected (irrelevant partitions with too low matching degree are removed), and they are organized in the format of "structured intent ID-partition identifier-total matching degree-feature matching details" to generate a mapping relationship table, which can directly reflect the correlation strength between the query intent and the target partition.

[0032] Preferably, in a specific implementation of the mapping relationship optimization component in a scenario, the mapping relationship table is the processing object. First, redundancy removal is performed: using a partition identifier hash deduplication algorithm, entries with the same "partition identifier" in the mapping relationship table are merged (retaining the entry with the highest total matching degree), while invalid entries in the "feature matching details" where all feature matching degrees are below 0.2 (such entries have no practical value for the query) are deleted, generating a de-redundant mapping table. Then, based on the de-redundant mapping table, priority sorting is performed: entries are sorted from highest to lowest "total matching degree," and when total matching degrees are the same, priority is given to sorting... Entries with higher matching scores for the "Cross-Domain Association Identifier" element (because this element is more critical for cross-domain queries) are processed to generate a sorted mapping table. Finally, using the sorted mapping table as the processing object, mapping criteria are added to each entry (e.g., "Matching score 0.8: Data domain matching 0.3 + Association identifier matching 0.4 + Filter condition matching 0.1"). The table is formatted with a fixed structure of "Sorting number - Partition identifier - Total matching score - Mapping criteria" to generate a query-partition mapping table. This table is more concise than the original mapping table, and the sorting results can directly guide the priority calculation of the subsequent cached dynamic sorting component, improving query efficiency.

[0033] Optionally, the cache dynamic sorting component is specifically used to: determine the target data partition to be queried based on the query-partition mapping table; calculate the time decay query heat value of each target data partition to be queried by combining historical query frequency statistics and query time decay factor; calculate the cache validity period of each target data partition to be queried by combining data update frequency data; identify the access cycle characteristics of each target data partition to be queried by long-term cycle pattern mining and generate cycle feature coefficients; and generate cache priority sorting by integrating time decay query heat value, cache validity period and cycle feature coefficients.

[0034] Optionally, the cache dynamic sorting component specifically includes the following four modules: Target data partition determination module: used to perform structured parsing of the query-partition mapping table to determine the target data partition to be queried and then generate the target partition attribute table; Time decay heat calculation module: Based on the target partition attribute table and combined with historical query frequency statistics, it calculates the query time decay factor and historical query distribution characteristic coefficient for each target data partition to be queried, so as to generate a time decay query heat value. Cache validity period calculation module: used to calculate the cache validity period of each target data partition to be queried based on the data update frequency data, and then calculate the periodic feature coefficient by mining the long-term periodic pattern. Multi-dimensional priority fusion module: It is used to perform weighted fusion of time decay query popularity value, cache validity period and periodic feature coefficient to calculate priority score, sort each target data partition to be queried in descending order of priority score and mark the score composition details to generate cache priority ranking.

[0035] Optionally, the target data partition determination module is used to perform structured parsing of the query-partition mapping table to determine the target data partition to be queried and then generate the target partition attribute table. Specifically, it includes the following three components: Partition identifier parsing component: used to perform field-level structured parsing on query-partition mapping table, locate the unique identifier of the target data partition to be queried through the preset partition identifier extraction rule base, and use semantic role labeling to identify the data filtering range and related data requirements in the query matching conditions to generate a partition identifier-matching condition lookup table; Partition Attribute Supplement Component: Based on the partition identifier-matching condition lookup table, it retrieves the complete attributes of the corresponding target data partition, the type identifier of the associated data source, and the data update history identifier from the semantic partition index table through the partition unique identifier association, in order to supplement the corresponding entries in the partition identifier-matching condition lookup table and generate the initial partition attribute table; The attribute table integration component is used to remove duplicate entries from the initial partition attribute table using a partition identifier hash deduplication algorithm and remove invalid fields using null value detection rules. Then, it performs structured arrangement according to a fixed field order of "partition identifier - data domain - associated data source - query matching conditions" to generate the target partition attribute table.

[0036] Preferably, the specific implementation process of the partition identifier parsing component is as follows: First, taking the query-partition mapping table as the processing object, perform field-level structured parsing: extract the content of the fields "sort number", "partition identifier", "total matching degree", and "mapping basis" from the query-partition mapping table row by row (e.g., "sort number: 1; partition identifier: ORD-2024Q2-001; total matching degree: 0.8; mapping basis: data domain matching 0.3 + association identifier matching 0.4 + filter condition matching 0.1"). Organize the extracted field information according to the format "row number-field name-field value" to generate the preliminary parsing result of the mapping table. Then, using the preliminary parsing result of the mapping table and the preset partition identifier extraction rule base as a common basis, locate the unique identifier of the target data partition to be queried: the partition identifier extraction rule base is designed for the identifier format of multi-source heterogeneous partitions in data weaving scenarios, including templates such as "data domain-time range-partition number" and "data source type-data domain-partition ID". The unique identifier of the partition is determined by rule matching. The unique identifier is extracted from the "Partition Identifier" field of the initial parsing result of the mapping table (e.g., extracting the unique identifier "ORD-2024Q2-001" from "ORD-2024Q2-001") to generate a list of unique partition identifiers. Then, using the "Mapping Basis" field from the initial parsing result as the processing object, the query matching conditions are extracted (e.g., extracting "Amount > 1000 yuan, Time = 2024Q2" from the original conditions associated with "Filter Condition Match 0.1"). Semantic role labeling technology is used to identify the "Data Filtering Range" (e.g., "Time = 2024Q2, Amount > 1000 yuan") and "Associated Data Requirements" (e.g., "Associated Payment Domain Data") in the conditions, generating condition parsing results. Finally, the list of unique partition identifiers and the condition parsing results are integrated in the format "Unique Partition Identifier - Data Filtering Range - Associated Data Requirements" to generate a partition identifier-matching condition lookup table. This table can directly provide accurate identifiers and conditional basis for subsequent attribute supplementation.

[0037] Preferably, in the specific technical implementation of the partition attribute supplementation component, the partition identifier-matching condition lookup table and the semantic partition index table are used as common processing objects. First, identifier association preparation is performed: all "partition unique identifiers" are extracted from the partition identifier-matching condition lookup table, and an association index is established in the format of "identifier-row number" (to ensure that the subsequently supplemented attributes can correspond to the original table entries), generating an identifier association index table; then, using the "partition unique identifier" in the identifier association index table as the query key, the complete attributes of the corresponding partition in the semantic partition index table are retrieved (including "data domain range", "cache current status", "cross-domain association identifier", such as "data domain range: order domain - 2024Q2; cache current status: not cached"), the type identifier of the associated data source (such as "data source type: MySQL relational database - order database; interface protocol: JDBC"), and the data update history identifier (such as "last update time: 20 (Updated 24-06-30 23:59; Update frequency: once daily) This information is organized into the format of "Partition Unique Identifier - Complete Attribute - Data Source Type Identifier - Update History Identifier" to generate an attribute supplementary dataset. Finally, based on the partition identifier-matching condition lookup table, the information in the attribute supplementary dataset is added to each entry in the table one by one (e.g., for the entry "Partition Unique Identifier: ORD-2024Q2-001", add "Data Domain Range: Order Domain-2024Q2; Data Source Type: MySQL Relational Database-Order Database"). After the addition is completed, the "Partition Unique Identifier - Data Filtering Range - Related Data Requirements" field is retained according to the original table structure, and the "Complete Attribute - Data Source Type Identifier - Update History Identifier" field is added to generate the initial partition attribute table. This table adds key attributes related to the partition operation and data source compared to the original lookup table, providing a more comprehensive basis for subsequent cache sorting.

[0038] Preferably, in a specific implementation of the attribute table integration component in a scenario, the initial partition attribute table is used as the processing object. First, partition identifier hash deduplication is performed: a hash value is calculated for the "unique partition identifier" field in the initial partition attribute table. Duplicate entries are identified by hash value comparison (if two entries have the same hash value, they are considered duplicates). The most recently updated entry among the duplicate entries is retained (to ensure data timeliness), and the remaining duplicate entries are deleted, generating a deduplicated partition attribute table. Next, based on the deduplicated partition attribute table, null value detection and invalid field removal are performed: null value detection rules are used (e.g., if the "complete attribute" or "data source type identifier" fields are empty, they are considered invalid), identifying and removing entries containing invalid fields (such entries are invalid due to missing attributes). The method participates in subsequent cache calculations), while deleting redundant temporary fields in the table (such as the "row number" field in the identifier related index table), generating a cleaned partition attribute table; finally, using the cleaned partition attribute table as the processing object, the order of the fields is rearranged according to the fixed field order of "partition identifier - data domain (extracted from complete attributes) - related data source (extracted from data source type identifier) ​​- query matching conditions (including data filtering range + related data requirements)" to ensure that the information format of each field is uniform (such as the "data domain" field being uniformly formatted as "main domain - subdomain"), generating a target partition attribute table. This table has a standardized format and no redundancy, and can be directly used as input data for the time decay heat calculation module and the cache validity period calculation module, avoiding subsequent processing anomalies caused by inconsistent formats.

[0039] Optionally, the time decay popularity calculation module is used to calculate the query time decay factor and historical query distribution characteristic coefficient for each target data partition to be queried, based on the target partition attribute table and combined with historical query frequency statistics, in order to generate a time decay query popularity value. Specifically, it includes the following three components: Attenuation factor configuration component: Based on the target partition attribute table, it is used to automatically identify the data domain sensitivity level and query timeliness requirements of each target data partition to be queried through business tags, and call the dynamic rule engine to match the "data domain sensitivity-timeliness requirements-attenuation coefficient" association model to generate a query time attenuation factor for each target data partition to be queried. Distribution characteristic coefficient calculation component: Based on the target partition attribute table, it retrieves the historical query logs of each target data partition to be queried, uses time series clustering analysis to analyze the concentration index of the query period, evaluates the query frequency fluctuation amplitude index through fluctuation trend fitting, and then integrates the query period concentration index and the query frequency fluctuation amplitude index through a multi-dimensional feature fusion model to generate historical query distribution characteristic coefficients. Popularity Value Integration Component: Based on the query time decay factor, historical query distribution characteristic coefficient, and query frequency in multiple time periods from historical query frequency statistics, this component performs multi-parameter collaborative calculation through a weighted fusion model and structures the calculation results in the format of "partition identifier - time decay query popularity value" to generate a time decay query popularity value.

[0040] Preferably, the specific implementation process of the attenuation factor configuration component is as follows: First, taking the target partition attribute table as the processing object, extract the business tags associated with the "data domain attribute" field in the table—match the preset business tag library (including "data domain sensitivity level tag" and "query timeliness requirement tag" from the "partition identifier-data domain" entry (such as "ORD-2024Q2-001-order domain"), for example, "high data domain sensitivity level" for order domain and payment domain, and "low data domain sensitivity level" for log domain and backup domain; "high query timeliness requirement" for real-time reconciliation scenario, and "low query timeliness requirement" for historical report scenario) to automatically identify the data domain sensitivity level and query timeliness requirement of each target data partition to be queried, and generate a partition business tag list; then, taking the partition business tag list as input, call the pre-built dynamic rule engine—this engine has a built-in "data domain sensitivity-timeliness requirement-attenuation coefficient" association model, which is designed for data weaving scenarios. The design incorporates the value decay characteristics of multi-source data (e.g., "high data domain sensitivity level + high query timeliness requirement" corresponds to a higher decay rate because this type of data is frequently updated, and the reference value of old query records for the current query decreases rapidly; "low data domain sensitivity level + low query timeliness requirement" corresponds to a lower decay rate because this type of data is highly stable, and old query records still have some value). Initial decay coefficients for each partition are output through precise tag matching. Finally, using the initial decay coefficients as the processing object, the coefficients are fine-tuned in conjunction with the "data update history identifier" (e.g., "recent update frequency: 3 times per day" and "update interval fluctuation: ±1 hour") in the target partition attribute table. If the data update frequency is higher than the average level of the same domain, the initial decay coefficient is increased by 8%-12% (to avoid excessive deviation between old query data and current data); otherwise, it is decreased by 3%-5%. Ultimately, a query time decay factor is generated for each target data partition to be queried, ensuring that the factor is deeply adapted to the business attributes and update characteristics of the partition.

[0041] Preferably, in the specific technical implementation of the distribution characteristic coefficient calculation component, firstly, the target partition attribute table is used as the processing object, and the "partition identifier" (such as "ORD-2024Q2-001" and "PAY-2024Q2-002") in the table is extracted. Based on the "partition identifier", the historical query logs of each target data partition to be queried for the past 90 days are retrieved (the logs contain the fields of "query timestamp", "query count" and "query source" to ensure that the logs correspond one-to-one with the partitions and avoid cross-partition data interference), generating a partition-historical log association set. Then, the historical query logs in the partition-historical log association set are used as the processing object, and a time series clustering algorithm (such as density-based DBSCAN clustering, with the clustering dimension set to "query timestamp (divided by hour) - query count") is used to analyze the concentration index of query time periods. For example, if 70% of the queries in a certain order domain partition are concentrated in two time periods, 9:00-11:00 and 14:00-16:00 every day, the "high query time period concentration index" is output after clustering. If the queries are evenly distributed in each time period, the "low query time period set" is output. The system first generates a set of time-period concentration indicators. Then, using the "daily query frequency" in the historical query log as the processing object, it performs linear trend fitting using a sliding window (window size set to 7 days) to calculate the deviation rate between the actual daily query frequency and the fitted trend line (deviation rate = |actual frequency - fitted frequency| / fitted frequency). The lower the deviation rate, the "lower value" (frequency stable) the query frequency fluctuation index is, and the higher the deviation rate, the "higher value" is, generating a set of frequency fluctuation indexes. Finally, using the time-period concentration index set and the frequency fluctuation index set as common inputs, it performs integration through a multi-dimensional feature fusion model. The model allocates weights according to "time-period concentration index accounting for 60% and frequency fluctuation index accounting for 40%" (because partitions with concentrated time periods are more likely to improve caching efficiency through targeted preloading, hence the higher weight). It calculates the comprehensive coefficient for each partition and generates historical query distribution characteristic coefficients. This coefficient differs from the traditional single indicator that only focuses on query frequency, and can more comprehensively reflect the stability of the temporal distribution of partition queries, providing a more accurate basis for judging cache priority.

[0042] Preferably, in a specific implementation of the popularity value integration component in a scenario, the historical query frequency statistics are first used as the processing object to extract the multi-time period query frequency (including the query frequency of the last 7 days and the query frequency of the last 30 days) of each target data partition to be queried. The selection of multiple time periods is to balance short-term popularity (last 7 days) and long-term popularity (last 30 days) and avoid misjudgment caused by the frequency of a single time period (such as a short-term sudden query that increases popularity). The frequency is then organized in the format of "partition identifier - frequency of the last 7 days - frequency of the last 30 days" to generate a multi-time period frequency statistics table. Then, the query time decay factor and historical query data are used to calculate the frequency. The distribution characteristic coefficient and the frequency statistics table for multiple time periods are used as common processing objects to construct a weighted fusion model: The first step is to assign weights to the frequencies for multiple time periods (60% for frequencies in the last 7 days and 40% for frequencies in the last 30 days, because recent queries better reflect current business needs), and calculate the basic frequency score (e.g., if a partition has 100 frequencies in the last 7 days and 300 frequencies in the last 30 days, the basic frequency score = 100 × 0.6 + 300 × 0.4 = 180); The second step is to multiply the basic frequency score by the query time decay factor (to decay the value of the old frequency, such as a decay factor of 0.8, then after decay...). The score is calculated as follows: 180 × 0.8 = 144. The third step involves multiplying the decayed score by the historical query distribution characteristic coefficient (to enhance the value of stable partitions; for example, if the distribution characteristic coefficient is 0.9, the initial popularity value is 144 × 0.9 = 129.6), generating an initial popularity value. Then, using this initial popularity value as the processing object, outlier correction is performed—if the initial popularity value of a partition exceeds three times the average of all partitions, it is corrected using the "average × 2" method (to avoid distortion of the coefficients due to extreme abnormal query data, such as thousands of invalid queries generated by testing in a partition), generating the corrected popularity value. The value is then processed using the corrected popularity value. It is then structured in the format of "partition identifier - time decay query popularity value" (e.g., "ORD-2024Q2-001-129.6") to ensure that the popularity value of each partition can be traced back to the three input parameters: "multi-time period frequency", "query time decay factor", and "historical query distribution characteristic coefficient". This generates a time decay query popularity value table, which can directly provide the core basis for the popularity dimension of the cache dynamic sorting component. Compared with the traditional popularity calculation based only on query frequency, it is more adaptable to the dynamic value changes of multi-source data in data weaving scenarios.

[0043] Optionally, the cache validity period calculation module is used to calculate the cache validity period of each target data partition to be queried based on the data update frequency data, and then calculate the periodic feature coefficient by mining the long-term periodic pattern. Specifically, it includes the following two components: Validity period calculation component: used to determine the average data update interval for updating the target data partition based on the target partition attribute table, and to generate the cache validity period; Periodic pattern mining component: Based on cache validity period, it identifies the potential access period of each target data partition to be queried through sliding window and Fourier transform, and then combines periodic continuity, periodic regularity, peak duration intensity and rolling prediction error to perform multi-dimensional reasoning on the potential access period to calculate periodic feature coefficients.

[0044] Preferably, the specific implementation process of the validity period calculation component is as follows: First, taking the target partition attribute table as the processing object, extract the information of the "data update history identifier" field in the table—including the "last 10 update timestamps" and "single update data volume percentage" (e.g., a certain update covers 30% of the partition data) of each target data partition to be queried, and organize it according to the format of "partition identifier - update timestamp list - single update percentage list" to generate the original update feature dataset; then, based on the original update feature dataset, calculate the initial average data update interval: for each partition, use the "last 10 update timestamps" to calculate the time difference (unit: hours) between two adjacent updates, and take the average value as the initial average interval (e.g., if the time differences are 24, 24, and 48 hours respectively, the initial average interval = (24+24+48) / 3 = 32 hours), and generate the partition initial average interval table; then, based on the "data domain" and "related data source type identifier" in the target partition attribute table, determine the update feature analysis and correction data: for the "transaction domain" and "payment domain" For high-frequency update partitions such as "archived domain" and "log domain", an "update volatility coefficient" is introduced (volatility coefficient = standard deviation / initial average interval; if the volatility coefficient > 0.3, it is judged as high volatility). The correction weight for high volatility partitions is set to 1.2 (to shorten the validity period and reduce the risk of cache expiration). For low-frequency update partitions such as "archived domain" and "log domain", a "business importance coefficient" is introduced (e.g., 1.1 for core business partitions and 0.9 for non-core partitions). The correction weight is positively correlated with business importance. A set of correction coefficients is generated. Finally, the cache validity period is calculated using the initial average interval table and the set of correction coefficients as common processing objects: validity period = initial average interval × correction coefficient × (1 - maximum single update percentage) (e.g., initial interval 32 hours, correction coefficient 1.2, maximum single update percentage 30%, then validity period = 32 × 1.2 × 0.7 = 26.88 hours). The cache validity period table is generated by organizing the data in the format of "partition identifier - cache validity period (hours)". This table is different from the traditional fixed validity period setting and can dynamically adapt to the update frequency and business characteristics of different partitions.

[0045] Preferably, in the specific technical implementation of the periodic pattern mining component, the periodic analysis window is first determined by taking the cache validity period table and the historical access logs (including "access timestamp" and "access count") of each target data partition as common processing objects: 1.5 times the cache validity period is set as the basic window length (e.g., if the cache validity period is 24 hours, the window length = 36 hours) to ensure that the window covers at least one complete cache lifecycle and avoids missing periodic features due to the window being too short; access data for the corresponding time period is extracted according to the format of "partition identifier - window length - historical access log fragment" to generate a partition-window access dataset; then, the "access timestamp - access count" sequence in the partition-window access dataset is taken as the processing object, and a sliding window (sub-window length = 1 / 3 of the window length) and Fourier transform are used to perform potential access period identification: the access sequence in the time domain is converted to the frequency domain through Fourier transform, the three frequencies with the highest power spectral density (corresponding to the most likely period) are extracted, converted to period duration (e.g., frequency 0.5 / hour corresponds to a period of 2 hours), and a potential period candidate set is generated (each partition contains 1-3 candidate periods); then the potential period is further analyzed. Multi-dimensional inference is performed on the candidate period set: "Period continuity" (number of times the period appears in 3 consecutive windows / 3, the higher the number, the more continuous), "Period regularity" (the deviation rate between the actual access time and the period prediction time, the lower the number, the more regular), "Peak duration intensity" (average duration of the peak access within the period / period duration, the higher the number, the more stable the peak), and "Rolling prediction error" (error rate of predicting the N+1th period based on the previous N periods, the lower the number, the better the predictability) are calculated to generate four-dimensional evaluation indicators. Finally, based on the four-dimensional evaluation indicators, a weighted fusion (continuity 30%, regularity 30%, peak intensity 20%, prediction error 20%) is used to calculate the comprehensive score of each candidate period. The period with the highest score is selected as the optimal period, and its comprehensive score is the period characteristic coefficient of the partition (coefficient range 0-1, the higher the number, the more significant the period characteristic). The coefficients are organized in the format of "partition identifier - optimal period (hour) - period characteristic coefficient" to generate a period characteristic coefficient table. Compared with the traditional period analysis that only relies on frequency, this table can more accurately reflect the access pattern of cross-domain data in the data weaving scenario, providing a time dimension basis for cache preloading.

[0046] Optionally, the scheduling and query execution component specifically includes the following modules: Partition Threshold Classification Module: Used to parse the partition ranking and priority score in the cache priority sorting table, generate the core information of the sorting table, and generate the partition threshold classification table based on the core information of the sorting table; High-quality partition association preloading module: used to perform association preloading based on the association rule mining results of the partition threshold classification table and historical query sequences to generate a high-quality partition cache index table; Low-priority partition cache management module: used to generate a low-priority partition data loading result table based on the partition threshold classification table and the high-priority partition cache index table; Cache optimization module: Based on the high-priority partition cache index table and the low-priority partition data loading result table, it generates a cache optimization strategy based on multi-dimensional loading performance to optimize the caching mechanism of the target data partition to be queried.

[0047] Optionally, the partition threshold classification module is specifically used to: parse the partition ranking and priority score in the cache priority sorting table, generate the core information of the sorting table, adjust the preset ratio threshold based on the core information of the sorting table and the dynamic threshold rules adapted to the business scenario, and generate the dynamically adjusted threshold; based on the dynamically adjusted threshold, divide the target data partition to be queried into a high-priority pre-loading partition and a low-priority cache management partition to generate a partition threshold classification table.

[0048] Preferably, the specific implementation process of the partition threshold classification module is as follows: First, taking the cache priority sorting table as the processing object, the core information of the sorting table is extracted: the "partition identifier", "priority ranking" and "priority score" fields in the table are parsed row by row, and the target partition attribute table (the previously generated target partition attribute table) is associated to supplement the "data domain" information of each partition (such as "order domain" and "log domain"). The data is sorted according to the fixed format of "partition identifier-priority ranking-priority score-data domain", and invalid entries with empty "priority score" are removed to generate the core information of the sorting table. Then, based on the core information of the sorting table, the preset ratio threshold is adjusted in combination with the dynamic threshold rules adapted to the business scenario: First, the current business scenario type is identified (judged by system load monitoring data, such as CPU utilization ≥70% and request queue length ≥ preset threshold, which is determined as "query peak scenario", otherwise it is "query off-peak scenario"). The preset dynamic threshold rule library is retrieved - the rule library designs differentiated adjustment strategies for different scenarios and data domains (such as in the query peak scenario, the preset ratio threshold of core data domains such as "transaction domain" and "payment domain" is reduced by 20%-30% to increase the pre-added high priority). Increase the number of partitions to improve query response speed; during off-peak query scenarios, increase the preset proportion threshold of non-core data domains such as "log domain" and "archive domain" by 10%-15% to reduce unnecessary preloading and save cache resources. Then, based on the "data domain" in the core information of the sorting table, match the corresponding rules to perform scenario-based adjustments on the initial preset proportion threshold (e.g., default 50%) to generate dynamically adjusted thresholds (e.g., during peak hours, the "transaction domain" threshold is reduced from 50% to 35%, and during off-peak hours, the "log domain" threshold is increased from 50% to 60%). Finally, use the dynamically adjusted thresholds and the core information of the sorting table... For shared processing objects, target partition classification is performed: partitions whose "priority ranking" in the core information of the sorting table is above the dynamically adjusted threshold (e.g., the top 35% of partitions when the threshold is 35%) are marked as "high-priority preloaded partitions," and those below the threshold are marked as "low-priority cache management partitions." Simultaneously, a "partition type determination basis" field is added (e.g., "high-priority preloaded partition: ranking 12 / total 34, threshold 35%), and the data is structured in the format of "partition identifier - priority ranking - priority score - data domain - partition type - determination basis" to generate a partition threshold classification table. This module differs from the traditional fixed-ratio threshold classification method. By dynamically adjusting the threshold based on business scenario load and data domain importance, it ensures rapid response of core partitions during peak query periods and avoids wasting cache resources during off-peak periods, making it more adaptable to the varied query needs in data weaving scenarios.

[0049] Optionally, the high-priority partition association preloading module is specifically used to: extract the identification information of high-priority preloaded partitions from the partition threshold classification table; then, based on the identification information of high-priority preloaded partitions, retrieve the association rule mining results of historical query sequences to generate partition association rules; and based on the partition association rules, perform distributed association preloading on high-priority preloaded partitions and associated partitions, and construct a hierarchical cache index according to the partition data structure characteristics to generate a high-priority partition cache index table.

[0050] Preferably, the specific implementation process of the high-priority partition association preloading module is as follows: First, taking the partition threshold classification table as the processing object, the high-priority preloading partition identifier extraction is performed: Entries marked as "high-priority preloading partition" in the "partition type" field of the filter table are marked as "high-priority preloading partition", and the "partition identifier" (such as "ORD-2024Q2-001" "PAY-2024Q2-002") of these entries are extracted. At the same time, the target partition attribute table is supplemented with the "data domain" (such as "order domain" "payment domain") and "association data source type identifier" (such as "MySQL-order database" "Redis-payment cache") corresponding to each identifier. The data is then organized according to the format of "high-priority partition identifier-data domain-data source type", and invalid entries with empty "data source type identifier" are removed to generate a high-priority partition identifier list. Next, based on the high-priority partition identifier list, the association rule mining results of the historical query sequence are retrieved: The historical query sequence rule base stores the "frequently queried itemsets" of each partition under the data weaving scenario (e.g., if the co-query frequency ratio of the order domain partition and the payment domain partition is ≥40%, it is determined to be a high-priority partition). The system uses a combination of criteria, including "frequent associations" and "query time-series dependency paths" (e.g., if the probability of querying the logistics domain within 30 minutes after querying the order domain partition is ≥60%, it is considered a time-series dependency). It precisely matches the "high-priority partition identifier" in the high-priority partition identifier list with the "data domain" in the association rules in the database, filtering out the "association partition identifier," "association type" (co-query / time-series dependency), and "association confidence" (e.g., 0.85) directly associated with the high-priority partition. These are then organized in the format "high-priority partition identifier - associated partition identifier - association type - association confidence" to generate partition association rules. Subsequently, based on these rules, distributed association preloading is performed on the high-priority preloaded partitions and associated partitions: first, loading is divided into shards by "data domain" (e.g., order domain and payment domain each have one shard), and each shard is allocated an independent distributed loading node (to avoid resource contention when loading different data domains and adapt to the parallel loading requirements of multi-source heterogeneous data). Then, the loading order is determined based on the "association confidence" in the partition association rules (associated partitions with an association confidence ≥0.8 are loaded synchronously with the high-priority partition, while those <0.8 are loaded synchronously).The loading process involves a 5-minute delay to reduce unnecessary real-time resource consumption. During loading, the "loading status" (success / failure), "loading duration" (e.g., 20 seconds), and "data integrity verification result" (e.g., verification passed) of each partition are recorded, generating a pre-loading execution result. Finally, using the pre-loading execution result as the processing object, a hierarchical cache index is constructed according to the partition data structure characteristics: the partition data with "loading status" of "success" and "data integrity verification result" of "pass" in the pre-loading execution result are filtered. For structured data corresponding to the "data domain" (e.g., order table, payment table), a B+ tree index is used (adapting to range query scenarios and improving the efficiency of field-based filtering). For semi-structured data (e.g., order logs, payment transaction JSON), an inverted index is used (adapting to keyword query scenarios and improving the efficiency of content-based retrieval). At the same time, the "index identifier," "index storage path," "index construction time," and the corresponding "high-priority partition identifier / related partition identifier" are recorded and structured in the format of "index identifier-partition identifier-data structure type-index type-storage path-construction time" to generate a high-priority partition cache index table. This module differs from traditional single-partition preloading methods. It achieves cross-partition collaborative preloading through association rule mining and combines it with hierarchical indexes adapted to data structures. This reduces loading latency for subsequent cross-domain queries and improves query efficiency for different data types, better meeting the efficient access needs of multi-partition, multi-structure data in data weaving scenarios.

[0051] Optionally, the low-priority partition cache management module is specifically used for: firstly, extracting the attribute information of the low-priority cache management partition from the partition threshold classification table; then, based on the attribute information of the low-priority cache management partition, initializing the local cache management partition using a weighted LRU (Least Recently Used) strategy; next, checking the existence status of the low-priority cache management partition in the local cache based on the global cache index integrated from the high-priority partition cache index table; if the target data exists, directly reading and updating the weighted LRU queue sort; if not, locating the source data partition based on the query-partition mapping table and performing incremental loading to generate a low-priority partition data loading result table.

[0052] Preferably, the specific implementation process of the low-priority partition cache management module is as follows: First, taking the partition threshold classification table as the processing object, the low-priority cache management partition attribute information extraction is performed: Entries marked as "low-priority cache management partition" in the "partition type" field of the filter table are selected, and the "partition identifier," "data domain," and "priority score" of these entries are extracted. At the same time, the "data size" (e.g., 500MB), "recent access timestamp," and "associated data source type identifier" (e.g., "HDFS log storage," "MongoDB document repository") are supplemented by associating with the target partition attribute table. The data is then organized in the format of "low-priority partition identifier-data domain-priority score-data size-recent access time-data source type," and invalid entries with empty "data size" or "data source type identifier" are removed (to avoid failure due to missing parameters during subsequent loading), generating a low-priority partition attribute list. The "data size," "data domain," and "recent access time" in this list will directly serve as the core calculation basis for the subsequent weighted LRU strategy, adapting to the management requirements of "resource occupancy control priority" for low-priority partitions.

[0053] Next, based on the list of low-priority partition attributes, a weighted LRU strategy is used to initialize the local cache management queue: First, the three-dimensional weight dimensions and allocation ratios of the weighted LRU are determined—"Data Access Value Weight" (40%, based on "Data Domain", such as "Transaction Auxiliary Domain" weight 0.6, "Log Archive Domain" weight 0.3, because auxiliary domain data still requires a certain level of access guarantee), "Cache Occupancy Cost Weight" (30%, calculated based on "Data Size", the smaller the data, the higher the weight, such as less than 50MB weight 0.9, more than 500MB weight 0.3, to avoid large-sized low-priority data occupying too much cache), and "Data Freshness Weight" (30%, calculated based on "Recent Access Time", accessed within the last 7 days). Data that has been accessed has a weight of 0.8, and data that has not been accessed for more than 30 days has a weight of 0.2, prioritizing data that may be reused in the near future. For each partition in the list of low-priority partition attributes, a weighted score is calculated based on "(data access value × 0.4) + (cache cost × 0.3) + (data freshness × 0.3)". The partitions are sorted from high to low according to their weighted scores to build a local cache management queue. The initial position of each partition in the queue is recorded (e.g., rank 15 / total 40), generating an initial weighted LRU queue. This queue differs from the traditional LRU queue that sorts only by "access time". It balances "data availability" and "cache resource consumption" through multi-dimensional weighting, adapting to the "low priority but not worthless" characteristics of low-priority partitions in data weaving scenarios.

[0054] Subsequently, using the initial weighted LRU queue and the high-priority partition cache index table as common processing objects, a local cache existence check is performed: First, the "partition identifier - index storage path - cache node IP" information is extracted from the high-priority partition cache index table and integrated into a global cache index (because low-priority partitions may have implicit data dependencies with high-priority partitions, the global index can avoid omissions in local cache checks); then, each low-priority partition in the initial weighted LRU queue is traversed, using the "low-priority partition identifier" as the query key, and the corresponding cache node and storage path are matched in the global cache index to check whether the partition data exists in the local cache node (the check logic is: determine whether there is a complete data verification file under the cache path, and whether the file modification time is later than the "most recently accessed timestamp"); if the target data exists, the data is directly read and the initial weighted LRU queue sort is updated—the "most recently accessed timestamp" of the partition is updated. The "Access Time" is updated to the current time, the weighted score is recalculated, and the queue position is adjusted (e.g., from ranking 15 to ranking 10), generating a cache hit update result. If the target data does not exist, the source data partition is located based on the query-partition mapping table (extracting "low-priority partition identifier - source data storage path - data loading method" from the table, such as "LOG-2024Q2-001-HDFS / path / logs-incremental slice loading"), and incremental loading is performed (only data slices newly added or modified in the last 30 days are loaded, rather than full loading, reducing loading bandwidth and time consumption). After loading is completed, the data is written to the local cache, and the partition is added to the end of the initial weighted LRU queue (sorted by the newly calculated weighted score), generating a cache miss loading result. The cache hit update result and the cache miss loading result are integrated to obtain the low-priority partition cache processing details.

[0055] Finally, taking the low-priority partition cache processing details as the processing object, the result table is structured and organized: extracting the "partition identifier," "loading status" (hit / miss), "loading duration" (reading time when hit, such as 200ms; incremental loading time when miss, such as 5s), "position after weighted LRU queue update," and "data integrity verification result" (if the number of data records verified after loading is consistent with the source data, it is "passed") for each low-priority partition, and arranging them in a fixed format of "low-priority partition identifier - loading status - loading duration - queue position - integrity verification - data source type" to ensure that each field can be traced back to the previous processing stage (such as "loading duration" corresponding to the actual time spent on cache check or incremental loading), generating a low-priority partition data loading result table; this table can directly provide the subsequent cache optimization module with "resource consumption - access efficiency" data support for low-priority partitions, avoiding the problem of "uncontrollable status" in traditional low-priority partition management.

[0056] Optionally, the cache optimization module is specifically used to: first extract the preloading performance index from the high-priority partition cache index table; then combine it with the low-priority partition data loading result table to generate a cache optimization strategy based on multi-dimensional loading performance data, and optimize the caching mechanism of the target data partition to be queried by dynamically adjusting the preloading trigger threshold according to the real-time query load data through the load prediction model.

[0057] Preferably, the specific implementation process of the cache optimization module is as follows: First, taking the high-priority partition cache index table as the processing object, the preloading performance index is extracted: Entries with "loading status" of "successful" are filtered from the table, and the "preloading success rate" (number of successfully preloaded partitions / total number of preloaded partitions), "index query time" (average time spent locating data through the index), and "cache hit rate" (the proportion of cache hits in query requests based on high-priority partitions) are extracted. At the same time, the "preloaded partition data domain" (such as "order domain" and "payment domain") is supplemented by associating with the partition threshold classification table. The data domains are organized in the format of "data domain - preloading success rate - index query time - cache hit rate". Abnormal data domain entries with a "preloading success rate" of less than 50% are removed (to avoid extreme data interfering with strategy formulation), generating a high-priority partition preloading performance index set. This index set focuses on the core dimensions of "preloading effectiveness" and "query efficiency" of high-priority partitions, providing a quantitative basis for subsequent optimization strategies on the high-priority partition side, which is different from the traditional optimization method that only focuses on "whether preloading occurs" and ignores performance feedback.

[0058] Next, using the high-priority partition preloading performance indicator set and the low-priority partition data loading result table as common processing objects, multi-dimensional loading performance data is integrated: "loading latency" (average loading time), "cache utilization rate" (low-priority partition cache utilization / total cache capacity), and "data reuse rate" (the proportion of low-priority partitions repeatedly accessed within 7 days) are extracted from the low-priority partition data loading result table and organized in the format of "data domain-loading latency-cache utilization rate-data reuse rate" to generate the low-priority partition loading performance indicator set; then, the high-priority partition preloading performance indicator set and the low-priority partition loading performance indicator set are associated and integrated according to "data domain" to supplement "cross-domain association performance" (such as the response time of collaborative queries between the high-priority order domain and the low-priority log domain), forming a multi-dimensional loading performance dataset; this dataset covers high-priority / low-priority partitions and single-domain / cross-domain scenarios, avoiding the global performance imbalance caused by traditional optimization focusing only on a single partition type, and is more suitable for the characteristics of multi-partition collaboration in data weaving scenarios.

[0059] Then, based on the multi-dimensional loading performance dataset, a cache optimization strategy based on the multi-dimensional loading performance data is generated: For data domains with "low preloading success rate (<70%) and high index query time (>500ms)" (such as the high-priority payment domain), a "preloading sharding adjustment strategy" is formulated (the original single shard loading is split into 2-3 small shards for parallel loading, reducing the impact of single shard loading failure on the overall performance, while optimizing the index structure to reduce query time); For data domains with "high low-priority partition loading latency (>3s) and high data reuse rate (>60%)" (such as the low-priority user profile domain), Develop a "low-optimal cache expansion strategy" (while keeping the total cache capacity unchanged, release 10%-15% of the cache space from low-optimal partitions with a reuse rate of <30% and allocate it to high-reuse-rate partitions); for data domain combinations with "poor cross-domain association performance (collaborative query time >1s)" (such as high-optimal order domain and low-optimal logistics domain), develop a "cross-domain preloading linkage strategy" (when the high-optimal order domain is preloaded, the lightweight preloading of the core data of the low-optimal logistics domain is triggered simultaneously); organize these strategies in the format of "strategy ID-applicable scenario-optimization goal-execution steps" to generate a cache optimization strategy list.

[0060] Subsequently, using the cache optimization strategy list and real-time query load data as common processing objects, a load prediction model was introduced to dynamically adjust the preloading trigger threshold: First, real-time query load data, including "CPU utilization," "memory usage," "request queue length," and "cache IO throughput," was collected and organized in the format of "timestamp-CPU utilization-memory usage-request queue length-IO throughput" to generate a real-time load dataset; then, the real-time load dataset was input into the load prediction model (the model is trained based on the historical load data of the past 30 days, and uses a combination of sliding window statistics and exponential smoothing algorithms to predict the load change trend within the next 30 minutes, such as predicting that CPU utilization will increase from...). If the current load is increased from 60% to 80%, a load prediction result is generated. Then, the adjustment rules in the cache optimization strategy list are matched according to the load prediction result: if the load peak is predicted (CPU utilization ≥ 75% or request queue length ≥ preset threshold 100), the preload trigger threshold is lowered by 10%-15% (e.g., from 35% to 25%) to reduce the number of high-priority partitions preloaded to reduce system load; if the load trough is predicted (CPU utilization ≤ 40% and request queue length ≤ 30), the preload trigger threshold is increased by 5%-10% (e.g., from 35% to 40%) to increase preload coverage to improve subsequent query efficiency; if the load is predicted to be stable, the threshold is kept unchanged.

[0061] Finally, using the threshold adjustment results and cache optimization strategy list as processing objects, the optimization implementation and recording are carried out: the adjusted preload trigger threshold is synchronized to the partition threshold classification module (for subsequent partition classification), and strategies such as "preload shard adjustment" and "low-priority cache expansion" are synchronized to the high-priority partition associated preload module and the low-priority partition cache management module; at the same time, the "threshold before adjustment - threshold after adjustment - load prediction basis - optimization strategy content - expected performance target" (e.g., "35% before adjustment - 25% after adjustment - predicted CPU to rise to 80% - execution of preload shard adjustment - expected preload success rate to increase to 80%+") are recorded and organized in the format of "adjustment time - adjustment content - basis - expected target" to generate a cache optimization execution report; this report realizes full-link traceability of the optimization process, and through load prediction to drive dynamic threshold adjustment, it avoids the problem of "either overload or resource waste" of traditional static thresholds when the load fluctuates, ensuring that the caching mechanism always maintains high efficiency under the variable load of data weaving scenarios.

[0062] Figure 2 This is a flowchart illustrating an efficient query acceleration and caching optimization method for data weaving, according to an embodiment of this application. Figure 2 As shown, it includes: Based on multi-source heterogeneous semantic metadata, the multi-source heterogeneous dataset is dynamically partitioned, and cross-domain association identifiers of data weaving are embedded to generate a semantic partition index table. The query intent is extracted from the user's query request to generate a query intent representation. The mapping relationship between the query intent representation and the target data partition to be queried is established based on the semantic partition index table to generate a query-partition mapping table. Generate a cache priority sorting table based on the query-partition mapping table and historical query frequency statistics; Based on the cache priority sorting table and preset ratio threshold, the caching mechanism of the target data partition to be queried is optimized.

[0063] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A high-efficiency query acceleration and caching optimization device for data weaving, characterized in that, include: Semantic partitioning component: used to dynamically partition multi-source heterogeneous datasets based on multi-source heterogeneous semantic metadata, while embedding cross-domain association identifiers of data weaving to generate a semantic partitioning index table; Query intent mapping component: used to extract intent from user-initiated query requests to generate query intent representations, and to establish a mapping relationship between the query intent representations and the target data partitions to be queried based on the semantic partition index table, so as to generate a query-partition mapping table. Cache dynamic sorting component: used to generate a cache priority sorting table based on the query-partition mapping table and historical query frequency statistics; Scheduling and query execution component: used to optimize the caching mechanism of the target data partition to be queried based on the cache priority sorting table and preset ratio threshold; The semantic partitioning component is used to dynamically partition multi-source heterogeneous datasets based on multi-source heterogeneous semantic metadata and embed cross-domain association identifiers to generate a semantic partitioning index table. Specifically, it includes the following two modules: Parsing Enhancement Module: Used to perform structured parsing of multi-source heterogeneous semantic metadata to extract the basic dimensions of data domain affiliation and data type, generate basic semantic dimensions to complete the hidden relationships between different data sources through cross-domain data association rules, and generate enhanced semantic metadata; Partitioning and Identifier Embedding Module: Used to determine the partitioning granularity based on enhanced semantic metadata, to perform dynamic partitioning of multi-source heterogeneous datasets to determine partitioning mapping relationships, and then embed cross-domain association identifiers woven from data into the partitioning mapping relationships to generate a semantic partitioning index table; The query intent mapping component is used to extract the user's query request intent and establish a mapping relationship between the target data partitions to be queried, in order to generate a query-partition mapping table. Specifically, it includes the following two modules: Intent extraction module: It is used to perform statement-by-state grammatical parsing on the query request initiated by the user to extract the data domains and filtering conditions involved in the query, and generate grammatical parsing results accordingly. Based on the grammatical parsing results, it integrates the user's historical query context to supplement implicit needs, so as to generate a structured query intent representation. Partition Mapping Module: Used to extract cross-domain association identifiers and partition attributes from the semantic partition index table, and then establish a structured query intent representation of the matching relationship of the target data partition to be queried, so as to generate a query-partition mapping table.

2. The apparatus according to claim 1, characterized in that, The scheduling and query execution component is specifically used to: optimize the caching mechanism of the target data partition to be queried by performing association preloading based on the cache priority sorting table and preset ratio threshold, and according to the association rule mining results of historical query sequences.

3. The apparatus according to claim 1, characterized in that, The parsing enhancement module is used to perform structured parsing of multi-source heterogeneous semantic metadata to extract basic dimensions such as data domain affiliation and data type, and generate basic semantic dimensions to complete the hidden relationships between different data sources through cross-domain data association rule mining, thereby generating enhanced semantic metadata. Specifically, it includes the following three components: Basic Dimension Extraction Component: Used to perform structured parsing of multi-source heterogeneous semantic metadata to extract information on data domain affiliation, data type, and field meaning descriptions to generate basic semantic dimensions; Cross-domain association completion component: Based on the basic semantic dimension set, it performs association rule mining on multi-source heterogeneous semantic metadata through cross-domain data association rules to identify hidden association relationships between semantic metadata from different data sources; Enhanced metadata integration component: Used to align fields between basic semantic dimensions and hidden relationships to generate enhanced semantic metadata.

4. The apparatus according to claim 1, characterized in that, The partitioning and identifier embedding module is used to determine the partitioning granularity based on enhanced semantic metadata, so as to perform dynamic partitioning of multi-source heterogeneous datasets to determine the partitioning mapping relationship. Then, cross-domain association identifiers woven from data are embedded in the partitioning mapping relationship to generate a semantic partitioning index table. Specifically, it includes the following three components: Partition granularity rule generation component: used to determine the density of data domain affiliation and association based on the enhanced semantic meta dataset, to calculate the optimal partition granularity threshold for different data domains, and to formulate partitioning rules according to the optimal partition granularity threshold to generate a dynamic partitioning rule table; Multi-source data partitioning execution component: Used to perform distributed dynamic partitioning processing on multi-source heterogeneous datasets based on a dynamic partitioning rule table, so as to allocate data of different data domains and different correlation strengths to corresponding partitions and generate partition mapping relationships; Cross-domain identifier embedding component: used to parse the cross-domain association requirement expression of data weaving to generate a unique cross-domain association identifier, and embed the unique cross-domain association identifier into the partition mapping relationship to generate a semantic partition index table.

5. The apparatus according to claim 1, characterized in that, The intent extraction module performs statement-by-statement syntactic parsing on user-initiated query requests to extract the data domains and filtering conditions involved in the query, and generates syntactic parsing results accordingly. Based on the syntactic parsing results, it integrates the user's historical query context to supplement implicit requirements, thereby generating a structured query intent representation. Specifically, it includes the following three components: Syntax parsing component: Used to perform statement-by-state syntax parsing on user-initiated query requests to identify data entities, operational logic, data fields, and filtering conditions in the query, and organize them into key-value pair structures according to a preset format to generate query syntax parsing results; Query Context Fusion Component: This component extracts data domain preferences and filtering logic related to the current query from the user's historical query records and integrates them into the query syntax parsing results to generate context-enhanced intent fragments. Structured Intent Component: Used to integrate context-enhanced intent fragments by "data domain - filtering conditions - aggregation requirements - related data identifiers" to generate a structured query intent representation.

6. The apparatus according to claim 1, characterized in that, Partition Mapping Module: This module extracts cross-domain association identifiers and partition attributes from the semantic partition index table, then establishes a structured query intent representation of the matching relationship between the target data partitions to be queried, in order to generate a query-partition mapping table. It specifically includes the following three components: Mapping rule generation component: Used to extract partition attributes from semantic partition index table to obtain the data domain range, cross-domain association identifier and query adaptation conditions of each target data partition to be queried, and formulate rules in the format of "intent element-partition attribute-matching weight" to generate intent-partition mapping rules; Mapping Relationship Matching Component: Based on the structured query intent representation, it calculates the matching degree between each intent element and the partition attribute by referring to the intent-partition mapping rule set, and filters the target data partitions whose matching degree meets the preset threshold to generate a mapping relationship table; Mapping relationship optimization component: used to remove redundancy and prioritize mapping entries in the mapping relationship table, remove duplicate partition mapping entries, sort by matching degree from high to low and mark the mapping basis to generate a query-partition mapping table.

7. The apparatus according to claim 1, characterized in that, The cache dynamic sorting component is specifically used to: determine the target data partition to be queried based on the query-partition mapping table; calculate the time decay query heat value of each target data partition to be queried by combining historical query frequency statistics and query time decay factor; calculate the cache validity period of each target data partition to be queried by combining data update frequency data; identify the access cycle characteristics of each target data partition to be queried by mining long-term cycle patterns and generate cycle feature coefficients; and generate cache priority sorting by integrating time decay query heat value, cache validity period and cycle feature coefficients.

8. The apparatus according to claim 7, characterized in that, The cache dynamic sorting component specifically includes the following four modules: Target data partition determination module: used to perform structured parsing of the query-partition mapping table to determine the target data partition to be queried and then generate the target partition attribute table; Time decay heat calculation module: Based on the target partition attribute table and combined with historical query frequency statistics, it calculates the query time decay factor and historical query distribution characteristic coefficient for each target data partition to be queried, so as to generate a time decay query heat value. Cache validity period calculation module: used to calculate the cache validity period of each target data partition to be queried based on the data update frequency data, and then calculate the periodic feature coefficient by mining the long-term periodic pattern. Multi-dimensional priority fusion module: It is used to perform weighted fusion of time decay query popularity value, cache validity period and periodic feature coefficient to calculate priority score, sort each target data partition to be queried in descending order of priority score and mark the score composition details to generate cache priority ranking.

Images