Cloud information retrieval acceleration system

By quantifying the uncertainty of data flow and dynamically adjusting index configuration and optimizing query routing, the cloud information retrieval system in the existing technology is difficult to cope with data volume changes and query requirements, and improves query efficiency and system performance.

CN118981469AInactive Publication Date: 2024-11-19DALIAN BINGXI HUILI NETWORK TECH CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411419960.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-12
Publication Date
2024-11-19
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing cloud information retrieval acceleration system lacks an adaptive adjustment mechanism, making it difficult to cope with changes in data volume and rapid changes in query requirements, resulting in query delays and performance limitations.

Method used

The data flow entropy calculation module is used to quantify the uncertainty of the data flow, generate information entropy eigenvalues, dynamically adjust index configuration and query routing, and process composite queries through inverted indexes and multi-level B-tree indexes to optimize query routing selection.

Benefits of technology

It significantly improves the efficiency and accuracy of query processing, enhances the adaptability and flexibility of the system, and reduces the response time and concurrent processing capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118981469A_ABST
    Figure CN118981469A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of data retrieval, in particular to a cloud information retrieval acceleration system which comprises a data stream entropy calculation module, a data stream entropy calculation module, a data stream entropy calculation module, a data stream entropy calculation module, a data stream entropy calculation module, a data stream entropy calculation module and a data stream entropy calculation module. According to the method, the efficiency and accuracy of query processing are remarkably optimized by quantizing the uncertainty of the data flow and distributing the index path according to the uncertainty, the introduction of the information entropy is used for measuring the complexity of the data flow, and an innovative method is provided for optimizing the index structure, so that the system can accurately process various complex queries; index configuration is dynamically adjusted to cope with changes of data and query modes, adaptability and flexibility of the system are remarkably improved, in addition, response time is further shortened according to a strategy of intelligently selecting processing nodes according to system loads and data access modes, and concurrent processing capacity is enhanced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data retrieval technology, and in particular to a cloud information retrieval acceleration system. Background Art

[0002] The field of data retrieval technology involves the construction, management, or processing of data structures, with a particular emphasis on improving the efficiency and accuracy of retrieving specific information from large-scale data sets. This field includes a variety of technologies and methods, such as indexing technology, search algorithm optimization, data classification and clustering, and intelligent query processing. In modern applications, data retrieval technology continues to integrate machine learning and artificial intelligence to improve performance in processing complex queries and big data environments. In addition, with the rapid increase in data volume, efficient data retrieval systems have become particularly important for achieving fast and accurate information access, especially in areas such as Internet search, enterprise information systems, and scientific research data analysis.

[0003] Among them, cloud-based information retrieval acceleration systems refer to systems that use the resources of cloud computing platforms to optimize and accelerate information retrieval. The main uses of such systems include improving the response time of data queries and the ability to handle a large number of concurrent requests. By being deployed on cloud platforms, these systems are able to dynamically scale resources to cope with demand peaks while leveraging the high availability and elasticity of cloud computing to ensure the stability and reliability of data retrieval services. This is particularly important in applications such as business intelligence, online analytical processing (OLAP), and big data analysis that require processing large-scale data sets and complex queries.

[0004] Existing technologies often fail to effectively cope with the rapid changes in data and query requirements, and lack an adaptive adjustment mechanism, which makes it difficult to meet the requirements of efficient processing in a big data environment. For example, when the amount of data increases dramatically or the data type changes, a fixed index structure may cause query delays or fail to effectively support new query types. This static processing method limits system performance and user experience in applications that require fast data analysis and decision support. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and propose a cloud information retrieval acceleration system.

[0006] In order to achieve the above purpose, the present invention adopts the following technical solution: The cloud information retrieval acceleration system includes: The data stream entropy calculation module receives the data stream, records the value and frequency of each data packet, performs information entropy value calculation on the recorded data, quantifies the uncertainty of the data stream, and generates information entropy feature values; The data pre-classification index module allocates data streams to corresponding index paths based on the information entropy feature value and the quantized level of data complexity, applies inverted index to process compound queries, and processes basic queries through multi-level B-tree index to generate index allocation records; The index dynamic adjustment module monitors the data update frequency and the changes in the query mode, and adjusts the index configuration in combination with the index allocation record. The adjustment actions include data category migration and index type change, matching the evolution of query requirements, and generating index adjustment logs. The query route optimization module analyzes the instant query request, selects the data node for processing the query according to the data access mode and the current load of the system, adjusts the query path using the index adjustment log, and generates an optimized query route.

[0007] As a further solution of the present invention, the step of obtaining the information entropy characteristic value is specifically as follows: Receive the data stream and count each packet type, record the number of times the packet occurs, using the formula: ; Calculate each packet In the time window The weighted cumulative occurrence times in generate the weighted occurrence frequency of each data packet; in, Indicates the target packet type The weighted cumulative occurrence of It is a data packet The weight of Indicates at time Data Pack Whether the indicator function appears, is the monitoring time window; Sum up the occurrence frequencies of all the packet types and calculate the total occurrence frequency using the formula: ; Sum up the cumulative occurrence frequencies of all types of data packets to obtain the total data packet occurrence frequency; in, is the sum of the occurrence frequencies of all packet types, is the number of data packet types; Based on the weighted occurrence frequency of the data packets and the total occurrence frequency of the data packets, the uncertainty of the data stream is evaluated using the formula: , standardize the information entropy value to compare the differentiated data streams and obtain the information entropy feature value of the data packet; in, Represents the number of differentiated packet types, It is a logarithmic function with base 2. H represents the information entropy of the data stream and quantifies the uncertainty of the data stream.

[0008] As a further solution of the present invention, the step of obtaining the index allocation record is specifically as follows: Based on the information entropy eigenvalue, the data complexity is calculated using the formula: , generate data complexity; in, Adjust the secondary effect of information entropy, Adjust the square root effect of information entropy, Adjust the logarithmic effect of information entropy, 1 represents the data complexity after calculation; According to the data complexity, the data stream is allocated to the corresponding index path using the formula: , generate index path; in, is the interval value of the complexity partition, is the index path assigned according to complexity; Apply the index path and use the inverted index to process compound queries. Basic queries are processed through multi-level B-tree indexes using the formula: , generate index allocation records; in, 1 represents the query type, Represents an index allocation record.

[0009] As a further solution of the present invention, the steps of obtaining the index adjustment log are specifically as follows: Monitor data update frequency and query mode changes using the formula: ; Calculate the adjusted data update frequency and generate a new update frequency; in, represents the weight factor, Represents the previous update frequency, Represents the current update frequency, Represents the new update frequency; In combination with the index allocation record and the new update frequency, the formula is adopted: , calculate and generate the index adjustment threshold; in, Represents the index to assign record weights, represents the update frequency weight, Represents the index adjustment threshold; According to the index adjustment threshold, data category migration and index type change are performed using the formula: , generate index adjustment log; in, Represents the query mode, Represents adjustment action, Represents the index adjustment log, Represents the index adjustment threshold, Is a custom log function, Represents the sine function.

[0010] As a further solution of the present invention, the step of acquiring the data node is specifically as follows: Analyze ad hoc query requests and data access patterns using the formula: , calculate and generate weighted query frequency ; in, is the total number of query requests, is the weight factor, = is the frequency of query requests, is the regularization parameter; Based on the weighted query frequency and the current load data of the system, the formula is adopted: , select the data node with the best load ratio and generate the selected data node ; in, is the total number of data nodes, Is a node The total capacity of Is a node The current usage rate, is a small constant to prevent division by zero; Using the data node , according to the distribution of query types, the formula is used: ; Determine the data nodes that process the query and generate the selected data nodes ; in, is the category distribution of query requests, is the adjustment factor, is the entropy of the query distribution.

[0011] As a further solution of the present invention, the step of obtaining the optimized query route is specifically as follows: Analyze the current query configuration using the formula: ; Express the impact analysis of the value in logarithmic form, calculate the configuration weight of the current query path, and generate the weight of the current query path ; in, Indicates multiple records in the index adjustment log. is the impact factor of each record; According to the current query path weight and system dynamic load data, the formula is adopted: ; Adjust the query path to match the system load changes and generate the adjusted query path ; in, is the attenuation coefficient, It is the dynamic load index of the system; Combining the adjusted query path and current query requirements, the formula is adopted: ; Optimize query routing to match new query requirements , generate optimized query routing ; in, is the coefficient of variation, Indicates the variability of new query requirements.

[0012] Compared with the prior art, the advantages and positive effects of the present invention are: In the present invention, by quantifying the uncertainty of the data stream and allocating the index path accordingly, the efficiency and accuracy of query processing are significantly optimized. The introduction of information entropy as a measure of the complexity of the data stream provides an innovative method to optimize the index structure, enabling the system to accurately handle various complex queries. Dynamic adjustment of the index configuration to cope with changes in data and query patterns significantly improves the adaptability and flexibility of the system. In addition, the strategy of intelligently selecting processing nodes based on system load and data access patterns further reduces response time and enhances concurrent processing capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a system flow chart of the present invention; Figure 2 This is a flow chart of the steps for obtaining the information entropy characteristic value of the present invention; Figure 3 A flow chart of the steps for obtaining the index allocation record of the present invention; Figure 4 A flow chart of the steps for obtaining the index adjustment log of the present invention; Figure 5 A flowchart of the steps for obtaining a data node of the present invention; Figure 6 A flow chart of the steps for obtaining the optimized query routing of the present invention. DETAILED DESCRIPTION

[0014] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0015] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, in the description of the present invention, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0016] Example 1: Please refer to Figure 1 ,The cloud information retrieval acceleration system includes: The data stream entropy calculation module receives the data stream, records the value and frequency of each data packet, performs information entropy value calculation on the recorded data, quantifies the uncertainty of the data stream, and generates information entropy feature values; The data pre-classification index module allocates data streams to corresponding index paths based on information entropy feature values ​​and the quantitative level of data complexity, applies inverted indexes to handle compound queries, and processes basic queries through multi-level B-tree indexes to generate index allocation records. The index dynamic adjustment module monitors the data update frequency and query mode changes, and adjusts the index configuration in combination with the index allocation record. The adjustment actions include data category migration and index type change, matching the evolution of query requirements, and generating index adjustment logs. The query routing optimization module analyzes instant query requests, selects data nodes to process queries based on data access patterns and the current system load, adjusts query paths using index adjustment logs, and generates optimized query routes.

[0017] Information entropy feature values ​​include frequency statistics, uncertainty metrics, and value distribution indicators. Index allocation records include index path types, query complexity classifications, and data flow target indexes. Index adjustment logs include data migration records, index update types, and query adaptability records. Optimized query routing includes selected data nodes, path adjustment strategies, and current load analysis results.

[0018] See also Figure 2 , the specific steps for obtaining the information entropy eigenvalue are: Receive the data stream and count each packet type, record the number of times the packet occurs, using the formula: ; Calculate each packet In the time window The weighted cumulative occurrence times in generate the weighted occurrence frequency of each data packet; in, Indicates the target packet type The weighted cumulative occurrence of It is a data packet The weight of Indicates at time Data Pack Whether the indicator function appears, is the monitoring time window; Sum up the occurrence frequencies of all packet types and calculate the total occurrence frequency using the formula: ; Sum up the cumulative occurrence frequencies of all types of data packets to obtain the total data packet occurrence frequency; in, is the sum of the occurrence frequencies of all packet types, is the number of data packet types; Based on the weighted occurrence frequency of data packets and the total occurrence frequency of data packets, the uncertainty of the data flow is evaluated using the formula: ; Normalize the information entropy value to compare the differentiated data streams and obtain the information entropy feature value of the data packet; in, Represents the number of differentiated packet types, It is a logarithmic function with base 2, H represents the information entropy of the data stream, and quantifies the uncertainty of the data stream; Parameter details: : Target packet type The weighted cumulative occurrences of ; : Data Packet The weight of each data packet is used to adjust the influence of each data packet in the total frequency; : indicator function, at time Data Pack Whether it appears, if it appears, it is 1, otherwise it is 0; : Monitoring time window; Assumptions: The monitoring time window is 10 seconds. ; Packet Type There are 2 types (e.g., A and B); , weight ; The reception of data packets A and B within the time window is: ; Calculation process: Calculate the weighted cumulative occurrence count of packet A : ; Calculate the weighted cumulative occurrence count of packet B : ; Generate result: get the weighted occurrence frequency of data packet A , the weighted occurrence frequency of data packet B: .

[0019] formula: ; Parameter details: : The total occurrence frequency of all packet types; : Target packet type The weighted cumulative frequency of occurrence; : The number of data packet types; Assumptions: From step 1, packet A , data packet B ; Number of data packet types ; Calculation process: Summarize the weighted occurrence frequencies of all packet types: ; Generate results: Get the total packet frequency ; formula: ; Parameter details: : Information entropy of data stream; : Target packet type The weighted frequency of occurrence of : The total occurrence frequency of all packet types; : The number of data packet types; : Logarithmic function with base 2; Assumptions: From step 1 and step 2, packet A , data packet B , total frequency ; Number of data packet types ; Calculation process: Calculate the probability of each packet :

[0020] Calculating Information Entropy :

[0021] Generate results: Get the information entropy characteristic value of the data packet .

[0022] See also Figure 3 , the specific steps for obtaining index allocation records are: Based on the information entropy eigenvalue, the data complexity is calculated using the formula: , generate data complexity; in, Adjust the secondary effect of information entropy, Adjust the square root effect of information entropy, Adjust the logarithmic effect of information entropy, 1 represents the data complexity after calculation; According to the data complexity, the data flow is allocated to the corresponding index path using the formula: , generate index path; in, is the interval value of the complexity partition, is the index path assigned according to complexity; Apply the index path and use the inverted index to handle compound queries. Basic queries are handled through multi-level B-tree indexes using the formula: , generate index allocation records; Among them, Q1 represents the query type, and R represents the index allocation record; formula: ; Parameter interpretation and assumptions: Information entropy eigenvalue : represents the uncertainty of the data stream. Assume (Typical information entropy ranges from 0 to several digits, depending on the diversity of the data); Adjust the quadratic influence coefficient of information entropy : Assuming it is 0.5, it enhances the influence of high entropy value; Adjust the influence coefficient of the square root of information entropy : Assumed to be 1.5, indicating basic adjustment; Adjust the logarithmic influence coefficient of information entropy : Assumed to be 0.2, used to adjust the impact of small changes; Calculation process: calculate ; calculate ; calculate ; Substitute into the formula to calculate :

[0023] Result analysis: Calculated data complexity It represents the complexity evaluation value of data under the condition of given information entropy; formula: ; Parameter interpretation and assumptions: Data complexity : ; Interval value of complexity partition : Assuming 0.5, small intervals can provide index distribution; Calculation process: calculate ; calculate ; Result analysis: Index Path Indicates the index path number corresponding to the data complexity D2. The number will guide how to index the data so that the query can be processed quickly and efficiently; formula: ; Parameter interpretation and assumptions: Index path number : ; Query Type Q1: Hypothesis Represents a basic query type; Calculation process and result analysis: Simulate application index path To process the query type Q1, generate the index allocation record R. The function IndexApply depends on the implementation details of the system and is not simulated in detail here.

[0024] See also Figure 4 , the specific steps for obtaining index adjustment logs are: Monitor data update frequency and query mode changes using the formula: , calculate the adjusted data update frequency and generate a new update frequency; in, represents the weight factor, Represents the previous update frequency, Represents the current update frequency, Represents the new update frequency; Combine the index allocation record and the new update frequency, using the formula: , calculate and generate the index adjustment threshold; in, Represents the index to assign record weights, represents the update frequency weight, Represents the index adjustment threshold; According to the index adjustment threshold, perform data category migration and index type change, using the formula: , generate index adjustment log; Among them, Q2 represents the query mode, Action represents the adjustment action, Represents the index adjustment log, Represents the index adjustment threshold, Is a custom log function, Represents the sine function.

[0025] formula: ; Parameter details: : Weighting factor used to balance the impact of old frequency and current frequency, assuming , which means that the old frequencies have higher weights; : The previous update frequency is assumed to be 10 times / hour; : The current update frequency is assumed to be 16 times / hour; Calculation example:

[0026] New update frequency Explanation: Based on the given weight and the previous frequency, and with reference to the change in the current frequency, the new frequency is calculated to be approximately 38.5 times per hour; formula: ; Parameter details: : The weight of the index assigned to the record, assuming ; : The weight of the new update frequency, assuming ; : Index allocation record, assuming it is 5, indicating that there are 5 key index events; Calculation example:

[0027] This adjustment threshold Indicates how urgently the index needs to be adjusted; formula: ; Parameter details: : Adjust the threshold; Q2: Changes in query patterns, assuming Indicates that the query has become more complex; :Adjust the action, assuming Indicates data migration; Calculation example: ; ; Represents the index adjustment operations recorded based on adjustment thresholds, query complexity increases, and migration actions.

[0028] See also Figure 5 , the specific steps for obtaining data nodes are: Analyze ad hoc query requests and data access patterns using the formula: , calculate and generate weighted query frequency ; in, is the total number of query requests, is the weight factor, is the frequency of query requests, is the regularization parameter; Based on the weighted query frequency and the current load data of the system, the formula is used: , select the data node with the best load ratio and generate the selected data node ; in, is the total number of data nodes, Is a node The total capacity of Is a node The current usage rate, is a small constant to prevent division by zero; Leveraging Data Nodes , according to the distribution of query types, the formula is used: ; Determine the data nodes that process the query and generate the selected data nodes ; in, is the category distribution of query requests, is the adjustment factor, is the entropy of the query distribution; formula: ; in: : The total number of query requests; : No. The weight of a query request; Freq : No. The frequency of query requests; : Regularization parameter to balance the influence of the weight vector; Derivation and examples: Assume that there are three types of query requests in the system, and the frequencies and weights of multiple query requests are as follows: Frequency = times / hour, weight = ; Frequency = Second-rate Hours, weight = ; Frequency = Second-rate When, weight = ; Regularization parameter Set to 0.1 (a common small regularization coefficient, used for balanced calculations) Calculation process: Compute the weighted frequency sum: ; Compute the sum of the squares of the weight vector: ; Compute the regularization term: ; calculate ;

[0029] Result analysis: Calculated It represents the average query request density after reference weight adjustment, reflecting the weighted activity of query requests; formula: ; in: : The total number of data nodes; :node Total capacity; node Current usage rate; : A small constant to prevent zero division, set to 0.01; Derivation and examples: Assume there are 2 data nodes, and the capacity and utilization are as follows: Node 1: Total capacity , current usage ; Node 2: Total capacity , current usage ; Calculation process: For Node 1: ; For Node 2: ; Result analysis: Calculated For node 2, since the load ratio is the lowest (1.5), it means that node 2 has the highest remaining capacity ratio relative to its capacity; formula: ; in: Optimal node; Category distribution of query requests; : Adjustment coefficient to enhance the effect of entropy value, set to 1.5; : The entropy of query distribution is an indicator to measure the diversity of query distribution; Derivation and examples: Assuming query type distribution As follows, and calculate its entropy value: Type A: ; Type B: ; Type C: ; The entropy calculation formula is: ; calculate: Entropy calculation: ; Adjusted entropy value: ; Execution process: according to and the adjusted entropy value 2.22825, the system determines the data node through an internal decision function AllocateNode ,The function makes the decision by referring to the matching degree of the node’s ,technical specifications and the query type distribution; Result analysis: By using the AllocateNode function, the system selects a data node that can handle the current query distribution. It is the data node that best matches the current query requirements, which means that the node has the optimal configuration and resources to handle multiple types of queries.

[0030] See also Figure 6 , the specific steps for obtaining the optimized query route are: Analyze the current query configuration using the formula: ; Express the impact analysis of the value in logarithmic form, calculate the configuration weight of the current query path, and generate the weight of the current query path ; in, Indicates multiple records in the index adjustment log. is the impact factor of each record; According to the current query path weight and system dynamic load data, the formula is used: ; Adjust the query path to match the system load changes and generate the adjusted query path ; in, is the attenuation coefficient, It is the dynamic load index of the system; Combined with the adjusted query path and current query requirements, the formula is adopted: ; Optimize query routing to match new query requirements , generate optimized query routing ; in, is the coefficient of variation, Indicates the variability of new query requirements; Detailed explanation and derivation process: Assume the following parameters and data: : Total number of log records; : The values ​​of the log items are ; : Impact factor, set to ; formula: ; Parameter Description: : The item value of the log record, including the number of visits or weight; : The impact factor of each log record item, adjusting the contribution of each record to the final value; Calculation process and examples: Apply a logarithmic transformation to each log record to enhance the impact of small values: ; ; ; Apply the weights to compute the weighted sum: ; ; ; calculate : ; Result description: Indicates the impact of log records. A higher value indicates a more critical path configuration. formula: ; Parameter Description: : Attenuation coefficient, the degree of influence on the system load; : Dynamic load factor, assuming the system load is moderate; Calculation process and examples: Calculate the load adjustment factor: ; Adjust query path weights: ; Result description: It indicates the weight of the adjusted path after referring to the system load. A smaller value means that the impact of the system load is appropriately reduced. formula: ; Parameter Description: : Coefficient of variation, which affects the optimization adjustment due to the diversity of new query requirements; : The variability of new query requirements, indicating the uneven distribution of requirements; Calculation process and examples: Calculate the impact of query demand: ; Optimize query routing: ; Result description: =2.73 indicates that the query routing can better match the diversity of current query requirements after optimization.

[0031] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. Cloud information retrieval acceleration system, characterized in that: The system comprises: The data stream entropy calculation module receives the data stream, records the value and frequency of each data packet, performs information entropy value calculation on the recorded data, quantifies the uncertainty of the data stream, and generates information entropy feature values; The steps for obtaining the information entropy characteristic value are specifically as follows: Receive the data stream and count each packet type, record the number of times the packet occurs, using the formula: ; Calculate the weighted cumulative number of occurrences of each data packet i within the time window T to generate the weighted occurrence frequency of each data packet; in, represents the weighted cumulative occurrence count of target packet type i, is the weight of packet i, An indicator function indicating whether packet i appears at time t, where T is the monitoring time window; Sum up the occurrence frequencies of all the packet types and calculate the total occurrence frequency using the formula: ; Sum up the cumulative occurrence frequencies of all types of data packets to obtain the total data packet occurrence frequency; Where N is the sum of the occurrence frequencies of all packet types, and n1 is the number of packet types; Based on the weighted occurrence frequency of the data packets and the total occurrence frequency of the data packets, the uncertainty of the data stream is evaluated using the formula: ; Normalize the information entropy value to compare the differentiated data streams and obtain the information entropy feature value of the data packet; Where n2 represents the number of differentiated packet types, It is a logarithmic function with base 2, H represents the information entropy of the data stream, and quantifies the uncertainty of the data stream; The data pre-classification index module allocates data streams to corresponding index paths based on the information entropy feature value and the quantized level of data complexity, applies inverted index to process compound queries, and processes basic queries through multi-level B-tree index to generate index allocation records; The index dynamic adjustment module monitors the data update frequency and the changes in the query mode, and adjusts the index configuration in combination with the index allocation record. The adjustment actions include data category migration and index type change, matching the evolution of query requirements, and generating index adjustment logs. The query route optimization module analyzes the instant query request, selects the data node for processing the query according to the data access mode and the current load of the system, adjusts the query path using the index adjustment log, and generates an optimized query route.

2. The cloud information retrieval acceleration system according to claim 1, characterized in that: The steps for obtaining the index allocation record are specifically as follows: Based on the information entropy eigenvalue, the data complexity is calculated using the formula: , generate data complexity; in, Adjust the secondary effect of information entropy, Adjust the square root effect of information entropy, Adjust the logarithmic effect of information entropy, D1 represents the data complexity after calculation; According to the data complexity, the data stream is allocated to the corresponding index path using the formula: , generate index path; in, is the interval value of the complexity partition, is the index path assigned according to complexity; Apply the index path and use the inverted index to process compound queries. Basic queries are processed through multi-level B-tree indexes using the formula: , generate index allocation records; Among them, Q1 represents the query type, and R represents the index allocation record.

3. The cloud information retrieval acceleration system according to claim 2, characterized in that: The steps for obtaining the index adjustment log are specifically as follows: Monitor data update frequency and query mode changes using the formula: , calculate the adjusted data update frequency and generate a new update frequency; in, represents the weight factor, Represents the previous update frequency, Represents the current update frequency, Represents the new update frequency; In combination with the index allocation record and the new update frequency, the formula is adopted: , calculate and generate the index adjustment threshold; in, Represents the index to assign record weights, represents the update frequency weight, Represents the index adjustment threshold; According to the index adjustment threshold, data category migration and index type change are performed using the formula: , generate index adjustment log; Among them, Q2 represents the query mode, represents the adjustment action, L represents the index adjustment log, Represents the index adjustment threshold, Is a custom log function, Represents the sine function.

4. The cloud information retrieval acceleration system according to claim 3, characterized in that: The steps of obtaining the data node are specifically as follows: Analyze ad hoc query requests and data access patterns using the formula: , calculate and generate weighted query frequency ; Where n3 is the total number of query requests, is the weight factor, is the frequency of query requests, is the regularization parameter; Based on the weighted query frequency and the current load data of the system, the formula is adopted: , select the data node with the best load ratio and generate the selected data node ; Where m is the total number of data nodes, is the total capacity of node i, is the current utilization rate of node i, is a small constant to prevent division by zero; Using the data node , according to the distribution of query types, the formula is used: ; Determine the data nodes that process the query and generate the selected data nodes ; in, is the category distribution of query requests, is the adjustment factor, is the entropy of the query distribution.

5. The cloud information retrieval acceleration system according to claim 4, characterized in that: The steps for obtaining the optimized query route are specifically as follows: Analyze the current query configuration using the formula: ; Express the impact analysis of the value in logarithmic form, calculate the configuration weight of the current query path, and generate the weight of the current query path ; in, Indicates multiple records in the index adjustment log. is the impact factor of each record; According to the current query path weight and system dynamic load data, the formula is adopted: ; Adjust the query path to match the system load changes and generate the adjusted query path ; in, is the attenuation coefficient, and D2 is the dynamic load index of the system; Combining the adjusted query path and current query requirements, the formula is adopted: ; Optimize query routing to match new query requirements , generate optimized query routing ; in, is the coefficient of variation, Indicates the variability of new query requirements.

Citation Information

Cited By

  • Data table processing method and device, storage medium and electronic equipment

    CN119961265A

  • Mass data query method and related equipment

    CN121009117A