An index building method, device, program product, and storage medium

Abstract

This invention relates to an index construction method, device, program product, and storage medium, and pertains to the technical field of vector databases. The method includes: acquiring query vectors; calculating the coordinate region identifiers of each query vector in the vector space; statistically analyzing the query frequency of each coordinate region identifier within a preset time window to obtain regional query frequency distribution data; calculating the frequency growth rate of the regional query frequency distribution data of each coordinate region identifier at the current moment compared to the regional query frequency distribution data at the previous moment; calculating a query popularity fluctuation coefficient based on each frequency growth rate; monitoring the load parameters of the vector database system in real time and calculating a comprehensive system load index based on the load parameters; and constructing an index for each query vector based on the query popularity fluctuation coefficient and the comprehensive system load index. Implementing the technical solution provided in this application can achieve a balance between indexing effectiveness and system stability.

Description

Technical Field

[0001] This application relates to the technical field of vector databases, specifically to an index construction method, device, program product, and storage medium. Background Technology

[0002] With the rapid development of artificial intelligence technology, vector databases have been widely used as core infrastructure supporting applications such as similarity search and recommendation systems. Index construction, as a core technical aspect of vector databases, directly impacts the query performance and user experience of the entire system.

[0003] Existing vector database indexing technologies mainly include tree-based, hash-based, and graph-based indexes. To improve the intelligence of index building, some existing solutions have adopted dynamic index building strategies based on query popularity.

[0004] These solutions analyze query request QPS (queries per second) and concurrency in real time, calculate a popularity index using a sliding window, and compare the popularity index with a preset threshold to categorize high-popularity and low-popularity scenarios. For high-popularity scenarios, fine-grained indexes are built, such as increasing the number of index partitions and layers, enabling multi-level caching, and preloading high-frequency query results to improve query efficiency. For low-popularity scenarios, coarse-grained indexes are built, the number of index partitions is reduced, flat indexes or low-precision quantization are used, and incremental index updates are enabled to reduce resource consumption.

[0005] However, existing index building methods based on query popularity primarily make decisions based on a single dimension: query popularity. For example, they might initiate fine-grained index building as soon as high query popularity is detected. This could lead to resource-intensive operations being performed even when system resources are limited, creating resource competition with normal query services and causing system instability. Alternatively, it might miss opportunities for index optimization when system resources are sufficient but query patterns are stable, resulting in poor index performance. This traditional index building method makes it difficult for the system to achieve a balance between index performance and system stability. Summary of the Invention

[0006] This application provides an index building method, device, program product, and storage medium that can achieve a balance between indexing performance and system stability.

[0007] The first aspect of this application provides an index construction method, specifically including:

[0008] Obtain query vectors within a preset time window, calculate the coordinate region identifiers of each query vector in the vector space, and count the query frequency of each coordinate region identifier within the preset time window to obtain regional query frequency distribution data.

[0009] Calculate the frequency growth rate of the current regional query frequency distribution data and the previous regional query frequency distribution data for each of the coordinate regions, and calculate the query popularity fluctuation coefficient based on the frequency growth rate.

[0010] The system monitors the load parameters of the vector database system in real time and calculates the overall system load index based on the load parameters.

[0011] The query vectors are indexed and constructed based on the query popularity fluctuation coefficient and the system comprehensive load index.

[0012] By adopting the above technical solution, query vectors are acquired within a preset time window, and their coordinate region identifiers are calculated. This allows for the statistical analysis of regional query frequency distribution data, effectively capturing the distribution characteristics of query load in the vector space. Furthermore, by calculating the frequency growth rate of regional query frequency distribution data at adjacent times and obtaining the query popularity fluctuation coefficient, the dynamic trend of query patterns can be accurately reflected. Simultaneously, by monitoring system load parameters in real time and calculating the comprehensive system load index, the system's resource utilization can be comprehensively evaluated. Index construction decisions based on the query popularity fluctuation coefficient and the comprehensive system load index consider both the fluctuation characteristics of external query load and the availability of internal system resources, thereby optimizing indexing performance while ensuring system stability and achieving a balance between indexing performance and system stability.

[0013] Optionally, calculating the query popularity fluctuation coefficient based on the frequency growth rate of each of the aforementioned factors includes:

[0014] The query popularity fluctuation coefficient is calculated using the frequency growth rate and the preset first formula.

[0015] The first formula is: ;

[0016] Where H is the query popularity fluctuation coefficient. Let i be the frequency growth rate of the i-th coordinate region. Let be the actual query frequency of the i-th coordinate region within the preset time window, and n be the total number of coordinate region identifiers involved in the calculation. It is a weighted average of the frequency growth rate. The weighted standard deviation of the frequency growth rate. This is a preset value for the smallest positive number to prevent division by zero.

[0017] By adopting the above technical solution, a comprehensive index is constructed that reflects both the overall trend of query popularity and captures local fluctuations by weighting the frequency growth rate of each coordinate region according to the actual query frequency and combining it with the dispersion of the frequency growth rate. In this formula, the weighted average term ensures that the growth trend of high-frequency query regions receives more attention, while the standard deviation term can identify drastic changes in the distribution of query load. The fluctuation coefficient obtained by combining the two can accurately characterize the dynamic features of query patterns.

[0018] Optionally, calculating the system's comprehensive load index based on the load parameters includes:

[0019] The overall system load index is calculated using the load parameters and the preset second formula.

[0020] The second formula is: ;

[0021] Where L is the overall system load index. The standardized value of the j-th load parameter is obtained by dividing the original load parameter value by the theoretical maximum value of the load parameter, and its value ranges from [0,1]. m is the total number of load parameters. The maximum value among all standardized load parameter values. The minimum value among all standardized load parameter values. This is a preset value for the smallest positive number to prevent division by zero.

[0022] By adopting the above technical solution, various load parameters are standardized and their distribution characteristics are combined to construct a comprehensive index that can fully reflect the system's resource utilization status. The formula first standardizes load parameters of different dimensions to the [0,1] interval, ensuring the comparability of various indicators. Then, it uses an arithmetic mean to reflect the overall system load level and introduces a term representing the difference between the maximum and minimum values ​​to characterize the degree of load distribution imbalance. This calculation method not only considers the system's average load state but also identifies bottlenecks in certain resources, providing a more accurate metric for system resource scheduling.

[0023] Optionally, the step of indexing each query vector based on the query popularity fluctuation coefficient and the system comprehensive load index includes:

[0024] The query popularity fluctuation coefficient is compared with a preset fluctuation threshold to determine whether the external environment state is a first fluctuation or a second fluctuation, wherein the first fluctuation is less than the second fluctuation;

[0025] The system's overall load index is compared with a preset load threshold to determine whether the internal resource status is a first load or a second load, where the first load is less than the second load.

[0026] When the external environment is in the first fluctuation state and the internal resource state is in the first load state, the existing index structure of each query vector remains unchanged.

[0027] When the external environment is in the first fluctuation state and the internal resource state is in the second load state, the index building operation of each query vector is paused.

[0028] When the external environment state is in the second fluctuation and the internal resource state is in the first load, a first index is constructed for each of the query vectors;

[0029] When the external environment state is in the second fluctuation and the internal resource state is in the second load, a second index is constructed for each of the query vectors.

[0030] By adopting the above technical solution, the system's operating status is divided into two levels—external environment status (first fluctuation or second fluctuation) and internal resource status (first load or second load)—based on two dimensions: query popularity fluctuation coefficient and system comprehensive load index. Differentiated index building strategies are then implemented for different state combinations. Specifically, when the system is in a low-fluctuation, low-load state, the existing index structure remains unchanged to avoid unnecessary resource consumption. When in a low-fluctuation, high-load state, index building operations are paused to ensure system stability. When in a high-fluctuation, low-load state, sufficient system resources are utilized for first-level index building to respond promptly to changes in query patterns. When in a high-fluctuation, high-load state, a relatively lightweight second-level index building method is adopted to moderately optimize the index structure while ensuring system stability. This two-dimensional adaptive index building strategy can flexibly adjust the index structure when query patterns change drastically, while also reasonably controlling the overhead of index building when system resources are strained, thereby achieving a dynamic balance between index performance and system stability.

[0031] Optionally, when the external environment state is in the second fluctuation and the internal resource state is in the first load, constructing a first index for each query vector includes:

[0032] When the external environment is in the second fluctuation state and the internal resource is in the first load state, the target coordinate region identifiers whose frequency growth rate exceeds the preset growth threshold are identified, and a preset number of target coordinate region identifiers are selected as refined construction objects according to the frequency growth rate.

[0033] The target vector space within the preset range where the refined construction object is located is divided according to a preset subdivision multiple. The coordinate region identifier of the query vector corresponding to the refined construction object is recalculated according to the divided target vector space to generate multiple sub-region identifiers.

[0034] A multi-level index structure is constructed based on the identifiers of each sub-region, and the remaining capacity of the system's comprehensive load index is obtained after the construction of the multi-level index structure.

[0035] Predict query hotspot areas based on historical query patterns, and construct preliminary index nodes corresponding to the query hotspot areas based on the remaining capacity. When the query hotspot area includes the query vector, the preliminary index nodes are invoked.

[0036] By adopting the above technical solution, a multi-dimensional optimization strategy is used for the first index construction when the system is in a high-fluctuation, low-load state. First, by identifying target coordinate regions with significant frequency growth rates and selecting a predetermined number of construction objects, index construction resources are prioritized for the most valuable regions. Second, the selected regions are finely divided spatially, and sub-region identifiers are generated to improve index accuracy, enabling queries to locate target data more quickly. Third, a multi-level index structure is constructed, ensuring query efficiency while reserving space for subsequent optimization by calculating remaining capacity. Finally, predictive construction is performed based on historical query patterns, preparing index nodes in advance for potential query hotspots, thereby achieving proactive optimization of query performance.

[0037] Optionally, when the external environment state is in the second fluctuation state and the internal resource state is in the second load state, constructing a second index for each of the query vectors includes:

[0038] When the external environment is in the second fluctuation state and the internal resource state is in the second load state, the remaining capacity of the system comprehensive load index is calculated based on the system comprehensive load index and the preset load threshold.

[0039] Calculate the maximum indexing capacity under the current resource conditions based on the remaining capacity, select the regions corresponding to the coordinate region identifiers in descending order of frequency growth rate as core hotspot regions, and construct an index for the query vectors in each core hotspot region based on the frequency growth rate corresponding to each core hotspot region until the system's overall load index reaches the maximum indexing capacity.

[0040] By adopting the above technical solution, when the system is under high fluctuation and high load, the remaining resource capacity of the system is calculated by the difference between the system's comprehensive load index and the preset load threshold. This determines the maximum indexing capacity that the system can currently handle. Then, based on the priority ranking of frequency growth rate, the most valuable core hotspot areas are indexed to a limited extent. This solution, by calculating the system's remaining capacity in real time, ensures that the indexing process does not lead to system overload, effectively maintaining system stability. At the same time, using the frequency growth rate as the ranking criterion ensures that limited system resources are prioritized for allocation to areas with the fastest growth in query demand, achieving maximum resource utilization efficiency. In addition, this solution dynamically adjusts the scope and progress of indexing based on the actual system load. By setting a maximum indexing capacity as an upper limit, necessary resource buffers are reserved for the system, avoiding system resource exhaustion caused by indexing.

[0041] Optionally, the step of indexing and constructing the query vectors within each of the core hotspot regions based on the frequency growth rate corresponding to each core hotspot region includes:

[0042] Calculate the priority weight of each core hotspot region based on the frequency growth rate corresponding to each core hotspot region, and calculate the single processing limit based on the remaining capacity.

[0043] The core hotspot regions are sorted according to the priority weights to obtain the sorting results. Target query vectors within each core hotspot region that do not exceed the single processing limit are selected according to the sorting results.

[0044] In the current core hotspot area, temporary index nodes are constructed for each selected target query vector. The temporary index nodes adopt a single-level hash structure. The temporary index nodes of the target query vector are merged with the corresponding existing index structure to obtain the merged index structure.

[0045] The query response time of the merged index structure is tested. When the query response time is shortened by more than or equal to the preset response time compared to before the merge, the target query vectors not exceeding the single processing limit are selected in the current core hot spot area until the query response time is shortened by less than the preset response time compared to before the merge. Then the index construction of the current core hot spot area is ended and the index construction of the next priority weight core hot spot area is transferred.

[0046] By adopting the above technical solution, priority weights are first calculated based on the frequency growth rate of core hotspot areas, and the upper limit of single processing is determined in conjunction with the remaining system capacity, thus achieving fine-grained control over the index building process. In specific execution, a strategy of selecting target query vectors in batches is adopted. A temporary index node with a single-level hash structure is built for each query vector, and query performance is gradually optimized through merging operations with the existing index structure. By continuously monitoring the query response time of the merged index structure, when the performance improvement is less than a preset threshold, the system automatically switches to the next priority core hotspot area, ensuring efficient utilization of system resources. This dynamically adjusted index building strategy not only continuously optimizes query performance while ensuring system stability but also avoids resource waste caused by over-optimization through a real-time response time feedback mechanism.

[0047] In a second aspect, this application provides an index building apparatus comprising: one or more processors and a memory; the memory being coupled to the one or more processors, the memory being used to store computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the index building apparatus to perform the method described in the first aspect and any possible implementation thereof.

[0048] Thirdly, this application provides a computer program product containing instructions that, when run on an index building device, cause the index building device to perform the method described in the first aspect and any possible implementation thereof.

[0049] Fourthly, this application provides a computer-readable storage medium including instructions that, when executed on an indexing device, cause the indexing device to perform the method described in the first aspect and any possible implementation thereof. Attached Figure Description

[0050] Figure 1 This is a system architecture diagram of an index building system provided in an embodiment of this application;

[0051] Figure 2 This is a flowchart illustrating an index construction method provided in an embodiment of this application;

[0052] Figure 3 This is a schematic diagram illustrating the principle of a second index construction provided in an embodiment of this application;

[0053] Figure 4 This is an exemplary hardware structure diagram of an index building device provided in an embodiment of this application. Detailed Implementation

[0054] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0055] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0056] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0057] Figure 1 An index building system architecture is illustrated. For example... Figure 1 As shown, the system architecture may include a query client 011, a network 012, and an electronic device 013. The network 012 provides a data transmission link between the query client 011 and the electronic device 013. The network 012 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0058] The query client 011 can send a query request to the electronic device 013 via network 012. The query client 011 is mainly responsible for generating query vectors and initiating similarity search requests according to preset rules.

[0059] The query client 011 is hardware, which can be a client device with vector generation and query sending functions, including but not limited to basic components such as mobile terminals, web browsers, and applications.

[0060] Electronic device 013 is responsible for receiving query requests and performing comprehensive analysis and processing, including core functions such as calculating the coordinate region identifier of the query vector, query frequency statistics, popularity fluctuation analysis, system load monitoring, and index construction. Electronic device 013 can adaptively construct indexes based on query popularity fluctuations and system load status, calculate the optimal index processing strategy, and combine it with preset construction rules to ultimately achieve dynamic optimization of the index structure. These analysis and processing results can be used to improve the response efficiency of subsequent queries.

[0061] It should be noted that electronic devices can be either hardware or software. When an electronic device is hardware, it can be implemented as a distributed cluster of multiple electronic devices or as a single electronic device. When an electronic device is software, it can be implemented as multiple software programs or software modules (e.g., multiple software programs or software modules used to provide distributed processing) or as a single software program or software module. No specific limitations are set here.

[0062] It should be understood that Figure 1 The number of query clients 011, networks 012, and electronic devices 013 shown is merely illustrative. Depending on implementation needs, there can be any number of query clients 011, networks 012, and electronic devices 013. In particular, if the query request does not need to be sent remotely, the above system architecture may exclude network 012 and include only query clients 011 or electronic devices 013.

[0063] This application provides an index construction method, referencing... Figure 2 , Figure 2 This is a flowchart illustrating an index construction method provided in an embodiment of this application, including steps S101 to S104, as follows:

[0064] S101: Obtain the query vector within the preset time window, calculate the coordinate region identifier of each query vector in the vector space, count the query frequency of each coordinate region identifier within the preset time window, and obtain the regional query frequency distribution data.

[0065] In this embodiment, the coordinate region identifier refers to a digital mark used to uniquely identify the spatial region to which each query vector belongs after the vector space is divided into grids using a specific encoding method. Specifically, the coordinate region identifier achieves rapid location and classification of the query vector space by dividing the high-dimensional vector space into grids according to a preset granularity and assigning a unique numerical number to each grid region. For example, in a two-dimensional space, the coordinate range [0,100]×[0,100] can be divided into a 10×10 grid, and each grid corresponds to a coordinate region identifier, such as "A1", "B3", etc. When the query vector coordinates are (25,67), the corresponding coordinate region identifier is "C7".

[0066] Specifically, the query vectors within a preset time window are obtained. Using a preset spatial partitioning algorithm, the coordinates of each query vector are mapped to a corresponding grid region, and a unique coordinate region identifier is assigned to each grid region. The query frequency of each coordinate region identifier within the preset time window is counted. All query frequencies within the preset time window are iterated through, and the occurrence count of each coordinate region identifier is accumulated to obtain the region query frequency distribution data.

[0067] S102: Calculate the frequency growth rate of the current regional query frequency distribution data and the previous regional query frequency distribution data for each coordinate region identifier, and calculate the query popularity fluctuation coefficient based on the frequency growth rate.

[0068] In this embodiment, the query popularity fluctuation coefficient is a comprehensive index that quantifies the degree of dynamic change in the query pattern of the entire vector space by statistically analyzing the frequency changes of the identifiers of each coordinate region at different times. The larger the value, the more drastic the fluctuation of the query pattern; the smaller the value, the more stable the query pattern.

[0069] Specifically, firstly, obtain the frequency distribution data of each coordinate region identifier at the current time and the previous time. By comparing the difference in query frequency of each coordinate region identifier at the two times, calculate the frequency growth rate of each coordinate region identifier. The calculation formula is (current frequency - previous time frequency) / previous time frequency, and obtain the set of frequency growth rates of all coordinate region identifiers.

[0070] Next, the query popularity fluctuation coefficient is calculated using the growth rate of each frequency and a preset first formula, where the first formula is: Where H is the query popularity fluctuation coefficient. Let i be the frequency growth rate of the i-th coordinate region. Let be the actual query frequency of the i-th coordinate region within the preset time window, and n be the total number of coordinate region identifiers involved in the calculation. It is a weighted average of the frequency growth rate. The weighted standard deviation of the frequency growth rate. This is a preset value for the smallest positive number to prevent division by zero.

[0071] The first formula uses a two-factor architecture design, the first part... The weighted average of the frequency growth rate is calculated, with actual query frequency as the weight to ensure that the active query region dominates the calculation. (Part Two) The fluctuation correction factor quantifies the dispersion of the growth rate by the ratio of the standard deviation to the mean. This approach considers both the overall level of the growth rate and the relative intensity of fluctuations. When the standard deviation of the growth rate is large, the fluctuation correction factor increases, allowing the query popularity fluctuation coefficient to sensitively capture the instability of query patterns. Simultaneously, weighted processing avoids noise interference in low-frequency regions, and the introduction of [missing information] effectively prevents division-by-zero anomalies, ensuring robustness of the calculation. Ultimately, this yields a query popularity fluctuation coefficient that accurately quantifies the drastic changes and trend stability of query patterns.

[0072] For example, suppose a vector database system has four coordinate regions: Region 1 had a frequency of 100 at the previous time step and a current frequency of 120; Region 2 had a frequency of 80 at the previous time step and a current frequency of 70; Region 3 had a frequency of 60 at the previous time step and a current frequency of 90; and Region 4 had a frequency of 40 at the previous time step and a current frequency of 35. Then the frequency growth rates for each region would be as follows: =0.2、 =−0.125、 =0.5、 =−0.125, the average frequency growth rate =0.1125, standard deviation =0.2602, with the query frequency of each region at the current time as the weight. =120、 =70、 =90、 =35, let... =0.001, the weighted average value is calculated to be 0.1774 using the formula, the fluctuation correction factor is 3.292, and the final query result is the heat fluctuation coefficient H = 0.1774 × 3.292 = 0.584.

[0073] S103: Monitors the load parameters of the vector database system in real time and calculates the comprehensive load index of the system based on the load parameters.

[0074] In this embodiment, the system comprehensive load index refers to a composite indicator that comprehensively quantifies the current operating status and resource consumption of a vector database system by statistically analyzing load parameters from multiple dimensions. Specifically, the system comprehensive load index standardizes and weights multiple load parameters such as CPU utilization, memory usage, disk I / O load, and network bandwidth usage to form a unified metric that comprehensively reflects the overall performance of the system. A higher value indicates a heavier system load, while a lower value indicates a smoother system operation. For example, when some system resources become bottlenecked while other resources are relatively abundant, the system comprehensive load index will increase accordingly, helping to identify unbalanced load conditions in the system.

[0075] Specifically, real-time monitoring of the vector database system's load parameters includes continuously collecting key performance indicators such as CPU utilization, memory usage, disk read / write speed, network bandwidth, and query response time. These load parameters are periodically acquired and preprocessed. Then, a comprehensive system load index is calculated based on the load parameters using a pre-defined second formula, where the second formula is: Where L is the overall system load index, The standardized value of the j-th load parameter is obtained by dividing the original load parameter value by the theoretical maximum value of the load parameter, and its value ranges from [0,1]. m is the total number of load parameters. The maximum value among all standardized load parameter values. The minimum value among all standardized load parameter values. This is a preset value for the smallest positive number to prevent division by zero.

[0076] The second formula uses a binary summation structure, the first part... The weighted concentration of load parameters is calculated by the ratio of the sum of squares to the linear sum. The square operation amplifies the influence weight of high-load parameters, making the system more sensitive to resource bottlenecks. (Part Two) The extreme ratio of load parameters is calculated to quantify the degree of load imbalance among different resources. The advantage of this approach is that it considers both the overall load level of the system and the balance of load distribution, ultimately yielding a comprehensive system load index that accurately reflects the overall operating status and resource utilization efficiency of the system.

[0077] For example, suppose a vector database system monitors five load parameters, and the CPU utilization is 75% after standardization. =0.75, memory usage is 60% after standardization =0.60, disk I / O load after standardization is 45%. =0.45, network bandwidth utilization is 80% after standardization. =0.80, query response time after 30% standardization =0.30, then the sum of squares of the parameters is =1.855, linear sum is =2.9, =0.8, =0.30, let =0.001, the first part is calculated to be 0.639 and the second part is 2.659 by the second formula. The final comprehensive system load index L = 0.639 + 2.659 = 3.298.

[0078] S104: Construct an index for each query vector based on the query popularity fluctuation coefficient and the system's overall load index.

[0079] In this embodiment, index building refers to employing different index structure building strategies for each query vector based on the current external environment and internal resource status of the system, in order to achieve a dynamic balance between query performance and system resource consumption. Specifically, index building intelligently selects different operation modes, such as maintaining existing indexes, pausing construction, or starting new indexes, by comprehensively considering the fluctuation of query popularity and the system load level. This ensures that when system resources are sufficient, the index structure is actively optimized to improve query efficiency; when the system load is too heavy, additional index building overhead is avoided; and when the query mode is stable, unnecessary index reconstruction is reduced.

[0080] Specifically, indexing is performed on each query vector based on the query popularity fluctuation coefficient and the system's overall load index. First, the query popularity fluctuation coefficient is compared with a preset fluctuation threshold to determine whether the external environment state is in the first or second fluctuation phase, where the first fluctuation is less than the second. Next, the system's overall load index is compared with a preset load threshold to determine whether the internal resource state is in the first or second load phase, where the first load is less than the second load. Then, an appropriate indexing strategy is executed based on the combination of the external environment state and the internal resource state: when the external environment state is in the first fluctuation phase and the internal resource state is in the first load phase, the existing index structure for each query vector remains unchanged; when the external environment state is in the first fluctuation phase and the internal resource state is in the second load phase, the indexing operation for each query vector is paused; when the external environment state is in the second fluctuation phase and the internal resource state is in the first load phase, the first index is built for each query vector; and when the external environment state is in the second fluctuation phase and the internal resource state is in the second load phase, the second index is built for each query vector.

[0081] Based on the above embodiments, as an optional embodiment, S104: the step of constructing an index for each query vector according to the query popularity fluctuation coefficient and the system comprehensive load index may specifically include the following steps:

[0082] S401: Compare the query popularity fluctuation coefficient with the preset fluctuation threshold to determine whether the external environment state is the first fluctuation or the second fluctuation, where the first fluctuation is less than the second fluctuation.

[0083] In this embodiment, the external environment state refers to a binary state classification of the external query environment faced by the vector database system, based on a comparison between the query popularity fluctuation coefficient and a preset fluctuation threshold. This state classification is used to quantify the fluctuation intensity of user query behavior, providing a basis for decision-making in subsequent index construction strategies.

[0084] Specifically, the query popularity fluctuation coefficient is compared with a preset fluctuation threshold. When the query popularity fluctuation coefficient is less than or equal to the preset fluctuation threshold, the external environment state is determined to be the first fluctuation. At this time, the query popularity fluctuation coefficient value is below the threshold, and the degree of change in user query behavior does not exceed the preset limit. When the query popularity fluctuation coefficient is greater than the preset fluctuation threshold, the external environment state is determined to be the second fluctuation. At this time, the query popularity fluctuation coefficient value is above the threshold, and the degree of change in user query behavior exceeds the preset limit. The first fluctuation is less than the second fluctuation, that is, the query popularity fluctuation coefficient value under the first fluctuation state is lower than the value under the second fluctuation state.

[0085] S402: Compare the overall system load index with the preset load threshold to determine the internal resource status as either the first load or the second load, where the first load is less than the second load.

[0086] In this embodiment, the internal resource status refers to a binary classification of the usage of computing resources, storage resources, and network resources within the vector database system, based on a comparison between the system's overall load index and a preset load threshold. This status classification quantifies the current resource consumption level of the system, providing a resource constraint basis for subsequent index construction strategies.

[0087] Specifically, the overall system load metric is compared with a preset load threshold. When the overall system load metric is less than or equal to the preset load threshold, the internal resource state is determined to be "first load." In this case, the overall system load metric is below the threshold, the utilization rates of critical resources such as CPU, memory, and disk do not exceed preset limits, and the system has the processing capacity to execute additional index building tasks. When the overall system load metric is greater than the preset load threshold, the internal resource state is determined to be "second load." In this case, the overall system load metric is above the threshold, the utilization rates of various resources exceed preset limits, and the system processing capacity has reached the preset upper limit. The first load is less than the second load, meaning the overall system load metric under the first load state is lower than the value under the second load state.

[0088] S403: When the external environment is in the first fluctuation state and the internal resource state is in the first load state, keep the existing index structure of each query vector unchanged.

[0089] Specifically, when the external environment is at its first fluctuation and the internal resources are at their first load, this combination indicates that the current query pattern is relatively stable, user query behavior is changing little, and the existing index structure can effectively support the current query needs. The first load indicates that system resources are sufficient; although there is the ability to optimize the index, the necessity for index reconstruction is low due to the stability of the query pattern. Therefore, the system chooses to maintain the status quo, keeping the existing index structure configuration of each query vector unchanged. This avoids unnecessary index adjustments during periods of stable query patterns, ensuring system stability and rational resource utilization, while also reserving resource space to cope with potential changes in query patterns or system load.

[0090] S404: When the external environment is in the first fluctuation state and the internal resource state is in the second load state, pause the index building operation of each query vector.

[0091] Specifically, when the external environment is in the first fluctuation state and the internal resource state is in the second load state, under this combination, the first fluctuation indicates that the current query pattern is relatively stable, user query behavior changes little, the existing index structure can basically meet the current query needs, and the urgency of index optimization is relatively low; the second load indicates that system resources are strained, the utilization rate of key resources such as CPU and memory is high, and the system processing capacity is approaching its bottleneck state. Therefore, the system chooses to suspend the index building operation, prioritizing the processing of user query requests with valuable system resources, ensuring the stability and response speed of the query service under resource constraints, and avoiding the overall system performance degradation caused by the index building task competing with the query processing task for resources.

[0092] S405: When the external environment is in the second fluctuation state and the internal resource state is in the first load state, construct the first index for each query vector.

[0093] In this embodiment, the first index construction refers to the adaptive index optimization operation performed on each query vector under the condition that the query pattern changes and the system resources are sufficient. Specifically, the first index construction includes multiple steps such as identifying and refining high-frequency query regions, subdividing the target vector space to generate a multi-level index structure, and constructing preparatory index nodes based on historical query pattern prediction and remaining resource capacity. Through this series of refined index construction processes, the system can proactively adapt to the changing trends of query patterns, ensuring current query performance while preparing for possible future query hotspots.

[0094] Specifically, when the external environment is in the second fluctuation state and the internal resource state is in the first load state, the system performs the first index construction operation on each query vector. In this state combination, the second fluctuation indicates that the current query pattern is changing, user query behavior is showing new trends or hotspots, and the existing index structure may not be fully adaptable to the new query requirements; the first load state indicates that the system resources are sufficient and have the ability to optimize and reconstruct the index. Therefore, the system initiates the first index construction process: First, it identifies target coordinate region identifiers whose frequency growth rate exceeds a preset growth threshold, and selects a preset number of target coordinate region identifiers as refined construction objects; then, it divides the target vector space within the preset range of the refined construction objects according to a preset subdivision multiple, recalculates the coordinate region identifiers, and generates multiple sub-region identifiers; next, it constructs a multi-level index structure based on each sub-region identifier and obtains the remaining capacity of the system's overall load index after construction; finally, it predicts query hotspot areas based on historical query patterns and uses the remaining capacity to construct corresponding reserve index nodes to achieve proactive adaptation to changes in query patterns.

[0095] Based on the above embodiments, as an optional embodiment, S405: When the external environment state is in the second fluctuation and the internal resource state is in the first load, the step of constructing a first index for each query vector may specifically include the following steps:

[0096] S501: When the external environment is in the second fluctuation state and the internal resource state is in the first load state, identify the target coordinate area identifiers whose frequency growth rate exceeds the preset growth threshold, and select a preset number of target coordinate area identifiers as refined construction objects according to the frequency growth rate.

[0097] In this embodiment of the application, the refined construction object refers to the set of specific coordinate region identifiers that need to be optimized for deep indexing, selected based on the analysis of query frequency growth trends.

[0098] Specifically, when the external environment is in the second fluctuation state and the internal resource state is in the first load state, the system first analyzes the historical query data of each target coordinate region identifier, calculates the frequency growth rate of each target coordinate region identifier, and compares it with a preset growth threshold to identify target coordinate region identifiers whose frequency growth rate exceeds the threshold. Then, the system sorts these candidate regions according to the magnitude of the frequency growth rate and selects a preset number of target coordinate region identifiers as refinement construction objects in descending order.

[0099] S502: Divide the target vector space within the preset range where the refined construction object is located according to the preset subdivision multiple, and recalculate the coordinate region identifier of the query vector corresponding to the refined construction object according to the divided target vector space to generate multiple sub-region identifiers.

[0100] Specifically, first, the system determines the target vector space within a preset range corresponding to the coordinate region identifier of each refined construction object, including the boundary coordinates and spatial dimension information of the coordinate region. Then, the system proportionally divides the target vector space according to a preset subdivision factor. For example, when the subdivision factor is 4, the original region is divided into 4 equal parts in each dimension, thus generating 16 sub-regions. Next, the system recalculates the coordinate region identifier of the query vector corresponding to the refined construction object according to the divided target vector space, assigning a unique identifier code to each sub-region, generating multiple sub-region identifiers.

[0101] S503: Construct a multi-level index structure based on the identifiers of each sub-region, and obtain the remaining capacity of the system's comprehensive load index after constructing the multi-level index structure.

[0102] In the embodiments of this application, the multi-level index structure refers to a hierarchical index organization architecture based on sub-region identifiers. The multi-level index structure achieves hierarchical management and rapid positioning of query vectors by establishing index nodes at different granularities.

[0103] Specifically, firstly, the system categorizes and organizes multiple sub-region identifiers according to their spatial location and hierarchical affiliation, establishing parent-child and adjacency relationships between them. Then, based on these parent-child and adjacency relationships, the system constructs a multi-level index structure, including creating root node indexes, intermediate level indexes, and leaf node indexes. Each index node stores the identifier information, spatial range, and pointers to lower-level nodes for its corresponding region. Next, the system allocates corresponding storage space and computing resources to each index level and establishes query paths and data transfer between levels. Finally, by monitoring the CPU, memory, and storage load parameters consumed during index construction, the system calculates the overall system load index according to the second formula in S103. Based on the calculated overall system load index, and then using the difference between the system's theoretical maximum load capacity and the current overall system load index, the system's remaining capacity is calculated.

[0104] S504: Predict query hotspot areas based on historical query patterns, and construct the corresponding preparatory index nodes based on the remaining capacity. When the query hotspot area includes a query vector, the preparatory index node is invoked.

[0105] In this embodiment of the application, the reserve index node refers to a backup index structure built in advance based on the query hotspot prediction mechanism, which is used to cope with the high-frequency query demand that may occur in the future.

[0106] Specifically, firstly, the system collects and analyzes historical query data, including query frequency distribution, temporal variation patterns, and spatial clustering characteristics. It then uses machine learning algorithms or statistical analysis methods to predict the location and scope of areas that may become query hotspots in the future. Next, the system divides the remaining capacity by the resource cost required for a single pre-index node to determine the construction quota. Simultaneously, it ranks the importance of each query hotspot area based on factors such as historical query frequency and regional coverage in the prediction results. Then, within the construction quota limit, the system sequentially selects the query hotspot areas with the highest importance ranking to build pre-index nodes. The specific construction process includes first determining the spatial boundaries and coordinate range of the query hotspot areas; then extracting the feature information and location identifiers of all vector data within the area; establishing a fast retrieval mapping table for vectors within the area; creating an index node data structure containing key information such as area identifiers, spatial range parameters, vector feature summaries, and retrieval path pointers; allocating independent memory storage space and computing thread resources to the pre-index nodes; establishing the association link and calling interface between the pre-index nodes and the main index structure; and finally, marking the completed pre-index nodes as pending activation and incorporating them into the system's index management framework until the remaining capacity is exhausted or the preset node limit is reached. When the actual query vector falls into the predicted query hotspot area, the system immediately calls the corresponding prepared index node, converting the prepared index node from the pending state to the active state, and directly providing fast response service for query requests.

[0107] S406: When the external environment is in the second fluctuation state and the internal resource state is in the second load state, construct the second index for each query vector.

[0108] In this embodiment of the application, the second index construction refers to a resource-constrained index construction strategy executed for the query vector under a specific system state where the external environment state is the second fluctuation and the internal resource state is the second load.

[0109] Specifically, when the external environment is in the second fluctuation state and the internal resource state is in the second load state, the system compares the current overall system load index with a preset load threshold to determine the remaining capacity of the overall system load index. Then, based on the calculated remaining capacity, the system calculates the maximum index processing capacity under the current resource conditions, determining the upper limit of the index building tasks the system can handle without overloading. Next, the system sorts the regions corresponding to each coordinate region from high to low frequency growth rate, selecting the regions with the highest frequency growth rate as core hotspot regions. These core hotspot regions represent the regions with the fastest growth in query demand and the highest priority for index building. Finally, based on the frequency growth rate corresponding to each core hotspot region, the system performs ordered index building operations on the query vectors within these core hotspot regions, continuously building indexes until the overall system load index reaches the pre-calculated maximum index processing capacity.

[0110] like Figure 3 As shown, Figure 3 This is a schematic diagram illustrating the principle of a second index construction provided in an embodiment of this application.

[0111] Based on the above embodiments, as an optional embodiment, S406: When the external environment state is a second fluctuation and the internal resource state is a second load, the step of constructing a second index for each query vector may specifically include the following steps:

[0112] S601: When the external environment is in the second fluctuation state and the internal resource state is in the second load state, calculate the remaining capacity of the system comprehensive load index based on the system comprehensive load index and the preset load threshold.

[0113] Specifically, when the external environment is in the second fluctuation state and the internal resource state is in the second load state, the system obtains the current system comprehensive load index value, then the system reads the preset load threshold, and then the system calculates the difference between the preset load threshold and the current system comprehensive load index to obtain the remaining capacity of the system comprehensive load index.

[0114] S602: Calculate the maximum indexing capacity under the current resource conditions based on the remaining capacity, select the regions corresponding to the coordinate region identifiers as core hotspot regions in descending order of frequency growth rate, and build an index for the query vectors in each core hotspot region based on the frequency growth rate corresponding to each core hotspot region until the system's overall load index reaches the maximum indexing capacity.

[0115] In this embodiment, the maximum index processing capacity refers to the maximum amount of processing that the system can perform index building operations under the current system resource constraints. It is determined by comprehensively considering factors such as remaining capacity, index building complexity, and system performance requirements.

[0116] Specifically, the system calculates the maximum indexing capacity under current resource conditions based on remaining capacity. First, the system calculates the priority weight of each core hotspot region based on its frequency growth rate, and simultaneously calculates the single-processing limit based on remaining capacity and the estimated resource consumption of a single index node. Then, the system sorts the core hotspot regions from highest to lowest priority weight, obtaining the sorting result. It processes each core hotspot region sequentially according to the sorting result, selecting target query vectors within each core hotspot region that do not exceed the single-processing limit as the processing objects for the current batch. Next, the system constructs temporary index nodes for each selected target query vector within the current core hotspot region. These temporary index nodes use a single-level hash structure to reduce construction complexity and resource consumption. Finally, the constructed temporary index nodes for the target query vectors are merged with the corresponding existing index structure to obtain the merged index structure. Finally, the system tests the query response time of the merged index structure to evaluate the index optimization effect. When the query response time is shortened by more than or equal to the preset response time compared to before the merge, the system continues to select the next batch of target query vectors not exceeding the single processing limit in the current core hotspot area for index building. When the query response time is shortened by less than the preset response time compared to before the merge, it indicates that the index optimization effect of the current area has reached the point of diminishing marginal returns. The system ends the index building of the current core hotspot area and moves to the core hotspot area with the next priority weight to continue the index building operation until the system's overall load index reaches the maximum index processing capacity.

[0117] Based on the above embodiments, as an optional embodiment, S602: the step of indexing and constructing the query vectors in each core hotspot region according to the frequency growth rate corresponding to each core hotspot region may specifically include the following steps:

[0118] S701: Calculate the priority weight of each core hotspot area based on the frequency growth rate corresponding to each core hotspot area, and calculate the upper limit of single processing based on the remaining capacity.

[0119] Specifically, firstly, the system obtains the frequency growth rate of each core hotspot area within the statistical period. Then, the system uses a weighted algorithm to standardize the frequency growth rate. Through a preset weight base and normalization formula, the frequency growth rate of each core hotspot area is converted into a corresponding priority weight. The calculation formula is: Priority Weight = (Current Area Frequency Growth Rate / Sum of Frequency Growth Rates of All Core Hotspot Areas) × Weight Base. Simultaneously, the system calculates the single-processing limit based on the remaining capacity and the average resource consumption of a single query vector index, i.e., Single-Processing Limit = Remaining Capacity / Average Resource Consumption of a Single Index Node.

[0120] S702: Sort each core hotspot region according to priority weight, obtain the sorting result, and select the target query vector within each core hotspot region according to the sorting result, which does not exceed the single processing limit.

[0121] In this embodiment of the application, the target query vector refers to the set of specific query vectors selected from each core hot spot area according to priority sorting and processing capacity limitations. These vectors represent the actual processing objects of the current batch index building task.

[0122] Specifically, first, the system obtains the priority weights of each core hotspot region, ranking the regions with the highest priority weights first, forming a sorted result from high to low. Then, the system processes each core hotspot region sequentially according to the sorted result, starting with the highest-ranked core hotspot region, directly selecting query vectors within that region as candidate target query vectors. Next, based on the single-processing limit, the system sequentially selects query vectors from the current core hotspot region in quantities not exceeding the single-processing limit as target query vectors.

[0123] S703: In the current core hotspot area, temporary index nodes are constructed for each selected target query vector. The temporary index nodes adopt a single-level hash structure. The temporary index nodes of the target query vector are merged with the corresponding existing index structure to obtain the merged index structure.

[0124] In the embodiments of this application, a temporary index node refers to a lightweight index structure unit created separately for the target query vector, serving as a transitional data structure between the new query vector and the existing index system.

[0125] Specifically, first, the system obtains a set of target query vectors. For each target query vector in the set, it performs a temporary index node construction operation, using a single-level hash structure as the basic architecture of the temporary index nodes. A hash function maps the feature values ​​of the target query vectors to fixed-length hash codes, establishing a direct mapping relationship between vector identifiers and hash codes. Then, the system allocates independent storage space and access interfaces for each temporary index node. Next, the system performs an index merging operation, integrating the temporary index nodes of the target query vectors with the corresponding existing index structures. Through index node linking, hash table expansion, and metadata updates, the mapping relationships and data content in the temporary index nodes are integrated into the existing index structure. Finally, the system generates the merged index structure.

[0126] S704: Perform query response time testing on the merged index structure. If the query response time is shortened by more than or equal to the preset response time compared to before the merge, continue to select target query vectors within the current core hotspot area that do not exceed the single processing limit until the query response time is shortened by less than the preset response time compared to before the merge. Then, end the index building of the current core hotspot area and transfer to the index building of the next priority weight core hotspot area.

[0127] In this embodiment of the application, query response time testing refers to the performance evaluation process of performing standardized query operations on the merged index structure and measuring the response time of the merged index structure, which is used to quantify the index optimization effect and guide subsequent processing decisions.

[0128] Specifically, first, the system obtains the merged index structure, performs multiple query response time tests on the merged index structure, records the complete response time of each query, and calculates the average response time. Simultaneously, it obtains the baseline response time data of the index structure before merging, subtracts the average response time from the baseline response time data, and obtains the response time reduction margin. Next, the system compares the response time reduction margin with a preset response time. When the reduction margin is greater than or equal to the preset response time, the system continues to select a new target query vector within the current core hotspot region, following the method in step S702, not exceeding the single processing limit, and repeats steps S703 and S704 to continuously optimize the index performance of the current region. Finally, when the query response time reduction margin is less than the preset response time, the system ends the index building task for the current core hotspot region, transfers to the next priority-weighted core hotspot region according to the determined priority ranking results, and begins a new round of index building.

[0129] The following describes an exemplary index building device provided in an embodiment of this application. Figure 4 This is an exemplary hardware structure diagram of an index building device provided in an embodiment of this application.

[0130] In some embodiments, the index building apparatus is a computer device or includes a computer device. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device stores data. The network interface of the computer device is used to communicate with other external terminals or servers via a network connection. In some embodiments, the network interface can be a wired network interface; in some embodiments, the network interface can also be a wireless network interface. When the computer program is executed by the processor, it implements the methods in the embodiments of this application.

[0131] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0132] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0133] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0134] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0135] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. An index construction method, characterized by, The method includes: Obtain query vectors within a preset time window, calculate the coordinate region identifiers of each query vector in the vector space, and count the query frequency of each coordinate region identifier within the preset time window to obtain regional query frequency distribution data. Calculate the frequency growth rate of the regional query frequency distribution data at the current time and the regional query frequency distribution data at the previous time for each of the aforementioned coordinate regions, and calculate the query popularity fluctuation coefficient based on each of the aforementioned frequency growth rates, including: The query popularity fluctuation coefficient is calculated using the frequency growth rate and the preset first formula. The first formula is: ; H is a query popularity fluctuation coefficient, is a frequency growth rate of the i-th coordinate region, is an actual query frequency of the i-th coordinate region within the preset time window, and n is the total number of coordinate region identifiers participating in the calculation, is a weighted average of the frequency growth rate, is a weighted standard deviation of the frequency growth rate, is a preset small positive number to prevent division by zero; Real-time monitoring of the load parameters of the vector database system, and calculation of the system's comprehensive load index based on the load parameters, including: The overall system load index is calculated using the load parameters and the preset second formula. The second formula is: ; wherein L is a system comprehensive load index, is a standardized value of the jth load parameter, obtained by dividing the original load parameter value by the theoretical maximum value of the load parameter, with a value range of [0, 1], and m is the total number of load parameters, is the maximum value among all the load parameter standardized values, is the minimum value among all the load parameter standardized values, is a preset minimum positive number for preventing division by zero; The query vectors are indexed and constructed based on the query popularity fluctuation coefficient and the system comprehensive load index, including: The query popularity fluctuation coefficient is compared with a preset fluctuation threshold to determine whether the external environment state is a first fluctuation or a second fluctuation, wherein the first fluctuation is less than the second fluctuation; The system's overall load index is compared with a preset load threshold to determine whether the internal resource status is a first load or a second load, where the first load is less than the second load. When the external environment is in the first fluctuation state and the internal resource state is in the first load state, the existing index structure of each query vector remains unchanged. When the external environment is in the first fluctuation state and the internal resource state is in the second load state, the index building operation of each query vector is paused. When the external environment state is in the second fluctuation and the internal resource state is in the first load, a first index is constructed for each of the query vectors; When the external environment state is in the second fluctuation and the internal resource state is in the second load, a second index is constructed for each of the query vectors.

2. The index construction method of claim 1, wherein, When the external environment state is in the second fluctuation and the internal resource state is in the first load, the first index is constructed for each of the query vectors, including: When the external environment is in the second fluctuation state and the internal resource is in the first load state, the target coordinate region identifiers whose frequency growth rate exceeds the preset growth threshold are identified, and a preset number of target coordinate region identifiers are selected as refined construction objects according to the frequency growth rate. The target vector space within the preset range where the refined construction object is located is divided according to a preset subdivision multiple. The coordinate region identifier of the query vector corresponding to the refined construction object is recalculated according to the divided target vector space to generate multiple sub-region identifiers. A multi-level index structure is constructed based on the identifiers of each sub-region, and the remaining capacity of the system's comprehensive load index is obtained after the construction of the multi-level index structure. Predict query hotspot areas based on historical query patterns, and construct preliminary index nodes corresponding to the query hotspot areas based on the remaining capacity. When the query hotspot area includes the query vector, the preliminary index nodes are invoked.

3. The index building method of claim 1, wherein, When the external environment state is in the second fluctuation state and the internal resource state is in the second load state, the second index is constructed for each of the query vectors, including: When the external environment is in the second fluctuation state and the internal resource state is in the second load state, the remaining capacity of the system comprehensive load index is calculated based on the system comprehensive load index and the preset load threshold. Calculate the maximum indexing capacity under the current resource conditions based on the remaining capacity, select the regions corresponding to the coordinate region identifiers in descending order of frequency growth rate as core hotspot regions, and construct an index for the query vectors in each core hotspot region based on the frequency growth rate corresponding to each core hotspot region until the system's overall load index reaches the maximum indexing capacity.

4. The index construction method of claim 3, wherein, The step of indexing and constructing the query vectors within each core hotspot region based on the frequency growth rate corresponding to each core hotspot region includes: Calculate the priority weight of each core hotspot region based on the frequency growth rate corresponding to each core hotspot region, and calculate the single processing limit based on the remaining capacity. The core hotspot regions are sorted according to the priority weights to obtain the sorting results. Target query vectors within each core hotspot region that do not exceed the single processing limit are selected according to the sorting results. In the current core hotspot area, temporary index nodes are constructed for each selected target query vector. The temporary index nodes adopt a single-level hash structure. The temporary index nodes of the target query vector are merged with the corresponding existing index structure to obtain the merged index structure. The query response time of the merged index structure is tested. When the query response time is shortened by more than or equal to the preset response time compared to before the merge, the target query vectors not exceeding the single processing limit are selected in the current core hot spot area until the query response time is shortened by less than the preset response time compared to before the merge. Then the index construction of the current core hot spot area is ended and the index construction of the next priority weight core hot spot area is transferred.

5. An indexing apparatus, comprising: The index building apparatus includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, and the one or more processors invoking the computer instructions to cause the index building apparatus to perform the method as described in any one of claims 1-4.

6. A computer program product comprising instructions, characterized in that, When the computer program product is run on an index building device, the index building device performs the method as described in any one of claims 1-4.

7. A computer-readable storage medium comprising instructions, wherein: When the instructions are executed on the index building device, the index building device performs the method as described in any one of claims 1-4.

Images