A Data Relevance Recommendation Method Based on Vectorized Information Matching

Abstract

This invention relates to the field of vector retrieval technology, specifically to a data relevance recommendation method based on vectorized information matching, which solves the technical problem of low performance in existing systems. The method includes: acquiring a sampling period log set; the sampling period log set includes multiple query records, each query record including a traversal start vector, a result target vector, and the number of distance calculations; performing geometric feature analysis of the query trajectory based on the sampling period log set to determine the center displacement vector and the trajectory dispersion coefficient; the center displacement vector characterizes the geometric deviation between the index entry node and the actual query hotspot area; the trajectory dispersion coefficient characterizes the consistency of the query trajectory direction with respect to the center displacement vector; performing a vector index structure update based on the center displacement vector and the trajectory dispersion coefficient; and performing data relevance recommendation using the updated vector index structure.

Description

Technical Field

[0001] This invention relates to the field of vector retrieval technology, and more specifically to a data relevance recommendation method based on vectorized information matching. Background Technology

[0002] In the era of big data, vector retrieval technology is widely used in recommender systems, image search, semantic matching, and other fields. With the continuous growth of data volume and the dynamic changes in user behavior, efficient vector index structures and accurate data-related recommendation functions are crucial for ensuring system performance. Graph-based approximate nearest neighbor search methods have become the mainstream choice in industry due to their high query efficiency, while the core requirement of data-related recommendation is to quickly match target data that meets user query features.

[0003] Existing vector index structures are typically built statically, with their topology remaining largely fixed after index creation. However, in real-world applications, user query intent and data hotspots change dynamically over time. When high-frequency query areas deviate geometrically from the fixed entry point of the index, the search algorithm needs to navigate through numerous intermediate nodes to reach the target area. This results in feature redundancy and insufficient identification of key matching features in the index structure of traditional data relevance recommendation methods, ultimately impacting system performance. Summary of the Invention

[0004] To address the problem of low performance in existing systems, the present invention aims to provide a data relevance recommendation method based on vectorized information matching. The specific technical solution adopted is as follows: This application provides a data relevance recommendation method based on vectorized information matching, including: Obtain the sampling period log set; the sampling period log set includes multiple query records, each query record includes the traversal start vector, the result target vector, and the number of distance calculations; Based on the sampling period log set, the geometric feature analysis of the query trajectory is performed to determine the center displacement vector and the trajectory dispersion coefficient; the center displacement vector represents the degree of geometric deviation between the index entry node and the actual query hotspot area; the trajectory dispersion coefficient represents the degree of consistency between the direction of the query trajectory and the vector direction of the center displacement vector. Perform vector index structure update based on the center displacement vector and the trajectory discrete coefficients; Recommendations based on data relevance are made using the updated vector index structure.

[0005] The present invention has the following beneficial effects: In view of the low performance of existing technology systems, this application provides a data relevance recommendation method based on vectorized information matching. This application can comprehensively capture the geometric features and computing power consumption of the query process by collecting a sampling periodic log set containing the traversal start vector, the result target vector, and the number of distance calculations. Based on the sampling periodic log set, the geometric feature analysis of the query trajectory is performed. The obtained center displacement vector and trajectory dispersion coefficient can accurately quantify the degree of misalignment and morphological changes between the index structure and the data distribution. Thus, the vector index structure is updated based on the center displacement vector and trajectory dispersion coefficient, realizing the adaptive adjustment of the index structure to dynamic query hotspots. Finally, data relevance recommendation is performed through the updated vector index structure. This effectively solves the problem of low retrieval efficiency and wasted computing power caused by the misalignment of the fixed entry vector index with dynamic hotspots in the existing technology, and significantly improves the overall performance of the data relevance recommendation system. Attached Figure Description

[0006] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0007] Figure 1 This is a flowchart illustrating a data relevance recommendation method based on vectorized information matching, provided in one embodiment of the present invention. Figure 2 This is a schematic diagram of the hardware structure of a data relevance recommendation device based on vectorized information matching, provided in one embodiment of the present invention. Detailed Implementation

[0008] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a data relevance recommendation method based on vectorized information matching proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0009] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0010] In all division and logarithmic operations covered in this application, a smoothing mechanism is employed to prevent computer program crashes or invalid values ​​from being generated due to a zero denominator or a zero input. Specifically, a positive correction factor is superimposed on the denominator term of the division operation or the argument term of the logarithmic function. For example, the value is This ensures the robustness and feasibility of the algorithm under extreme conditions.

[0011] The normalization function mentioned in this application Unless otherwise specified, all values ​​are normalized using maximum and minimum values. The maximum and minimum values ​​are preset empirical extreme values ​​derived from a large amount of historical experimental data. If the calculated result exceeds the [0,1] interval, it is restricted to the [0,1] range by a truncation function (i.e., if the result is less than 0, it is taken as 0, and if it is greater than 1, it is taken as 1) to eliminate the influence of outliers on the evaluation index.

[0012] In view of the low performance of existing technology systems, this application provides a data relevance recommendation method based on vectorized information matching. This application can comprehensively capture the geometric features and computing power consumption of the query process by collecting a sampling periodic log set containing the traversal start vector, the result target vector, and the number of distance calculations. Based on the sampling periodic log set, the geometric feature analysis of the query trajectory is performed. The obtained center displacement vector and trajectory dispersion coefficient can accurately quantify the degree of misalignment and morphological changes between the index structure and the data distribution. Thus, the vector index structure is updated based on the center displacement vector and trajectory dispersion coefficient, realizing the adaptive adjustment of the index structure to dynamic query hotspots. Finally, data relevance recommendation is performed through the updated vector index structure. This effectively solves the problem of low retrieval efficiency and wasted computing power caused by the misalignment of the fixed entry vector index with dynamic hotspots in the existing technology, and significantly improves the overall performance of the data relevance recommendation system.

[0013] The following description, in conjunction with the accompanying drawings, details a specific scheme for a data relevance recommendation method based on vectorized information matching provided by the present invention.

[0014] Please see Figure 1 The diagram illustrates a flowchart of a data relevance recommendation method based on vectorized information matching, according to an embodiment of the present invention. The method includes the following steps: Step 101: Obtain the sampling period log set.

[0015] The sampling period log set includes multiple query records, each of which includes the traversal start vector, the result target vector, and the number of distance calculations.

[0016] For example, vector indexes can adopt a hierarchical graph index structure (such as Hierarchical Navigable Small World Graphs (HNSW)). In the actual operation of the vector retrieval system, each user's retrieval request triggers the nearest neighbor search process in the graph index. In order to accurately capture the matching status between index entry points and actual query hotspots, this application can collect key data in the query process within a set time period to construct a sampling period log set.

[0017] In a vector retrieval system, when a user initiates a query, the system traverses from the index entry node until it finds the approximate nearest neighbor. The traversal start vector is the feature vector of the initial node from which the query request begins navigation in the vector index, directly determined by the current index entry node, representing the starting geometric position of the query. The result target vector is the feature vector of the node finally matched by the query request, reflecting the geometric position corresponding to the user's true search intent. The number of distance calculations is the total number of vector distance (such as Euclidean distance and cosine similarity) calculations performed during the query process, quantifying the computational power consumption and retrieval difficulty of the query. By associating these three data items with the query records, the geometric trajectory and resource consumption of a single query can be comprehensively reflected, providing basic data support for subsequent analysis.

[0018] Step 102: Perform geometric feature analysis of the query trajectory based on the sampling period log set to determine the center displacement vector and trajectory dispersion coefficient.

[0019] Among them, the center displacement vector represents the degree of geometric deviation between the index entry node and the actual query hotspot area, and the trajectory dispersion coefficient represents the degree of consistency between the query trajectory direction and the vector direction of the center displacement vector.

[0020] After obtaining the sampling period log set, this application can mine the matching relationship between the index structure and the query distribution through geometric feature analysis. The center displacement vector can quantify the overall deviation between the average starting position corresponding to the index entry and the center of the target region corresponding to the actual high-frequency query. The severity of the deviation can be intuitively judged through this center displacement vector.

[0021] The trajectory dispersion coefficient focuses on the directional characteristics of the query trajectory. If the direction of most query trajectories is consistent with the direction of the central displacement vector, it indicates that the query hotspots exhibit overall migration characteristics. If the directions are dispersed, it indicates that the query hotspots exhibit multi-peak splitting characteristics.

[0022] In this way, this application can comprehensively depict the dynamic changes in query distribution through the above two indicators, providing accurate decision-making basis for subsequent index updates and avoiding efficiency decline caused by blind adjustments.

[0023] Step 103: Update the vector index structure based on the center displacement vector and the trajectory discrete coefficients.

[0024] In some embodiments, this application can adjust the topology of the vector index based on two core indicators: the center displacement vector and the trajectory dispersion coefficient. If the center displacement vector indicates a significant deviation, and the trajectory dispersion coefficient reflects a consistent query direction, then the entry nodes are adjusted to align with hotspot areas; if the trajectory dispersion coefficient reflects a dispersed query direction, then new long-range connection edges are used to optimize the retrieval path in multi-peak areas. This ensures that the adjustment of the index structure matches the dynamic changes in query distribution, avoiding the limitations of existing fixed entry strategies.

[0025] In one possible implementation, after updating the vector index structure, this application can also clear the sampling period log set and reset the sampling time window to start a new round of sampling period log set data collection.

[0026] The clearing of the sampling period log set is to prevent historical data from the old period from interfering with the analysis of the new period. Because the index structure has been updated, the index state corresponding to the old data is inconsistent with the new state, and continuing to use it will lead to distorted analysis results. Resetting the sampling time window includes operations such as resetting the time window timer to zero and resetting the sample counter, enabling the system to restart data collection based on the updated index structure and start a new round of runtime data sampling. This reset mechanism ensures that each optimization period uses independent and up-to-date data, avoiding the cumulative interference of historical data.

[0027] Step 104: Perform data relevance recommendations using the updated vector index structure.

[0028] First, upon receiving a new query request, this application can vectorize the query request, transforming it into a feature vector with the same vector dimensions as the vector index structure. This feature vector is generated using the same feature extraction algorithm as the original data, ensuring that similarity calculations are performed in the same vector space.

[0029] Subsequently, this application can perform an approximate nearest neighbor search based on the updated and optimized vector index structure, thereby achieving vectorized information matching. This involves querying the database for the most similar target vector and determining the data information corresponding to the target vector. Since the index structure has been optimized based on the geometric distribution characteristics of historical query trajectories, the search process can significantly improve retrieval efficiency while maintaining accuracy.

[0030] Finally, this application can generate recommendation results based on the data information corresponding to the target vector of the search results, and present them to end users through API interfaces or user interfaces. At the same time, the user's subsequent interaction behavior (such as clicks, dwell time, further filtering, etc.) will be recorded as new query records for the next round of index structure updates, forming a closed-loop optimization mechanism.

[0031] Based on the above technical solution, this application can comprehensively capture the geometric features and computing power consumption of the query process by collecting a sampling periodic log set containing the traversal start vector, the result target vector, and the number of distance calculations. Based on the sampling periodic log set, the geometric feature analysis of the query trajectory is performed. The obtained center displacement vector and trajectory dispersion coefficient can accurately quantify the degree of misalignment and morphological changes between the index structure and the data distribution. Thus, the vector index structure is updated based on the center displacement vector and trajectory dispersion coefficient, realizing the adaptive adjustment of the index structure to dynamic query hotspots. Finally, data relevance recommendation is performed through the updated vector index structure, effectively solving the problem of low retrieval efficiency and wasted computing power caused by the misalignment of the fixed entry vector index with dynamic hotspots in the existing technology, and significantly improving the overall performance of the data relevance recommendation system.

[0032] As a possible embodiment of this application, step 101 above can be implemented through the following steps: Step 201: Within the sampling time window, for each received query request, take the feature vector of the initial node accessed by the query request in the graph index as the traversal start vector, take the feature vector of the result node returned by the query request as the result target vector, and count the total number of vector distance calculations performed by the query request as the distance calculation count.

[0033] In vector retrieval systems, graph index structures typically consist of multi-level graphs. Queries begin at the top-level entry node and traverse downwards layer by layer using a greedy search strategy. The sampling time window refers to the set continuous data collection period. Its length can be flexibly configured according to the query frequency and hotspot change rate of the vector retrieval system. For example, the sampling time window can be set to 60 seconds, ensuring sufficient sample data collection while promptly capturing hotspot changes.

[0034] The initial node accessed in the graph index is the global entry node of the index. This node is the fixed starting point for nearest neighbor search, and its feature vector is the traversal start vector. The result node is the approximate nearest neighbor node finally returned by the query request, and its feature vector is the result target vector. The number of distance calculations can be counted in real time using an atomic counter to ensure the accuracy of the statistical results. This number of distance calculations directly reflects the retrieval difficulty and computational power consumption of the query from the starting node to the target node.

[0035] For example, within the sampling time window, this application can perform Bernoulli sampling on all received query requests, with a sampling probability set to 0.01. This ensures sample representativeness while avoiding system performance overhead caused by full data collection. For each sampled query request, data collection is triggered at key moments in the search algorithm execution (when accessing the initial node or returning the result node), and the number of distance calculations is counted in real time to ensure the correlation and timeliness of the three core data items.

[0036] Step 202: Perform data preprocessing operations after the sampling time window ends to obtain the sampling period log set.

[0037] In one possible implementation, the data preprocessing operation includes the following steps: Filter query records where the number of distance calculations is less than the preset filtering threshold; If the number of filtered query records is less than the sample size threshold, the currently obtained query records are deemed invalid, and sampling begins again. If the number of filtered query records is greater than or equal to the sample size threshold, vector normalization is performed on the traversal start vector and result target vector of the filtered query records.

[0038] The preset filtering threshold is a critical value used to distinguish between valid queries and unstructured bottleneck queries. Its value can be determined based on the statistical characteristics of historical query data, for example, it can be set to 20% of the historical average number of distance calculations. Queries with extremely low distance calculation counts are usually due to accidental factors such as cache hits or the target node being located near the entry node. They cannot reflect the true matching status between the index structure and the query distribution. Retaining such records would interfere with the accuracy of subsequent analysis results, so they need to be filtered out.

[0039] The sample size threshold is the minimum sample size to ensure the statistical analysis results are significant; for example, it can be set to 500. If the number of filtered samples is too small, the statistical results may be random and cannot truly reflect the overall characteristics of the query distribution. Performing geometric analysis based on such samples will lead to errors in index update decisions, so they need to be deemed invalid and resampled. If the sample size meets the requirements, vector normalization processing is then performed on the traversal start vector and result target vector of the filtered query records.

[0040] The purpose of vector normalization is to unify the magnitude of the vectors to meet the requirements of subsequent geometric analysis. For example, this application can perform L2 normalization on the vectors to normalize the magnitude of the vectors to 1, constrain all vector endpoints to the unit hypersphere, and ensure the consistency and accuracy of geometric calculations.

[0041] Based on the above technical solution, this application ensures the relevance and timeliness of data collection by collecting the traversal start vector, result target vector, and distance calculation count within a set sampling time window. Through preprocessing operations after the sampling time window ends, high-quality input data is provided for subsequent geometric feature analysis, enabling the system to perform accurate topology optimization based on real and effective runtime data, thereby improving the reliability and effectiveness of index adaptive adjustment.

[0042] As a possible embodiment of this application, step 102 above can be implemented through the following steps: Step 301: Determine the average starting point vector based on the traversal starting vector of each query record in the sampling period log set, and determine the weighted target center vector based on the result target vector and the number of distance calculations of each query record in the sampling period log set.

[0043] The starting point average vector is the arithmetic mean of all traversal starting vectors. This application initializes a zero vector with the same dimension as the node's feature vector as an accumulator. It iterates through all query records in the sampling period log set, accumulating the traversal starting vector of each query record into the accumulator. After traversal, the accumulated vector in the accumulator is divided by the total number of query records to obtain the starting point average vector. This starting point average vector represents the average starting position of all queries within the current time window, reflecting the geometric positional characteristics corresponding to the index entry node.

[0044] It should be noted that during the initialization of the query algorithm, due to dynamic adjustments in the neighbor relationships within a layer, local graph pruning, or the algorithm's own random selection mechanism, the first node actually visited (i.e., the traversal starting vector) may not always strictly correspond to the preset entry node, but rather exhibit slight random fluctuations within the vicinity of the entry node. Therefore, this application can characterize the statistical average position of the starting point entering the graph index structure by calculating the arithmetic mean of the traversal starting vectors of each query record in the sampled periodic log set, thereby more accurately reflecting the distribution of the actual query's starting position.

[0045] The weighted target center vector is obtained by weighting the result target vector based on the number of distance calculations. This application can initialize a zero vector with the same dimension as the node's feature vector as an accumulator. It iterates through all query records in the sampling period log set, multiplying the result target vector of each query record by the corresponding number of distance calculations (scalar multiplication of vector), obtaining a weighted vector, and accumulating it into the accumulator. After the iteration is complete, the magnitude of the accumulator's accumulated vector is calculated, and the accumulated vector is divided by this magnitude to achieve weighted averaging, thus obtaining the weighted target center vector.

[0046] Since queries with higher distance calculation counts are more difficult to retrieve and better reflect the bottleneck of the index structure, this application can highlight the importance of the target region corresponding to such queries by weighting, so that the calculated target center can better reflect the distribution characteristics of actual high-frequency and high-consumption queries.

[0047] Step 302: Use the difference vector from the average vector at the starting point to the weighted target center vector as the center displacement vector.

[0048] For example, the center displacement vector satisfies the following formula: in, The center displacement vector, For the weighted target center vector, This is the starting point average vector. The starting point average vector represents the overall offset direction and degree of the current index entry position relative to the actual high-frequency query region. The direction of the starting point average vector indicates the theoretical direction in which the index entry should be adjusted, and the magnitude of the starting point average vector quantifies the current structural lag. Center displacement vector. The larger the modulus, the more serious the deviation, and the more necessary it is to adjust the index structure.

[0049] Therefore, this center displacement vector intuitively reflects the misalignment between the index structure and the query distribution. It should be noted that the center displacement vector is a virtual vector and does not necessarily correspond to an actual node in the graph index; it is mainly used to guide subsequent index update strategies.

[0050] Step 303: Determine the trajectory dispersion coefficients based on the center displacement vector and the sampling period log set.

[0051] In one possible implementation, this application can determine the individual trajectory vector for each query record in the sampling period log set based on the traversal start vector and the result target vector of the query record.

[0052] For example, the individual trajectory vector satisfies the following formula: in, For the first Individual trajectory vectors for each query record. For the first The result target vector of each query record For the first The traversal start vector of each query record. The individual trajectory vector represents the displacement path of a single query from the entry point to the result. In the feature space of the index structure, this individual trajectory vector connects the start and end positions of the query, intuitively reflecting the geometric path traversed by the search algorithm in this query.

[0053] Then, for each body trajectory vector, the vertical deviation component of the individual trajectory vector in the vector direction perpendicular to the central displacement vector is obtained by vector projection decomposition.

[0054] For example, the vertical deviation component satisfies the following formula: in, For the first The vertical deviation component of the individual trajectory vector of each query record in the direction perpendicular to the center displacement vector. For the first Individual trajectory vectors for each query record. The center displacement vector, The vertical deviation component is the projection vector of the individual trajectory vector in the direction perpendicular to the center displacement vector. The difference between the individual trajectory vector and the projection vector can be used to obtain the vertical deviation component.

[0055] The vertical deviation component characterizes the degree to which an individual query trajectory deviates from the overall displacement direction. Geometrically, it is perpendicular to the central displacement vector. The larger the magnitude of the vertical deviation component, the worse the consistency between the direction of the query and the overall displacement direction.

[0056] Thus, this application can determine the trajectory dispersion coefficient by combining the vertical deviation component and distance calculation times of each query record in the sampling period log set with the center displacement vector.

[0057] For example, the trajectory dispersion coefficients satisfy the following formula: in, The coefficients of dispersion of the trajectory. The number of records queried in the sample period log set. For the first The number of distance calculations for each query record. For the first The vertical deviation component of the individual trajectory vector of each query record in the direction perpendicular to the center displacement vector. The center displacement vector. This is a safety parameter used to correct fractions where the denominator is 0; its specific value can be determined based on... The value of the value determines the outcome, such as .

[0058] The overall vertical deviation, which is a weighted average of the magnitudes of the vertical deviation components of each query record based on the number of calculations for the corresponding distance, reflects the average degree of deviation of all query records relative to the central displacement direction. The larger the value, the higher the degree of dispersion of the overall query direction. The coefficient of variation (COP) is the magnitude of the center displacement vector, reflecting the overall deviation. Therefore, the COP quantifies the relative degree of directional dispersion while considering the overall deviation magnitude, avoiding misjudgments caused by judging solely by absolute deviation values ​​(e.g., the same vertical deviation component magnitude has different meanings under different overall deviation magnitudes). A smaller COP indicates a higher consistency between the query trajectory direction and the center displacement vector direction, and a stronger tendency for the query distribution to migrate overall; a larger COP indicates a higher degree of dispersion in the query trajectory direction, and a stronger tendency for the query distribution to split into multiple peaks.

[0059] Based on the above technical solution, this application determines the average starting point vector based on the traversal start vector of each query record in the sampling period log set, accurately reflecting the average starting position of the query. It determines the weighted target center vector based on the result target vector and the number of distance calculations for each query record in the sampling period log set, highlighting the importance of the target region corresponding to high-consumption queries and making the target center more closely match the actual query hotspots. Thus, the difference vector from the average starting point vector to the weighted target center vector is used as the center displacement vector, accurately quantifying the overall deviation between the index entry point and the query hotspots. Furthermore, the trajectory dispersion coefficient is calculated, providing a key basis for judging the query distribution pattern. The above technical solution ensures that the obtained indicators can truly reflect the matching state between the index structure and the query distribution, providing accurate decision support for subsequent index updates.

[0060] As a possible embodiment of this application, step 103 above can be implemented through the following steps: Step 401: Determine the dynamic displacement judgment threshold based on the result target vector and weighted target center vector of each query record in the sampling period log set.

[0061] In some embodiments, this application may traverse the sampling period log set, calculate the Euclidean distance between each result target vector and the weighted target center vector, use the arithmetic mean of the obtained Euclidean distances as the average distribution radius, and determine the dynamic displacement judgment threshold based on the average distribution radius.

[0062] For example, the dynamic displacement determination threshold satisfies the following formula: in, The threshold for determining dynamic displacement. The radius of the average distribution. This is a ratio coefficient, which can be flexibly adjusted according to the system's sensitivity to hotspot changes; for example, it can be set to 0.2.

[0063] Because the sampled data exhibits certain statistical fluctuations, minute center displacements may simply be caused by noise rather than genuine changes in the query distribution. Performing index updates for such minute displacements would lead to frequent system oscillations, impacting system stability. The dynamic displacement threshold adapts to different query distributions. When the query distribution is dense, the average distribution radius is small, and the dynamic displacement threshold decreases accordingly, increasing sensitivity to displacement. Conversely, when the query distribution is sparse, the average distribution radius is large, and the dynamic displacement threshold increases accordingly, avoiding misjudgments caused by sparse distribution.

[0064] Step 402: If the magnitude of the center displacement vector is greater than the dynamic displacement determination threshold, perform vector index structure update based on the trajectory discrete coefficient.

[0065] If the magnitude of the center displacement vector exceeds the dynamic displacement determination threshold, it indicates a significant structural lag in the current index structure, necessitating a vector index structure update. If the magnitude of the center displacement vector is less than or equal to the dynamic displacement determination threshold, the current index structure is in a steady state and requires no update, allowing direct entry into the next sampling cycle.

[0066] In one possible implementation, if the trajectory dispersion coefficient is less than the dispersion determination threshold, this application can perform an entry pointer update operation.

[0067] The dispersion threshold is a critical value that distinguishes between single-peak migration and multi-peak splitting. Its value can be determined through experimental calibration, for example, it can be set to 1.0. When the trajectory dispersion coefficient is less than this dispersion threshold, it indicates that the direction of most query trajectories is basically consistent with the direction of the center displacement vector. The query hotspot has undergone a holistic positional migration in the vector space. At this time, by adjusting the position of the index entry node, the new hotspot center can be directly aligned, eliminating the misalignment between the entry and the hotspot, reducing the initial jump steps of the query, significantly reducing the average search path length, and improving retrieval efficiency.

[0068] In some embodiments, the entry pointer update operation includes the following steps: using the weighted target center vector as the query vector, performing an approximate nearest neighbor search in the current graph index to determine the target node that is closest to the weighted target center vector, and then updating the node pointer of the index entry node of the vector index based on the target node.

[0069] The weighted target center vector represents the geometric center of the query hotspot region. However, this weighted target center vector is a virtual coordinate system, not an actual node in the graph index. Therefore, it needs to be mapped to a physically existing node in the graph index. This application can determine the target node closest to the weighted target center vector through an approximate nearest neighbor search. This approximate nearest neighbor search can employ a greedy search algorithm, and this target node becomes a new candidate entry node.

[0070] For example, this application can use the metadata update interface of the graph index to modify the value of the global entry pointer from the identifier of the original entry node to the identifier of the target node. This operation is typically atomic, ensuring consistency in a multi-threaded environment. After the update is complete, all subsequent new queries will traverse based on the target node.

[0071] In another possible implementation, if the trajectory dispersion coefficient is greater than or equal to the dispersion determination threshold, this application can perform a long-range edge addition operation.

[0072] When the trajectory dispersion coefficient is greater than or equal to the dispersion threshold, it indicates that although the query trajectory has overall displacement, the individual directions are extremely divergent, and the query hotspots exhibit a multi-peak splitting pattern. In this case, blindly moving the entry node to the position corresponding to the weighted target center vector may cause the entry point to fall into a sparse data region, thus increasing search overhead. Therefore, the optimal strategy is to keep the original entry node unchanged and provide a shortcut for the dispersed, high-cost queries by adding new long-range connecting edges, thereby optimizing the retrieval path.

[0073] In some embodiments, the long-range edge addition operation includes the following steps: sorting the query records in descending order according to the number of distance calculations in the sampling period log set, selecting the top-ranked query records, extracting the nodes pointed to by the result target vectors of the multiple query records, constructing a high-time-consuming node set, and then, in the highest level where the index entry node of the current vector index is located, establishing a unidirectional connection edge from the index entry node to each node in the high-time-consuming node set at the corresponding level (or mapping the bottom node upwards).

[0074] The number of selected query records can be flexibly configured based on the capacity of the neighbor table in the graph index. For example, the top 5 query records can be selected. These query records correspond to the most expensive queries under the current index structure. High-cost queries usually reflect a severe mismatch between the index structure and the query distribution, and optimizing these queries can bring the greatest performance benefits. Queries with higher distance calculation counts have longer retrieval paths and greater computational consumption. The corresponding nodes are more likely to reflect key hotspot areas in multi-peak split scenarios. Therefore, selecting such query records can accurately locate the high-cost paths that need optimization.

[0075] It should be noted that multiple queries may converge to the same node, resulting in a reduction in the actual number of nodes in the high-time-consuming node set. This application can deduplicate nodes and reorder them based on the cumulative distance calculation count to ensure that the high-time-consuming node set contains the truly high-overhead target regions.

[0076] Then, this application can establish unidirectional connection edges. First, check the capacity of the neighbor table of the entry node: if the neighbor table is not full, directly add unidirectional directed edges pointing to each node in the high-time-consuming node set in the neighbor table; if the neighbor table is full, calculate the distance between the existing nodes in the neighbor table and the entry node, remove the node with the farthest distance to free up space, and then add new connection edges. The direction of the unidirectional connection edges is from the entry node to the high-time-consuming nodes. These newly added unidirectional connection edges provide a direct channel for high-time-consuming queries, enabling them to jump to the vicinity of the target area in one step, greatly reducing the number of intermediate jumps.

[0077] It should be noted that these connecting edges are unidirectional (from the entry point to the destination), which will not disrupt the hierarchy and connectivity of the graph structure, and will not significantly increase memory usage.

[0078] Based on the above technical solution, this application calculates a dynamic displacement judgment threshold based on the result target vector and the weighted target center vector. This allows the dynamic displacement judgment threshold to adapt to the sparsity of the query distribution. By comparing the magnitude of the center displacement vector with the dynamic displacement judgment threshold, it determines whether to trigger an update. This ensures that an update operation is only performed when there is a significant deviation between the index entry and the query hotspot, avoiding frequent system oscillations caused by statistical noise and improving system stability and resource utilization. Simultaneously, by comparing the trajectory dispersion coefficient with the dispersion judgment threshold, it achieves accurate judgment of the query distribution pattern and executes differentiated index update operations accordingly. In a single-peak migration scenario, the entry pointer update operation can directly align with the hotspot center, quickly eliminating the misalignment between the index entry and the hotspot. In a multi-peak splitting scenario, the long-range edge addition operation avoids the risk of the entry moving to a sparse region and optimizes the retrieval path of the dispersed hotspots through shortcuts. This differentiated strategy ensures accurate matching between the index update method and the query distribution pattern, further improving the adaptive capability and retrieval efficiency of the vector index.

[0079] This application also provides a hardware structure diagram of a data relevance recommendation device based on vectorized information matching (denoted as data relevance recommendation device 20 based on vectorized information matching), see [link to diagram]. Figure 2 The data relevance recommendation device 20 based on vectorized information matching includes a processor 21, and optionally, a memory 22 connected to the processor 21.

[0080] In the first possible implementation, see Figure 2The data relevance recommendation device 20 based on vectorized information matching also includes a transceiver 23. The processor 21, memory 22, and transceiver 23 are connected via a bus. The transceiver 23 is used to communicate with other devices or communication networks. Optionally, the transceiver 23 may include a transmitter and a receiver. The device in the transceiver 23 that implements the receiving function can be considered as a receiver, which is used to perform the receiving steps in the embodiments of this application. The device in the transceiver 23 that implements the transmitting function can be considered as a transmitter, which is used to perform the transmitting steps in the embodiments of this application.

[0081] Based on the first possible implementation method Figure 2 The schematic diagram shown can be used to illustrate the structure of the data relevance recommendation device based on vectorized information matching involved in the above embodiments.

[0082] in, Figure 2 The diagram can also illustrate the system chip in a data relevance recommendation device based on vectorized information matching. In this case, the actions performed by the aforementioned data relevance recommendation device based on vectorized information matching can be implemented by this system chip. The specific actions performed are detailed above and will not be repeated here.

[0083] In implementation, each step of the method provided in this embodiment can be completed by integrated logic circuits in the processor or by instructions in software form. The steps of the method disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0084] It should be noted that the various embodiments of this application can be referenced or learned from each other. For example, the same or similar steps, method embodiments, system embodiments and device embodiments can be referenced from each other without limitation.

[0085] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0086] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A data relevance recommendation method based on vectorized information matching, characterized in that, include: Obtain the sampling period log set; The sampling period log set includes multiple query records, each query record including the traversal start vector, the result target vector, and the number of distance calculations; Based on the sampling period log set, the geometric feature analysis of the query trajectory is performed to determine the center displacement vector and the trajectory dispersion coefficient; the center displacement vector represents the degree of geometric deviation between the index entry node and the actual query hotspot area; the trajectory dispersion coefficient represents the degree of consistency between the direction of the query trajectory and the vector direction of the center displacement vector. Perform vector index structure update based on the center displacement vector and the trajectory discrete coefficients; Recommendations based on data relevance are made using the updated vector index structure.

2. The data relevance recommendation method based on vectorized information matching according to claim 1, characterized in that, Obtain the sampling period log set, including: Within the sampling time window, for each received query request, the feature vector of the initial node accessed by the query request in the graph index is used as the traversal start vector, the feature vector of the result node returned by the query request is used as the result target vector, and the total number of vector distance calculations performed by the query request is counted as the distance calculation count. After the sampling time window ends, data preprocessing is performed to obtain the sampling period log set.

3. The data relevance recommendation method based on vectorized information matching according to claim 2, characterized in that, The data preprocessing operation includes the following steps: Filter query records where the number of distance calculations is less than the preset filtering threshold; If the number of filtered query records is less than the sample size threshold, the currently obtained query records are deemed invalid, and sampling begins again. If the number of filtered query records is greater than or equal to the sample size threshold, the traversal start vector and result target vector of the filtered query records are normalized.

4. The data relevance recommendation method based on vectorized information matching according to claim 1, characterized in that, Based on the sampling period log set, perform geometric feature analysis of the query trajectory to determine the center displacement vector and trajectory dispersion coefficients, including: The starting point average vector is determined based on the traversal start vector of each query record in the sampling period log set, and the weighted target center vector is determined based on the result target vector and the number of distance calculations of each query record in the sampling period log set. The difference vector from the average vector of the starting point to the weighted target center vector is used as the center displacement vector; The trajectory dispersion coefficients are determined based on the center displacement vector and the sampling period log set.

5. The data relevance recommendation method based on vectorized information matching according to claim 4, characterized in that, Based on the center displacement vector and the sampling period log set, the trajectory discrete coefficients are determined, including: For each query record in the sampling period log set, the individual trajectory vector is determined based on the traversal start vector and result target vector of the query record; For each body trajectory vector, the vertical deviation component of the individual trajectory vector in the vector direction perpendicular to the central displacement vector is obtained by vector projection decomposition; The trajectory dispersion coefficient is determined by combining the vertical deviation component and distance calculation count of each query record in the sampling period log set with the center displacement vector.

6. The data relevance recommendation method based on vectorized information matching according to claim 4, characterized in that, Based on the center displacement vector and the trajectory discrete coefficients, the vector index structure is updated, including: The dynamic displacement judgment threshold is determined based on the result target vector of each query record in the sampling period log set and the weighted target center vector. If the magnitude of the center displacement vector is greater than the dynamic displacement determination threshold, the vector index structure is updated based on the trajectory discrete coefficient.

7. The data relevance recommendation method based on vectorized information matching according to claim 6, characterized in that, Based on the trajectory discrete coefficients, the vector index structure is updated, including: If the trajectory dispersion coefficient is less than the dispersion determination threshold, perform an entry pointer update operation; If the trajectory dispersion coefficient is greater than or equal to the dispersion determination threshold, a long-range edge addition operation is performed.

8. The data relevance recommendation method based on vectorized information matching according to claim 7, characterized in that, The entry pointer update operation includes the following steps: Using the weighted target center vector as the query vector, perform an approximate nearest neighbor search in the current graph index to determine the target node that is closest to the weighted target center vector; The node pointer of the index entry node of the updated vector index is updated based on the target node.

9. The data relevance recommendation method based on vectorized information matching according to claim 7, characterized in that, The operation of adding long-range edges includes the following steps: Based on the number of distance calculations for each query record in the sampling period log set, sort them in descending order and select the top-ranked query records. Extract the nodes pointed to by the result target vectors of the multiple query records to construct a set of high-time-consuming nodes; Establish a one-way connection edge between the index entry node of the current vector index and each node in the set of high-time-consuming nodes.

10. The data relevance recommendation method based on vectorized information matching according to claim 1, characterized in that, After performing a vector index structure update based on the center displacement vector and the trajectory discrete coefficients, the method further includes: Clear the sampling period log set and reset the sampling time window to start a new round of sampling period log set data collection.

Images