Multimodal vector index update method, apparatus, device, and medium

Abstract

This application provides a method, apparatus, device, and medium for updating a multimodal vector index. The method includes: receiving a data update instruction for a multimodal vector index; determining the operational multimodal vector corresponding to the data update instruction; determining the first core cluster center point closest to the operational multimodal vector; determining whether the data update instruction has a global impact based on whether the operational multimodal vector falls into the target sub-index structure corresponding to the first core cluster center point; performing corresponding operations on the operational multimodal vector according to the update operation type within the target sub-index structure to obtain the updated multimodal vector index; and, if the data update instruction has a global impact, determining an unstable sub-index structure from multiple sub-index structures; adjusting the core cluster center point of the unstable sub-index structure; and re-dividing the sub-index structure based on the adjusted core cluster center point to obtain the updated sub-index structure, thereby effectively improving the intelligence of index updates.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a multimodal vector index update method, apparatus, device, and medium. Background Technology

[0002] With the widespread application of Large Language Models (LLM) and Retrieval-augmented Generation (RAG) technologies, multimodal retrieval has become crucial for enhancing the cognitive and interactive capabilities of Artificial Intelligence (AI) systems. The core of this field is mapping data from different modalities, such as text, images, and audio, to a unified vector space using embedding techniques, and then leveraging vector databases for efficient Approximate Nearest Neighbor (ANN) searches. The maintenance mechanism of multimodal vector indexes has become a bottleneck restricting the performance and usability of multimodal retrieval; after initial construction, vector indexes typically require updates to incorporate new data.

[0003] When updating indexes, most methods employ a uniform approach, either attempting updates locally on all items or triggering a global adjustment after a certain number of updates have accumulated. This can lead to unnecessary index structure traversal, wasting resources. Furthermore, updates may cause the adjustment path to spread, resulting in significant fluctuations in the index structure and affecting index stability. Summary of the Invention

[0004] In view of this, this application provides a multimodal vector index update method, apparatus, device and medium, which determines whether there is a global impact on the data update instruction and performs index update accordingly, thereby realizing active control of index update, and restricting the update operation to be executed within the target sub-index structure, reducing index structure fluctuation.

[0005] Specifically, this application is implemented through the following technical solution: According to a first aspect of this application, a multimodal vector index update method is provided, the method comprising: The system receives a data update instruction for a multimodal vector index; the data update instruction carries an update operation type and update data information; the nodes in the multimodal vector index are used to store multimodal vectors; the multimodal vector index is pre-divided into multiple sub-index structures, and the sub-index structures are obtained based on the corresponding core cluster center points. Based on the updated data information, determine the operation multimodal vector corresponding to the data update instruction; The operation multimodal vector is matched with multiple core cluster centers to determine the first core cluster center closest to the operation multimodal vector. Based on whether the operation multimodal vector falls into the target sub-index structure corresponding to the first core cluster center, it is determined whether the data update instruction has a global impact. Within the target sub-index structure, the corresponding operation is performed on the operation multimodal vector according to the update operation type to obtain the updated multimodal vector index; When the data update instruction has a global impact, an unstable sub-index structure is determined from the multiple sub-index structures, the core cluster center point of the unstable sub-index structure is adjusted, and the sub-index structure is re-divided according to the adjusted core cluster center point to obtain the updated sub-index structure.

[0006] In one optional implementation, the method further includes: Based on the update operation type and the update data information, an update event object corresponding to the data update instruction is generated; Obtain a snapshot of the metadata of the multimodal vector index; Perform a consistency check on the update event object and the metadata snapshot to obtain the check result; The step of determining the operation multimodal vector corresponding to the data update instruction based on the updated data information includes: If the verification result indicates that the data update instruction passes, the operation multimodal vector corresponding to the data update instruction is determined based on the updated data information, and the operation multimodal vector is added to the update event object.

[0007] In one optional implementation, the method further includes: From the multiple core cluster centers, select the second core cluster center with the highest semantic correlation between the stored multimodal vector and the operational multimodal vector; Based on the vector distance between the operation multimodal vector and the second core cluster center point, and the coverage radius corresponding to the second core cluster center point, the update impact metric value corresponding to the data update instruction is determined, and the update impact metric value is added to the update event object.

[0008] In one optional implementation, the update operation type includes a add operation, a modify operation, or a delete operation; the data update information includes a data identifier, multimodal raw data payload, and a timestamp. The step of determining the operation multimodal vector corresponding to the data update instruction based on the updated data information includes: When the update operation type is a new operation or a modification operation, the data identifier, the original multimodal data payload, and the timestamp are jointly encoded according to the spatial dimension and measurement method of the multimodal vector index to obtain the operation multimodal vector corresponding to the data update instruction. When the update operation type is a delete operation, a frozen vector matching the data identifier is found in the metadata of the multimodal vector index, and the frozen vector is determined as the operation multimodal vector.

[0009] In one optional implementation, matching the operational multimodal vector with multiple core cluster centers to determine the first core cluster center closest to the operational multimodal vector includes: Based on the vector distance between the operation multimodal vector and the second core cluster center point, and the coverage radius corresponding to the second core cluster center point, the operation shunting confidence level corresponding to the data update instruction is determined; For each core cluster center point, a structure-aware distance score of the operation multimodal vector relative to the core cluster center point is determined based on the vector distance between the operation multimodal vector and the core cluster center point, the coverage radius corresponding to the core cluster center point, and the operation shunting confidence. The core cluster center point corresponding to the minimum structure perception distance score is determined as the first core cluster center point that is closest to the operation multimodal vector, and the minimum structure perception distance score is added to the update event object corresponding to the data update instruction. The step of determining whether the data update instruction has a global impact based on whether the operational multimodal vector falls into the target sub-index structure corresponding to the first core cluster center point includes: If the minimum structure-aware distance score is less than or equal to the structure-aware distance threshold, it is determined that the operational multimodal vector falls into the target sub-index structure corresponding to the first core cluster center point; If the multimodal vector of the operation falls into the target sub-index structure corresponding to the first core cluster center point, it is determined that the data update instruction has no global impact.

[0010] In one optional implementation, the step of performing corresponding operations on the multimodal vector according to the update operation type within the target sub-index structure to obtain the updated multimodal vector index includes: When the update operation type is a new operation, find the first leaf node with the smallest distance to the multimodal vector of the operation within the target sub-index structure; Based on the current number of carrying vectors of the first leaf node, the capacity limit of the first leaf node, and the minimum structure perception distance score corresponding to the operation multimodal vector, the local reconstruction pressure of the first leaf node is determined. When the local reconstruction pressure is less than or equal to the splitting threshold, the operational multimodal vector is inserted into the first leaf node to obtain the updated multimodal vector index.

[0011] In one optional implementation, the method further includes: When the local reconstruction pressure is greater than the splitting threshold, two target multimodal vectors are selected from the multimodal vectors stored in the first leaf node, and a split second leaf node is constructed with each target multimodal vector as the center. For each multimodal vector in the first leaf node other than the target multimodal vector, the target multimodal vector with the smallest distance from the two target multimodal vectors is determined, and the multimodal vector is assigned to the second leaf node corresponding to the target multimodal vector with the smallest distance. Based on the two split second leaf nodes, the pointers of the parent nodes of the first leaf nodes are adjusted to obtain the updated multimodal vector index.

[0012] In one optional implementation, the step of performing corresponding operations on the multimodal vector according to the update operation type within the target sub-index structure to obtain the updated multimodal vector index includes: When the update operation type is a modification operation, the third leaf node to which the multimodal vector to be modified belongs is found within the target sub-index structure; Determine the subtree path where the third leaf node is located, and determine the fourth leaf node from the subtree path where the operation multimodal vector is to be inserted; The multimodal vector to be modified is deleted from the third leaf node, and the multimodal vector to be modified is inserted into the fourth leaf node to obtain the updated multimodal vector index.

[0013] In one optional implementation, the step of performing corresponding operations on the multimodal vector according to the update operation type within the target sub-index structure to obtain the updated multimodal vector index includes: In the case that the update operation type is a deletion operation, the fifth leaf node to which the operation multimodal vector belongs is found within the target sub-index structure; Based on the load state of the fifth leaf node, determine whether the fifth leaf node satisfies the sparsity condition; If the fifth leaf node does not meet the sparsity condition, the operation multimodal vector is deleted from the fifth leaf node to obtain the updated multimodal vector index. If the fifth leaf node satisfies the sparsity condition, the operation multimodal vector is deleted from the fifth leaf node, and the fifth leaf node after vector deletion is locally balanced to obtain the updated multimodal vector index.

[0014] In one optional implementation, the local balancing process for the fifth leaf node after vector deletion includes: Based on the load status of the fifth leaf node after vector deletion, determine whether the fifth leaf node after vector deletion meets the merging condition; If the fifth leaf node after vector deletion does not meet the merging condition, the layout of the fifth leaf node after vector deletion is adjusted to obtain the updated multimodal vector index. If the fifth leaf node after vector deletion meets the merging condition, the multimodal vector stored in the fifth leaf node after vector deletion is merged into the leaf node with the smallest distance from the fifth leaf node in the target sub-index structure to obtain the updated multimodal vector index.

[0015] In one optional implementation, the step of determining an unstable sub-index structure from the plurality of sub-index structures when the data update instruction has a global impact, adjusting the core cluster centers of the unstable sub-index structure, and re-dividing the sub-index structure according to the adjusted core cluster centers to obtain an updated sub-index structure includes: If the data update instruction has a global impact, the update event object corresponding to the data update instruction is written into the buffer queue; Based on the update impact metric and minimum structure perception distance score recorded in each update event object in the buffer queue, the global evolution pressure corresponding to the buffer queue is determined. When the global evolution pressure is greater than the adjustment threshold and the load is in a low occupancy range, an unstable sub-index structure is determined from the sub-index structures corresponding to multiple update event objects in the buffer queue, and the update event object corresponding to the unstable sub-index structure is taken out from the buffer queue. The core cluster centers of the unstable substructure are adjusted, and the sub-index structure is re-divided based on the adjusted core cluster centers to obtain the updated sub-index structure.

[0016] In one optional implementation, adjusting the core cluster centers of the unstable substructure includes: Identify the target update event object that has a dominant influence on the core cluster centers of the unstable substructure from the buffer queue; Based on the operation multimodal vector and minimum structure-aware distance score recorded in the target update event object, the core cluster center points of the unstable substructure are adjusted to obtain the adjusted core cluster center points.

[0017] In one optional implementation, the method further includes: If the updated multimodal vector index or the updated sub-index structure is obtained, the metadata of the multimodal vector index is updated according to the updated multimodal vector index or the updated sub-index structure to obtain the updated metadata. Based on the updated metadata, the target metadata of the multimodal vector index is determined.

[0018] In one optional implementation, the method further includes: Based on the core cluster centers before adjustment, the coverage radius corresponding to each core cluster center before adjustment, and the core cluster centers after adjustment, the index stability convergence metric of the updated metadata compared to the metadata before update is determined. The step of determining the target metadata for the multimodal vector index based on the updated metadata includes: If the index stability convergence metric is within a stable range, the updated metadata is determined as the target metadata of the multimodal vector index.

[0019] According to a second aspect of this application, a multimodal vector index update apparatus is provided, the apparatus comprising: The instruction receiving module is used to obtain data update instructions for the multimodal vector index; the data update instructions carry update operation type and update data information; the nodes in the multimodal vector index are used to store multimodal vectors; the multimodal vector index is pre-divided into multiple sub-index structures, and the sub-index structures are obtained based on the corresponding core cluster center points; The vector determination module is used to determine the operation multimodal vector corresponding to the data update instruction based on the updated data information. The global judgment module is used to match the operation multimodal vector with multiple core cluster centers, determine the first core cluster center closest to the operation multimodal vector, and determine whether the data update instruction has a global impact based on whether the operation multimodal vector falls into the target sub-index structure corresponding to the first core cluster center. A local update module is used to perform corresponding operations on the multimodal vector of the operation according to the update operation type within the target sub-index structure to obtain the updated multimodal vector index; The global update module is used to determine the unstable sub-index structure from the multiple sub-index structures when the data update instruction has a global impact, adjust the core cluster center point of the unstable sub-index structure, and re-divide the sub-index structure according to the adjusted core cluster center point to obtain the updated sub-index structure.

[0020] According to a third aspect of this application, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the multimodal vector index update method described in the first aspect above.

[0021] According to a fourth aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the multimodal vector index update method described in the first aspect.

[0022] The multimodal vector index update method, apparatus, device, and medium provided in this application continuously quantifies and intelligently distributes the structural impact of a data update instruction upon receipt. It determines whether the data update instruction has a global impact and processes it accordingly. When there is no global impact, only index updates are performed. When a global impact exists, adjustments to the core cluster centers are added to the index update, transforming index maintenance from passive repair to proactive, structure-aware control, effectively improving the intelligence and accuracy of index updates. Furthermore, a boundary-constrained local reconstruction mechanism is adopted, restricting update operations to within the target sub-index structure, preventing the spread of operations, and enabling updates with minimal disturbance, reducing index structure fluctuations. Adjustments that disrupt cluster boundaries are only performed when a global impact exists, reducing the cascading disturbance of updates to the overall index topology at the structural level. This maintains both index consistency and high-throughput write capabilities in highly dynamic data scenarios, ensuring the efficiency, stability, and sustainability of cross-modal hybrid retrieval in highly dynamic data environments.

[0023] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this disclosure.

[0024] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0025] Figure 1This is a flowchart illustrating a multimodal vector index update method according to an exemplary embodiment of this application; Figure 2 This is a schematic diagram illustrating a partial update process in an exemplary embodiment of this application; Figure 3 This is a schematic diagram illustrating a global impact update process in an exemplary embodiment of this application; Figure 4 This is a schematic diagram illustrating a metadata update process according to an exemplary embodiment of this application; Figure 5 This is a schematic diagram illustrating a multimodal vector index update process according to an exemplary embodiment of this application; Figure 6 This is a schematic diagram of a multimodal vector index update apparatus shown in an exemplary embodiment of this application; Figure 7 This is a schematic diagram of the structure of a computer device shown in an exemplary embodiment of this application. Detailed Implementation

[0026] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0027] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0028] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0029] Research has revealed that vector retrieval is evolving from single-vector retrieval to hybrid retrieval that integrates keywords, vectors, and even graph relationships to balance semantic breadth and matching accuracy. Indexing strategies are shifting from static construction to dynamic adaptability to handle the continuous influx of real-time data. During this transformation, the maintenance mechanism of multimodal vector indexes has become a bottleneck restricting the performance and availability of multimodal retrieval. Vector indexes (such as Hierarchical Navigable Small World (HNSW) and Inverted File Index (IVF)) typically require periodic full reconstructions to absorb new data after initial construction, a time-consuming and resource-intensive process that can interrupt service. To address this, incremental index updates have emerged, employing methods such as parent document retrieval or asynchronous index construction to reduce the impact on online queries.

[0030] When updating indexes, most updates are performed periodically rather than responding in real time to individual update commands. Furthermore, a uniform approach is often used, either attempting updates locally on all updates or triggering a global adjustment after a certain number of updates have accumulated. This results in a failure to predict the scope of impact based on the spatial relationship between the update command content and the existing index structure at the initial arrival of the update command. Consequently, many small updates that could have been handled locally trigger unnecessary index structure traversal or probing, while critical update commands that could potentially cause distribution drift are ignored and cannot be identified in a timely manner. This leads to high computational redundancy in the index maintenance process, with a significant amount of resources consumed in unnecessary internal reorganization, thus crowding out resources available for providing external retrieval services and impacting overall throughput.

[0031] For schemes supporting incremental operations, when handling local insertions, simple heuristics (such as node capacity limits) are typically used to determine whether to split a node or merge adjacent regions. This approach lacks a quantitative consideration of the relationship between the current update and the stability of the local region structure. A simple insertion might trigger a node split simply because it reaches a fixed threshold. This split could further lead to an imbalance in the upper-level index structure, creating a chain reaction that spreads adjustments that should be confined to a small area to a wider range of index paths. This overreaction causes frequent fluctuations in the index structure, resulting in a long-term unstable topology. Consequently, the internal paths of the index change frequently, increasing the complexity and unpredictability of maintenance operations and severely impacting the consistency of retrieval performance. User query latency may fluctuate significantly depending on the different internal states of the index.

[0032] When implementing multimodal vector index updates, there are common problems such as coarse maintenance decisions and uncontrollable perturbation range, resulting in low retrieval quality, unstable retrieval performance, and poor long-term adaptability.

[0033] Based on the above research, this application provides a multimodal vector index update method. Upon receiving a data update instruction, it continuously quantifies and intelligently distributes the impact on the structure. By judging whether the data update instruction has a global impact, it processes the data accordingly. When there is no global impact, it only updates the index. When there is a global impact, it adjusts the core cluster center points in addition to updating the index, transforming index maintenance from passive repair to proactive control based on structure awareness, effectively improving the intelligence and accuracy of index updates. Furthermore, it adopts a boundary-constrained local reconstruction mechanism, restricting the update operation to be executed within the target sub-index structure, avoiding the spread of the operation, and enabling updates with minimal perturbation, reducing index structure fluctuations, and ensuring the efficiency, stability, and sustainability of cross-modal hybrid retrieval in highly dynamic data environments.

[0034] To facilitate understanding of this embodiment, a detailed description of the multimodal vector index update method disclosed in this application is provided first. The execution entity of the multimodal vector index update method provided in this application is generally an electronic device with a certain computing power. This electronic device can be a server, which can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, big data, and artificial intelligence platforms. In other embodiments, the electronic device can also be a terminal device, which can be a mobile device, terminal, handheld device, computing device, vehicle-mounted device, etc. In other embodiments, the multimodal vector index update method can also be applied to an implementation environment composed of a terminal and a server. Furthermore, the multimodal vector index update method can also be implemented by a processor calling computer-readable instructions stored in memory.

[0035] The following description, in conjunction with the accompanying drawings, illustrates a multimodal vector index update method provided in an embodiment of this application.

[0036] See Figure 1 The diagram shown is a flowchart illustrating a multimodal vector index update method according to an exemplary embodiment of this application. Figure 1 As shown in the figure, the multimodal vector index update method provided in this embodiment includes steps S101 to S105, wherein: S101: Receive a data update instruction for the multimodal vector index; the data update instruction carries the update operation type and update data information; the nodes in the multimodal vector index are used to store multimodal vectors; the multimodal vector index is pre-divided into multiple sub-index structures, and the sub-index structures are obtained based on the corresponding core cluster center points.

[0037] Here, the multimodal vector index is an index structure capable of mapping multimodal data such as text, images, and audio to the same vector space and managing them uniformly. The nodes in the multimodal vector index are used to store multimodal vectors, which are vectors extracted and fused from multimodal data (such as text, images, and audio) by an encoder. The type of the multimodal vector index can be a tree structure, graph structure, inverted index structure, etc., and is not specifically limited here.

[0038] The core cluster centers are pre-defined core points in the multimodal index, representing different data distribution regions. They form the basis for update routing and structure management, and each core cluster center corresponds to a multimodal vector. The sub-index structure is a local index region clustered from a single core cluster center, representing the smallest unit for independent local updates. Both the core cluster centers and the sub-index structure are recorded in the metadata of the multimodal vector index.

[0039] In this step, we do not simply passively receive data update instructions, but rather take over the instruction in a structured manner as the starting point to trigger subsequent incremental optimization links. In other words, we no longer treat the data update instruction as an isolated write operation that needs to be executed immediately, but rather as a task unit that can be analyzed, planned, and tracked.

[0040] Optionally, after receiving a data update instruction for a multimodal vector index, the data update instruction can be verified to determine whether the instruction applies to the currently valid index version.

[0041] In some possible implementations, the method further includes: Based on the update operation type and the update data information, an update event object corresponding to the data update instruction is generated; Obtain a snapshot of the metadata of the multimodal vector index; Perform a consistency check on the update event object and the metadata snapshot to obtain the check result; In the above steps, the data update instruction is mapped to an update event object. The update event object is a structured data carrier that records the update operation type and the update data information. In subsequent processing, intermediate results will be added to the update event object, such as operations on multimodal vectors, updates to impact metrics, and minimum structure-aware distance scores.

[0042] Obtain a metadata snapshot of the multimodal vector index, which may represent the current index version. Perform a consistency check on the update event object and the metadata snapshot to obtain a check result, determining whether the current index version is consistent with the version expected by the instruction. The check result includes pass or fail.

[0043] For example, a data update instruction indicates that a data item should be modified or deleted. The existence of the data item in the current index can be determined by a metadata snapshot. If it does not exist, the data update instruction is invalid, and the verification result is a failure.

[0044] Understandably, if the validation fails, it means the current index version is inconsistent with the version expected by the instruction, indicating that the index status has changed after receiving the data update instruction. In this case, execution cannot continue, as forcing an update might result in data loss or corruption. If the validation passes, it means the current index version is consistent with the version expected by the instruction, and execution can continue.

[0045] S102: Based on the updated data information, determine the operation multimodal vector corresponding to the data update instruction.

[0046] In the foregoing embodiments, a verification result can be obtained. Accordingly, determining the operation multimodal vector corresponding to the data update instruction based on the updated data information includes: If the verification result indicates that the data update instruction passes, the operation multimodal vector corresponding to the data update instruction is determined based on the updated data information, and the operation multimodal vector is added to the update event object.

[0047] It is understood that only when the verification passes and the current index version is consistent with the version expected by the instruction can subsequent processing be carried out, and the operation multimodal vector corresponding to the data update instruction is determined based on the updated data information.

[0048] Here, the operation multimodal vector is added to the update event object, thereby ensuring the continuous semantic position awareness of the corresponding data item during the index evolution process.

[0049] In this way, by constructing corresponding update event objects for data update instructions and performing consistency checks in conjunction with metadata snapshots of multimodal vector indexes, the matching between the updated data and the current metadata state of the index can be verified in advance before the index update operation is executed. Only after the verification is passed will the multimodal vector be determined and added to the update event object. This effectively avoids index update errors, structural disorder, and retrieval failures caused by metadata anomalies, state conflicts, or data mismatches, and effectively improves the reliability and stability of the multimodal vector index update process.

[0050] In other possible implementations, a retry mechanism or error message is triggered if the verification result indicates that the data has failed, in order to ensure data consistency.

[0051] For example, a new metadata snapshot of the multimodal vector index can be retrieved, and a consistency check can be performed on the update event object and the metadata snapshot. If a preset number of consecutive checks fail, it indicates that the index version has been modified frequently, and an error message can be displayed. The error message could be something like: "Update failed: The data being operated on was modified by another process before submission. Please retrieve the latest data and try again." This clarifies to the user that the error is not due to network issues or data loss, but rather a concurrent conflict in the index version. In this case, the user can retrieve the latest metadata version of the multimodal vector index and modify the data update instructions accordingly.

[0052] In some possible implementations, the method further includes: From the multiple core cluster centers, select the second core cluster center with the highest semantic correlation between the stored multimodal vector and the operational multimodal vector; Based on the vector distance between the operation multimodal vector and the second core cluster center point, and the coverage radius corresponding to the second core cluster center point, the update impact metric value corresponding to the data update instruction is determined, and the update impact metric value is added to the update event object.

[0053] In the above steps, multiple multimodal vectors corresponding to the core cluster centers can be extracted from the metadata, and the semantic correlation between each extracted multimodal vector and the operational multimodal vector can be determined. Then, the core cluster center to which the multimodal vector with the highest semantic correlation belongs is selected and determined as the second core cluster center.

[0054] The ratio of the vector distance between the operational multimodal vector and the second core cluster center point to the coverage radius corresponding to the second core cluster center point is determined as the update impact metric for the data update instruction. This update impact metric can characterize the strength of the potential perturbation of the index structure by the data update instruction at its source. The coverage radius corresponding to the second core cluster center point is the radius of the sub-index structure corresponding to the second core cluster center point.

[0055] For example, the updated impact metric can be determined by the following formula (1): (1) in, This indicates that updating affects the metric. This indicates the operation of multimodal vectors. This represents the stored multimodal vector corresponding to the second core cluster center point. This represents the vector distance between the multimodal vector and the second core cluster center. This represents the coverage radius corresponding to the second core cluster center point. Using formula (1), the data update instruction is embedded into the existing spatial structure of the index for quantization in the form of the ratio of vector distance to the cluster structure scale of the core cluster center point. The effect is that before the index is actually modified, a continuous quantized signal that is directly connected to the subsequent local update or global impact determination is formed, thereby avoiding coarse-grained diversion by simply relying on discrete operation types.

[0056] Here, the update impact metric is added to the update event object, thereby ensuring that there is continuous semantic basis related to the core clustering structure (i.e., the target sub-index structure) during the index evolution process.

[0057] In this way, by selecting the second core cluster center point with the highest semantic correlation to the multimodal vector, and combining the vector distance and the corresponding coverage radius to determine the update impact metric and add it to the update event object, the impact of this update on the index structure can be quantified before the update is executed. This provides a data basis for subsequent judgment on whether there is a global impact. Adding the update impact metric to the update event object enables unified management of update impact information and update events, thereby improving the accuracy of index updates.

[0058] In some possible implementations, the update operation type includes a add operation, a modify operation, or a delete operation; the data update information includes a data identifier, the multimodal raw data payload, and a timestamp; The step of determining the operation multimodal vector corresponding to the data update instruction based on the updated data information includes: When the update operation type is a new operation or a modification operation, the data identifier, the original multimodal data payload, and the timestamp are jointly encoded according to the spatial dimension and measurement method of the multimodal vector index to obtain the operation multimodal vector corresponding to the data update instruction. When the update operation type is a delete operation, a frozen vector matching the data identifier is found in the metadata of the multimodal vector index, and the frozen vector is determined as the operation multimodal vector.

[0059] In the above steps, the data update instructions can be parsed to eliminate ambiguities that may be introduced by different operation paths in the subsequent index maintenance phase. When parsing the data update instructions, the data update event can be mapped to one of three categories—addition update stream, modification update stream, or deletion update stream—based on the update operation type, thereby locking in a unique processing path.

[0060] If the update operation type is a new operation or a modification operation, the multimodal encoding and alignment module, which is of the same origin as the unified multimodal vector index, is invoked. The spatial dimension and metric method of the multimodal vector index are used to jointly encode the data identifier, the original multimodal data payload, and the timestamp, generating an operation multimodal vector that is completely consistent with the index spatial dimension and metric method.

[0061] If the update operation type is a deletion operation, it means that the corresponding multimodal vector is currently stored in the multimodal vector index. Therefore, there is no need to perform vector recalculation. Instead, the frozen vector that matches the data identifier is directly located through the data item location mapping table maintained in the metadata.

[0062] It can be understood that a frozen vector is a fixed vector that is determined and stored when it is first written to the multimodal index. When a new data item (such as an image or text) is added to the multimodal index for the first time, a corresponding operation multimodal vector can be encoded. This vector is not only added to the multimodal vector index but also stored in the data item location mapping table within the multimodal vector index's metadata. When a data update instruction for a delete operation is received, the original data may no longer exist, so it is necessary to query the data item location mapping table.

[0063] The data item location mapping table is index metadata in a key-value pair structure. The key indicates a data identifier, and the value indicates the multimodal vector corresponding to the data item and its storage location in the multimodal vector index. The storage location includes, but is not limited to, the core cluster center, the leaf node of the stored sub-index structure, and the offset within the node. Thus, by searching the data item location mapping table using the data identifier, a frozen vector matching the data identifier and its corresponding storage location can be found. This frozen vector is then determined as the operational multimodal vector.

[0064] Optionally, a structured deletion marker pointing to the storage location can be generated based on the storage location corresponding to the frozen vector.

[0065] In this way, operation multimodal vectors are generated differently for different update operation types such as adding, modifying, and deleting. When adding or modifying, the data identifier, multimodal original data payload, and timestamp are jointly encoded according to the index space dimension and measurement method to ensure that the generated vector is consistent with the index. When deleting, the matching frozen vector is found in the metadata through the data identifier, reducing the data processing steps caused by vector recalculation. Through the diversified operation multimodal vector determination method, different update operation types can be accurately adapted, ensuring the standardization and accuracy of operation multimodal vectors.

[0066] S103: Match the operation multimodal vector with multiple core cluster centers to determine the first core cluster center closest to the operation multimodal vector, and determine whether the data update instruction has a global impact based on whether the operation multimodal vector falls into the target sub-index structure corresponding to the first core cluster center.

[0067] In this step, the operation multimodal vector can be embedded into the clustering skeleton of the multimodal vector index for structural influence determination. The clustering skeleton consists of multiple core cluster centers that are pre-generated and continuously maintained during the index building phase. Each core cluster center corresponds to a stable sub-index coverage area (i.e., sub-index structure), and its corresponding storage vector and coverage radius are kept updated synchronously in the metadata.

[0068] Specifically, the operational multimodal vector is compared and evaluated in terms of distance with multiple pre-defined and recorded core cluster centers. Based on the comparison results, the first core cluster center closest to the operational multimodal vector is selected from the multiple core cluster centers, and the sub-index result corresponding to the first core cluster center is determined as the target sub-index structure. It is then determined whether the operational multimodal vector falls into the target sub-index structure. Based on the result of whether the operational multimodal vector falls into the target sub-index structure, it is determined whether the data update instruction has a global impact, thereby classifying this data update as a local update or a global impact update.

[0069] In some possible implementations, matching the operational multimodal vector with a plurality of core cluster centroids to determine the first core cluster centroid closest to the operational multimodal vector includes: Based on the vector distance between the operation multimodal vector and the second core cluster center point, and the coverage radius corresponding to the second core cluster center point, the operation shunting confidence level corresponding to the data update instruction is determined; For each core cluster center point, a structure-aware distance score of the operation multimodal vector relative to the core cluster center point is determined based on the vector distance between the operation multimodal vector and the core cluster center point, the coverage radius corresponding to the core cluster center point, and the operation shunting confidence. The core cluster center point corresponding to the minimum structure perception distance score is determined as the first core cluster center point that is closest to the operation multimodal vector, and the minimum structure perception distance score is added to the update event object corresponding to the data update instruction. The step of determining whether the data update instruction has a global impact based on whether the operational multimodal vector falls into the target sub-index structure corresponding to the first core cluster center point includes: If the minimum structure-aware distance score is less than or equal to the structure-aware distance threshold, it is determined that the operational multimodal vector falls into the target sub-index structure corresponding to the first core cluster center point; If the multimodal vector of the operation falls into the target sub-index structure corresponding to the first core cluster center point, it is determined that the data update instruction has no global impact.

[0070] In the above steps, the sum of the operation multimodal vector and the coverage radius corresponding to the second core cluster center point can be determined. Then, the ratio of the vector distance between the operation multimodal vector and the second core cluster center point to the sum can be determined. The difference between 1 and the ratio is determined as the operation split confidence score corresponding to the data update instruction. The operation split confidence score can measure the consistency between the operation multimodal vector and the current index cluster structure.

[0071] For example, the operation shunting confidence level can be determined by the following formula (2): (2) in, This indicates the confidence level of the operation split. This indicates the operation of multimodal vectors. This represents the stored multimodal vector corresponding to the second core cluster center point. This represents the vector distance between the multimodal vector and the second core cluster center. This represents the coverage radius corresponding to the second core cluster center point. Through formula (2), the vector distance is normalized to the joint space of the vector scale and the cluster scale, so that the operation split confidence can be quantified into a continuous value directly related to the degree of index structure fit. Without introducing additional traversal, this provides a stable and reusable metric for subsequent structure-aware distance scoring.

[0072] For each core cluster center, the ratio of the vector distance between the operational multimodal vector and the core cluster center, to the coverage radius corresponding to the core cluster center, and the operational split confidence score, is determined as the structure-aware distance score of the operational multimodal vector relative to the core cluster center. This structure-aware distance score transforms simple geometric distance into a decision metric that can be directly used to update splits. Furthermore, based on the structure-aware distance scores of the operational multimodal vector relative to each core cluster center, the minimum structure-aware distance score corresponding to the operational multimodal vector and the first core cluster center corresponding to the minimum structure-aware distance score are determined.

[0073] For example, the minimum structure perception distance score can be determined by the following formula (3): (3) in, This represents the minimum structure perception distance score. This indicates the operation of multimodal vectors. Indicates the first Each core cluster center corresponds to a stored multimodal vector. Represents the operation of the multimodal vector and the first Vector distance between the core cluster centers Indicates the first The coverage radius corresponding to each core cluster center point The distance term is represented by the operation split confidence. Using formula (3), the distance term is normalized according to the clustering scale and modulated using the operation split confidence, so that the data update instruction reflects both its semantic offset magnitude and structural matching stability in the numerical space.

[0074] In this embodiment, the structure-aware distance score is determined based on vector distance. In other embodiments, the structure-aware distance score can also be determined based on local density gradient or sub-index structure load change rate.

[0075] To avoid making overly aggressive or sluggish structural judgments based solely on the absolute distance between the operational multimodal vector and the nearest core cluster centroid, this embodiment compares the minimum structure-aware distance score with a preset structure-aware distance threshold in the index maintenance strategy. When the minimum structure-aware distance score is less than or equal to the structure-aware distance threshold, it is determined that the operational multimodal vector falls into the target sub-index structure corresponding to the first core cluster centroid, thus confirming that the data update instruction has no global impact, i.e., classifying the data update instruction as a local update. Conversely, when the minimum structure-aware distance score is greater than the structure-aware distance threshold, it is determined that the operational multimodal vector does not fall into the target sub-index structure corresponding to the first core cluster centroid, thus confirming that the data update instruction has a global impact that may exert extrapolation pressure on the existing cluster boundaries and cause significant changes in the index structure.

[0076] The specific value of the structure-aware distance threshold is set according to the actual index update needs, and is not specifically limited here.

[0077] In this way, by combining vector distance, the coverage radius corresponding to the core cluster center point, and the operation diversion confidence, the structure-aware distance score is determined, and then the nearest first core cluster center point is determined. Based on the comparison between the minimum structure-aware distance score and the preset threshold, it is determined whether the multimodal vector of the operation falls into the target sub-index structure. This achieves refined structure awareness and accurate diversion judgment for update operations, which not only improves the accuracy and rationality of core cluster center point matching, but also distinguishes whether the update operation has a global impact on the index as a whole, providing a basis for whether to adopt global adjustment in the future, effectively reducing unnecessary global index reconstruction, and ensuring the stability and update efficiency of the index structure.

[0078] S104: Within the target sub-index structure, perform the corresponding operation on the operation multimodal vector according to the update operation type to obtain the updated multimodal vector index.

[0079] In this step, an efficient local index reconstruction mechanism is adopted to freeze the target sub-index structure. The target sub-index structure is used as the boundary of operation, and the operation is performed only on the target sub-index structure. Within the target sub-index structure, the corresponding operation is performed on the multimodal vector of the operation according to the update operation type to prevent concurrent writes from spreading to other areas, thereby obtaining the updated multimodal vector index.

[0080] In some possible implementations, the step of performing corresponding operations on the multimodal vector according to the update operation type within the target sub-index structure to obtain the updated multimodal vector index includes: When the update operation type is a new operation, find the first leaf node with the smallest distance to the multimodal vector of the operation within the target sub-index structure; Based on the current number of carrying vectors of the first leaf node, the capacity limit of the first leaf node, and the minimum structure perception distance score corresponding to the operation multimodal vector, the local reconstruction pressure of the first leaf node is determined. When the local reconstruction pressure is less than or equal to the splitting threshold, the operational multimodal vector is inserted into the first leaf node to obtain the updated multimodal vector index.

[0081] In the above steps, for the new operation, a restricted proximity search is performed within the target sub-index structure to find the first leaf node with the smallest distance to the multimodal vector of the operation, and the first leaf node is used as a candidate insertion position. To avoid structural degradation caused by local density abrupt changes, the local reconstruction pressure of the first leaf node is determined before actual insertion to evaluate the load state and structural resilience of the first leaf node and to determine whether to trigger subtree splitting.

[0082] Specifically, the ratio of the current number of carrying vectors of the first leaf node to the upper limit of the capacity of the first leaf node can be determined, and the product of this ratio and the minimum structure perception distance score corresponding to the operation multimodal vector can be determined as the local reconstruction pressure of the first leaf node.

[0083] The specific value of the upper limit of the capacity of the first leaf node is set according to the index design requirements, and no specific limit is made here.

[0084] For example, the local reconstruction pressure of the first leaf node can be determined by the following formula (4): (4) in, Represents the first leaf node Local reconstruction pressure, Represents the first leaf node The current number of vectors carried. Represents the first leaf node Maximum capacity The minimum structure perception distance score is represented by formula (4). The load ratio of the first leaf node is coupled with the minimum structure perception distance score, so that the system triggers the minimum range of subtree splits only in local areas with high density and decreased structural stability, thus suppressing the index expansion caused by frequent splits.

[0085] The local reconstruction pressure is compared with the splitting threshold. If the local reconstruction pressure is less than or equal to the splitting threshold, it means that the subtree split has not been triggered. The operation multimodal vector can be directly inserted into the first leaf node to obtain the updated multimodal vector index.

[0086] The specific value of the splitting threshold is obtained based on statistical analysis of historical load data of the multimodal vector index. Specifically, the entire lifecycle of each leaf node from creation to splitting can be recorded, including the load ratio of each split (i.e., the ratio of the current number of vectors carried by the leaf node to its capacity limit) and the minimum structure-aware distance score corresponding to the multimodal vectors entering that leaf node at that time. An empirical distribution is fitted through a large amount of data, and then a reasonable trigger boundary is determined based on the tolerance for index expansion rate and query performance fluctuations, serving as the splitting threshold. In engineering practice, the specific value of the splitting threshold often falls between 1.5 and 2.0.

[0087] In this way, within the target sub-index structure, the nearest first leaf node is determined for the new operation. The local reconstruction pressure is determined by combining the number of vectors currently carried by the first leaf node, the capacity limit, and the minimum structure perception distance score. Vector insertion is performed when the local reconstruction pressure does not exceed the splitting threshold. This enables fine-grained evaluation and controllable updates of the leaf node insertion load, avoiding leaf node overload or index structure mutations caused by blind insertion. While ensuring efficient insertion of multimodal vectors, the internal structure of the sub-index remains stable, reducing the frequency of local reconstruction triggers and improving the reliability and retrieval efficiency of the index during dynamic updates.

[0088] In some possible implementations, the method further includes: When the local reconstruction pressure is greater than the splitting threshold, two target multimodal vectors are selected from the multimodal vectors stored in the first leaf node, and a split second leaf node is constructed with each target multimodal vector as the center. For each multimodal vector in the first leaf node other than the target multimodal vector, the target multimodal vector with the smallest distance from the two target multimodal vectors is determined, and the multimodal vector is assigned to the second leaf node corresponding to the target multimodal vector with the smallest distance. Based on the two split second leaf nodes, the pointers of the parent nodes of the first leaf nodes are adjusted to obtain the updated multimodal vector index.

[0089] In the above steps, if the local reconstruction pressure is greater than the splitting threshold, it indicates that the first leaf node is overloaded. At this time, the minimum range of subtree splitting is triggered, that is, the first leaf node is split into two second leaf nodes.

[0090] Two target multimodal vectors are selected from the multimodal vectors stored in the first leaf node. Specifically, the two multimodal vectors with the greatest distance from each other in the first leaf node can be determined as target multimodal vectors, or two target multimodal vectors can be selected through 2-means clustering to maximize the differentiation of subsequent data. A second leaf node is constructed with each target multimodal vector as the center. For each multimodal vector in the first leaf node (excluding the target multimodal vectors), the vector distance between each multimodal vector and each target multimodal vector is determined. The target multimodal vector with the smallest distance from the first target multimodal vector is determined, and this multimodal vector is assigned to the second leaf node corresponding to the target multimodal vector with the smallest distance. Based on the two split second leaf nodes, the pointers of the parent node of the first leaf node are adjusted. The pointers to the first leaf node recorded in the parent node of the first leaf node are replaced with pointers to the two second leaf nodes. The routing information of the parent node of the first leaf node is updated as needed to obtain the updated multimodal vector index.

[0091] In this way, when the local reconstruction pressure exceeds the splitting threshold, the splitting process is only performed on the first leaf node within the current target sub-index structure. The entire splitting and redistribution process is restricted to the target sub-index structure and will not spread to other sub-index structures, nor will it immediately trigger the global reconstruction of the upper-level index. This achieves the release of pressure on the leaf nodes while minimizing the disturbance to the overall index topology.

[0092] In some possible implementations, the step of performing corresponding operations on the multimodal vector according to the update operation type within the target sub-index structure to obtain the updated multimodal vector index includes: When the update operation type is a modification operation, the third leaf node to which the multimodal vector to be modified belongs is found within the target sub-index structure; Determine the subtree path where the third leaf node is located, and determine the fourth leaf node from the subtree path where the operation multimodal vector is to be inserted; The multimodal vector to be modified is deleted from the third leaf node, and the multimodal vector to be modified is inserted into the fourth leaf node to obtain the updated multimodal vector index.

[0093] The specific steps for finding the third leaf node to which the multimodal vector to be modified belongs in the above steps are similar to the steps for finding the frozen vector that matches the data identifier from the metadata of the multimodal vector index. For details, please refer to the foregoing embodiments, and they will not be repeated here.

[0094] After determining the third leaf node, starting from the third leaf node, backtrack upwards along the parent pointer until the root of the target sub-index structure, which is the level corresponding to the first core cluster center point. The path from the root of the target sub-index structure to the third leaf node is determined as the subtree path where the third leaf node is located.

[0095] Given the third leaf node and the subtree path, the multimodal vector to be modified is marked as pending replacement within the third leaf node. This ensures that if concurrent queries access this location during the modification process, it is known that the vector is being changed. Then, within the target sub-index structure, an insertion path lookup operation is performed on the subtree path to determine the fourth leaf node from which the multimodal vector to be inserted. The fourth leaf node may be the same as the third leaf node or a different leaf node.

[0096] When the fourth leaf node and the third leaf node are the same leaf node, the multimodal vector to be modified can be deleted from the third leaf node, and the operation multimodal vector can be inserted into the fourth leaf node. Local rearrangement can be performed in the fourth leaf node as needed, and the replacement state marker of the multimodal vector to be modified can be deleted to obtain the updated multimodal vector index.

[0097] If the fourth leaf node and the third leaf node are different leaf nodes, the multimodal vector to be modified and its state marker to be replaced are deleted from the third leaf node, and the operation multimodal vector is inserted into the fourth leaf node to obtain the updated multimodal vector index.

[0098] In this way, without changing the boundaries of the target sub-index structure, vector modifications are made based on the same subtree path, ensuring cluster consistency and continuity, and avoiding traversal and disturbance of the overall index topology.

[0099] In some possible implementations, the step of performing corresponding operations on the multimodal vector according to the update operation type within the target sub-index structure to obtain the updated multimodal vector index includes: In the case that the update operation type is a deletion operation, the fifth leaf node to which the operation multimodal vector belongs is found within the target sub-index structure; Based on the load state of the fifth leaf node, determine whether the fifth leaf node satisfies the sparsity condition; If the fifth leaf node does not meet the sparsity condition, the operation multimodal vector is deleted from the fifth leaf node to obtain the updated multimodal vector index. If the fifth leaf node satisfies the sparsity condition, the operation multimodal vector is deleted from the fifth leaf node, and the fifth leaf node after vector deletion is locally balanced to obtain the updated multimodal vector index.

[0100] The specific steps for finding the fifth leaf node to which the operational multimodal vector belongs are similar to the steps for finding the frozen vector matching the data identifier from the metadata of the multimodal vector index, as described in the foregoing embodiments, and will not be repeated here. Once the fifth leaf node is determined, a structured lazy deletion marker can be generated for the operational multimodal vector.

[0101] When determining whether the fifth leaf node meets the sparsity condition based on its load status, specifically, the number of currently active vectors in the fifth leaf node can be detected. The active vectors are the multimodal vectors currently in a valid state among those stored in the fifth leaf node, excluding multimodal vectors marked as deleted, pending modification, or invalid. The ratio of the number of currently active vectors in the fifth leaf node to its capacity limit is determined. If this ratio is greater than or equal to a sparsity threshold, the fifth leaf node is determined not to meet the sparsity condition; if the ratio is less than the sparsity threshold, the fifth leaf node is determined to meet the sparsity condition.

[0102] The specific value of the sparse threshold is set according to the needs of space utilization and query efficiency during index design, and is not specifically limited here.

[0103] If the fifth leaf node does not meet the sparsity condition, the operational multimodal vector and its structured lazy deletion marker can be directly deleted from the fifth leaf node to obtain the updated multimodal vector index. If the fifth leaf node meets the sparsity condition, it indicates that the fifth leaf node is relatively sparse. After deleting the operational multimodal vector and its structured lazy deletion marker from the fifth leaf node, a local balancing process is triggered on the fifth leaf node after vector deletion to obtain the updated multimodal vector index.

[0104] In this way, deletion operations do not need to traverse the entire index topology. All operations are limited to the target sub-index and do not backtrack to higher-level index structures, thereby limiting the scope of topology disturbance and avoiding unnecessary fluctuations in the global index structure.

[0105] In some possible implementations, the local balancing process for the fifth leaf node after vector deletion includes: Based on the load status of the fifth leaf node after vector deletion, determine whether the fifth leaf node after vector deletion meets the merging condition; If the fifth leaf node after vector deletion does not meet the merging condition, the layout of the fifth leaf node after vector deletion is adjusted to obtain the updated multimodal vector index. If the fifth leaf node after vector deletion meets the merging condition, the multimodal vector stored in the fifth leaf node after vector deletion is merged into the leaf node with the smallest distance from the fifth leaf node in the target sub-index structure to obtain the updated multimodal vector index.

[0106] In the above steps, based on the current number of vectors carried by the fifth leaf node after vector deletion and the upper limit of the fifth leaf node's capacity, the vector storage ratio of the fifth leaf node after vector deletion is determined. Specifically, the ratio of the current number of vectors carried by the fifth leaf node after vector deletion to the upper limit of the fifth leaf node's capacity can be determined as the vector storage ratio of the fifth leaf node after vector deletion. This vector storage ratio is compared with a merging threshold. If the vector storage ratio is less than the merging threshold, the fifth leaf node after vector deletion is determined to meet the merging condition. If the vector storage ratio is greater than or equal to the merging threshold, the fifth leaf node after vector deletion is determined not to meet the merging condition. The specific data for the merging threshold is set according to the actual index update needs and is not specifically limited here. For example, the merging threshold is 10%.

[0107] If the fifth leaf node after vector deletion does not meet the merging condition, it means that the number of remaining vectors in the fifth leaf node after vector deletion is large. The layout of the fifth leaf node after vector deletion can be adjusted, such as by reorganizing the internal storage layout of the node, to obtain the updated multimodal vector index without the need for merging across nodes.

[0108] If the fifth leaf node after vector deletion meets the merging condition, the multimodal vector stored in the fifth leaf node after vector deletion is merged into the leaf node with the smallest distance from the fifth leaf node in the target sub-index structure, resulting in an updated multimodal vector index. After merging, the fifth leaf node becomes an empty node, and can be removed from the pointer list of its parent node.

[0109] In this way, when performing local balancing on the fifth leaf node after deleting the vector, the entire operation is completely confined to the target sub-index, without spreading outwards or causing disturbances, and without triggering global reconstruction of the upper-level index. This effectively avoids the leaf node sparsity problem to maintain the rationality of the local structure, while also controlling the range of index topology changes and avoiding unnecessary fluctuations in the overall structure of the multimodal vector index.

[0110] In some possible implementations, when the fourth leaf node and the third leaf node are different leaf nodes, the third leaf node may become empty or have too low a load due to the deletion operation. Therefore, a local balancing process can be performed on the third leaf node after vector deletion to obtain an updated multimodal vector index. The specific steps for performing the local balancing process on the third leaf node after vector deletion are similar to the steps for performing the local balancing process on the fifth leaf node after vector deletion, and are detailed in the aforementioned embodiments, and will not be repeated here.

[0111] In some possible implementations, after the local reconstruction is completed and the updated multimodal vector index is obtained, the processing result of this data update instruction and a summary of local structural changes can be written into the index maintenance log.

[0112] To more clearly illustrate the process of partial updates, see [link to documentation]. Figure 2 This is a schematic diagram illustrating a partial update process in an exemplary embodiment of this application. Figure 2 As shown, the target sub-index structure is frozen, and update operation types are distinguished. When the update operation type is a modification operation, the third leaf node is searched within the target sub-index structure, and the fourth leaf node is determined from the subtree path of the third leaf node. The multimodal vector to be modified is deleted from the third leaf node, and the operation multimodal vector is inserted into the fourth leaf node to obtain the updated multimodal vector index. When the update operation type is an addition operation, the first leaf node is searched within the target sub-index structure, and the local reconstruction pressure of the first leaf node is determined. It is determined whether the local reconstruction pressure is greater than the splitting threshold. If so, the first leaf node is split into two second leaf nodes and vectors are assigned to obtain the updated multimodal vector index. If not, the operation multimodal vector is inserted into the first leaf node to obtain the updated multimodal vector index. When the update operation type is a deletion operation, the fifth leaf node is searched within the target sub-index structure. It is determined whether the fifth leaf node satisfies the sparsity condition. If not, the operation multimodal vector is deleted from the fifth leaf node to obtain the updated multimodal vector index. If so, the multimodal vector is deleted from the fifth leaf node, and a local balancing process is performed on the fifth leaf node after vector deletion to obtain the updated multimodal vector index. The specific steps are described in the aforementioned embodiment and will not be repeated here. This effectively avoids the chain reaction and index structure oscillation problems caused by simple threshold triggering in traditional incremental updates. By physically isolating the disturbance range and introducing dynamic pressure judgment, the impact of local updates is truly localized, effectively suppressing invalid index expansion and ensuring the overall stability of the index topology and the consistency of the retrieval path.

[0113] S105: When the data update instruction has a global impact, an unstable sub-index structure is determined from the multiple sub-index structures, the core cluster center point of the unstable sub-index structure is adjusted, and the sub-index structure is re-divided according to the adjusted core cluster center point to obtain the updated sub-index structure.

[0114] In this step, selective reconstruction is performed for cases where the data update instruction has a global impact. The most unstable sub-index structure is selected from the multiple sub-index structures, the core cluster center point of the unstable sub-index structure is adjusted, and the sub-index structure is re-divided according to the adjusted core cluster center point to obtain the updated sub-index structure to adapt to the data distribution evolution.

[0115] In some possible implementations, when the data update instruction has a global impact, determining an unstable sub-index structure from the plurality of sub-index structures, adjusting the core cluster centers of the unstable sub-index structure, and re-dividing the sub-index structure based on the adjusted core cluster centers to obtain an updated sub-index structure includes: If the data update instruction has a global impact, the update event object corresponding to the data update instruction is written into the buffer queue; Based on the update impact metric and minimum structure perception distance score recorded in each update event object in the buffer queue, the global evolution pressure corresponding to the buffer queue is determined. When the global evolution pressure is greater than the adjustment threshold and the load is in a low occupancy range, an unstable sub-index structure is determined from the sub-index structures corresponding to multiple update event objects in the buffer queue, and the update event object corresponding to the unstable sub-index structure is taken out from the buffer queue. The core cluster centers of the unstable substructure are adjusted, and the sub-index structure is re-divided based on the adjusted core cluster centers to obtain the updated sub-index structure.

[0116] Traditional dynamic index updates largely adapt to changes in data distribution passively and abruptly, such as re-clustering after detecting significant changes. This approach treats the continuous, slow evolution of data distribution as discrete, drastic reconstruction events. Between two reconstruction cycles, the index structure gradually becomes decoupled from the real data distribution, leading to a continuous decline in retrieval accuracy. Furthermore, when reconstruction occurs, it consumes significant resources and may interrupt service, causing a performance precipitate. This makes it impossible to maintain the continuity of retrieval service while allowing the index structure to follow the data distribution evolution in a gliding rather than leaping manner; the evolution process is not smooth.

[0117] In this embodiment of the disclosure, when it is determined that the data update instruction has a global impact, index structure adjustment is not immediately intervened. Instead, the update event objects corresponding to the data update instruction are written into a buffer queue in chronological order. The buffer queue is a delayed processing queue for structure evolution management. The buffer queue is used to temporarily store the update event objects corresponding to each data update instruction determined to have a global impact, so as to form a continuous observation of the data distribution evolution trend.

[0118] The update impact metric and minimum structure-aware distance score recorded in each update event object in the buffer queue are multiplied and accumulated. The ratio of the accumulated value to the number of update event objects in the buffer queue is determined as the global evolution pressure corresponding to the buffer queue, so as to quantify the comprehensive impact intensity of the current buffer state on the stability of the existing index.

[0119] For example, the global evolutionary pressure can be determined by the following formula (5): (5) in, Indicates global evolutionary pressure. This represents the update event object in the buffer queue. Minimum structure perception distance score. Indicates the first The update impact metric recorded in each update event object. Through formula (5), the structural offset intensity and update disturbance amplitude can be jointly accumulated, so that the structural pressure of the index in the time dimension can be characterized by a single continuous quantity, thereby avoiding coarse-grained triggering based solely on the number of updates, and accurately reflecting the true degree of compression of the index space shape by new data.

[0120] The global evolution pressure is compared with an adjustment threshold, and the load is detected. If the global evolution pressure is greater than the adjustment threshold and the load is in a low-occupancy range, asynchronous selective reconstruction of unstable substructures is performed. Unstable sub-index structures are determined from the sub-index structures corresponding to multiple update event objects in the buffer queue, and the update event objects corresponding to the unstable sub-index structures are retrieved from the buffer queue.

[0121] The specific data for adjusting the threshold is set according to the actual index update needs, and is not specifically limited here. When the Central Processing Unit (CPU) utilization is less than the corresponding threshold (e.g., 30%) and the query latency is less than a preset proportion of the normal value (e.g., 50%), the load is determined to be in a low utilization range.

[0122] Here, asynchronous processing means that when it is determined that the data update instruction has a global impact, the main thread simply writes the update event object corresponding to the data update instruction into the buffer queue and returns immediately without blocking the client's write request. The truly time-consuming selective reconstruction operation is executed independently and non-blockingly by the background thread when the global evolution pressure is greater than the adjustment threshold and the load is in a low occupancy range.

[0123] In this embodiment, the buffer triggering condition for determining the unstable sub-index structure is that the global evolution pressure is greater than the adjustment threshold and the load is in the low occupancy range, which is determined by the statistical distribution offset. In other embodiments, the determination of the unstable sub-index structure can also be triggered by the magnitude of retrieval performance degradation.

[0124] When determining unstable sub-index structures from the sub-index structures corresponding to multiple update event objects in the buffer queue, specifically, unstable sub-index structures are selected from the sub-index structures corresponding to multiple update event objects in the buffer queue based on the buffer vector density distribution corresponding to the buffer queue and the historical coverage radius fluctuations corresponding to each core cluster center point. The buffer density distribution is represented by a density heatmap formed by projecting the multimodal operation vectors recorded in all update event objects in the buffer queue onto a vector space, dividing it into several grid cells, and counting the number of vectors falling within each grid cell. The historical coverage radius fluctuations corresponding to each core cluster center point are the changes in the coverage radius of each core cluster center point within a historical time period. Based on the density heatmap and the changes in the coverage radius of each core cluster center point within a historical time period, sub-index structures corresponding to core cluster center points surrounded by high-density multimodal operation vectors and whose own coverage radius fluctuations are significantly increased are selected and determined as unstable sub-index structures. The number of unstable sub-index structures may be one or more. When there are multiple unstable sub-index structures, in a single asynchronous optimization, unstable sub-index structures can be selected in descending order of instability to adjust the core cluster center points.

[0125] After retrieving an unstable sub-index structure from the buffer queue each time, the core cluster center point of the unstable sub-index structure is adjusted, and the sub-index structure is re-divided based on the adjusted core cluster center point to obtain an updated sub-index structure. The step of determining unstable sub-index structures from the sub-index structures corresponding to multiple update event objects in the buffer queue is repeated until the step of determining the global evolution pressure corresponding to the buffer queue is based on the update impact metric and minimum structure perception distance score recorded in each update event object in the buffer queue. This process continues until the number of unstable sub-index structures processed reaches a first number, or the processing time reaches the target maintenance time, or the global evolution pressure is less than or equal to the adjustment threshold. At this point, the processing of unstable sub-index structures is stopped, the asynchronous processing is completed, and the updated sub-index structure is obtained.

[0126] The specific values ​​of the second quantity and the target maintenance time are set according to the actual index update needs, and are not specifically limited here.

[0127] In this way, by using globally controlled evolution based on buffering and asynchronous triggering, the performance fluctuations and structural oscillations caused by real-time global reconstruction are avoided, and the index structure is optimized smoothly and gradually. While ensuring that the index structure matches changes in data distribution, the interference with online retrieval services is minimized, and the stability and continuous service capability of multimodal vector indexes in highly dynamic data environments are improved.

[0128] In some possible implementations, adjusting the core cluster centers of the unstable substructure includes: Identify the target update event object that has a dominant influence on the core cluster centers of the unstable substructure from the buffer queue; Based on the operation multimodal vector and minimum structure-aware distance score recorded in the target update event object, the core cluster center points of the unstable substructure are adjusted to obtain the adjusted core cluster center points.

[0129] In the above steps, for each unstable sub-index structure retrieved from the buffer queue, target update event objects that have a dominant influence on the core cluster centers of the unstable sub-structure can be selected from multiple update event objects in the buffer queue. Specifically, for each update event object in the buffer queue, scores can be determined under three dimensions: spatial proximity, directional consistency, and event influence; the spatial proximity score, directional consistency score, and event influence score are weighted and summed to obtain the target score of the update event object; based on the target scores of each update event object, the target update event object is selected.

[0130] The spatial proximity score is determined based on the distance between the operational multimodal vector recorded in the updated event object and the multimodal vector corresponding to the core cluster center point of the unstable substructure. A larger distance results in a higher spatial proximity score. The directional consistency score is determined based on the offset direction of the operational multimodal vector recorded in the updated event object relative to the multimodal vector corresponding to the core cluster center point of the unstable substructure. A smaller offset results in a higher directional consistency score. The event influence score is the update influence metric recorded in the updated event object. A larger update influence metric results in a higher event influence score.

[0131] Optionally, update event objects with target scores greater than a score threshold can be identified as the target update event objects, and the second largest number of update event objects with the highest target scores can also be identified as the target update event objects. The specific values ​​of the score threshold and the second number are set according to the actual index update needs and are not specifically limited here.

[0132] Using the multimodal vector of operations and the minimum structure-aware distance score recorded in the target update event object, the core cluster center points of the unstable substructure are iteratively fine-tuned to obtain the adjusted core cluster center points.

[0133] For example, the iterative fine-tuning process can be represented by the following formula (6): (6) in, Indicates the adjusted number Each core cluster center corresponds to a stored multimodal vector. Indicates the number before adjustment Each core cluster center corresponds to a stored multimodal vector. This represents a set of objects that represent the target update event objects. Indicates the target update event object, This represents the multimodal vector of operations recorded in the target update event object. This represents the minimum structure-aware distance score recorded in the target update event object. The index evolution rate factor is used to control the displacement magnitude of the core cluster center points in a single asynchronous optimization. The value of the index evolution rate factor is determined based on historical adjustment data and the expected rate of data change, and is not specifically limited here. For example, it can be 0.01-0.1. Through formula (6), the weighted offset after direction normalization is used to ensure that the core cluster center points are adjusted in a controlled manner only along the direction of high structure offset density, thereby suppressing index instability caused by excessive drift while absorbing new data distribution characteristics.

[0134] In this way, by making adjustments in the above manner, the adjustment of the core cluster center points can be made to better reflect the actual changes in data distribution, avoid structural disturbances caused by indiscriminate global adjustments, maintain the stability of cluster boundaries while progressively optimizing the index structure, and ensure the smoothness of global index evolution.

[0135] In some implementations, after obtaining the updated sub-index structure, the reconstruction results and buffer statistical summaries can be recorded. The reconstruction results include the actual changes to the sub-index structure after fine-tuning and optimization, including but not limited to the multimodal vectors stored corresponding to the adjusted core cluster centers, the coverage radius corresponding to the adjusted core cluster centers, the version identifier of the affected sub-index structure, and structure adjustment records. The buffer statistical summaries include aggregated information for monitoring, diagnosis, and subsequent triggering decisions, including but not limited to the number of update event objects processed, the average minimum structure perception distance score, the average update impact metric, global evolutionary pressure, density distribution characteristics, and time windows.

[0136] To more clearly illustrate the process of partial updates, see [link to documentation]. Figure 3 This is a schematic diagram illustrating a global impact update process, as shown in an exemplary embodiment of this application. Figure 3As shown, when a data update instruction has a global impact, the update event object corresponding to the data update instruction is written to a buffer queue to determine the global evolution pressure corresponding to the buffer queue. It is then determined whether the global evolution pressure exceeds an adjustment threshold. If not, monitoring continues. If so, it is determined whether the load is in a low-occupancy range. If not, monitoring continues. If so, an unstable sub-index structure is identified, and the core cluster centers of the unstable sub-structure are adjusted. The sub-index structure is then re-divided based on the adjusted core cluster centers to obtain the updated sub-index structure. Specific steps are described in the aforementioned embodiment and will not be repeated here. This controlled adjustment of the core cluster centers of the unstable sub-index structure along the high-density offset direction helps overcome the passive and abrupt defects of reconstruction operations in traditional schemes. Through buffer accumulation effect judgment and low-load asynchronous execution, the index structure achieves smooth and gradual adaptation to the long-term evolution of data distribution, avoiding performance cliffs caused by frequent or drastic reconstructions and ensuring high availability and stable performance of the retrieval service.

[0137] In some possible implementations, the method further includes: If the updated multimodal vector index or the updated sub-index structure is obtained, the metadata of the multimodal vector index is updated according to the updated multimodal vector index or the updated sub-index structure to obtain the updated metadata. Based on the updated metadata, the target metadata of the multimodal vector index is determined.

[0138] In the above steps, after completing the targeted local index adjustment or asynchronous global optimization, the metadata of the multimodal vector index is updated synchronously, including but not limited to the sub-index identifiers that changed during the update process, the multimodal vectors stored corresponding to the core cluster centers, the coverage radius corresponding to the updated core cluster centers, and the storage location of the multimodal vectors (e.g., updating from the old node address to the new node address). The updated metadata is then written atomically to the index metadata storage area to obtain the updated metadata, thereby eliminating the state split between local structural adjustments and global structural evolution in the time dimension.

[0139] Based on the updated metadata, the target metadata of the multimodal vector index is determined, ensuring that subsequent cross-modal retrieval requests can be executed based on the latest and most optimized index path in the target metadata, thereby maintaining high-efficiency retrieval performance without excessively sacrificing index consistency.

[0140] In some possible implementations, the method further includes: Based on the core cluster centers before adjustment, the coverage radius corresponding to each core cluster center before adjustment, and the core cluster centers after adjustment, the index stability convergence metric of the updated metadata compared to the metadata before update is determined. The step of determining the target metadata for the multimodal vector index based on the updated metadata includes: If the index stability convergence metric is within a stable range, the updated metadata is determined as the target metadata of the multimodal vector index.

[0141] In the above steps, to avoid index path jitter caused by frequent metadata updates, an index stability convergence metric is determined compared to the metadata before the update. This metric is used to evaluate the overall offset of the updated metadata state compared to the metadata before the update. Specifically, the vector distances between each core cluster center point before and after the adjustment are summed to obtain a first summed value. The coverage radii corresponding to each core cluster center point before the adjustment are summed to obtain a second summed value. The ratio of the first summed value to the second summed value is determined as the index stability convergence metric.

[0142] For example, the index stability convergence metric is determined by the following formula (7): (7) in, This represents the index stability convergence metric. Indicates the adjusted number Each core cluster center corresponds to a stored multimodal vector. Indicates the number before adjustment Each core cluster center corresponds to a stored multimodal vector. Indicates the adjusted number The core cluster centers and the first one before adjustment Vector distance between the core cluster centers Indicates the number before adjustment The coverage radius corresponding to each core cluster center point. By using formula (7), the overall displacement of the core cluster center points is normalized to the original structural scale, so that the stability convergence level of the index structure within a maintenance cycle can be characterized by a single continuous value, ensuring that the index maintains predictable retrieval path depth and neighborhood connectivity while absorbing new data distribution characteristics.

[0143] When the index stability convergence metric is within a stable range, the updated metadata is determined as the target metadata of the multimodal vector index, and a new version identifier of the target metadata is published to the outside world. This allows subsequent cross-modal retrieval requests to be executed under a unified, converged index view, thus naturally connecting to the path selection and efficient retrieval process of real-time query vectors in the cross-modal hybrid retrieval stage.

[0144] The specific value of the stable interval is set according to the actual index update needs, and no specific limitation is made here.

[0145] Optionally, if the index stability convergence metric is not in the stable range, the sub-index structure can be adjusted, for example, by performing a local rollback, secondary fine-tuning, or internal correction of the structure for sub-index structures with excessive drift, and then redetermining the index stability convergence metric until the index stability convergence metric is in the stable range. The updated metadata is then determined as the target metadata of the multimodal vector index.

[0146] In this way, the updated metadata is confirmed as the target metadata only when the index's stable convergence metric is in a stable range. This enables quantitative verification of the convergence state after the index structure evolves, ensuring the stability of the structure after the adjustment of the core cluster center point and the reconstruction of the sub-index, and avoiding metadata failure or retrieval anomalies due to frequent structural fluctuations.

[0147] For a clearer illustration of the metadata update process, see [link to documentation]. Figure 4 This is a schematic diagram illustrating a metadata update process, as shown in an exemplary embodiment of this application. Figure 4 As shown, after obtaining the updated multimodal vector index or the updated sub-index structure, the metadata of the multimodal vector index is updated to obtain the updated metadata. The index stability convergence metric of the updated metadata compared to the metadata before the update is determined. It is then determined whether the index stability convergence metric is within a stable range. If not, the sub-index structure is adjusted, and the index stability convergence metric of the updated metadata compared to the metadata before the update is re-determined. If yes, the updated metadata is determined as the target metadata for the multimodal vector index. Specific steps are described in the aforementioned embodiments and will not be repeated here.

[0148] To more clearly illustrate the process of updating the multimodal vector index, see [link to documentation]. Figure 5 This is a schematic diagram illustrating a multimodal vector index update process, as shown in an exemplary embodiment of this application. Figure 5As shown, the system receives data update instructions for the multimodal vector index and determines whether these instructions have a global impact. This allows each data update instruction to be automatically and in real-time classified as either a local update or a globally impactful update, thus guiding it through different subsequent processing flows. This approach effectively avoids the inefficient, uniform processing of updates common in traditional methods, enabling pre-judgment before actual updates. It directs numerous regular updates conforming to the existing distribution to efficient local paths while accurately identifying critical updates with global impacts that could cause distribution drift. This significantly reduces unnecessary global computational probes from the outset, allowing for precise allocation of maintenance resources and ensuring low overhead and high efficiency in index updates.

[0149] If not, the updated multimodal vector index is obtained, and local updates are processed at high speed through real-time traffic splitting and constrained reconstruction to ensure the immediate availability and efficiency of the index. If so, based on the updated multimodal vector index, an updated sub-index structure is also obtained. Through buffered observation and controlled optimization, fundamental changes in data distribution are slowly absorbed to ensure the long-term adaptability and stability of the index. The metadata of the multimodal vector index is updated to determine the target metadata of the multimodal vector index. This allows the two updates to be coordinated and state-transferred through a unified quantitative indicator system (such as update impact metric, structure-aware distance score, global evolution pressure, etc.), and a consistent latest index view is provided externally by an atomic metadata convergence and publication mechanism. This disclosed embodiment simultaneously meets the combined requirements of high efficiency, high stability, and high adaptability for index maintenance in a highly dynamic environment. Compared with traditional solutions, it can achieve continuous optimization of index performance with minimal overhead during continuous retrieval services, representing a paradigm shift in dynamic vector index management from discrete reconstruction to continuous intelligent evolution. The specific steps are described in the foregoing embodiments and will not be repeated here.

[0150] The multimodal vector index update method provided in this application continuously quantifies and intelligently distributes the structural impact of a data update instruction upon receipt. It determines whether the data update instruction has a global impact and processes it accordingly. If no global impact exists, only the index is updated. If a global impact exists, adjustments to the core cluster centers are added to the index update, transforming index maintenance from passive repair to proactive, structure-aware control, effectively improving the intelligence and accuracy of index updates. Furthermore, a boundary-constrained local reconstruction mechanism is employed, restricting update operations to within the target sub-index structure to prevent operation propagation. This minimizes disturbances during updates, reducing index structure fluctuations. Adjustments that disrupt cluster boundaries are only performed when a global impact exists, reducing cascading disturbances to the overall index topology at the structural level. This allows for simultaneous maintenance of index consistency and high-throughput write capabilities in highly dynamic data scenarios, ensuring the efficiency, stability, and sustainability of cross-modal hybrid retrieval in highly dynamic data environments.

[0151] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0152] Corresponding to the aforementioned embodiments of the multimodal vector index update method, this application also provides embodiments of a multimodal vector index update apparatus.

[0153] Please see Figure 6 This is a schematic diagram illustrating a multimodal vector index update apparatus according to an exemplary embodiment of this application. Figure 6 As shown in the figure, the multimodal vector index update device 600 provided in this application embodiment includes: The instruction receiving module 601 is used to obtain a data update instruction for the multimodal vector index; the data update instruction carries an update operation type and update data information; the nodes in the multimodal vector index are used to store multimodal vectors; the multimodal vector index is pre-divided into multiple sub-index structures, and the sub-index structures are obtained based on the corresponding core cluster center points. The vector determination module 602 is used to determine the operation multimodal vector corresponding to the data update instruction based on the updated data information. The global judgment module 603 is used to match the operation multimodal vector with multiple core cluster center points, determine the first core cluster center point that is closest to the operation multimodal vector, and determine whether the data update instruction has a global impact based on whether the operation multimodal vector falls into the target sub-index structure corresponding to the first core cluster center point. The local update module 604 is used to perform corresponding operations on the multimodal vector of the operation according to the update operation type within the target sub-index structure to obtain the updated multimodal vector index; The global update module 605 is used to determine an unstable sub-index structure from the multiple sub-index structures when the data update instruction has a global impact, adjust the core cluster center point of the unstable sub-index structure, and re-divide the sub-index structure according to the adjusted core cluster center point to obtain the updated sub-index structure.

[0154] In an optional implementation, the instruction receiving module 601 is further configured to: Based on the update operation type and the update data information, an update event object corresponding to the data update instruction is generated; Obtain a snapshot of the metadata of the multimodal vector index; Perform a consistency check on the update event object and the metadata snapshot to obtain the check result; The vector determination module 602 is specifically used for: If the verification result indicates that the data update instruction passes, the operation multimodal vector corresponding to the data update instruction is determined based on the updated data information, and the operation multimodal vector is added to the update event object.

[0155] In an optional implementation, the vector determination module 602 is further configured to: From the multiple core cluster centers, select the second core cluster center with the highest semantic correlation between the stored multimodal vector and the operational multimodal vector; Based on the vector distance between the operation multimodal vector and the second core cluster center point, and the coverage radius corresponding to the second core cluster center point, the update impact metric value corresponding to the data update instruction is determined, and the update impact metric value is added to the update event object.

[0156] In one optional implementation, the update operation type includes a add operation, a modify operation, or a delete operation; the data update information includes a data identifier, multimodal raw data payload, and a timestamp. The vector determination module 602 is specifically used for: When the update operation type is a new operation or a modification operation, the data identifier, the original multimodal data payload, and the timestamp are jointly encoded according to the spatial dimension and measurement method of the multimodal vector index to obtain the operation multimodal vector corresponding to the data update instruction. When the update operation type is a delete operation, a frozen vector matching the data identifier is found in the metadata of the multimodal vector index, and the frozen vector is determined as the operation multimodal vector.

[0157] In an optional implementation, when the global judgment module 603 matches the operational multimodal vector with multiple core cluster centers to determine the first core cluster center closest to the operational multimodal vector, it is specifically used for: Based on the vector distance between the operation multimodal vector and the second core cluster center point, and the coverage radius corresponding to the second core cluster center point, the operation shunting confidence level corresponding to the data update instruction is determined; For each core cluster center point, a structure-aware distance score of the operation multimodal vector relative to the core cluster center point is determined based on the vector distance between the operation multimodal vector and the core cluster center point, the coverage radius corresponding to the core cluster center point, and the operation shunting confidence. The core cluster center point corresponding to the minimum structure perception distance score is determined as the first core cluster center point that is closest to the operation multimodal vector, and the minimum structure perception distance score is added to the update event object corresponding to the data update instruction. The global judgment module 603, when used to determine whether the data update instruction has a global impact based on whether the operational multimodal vector falls into the target sub-index structure corresponding to the first core cluster center point, is specifically used for: If the minimum structure-aware distance score is less than or equal to the structure-aware distance threshold, it is determined that the operational multimodal vector falls into the target sub-index structure corresponding to the first core cluster center point; If the multimodal vector of the operation falls into the target sub-index structure corresponding to the first core cluster center point, it is determined that the data update instruction has no global impact.

[0158] In one optional implementation, the local update module 604 is specifically used for: When the update operation type is a new operation, find the first leaf node with the smallest distance to the multimodal vector of the operation within the target sub-index structure; Based on the current number of carrying vectors of the first leaf node, the capacity limit of the first leaf node, and the minimum structure perception distance score corresponding to the operation multimodal vector, the local reconstruction pressure of the first leaf node is determined. When the local reconstruction pressure is less than or equal to the splitting threshold, the operational multimodal vector is inserted into the first leaf node to obtain the updated multimodal vector index.

[0159] In an optional implementation, the local update module 604 is further configured to: When the local reconstruction pressure is greater than the splitting threshold, two target multimodal vectors are selected from the multimodal vectors stored in the first leaf node, and a split second leaf node is constructed with each target multimodal vector as the center. For each multimodal vector in the first leaf node other than the target multimodal vector, the target multimodal vector with the smallest distance from the two target multimodal vectors is determined, and the multimodal vector is assigned to the second leaf node corresponding to the target multimodal vector with the smallest distance. Based on the two split second leaf nodes, the pointers of the parent nodes of the first leaf nodes are adjusted to obtain the updated multimodal vector index.

[0160] In one optional implementation, the local update module 604 is specifically used for: When the update operation type is a modification operation, the third leaf node to which the multimodal vector to be modified belongs is found within the target sub-index structure; Determine the subtree path where the third leaf node is located, and determine the fourth leaf node from the subtree path where the operation multimodal vector is to be inserted; The multimodal vector to be modified is deleted from the third leaf node, and the multimodal vector to be modified is inserted into the fourth leaf node to obtain the updated multimodal vector index.

[0161] In one optional implementation, the local update module 604 is specifically used for: In the case that the update operation type is a deletion operation, the fifth leaf node to which the operation multimodal vector belongs is found within the target sub-index structure; Based on the load state of the fifth leaf node, determine whether the fifth leaf node satisfies the sparsity condition; If the fifth leaf node does not meet the sparsity condition, the operation multimodal vector is deleted from the fifth leaf node to obtain the updated multimodal vector index. If the fifth leaf node satisfies the sparsity condition, the operation multimodal vector is deleted from the fifth leaf node, and the fifth leaf node after vector deletion is locally balanced to obtain the updated multimodal vector index.

[0162] In one optional implementation, the local update module 604, when performing local balancing on the fifth leaf node after vector deletion, is specifically used for: Based on the load status of the fifth leaf node after vector deletion, determine whether the fifth leaf node after vector deletion meets the merging condition; If the fifth leaf node after vector deletion does not meet the merging condition, the layout of the fifth leaf node after vector deletion is adjusted to obtain the updated multimodal vector index. If the fifth leaf node after vector deletion meets the merging condition, the multimodal vector stored in the fifth leaf node after vector deletion is merged into the leaf node with the smallest distance from the fifth leaf node in the target sub-index structure to obtain the updated multimodal vector index.

[0163] In one optional implementation, the global update module 605 is specifically used for: If the data update instruction has a global impact, the update event object corresponding to the data update instruction is written into the buffer queue; Based on the update impact metric and minimum structure perception distance score recorded in each update event object in the buffer queue, the global evolution pressure corresponding to the buffer queue is determined. When the global evolution pressure is greater than the adjustment threshold and the load is in a low occupancy range, an unstable sub-index structure is determined from the sub-index structures corresponding to multiple update event objects in the buffer queue, and the update event object corresponding to the unstable sub-index structure is taken out from the buffer queue. The core cluster centers of the unstable substructure are adjusted, and the sub-index structure is re-divided based on the adjusted core cluster centers to obtain the updated sub-index structure.

[0164] In one optional implementation, when adjusting the core clustering centers of the unstable substructure, the global update module 605 is specifically used for: Identify the target update event object that has a dominant influence on the core cluster centers of the unstable substructure from the buffer queue; Based on the operation multimodal vector and minimum structure-aware distance score recorded in the target update event object, the core cluster center points of the unstable substructure are adjusted to obtain the adjusted core cluster center points.

[0165] In an optional embodiment, the multimodal vector index update device 600 further includes a metadata update module 606, the metadata update module 606 being used for: If the updated multimodal vector index or the updated sub-index structure is obtained, the metadata of the multimodal vector index is updated according to the updated multimodal vector index or the updated sub-index structure to obtain the updated metadata. Based on the updated metadata, the target metadata of the multimodal vector index is determined.

[0166] In an optional implementation, the metadata update module 606 is further configured to: Based on the core cluster centers before adjustment, the coverage radius corresponding to each core cluster center before adjustment, and the core cluster centers after adjustment, the index stability convergence metric of the updated metadata compared to the metadata before update is determined. When the metadata update module 606 determines the target metadata of the multimodal vector index based on the updated metadata, it is specifically used for: If the index stability convergence metric is within a stable range, the updated metadata is determined as the target metadata of the multimodal vector index.

[0167] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0168] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0169] Based on the same technical concept, this application also provides a computer device 700, referring to... Figure 7 The diagram shown is a schematic representation of the structure of a computer device according to an exemplary embodiment of this application, comprising: The processor 710, memory 720, and bus 730 are included. The memory 720 is used to store execution instructions and includes main memory 721 and external memory 722. The main memory 721, also known as internal memory, is used to temporarily store the operation data in the processor 710 and the data exchanged with external memory 722 such as hard disk. The processor 710 exchanges data with external memory 722 through main memory 721.

[0170] In this embodiment, the memory 720 is specifically used to store application code that executes the solution of this application, and its execution is controlled by the processor 710. That is, when the computer device 700 is running, the processor 710 communicates with the memory 720 through the bus 730, or the processor 710 communicates with the memory 720 through other means, so that the processor 710 executes the application code stored in the memory 720, and then executes the steps of the multimodal vector index update method described in any of the foregoing embodiments.

[0171] The memory 720 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0172] Processor 710 may be an integrated circuit chip with signal processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0173] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the computer device 700. In other embodiments of this application, the computer device 700 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0174] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the multimodal vector index update method described in the above-described method embodiments. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0175] This disclosure also provides a computer program product storing a computer program. When the computer program is run by a processor, it executes the steps of the multimodal vector index update method provided in any of the above embodiments of this disclosure. For details, please refer to the above method embodiments, which will not be repeated here.

[0176] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium, which can be a volatile or non-volatile computer-readable storage medium. In another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0177] Furthermore, embodiments of the subject matter and functional operation described in this specification can be implemented in the following ways: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. Alternatively or additionally, program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiving device for execution by the data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or combinations thereof.

[0178] The processing and logic flow described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform corresponding functions by operating on input data and generating output. The processing and logic flow can also be executed by dedicated logic circuitry—such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits), and the device can also be implemented as dedicated logic circuitry.

[0179] Suitable computers for executing computer programs include, for example, general-purpose and / or special-purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory and / or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer will be operatively coupled to such mass storage devices to receive data from or transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0180] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0181] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0182] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0183] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0184] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A multi-modal vector index update method, characterized by, The method includes: The system receives a data update instruction for a multimodal vector index; the data update instruction carries an update operation type and update data information; the nodes in the multimodal vector index are used to store multimodal vectors; the multimodal vector index is pre-divided into multiple sub-index structures, and the sub-index structures are obtained based on the corresponding core cluster center points. Based on the updated data information, determine the operation multimodal vector corresponding to the data update instruction; The operation multimodal vector is matched with multiple core cluster centers to determine the first core cluster center closest to the operation multimodal vector. Based on whether the operation multimodal vector falls into the target sub-index structure corresponding to the first core cluster center, it is determined whether the data update instruction has a global impact. Within the target sub-index structure, the corresponding operation is performed on the operation multimodal vector according to the update operation type to obtain the updated multimodal vector index; When the data update instruction has a global impact, an unstable sub-index structure is determined from the multiple sub-index structures, the core cluster center point of the unstable sub-index structure is adjusted, and the sub-index structure is re-divided according to the adjusted core cluster center point to obtain the updated sub-index structure.

2. The method of claim 1, wherein, The method further includes: Based on the update operation type and the update data information, an update event object corresponding to the data update instruction is generated; Obtain a snapshot of the metadata of the multimodal vector index; Perform a consistency check on the update event object and the metadata snapshot to obtain the check result; The step of determining the operation multimodal vector corresponding to the data update instruction based on the updated data information includes: If the verification result indicates that the data update instruction passes, the operation multimodal vector corresponding to the data update instruction is determined based on the updated data information, and the operation multimodal vector is added to the update event object.

3. The method of claim 2, wherein, The method further includes: From the multiple core cluster centers, select the second core cluster center with the highest semantic correlation between the stored multimodal vector and the operational multimodal vector; Based on the vector distance between the operation multimodal vector and the second core cluster center point, and the coverage radius corresponding to the second core cluster center point, the update impact metric value corresponding to the data update instruction is determined, and the update impact metric value is added to the update event object.

4. The method of claim 1, wherein, The update operation types include add, modify, or delete operations; the data update information includes data identifier, multimodal raw data payload, and timestamp; The step of determining the operation multimodal vector corresponding to the data update instruction based on the updated data information includes: When the update operation type is a new operation or a modification operation, the data identifier, the original multimodal data payload, and the timestamp are jointly encoded according to the spatial dimension and measurement method of the multimodal vector index to obtain the operation multimodal vector corresponding to the data update instruction. When the update operation type is a delete operation, a frozen vector matching the data identifier is found in the metadata of the multimodal vector index, and the frozen vector is determined as the operation multimodal vector.

5. The method of claim 1, wherein, The step of matching the operational multimodal vector with multiple core cluster centers to determine the first core cluster center closest to the operational multimodal vector includes: Based on the vector distance between the operation multimodal vector and the second core cluster center point, and the coverage radius corresponding to the second core cluster center point, the operation shunting confidence level corresponding to the data update instruction is determined; For each core cluster center point, a structure-aware distance score of the operation multimodal vector relative to the core cluster center point is determined based on the vector distance between the operation multimodal vector and the core cluster center point, the coverage radius corresponding to the core cluster center point, and the operation shunting confidence. The core cluster center point corresponding to the minimum structure perception distance score is determined as the first core cluster center point that is closest to the operation multimodal vector, and the minimum structure perception distance score is added to the update event object corresponding to the data update instruction. The step of determining whether the data update instruction has a global impact based on whether the operational multimodal vector falls into the target sub-index structure corresponding to the first core cluster center point includes: If the minimum structure-aware distance score is less than or equal to the structure-aware distance threshold, it is determined that the operational multimodal vector falls into the target sub-index structure corresponding to the first core cluster center point; If the multimodal vector of the operation falls into the target sub-index structure corresponding to the first core cluster center point, it is determined that the data update instruction has no global impact.

6. The method according to claim 1, characterized in that, Within the target sub-index structure, performing corresponding operations on the multimodal vector according to the update operation type to obtain the updated multimodal vector index includes: When the update operation type is a new operation, find the first leaf node with the smallest distance to the multimodal vector of the operation within the target sub-index structure; Based on the current number of carrying vectors of the first leaf node, the capacity limit of the first leaf node, and the minimum structure perception distance score corresponding to the operation multimodal vector, the local reconstruction pressure of the first leaf node is determined. When the local reconstruction pressure is less than or equal to the splitting threshold, the operational multimodal vector is inserted into the first leaf node to obtain the updated multimodal vector index.

7. The method according to claim 6, characterized in that, The method further includes: When the local reconstruction pressure is greater than the splitting threshold, two target multimodal vectors are selected from the multimodal vectors stored in the first leaf node, and a split second leaf node is constructed with each target multimodal vector as the center. For each multimodal vector in the first leaf node other than the target multimodal vector, the target multimodal vector with the smallest distance from the two target multimodal vectors is determined, and the multimodal vector is assigned to the second leaf node corresponding to the target multimodal vector with the smallest distance. Based on the two split second leaf nodes, the pointers of the parent nodes of the first leaf nodes are adjusted to obtain the updated multimodal vector index.

8. The method according to claim 1, characterized in that, Within the target sub-index structure, performing corresponding operations on the multimodal vector according to the update operation type to obtain the updated multimodal vector index includes: When the update operation type is a modification operation, the third leaf node to which the multimodal vector to be modified belongs is found within the target sub-index structure; Determine the subtree path where the third leaf node is located, and determine the fourth leaf node from the subtree path where the operation multimodal vector is to be inserted; The multimodal vector to be modified is deleted from the third leaf node, and the multimodal vector to be modified is inserted into the fourth leaf node to obtain the updated multimodal vector index.

9. The method according to claim 1, characterized in that, Within the target sub-index structure, performing corresponding operations on the multimodal vector according to the update operation type to obtain the updated multimodal vector index includes: In the case that the update operation type is a deletion operation, the fifth leaf node to which the operation multimodal vector belongs is found within the target sub-index structure; Based on the load state of the fifth leaf node, determine whether the fifth leaf node satisfies the sparsity condition; If the fifth leaf node does not meet the sparsity condition, the operation multimodal vector is deleted from the fifth leaf node to obtain the updated multimodal vector index. If the fifth leaf node satisfies the sparsity condition, the operation multimodal vector is deleted from the fifth leaf node, and the fifth leaf node after vector deletion is locally balanced to obtain the updated multimodal vector index.

10. The method according to claim 9, characterized in that, The local balancing process for the fifth leaf node after vector deletion includes: Based on the load status of the fifth leaf node after vector deletion, determine whether the fifth leaf node after vector deletion meets the merging condition; If the fifth leaf node after vector deletion does not meet the merging condition, the layout of the fifth leaf node after vector deletion is adjusted to obtain the updated multimodal vector index. If the fifth leaf node after vector deletion meets the merging condition, the multimodal vector stored in the fifth leaf node after vector deletion is merged into the leaf node with the smallest distance from the fifth leaf node in the target sub-index structure to obtain the updated multimodal vector index.

11. The method according to claim 1, characterized in that, When the data update instruction has a global impact, the method involves determining an unstable sub-index structure from the multiple sub-index structures, adjusting the core cluster centers of the unstable sub-index structure, and re-dividing the sub-index structure based on the adjusted core cluster centers to obtain the updated sub-index structure, including: If the data update instruction has a global impact, the update event object corresponding to the data update instruction is written into the buffer queue; Based on the update impact metric and minimum structure perception distance score recorded in each update event object in the buffer queue, the global evolution pressure corresponding to the buffer queue is determined. When the global evolution pressure is greater than the adjustment threshold and the load is in a low occupancy range, an unstable sub-index structure is determined from the sub-index structures corresponding to multiple update event objects in the buffer queue, and the update event object corresponding to the unstable sub-index structure is taken out from the buffer queue. The core cluster centers of the unstable substructure are adjusted, and the sub-index structure is re-divided based on the adjusted core cluster centers to obtain the updated sub-index structure.

12. The method according to claim 11, characterized in that, The adjustment of the core cluster centers of the unstable substructure includes: Identify the target update event object that has a dominant influence on the core cluster centers of the unstable substructure from the buffer queue; Based on the operation multimodal vector and minimum structure-aware distance score recorded in the target update event object, the core cluster center points of the unstable substructure are adjusted to obtain the adjusted core cluster center points.

13. The method according to claim 1, characterized in that, The method further includes: If the updated multimodal vector index or the updated sub-index structure is obtained, the metadata of the multimodal vector index is updated according to the updated multimodal vector index or the updated sub-index structure to obtain the updated metadata. Based on the updated metadata, the target metadata of the multimodal vector index is determined.

14. The method according to claim 13, characterized in that, The method further includes: Based on the core cluster centers before adjustment, the coverage radius corresponding to each core cluster center before adjustment, and the core cluster centers after adjustment, the index stability convergence metric of the updated metadata compared to the metadata before update is determined. The step of determining the target metadata for the multimodal vector index based on the updated metadata includes: If the index stability convergence metric is within a stable range, the updated metadata is determined as the target metadata of the multimodal vector index.

15. A multimodal vector index update device, characterized in that, The device includes: The instruction receiving module is used to obtain data update instructions for the multimodal vector index; the data update instructions carry update operation type and update data information; the nodes in the multimodal vector index are used to store multimodal vectors; the multimodal vector index is pre-divided into multiple sub-index structures, and the sub-index structures are obtained based on the corresponding core cluster center points; The vector determination module is used to determine the operation multimodal vector corresponding to the data update instruction based on the updated data information. The global judgment module is used to match the operation multimodal vector with multiple core cluster centers, determine the first core cluster center closest to the operation multimodal vector, and determine whether the data update instruction has a global impact based on whether the operation multimodal vector falls into the target sub-index structure corresponding to the first core cluster center. A local update module is used to perform corresponding operations on the multimodal vector of the operation according to the update operation type within the target sub-index structure to obtain the updated multimodal vector index; The global update module is used to determine the unstable sub-index structure from the multiple sub-index structures when the data update instruction has a global impact, adjust the core cluster center point of the unstable sub-index structure, and re-divide the sub-index structure according to the adjusted core cluster center point to obtain the updated sub-index structure.

16. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the multimodal vector index update method according to any one of claims 1 to 14.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the multimodal vector index update method according to any one of claims 1 to 14.

Images