An efficient retrieval method and system for intelligent material databases based on multimodal vector fusion
By constructing a real-time adaptive closed loop and utilizing multimodal vector fusion and intent determination networks, the retrieval weights are dynamically adjusted and the index is updated. This solves the problem of the inability to capture user intent in real time in existing technologies, realizes adaptive optimization and continuous learning of the retrieval system, and improves retrieval performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HANGZHOU SHEARING INK TECHNOLOGY CO LTD
- Filing Date
- 2026-02-10
- Publication Date
- 2026-05-26
AI Technical Summary
Existing technologies cannot capture users' potential intentions in real time, which prevents retrieval systems from self-optimizing and from effectively utilizing implicit feedback, thus limiting the long-term improvement of retrieval results.
A real-time adaptive closed loop of 'perception-decision-execution-evolution' is constructed. The retrieval weights are adjusted in real time through multimodal vector fusion and intent determination network, and the incremental index blocks are combined for dynamic coordination and index updates to achieve adaptive optimization of the system.
It achieves real-time personalized adaptation and adaptive optimization of the retrieval process, improving the immediate relevance of retrieval results and user satisfaction. At the same time, it has the ability to continuously learn and evolve, ensuring retrieval performance and response speed.
Smart Images
Figure CN122087191A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of information retrieval technology, specifically to an efficient retrieval method and system for intelligent material databases based on multimodal vector fusion, which is particularly suitable for scenarios where real-time understanding of retrieval intent and adaptive optimization of the system are achieved by analyzing users' implicit interaction behavior. Background Technology
[0002] With the development of multimedia technology, the materials stored in intelligent material libraries are increasingly exhibiting multimodal characteristics, such as images, text, audio, and video. How to efficiently and accurately retrieve materials that match user intent from massive amounts of multimodal data has become a key challenge for the industry.
[0003] Current mainstream solutions are primarily based on multimodal vector fusion technology, which embeds materials from different modalities (such as image features and text features) into a unified vector space and performs retrieval by calculating the similarity between the query vector and the material vectors. However, existing technologies have the following fundamental limitations:
[0004] First, at the interaction level, the system cannot capture users' latent intentions in real time. Most systems rely on explicit feedback to optimize retrieval, which disrupts the user's natural interaction flow and requires users to precisely express their vague intentions, resulting in limited effectiveness. More importantly, the rich preference information contained in a large number of implicit user behaviors (such as hovering and dwelling) is completely wasted. The system cannot effectively perceive and respond to subtle changes in user interests during the interaction process.
[0005] Furthermore, at the system architecture level, this leads to a deeper dilemma: the retrieval system is static and cannot evolve. Because it cannot effectively utilize implicit feedback, the system lacks a mechanism to continuously learn from user interactions and optimize itself. Its core index remains fixed once established, failing to reflect the changing preferences of the user group over time, preventing the system from becoming "smarter with use" and fundamentally limiting the long-term potential for improvement in retrieval performance.
[0006] Therefore, there is an urgent need for an intelligent retrieval solution that can understand users' implicit intentions in real time and enable the retrieval system to continuously learn and adapt and evolve. Summary of the Invention
[0007] To address the dual shortcomings of existing technologies, namely the inability to capture users' potential intentions in real time and the inability of static systems to evolve, this invention aims to provide an intelligent retrieval scheme by constructing a real-time adaptive closed loop of "perception-decision-execution-evolution," thereby simultaneously achieving real-time personalized adaptation for a single retrieval and continuous learning and evolution of the system.
[0008] To address the aforementioned problems, in a first aspect, this invention provides an efficient intelligent material library retrieval method based on multimodal vector fusion, comprising the following steps:
[0009] S1: In response to the user's initial query, perform a multimodal search from the intelligent material library to obtain and display the initial result set;
[0010] S2: Monitor users' implicit interaction behavior with the initial result set in real time, and generate behavioral sequence features based on this;
[0011] S3: Input the behavioral sequence features into the intent determination network to determine at least one target modality that the user's intent is focused on in real time;
[0012] S4: Based on the determination result of the target modality, perform dynamic coordination and index update operations simultaneously. This operation includes: dynamically adjusting the weight distribution of each modality vector in the multimodal fusion retrieval according to the target modality, and adding the vector associated with the target modality to an incremental index block independent of the main index.
[0013] S5: Integrate behavioral sequence features with the initial query to generate a refined query vector;
[0014] S6: Based on the dynamically adjusted weight allocation, a refined query vector is used to query the main index and incremental index blocks in parallel, and the returned search results are merged, sorted, and output.
[0015] Through the aforementioned synchronous execution of dynamic coordination and index update operations, the present invention can achieve real-time personalized adaptation of the retrieval process and adaptive optimization of the system.
[0016] Furthermore, in step S3, the intent determination network is a network based on a two-layer gated recurrent unit. Its first layer is used to encode the temporal dependency of the behavior sequence, and its second layer is used to output the modality focusing probability distribution. The determination threshold is dynamically adjusted according to the variance of the probability distribution.
[0017] Furthermore, the second layer of the intent determination network is coupled with an attention mechanism module. The attention mechanism module is used to assign different weights to the hover trajectory dispersion and regional dwell time distribution in the behavior sequence features according to the context of the current query, so as to calculate the weighted behavior feature vector, which is then input into the second layer for intent determination.
[0018] Furthermore, in step S4, the step of dynamically adjusting the weights specifically includes: increasing the weight contribution coefficient of the target mode vector, and performing sparsity suppression normalization processing on the non-target mode vectors, wherein the total increase in weight is equal to the total decrease in weight.
[0019] Furthermore, the non-target mode vectors are subjected to sparsity suppression normalization processing, including calculating the vector's... Norm and scale it to the original norm times, of which At the same time, its allocation coefficient in the fusion weight is reduced to half of the original coefficient. times, of which .
[0020] Furthermore, in step S4, the weight allocation of each modality vector in the multimodal fusion retrieval is dynamically adjusted according to the target modality. Specifically, the adjusted weights are calculated using the following formula: For the target modality Weight For non-target modes Weight ;in, and The adjustment factor is dynamically calculated based on the entropy value of the focusing probability distribution output by the network for this intent determination, and it satisfies the principle of weight conservation.
[0021] Furthermore, in step S4, the incremental index block adopts a data structure combining a Bloom filter and a hash table; in step S6, during parallel querying, a Bloom filter is first used for fast comparison filtering, and then a hash table is used to perform accurate similarity calculation in the remaining set.
[0022] Furthermore, the coordination method between the incremental index block and the main index also includes: setting a dynamic merging trigger condition, which occurs when the number of vectors in the incremental index block grows to the number of modal vectors corresponding to the main index. Double time ( Alternatively, when the average popularity of vectors in the incremental index block is lower than a set threshold, an automatic merging and deduplication operation is triggered, and the results are synchronized to the main index while the incremental index block is cleared.
[0023] Further, in step S5, the behavioral sequence features are fused with the initial query to generate a refined query vector. Specifically, the behavioral sequence features are mapped to a behavioral preference vector of the same dimension as the initial query vector through a lightweight fully connected network. Then, the behavioral preference vector and the initial query vector are summed element by element to obtain the refined query vector.
[0024] Secondly, the present invention also provides a system for implementing the above method, the system comprising:
[0025] The initial retrieval module is used to respond to the initial query and return the result set;
[0026] The behavior monitoring and feature extraction module is used to monitor implicit interaction behaviors and extract behavioral sequence features;
[0027] The real-time intent determination module is used to run the intent determination network and determine the target modality.
[0028] The dynamic coordination and incremental index management module is used to synchronously perform dynamic weight adjustments and updates to incremental index blocks;
[0029] The query refinement and hybrid retrieval module is used to generate refined query vectors and perform parallel hybrid retrieval.
[0030] Compared with the prior art, the present invention has the following beneficial effects:
[0031] 1. This invention analyzes implicit interaction behaviors such as hovering trajectories and dwell times in real time, and utilizes an intent determination network based on GRU and attention mechanisms for deep temporal modeling and feature weighting. This enables real-time and accurate capture of the modalities focused on by the user's potential intent without requiring additional user input. This effectively overcomes the pain points of traditional methods, such as strong reliance on explicit feedback and ambiguous user intent expression, achieving a seamless understanding of user preferences.
[0032] 2. This invention achieves dual optimization of short-term personalized enhancement and long-term system self-evolution through the core mechanism of "synchronous execution of dynamic coordination and index update":
[0033] Short-term personalization enhancement: Based on real-time intent determination, the multimodal fusion weights are dynamically adjusted and refined query vectors are generated, enabling the results of a single search to adapt to the user's current preferences in an immediate and sensitive manner, significantly improving the immediate relevance of search results and user satisfaction.
[0034] Long-term system self-evolution: By synchronously writing the material vectors of intent focus into independent incremental index blocks, the system can continuously and with low loss accumulate implicit preference data from massive user interactions. This is combined with preset dynamic merging trigger conditions (such as data volume thresholds). The system can periodically integrate and optimize incremental data into the main index, thereby enabling the core retrieval model of the system to continuously evolve with use, providing a data foundation for subsequent retrieval based on historical group preferences, and realizing the adaptive evolution capability of the retrieval system.
[0035] 3. This invention, through its innovative system architecture, effectively ensures overall retrieval performance while introducing advanced intelligence. The core mechanism of this architecture is "synchronous execution of dynamic coordination and index updates." It guarantees the atomicity of the two key sub-steps—weight adjustment and index update—through a transaction controller. This eliminates the data inconsistency window that can occur with traditional asynchronous operations from within the computer system, effectively avoiding concurrency conflicts and ensuring real-time and high reliability of intent understanding and system adjustments. Combined with this mechanism, incremental index blocks are managed using a high-efficiency data structure combining Bloom filters and hash tables, and are retrieved in parallel with the main index, along with an intelligent hybrid sorting strategy. This series of collaborative designs enables the system to maintain excellent retrieval response speed and overall throughput while achieving real-time personalized adaptation and continuous data accumulation, meeting the stringent practical requirements of high-efficiency retrieval for large-scale intelligent material libraries. Attached Figure Description
[0036] Figure 1 This is a flowchart of an efficient intelligent material library retrieval method based on multimodal vector fusion in the first embodiment of the present invention;
[0037] Figure 2 This is a structural block diagram of an intelligent material library efficient retrieval system based on multimodal vector fusion in the second embodiment of the present invention. Detailed Implementation
[0038] To make the objectives, technical solutions, and advantages of the present invention clearer, two embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.
[0039] First implementation method:
[0040] Figure 1 This paper presents an efficient intelligent material library retrieval method based on multimodal vector fusion. The core of the method provided in this embodiment lies in constructing a real-time adaptive closed loop of "perception-decision-execution-evolution," including the following steps:
[0041] S1. Respond to the initial query and display the result set:
[0042] The system receives an initial query from the user (usually text). This query is converted into an initial query vector by a text encoder in a pre-trained multimodal model (such as CLIP, ViLT, etc.). The system uses this vector to retrieve the main index of the intelligent material library in a multimodal fusion vector space with initial default weights, and obtains and displays an initial result set containing materials of multiple modalities.
[0043] S2. Real-time monitoring of implicit interactions and generation of behavioral sequence features:
[0044] The system monitors user interaction events on the initial result set in real time and extracts raw behavioral data, including at least the dispersion of hover trajectory and the distribution of dwell time in the region. These implicit interactive behaviors include, but are not limited to, cursor movement trajectories, dwell times in specific areas, scrolling speed, and other operations that indirectly reflect user interests.
[0045] Hover trajectory dispersion: Quantified by calculating the standard deviation of the coordinate sequence of the cursor within a single preview area. Lower dispersion generally indicates careful examination, while higher dispersion indicates rapid browsing.
[0046] Area dwell time distribution: This is obtained by statistically analyzing the percentage of cumulative dwell time of the cursor in different modalities (such as image and video areas) of the material preview area.
[0047] These features are organized chronologically to form behavioral sequence features, in order to capture the temporal changes in user interests.
[0048] S3. Real-time determination of target modality based on intent determination network:
[0049] The behavioral sequence features are input into the intent determination network. A preferred structure for this network is a two-layer gated recurrent unit (GRU) network.
[0050] Its first layer, GRU, is used to encode the temporal dependencies of behavioral sequences.
[0051] In a further optimized embodiment, the second layer of the network is coupled with an attention mechanism module. This module can dynamically assign different importance weights to different dimensions of the behavioral sequence features (such as hover trajectory dispersion and region dwell time) based on the context of the current query, thereby generating a weighted, more expressive behavioral feature vector, which is then input into the second layer.
[0052] The final output of the network is a modality focusing probability distribution (e.g., , The system dynamically adjusts the judgment threshold based on the variance of the distribution, and determines the modality with a focusing probability higher than the threshold as the target modality for this interaction, and determines the modality with a probability significantly lower than another threshold as a non-target modality.
[0053] S4. Synchronously perform dynamic coordination and index updates:
[0054] This step is the core of the method. Based on the same decision result of S3, it ensures that the following two operations are executed synchronously, i.e., processed as an indivisible unit of work. In a preferred software engineering implementation, this synchronization can be guaranteed by a transaction controller or by sending atomic messages to a message queue, ensuring that the operation only takes effect if both sub-steps succeed; otherwise, it rolls back. By ensuring the synchronous atomic execution of "dynamic weight coordination" and "incremental index update" through a transaction controller, not only is real-time consistency between user intent and system state changes guaranteed, but the data race conditions and index inconsistencies that may arise from traditional asynchronous operations are also effectively avoided, improving the internal reliability and execution efficiency of the computer system when handling continuous interactive feedback.
[0055] Operation 1: Dynamic Weight Coordination
[0056] Based on the determination result of the target modality, the weight allocation of each modality vector in the multimodal fusion retrieval is adjusted in real time.
[0057] One implementation involves increasing the weight contribution coefficient of the target mode vector and performing sparsity suppression normalization on the non-target mode vector. This process includes calculating the weight contribution coefficient of the non-target mode vector. Norm and scale it to the original norm times ( At the same time, its allocation coefficient in the fusion weights is reduced to half of the original coefficient. times ( And ensure that the total increase in all modal weights is equal to the total decrease, maintaining the total sum conservation.
[0058] Another preferred embodiment uses the following formula for precise calculation:
[0059] For the target mode The adjusted weights are: ;
[0060] For non-target modes The adjusted weights are: ;
[0061] in, and To determine the focusing probability distribution of the network output based on this intent entropy value The adjustment factor is calculated dynamically. One specific calculation method is as follows:
[0062] ;
[0063] in, The Shannon entropy is the probability distribution. For the total number of modes, and for A preset proportional coefficient within the range is used to control the adjustment range. Entropy value. The lower the value (the clearer the intention). The larger the value, the greater the increase in the target modality weight. The smaller the value, the stronger the non-target mode suppression. The system ensures that the total weight conservation is satisfied after adjustment. The adjusted weight vector will take effect immediately.
[0064] The two weight adjustment implementation methods described above are parallel optional schemes. In actual systems, either one can be implemented, and both can achieve the purpose of dynamically adjusting the weight allocation according to the target mode.
[0065] Operation 2: Incremental Index Update
[0066] Simultaneously, the system adds the material vectors associated with the target modality and those that received high user attention during the interaction to an incremental index block independent of the main index. This block is physically separate from the main index but logically related, forming the system's dynamic memory unit. This incremental index block preferably uses a data structure combining a Bloom filter and a hash table. This design is a generally preferred solution; regardless of the aforementioned weight adjustment method used by the system, this structure can be applied to support efficient retrieval in subsequent steps and avoid a decrease in retrieval speed due to data growth.
[0067] The synchronous execution ensured by the aforementioned transaction control mechanism guarantees the atomicity and data consistency of the two key sub-steps: "dynamic weight coordination" and "incremental index update." This not only achieves instantaneous linkage between user intent determination and system adaptive adjustment in business logic, but also brings significant benefits to the internal performance of the computer system: it eliminates the "time window" of brief inconsistency between weight and index states that may exist in traditional asynchronous or step-by-step execution modes, avoiding retrieval result disturbances or concurrency conflicts caused by this, thereby improving the overall reliability and response efficiency of the system when handling high concurrency and real-time interactive feedback. This design is one of the core technical guarantees for the efficient and stable retrieval achieved by this invention.
[0068] S5. Generate refined query vectors:
[0069] The behavioral sequence features are mapped to a behavioral preference vector of the same dimension as the initial query vector through a lightweight fully connected network. Then, this behavioral preference vector is summed element-wise with the initial query vector to generate the final refined query vector. This vector simultaneously contains the user's original query semantics and its implicitly expressed preferences.
[0070] S6. Parallel Hybrid Retrieval and Result Fusion:
[0071] Based on the multimodal fusion weights adjusted by S4, a retrieval is initiated using the refined query vector generated by S5.
[0072] Parallel Queries: Queries are sent simultaneously to both the main index retrieval engine and the incremental index retrieval engine. The incremental index blocks employ a data structure combining Bloom filters and hash tables to improve efficiency. During a query, a Bloom filter is first used to quickly compare the refined query vector to eliminate a large number of impossible matches. Then, a hash table is used to perform precise similarity calculations only on a very small candidate set.
[0073] Merged Ranking: Search results from the main index and incremental index are aggregated. The system assigns higher priority weights to results from the incremental index, and then re-ranks them by comprehensively considering factors such as similarity scores and source priority, ultimately generating and outputting a new, optimized result set.
[0074] To better understand the above method, the following two examples will illustrate each step in detail:
[0075] Example 1: Weight allocation method using dynamic calculation of adjustment factors based on entropy values.
[0076] Taking the intelligent design material library search as an example, suppose a user (a graphic designer) enters the text query "summer refreshing drink poster".
[0077] (Corresponding to S1) The system returns a mixed initial result set containing images, videos, and templates.
[0078] (Corresponding to S2, refined features) The behavior monitoring module discovered that the user's cursor exhibited slow, low-dispersion movement and lingering for several seconds over several high-saturation beverage photos, while quickly swiping over several video previews. Calculations showed that 85% of the time was concentrated in the image area.
[0079] (Corresponding to S3, demonstrating network operation) The aforementioned sequence features are input into the intent determination network. The network's first layer, GRU, captures the temporal pattern; the attention module, combined with the "poster" context in the query, assigns higher weight to the "region dwell time" feature. Ultimately, the network outputs the focus probability distribution as follows: , , , Due to the large variance, the system explicitly identifies the image as the target modality.
[0080] (Corresponding to S4, demonstrating synchronization operations and specific calculations) The system executes synchronously through the transaction controller:
[0081] Dynamic coordination: Calculating the entropy of the probability distribution (Intent clear). Setting , Calculated according to the formula , Increase the image weight from 0.25 to Suppress video weight to .
[0082] Index update: Write the vectors of the beverage images that the user is most interested in as an atomic operation to the incremental index block.
[0083] (Corresponding to S5) Behavioral features are mapped to "visual preference vectors", which are added to the original text query vector to generate a refined query vector containing the compound semantics of "summer drinks, high-quality photography, static posters".
[0084] (Corresponding to S6, demonstrating hybrid retrieval) Under the new weight space, this vector is used to retrieve the main index and incremental index in parallel. The relevant images in the incremental index are quickly hit and given high priority. After being mixed with other highly similar images in the main index and weighted, the final result set significantly highlights high-quality images of the same type at the top.
[0085] The following example demonstrates that the invention is equally effective when different weighting strategies are employed.
[0086] Example 2: Weight allocation method using sparsity suppression normalization processing
[0087] To demonstrate the feasibility of an alternative weighting method, consider a scenario where another user searches for "city night view". The initial system search returns a mixture of images and videos. Monitoring and analysis show that the user exhibits significant focusing behavior (low hover dispersion, long dwell time) on several "time-lapse style" videos. Based on this, the intent determination network outputs the target modality as video, and non-target modalities including images.
[0088] The system employs a sparsity suppression scheme and sets parameters. , For non-target modalities (such as images), the system first calculates their multimodal fusion vector. The norm is calculated and scaled to 0.3, then its allocation coefficient in the global fusion weights is reduced to 0.2 times the original coefficient. For the target modality (video), its weight contribution coefficient is increased accordingly based on the principle of weight sum conservation.
[0089] After the above processing, the weight of video modalities in subsequent fusion retrieval is significantly enhanced, while the vector representations and weights of non-focused modalities such as images are effectively suppressed. This also enables a rapid and accurate adjustment of the retrieval focus to the user's intent, forming a technical path parallel to "Example 1".
[0090] Subsequently, based on the adjusted fusion weight configuration and the generated refined query vector, the system will continue to perform parallel hybrid retrieval and result fusion sorting (corresponding to S5 and S6), and synchronously write the video material vectors focused on this time into the incremental index block to participate in the system's long-term self-evolution process.
[0091] It should be noted that in any of the above examples, the long-term self-evolution of the system is achieved by managing incremental index blocks. When the number of vectors in this block grows to the number of modal vectors corresponding to the primary index... times (for example, When the average popularity of a vector is below a threshold (typically 0.05), automatic merging and deduplication operations are triggered, synchronizing stable group preference knowledge to the main index, and then clearing the incremental blocks. This allows the entire system to learn continuously.
[0092] The above examples fully demonstrate how this invention achieves both short-term personalization enhancement and long-term system self-evolution. In this interaction, the system accurately matched the user's potential intent through real-time intent determination and weight adjustment, reflecting short-term personalization enhancement. Simultaneously, the high-quality image vectors written into the incremental index will enter the main index in subsequent merging operations, enabling subsequent searches (including queries from other users) to benefit from this knowledge, demonstrating the system's long-term self-evolution capability.
[0093] Second implementation method:
[0094] Figure 2 This paper presents an efficient intelligent material library retrieval system based on multimodal vector fusion. The system provided in this embodiment is used to implement the retrieval method described in the first embodiment. This system can be deployed on a server cluster or in the cloud and includes the following functional modules:
[0095] Initial retrieval module: This module receives the user's initial query request, calls the multimodal encoder to convert the query into a vector, accesses the main index under the initial weight configuration, performs a preliminary retrieval to generate and return an initial result set to the interactive interface.
[0096] Behavior monitoring and feature extraction module: Tightly coupled with the front-end interface, this module captures cursor movement and dwell events on the result set in real time. It includes a built-in feature calculation unit to calculate the hover trajectory dispersion and regional dwell time distribution from the raw logs in real time, and organizes them into time-series features.
[0097] Real-time Intent Determination Module: This module loads and runs the intent determination network (such as a pre-trained GRU-attention model). It receives behavioral sequence features from the previous module, performs forward inference in real time, outputs the modality focusing probability distribution, and determines the target modality and non-target modality based on dynamic threshold logic. This module typically provides low-latency inference capabilities as a backend service.
[0098] Dynamic Coordination and Incremental Index Management Module: The core control and storage module of this system. It receives the output from the intent determination module and ensures the synchronous execution of two key responsibilities through its internal transaction controller:
[0099] Weight Coordinator: Maintains the current multimodal fusion weight configuration and updates it in real time by calling a predefined weight adjustment algorithm (such as the formula above) based on the judgment result.
[0100] Incremental Index Manager: Manages incremental index blocks independent of the main index. These blocks typically reside in high-performance memory or SSD storage, employing Bloom filters for fast existence checks and utilizing hash tables to store the mapping between vector IDs and vectors for efficient insertion and retrieval. The manager is responsible for writing target modality-related vectors into this block and, when preset conditions are met (such as the number reaching a threshold θ), scheduling background tasks to merge the incremental index with the main index.
[0101] Query Refinement and Hybrid Search Module: This module contains two sub-units:
[0102] Query Refinement Unit: Embedded with a lightweight fully connected network, responsible for fusing behavioral features into the initial query vector to generate a refined query vector.
[0103] Hybrid retrieval engine: Receives refined query vectors and updated weight configurations, and initiates parallel retrieval. It simultaneously sends query requests to both the main index search engine and the aforementioned incremental index manager, and integrates the returned results from both sources according to a preset fusion and sorting strategy (such as weighted scoring) to generate the final sorted list.
[0104] The modules interact with each other through predefined interface protocols, forming a collaborative pipeline. For example, the behavior monitoring module encapsulates feature sequences in JSON format and sends them to the intent determination module via a RESTful API. The intent determination module simultaneously writes the results to a shared message queue (such as Kafka) or cache (such as Redis), and the dynamic coordination module and query refinement module, acting as consumers, read these simultaneously to trigger synchronization operations. The hybrid search engine obtains the latest weight configuration and incremental index access endpoints through the query service discovery center. This loosely coupled architecture ensures the system's scalability and high availability.
[0105] In light of current practical needs, the scope of protection of the above-described embodiments adopted in this invention is not limited thereto. Any changes made within the scope of knowledge possessed by those skilled in the art without departing from the inventive concept, such as intending to use a Transformer architecture for the decision network, employing other mathematical forms for weight adjustment, or using different data structures for incremental indexing, should all fall within the scope of protection of this invention.
Claims
1. A highly efficient retrieval method for an intelligent material library based on multimodal vector fusion, characterized in that, Includes the following steps: S1: In response to the user's initial query, perform a multimodal search from the intelligent material library to obtain and display the initial result set; S2: Monitor the implicit interaction behavior of users with the initial result set in real time, and generate behavioral sequence features based on this; S3: Input the behavioral sequence features into the intent determination network to determine at least one target modality that the user's intent is focused on in real time; S4: Based on the determination result of the target modality, perform dynamic coordination and index update operations simultaneously. This operation includes: dynamically adjusting the weight distribution of each modality vector in the multimodal fusion retrieval according to the target modality, and adding the vector associated with the target modality to an incremental index block independent of the main index. S5: Fuse the behavioral sequence features with the initial query to generate a refined query vector; S6: Based on the dynamically adjusted weight allocation, the refined query vector is used to query the main index and the incremental index blocks in parallel, and the returned search results are merged, sorted, and output.
2. The efficient retrieval method for intelligent material database based on multimodal vector fusion according to claim 1, characterized in that, In step S3, the intent determination network is a network based on a two-layer gated recurrent unit. Its first layer is used to encode the temporal dependency of the behavior sequence, and its second layer is used to output the modality focusing probability distribution. The determination threshold is dynamically adjusted according to the variance of the probability distribution.
3. The efficient retrieval method for intelligent material database based on multimodal vector fusion according to claim 2, characterized in that, The second layer of the intent determination network is coupled to an attention mechanism module. The attention mechanism module is used to assign different weights to the hover trajectory dispersion and regional dwell time distribution in the behavior sequence features according to the context of the current query, so as to calculate the weighted behavior feature vector, and then input it into the second layer for intent determination.
4. The efficient retrieval method for intelligent material database based on multimodal vector fusion according to claim 1, characterized in that, In step S4, the step of dynamically adjusting the weights specifically includes: increasing the weight contribution coefficient of the target mode vector, and performing sparsity suppression normalization processing on the non-target mode vectors, wherein the total increase in weights is equal to the total decrease in weights.
5. The efficient retrieval method for intelligent material database based on multimodal vector fusion according to claim 4, characterized in that, The sparsity suppression and normalization process for non-target mode vectors includes calculating the vector's... Norm and scale it to the original norm times, of which At the same time, its allocation coefficient in the fusion weight is reduced to half of the original coefficient. times, of which .
6. The efficient retrieval method for intelligent material database based on multimodal vector fusion according to claim 1, characterized in that, In step S4, the weight allocation of each modality vector in the multimodal fusion retrieval is dynamically adjusted according to the target modality. Specifically, the adjusted weights are calculated using the following formula: For the target modality... Weight For non-target modes Weight ;in, and The adjustment factor is dynamically calculated based on the entropy value of the focusing probability distribution output by the network for this intent determination, and it satisfies the principle of weight conservation.
7. The efficient retrieval method for intelligent material database based on multimodal vector fusion according to claim 1, characterized in that, In step S4, the incremental index block adopts a data structure combining a Bloom filter and a hash table; in step S6, during the parallel query, the Bloom filter is first used for fast comparison filtering, and then the hash table is used to perform accurate similarity calculation in the remaining set.
8. The efficient retrieval method for intelligent material database based on multimodal vector fusion according to claim 1, characterized in that, The coordination method between the incremental index block and the main index further includes: setting a dynamic merging trigger condition, which occurs when the number of vectors in the incremental index block increases to the number of modal vectors corresponding to the main index. When the average popularity of vectors in the incremental index block is lower than a set threshold, an automatic merging and deduplication operation is triggered, and the results are synchronized to the main index, while the incremental index block is cleared.
9. The efficient retrieval method for intelligent material database based on multimodal vector fusion according to claim 1, characterized in that, In step S5, fusing the behavioral sequence features with the initial query to generate a refined query vector specifically involves: mapping the behavioral sequence features to a behavioral preference vector of the same dimension as the initial query vector through a lightweight fully connected network, and then summing the behavioral preference vector and the initial query vector element by element to obtain the refined query vector.
10. A system for implementing the efficient intelligent material library retrieval method based on multimodal vector fusion as described in any one of claims 1-9, characterized in that, include: The initial retrieval module is used to respond to the initial query and return the result set; The behavior monitoring and feature extraction module is used to monitor implicit interaction behaviors and extract behavioral sequence features; The real-time intent determination module is used to run the intent determination network and determine the target modality; The dynamic coordination and incremental index management module is used to synchronously perform dynamic weight adjustments and updates to incremental index blocks; The query refinement and hybrid retrieval module is used to generate refined query vectors and perform parallel hybrid retrieval.