Multi-path recall retrieval method and system based on dynamic weight distribution and storage medium

Abstract

The invention discloses a multi-path recall mixed retrieval method and system based on intelligent dynamic weight distribution, and aims to solve the problems that semantic comprehension and keyword matching are difficult to balance and the adaptability is poor due to the adoption of a fixed weight in the existing retrieval technology. The invention provides a multi-path recall mechanism fusing vector semantic retrieval, BM25 keyword retrieval and entity retrieval. A query feature vector containing 13-dimensional features such as semantic complexity, keyword density and entity coverage rate is constructed, a query type is recognized in combination with an SVM and a random forest integration model, a dynamic weight distribution algorithm is designed, and the final weight of each retrieval path is calculated in real time. And an adaptive multi-source enhanced reciprocal ranking fusion (AMSE-RRF) algorithm is further adopted to carry out optimization fusion on multiple paths of results, and a depth reordering model can be selected to improve the precision. According to the method, the accuracy and robustness of retrieval can be remarkably improved in multiple scenes of medical treatment, finance, government affairs and the like according to a millisecond-level self-adaptive adjustment strategy of query features.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of information retrieval and artificial intelligence, in particular to a multi-path recall retrieval method and system based on dynamic weight distribution and a storage medium. BACKGROUND

[0002] With the rapid development of artificial intelligence and big data technology, information retrieval systems play an increasingly critical role in knowledge-intensive scenarios such as intelligent question answering, enterprise knowledge base, government service, medical diagnosis assistance, financial risk control, etc. Users' expectations for retrieval systems have shifted from simple keyword matching to intelligent services that can understand semantics and accurately respond to complex problems. Current mainstream retrieval techniques mainly fall into two categories: keyword matching-based retrieval methods and semantic understanding-based vector retrieval methods.

[0003] Among them, the keyword matching-based retrieval method, such as the classic BM25 (Best Matching 25) algorithm, calculates the relevance score of documents and queries by counting the frequency of query terms in documents, inverse document frequency, and field weight. This method has the advantages of high efficiency, accurate recall results, and strong interpretability, especially when dealing with queries containing explicit keywords. However, its core limitation is the lack of semantic understanding, making it difficult to handle synonym replacement, contextual ambiguity, and sentence transformation, resulting in low recall rates in scenarios where keywords do not completely match but are semantically related.

[0004] On the other hand, semantic understanding-based vector retrieval methods (also known as "vector similarity retrieval" or "semantic search") use deep learning models (such as BERT, Sentence-BERT, etc.) to map text to high-dimensional vector space, and measure semantic relevance by calculating the cosine similarity between query and document vectors. This method can capture semantic relationships between words, effectively handle synonymous expressions and contextual reasoning, and show significant advantages in complex semantic queries. However, it also has obvious defects: first, for low-frequency words, emerging concepts or professional terms, due to insufficient training data, their semantic vector representation is often inaccurate or has "semantic drift" problems; second, the recall results of vector retrieval may contain semantically related but keyword missing documents, affecting the accuracy of the results; in addition, vector retrieval is highly dependent on models and training data, with limited cross-domain adaptability.

[0005] To overcome the limitations of single retrieval method, researchers have proposed a hybrid retrieval strategy, which uses both keyword retrieval and vector retrieval, and fuses the multi-path recall results. Common fusion methods include weighted summation, RRF (Reciprocal Rank Fusion), etc. However, existing hybrid retrieval techniques generally use a fixed weight fusion strategy, i.e. the weights of each retrieval path are pre-set at the system design stage (e.g. vector retrieval accounts for 60% and keyword retrieval accounts for 40%), and remain unchanged in all query scenarios. This static weight allocation method has serious technical defects:

[0006] 1. Weight solidification, lack of adaptability: Different types of user queries have significant differences in the demand for semantic understanding and keyword exact match. For example, "What are the symptoms of pneumonia?" This kind of question relies more on semantic understanding, while "What disease does ICD-10 code J18.9 correspond to?" It highly depends on keyword exact match. Fixed weight cannot be dynamically adjusted according to query semantic features, resulting in decreased retrieval performance in certain scenarios; 2. Weak scene awareness: Existing methods lack comprehensive analysis of multi-dimensional features such as query complexity, entity density, keyword distribution, and semantic depth, and cannot intelligently determine which retrieval strategy is more suitable for the current query; 3. Poor handling of low-frequency words and professional terms: In professional fields such as medicine, law, and technology, queries often contain a large number of low-frequency professional terms. The semantic representation ability of vector models for such words is limited, and if vector retrieval is still given a high weight at this time, it is easy to miss key information. The fixed weight strategy cannot perceive such risks and compensate.

[0007] Therefore, existing technologies cannot achieve stable and optimal retrieval results in diversified and complex practical application scenarios. There is an urgent need for a hybrid retrieval method that can intelligently perceive query features and dynamically adjust multi-path retrieval weights to achieve adaptive optimization of retrieval strategies, balancing accuracy and semantic understanding ability, thereby improving the intelligence level and user experience of information retrieval systems. The present invention is proposed in this background, aiming to solve the above technical bottlenecks and provide a multi-path recall hybrid retrieval strategy based on intelligent dynamic weight allocation. SUMMARY

[0008] The present invention aims to provide a multi-path recall retrieval method based on dynamic weight allocation to solve the problem of existing retrieval techniques being difficult to achieve stable and optimal retrieval results in diversified and complex practical application scenarios as described in the background.

[0009] To achieve the above-mentioned purpose, the present invention provides the following technical solutions:

[0010] A multi-path recall retrieval method based on dynamic weight distribution, comprising:

[0011] Step S1: receiving a user query request, text pre-processing the query request, and extracting a multi-dimensional query feature vector containing an exact match score, semantic complexity, keyword density, entity coverage, syntactic structure complexity, semantic similarity variance, query length normalization value, TF-IDF weight, semantic embedding cosine similarity, syntactic dependency tree depth, entity density, interrogative word weight coefficient, and query complexity index;

[0012] Step S2: based on the multi-dimensional query feature vector, using a support vector machine and a random forest ensemble learning model to classify the user query type, and identifying one of keyword-oriented, semantic-oriented, balanced, or complex;

[0013] Step S3: according to the identified query type and the multi-dimensional query feature vector, calculating the maximum weight of the vector semantic retrieval path, the keyword retrieval path, and the entity retrieval path through a dynamic weight distribution algorithm;

[0014] Step S4: performing three-way recall in parallel: a. Vector semantic retrieval: converting the query into a semantic vector and performing approximate nearest neighbor search in the vector database to obtain a set of semantically related documents; b. Keyword retrieval: keyword matching on the query and retrieving a set of related documents in the inverted index; c. Entity retrieval: based on the named entity recognition result, retrieving related entities and their associated documents in the knowledge graph or structured database; Step S5: merging and sorting the three-way recall results to output the final retrieval results.

[0015] Further, the dynamic weight distribution algorithm in step S3 includes:

[0016] Step S31: calculating the probability distribution of each query type:

[0017]

[0018] Where F is the multi-dimensional query feature vector, c is the query type, And are model parameters;

[0019] Step S32: calculating the base weight:

[0020]

[0021] Where, is the preset weight configuration for each query type;

[0022] Step S33: Calculate the dynamic adjustment factor:

[0023]

[0024] wherein, is the ith feature, is the corresponding weight coefficient, and β and γ are adjustable parameters;

[0025] Step S34: Calculate the final weight:

[0026]

[0027] wherein, denotes an element-wise product, denotes a normalization process to make the sum of weights equal to 1.

[0028] Further, in step S5, the fusion ranking adopts the AMSE-RRF algorithm, which includes:

[0029] Step S51: Perform deduplication and score standardization on each route recall result;

[0030] Step S52: Calculate the ranking confidence of each document in each route:

[0031]

[0032] wherein, is the ranking of one route, is a parameter;

[0033] Step S53: Calculate the basic RRF fusion score:

[0034]

[0035] wherein, is the final weight obtained in step S3, and k is a smoothing constant;

[0036] Step S54: Introduce a score enhancement term:

[0037]

[0038] wherein, is the normalized original relevance score, is an adaptive activation function;

[0039] Step S55: Calculate the final fusion score:

[0040]

[0041] ​wherein a, b, g are fusion coefficients and a+b+g = 1.

[0042] Further, the vector semantic retrieval generates query vectors using a pre-trained language model and performs approximate nearest neighbor search in the vector database with sub-linear time complexity O(log n) through HNSW graph indexing. Further, the keyword retrieval uses an improved BM25F algorithm, combined with field weight, length normalization and query expansion techniques, including synonym, near-synonym and pinyin similar word expansion. Further, the entity retrieval extracts entities such as names, places, organizations, times, and professional terms from the query based on a named entity recognition model, and performs entity linking and relationship reasoning in the knowledge graph to return relevant documents. Further, in the AMSE-RRF fusion algorithm, the score normalization uses the Z-score method or the min-max normalization to map the original scores of each path to a unified interval.

[0043] Further, after the result fusion, the Top-20 candidate documents are re-ranked based on deep learning, using a pre-trained re-ranking model to calculate query-document relevance, and using a weighted method to fuse the re-ranking score and the fusion score, wherein the re-ranking score weight is not less than 0.7.

[0044] The multi-path recall retrieval system based on dynamic weight distribution includes:

[0045] The query preprocessing and feature extraction module is used for cleaning, tokenizing, and entity recognition of user input queries, and generates a 13-dimensional query feature vector.

[0046] The query type identification module is used to identify the query type as keyword-oriented, semantic-oriented, balanced, or complex based on the 13-dimensional query feature vector using a support vector machine and a random forest ensemble classifier.

[0047] The dynamic weight calculation module is used to calculate the maximum weight of the vector semantic retrieval, keyword retrieval, and entity retrieval based on the query type and 13-dimensional feature vector through the dynamic weight distribution algorithm.

[0048] The multi-path recall module includes: a. The vector semantic retrieval unit implements semantic neighbor search based on the vector database and HNSW indexing. b. The keyword retrieval unit implements keyword matching based on the improved BM25F algorithm and inverted indexing. c. Entity retrieval unit based on the pipeline of "large model entity extraction + entity linking (disambiguation) + knowledge graph relationship retrieval": first, extract entities from the query and link them to the unique nodes of the graph; then, retrieve the entity and its neighborhood relationships, and output the relevant documents and structured relationship views; A result fusion module is configured to fuse the three-way recall results by using an AMSE-RRF algorithm, including score standardization, ranking confidence evaluation, weighted RRF calculation, score enhancement, and diversity optimization. A reordering module is configured to perform deep semantic reordering of the fused Top-K results based on a Transformer architecture, and output a final ranking list. The system realizes millisecond-level adaptive retrieval strategy adjustment through a dynamic weight distribution mechanism, and improves the retrieval accuracy and robustness in complex query scenarios.

[0049] A computer-readable storage medium stores a computer program, which is executed by a processor to implement the multi-way recall retrieval method based on dynamic weight distribution.

[0050] Compared with the prior art, the present application has the following advantages: (1) Significantly improve the retrieval accuracy and recall rate: by fusing vector semantic retrieval, BM25 keyword retrieval and entity retrieval three strategies, taking into account the semantic understanding ability and keyword exact match advantage, effectively covering the diversified query demand, especially in dealing with synonyms, professional terms and complex semantic problems, significantly improving the recall rate and ranking accuracy of relevant documents; (2) Intelligent adaptive retrieval strategy: innovatively introduce a dynamic weight distribution mechanism based on 13-dimensional query features, combined with a machine learning model to identify query types (such as keyword-oriented, semantic-oriented, etc.) in real time, dynamically adjust the weight of each retrieval path at a millisecond level, overcoming the poor adaptability of traditional fixed weight method, making the retrieval strategy more in line with the actual query intention; (3) Enhance system robustness and cross-domain applicability: the dynamic weight mechanism has the ability to identify and compensate for low-frequency words, emerging concepts and professional terms, and performs stably in medical, legal, financial and other professional fields; multi-dimensional feature perception and adaptive fusion strategy ensure that the system maintains high performance in different application scenarios, with strong generalization ability; (4) Optimize fusion efficiency and result quality: use the adaptive multi-source enhanced inverse ranking fusion (AMSE-RRF) algorithm, combined with ranking confidence evaluation, score standardization and diversity reward, to realize efficient and accurate result fusion; support deep learning reordering to further improve the relevance of Top results; (5) High practicality and scalability: modular system architecture, support micro-service deployment, flexible integration into intelligent question answering, knowledge base management, government service and other practical business systems, significantly improve information acquisition efficiency and user experience. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 Flow chart of the multi-channel recall retrieval method based on dynamic weight distribution of the application; Figure 2 Detailed diagram of query type classification of the application; Figure 3 Detailed diagram of multi-channel recall mechanism of the application; Figure 4 Detailed flow chart of MSE-RRF fusion algorithm of the application; Figure 5 Application scenario example diagram of the application. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only a part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0053] Please refer to Figure 1 The application provides a technical solution: a multi-channel recall retrieval method based on dynamic weight distribution, including the following core mechanisms: (1) Multi-channel recall mechanism: use vector database for semantic retrieval to obtain documents similar in semantics to user questions; use keyword matching based on BM25 algorithm for accurate retrieval to extract documents with high matching degree of keywords; use entity recognition for structured information retrieval; (2) Intelligent weight adjustment mechanism: use a deep query analysis algorithm based on multi-dimensional feature fusion to automatically identify user question types (keyword-oriented, semantic-oriented, balanced, and complex) by constructing a query feature vector space and using support vector machine (SVM) and random forest ensemble learning method, dynamically calculate the weight of each retrieval path according to 13-dimensional query features, and realize millisecond-level adaptive weight allocation strategy.

[0054] Definition of 13-dimensional query feature vector: Accurate matching score: calculate the proportion and weight of accurately matched words in the query; Semantic complexity: comprehensive evaluation based on word diversity and syntactic structure complexity; Keyword density: density distribution of effective keywords in the query; Entity Coverage: The extent of named entities covered in the query; Syntactic Complexity: Quantification of structural complexity based on dependency parsing; Semantic Similarity Variance: Variance of semantic similarity among query terms; Query Length Normalized Value: Normalized query length feature; TF-IDF Weight: TF-IDF weighted average of query terms; Semantic Embedding Cosine Similarity: Average similarity of query vector with domain knowledge base; Syntactic Dependency Tree Depth: Maximum depth of syntactic dependency analysis tree; Entity Density: Density of named entities per unit length; Interrogative Word Weight Coefficient: Importance weight of interrogative word type in query; Query Complexity Index: Comprehensive query complexity evaluation of multi-dimensional features.

[0055] Intelligent Dynamic Weight Allocation Algorithm: The dynamic weight allocation mechanism is the core innovation of this method. Through multi-level feature perception and real-time weight optimization, the intelligent adjustment of retrieval strategy is realized. The dynamic weight allocation algorithm has the following technical features: Multi-dimensional Feature Driven: Based on 13-dimensional query feature vector for weight calculation, including semantic complexity, keyword density, entity coverage, etc. Key indicators ensure the scientificity and accuracy of weight allocation; Real-time Adaptive Adjustment: Adopting millisecond-level response weight calculation algorithm, it can adjust the weight allocation of each retrieval path in real time according to the changes of query features, avoiding the limitations of traditional fixed weight method; Scenario-aware Optimization: Through query type identification and feature analysis, it automatically selects the optimal weight configuration strategy for different application scenarios (such as medical, financial, education, etc.); Mathematical Model Precision: Adopting a mathematical framework combining probability theory and information theory, ensuring the theoretical rigor and practical effectiveness of weight allocation.

[0056] The weight calculation formula is as follows:

[0057] Step 1: Calculate the probability distribution of each query type:

[0058] (1)

[0059] Where, F is the multi-dimensional query feature vector, c is the query type, and are model parameters;

[0060] Step 2: Calculate the base weight:

[0061] (2)

[0062] wherein, is the preset weight configuration for each query type;

[0063] Step S3: Calculate the dynamic adjustment factor:

[0064] (3)

[0065] wherein, is the i-th feature, is the corresponding weight coefficient, and β and γ are adjustable parameters;

[0066] Step S4: Calculate the final weight:

[0067] (4)

[0068] wherein, denotes the element-wise product, denotes the normalization processing to make the sum of weights equal to 1.

[0069] Technical advantages of dynamic weight distribution: By real-time sensing of query feature changes, automatically adjusting the retrieval strategy weight, compared with the traditional fixed weight method, the retrieval accuracy is improved by 15-25% in complex query scenarios, the response time is controlled within milliseconds, and it has good practicability and expansibility.

[0070] (3) Optimization and fusion mechanism: adopt Adaptive Multi-Source Enhanced RRF (AMSE-RRF) algorithm for multi-path recall result fusion, which introduces score distribution perception, ranking confidence evaluation and multi-level fusion strategy on the basis of traditional RRF, combines dynamic weight and Z-score standardization, and realizes efficient fusion with sub-linear time complexity O(n log n); optionally use deep reordering model based on Transformer architecture for semantic-level secondary sorting, and finally output the retrieval result list with the highest relevance.

[0071] AMSE-RRF fusion algorithm:

[0072] Step 1: De-duplicate and score standardization processing is performed on each path recall result;

[0073] Step 2: Calculate the ranking confidence of each document in each path:

[0074] (5)

[0075] wherein, Ranking for one of the routes, Parameters;

[0076] Step 3: Calculate the basic RRF fusion score:

[0077] (6)

[0078] wherein, is the maximum weight obtained in step S3, and k is a smoothing constant;

[0079] Step 4: Introduce score enhancement item:

[0080] (7)

[0081] wherein, is the normalized original correlation score, is an adaptive activation function;

[0082] Step 5: Calculate the final fusion score:

[0083] (8)

[0084] wherein, α, β, γ are fusion coefficients and α+β+γ=1.

[0085] Specifically, the following steps are implemented:

[0086] 1. Query preprocessing and analysis A multi-level text preprocessing pipeline is adopted: first, text cleaning is performed using regular expressions and Unicode standardization; then, accurate segmentation is performed using a BERT-based Chinese segmenter combined with a custom dictionary; then, 13 types of entities such as names, places, and organizations are extracted using a named entity recognition (NER) model; finally, a complete 13-dimensional query feature vector (including accurate matching score, semantic complexity, keyword density, entity coverage, syntactic structure complexity, semantic similarity variance, query length normalization value, TF-IDF weight, semantic embedding cosine similarity, syntactic dependency tree depth, entity density, interrogative word weight coefficient, query complexity index) is constructed, and a trained SVM+random forest ensemble classifier is used for query type recognition (( Figure 2 ), improving classification accuracy;

[0087] 2. Multi-path recall ( Figure 3 ) (1) Vector semantic retrieval: High-performance vector database, multi-dimensional index model generates query vectors, and the HNSW (Hierarchical Navigable Small World) graph index algorithm achieves approximate nearest neighbor search with sublinear time complexity of O(log n), supporting dual measures of cosine similarity and Euclidean distance; (2) Keyword precise retrieval: Based on the improved BM25F algorithm, field weight and length normalization are introduced, and intelligent query statements are constructed by combining query expansion technology (synonyms, near-synonyms, similar pinyin) and using inverted index to achieve millisecond-level precise matching; (3) Entity retrieval: The pipeline of "large model entity extraction + entity linking (disambiguation) + knowledge graph relationship retrieval" is adopted: first, the entity is extracted from the query and linked to the unique node of the graph; then, the entity and its neighborhood relationship are retrieved based on this, and the relevant documents and structured relationship view are output.

[0088] 3. Dynamic weight calculation

[0089] The weights of each retrieval path are dynamically calculated based on the query type and characteristics: (1) Keyword-oriented: BM25 weights dominate, while vector weights and entity weights are secondary; (2) Semantic-oriented: Vector weights are dominant, while BM25 weights and entity weights are secondary; (3) Balanced type: Vector weights and BM25 weights are evenly distributed, with entity weights participating moderately; (4) Complex type: Vector weights are slightly higher, BM25 weights are next, and entity weights are moderately involved.

[0090] 4. Optimize the RRF fusion algorithm

[0091] An improved Reciprocal Rank Fusion (RRF) algorithm is adopted. Figure 4 Specifically, it includes: (1) Preprocessing stage: Deduplication is performed on the multi-path recall results, and duplicate items are identified based on document ID and content hash value; score standardization is implemented to map the scores of different search paths to the [0,1] interval to ensure score comparability; (2) Fusion calculation: The improved RRF formula is adopted: ,in The weights are dynamic, and k=60 is the smoothing parameter. The ranking of the document in the i-th path search; (3) Score Enhancement: Introduce an original score enhancement factor of 0.3, resulting in a final score of Balance ranking information and original relevance scores.

[0092] 5. Intelligent reordering mechanism

[0093] A deep learning-based secondary re-ranking strategy improves the relevance of the final result: (1) Candidate selection: The top 20 candidate documents are selected from the fusion results and re-ranked to balance computational efficiency and performance improvement; (2) Feature construction: Construct multi-dimensional feature vectors for query-document pairs, including semantic similarity, keyword matching degree, entity overlap rate, document quality score, etc. (3) Model inference: The query-document relevance score is calculated using a pre-trained deep learning re-ranking model, which supports batch processing to improve efficiency; (4) Score fusion: A weighted fusion strategy is adopted. The reordering score has a higher weight to ensure accuracy.

[0094] Typical application scenario examples ( Figure 5 )

[0095] (1) Intelligent Medical Knowledge Base System

[0096] Deployment Architecture: A microservice architecture is adopted, including modules for medical literature retrieval, case analysis, and drug information query. Addressing the characteristics of numerous medical terms and complex synonyms, a customized medical domain embedding model is trained, and an entity library containing standard medical ontologies such as ICD-10 and SNOMED CT is constructed. Through intelligent retrieval technology, it can significantly improve doctors' query efficiency and the accuracy of diagnostic assistance, providing strong support for medical decision-making.

[0097] (2) Intelligent Question and Answer System for Financial Risk Control

[0098] Deployment Architecture: The system integrates modules for risk assessment, compliance checks, and policy interpretation. Addressing the frequent updates to financial regulations, it implements incremental index updates and version management. The system significantly reduces compliance query response time, improves the accuracy of regulatory document retrieval, effectively reduces compliance risks, and enhances the risk control efficiency of financial institutions.

[0099] (3) Intelligent recommendation platform for educational resources

[0100] Deployment Architecture: A multi-layered knowledge graph is constructed, encompassing subject knowledge points, teaching resources, and learning paths. Search weights are dynamically adjusted based on students' learning behavior and knowledge mastery. The system significantly improves the accuracy of personalized resource recommendations, enhances student learning outcomes, and provides technical support for personalized education.

[0101] (4) Intelligent consultation system for government services

[0102] Deployment Architecture: Integrates multi-source information such as policies and regulations, procedures, and frequently asked questions. Supports multi-turn dialogues and contextual understanding, enabling intelligent upgrades to government services. The system significantly improves citizen satisfaction with inquiries, substantially reduces the workload of human customer service representatives, and enhances the efficiency of government services.

[0103] (5) Intelligent analysis system for handling police calls

[0104] Deployment Architecture: Integrates multi-source data including historical police report databases, geographic information systems, and personnel archives. Addressing the unstructured nature of police report texts, a multi-dimensional entity recognition model is constructed, incorporating elements such as personnel, location, time, and case type.

[0105] Core application functions: 1) Intelligent police situation early warning: Based on the spatiotemporal distribution patterns of historical police situation data and combined with the characteristics of current police situations, it realizes early warning analysis of high-risk areas and time periods, and provides decision support for police force deployment; 2) Entity Relationship Graph Construction: Automatically extracts entity information such as people, locations, items, and vehicles from police reports, constructs a multi-dimensional relationship network, and supports case association analysis and suspect social relationship mining; 3) Intelligent matching of similar cases: Through multi-path recall technology, similar historical cases can be quickly retrieved to provide reference cases and handling experience for police officers and improve the efficiency of case handling. 4) Situational awareness of police situation: Real-time analysis of regional police situation trends, identification of abnormal police situation patterns, and intelligent support for command and dispatch and emergency response.

[0106] The system can significantly improve the efficiency of receiving and handling police calls, provide intelligent support for early warning of police incidents, case investigation, and resource allocation, and effectively improve the law enforcement efficiency and social security prevention and control capabilities of public security organs.

[0107] (6) Enterprise technical document management system

[0108] Deployment Architecture: Supports parsing multiple document formats (PDF, Word, PPT, code files, etc.) and constructing a technical knowledge graph. The system can significantly improve the efficiency of technical document retrieval, shorten the technical training cycle for new employees, and enhance the efficiency of enterprise knowledge management and transfer.

[0109] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A multi-path recall retrieval method based on dynamic weight allocation, characterized in that, include: Step S1: Receive user query requests, perform text preprocessing on the query requests, and extract a multi-dimensional query feature vector including exact match score, semantic complexity, keyword density, entity coverage, syntactic structure complexity, semantic similarity variance, query length normalization value, TF-IDF weight, semantic embedding cosine similarity, syntactic dependency tree depth, entity density, interrogative word weight coefficient, and query complexity index. Step S2: Based on the multidimensional query feature vector, the user query type is classified using the support vector machine and random forest ensemble learning model, and identified as one of keyword-oriented, semantic-oriented, balanced, or complex types. Step S3: Based on the identified query type and the multidimensional query feature vector, calculate the final weights of the vector semantic retrieval path, keyword retrieval path, and entity retrieval path using a dynamic weight allocation algorithm; Step S4: Perform three-way recall in parallel: a. Vector semantic retrieval: Convert the query into a semantic vector, perform an approximate nearest neighbor search in the vector database, and obtain a set of semantically related documents; b. Keyword search: Match the query with keywords and retrieve relevant document sets from the inverted index; c. Entity retrieval: Based on the named entity recognition results, retrieve relevant entities and their associated documents in knowledge graphs or structured databases; Step S5: Merge and sort the three recall results, and output the final search results.

2. The multi-path recall retrieval method based on dynamic weight allocation according to claim 1, characterized in that, The dynamic weight allocation algorithm in step S3 includes: Step S31: Calculate the probability distribution of each query type: Where F is the multidimensional query feature vector, and c is the query type. and These are model parameters; Step S32: Calculate the basic weights: in, Preset weight configurations for each query type; Step S33: Calculate the dynamic adjustment factor: in, For the i-th feature, These are the corresponding weighting coefficients, and β and γ are adjustable parameters; Step S34: Calculate the final weights: in, Represents element-wise product. This indicates that the normalization process makes the sum of the weights equal to 1.

3. The multi-path recall retrieval method based on dynamic weight allocation according to claim 1, characterized in that, In step S5, the fusion sorting adopts the AMSE-RRF algorithm, including: Step S51: Perform deduplication and score standardization on the recall results for each route; Step S52: Calculate the confidence score of each document's ranking in each path: ； in, For the ranking of one of the routes, For parameters; Step S53: Calculate the base RRF fusion score: in, The final weights are obtained in step S3, and k is a smoothing constant; Step S54: Introduce fractional enhancement terms: in, The original relevance score after standardization. An adaptive activation function; Step S55: Calculate the final fusion score: Where α, β, and γ are fusion coefficients and α+β+γ=1.

4. The multi-path recall retrieval method based on dynamic weight allocation according to claim 1, characterized in that, The vector semantic retrieval uses a pre-trained language model to generate query vectors, and performs an approximate nearest neighbor search in the vector database with a sublinear time complexity of O(log n) through the HNSW graph index.

5. The multi-path recall retrieval method based on dynamic weight allocation according to claim 1, characterized in that, The keyword retrieval uses an improved BM25F algorithm, combined with field weights, length normalization, and query expansion techniques. The query expansion includes synonyms, near-synonyms, and pinyin similar words.

6. The multi-path recall retrieval method based on dynamic weight allocation according to claim 1, characterized in that, The entity retrieval is based on a named entity recognition model to extract entities such as names of people, places, organizations, times, and professional terms from the query, and performs entity linking and relationship reasoning in the knowledge graph to return relevant documents.

7. The multi-path recall retrieval method based on dynamic weight allocation according to claim 3, characterized in that, In the AMSE-RRF fusion algorithm, score standardization uses the Z-score method or min-max normalization to map the original scores of each path to a unified interval.

8. The multi-path recall retrieval method based on dynamic weight allocation according to claim 1, characterized in that, After the results are fused, the Top-20 candidate documents are re-ranked based on deep learning. A pre-trained re-ranking model is used to calculate the query-document relevance, and the re-ranking score and the fused score are fused in a weighted manner, wherein the weight of the re-ranking score is not less than 0.

7.

9. A multi-path recall retrieval system based on dynamic weight allocation, characterized in that, include: The query preprocessing and feature extraction module is used to clean, segment, and identify entities from user-input queries, and generate a 13-dimensional query feature vector. The query type identification module is used to identify the query type as keyword-oriented, semantic-oriented, balanced, or complex based on the 13-dimensional query feature vector using a support vector machine and random forest ensemble classifier. The dynamic weight calculation module is used to calculate the final weights of vector semantic retrieval, keyword retrieval, and entity retrieval based on the query type and 13-dimensional feature vector, using the dynamic weight allocation algorithm described in claim 1. The multi-channel recall module includes: a. Vector semantic retrieval unit, which realizes semantic nearest neighbor search based on vector database and HNSW index; b. Keyword retrieval unit, which achieves keyword matching based on the improved BM25F algorithm and inverted index; c. Entity retrieval unit, based on the pipeline of "large model entity extraction + entity linking (disambiguation) + knowledge graph relationship retrieval": first, extract entities from the query and link them to unique nodes in the graph; then, retrieve the entity and its neighborhood relationships based on this, and output related documents and structured relationship views; The results fusion module is used to fuse the three-way recall results using the AMSE-RRF algorithm, including score standardization, ranking confidence assessment, weighted RRF calculation, score enhancement, and diversity optimization. The reordering module is used to perform deep semantic reordering of the fused Top-K results based on the Transformer architecture and output the final sorted list. The system achieves millisecond-level adaptive retrieval strategy adjustment through a dynamic weight allocation mechanism, thereby improving retrieval accuracy and robustness in complex query scenarios.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the multi-way recall retrieval method based on dynamic weight allocation as described in any one of claims 1 to 8.

Images