Data search enhancement and rearrangement method, system and device

Abstract

The present invention belongs to the field of data search enhancement, and specifically relates to a data search enhancement rearrangement method, system and device, which aims to solve the problem of low accuracy of existing RAG technology due to local matching, single representation and noise interference. The present invention includes: constructing a keyword library; dividing professional knowledge and documents into multiple text blocks to generate a first high-dimensional feature vector; semantically segmenting each text block and extracting the second high-dimensional feature vector of each semantic unit; extracting keywords of query information and high-dimensional feature vectors of each rewritten query; screening multiple text blocks that match the keywords of the query information; matching the high-dimensional feature vectors corresponding to the keywords of the query information with the first high-dimensional feature vector, and performing secondary screening based on the transition probability distance; calculating the spherical divergence distance between the high-dimensional feature vector corresponding to the rewritten query and the second high-dimensional feature vector after the secondary screening, and rearranging. The present invention combines multiple semantic features to improve search accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of data search enhancement, and in particular relates to a data search enhancement rearrangement method, system and device. Background Art

[0002] In recent years, large language models (LLMs), such as ChatGPT, have demonstrated capabilities approaching or even exceeding human performance in text comprehension, generation, and logical reasoning, driving the intelligent transformation of various industries. However, the core technical framework of large models still has significant flaws:

[0003] Randomness of autoregressive prediction: The model generates content word by word based on historical tokens, and its probabilistic output mechanism leads to uncertainty in the generated results.

[0004] Pre-training data noise and knowledge limitations: The lack of timeliness and accuracy of the training corpus makes the model prone to "hallucinations" in open-domain or professional field tasks, that is, generating content that is inconsistent with facts or logically incorrect.

[0005] To alleviate these issues, Retrieval-Augmented Generation (RAG) has become a mainstream solution. It retrieves information from external knowledge bases or the internet, feeding relevant text snippets into the model as context to constrain the accuracy of generated content. However, existing RAG solutions still face the following bottlenecks:

[0006] Insufficient retrieval accuracy: Traditional methods rely on local similarity metrics such as cosine similarity for retrieval, which makes it difficult to distinguish content that is semantically similar but factually conflicting.

[0007] Lack of distribution modeling: Existing solutions use a single vector to represent queries and text blocks, ignoring the semantic distribution characteristics of text. This leads to poor adaptability to long-tail queries or complex semantic scenarios.

[0008] Significant noise interference: In the candidate text blocks retrieved initially, the misleading effects of low-relevance content on the generation phase are difficult to eliminate through simple threshold screening, and the accuracy rate in actual applications is usually only 70%-80%.

[0009] Based on this, the present invention proposes a data search enhancement and rearrangement method, system and device. Summary of the Invention

[0010] In order to solve the above-mentioned problems in the prior art, namely, the problem that the existing RAG technology has low accuracy due to local matching, single representation and noise interference, the present invention provides a data search enhancement and rearrangement method, system and device.

[0011] A first aspect of the present invention provides a data search enhancement and rearrangement method, the method comprising:

[0012] Collect existing domain expertise and documents, extract keywords, and build a keyword library corresponding to the domain;

[0013] Segmenting the professional knowledge and documents into multiple text blocks of fixed length, extracting keywords from each text block based on a keyword library, and generating a first high-dimensional feature vector using a text representation model;

[0014] Perform semantic segmentation on each text block to generate multiple semantic units, and extract the high-dimensional feature vector of each semantic unit through the text representation model to obtain the second high-dimensional feature vector;

[0015] Extracting keywords of the input query information based on the keyword library, and extracting high-dimensional feature vectors corresponding to the keywords of the query information; obtaining multiple semantically equivalent rewritten queries of the query information, and extracting high-dimensional feature vectors corresponding to each rewritten query;

[0016] Filtering multiple text blocks that match the keywords of the query information as candidate text blocks;

[0017] Matching the high-dimensional feature vector corresponding to the keyword of the query information with the first high-dimensional feature vector, and performing a secondary screening on the candidate text blocks based on the transition probability distance;

[0018] The spherical divergence distance between the high-dimensional feature vector corresponding to the rewritten query and the second high-dimensional feature vector after secondary screening is calculated, and the text blocks after secondary screening are rearranged based on the spherical divergence distance.

[0019] Furthermore, the candidate text blocks are screened again based on the transition probability distance, and the method is as follows:

[0020] Combining the high-dimensional feature vector corresponding to the keyword of the query information and the first high-dimensional feature vectors of all candidate text blocks into a joint feature set;

[0021] Calculating the cosine similarity between each feature vector and other feature vectors in the joint feature set;

[0022] For each eigenvector, retain the similarities corresponding to the first K neighbors with the highest cosine similarity, set the remaining similarities to zero, obtain the similarity matrix, and normalize it row by row to obtain the probability transfer matrix;

[0023] Based on the probability transfer matrix, the transition probability distance between the high-dimensional feature vector corresponding to the keyword of the query information and the first high-dimensional feature vector of each candidate text block is calculated, and the candidate text blocks are secondary screened according to the order of the transition probability distance and a preset distance threshold.

[0024] Furthermore, the transition probability distance , which is calculated as follows:

[0025] ;

[0026] in, is the high-dimensional feature vector corresponding to the keyword of the query information, For the i The first high-dimensional feature vector of the candidate text block, N is the number of candidate text blocks, for Transfer to j The probability of a feature vector, for Transfer to j The probability of a feature vector.

[0027] Furthermore, the similarity matrix is ​​calculated as follows:

[0028] ;

[0029] in, is the first in the joint feature set j feature vectors, for and The similarity matrix between .

[0030] Furthermore, the spherical divergence distance is calculated by:

[0031] The high-dimensional feature vector set of each rewritten query is recorded as , the second highest dimensional feature vector set after secondary screening is recorded as ;in, is the number of eigenvectors, m for 's index;

[0032] calculate and The Euclidean distance between any two eigenvectors ;in, , ;

[0033] The spherical divergence distance of each set of feature vectors is calculated based on the Euclidean distance.

[0034] Furthermore, the spherical divergence distance of each set of feature vectors is calculated based on the Euclidean distance , the method is:

[0035] ;

[0036] in, represents the intra-group distance between high-dimensional feature vectors of the rewritten query, is the intra-group distance between the second highest dimensional feature vectors after secondary screening, It is a loop index variable used to traverse the elements in the collection. Its value range is , for The feature vectors.

[0037] Furthermore, the text blocks after the secondary screening are rearranged based on the spherical divergence distance, and the method is as follows:

[0038] The text blocks corresponding to the second highest dimensional feature vectors after the secondary screening are sorted in ascending order based on the spherical divergence distance to obtain a rearrangement result.

[0039] In a second aspect, the present invention provides a data search enhancement and rearrangement system. Based on a data search enhancement and rearrangement method, the system includes:

[0040] A keyword library construction module is configured to collect professional knowledge and documents in existing fields and extract keywords to build a keyword library corresponding to the field;

[0041] a first high-dimensional feature vector generation module configured to segment the professional knowledge and documents into a plurality of text blocks of fixed length, extract keywords from each text block based on a keyword library, and generate a first high-dimensional feature vector using a text representation model;

[0042] A second high-dimensional feature vector generation module is configured to perform semantic segmentation on each text block to generate multiple semantic units, and extract a high-dimensional feature vector of each semantic unit through a text representation model to obtain a second high-dimensional feature vector;

[0043] a query information processing module configured to extract keywords from the input query information based on the keyword library, and extract high-dimensional feature vectors corresponding to the keywords of the query information; obtain multiple semantically equivalent rewritten queries of the query information, and extract high-dimensional feature vectors corresponding to each rewritten query;

[0044] a candidate text block preliminary screening module configured to screen multiple text blocks that match the keywords of the query information as candidate text blocks;

[0045] a candidate text block secondary screening module configured to match the high-dimensional feature vector corresponding to the keyword of the query information with the first high-dimensional feature vector, and perform secondary screening on the candidate text blocks based on the transition probability distance;

[0046] The rearrangement module is configured to calculate the spherical divergence distance between the high-dimensional feature vector corresponding to the rewritten query and the second high-dimensional feature vector after secondary screening, and rearrange the text blocks after secondary screening based on the spherical divergence distance.

[0047] A third aspect of the present invention provides an electronic device, comprising:

[0048] at least one processor; and

[0049] a memory communicatively connected to at least one of the processors; wherein,

[0050] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the above-mentioned data search enhancement and rearrangement method.

[0051] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to be executed by the computer to implement the above-mentioned data search enhancement and rearrangement method.

[0052] Beneficial effects of the present invention:

[0053] Improve domain adaptability and retrieval accuracy: A text block screening mechanism based on a domain keyword library, combined with keyword matching and high-dimensional semantic feature matching, effectively distinguishes text content with similar semantics but conflicting facts, significantly improving the accuracy of retrieval in professional fields and open domains.

[0054] Enhanced complex semantic modeling capabilities: Through semantic unit segmentation and multi-granularity feature representation (first and second high-dimensional feature vectors) of text blocks, it breaks through the limitations of traditional single vector representation, accurately captures long-tail queries, polysemous expressions, and implicit semantic associations, and improves the robustness of semantic matching in complex scenarios.

[0055] Reduce noise interference and mismatch risks: Adopt a three-level filtering mechanism of "keyword primary screening - transfer probability secondary screening - spherical divergence rearrangement", model contextual relevance through transfer probability distance, combine spherical divergence distance to quantify semantic distribution differences, suppress the interference of low-relevance text blocks on the generation stage, and reduce the model's "hallucination" phenomenon.

[0056] Optimize the factual consistency of generated content: By re-ranking and optimizing search results, ensure that highly relevant text blocks are input into the large model first, strengthen the anchoring effect between the generated content and the authoritative knowledge base, and improve the logical accuracy and factual credibility of the generated results.

[0057] System flexibility and scalability: The modular design supports dynamic updates of the keyword library and independent optimization of the text representation model, adapting to different fields and multilingual scenarios, reducing deployment costs, and providing an efficient technical foundation for the industry's intelligent transformation. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0059] Figure 1 It is a flow chart of a data search enhancement and rearrangement method of the present invention. DETAILED DESCRIPTION

[0060] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0061] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0062] The present invention provides a data search enhancement and rearrangement method, the method comprising:

[0063] Step S10: Collecting professional knowledge and documents in existing fields and extracting keywords to build a keyword library corresponding to the field;

[0064] Step S20: Segment the professional knowledge and documents into multiple text blocks of fixed length, extract keywords from each text block based on a keyword library, and generate a first high-dimensional feature vector using a text representation model;

[0065] Step S30, performing semantic segmentation on each text block to generate multiple semantic units, and extracting a high-dimensional feature vector of each semantic unit through a text representation model to obtain a second high-dimensional feature vector;

[0066] Step S40: extracting keywords of the input query information based on the keyword library, and extracting high-dimensional feature vectors corresponding to the keywords of the query information; obtaining multiple semantically equivalent rewritten queries of the query information, and extracting high-dimensional feature vectors corresponding to each rewritten query;

[0067] Step S50, screening multiple text blocks that match the keywords of the query information as candidate text blocks;

[0068] Step S60, matching the high-dimensional feature vector corresponding to the keyword of the query information with the first high-dimensional feature vector, and performing a secondary screening on the candidate text blocks based on the transition probability distance;

[0069] Step S70 , calculating the spherical divergence distance between the high-dimensional feature vector corresponding to the rewritten query and the second high-dimensional feature vector after secondary screening, and rearranging the text blocks after secondary screening based on the spherical divergence distance.

[0070] In order to more clearly illustrate a data search enhancement and rearrangement method of the present invention, the following is combined with Figure 1 Steps S10 to S70 in the embodiment of the present invention are described in detail. Each step is described in detail as follows:

[0071] Step S10: Collecting professional knowledge and documents in existing fields and extracting keywords to build a keyword library corresponding to the field;

[0072] The professional knowledge in this embodiment refers to the expert knowledge and experience in the field, and the documents are relevant documents in the existing field. Based on the expert knowledge and experience in the field, combined with the relevant documents in the existing field, a large language model (such as deepseek-R1) is used to summarize and extract corresponding keywords to establish a keyword library.

[0073] Step S20: Segment the professional knowledge and documents into multiple text blocks of fixed length, extract keywords from each text block based on a keyword library, and generate a first high-dimensional feature vector using a text representation model;

[0074] In this embodiment, the document is first divided into different text blocks based on a fixed length (for example, 1024). For each text block, keywords are extracted using a large language model (LLM) based on a keyword library, and high-dimensional representations are extracted using a text representation model.

[0075] The BERT model can be used as the text representation model.

[0076] The high-dimensional feature vector in the present invention refers to converting a text block into a numerical vector of fixed dimensions (such as 768 dimensions, 1024 dimensions), where each dimension corresponds to a potential semantic or grammatical feature.

[0077] Step S30, performing semantic segmentation on each text block to generate multiple semantic units, and extracting a high-dimensional feature vector of each semantic unit through a text representation model to obtain a second high-dimensional feature vector;

[0078] In this embodiment, each text block is segmented into several semantics using a large language model (LLM), and then a text representation model is used to extract a high-dimensional representation of each semantic.

[0079] The present invention introduces the capability of Large Language Model (LLM), which can adaptively extract semantic information from text blocks segmented from the knowledge base, making up for the representation defects caused by missing / excessive semantics brought about by existing fixed-length segmentation.

[0080] Step S40: extracting keywords of the input query information based on the keyword library, and extracting high-dimensional feature vectors corresponding to the keywords of the query information; obtaining multiple semantically equivalent rewritten queries of the query information, and extracting high-dimensional feature vectors corresponding to each rewritten query;

[0081] In this embodiment, the large language model (LLM) is also used to extract keywords from the input query information, and rewrite them into several different expressions while preserving the semantics.

[0082] Step S50, screening multiple text blocks that match the keywords of the query information as candidate text blocks;

[0083] In this embodiment, preliminary corresponding candidate text blocks are screened based on the matching similarity technology (TF-IDF).

[0084] Step S60, matching the high-dimensional feature vector corresponding to the keyword of the query information with the first high-dimensional feature vector, and performing a secondary screening on the candidate text blocks based on the transition probability distance;

[0085] In this embodiment, the candidate text blocks are screened twice based on the transition probability distance, and the method is as follows:

[0086] Combining the high-dimensional feature vector corresponding to the keyword of the query information and the first high-dimensional feature vectors of all candidate text blocks into a joint feature set;

[0087] Calculating the cosine similarity between each feature vector and other feature vectors in the joint feature set;

[0088] For each eigenvector, retain the similarities corresponding to the first K neighbors with the highest cosine similarity, set the remaining similarities to zero, obtain the similarity matrix, and normalize it row by row to obtain the probability transfer matrix;

[0089] Based on the probability transfer matrix, the transition probability distance between the high-dimensional feature vector corresponding to the keyword of the query information and the first high-dimensional feature vector of each candidate text block is calculated, and the candidate text blocks are secondary screened according to the order of the transition probability distance and a preset distance threshold.

[0090] Assume that the high-dimensional feature vector corresponding to the query information and the second high-dimensional feature vector corresponding to the candidate text block are and , then we can calculate a Similarity matrix S , which satisfies:

[0091] ;

[0092] in, is the first in the joint feature set j feature vectors, for and The similarity matrix between .

[0093] calculate , we can get The probability transfer matrix P ,in , so our transition probability distance , which is calculated as follows:

[0094] ;

[0095] in, is the high-dimensional feature vector corresponding to the keyword of the query information, For the i The first high-dimensional feature vector of the candidate text block, N is the number of candidate text blocks, for Transfer to j The probability of a feature vector, for Transfer to j The probability of a feature vector.

[0096] Specifically, i Represents the index of the eigenvector, corresponding to the row number of the matrix.

[0097] i =0, indicating the high-dimensional feature vector corresponding to the query information keyword .

[0098] i =1, 2, ..., N When , it represents the first high-dimensional eigenvector .

[0099] j Represents the index of the target eigenvector, corresponding to the column number of the matrix.

[0100] j = 0, the target is a high-dimensional feature vector .

[0101] j =1, 2, ..., N When the target is j Candidate text block feature vector .

[0102] During the high-dimensional feature comparison and screening phase, we introduced the transition probability distance, a method distinct from the traditional cosine distance. This distance has excellent properties: its value range is [0, 1], and its distance is 0 only when two points have identical neighbors. This distance is also unaffected by the data distribution, providing a uniform distance standard for any data distribution. Compared to the cosine distance, this distance significantly improves screening robustness.

[0103] Step S70 , calculating the spherical divergence distance between the high-dimensional feature vector corresponding to the rewritten query and the second high-dimensional feature vector after secondary screening, and rearranging the text blocks after secondary screening based on the spherical divergence distance.

[0104] In this embodiment, the ball divergence distance is calculated as follows:

[0105] The high-dimensional feature vector set of each rewritten query is recorded as , the second highest dimensional feature vector set after secondary screening is recorded as ;in, is the number of eigenvectors, m for 's index;

[0106] calculate and The Euclidean distance between any two eigenvectors ;in, , ;

[0107] The spherical divergence distance of each set of feature vectors is calculated based on the Euclidean distance.

[0108] Calculate the spherical divergence distance of each set of eigenvectors based on the Euclidean distance , the method is:

[0109] ;

[0110] in, represents the intra-group distance between high-dimensional feature vectors of the rewritten query, is the intra-group distance between the second highest dimensional feature vectors after secondary screening, It is a loop index variable used to traverse the elements in the collection. Its value range is , for The feature vectors.

[0111] In this embodiment, the text blocks after the secondary screening are rearranged based on the spherical divergence distance, and the method is as follows:

[0112] The text blocks corresponding to the second highest dimensional feature vectors after the secondary screening are sorted in ascending order based on the spherical divergence distance to obtain a rearrangement result.

[0113] Although the various steps in the above embodiment are described in the above-mentioned order, those skilled in the art will understand that in order to achieve the effect of this embodiment, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in a reverse order. These simple changes are within the scope of protection of the present invention.

[0114] A data search enhancement and rearrangement system according to a second embodiment of the present invention is based on a data search enhancement and rearrangement method. The system includes:

[0115] A keyword library construction module is configured to collect professional knowledge and documents in existing fields and extract keywords to build a keyword library corresponding to the field;

[0116] a first high-dimensional feature vector generation module configured to segment the professional knowledge and documents into a plurality of text blocks of fixed length, extract keywords from each text block based on a keyword library, and generate a first high-dimensional feature vector using a text representation model;

[0117] A second high-dimensional feature vector generation module is configured to perform semantic segmentation on each text block to generate multiple semantic units, and extract a high-dimensional feature vector of each semantic unit through a text representation model to obtain a second high-dimensional feature vector;

[0118] a query information processing module configured to extract keywords from the input query information based on the keyword library, and extract high-dimensional feature vectors corresponding to the keywords of the query information; obtain multiple semantically equivalent rewritten queries of the query information, and extract high-dimensional feature vectors corresponding to each rewritten query;

[0119] a candidate text block preliminary screening module configured to screen multiple text blocks that match the keywords of the query information as candidate text blocks;

[0120] a candidate text block secondary screening module configured to match the high-dimensional feature vector corresponding to the keyword of the query information with the first high-dimensional feature vector, and perform secondary screening on the candidate text blocks based on the transition probability distance;

[0121] The rearrangement module is configured to calculate the spherical divergence distance between the high-dimensional feature vector corresponding to the rewritten query and the second high-dimensional feature vector after secondary screening, and to rearrange the text blocks after secondary screening based on the spherical divergence distance. Those skilled in the art will clearly understand that for ease and brevity of description, the specific operating process and related description of the system described above can be referred to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0122] It should be noted that the data search enhancement and rearrangement system provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be decomposed or combined. For example, the modules in the above embodiment can be combined into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps and are not regarded as improper limitations on the present invention.

[0123] An electronic device according to a third embodiment of the present invention includes:

[0124] at least one processor; and

[0125] a memory communicatively connected to at least one of the processors; wherein,

[0126] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the above-mentioned data search enhancement and rearrangement method.

[0127] A fourth embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to be executed by the computer to implement the above-mentioned data search enhancement and rearrangement method.

[0128] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes and related instructions of the storage device and processing device described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0129] Those skilled in the art should be able to appreciate that the modules and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two, and the programs corresponding to the software modules and method steps can be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art. In order to clearly illustrate the interchangeability of electronic hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0130] The terms "first", "second", etc. are used to distinguish similar objects, rather than to describe or indicate a particular order or sequence.

[0131] The term "comprise" or any other similar term is intended to cover non-exclusive inclusion such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0132] Thus far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art may make equivalent changes or substitutions to the relevant technical features, and the technical solutions after such changes or substitutions will fall within the scope of protection of the present invention.

Claims

1. A data search enhancement and rearrangement method, characterized in that: The method includes: Collect existing domain expertise and documents, extract keywords, and build a keyword library corresponding to the domain; Segmenting the professional knowledge and documents into multiple text blocks of fixed length, extracting keywords from each text block based on a keyword library, and generating a first high-dimensional feature vector using a text representation model; Perform semantic segmentation on each text block to generate multiple semantic units, and extract the high-dimensional feature vector of each semantic unit through the text representation model to obtain the second high-dimensional feature vector; Extracting keywords of the input query information based on the keyword library, and extracting high-dimensional feature vectors corresponding to the keywords of the query information; obtaining multiple semantically equivalent rewritten queries of the query information, and extracting high-dimensional feature vectors corresponding to each rewritten query; Filtering multiple text blocks that match the keywords of the query information as candidate text blocks; Matching the high-dimensional feature vector corresponding to the keyword of the query information with the first high-dimensional feature vector, and performing a secondary screening on the candidate text blocks based on the transition probability distance; Calculating the spherical divergence distance between the high-dimensional feature vector corresponding to the rewritten query and the second high-dimensional feature vector after secondary screening, and rearranging the text blocks after secondary screening based on the spherical divergence distance; The ball divergence distance is: The high-dimensional feature vector set of each rewritten query is recorded as , the second highest dimensional feature vector set after secondary screening is recorded as ;in, is the number of eigenvectors, m is 's index; calculate and The Euclidean distance between any two eigenvectors ;in, , ; Calculating the spherical divergence distance of each set of eigenvectors based on the Euclidean distance; Calculate the spherical divergence distance of each set of eigenvectors based on the Euclidean distance , the method is: ； in, represents the intra-group distance between high-dimensional feature vectors of the rewritten query, is the intra-group distance between the second highest dimensional feature vectors after secondary screening, It is a loop index variable used to traverse the elements in the collection. Its value range is , for The feature vectors.

2. A data search enhancement and rearrangement method according to claim 1, characterized in that: The candidate text blocks are screened twice based on the transition probability distance. The method is as follows: Combining the high-dimensional feature vector corresponding to the keyword of the query information and the first high-dimensional feature vectors of all candidate text blocks into a joint feature set; Calculating the cosine similarity between each feature vector and other feature vectors in the joint feature set; For each eigenvector, retain the similarities corresponding to the first K neighbors with the highest cosine similarity, set the remaining similarities to zero, obtain the similarity matrix, and normalize it row by row to obtain the probability transfer matrix; Based on the probability transfer matrix, the transition probability distance between the high-dimensional feature vector corresponding to the keyword of the query information and the first high-dimensional feature vector of each candidate text block is calculated, and the candidate text blocks are secondary screened according to the order of the transition probability distance and a preset distance threshold.

3. A data search enhancement and rearrangement method according to claim 2, characterized in that: Transition probability distance , which is calculated as follows: ； in, is the high-dimensional feature vector corresponding to the keyword of the query information, is the first high-dimensional feature vector of the i-th candidate text block, N is the number of candidate text blocks, for The probability of transferring to the jth eigenvector, for The probability of transferring to the jth eigenvector.

4. A data search enhancement and rearrangement method according to claim 3, characterized in that: The similarity matrix is ​​calculated as follows: ； in, is the jth eigenvector in the joint feature set, for and The similarity matrix between .

5. The data search enhancement and rearrangement method according to claim 1, characterized in that: The text blocks after the secondary screening are rearranged based on the spherical divergence distance, and the method is as follows: The text blocks corresponding to the second highest dimensional feature vectors after the secondary screening are sorted in ascending order based on the spherical divergence distance to obtain a rearrangement result.

6. A data search enhancement and rearrangement system, based on a data search enhancement and rearrangement method according to any one of claims 1 to 5, characterized in that: The system includes: A keyword library construction module is configured to collect professional knowledge and documents in existing fields and extract keywords to build a keyword library corresponding to the field; a first high-dimensional feature vector generation module configured to segment the professional knowledge and documents into a plurality of text blocks of fixed length, extract keywords from each text block based on a keyword library, and generate a first high-dimensional feature vector using a text representation model; A second high-dimensional feature vector generation module is configured to perform semantic segmentation on each text block to generate multiple semantic units, and extract a high-dimensional feature vector of each semantic unit through a text representation model to obtain a second high-dimensional feature vector; a query information processing module configured to extract keywords from the input query information based on the keyword library, and extract high-dimensional feature vectors corresponding to the keywords of the query information; obtain multiple semantically equivalent rewritten queries of the query information, and extract high-dimensional feature vectors corresponding to each rewritten query; a candidate text block preliminary screening module configured to screen multiple text blocks that match the keywords of the query information as candidate text blocks; a candidate text block secondary screening module configured to match the high-dimensional feature vector corresponding to the keyword of the query information with the first high-dimensional feature vector, and perform secondary screening on the candidate text blocks based on the transition probability distance; a rearrangement module configured to calculate a spherical divergence distance between a high-dimensional feature vector corresponding to the rewritten query and a second high-dimensional feature vector after secondary screening, and rearrange the text blocks after secondary screening based on the spherical divergence distance; The ball divergence distance is: The high-dimensional feature vector set of each rewritten query is recorded as , the second highest dimensional feature vector set after secondary screening is recorded as ;in, is the number of eigenvectors, m is 's index; calculate and The Euclidean distance between any two eigenvectors ;in, , ; Calculating the spherical divergence distance of each set of eigenvectors based on the Euclidean distance; Calculate the spherical divergence distance of each set of eigenvectors based on the Euclidean distance , the method is: ； in, represents the intra-group distance between high-dimensional feature vectors of the rewritten query, is the intra-group distance between the second highest dimensional feature vectors after secondary screening, It is a loop index variable used to traverse the elements in the collection. Its value range is , for The feature vectors.

7. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to at least one of the processors; wherein, The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement a data search enhancement and rearrangement method based on any one of claims 1-5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to be executed by the computer to implement a data search enhancement and rearrangement method according to any one of claims 1 to 5.

Images