Integrated knowledge management method and device, computer equipment and medium

Abstract

The embodiment of the invention discloses an integrated knowledge management method and device, computer equipment and a medium. The method comprises the following steps: constructing a vector database; obtaining a retrieval demand of a user, and vectorizing the retrieval demand to obtain a dense vector and a sparse vector of the retrieval demand; performing similarity calculation to obtain a first retrieval result and a second retrieval result, and fusing the first retrieval result and the second retrieval result in a reciprocal sorting fusion mode to obtain a similar fragment list; a rearrangement device based on a large model is used for rearranging the fused sorting result, similar document fragments are obtained, the document fragments with similarity scores smaller than a threshold value are filtered out, and a final fragment set is obtained; and structuring the document fragments in the final fragment set by using a large language model, extracting and outputting a Markdown document, and performing mind mapping visualization on the content. According to the invention, knowledge can be quickly managed through a data processing pipeline, and the knowledge acquisition efficiency in the online learning process of the user can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of online intelligent education technology, and in particular to an integrated knowledge management method, device, computer equipment and medium. Background Art

[0002] With the rapid development of Internet technology and online education, online education has become an important part of the education field. Online education platforms have brought together a large number of teaching resources, including but not limited to course content, handouts, academic literature, and community discussions, all of which have rich knowledge content and constitute a huge knowledge base. However, how to effectively integrate and deliver this huge amount of knowledge is a challenging task. For example, if a user wants to obtain specific knowledge on an education platform, it is usually done manually, by asking the platform operator, or by relying on Internet inquiries. Because knowledge is time-sensitive, and the relevant information on the Internet is huge and complicated. The above-mentioned traditional manual method lacks flexibility, and as knowledge is updated, it is also necessary to manually organize and collect this knowledge again.

[0003] The prior art has the following problems:

[0004] (1) Low efficiency of knowledge updating and integration: Relying on manual data search, filtering, and integration leads to a waste of human resources.

[0005] (2) Insufficient automation in document processing and information extraction: The processing and analysis efficiency of large-scale unstructured content (such as PDF handouts, mind map images, community discussion records, etc.) is insufficient.

[0006] (3) Knowledge visualization lacks flexibility: Currently, online education platforms mostly rely on manual organization of knowledge in advance and then organize it into visualization images. These visualization results are often static and cannot be changed according to the user's wishes. They lack flexibility and require professionals to organize the knowledge before visualization. Summary of the invention

[0007] The purpose of the embodiments of the present invention is to provide an integrated knowledge management method, apparatus, computer equipment and medium, which can quickly manage knowledge through a data processing pipeline and effectively improve the knowledge acquisition efficiency of users in the online learning process.

[0008] To achieve the above objectives, in a first aspect, an embodiment of the present invention provides an integrated knowledge management method, including:

[0009] Constructing a vector database; the vector database stores dense vectors and sparse vectors of slice contents;

[0010] Obtaining a user's search requirement, vectorizing the search requirement, and obtaining a dense vector and a sparse vector of the search requirement;

[0011] Calculate similarity based on the dense vector and sparse vector of the shard content and the search requirement to obtain a first search result and a second search result, and fuse the first search result and the second search result in a reciprocal sorting fusion manner to obtain a similar shard list;

[0012] Use the large model-based reranker LLMReranker to rerank the fused sorting results to obtain N similar document fragments, filter out document fragments with similarity scores less than the threshold, and obtain the final fragment set;

[0013] The document fragments in the final fragment set are structured using a large language model, Markdown documents are extracted and output, and the contents of the Markdown documents are visualized as a mind map.

[0014] As a specific implementation of the present application, constructing a vector database is specifically as follows:

[0015] Obtaining manually uploaded data files to be managed, and performing content parsing, extraction, cleaning and format processing on the data files to be managed according to different data file types to obtain parsed files;

[0016] Segment the parsed file using a sentence-based segmentation algorithm or a hierarchy-based segmentation algorithm to obtain segment contents;

[0017] Performing dense vectorization and sparse vectorization on the slice content to obtain a dense vector and a sparse vector of the slice content;

[0018] A vector database is used to store dense vectors and sparse vectors of the slice content.

[0019] As a specific implementation of this application, the parsed file is obtained as follows:

[0020] If the uploaded file is a PDF, the OCR module is used to identify the PDF content, and the PDF content is annotated using the PDF content annotation model. The header, footer, and directory are deleted, the table is converted into an image, and the image is saved as a link. Finally, the text, first and second level titles, and image links are converted into content that conforms to the Markdown format.

[0021] If the uploaded file is in Excel, CSV or JSON format, the content will be parsed according to fixed rules.

[0022] As a specific implementation method of the present application, the API is called or a locally deployed Embedding model is used to densely vectorize the shard content and retrieval requirements; a word frequency-based algorithm is used to sparsely vectorize the shard content and retrieval requirements.

[0023] As a specific implementation of the present application, similarity calculation is performed based on the dense vector and sparse vector of the shard content and the search requirement to obtain the first search result and the second search result, and the first search result and the second search result are merged in a reciprocal sorting fusion manner, specifically:

[0024] The dense retriever of the hybrid retriever calculates the similarity between the dense vector of the retrieval requirement and the dense vector of the shard content, and returns the top N most similar document shard DNs;

[0025] The sparse searcher of the hybrid searcher calculates the similarity between the sparse vector of the search requirement and the sparse vector of the shard content, and returns the top N most similar document shard SNs;

[0026] The document fragment DN and the document fragment SN are merged using the RRF algorithm.

[0027] As a specific implementation of the present application, the document fragment DN and the document fragment SN are merged using the RRF algorithm, specifically:

[0028] The RRF algorithm converts the similarity scores in the document shard DN and the document shard SN into reciprocals, and then adds up all the reciprocals to obtain the fused ranking score.

[0029] In a second aspect, an embodiment of the present invention provides an integrated knowledge management device, including:

[0030] A construction module, used to construct a vector database; the vector database stores dense vectors and sparse vectors of slice contents;

[0031] A demand acquisition module is used to acquire the user's search demand, vectorize the search demand, and obtain a dense vector and a sparse vector of the search demand;

[0032] A calculation fusion module is used to perform similarity calculation based on the dense vector and sparse vector of the shard content and the search requirement to obtain a first search result and a second search result, and fuse the first search result and the second search result in a reciprocal sorting fusion manner to obtain a similar shard list;

[0033] The re-ranking module is used to re-rank the fused sorting results using the large model-based re-ranker LLMReranker to obtain N similar document fragments, filter out document fragments with similarity scores less than a threshold, and obtain the final fragment set;

[0034] The visualization module is used to structure the document fragments in the final fragment set using a large language model, extract and output Markdown documents, and visualize the contents of the documents as a mind map.

[0035] In a third aspect, an embodiment of the present invention provides a computer device, comprising a non-volatile memory, a processor, a memory bus, a system bus, and an input / output bus, as well as a USB adapter, a graphics adapter, etc. connected to the input / output bus, and a computer program stored in the non-volatile memory and executable on the processor, wherein the processor implements the method described in the first aspect above when executing the computer program.

[0036] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. The computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the method of the first aspect.

[0037] The implementation of the present invention provides an efficient online education knowledge management solution through an integrated knowledge pipeline, which has the following advantages:

[0038] (1) It can automatically process and convert data files in various formats, perform content parsing, segmentation, vectorization, storage and retrieval, thereby avoiding repeated calculations.

[0039] (2) Through hybrid vector retrieval and large model rearrangement technology, accurate knowledge matching and sorting are achieved, and retrieval results are optimized based on threshold screening. Finally, knowledge is extracted into Markdown format and visualized as a mind map, which significantly improves the efficiency of knowledge management, search and application.

[0040] (3) It can effectively improve students’ knowledge acquisition efficiency in online education scenarios and promote personalized and adaptive online education. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the specific implementation of the present invention or the technical solution in the prior art, the drawings required for use in the specific implementation or the description of the prior art are briefly introduced below.

[0042] Figure 1 It is a schematic diagram of an application environment of an integrated knowledge management device in one embodiment of the present invention;

[0043] Figure 2 is a flow chart of a course-based knowledge management method in one embodiment of the present invention;

[0044] Figure 3 yes Figure 2 The sub-flow chart of step S1 is shown;

[0045] Figure 4 It is an example diagram of visual knowledge structure;

[0046] Figure 5 is a schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0049] In order to enable those skilled in the art to better understand the technical solutions provided by the embodiments of the present application, some technical terms involved in the embodiments of the present application are briefly explained below.

[0050] 1. OCR: Optical Character Recognition is a technology used to recognize printed or handwritten text from images or document scans and convert it into machine-encoded text. OCR technology allows computers to read text information in images and convert it into editable and searchable data.

[0051] 2. Vectorization: Text vectorization refers to the process of converting natural language text into numerical vectors, so that a vector can be used to represent a piece of text, so that many vector similarity algorithms or matrix algorithms can be directly used in text semantic analysis, similarity calculation, etc.

[0052] 3. Markdown: Markdown is a lightweight markup language designed to format documents through simple text symbols, making them easy to write and read. It uses intuitive grammatical rules to represent common text formats such as titles, lists, links, pictures, bold, italics, etc. without the need for complex typesetting tools.

[0053] The technical solution of the present invention will be clearly and detailedly described below in conjunction with the embodiments of the present invention.

[0054] An integrated knowledge management device provided by an embodiment of the present invention can be applied in Figure 1 In the application environment, the client can be a mobile phone, tablet, computer and other terminals, and the knowledge management background can be an application deployed on a local server or cloud server cluster. The online education platform uploads knowledge files to the knowledge management background, and users can view the corresponding knowledge content on the client when they need to search for knowledge points. The knowledge content is obtained through an integrated knowledge management device deployed in the knowledge management background.

[0055] like Figure 2 As shown, the integrated knowledge management method provided by the embodiment of the present invention includes the following steps:

[0056] S1, build vector database.

[0057] Specifically, Figure 3 As shown, step S1 includes:

[0058] S11, manually upload the data files to be managed, and parse, extract, clean and uniformly process the contents of the data files into md format according to the different types of data files.

[0059] Supported file formats include but are not limited to pdf, word, excel, txt, json and md.

[0060] If the uploaded file is a PDF, the OCR module is used to identify the PDF content, extract the text content, and use the PDF content annotation model to annotate different elements with type tags based on the location and content of the elements in the PDF, including "header, footer, directory, body, first-level title, second-level title, table, and picture". Delete the header, footer, and directory, convert the table into a picture, and then save the picture as a link. Finally, the body, first-level and second-level titles, and picture links are converted into content that conforms to the Markdown format.

[0061] If the uploaded file is a file type with a specific structure such as excel, csv, or json, the content is parsed according to fixed rules. For example, excel and csv can specify the header as the title of each row of content, the first column as the title of each row of content, or specify a column as the title of the content. Json files can specify specific fields as content titles.

[0062] S12, segmenting the parsed data file content.

[0063] The present invention uses two fragmentation modes: sentence-based fragmentation and hierarchy-based fragmentation.

[0064] Sentence-based segmentation method (SentenceSplit). According to Chinese punctuation marks, such as ".", ", etc.", the document content D is segmented to obtain S = [s1, ..., sn], which means that the document is segmented into n independent sentences according to the punctuation marks. Then these contents are merged at a fixed length threshold. For example, the specified length threshold is 1024, which means that S1, ... Sn are spliced ​​in sequence until the length is about to exceed 1024, then stop.

[0065] Hierarchical Split. Hierarchical Split is based on sentence-based segmentation. By setting multiple different length thresholds, the document is divided into multiple layers of segments with different maximum lengths. For example, if [1024, 512, 128] is specified, it means that the document content D is first segmented using the SentenceSplit method with a threshold of 1024 to obtain two segments DL1 = [Dl1, dl2]. Then, dl1 and dl2 are segmented using the SentenceSplit method with a threshold of 512 to obtain three segments DL2-1 = [dl2-1-1, dl2-1-2, dl2-1-3] and DL2-2 = [dl2-2-1, dl2-2-2, dl2-2-3]. The threshold length is 128, which is the same and will not be repeated.

[0066] S13, densely vectorize and sparsely vectorize the sliced ​​content to obtain dense_vector_context and sparse_vector_context of the sliced ​​content respectively.

[0067] Dense vectorization operations can usually be obtained by calling an API, such as OpenAI's Embedding API or Alibaba's DashVector API, or by using a locally deployed Embedding model, such as the BGE series model of Zhiyuan Research Institute. In this invention, the gte-Qwen2-7B-instruct model of Tongyi Qianwen is used to generate dense vectors.

[0068] Sparse vectors are usually generated using a word frequency-based algorithm such as BM25 or TFIDF. In the present invention, the BM25 algorithm is used to calculate and generate sparse vectors.

[0069] S14, using the vector database to save the obtained dense_vector_context and sparse_vector_context, so that they can be directly used when the user queries related knowledge, avoiding repetitive calculation work.

[0070] The vector database may select an online cloud database such as Alibaba's DashVector, or a local vector database may be deployed. In the present invention, a local vector database is constructed based on the Qdrant open source database.

[0071] S2, obtaining the user's search requirements, vectorizing the search requirements, and obtaining a dense vector and a sparse vector of the search requirements.

[0072] S3, performing similarity calculation based on the dense vector and sparse vector of the shard content and the search requirement to obtain a first search result and a second search result, and merging the first search result and the second search result in a reverse sorting fusion manner to obtain a similar shard list.

[0073] S4, use the large model-based reranker LLMReranker to rerank the fused sorting results to obtain N similar document fragments, filter out document fragments with similarity scores less than the threshold, and obtain the final fragment set.

[0074] Specifically, the user's search requirements are vectorized to obtain dense_vector_query and sparse_vector_query in the same manner as S13.

[0075] The hybrid retriever is used to calculate the similarity between the query vector and the context vector. The retrieval results of the dense vector and the retrieval results of the sparse vector are fused using the reciprocal sort fusion (RRF) method to obtain a similar fragment list Rs = [rs1, ...rsn], where rs1 to rsn represent the fused n similar document fragments.

[0076] The hybrid searcher consists of a dense searcher and a sparse searcher. The dense searcher calculates the similarity between the user query dense_vecotr_query and the dense_vector_context in S4, and returns the top N most similar document fragments represented as DN = [dn1,…,dnn]. Similarly, the sparse searcher returns the top N most similar document fragments represented as SN = [sn1,…,snn]. The dense searcher focuses on the similarity in text semantics, while the sparse searcher focuses on the similarity in text word frequency. The two have different measures of similarity, so the RRF algorithm is used to fuse the documents of DN and SN. The RRF algorithm converts the similarity scores in DN and SN into reciprocals, and then adds all the reciprocals to obtain the fused ranking score.

[0077]

[0078] Where k represents the ranking of a search result in the original search results (e.g., DN). The higher the ranking (i.e., the smaller k is), the larger its reciprocal is, and the higher the ranking in the fused results is. i is an empirical constant, usually 60. The fragmented results in DN and SN are fused according to the ranking scores of RRF to obtain RN = [rn1,…,rnn].

[0079] It should be noted that there are usually a large number of knowledge vectors in the knowledge management module, and the dense searcher and sparse searcher in S4 have the characteristics of fast speed but inaccurate effect. Therefore, in the present invention, the method in S4 is first used to quickly and preliminarily screen out a relatively similar candidate set from a huge knowledge vector database. The large language model can understand the complexity of natural language more finely, so it is effective to use a slower but more accurate large model to perform secondary sorting on the candidate set.

[0080] In this invention, based on the BGE big model of Zhiyuan Research Institute, a reranker LLMReranker is constructed to rerank the fused sorting results, and each group (query, rnn) is reranked to obtain N similar document fragments LLMR = [llmrn1, ..., llmrnn], and the documents in LLMR are sorted in descending order according to the similarity score. And through the pre-set threshold p, the document fragments with a similarity score less than p in LLMR are filtered out to obtain the final fragment set LLMRF = [llmrf1, ..., llmrfn].

[0081] S5, using a large language model to structure the document fragments in the final fragment set, extracting and outputting Markdown documents, and visualizing the contents of the Markdown documents in a mind map.

[0082] In this embodiment, the document fragments in LLMRF are structured and extracted and output as Markdown format documents based on the GLM large language model of Zhiyuan Research Institute, and the contents of the Markdown documents are visualized as mind maps, such as Figure 4 shown.

[0083] The implementation of the present invention provides an efficient online education knowledge management solution through an integrated knowledge pipeline, which has the following advantages:

[0084] (1) It can automatically process and convert data files in various formats, perform content parsing, segmentation, vectorization, storage and retrieval, thereby avoiding repeated calculations.

[0085] (2) Through hybrid vector retrieval and large model rearrangement technology, accurate knowledge matching and sorting are achieved, and retrieval results are optimized based on threshold screening. Finally, knowledge is extracted into Markdown format and visualized as a mind map, which significantly improves the efficiency of knowledge management, search and application.

[0086] (3) It can effectively improve students’ knowledge acquisition efficiency in online education scenarios and promote personalized and adaptive online education.

[0087] Based on the same inventive concept, the present invention also provides an integrated knowledge management device, comprising:

[0088] A construction module, used to construct a vector database; the vector database stores dense vectors and sparse vectors of slice contents;

[0089] A demand acquisition module is used to acquire the user's search demand, vectorize the search demand, and obtain a dense vector and a sparse vector of the search demand;

[0090] A calculation fusion module is used to perform similarity calculation based on the dense vector and sparse vector of the shard content and the search requirement to obtain a first search result and a second search result, and fuse the first search result and the second search result in a reciprocal sorting fusion manner to obtain a similar shard list;

[0091] The re-ranking module is used to re-rank the fused sorting results using the large model-based re-ranker LLMReranker to obtain N similar document fragments, filter out document fragments with similarity scores less than a threshold, and obtain the final fragment set;

[0092] The visualization module is used to structure the document fragments in the final fragment set using a large language model, extract and output Markdown documents, and visualize the contents of the documents as a mind map.

[0093] Optionally, an embodiment of the present invention further provides another integrated knowledge management device, including a knowledge management module and a knowledge retrieval module.

[0094] The knowledge management module includes: document upload and parsing module, document segmentation module, OCR module, vectorization module, and similarity calculation module. The knowledge retrieval module includes: vectorization module, similarity calculation module, knowledge segmentation rearrangement module, and visualization module. The vectorization module and similarity module are shared with the knowledge management module and knowledge retrieval module.

[0095] Document upload and parsing module: used to upload and parse the knowledge documents to be managed.

[0096] Document segmentation module: segment knowledge documents according to different methods.

[0097] OCR module: For images in knowledge documents or PDFs of image type, OCR recognition is used to obtain the text content.

[0098] Vectorization module: Densely vectorize and sparsely vectorize each text segment and knowledge point, representing them with a high-dimensional vector and a low-dimensional vector respectively, so as to obtain knowledge fragments and semantic information and keyword information of user queries.

[0099] Similarity calculation module: Cosine similarity is used to calculate the dense vectors of knowledge shards and user queries, and dot product operations of vectors are used to calculate the sparse vectors of knowledge shards and user queries to obtain the degree of correlation between knowledge shards and user queries.

[0100] Knowledge fragment re-ranking module: Use a large language model to semantically re-rank the first several similar document fragments.

[0101] Visualization module: Use a large language model to summarize several knowledge fragments and display the relationship between knowledge in the form of a mind map.

[0102] It should be noted that for more specific work processes of each module, please refer to the aforementioned method embodiments, which will not be repeated here.

[0103] Optionally, in another embodiment, the present invention further provides a computer device, which may be a cloud server or a local server, and its internal structure is as follows: Figure 5As shown. The computer device includes a processor, an I / O bridge, a memory bus, an input-output bus, a non-volatile storage medium, etc. The processor is connected to the I / O bridge through the system bus, the I / O bridge is connected to the memory bus and the input-output bus respectively, and the non-volatile storage medium is connected to the input-output bus through the memory bus. The USB adapter, the graphics adapter, and the network adapter are connected to the input-output bus. The USB adapter is connected to the keyboard and mouse, the graphics adapter is connected to the display, and the network adapter is connected to the network card.

[0104] The database of the computer device is used to store the data used in the above-mentioned large-model-based intelligent teaching method, including training data, reasoning data, etc. The network card of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an integrated knowledge management method is implemented.

[0105] Accordingly, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the steps of the above-mentioned integrated knowledge management method are implemented.

[0106] The computer-readable storage medium may be an internal storage unit of the system described in any of the foregoing embodiments, such as a hard disk or memory of the system. The computer-readable storage medium may also be an external storage device of the system, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the system. Furthermore, the computer-readable storage medium may also include both an internal storage unit of the system and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the system. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0107] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. An integrated knowledge management method, characterized in that: include: Build vector database; The vector database stores dense vectors and sparse vectors of slice contents; Obtaining a user's search requirement, vectorizing the search requirement, and obtaining a dense vector and a sparse vector of the search requirement; Calculate similarity based on the dense vector and sparse vector of the shard content and the search requirement to obtain a first search result and a second search result, and fuse the first search result and the second search result in a reciprocal sorting fusion manner to obtain a similar shard list; Use the large model-based reranker LLMReranker to rerank the fused sorting results to obtain N similar document fragments, filter out document fragments with similarity scores less than the threshold, and obtain the final fragment set; The document fragments in the final fragment set are structured using a large language model, Markdown documents are extracted and output, and the contents of the Markdown documents are visualized as a mind map.

2. The integrated knowledge management method according to claim 1, characterized in that: The specific steps of building a vector database are as follows: Obtaining manually uploaded data files to be managed, and performing content parsing, extraction, cleaning and format processing on the data files to be managed according to different data file types to obtain parsed files; Segment the parsed file using a sentence-based segmentation algorithm or a hierarchy-based segmentation algorithm to obtain segment contents; Performing dense vectorization and sparse vectorization on the slice content to obtain a dense vector and a sparse vector of the slice content; A vector database is used to store dense vectors and sparse vectors of the slice content.

3. The integrated knowledge management method according to claim 2, characterized in that: The parsed file is as follows: If the uploaded file is a PDF, the OCR module is used to identify the PDF content, and the PDF content is annotated using the PDF content annotation model. The header, footer, and directory are deleted, the table is converted into an image, and the image is saved as a link. Finally, the text, first and second level titles, and image links are converted into content that conforms to the Markdown format. If the uploaded file is in Excel, CSV or JSON format, the content will be parsed according to fixed rules.

4. The integrated knowledge management method according to claim 2, characterized in that: Call the API or use the locally deployed Embedding model to densely vectorize the shard content and retrieval requirements; use a word frequency-based algorithm to sparsely vectorize the shard content and retrieval requirements.

5. The integrated knowledge management method according to claim 1, characterized in that: Similarity calculation is performed based on the dense vector and sparse vector of the shard content and the search requirement to obtain a first search result and a second search result, and the first search result and the second search result are merged in a reciprocal sorting fusion manner, specifically: The dense retriever of the hybrid retriever calculates the similarity between the dense vector of the retrieval requirement and the dense vector of the shard content, and returns the top N most similar document shard DNs; The sparse searcher of the hybrid searcher calculates the similarity between the sparse vector of the search requirement and the sparse vector of the shard content, and returns the top N most similar document shard SNs; The document fragment DN and the document fragment SN are merged using the RRF algorithm.

6. The integrated knowledge management method according to claim 4, characterized in that: The RRF algorithm is used to merge the document fragment DN and the document fragment SN. Specifically: The RRF algorithm converts the similarity scores in the document shard DN and the document shard SN into reciprocals, and then adds up all the reciprocals to obtain the fused ranking score.

7. An integrated knowledge management device, characterized in that: include: Construction module, used to construct vector database; The vector database stores dense vectors and sparse vectors of slice contents; A demand acquisition module is used to acquire the user's search demand, vectorize the search demand, and obtain a dense vector and a sparse vector of the search demand; A calculation fusion module is used to perform similarity calculation based on the dense vector and sparse vector of the shard content and the search requirement to obtain a first search result and a second search result, and fuse the first search result and the second search result in a reciprocal sorting fusion manner to obtain a similar shard list; The re-ranking module is used to re-rank the fused sorting results using the large model-based re-ranker LLMReranker to obtain N similar document fragments, filter out document fragments with similarity scores less than a threshold, and obtain the final fragment set; A visualization module is used to structure the document fragments in the final fragment set using a large language model, extract and output Markdown documents, and visualize the contents of the Markdown documents in a mind map.

8. A computer device comprising a non-volatile memory, a processor, a memory bus, a system bus, and an input / output bus, as well as a USB adapter, a graphics adapter, etc. connected to the input / output bus, and a computer program stored in the non-volatile memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 6.

Images