Information retrieval method and device and storage medium

Abstract

The invention provides an information retrieval method and device and a storage medium, and the method comprises the steps: carrying out static denseness, context denseness and sparse coding on a retrieval text, matching an associated provision set according to the static denseness, context denseness and sparse coding, then extracting a triple of associated provisions through a large language model, and determining a retrieval result by combining the to-be-retrieved text, the triple set and the associated text. Text features are comprehensively captured through multiple codes, retrieval comprehensiveness and accuracy are improved, associated texts can be simplified by extracting triads, results have basis, logic coherence and traceability are achieved, the problems that traditional retrieval is low in efficiency and fuzzy in result are solved, rail transit operation and maintenance requirements are met, personnel cost is reduced, and field response speed is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, in particular to an information retrieval method, device and storage medium. BACKGROUND

[0002] In the field of rail transit operation and maintenance, there are many professionals involved, such as energy management, driving, passenger flow, security, etc. These professions are usually managed by multiple programs, increasing the complexity of operation and maintenance. In traditional rail transit operation and maintenance, professional personnel train operation and maintenance personnel and output training documents for future reference. However, due to the large and complex rail transit software and hardware systems, the amount of document content is large, and the operation and maintenance personnel cannot quickly and accurately retrieve related content. Moreover, operation and maintenance itself is highly dependent on experience, and the content of operation and maintenance documents is highly professional, making it difficult for personnel to hand over work and affecting the response speed of on-site operation and maintenance.

[0003] The existing rail transit field lacks an effective method for quickly retrieving operation and maintenance documents. Mainly, operation and maintenance personnel retrieve keywords by reading operation and maintenance documents. However, this method is limited by the description of the operation and maintenance documents, and if the document description is not accurate, it will affect the efficiency of retrieval. SUMMARY

[0004] The present application aims to address the deficiencies in the prior art by providing an information retrieval method, device and storage medium to solve the problem of low retrieval efficiency of operation and maintenance documents in the prior art.

[0005] To achieve the above-mentioned purpose, the technical solution adopted by the present application is as follows: In a first aspect, the present application provides an information retrieval method, comprising: encoding the text to be retrieved to obtain the static dense encoding, context dense encoding and sparse encoding of the text to be retrieved; determining a set of associated provisions matching the text to be retrieved based on the static dense encoding, context dense encoding and sparse encoding of the text to be retrieved, the set of associated provisions including at least one associated provision; extracting triples from each associated provision in the set of associated provisions based on a large language model to obtain a set of triples for each associated provision; determining the retrieval result of the text to be retrieved based on the large language model, the set of triples for each associated provision and the text to be retrieved.

[0006] Optionally, the encoding of the text to be retrieved to obtain the static dense encoding, context dense encoding and sparse encoding of the text to be retrieved comprises: inputting the to-be-retrieved text into a static word vector model for encoding processing to obtain the static dense code, the static dense code being used to represent semantic information of the to-be-retrieved text; inputting the to-be-retrieved text into a context dense vector model for encoding processing to obtain the context dense code, the context dense code being used to represent semantic information of the to-be-retrieved text and context of the to-be-retrieved text; performing feature extraction on the to-be-retrieved text feature extractor to obtain the sparse code, the sparse code being used to represent occurrence frequency of keywords in the to-be-retrieved text.

[0007] Optionally, determining the associated article set matched with the to-be-retrieved text according to the static dense code, the context dense code and the sparse code of the to-be-retrieved text comprises: matching the static dense code in a static dense vector library to obtain a first vector set corresponding to the to-be-retrieved text, the first vector set including a plurality of index vectors matched with the static dense code; matching the context dense code in a context dense vector library to obtain a second vector set corresponding to the to-be-retrieved text, the second vector set including a plurality of index vectors matched with the context dense code; matching the sparse code in a sparse vector library to obtain a third vector set corresponding to the to-be-retrieved text, the third vector set including a plurality of index vectors matched with the sparse code; determining the associated article set matched with the to-be-retrieved text according to the first vector set, the second vector set and the third vector set.

[0008] Optionally, determining the associated article set matched with the to-be-retrieved text according to the first vector set, the second vector set and the third vector set comprises: merging each index vector in the first vector set, the second vector set and the third vector set with the to-be-retrieved text to obtain a plurality of merged vectors; performing similarity calculation on each merged vector, and determining an index vector matched with the to-be-retrieved text from each merged vector according to a similarity result of each merged vector; adding a text corresponding to the index vector to the associated article set.

[0009] Optionally, performing triple extraction on each associated article in the associated article set based on a large language model to obtain a triple set of each associated article comprises: Each associated clause in the associated clause set is sequentially input into the large language model. The large language model extracts the subject information, action information, and object information from the associated clauses, and combines the subject information, action information, and object information into a triple of the associated clause and adds it to the triple set.

[0010] Optionally, determining the retrieval results for the text to be retrieved based on the large language model, the set of triples for each related text, and the text to be retrieved includes: Based on the set of triples of each related clause, construct the evidence chain graph corresponding to the text to be retrieved; Traverse the evidence chain graph to obtain the evidence chain information; The evidence chain information and the text to be retrieved are input into the large language model to obtain the retrieval results for the text to be retrieved.

[0011] Optionally, constructing the evidence chain graph corresponding to the text to be retrieved based on the set of triples of each associated clause includes: The text to be retrieved is used as the root node of the evidence chain graph; Each of the aforementioned related clauses is designated as the first child node of the root node; Based on the set of triples for each associated clause, the first child node is used as the parent node to generate the second child node corresponding to each associated clause. Based on the triples in each set of triples, the second child node is used as the parent node to generate the third child node corresponding to each triple.

[0012] Optionally, the method further includes: Obtain a document set, which includes multiple initial documents; Each of the initial documents is divided into blocks to obtain multiple text blocks corresponding to each initial document; Encode multiple text blocks corresponding to each initial document to obtain static dense encoding, context dense encoding, and sparse encoding for each text block; The static dense encoding of each text block is stored in the static dense vector library, the context dense encoding of each text block is stored in the context dense vector library, and the sparse encoding of each text block is stored in the sparse vector library.

[0013] Secondly, embodiments of this application also provide an electronic device, including: a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of an information retrieval method as described in any one of the first aspects.

[0014] In a third aspect, the embodiments of the present application further provide a computer readable storage medium, which stores a computer program. When the computer program is run by a processor, the steps of the information retrieval method according to any one of the first aspect are executed.

[0015] The beneficial effects of the present application are as follows: through the multi-encoding mode of static dense, context dense and sparse encoding, the semantic core, context association and keyword features of the text to be retrieved can be comprehensively captured, avoiding the limitations of single encoding, laying a foundation for accurate matching. By retrieving associated articles based on the three kinds of encoding, the retrieval coverage is expanded, and the relevance of the associated articles is ensured through subsequent screening, greatly improving the retrieval efficiency and accuracy of fragmented documents, and reducing the time cost of operation and maintenance personnel. By extracting triples of associated articles with the help of a large language model, core factual relationships can be refined, and scattered article structures can be structured. By combining the triple set and the text to be retrieved to generate a retrieval result, the output answer has clear basis, logical coherence and traceability, solving the problems of low efficiency and fuzzy results of traditional retrieval, and helping users to understand professional content and improve the response speed of on-site operation and maintenance, adapting to the professional needs of rail transit operation and maintenance.

[0016] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the following preferred embodiments are described in detail below, and the accompanying drawings are described as follows. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0018] Figure 1 A flowchart of an information retrieval method provided by an embodiment of the present application is shown; Figure 2 A flowchart of an encoding process provided by an embodiment of the present application is shown; Figure 3 A flowchart of determining an associated article set provided by an embodiment of the present application is shown; Figure 4 A flowchart of generating an associated article set provided by an embodiment of the present application is shown; Figure 5 A flowchart of generating a retrieval result provided by an embodiment of the present application is shown; Figure 6An example diagram of an evidence chain graph provided by an embodiment of the present application is shown. Figure 7 A flowchart of generating an evidence chain graph provided by an embodiment of the present application is shown. Figure 8 A flowchart of generating a database provided by an embodiment of the present application is shown. Figure 9 A structural schematic diagram of an information retrieval device provided by an embodiment of the present application is shown. Figure 10 A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will combine the accompanying drawings for the embodiments of the present application to make a clear and complete description of the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. The components of the embodiments of the present application described and shown in the accompanying drawings can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0020] It should be noted that the term “comprises” will be used in the embodiments of the present application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0021] The rail transit operation and maintenance field has the characteristics of wide professional span and large and complex software and hardware systems. In the traditional operation and maintenance mode, relevant professionals output training documents for subsequent operation and maintenance query and work reference, but due to the large content and strong professionalism of the operation and maintenance documents, and the high dependence of operation and maintenance work on personnel experience, specific work scenarios and technical problems have been formed in this field.

[0022] Firstly, operation and maintenance related documents are scattered in different professions and different program management systems, lack unified and efficient management methods, and have the characteristics of fragmented storage. The existing retrieval methods mainly rely on manual reading or keyword retrieval, which cannot adapt to fragmented and large data volume operation and maintenance documents, so it is difficult for operation and maintenance personnel to accurately locate the required content, which further affects the on-site operation and maintenance response speed.

[0023] Secondly, the operation and maintenance work requires the operation and maintenance personnel to have rich experience and strong professionalism. However, the current keyword retrieval mode is easily affected by the accuracy of document description and the matching degree of keywords. If the document description is not clear or the operation and maintenance personnel do not understand the relevant professional knowledge of the operation and maintenance document, the situation of missing and wrong keywords may occur, thereby causing low retrieval efficiency.

[0024] Based on this, the present application provides an information retrieval method. First, the uploaded operation and maintenance document is divided into blocks, and the text blocks of the operation and maintenance document are multi-encoded and stored in a vector database. When a user queries, the to-be-retrieved text is encoded and retrieved in multiple databases, and then strong correlation clauses are filtered through reordering. Then, the strong correlation clauses are extracted into clause triplets, combined with queries, clauses and sources to build an evidence chain, input into a large language model to generate answers and visualize the evidence chain. By using multi-encoding cross-database retrieval and reordering, the retrieval efficiency and accuracy of fragmented documents are improved, the time cost of operation and maintenance personnel is reduced, and through triplet extraction and evidence chain construction, the answer logic is traceable and verifiable, solving the problems of low efficiency, high cost and insufficient intelligence of traditional keyword retrieval.

[0025] It should be understood that the present application is only illustrated by taking the rail transit operation and maintenance field as an example, and should not be limited thereto. The information retrieval method of the present application can also be applied to other fields to realize information retrieval processing, such as querying clinical diagnosis and treatment specifications in the medical and health field, and retrieving legal regulations and file information in the legal and government affairs field.

[0026] Figure 1 is a flowchart of an information retrieval method provided by an embodiment of the present application. The execution subject of the method can be an electronic device or a server, as shown in Figure 1 The method comprises the following steps. S101, encoding processing is performed on the to-be-retrieved text to obtain static dense encoding, context dense encoding and sparse encoding of the to-be-retrieved text.

[0027] The to-be-retrieved text can be a question input by an operation and maintenance personnel or a text that needs to be retrieved. For example, the to-be-retrieved text input by the operation and maintenance personnel can be “how to monitor the server status of the rail transit system”, and the final output retrieval result is an answer to the to-be-retrieved text.

[0028] Optionally, the to-be-retrieved text can be encoded by different encoding models or encoding methods to obtain static dense encoding, context dense encoding and sparse encoding of the to-be-retrieved text, respectively.

[0029] The static dense code is a vector representation obtained by encoding the text to be searched by a static word vector model, can represent the core semantic information of the text to be searched, does not depend on the context, has a fixed vector dimension, and all elements are non-zero.

[0030] The context dense code is a vector generated by a context dense vector model, which can capture the semantic association of the text itself and the context before and after.

[0031] The sparse code is an encoding obtained by extracting text features by a feature extractor, which is used to represent the frequency feature of keywords in the text to be searched and reflect the importance of the keywords in the text to be searched.

[0032] By using multiple encoding methods to encode the text to be searched, the information of the text to be searched can be captured from three aspects of semantics, context association and keyword importance, the deep semantic understanding of the text to be searched is realized, and the retrieval accuracy of the text to be searched is improved.

[0033] S102, according to the static dense code, the context dense code and the sparse code of the text to be searched, determining a set of associated clauses matched with the text to be searched.

[0034] It should be noted that the operation and maintenance documents can be pre-processed by blocking and encoding. The text of the operation and maintenance document is processed by the same encoding method as the above S101 step to generate a database corresponding to the operation and maintenance document. The database includes a static dense vector library, a context dense vector library and a sparse vector library. After receiving the text to be searched input by the user and performing encoding processing, the static dense code, the context dense code and the sparse code of the text to be searched are matched in the static dense vector library, the context dense vector library and the sparse vector library respectively. The associated clauses matched with the text to be searched are added to the associated clause set, and finally the associated clause set is obtained.

[0035] The associated clause set includes at least one associated clause. The associated clause can be a text segment retrieved from the database of the operation and maintenance document, which is semantically related to the text to be searched and can provide support for answering the question to be searched. For example, the text to be searched is "how to monitor the server status of the rail transit system", and the associated clauses include "associated clause 1: the CPU and memory usage of the server should be collected every minute; associated clause 2: send an alarm message by email when the monitoring system detects abnormal load".

[0036] In a possible implementation, the text segments with high semantic relevance can be screened out by calculating the similarity of the text encoding to be retrieved and the text block encoding stored in the database, and these segments collectively constitute the associated provisions set.

[0037] By expanding the search range through multi-database search, potential relevant information can be ensured not to be missed, and effective content can be preliminarily focused through similarity screening, solving the problems of missed detection and false detection in traditional keyword search, and improving the comprehensiveness and preliminary accuracy of the search.

[0038] S103, based on the large language model, performing triple extraction on each associated provision in the associated provisions set to obtain a triple set of each associated provision.

[0039] The triple can be a structured data form composed of subject information, action information and object information, which is used to clearly present the core fact relationship in the text and form logical association of scattered text information.

[0040] Each associated provision can include at least one triple, and the triple set is a set of all triples corresponding to the associated provisions.

[0041] Optionally, the large language model can be an artificial intelligence model with natural language processing capability, which can complete text understanding, information extraction and content generation and other tasks.

[0042] Optionally, each provision in the associated provisions set can be input into the large language model in turn, and the model extracts the subject, action and object core information in the provision through deep understanding of the provision, and combines the three types of information into triples. Each triple clearly presents a complete fact relationship, and multiple triples collectively constitute the triple set of the provision.

[0043] For example, assuming that the associated provision is "the server CPU and memory usage rate should be collected every minute", after inputting it into the large language model, the output triple set can be represented as { "server-monitoring indicator-CPU usage rate", "server-monitoring indicator-memory usage rate"}, which includes two triples.

[0044] S104, based on the large language model, the triple set of each associated provision and the text to be retrieved, determining the search result of the text to be retrieved.

[0045] Optionally, an input sentence of the large language model can be generated based on the triple set and the text to be retrieved, and the input sentence is input into the large language model, and the large language model understands and generates the search result of the text to be retrieved.

[0046] The search results for the text to be searched can be answers to questions related to the text. While generating search results, a visual graph can also be generated as a chain of evidence based on related texts, sets of triples of related texts, and the text to be searched. This transforms complex operational regulations into simple and intuitive icons, allowing even operations and maintenance personnel unfamiliar with the documentation or lacking specialized knowledge to quickly understand the core of the regulations through the chain of evidence, thus achieving visualized integration and utilization of operational regulations.

[0047] It should be noted that in existing technologies, when using large language models to help understand operation and maintenance documents, the operation and maintenance documents and the text to be retrieved are generally directly input into the large language model, and the retrieval results are output. However, in this approach, the large language model cannot deeply understand the deep semantics of the operation and maintenance documents. By first extracting triples from related clauses, the large language model can deepen its understanding of related clauses. After extracting triples, the large language model can then generate retrieval results based on the triples, related clauses, and the text to be retrieved, thereby improving the accuracy of the retrieval results generated by the large language model.

[0048] In this embodiment, a multi-encoding approach—static dense, context-dense, and sparse encoding—comprehensively captures the semantic core, contextual relationships, and keyword features of the text to be retrieved, avoiding the limitations of a single encoding method and laying the foundation for accurate matching. By retrieving related clauses from multiple databases based on these three encoding methods, the search coverage is expanded. Subsequent filtering ensures the relevance of related clauses, significantly improving the retrieval efficiency and accuracy of fragmented documents and reducing the time cost for maintenance personnel. By extracting triples from related clauses using a large language model, core factual relationships can be extracted, structuring scattered clauses. By combining the triple set with the text to be retrieved to generate search results, the output answer is based on clear evidence, logically coherent, and traceable, solving the problems of low efficiency and ambiguous results in traditional retrieval. Simultaneously, it helps users understand professional content, improves on-site maintenance response speed, and adapts to the professional needs of rail transit maintenance.

[0049] The following is a further explanation of the encoding processing of the text to be retrieved, resulting in static dense encoding, context-dense encoding, and sparse encoding. Figure 2 As shown, the above step S101 includes: S201. Input the text to be retrieved into the static word vector model for encoding processing to obtain static dense encoding. Static dense encoding is used to represent the semantic information of the text to be retrieved.

[0050] Among them, static word vector models can map the text to be retrieved to a continuous vector space through specific algorithms, capturing the basic semantic relationships in the text to be retrieved. Static word vector models can be word-to-vector models (Word2Vec Model).

[0051] The text to be retrieved is input into a static word vector model. The model analyzes the semantic associations of words in the text and transforms the text into a static dense code with fixed dimensions.

[0052] S202. Input the text to be retrieved into the context-dense vector model for encoding processing to obtain the context-dense encoding. The context-dense encoding is used to represent the semantic information of the text to be retrieved and the context of the text to be retrieved.

[0053] Context-dense vector models can be bidirectional language representation models based on a transformer architecture. They can simultaneously combine the text itself with its surrounding context for semantic encoding, generating context-dense codes that accurately reflect the meaning of the retrieved text within its specific context. Context-dense vector models can be bidirectional encoder representation models (BERT models).

[0054] Input the text to be retrieved into the context-dense vector model. The model will analyze the text itself and the context information it is related to, and combine the two to generate a context-dense code.

[0055] S203. Input the text to be retrieved into the feature extractor for feature extraction to obtain sparse coding. Sparse coding is used to characterize the frequency of occurrence of keywords in the text to be retrieved.

[0056] Feature extractors are tools for extracting key features from text, focusing on the frequency of keywords in the text to be retrieved and their importance in the overall document set, thereby filtering out information that plays a key role in the meaning of the text as sparse encoding. Feature extractors can be Term Frequency-Inverse Document Frequency Weight Model (TF-IDF Model).

[0057] The text to be retrieved is input into the feature extractor, which focuses on counting the frequency of keywords in the text and, combined with the scarcity of keywords in the overall document set, generates sparse codes.

[0058] For example, suppose the text to be retrieved is represented as Treating the search text Three encoding methods were performed respectively to obtain... ,in, , These represent the corresponding static dense coding, context dense coding, and sparse coding, respectively.

[0059] In this embodiment of the application, the text to be retrieved is encoded in multiple ways to obtain encoding vectors of different dimensions. Among them, static dense encoding can ensure semantic benchmark, context dense encoding can take into account contextual accuracy, and sparse encoding highlights core keywords. The combination of the three provides comprehensive and three-dimensional text feature support for subsequent multi-database retrieval, thereby improving the comprehensiveness and accuracy of retrieval from the source.

[0060] The following explains the steps for determining the set of related clauses matching the text to be retrieved based on the static dense coding, contextual dense coding, and sparse coding of the text to be retrieved. Figure 3 As shown, the above step S102 includes: S301. Match the static dense vector library according to the static dense encoding to obtain the first vector set corresponding to the text to be retrieved.

[0061] The first vector set includes multiple index vectors that match the static dense encoding. These index vectors can be static dense vectors from a static dense vector library that match the static dense encoding of the text to be retrieved.

[0062] Optionally, by comparing the semantic similarity of the static dense encoding of the text to be retrieved with all the index vectors stored in the static dense vector library, and by calculating the similarity, multiple index vectors that are closest to the core semantics of the text to be retrieved are selected based on the results of the similarity calculation. These vectors together constitute the first vector set.

[0063] S302. Match the text in the context-dense vector library according to the context-dense encoding to obtain the second vector set corresponding to the text to be retrieved.

[0064] The second vector set includes multiple index vectors that match the context-dense encoding. These index vectors can be context-dense vectors from a context-dense vector library that match the context-dense encoding of the text to be retrieved.

[0065] Optionally, by densely encoding the context of the text to be retrieved and comparing its similarity with the index vectors in the dense context vector library, multiple index vectors that fit the context of the text to be retrieved are selected based on the similarity calculation results to form a second vector set.

[0066] S303. Match the sparse vector in the sparse vector library according to the sparse coding to obtain the third vector set corresponding to the text to be retrieved.

[0067] The third vector set includes multiple index vectors that match the sparse coding. These index vectors can be sparse vectors from a sparse vector library that match the sparse coding of the text to be retrieved.

[0068] Optionally, by matching the sparse encoding of the text to be retrieved with the index vector in the sparse vector library and calculating the similarity, multiple vectors with high relevance are selected based on the similarity calculation results to form a third vector set.

[0069] Continuing with the example above, let's look at statically dense coding. to The vectors in the set are compared by cosine similarity, and the top K index vectors with the highest similarity are selected to obtain the first vector set. Similarly, for sparse coding and keyword coding The cosine similarity of each vector with the vectors in sets S and K is calculated, and the top K index vectors with the highest scores are selected to obtain the second vector set. and the third vector set And put these 3K index vectors into a set. middle.

[0070] S304. Based on the first vector set, the second vector set, and the third vector set, determine the set of related clauses that match the text to be retrieved.

[0071] For each index vector in the first, second, and third vector sets, further filtering can be performed using a large language model. The index vectors are merged with the text to be retrieved and input into the large language model. The large language model outputs the similarity between the text to be retrieved and the index vectors, and then filters out the set of related texts that match the text to be retrieved from the index vectors based on the similarity.

[0072] It should be noted that when performing step S304 above, the method of similarity calculation can be different from the method of obtaining the first vector set, the second vector set, and the third vector set through the above filtering. For example, the first vector set, the second vector set, and the third vector set can be obtained by calculating the cosine similarity. Then, other similarity calculation methods can be used to filter out the set of related texts that match the text to be retrieved from the index vectors of the first vector set, the second vector set, and the third vector set. For example, the similarity can be determined by calculating the Pearson correlation coefficient or Euclidean distance, and the set of related texts can be obtained based on the similarity.

[0073] The following is a further explanation of how the set of related entries matching the text to be retrieved is determined based on the first vector set, the second vector set, and the third vector set. Figure 4 As shown, step S304 above includes: S401. Merge each index vector in the first vector set, the second vector set, and the third vector set with the text to be retrieved to obtain multiple merged vectors.

[0074] The merged vector refers to either the vector obtained by fusing the text to be retrieved with the text corresponding to the index vector, or the vector obtained by fusing the text to be retrieved with the index vector. The merged vector contains the semantic features of the query requirements and the semantic features of the entries corresponding to the index vector.

[0075] S402. Calculate the similarity of each merged vector and determine the index vector that matches the text to be retrieved from each merged vector based on the similarity results of each merged vector.

[0076] Optionally, the merged vectors can be input into a similarity calculation model, which calculates and outputs the similarity between the text to be retrieved and the index vector in the merged vectors. The merged vectors are then reordered based on their similarity. The top K merged vectors are selected according to the similarity ranking, and the index vector in the merged vectors is determined as the index vector that matches the text to be retrieved.

[0077] Continuing the example above, the similarity calculation model can be a cross-encoder model. Iterate through the index vector 'a' in set A, merge it with the query text 'q', input it into the cross-encoder model, output the similarity between 'a' and 'q', reorder the 3K index vectors based on the similarity, and return the top K index vectors as the query text. .

[0078] S403. Add the text corresponding to the index vector to the associated text set.

[0079] Optionally, the high-similarity index vectors determined by the similarity calculation model can be matched with their corresponding original text fragments, and these text fragments can be integrated to form the final set of related entries. Each entry in this set has undergone multiple rounds of screening and has a high semantic relevance to the text to be retrieved.

[0080] In this embodiment of the application, by further filtering the selected index vectors, the accuracy of similarity judgment can be improved, thereby obtaining relevant and semantically consistent entries that are related to the text to be retrieved and adding them to the set of related entries, ensuring that the selected related entries can better meet the user's query needs.

[0081] The following is a further explanation of the above-mentioned extraction of triples from the set of related clauses based on the large language model, resulting in the set of triples for each related clause. Step S103 above includes: Each related clause in the related clause set is sequentially input into the large language model. The large language model extracts the subject information, action information, and object information from the related clauses, and combines the subject information, action information, and object information into a triple of the related clause and adds it to the triple set.

[0082] In this context, the subject information can be the initiator of the action or the core description in the related clauses. The action information can be the link connecting the subject and the object, used to describe the subject's behavior or the relationship between the two. The object information can be the target or result of the action, the object to which the subject's behavior is applied.

[0083] For example, suppose the associated clause is "send an alarm via email when the monitoring system detects abnormal load". The extracted subject information is the monitoring system, the action information is the alarm method, and the object information is the email.

[0084] Optionally, each entry in the filtered set of related entries can be input into the large language model one by one. The large language model will perform deep semantic analysis on each entry, combine it with the professional context of the operation and maintenance document, accurately identify the core elements in the entry, extract the triples in the related entries, and combine the extracted triples according to fixed logic to form the triples of the related entries.

[0085] If a related clause contains multiple objects or actions, the related clause can be split into multiple triples to generate a set of triples for the related clause.

[0086] In this embodiment of the application, by transforming unstructured related clauses into a set of structured triples, complex professional expressions can become logically clear, thereby helping operation and maintenance personnel to quickly understand the core logic of related clauses, reducing the threshold for understanding professional knowledge, and providing a structured basis for large language models to generate organized answers.

[0087] The following is a further explanation of how the retrieval results are determined based on the large language model, the set of triples for each related clause, and the text to be retrieved. Figure 5 As shown, step S104 above includes: S501. Based on the set of triples of each related clause, construct the evidence chain graph corresponding to the text to be retrieved.

[0088] Among them, the evidence chain graph is a logical association system presented in the form of a graph. It consists of information such as the text to be retrieved, related clauses, and triples as nodes, and connects each node through edges. It is used to represent the hierarchical association between the question, the text, and the core information.

[0089] Reference Figure 6This is an example of an evidence chain graph, in which the text to be retrieved is the first node of the graph, and the first layer extending outward from the node of the text to be retrieved is the node of the related clauses, and each related clause extends outward to multiple triple nodes.

[0090] S502. Traverse the evidence chain graph to obtain evidence chain information.

[0091] Among them, the evidence chain information can be structured information obtained by traversing the evidence chain graph, including the text to be retrieved, the content of related clauses, the source of the clauses, the triple facts and the logical relationships between each element. The evidence chain graph can be deeply traversed to transform the evidence chain graph into evidence chain information in a string input format that can be accepted by the large language model.

[0092] S503. Input the evidence chain information and the text to be retrieved into the large language model to obtain the retrieval results of the text to be retrieved.

[0093] Optionally, the evidence chain information can be combined with the text to be retrieved into an input string, with the evidence chain information serving as the context and the text to be retrieved serving as the question, and the large language model can output the retrieval results of the text to be retrieved.

[0094] In this embodiment of the application, by establishing an evidence chain graph of the text to be retrieved, related clauses, and triples, the large language model can gain a deeper understanding of the retrieved information, thereby improving the accuracy of the large language model's answers.

[0095] The following is a further explanation of how the evidence chain graph corresponding to the text to be retrieved is constructed based on the set of triples of each related clause. Figure 7 As shown, the above S501 step includes: S701. Use the text to be retrieved as the root node of the evidence chain graph.

[0096] S702. Treat each related clause as the first child node of the root node.

[0097] S703. Based on the set of triples for each associated clause, take the first child node as the parent node and generate the second child node corresponding to each associated clause.

[0098] S704. Based on the triples in each triplet set, take the second child node as the parent node and generate the third child node corresponding to each triplet.

[0099] The first child node represents the associated clause, the second child node represents each triple in the triple set, and the third child node is used to represent the specific information in the triple, including the subject, object, and action.

[0100] Reference Figure 6 The evidence chain diagram shown is used in the construction of...Figure 6 When constructing the evidence chain graph shown, the text to be retrieved, "How to monitor the server status of the rail transit system" q, can be used as the first node of the graph. Two edges are extended from it, and the node of each edge is the first child node, which is the retrieved related text. Then, at least one edge is extended from each related text node, and the node of each edge is the second child node, representing a triple. For each triple, a third child node is generated for the subject, action, and object, thereby completing the construction of the evidence chain graph G.

[0101] Before performing text retrieval, the database must first be established based on existing operation and maintenance documentation, such as... Figure 8 As shown, the method of this application further includes: S801. Obtain a document collection, which includes multiple initial documents.

[0102] The document collection can be a collection of multiple operation and maintenance documents uploaded by users. The initial document is an operation and maintenance document, which contains operation and maintenance information under a specific profession or scenario.

[0103] S802. Divide each initial document into blocks to obtain multiple text blocks corresponding to each initial document.

[0104] A single initial document can be split into several independent text segments according to semantic logic or length rules, resulting in multiple text blocks corresponding to the initial document, with complete local semantics preserved in each text block.

[0105] S803. Encode the multiple text blocks corresponding to each initial document to obtain the static dense code, context dense code, and sparse code of each text block.

[0106] The encoding process for text blocks can be the same as that described in S101 above, and the specific steps will not be elaborated here. Specifically, three different encoding methods are used for each text block. Static dense encoding is generated using a static word vector model to capture the basic core semantics of the text block. Contextual dense encoding is generated using a context vector model to integrate the semantic relationships between the text block and its context. Finally, sparse encoding is generated using a feature extractor to highlight the frequency features of keywords in the text block.

[0107] S804. Store the static dense encoding of each text block in the static dense vector library, store the context dense encoding of each text block in the context dense vector library, and store the sparse encoding of each text block in the sparse vector library.

[0108] The three types of encoding generated from the text blocks are stored in corresponding vector databases: static dense encoding in a static dense vector database, context-dense encoding in a context-dense vector database, and sparse encoding in a sparse vector database. Furthermore, each encoding maintains a one-to-one correspondence with the corresponding text block and metadata to ensure rapid retrieval of the original information.

[0109] For example, a user uploads a set of documents. ,in Let N represent the nth document, and N represent the total number of documents. Divide into blocks to obtain a block set: ,in , Indicates the first There are 3 blocks, M is... The total number of document chunks, and They represent The corresponding text and metadata (source file, page number).

[0110] For block Static dense coding, context-dense coding, and sparse coding are performed respectively. .

[0111] in, , These are the Word2Vec, BERT, and TF-IDF models, respectively. , and These are the corresponding static dense coding, context dense coding, and sparse coding, respectively.

[0112] Save the above three encodings to a static dense vector library respectively. Context-dense vector library Sparse vector library middle.

[0113] In this embodiment of the application, by dividing the initial document into blocks and encoding it, the coded data can be classified, managed and stored, thus providing a foundation for the parallel processing of multiple databases. Subsequently, different encoding types can be searched simultaneously in the corresponding databases, thereby improving retrieval efficiency.

[0114] Based on the same inventive concept, this application also provides an information retrieval device corresponding to the information retrieval method. Since the principle of the device in this application is similar to that of the information retrieval method described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0115] Figure 9A schematic diagram of an information retrieval device provided in an embodiment of this application is shown. The device includes an encoding module 901, a filtering module 902, an extraction module 903, and a determination module 904.

[0116] The encoding module 901 is used to encode the text to be retrieved, and obtain the static dense encoding, context dense encoding and sparse encoding of the text to be retrieved. The filtering module 902 is used to determine the set of related clauses that match the text to be retrieved based on the static dense coding, context dense coding and sparse coding of the text to be retrieved. The set of related clauses includes at least one related clause. The extraction module 903 is used to extract triples from each related clause in the related clause set based on the large language model, so as to obtain the triple set of each related clause. The determination module 904 is used to determine the retrieval results of the text to be retrieved based on the large language model, the set of triples of each related text, and the text to be retrieved.

[0117] Optionally, the encoding module 901 is specifically used for: The text to be retrieved is input into a static word vector model for encoding, resulting in static dense encoding. Static dense encoding is used to represent the semantic information of the text to be retrieved. The text to be retrieved is input into a context-dense vector model for encoding to obtain context-dense encoding. Context-dense encoding is used to represent the semantic information of the text to be retrieved and its context. The feature extractor of the text to be retrieved extracts features and obtains sparse coding, which is used to characterize the frequency of occurrence of keywords in the text to be retrieved.

[0118] Optionally, the filtering module 902 is specifically used for: Based on the static dense coding, a first vector set corresponding to the text to be retrieved is obtained by matching in the static dense vector library. The first vector set includes multiple index vectors that match the static dense coding. Based on the context-dense coding, a second vector set corresponding to the text to be retrieved is obtained by matching in the context-dense vector library. The second vector set includes multiple index vectors that match the context-dense coding. Based on the sparse coding, a third vector set corresponding to the text to be retrieved is obtained by matching in the sparse vector library. The third vector set includes multiple index vectors that match the sparse coding. Based on the first vector set, the second vector set, and the third vector set, determine the set of related clauses that match the text to be retrieved.

[0119] Optionally, the filtering module 902 is specifically used for: The index vectors in the first vector set, the second vector set, and the third vector set are merged with the text to be retrieved to obtain multiple merged vectors; The similarity of each merged vector is calculated, and based on the similarity results of each merged vector, the index vector that matches the text to be retrieved is determined from each merged vector. Add the text corresponding to the index vector to the associated text set.

[0120] Optionally, the extraction module 903 is specifically used for: Each related clause in the related clause set is sequentially input into the large language model. The large language model extracts the subject information, action information, and object information from the related clauses, and combines the subject information, action information, and object information into a triple of the related clause and adds it to the triple set.

[0121] Optionally, module 904 is specifically used for: Based on the set of triples of each related clause, construct the evidence chain graph corresponding to the text to be retrieved; Traverse the evidence chain graph to obtain evidence chain information; The evidence chain information and the text to be retrieved are input into the large language model to obtain the retrieval results for the text to be retrieved.

[0122] Optionally, module 904 is specifically used for: The text to be retrieved is used as the root node of the evidence chain graph; Each related clause is designated as the first child node of the root node; Based on the set of triples for each related clause, the first child node is used as the parent node to generate the second child node corresponding to each related clause. Based on the triples in each set of triples, the second child node is used as the parent node to generate the third child node corresponding to each triple.

[0123] Optionally, the device also includes a building module for: Retrieve a collection of documents, which includes multiple initial documents; Each initial document is divided into blocks to obtain multiple text blocks corresponding to each initial document; Encode multiple text blocks corresponding to each initial document to obtain static dense encoding, context dense encoding, and sparse encoding for each text block; The static dense encoding of each text block is stored in the static dense vector library, the context dense encoding of each text block is stored in the context dense vector library, and the sparse encoding of each text block is stored in the sparse vector library.

[0124] This application's embodiments utilize multiple encoding methods—static dense, context-dense, and sparse encoding—to comprehensively capture the semantic core, contextual relationships, and keyword features of the text to be retrieved, avoiding the limitations of single encoding and laying the foundation for accurate matching. By retrieving related entries from multiple databases based on these three encoding methods, the search coverage is expanded. Subsequent filtering ensures the relevance of related entries, significantly improving the retrieval efficiency and accuracy of fragmented documents and reducing the time cost for maintenance personnel. By extracting triples from related entries using a large language model, core factual relationships can be extracted, structuring scattered entries. By combining the triple set with the text to be retrieved to generate search results, the output answers are clearly based, logically coherent, and traceable, solving the problems of low efficiency and ambiguous results in traditional retrieval. Simultaneously, it helps users understand professional content, improves on-site maintenance response speed, and adapts to the professional needs of rail transit maintenance.

[0125] Figure 10 This illustration shows a schematic diagram of an electronic device provided in an embodiment of this application, including: a processor 1001, a storage medium 1002, and a bus 1003. The storage medium 1002 stores machine-readable instructions executable by the processor 1001. When the electronic device runs an information retrieval method as described in the embodiment, the processor 1001 communicates with the storage medium 1002 via the bus 1003. The processor 1001 executes the machine-readable instructions, and the preamble of the method item of the processor 1001 executes the steps in the above-described information retrieval method.

[0126] This application also provides a computer-readable storage medium storing a computer program, which is executed by a processor, and the processor performs the steps in the above-described information retrieval method.

[0127] In this embodiment, the computer program, when run by the processor, can also execute other machine-readable instructions to perform other methods as described in the embodiments. For details on the specific execution steps and principles, please refer to the description of the embodiments, which will not be repeated here.

[0128] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0129] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0130] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0131] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0132] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0133] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. An information retrieval method, characterized in that, include: The text to be retrieved is encoded to obtain the static dense code, the context dense code, and the sparse code of the text to be retrieved; Based on the static dense coding, context dense coding, and sparse coding of the text to be retrieved, a set of related clauses matching the text to be retrieved is determined, and the set of related clauses includes at least one related clause. Based on the large language model, triples are extracted from each associated clause in the associated clause set to obtain a set of triples for each associated clause. Based on the large language model, the set of triples of each associated text, and the text to be retrieved, the retrieval results of the text to be retrieved are determined.

2. The method according to claim 1, characterized in that, The encoding process for the text to be retrieved, resulting in static dense encoding, context-dense encoding, and sparse encoding, includes: The text to be retrieved is input into a static word vector model for encoding processing to obtain the static dense encoding, which is used to represent the semantic information of the text to be retrieved. The text to be retrieved is input into a context-dense vector model for encoding to obtain the context-dense encoding, which is used to characterize the semantic information of the text to be retrieved and the context of the text to be retrieved. The feature extractor of the text to be retrieved extracts features to obtain the sparse code, which is used to characterize the frequency of occurrence of keywords in the text to be retrieved.

3. The method according to claim 1, characterized in that, The step of determining the set of related clauses matching the text to be retrieved based on the static dense encoding, context dense encoding, and sparse encoding of the text to be retrieved includes: The first vector set corresponding to the text to be retrieved is obtained by matching the static dense encoding in the static dense vector library. The first vector set includes multiple index vectors that match the static dense encoding. The second vector set corresponding to the text to be retrieved is obtained by matching the context-dense encoding in the context-dense vector library. The second vector set includes multiple index vectors that match the context-dense encoding. The sparse coding is matched in the sparse vector library to obtain a third vector set corresponding to the text to be retrieved. The third vector set includes multiple index vectors that match the sparse coding. Based on the first vector set, the second vector set, and the third vector set, a set of related clauses that match the text to be retrieved is determined.

4. The method according to claim 3, characterized in that, The step of determining the set of related texts matching the text to be retrieved based on the first vector set, the second vector set, and the third vector set includes: The index vectors in the first vector set, the second vector set, and the third vector set are merged with the text to be retrieved to obtain multiple merged vectors; The similarity of each of the merged vectors is calculated, and based on the similarity results of each merged vector, the index vector that matches the text to be retrieved is determined from each of the merged vectors. Add the text corresponding to the index vector to the set of associated clauses.

5. The method according to claim 1, characterized in that, The method involves extracting triples from each associated clause in the associated clause set based on a large language model, resulting in a set of triples for each associated clause, including: Each associated clause in the associated clause set is sequentially input into the large language model. The large language model extracts the subject information, action information, and object information from the associated clauses, and combines the subject information, action information, and object information into a triple of the associated clause and adds it to the triple set.

6. The method according to claim 1, characterized in that, The process of determining the retrieval results for the text to be retrieved based on the large language model, the set of triples for each related clause, and the text to be retrieved includes: Based on the set of triples of each related clause, construct the evidence chain graph corresponding to the text to be retrieved; Traverse the evidence chain graph to obtain the evidence chain information; The evidence chain information and the text to be retrieved are input into the large language model to obtain the retrieval results for the text to be retrieved.

7. The method according to claim 6, characterized in that, The step of constructing the evidence chain graph corresponding to the text to be retrieved based on the set of triples of each associated clause includes: The text to be retrieved is used as the root node of the evidence chain graph; Each of the aforementioned related clauses is designated as the first child node of the root node; Based on the set of triples for each associated clause, the first child node is used as the parent node to generate the second child node corresponding to each associated clause. Based on the triples in each set of triples, the second child node is used as the parent node to generate the third child node corresponding to each triple.

8. The method according to claim 1, characterized in that, The method further includes: Obtain a document set, which includes multiple initial documents; Each of the initial documents is divided into blocks to obtain multiple text blocks corresponding to each initial document; Encode multiple text blocks corresponding to each initial document to obtain static dense encoding, context dense encoding, and sparse encoding for each text block; The static dense encoding of each text block is stored in the static dense vector library, the context dense encoding of each text block is stored in the context dense vector library, and the sparse encoding of each text block is stored in the sparse vector library.

9. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of an information retrieval method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of an information retrieval method as described in any one of claims 1 to 8.

Images