Knowledge base construction method, information retrieval method and computer equipment

Abstract

The invention discloses a knowledge base construction method, an information retrieval method and computer equipment, and the construction method comprises the steps: processing to-be-processed text information, and obtaining slice contents of a plurality of text slices; neighbor slice association indexes of the adjacent text slices are determined, a target knowledge base is constructed based on the neighbor slice association indexes and the slice content of the multiple text slices, and the target knowledge base comprises the neighbor slice association indexes and the slice content of the multiple text slices. Content inquired by utilizing the knowledge base is high in continuity, and user requirements and information retrieval requirements of professional scenes can be met.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method for constructing a knowledge base, an information retrieval method, and a computer device. Background Technology

[0002] In related technologies, document content can be segmented and vectorized to build a knowledge base for users to query documents. However, the content retrieved using such a knowledge base has poor logical structure and cannot meet the information retrieval requirements of user needs and professional scenarios. Summary of the Invention

[0003] The purpose of this application is to provide a method for constructing a knowledge base, an information retrieval method, and a computer device, so as to utilize the logically strong content retrieved from the knowledge base, thereby meeting the information retrieval requirements of user needs and professional scenarios.

[0004] In a first aspect, embodiments of this application provide a method for constructing a knowledge base, including: The text information to be processed is processed to obtain the slice content of multiple text slices; Determine the neighbor slice association index of adjacent text slices. The neighbor slice association index is used to indicate the degree of association between two adjacent text slices. A target knowledge base is constructed based on the neighbor slice association index and the slice content of multiple text slices. The target knowledge base includes the neighbor slice association index and the slice content of the multiple text slices.

[0005] Using the above technical solution, the text to be processed can be used to obtain the content of multiple text slices, and the neighbor slice association index of adjacent text slices can be determined. This neighbor slice association index indicates the degree of association between two adjacent text slices. Therefore, a target knowledge base can be constructed based on the neighbor slice association index and the content of multiple text slices. When an information retrieval request is received, the initial text slice is obtained from the knowledge base. Since the neighbor slice association index is used to indicate the degree of association between two adjacent text slices, it can be determined whether the neighbor slice association index of the initial text slice meets the concatenation condition of neighbor slices. If the neighbor slice association index of the initial text slice meets the concatenation condition, it indicates that the degree of association between the initial text slice and its neighbor slices is relatively high. Therefore, based on the order of the neighbor slices and the initial text slice, the content of the neighbor slices of the initial text slice can be concatenated with the content of the initial text slice. As can be seen, the method in this application embodiment can determine whether it is necessary to splice the slice content of the neighboring slices into the slice content of the initial text slice based on the neighboring slice association index of the initial text slice. This can ensure the logical coherence of the final information retrieval results, thereby meeting the information retrieval requirements of user needs and professional scenarios.

[0006] In one possible implementation, determining the neighbor slice association index of adjacent text slices includes: determining the neighbor slice association index of each adjacent text slice; correspondingly, the target knowledge base includes: the neighbor slice association index of each adjacent text slice, and the slice content of multiple text slices.

[0007] When the neighbor slice association indicators of each adjacent text slice in multiple text slices are stored in the target knowledge base, a global search of text slices can be performed in the target knowledge base during the information retrieval process. This allows for the search of the maximum range of associated slices in the target knowledge base, with the initial text slice as the root node, thus ensuring the completeness of the final information retrieval results and avoiding information omissions.

[0008] In one possible implementation, the neighbor slice association index is used to indicate the semantic similarity of the slice content of two adjacent text slices; correspondingly, determining the neighbor slice association index of adjacent text slices includes: Determine the content of each text slice in adjacent text slices; determine the semantic similarity of the slice content based on the content of each text slice in adjacent text slices; and obtain the neighbor slice association index of adjacent text slices based on the semantic similarity of the slice content.

[0009] Since the semantic similarity of slice content can reflect the degree of semantic association between the slice content of adjacent text slices, the neighbor slice association index of adjacent text slices can be obtained based on the semantic similarity of slice content.

[0010] In one possible implementation, the neighbor slice association metric includes the preceding neighbor slice association metric. In this case, among two adjacent text slices, the text content of one text slice is preceding the text content of the other text slice. The preceding neighbor slice association metric can then be used to determine whether the content of the initial text slice can be concatenated with the content of its preceding neighbor slice, thus ensuring a high degree of coherence of the preceding content starting from the initial text slice in the final information retrieval results.

[0011] In one possible implementation, the neighbor slice association index includes the following neighbor slice association index. In this case, among two adjacent text slices, the text content of one text slice is the following text of the text content of the other text slice. The following neighbor slice association index can then be used to determine whether the slice content of the initial text slice can be concatenated with the slice content of its following neighbor slice, thus ensuring the coherence between the initial text slice and its following text in the information retrieval results.

[0012] In one possible implementation, the target knowledge base also includes semantic vectors for each text slice. These semantic vectors are used to achieve semantic similarity matching with the information retrieval request; correspondingly, the method further includes: extracting a slice summary for each of the multiple text slices based on the slice content of each of the multiple text slices; and vectorizing each slice summary to obtain a semantic vector for each of the multiple text slices.

[0013] In the method of this application embodiment, redundant information of the corresponding text slice summary has been removed from each slice summary, allowing the vector representation to focus more on the core content of the text slice. Therefore, each slice summary can be vectorized to obtain the semantic vector of the corresponding slice text and stored in the target knowledge base. Based on this, when retrieving the initial text slice from the target knowledge base using a vectorized retrieval method, the influence of redundant information in the text slices stored in the target knowledge base can be reduced, making the retrieved initial text slice match the information retrieval request. Thus, recursively retrieving from the target knowledge base with the initial text slice as the root node can ensure the accuracy of the final information retrieval results.

[0014] In one possible implementation, the method further includes: determining a corresponding slice keyword for each of the multiple text slices; correspondingly, the target knowledge base also includes: a corresponding slice keyword for each of the multiple text slices.

[0015] When the target knowledge base includes multiple text slices, each with its corresponding slice keywords, multiple candidate text slices can be selected from the target knowledge base first through slice keyword matching, thus completing the initial filtering of text slices. Then, semantic vector matching is used to further filter the initial text slices from the multiple candidate text slices, thus completing the secondary filtering of text slices. This dual filtering method of slice keyword matching and semantic vector matching not only takes into account whether the text slice contains slice keywords that match the information retrieval request, but also whether the text slice contains core content that matches the information retrieval request. Therefore, the embodiments of this application can reduce the interference of redundant information on the retrieval of the initial text slices and improve the accuracy of information retrieval results through the dual filtering method of slice keyword matching and semantic vector matching.

[0016] In one possible implementation, the method of this application embodiment may further include: deduplicating identical slice keywords among the slice keywords of multiple text slices. This can reduce the storage pressure on the target knowledge base.

[0017] Secondly, embodiments of this application provide a knowledge base construction apparatus, comprising: The acquisition module is used to process the text information to be processed, obtain the slice content of multiple text slices, and determine the neighbor slice association index of adjacent text slices. The neighbor slice association index is used to indicate the degree of association between two adjacent text slices. The organization module is used to construct a target knowledge base based on neighbor slice association indicators and the slice content of multiple text slices. The target knowledge base includes neighbor slice association indicators and the slice content of the multiple text slices.

[0018] Thirdly, embodiments of this application also provide an electronic device, including: Processor; and, Memory for stored programs; The program includes instructions, which the processor executes to cause the computer device to perform the method according to the first aspect or any possible implementation thereof according to the embodiments of this application.

[0019] Fourthly, embodiments of this application also provide a computer storage medium storing computer instructions that, when executed on an electronic device, cause the processor of the electronic device to perform the method described according to the first aspect of the embodiments of this application or any possible implementation thereof.

[0020] Fifthly, embodiments of this application also provide a computer program product, including a computer program, wherein the computer program, when executed by a processor, implements the method described in the first aspect or any possible implementation thereof.

[0021] The beneficial effects of the technical solutions of the second to fifth aspects of the embodiments of this application can be referred to the beneficial effects of the methods described in the first aspect or any possible implementation of the first aspect of the embodiments of this application, which will not be repeated here.

[0022] Sixthly, embodiments of this application also provide an information retrieval method, including: When an information retrieval request is received, an initial text slice is obtained from the knowledge base. The knowledge base includes neighbor slice association indicators and slice content of multiple text slices. The initial text slice corresponds to a neighbor slice association indicator. Determine whether the neighbor slice association index of the initial text slice meets the concatenation condition of the neighbor slice. The neighbor slice is the text slice that is adjacent to the initial text slice among multiple text slices. If the association index of the neighboring slices of the initial text slice meets the splicing condition of the neighboring slices, the neighboring slices of the initial text slice are obtained from the knowledge base based on the initial text slice; Based on the order of the neighbor slices and the initial text slice, the slice content of the neighbor slices of the initial text slice is concatenated with the slice content of the initial text slice.

[0023] As can be seen, in the method of this application embodiment, an initial text slice can be obtained from the knowledge base, and the neighbor slice association index of the initial text slice is used as a basis to determine whether it is necessary to splice the slice content of the neighbor slice into the slice content of the initial text slice. This can ensure the logical coherence of the final information retrieval results, thereby meeting the information retrieval requirements of user needs and professional scenarios.

[0024] In one possible implementation, the concatenation condition for neighboring slices may include the neighboring slice association index of the initial text slice being greater than or equal to a preset index.

[0025] In one possible implementation, the knowledge base includes: neighbor slice association indicators for each adjacent text slice; correspondingly, after the step of concatenating the slice content of the neighbor slices with the slice content of the initial text slice based on the order of the neighbor slices and the initial text slice, the method further includes: The neighboring slices of the initial text slice are updated to the initial text slice. The step of determining whether the association index of the neighboring slices of the initial text slice meets the concatenation condition of the neighboring slices is returned until the association index of the neighboring slices of the initial text slice no longer meets the concatenation condition of the neighboring slices.

[0026] When the knowledge base includes neighbor slice association indicators for each adjacent text slice, if the neighbor slice association indicators of the initial text slice meet the concatenation conditions of neighbor slices, the slice content of the initial text slice can not only be concatenated with the slice content of neighbor slices, but the neighbor slices can also be updated to the initial text slice, and the step of determining whether the neighbor slice association indicators of the initial text slice meet the concatenation conditions of neighbor slices is returned. In this way, more text slices associated with the initial text slice can be obtained recursively, thereby ensuring the completeness of the content of the final information retrieval results.

[0027] In one possible implementation, when the neighboring slice is the preceding neighboring slice of the initial text slice, and the neighboring slice is updated to the initial text slice, and the step of determining whether the neighboring slice association index of the initial text slice meets the splicing condition of the neighboring slice is returned, more text slices associated with the initial text slice can be obtained by recursion upwards.

[0028] In one possible implementation, when a neighboring slice is a subsequent neighboring slice of the initial text slice, and the neighboring slice is updated to the initial text slice, and the step of determining whether the association index of the neighboring slice of the initial text slice meets the splicing condition of the neighboring slice is returned, a downward recursive approach can be used to obtain more text slices associated with the initial text slice.

[0029] In one possible implementation, when the initial text slice is the first text slice among multiple text slices, its neighboring slices only include the following neighboring slices. In this case, all text slices associated with the initial text slice can be obtained simply by using a downward recursive approach.

[0030] In one possible implementation, when the initial text slice is the last text slice among multiple text slices, its neighboring slices only include the preceding neighboring slice. In this case, all text slices associated with the initial text slice can be obtained simply by using an upward recursive approach.

[0031] In one possible implementation, when the initial text slice is the text slice between the first and last of multiple text slices, and this initial text slice has both preceding and following neighbor slices, all text slices associated with the initial text slice can be obtained using upward and downward recursion. Here, the upward and downward recursion methods can retrieve text slices associated with the initial text slice synchronously or asynchronously.

[0032] In one possible implementation, the knowledge base also includes semantic vectors for each text slice, which are used to achieve semantic similarity matching with the information retrieval request. Accordingly, upon receiving an information retrieval request, initial text slices are obtained from the knowledge base, including: The information retrieval request is semantically matched with the text slices in the knowledge base to obtain the text slice with the highest matching degree, and this text slice is used as the initial text slice.

[0033] The initial text slice is the one with the highest matching degree among multiple text slices. This initial text slice has the highest semantic matching degree with the information retrieval request. Therefore, when recursively retrieving with this initial text slice as the root node, it can be ensured that the text context of the obtained information retrieval results is relatively coherent, thereby avoiding excessive text fragmentation in the retrieval.

[0034] In one possible implementation, the semantic vector is a vectorized version of the slice summary of the corresponding text slice. In this case, redundant information in the text slice has less interference with semantic vector matching, effectively improving the matching between the initial text slice and the information retrieval request, and ensuring the accuracy of the final information retrieval results.

[0035] In one possible implementation, each of the multiple text slices also includes a slice keyword for the text slice; correspondingly, before performing semantic vector matching between the information retrieval request and the text slices in the knowledge base to obtain the most relevant text slice, the following is also included: The query keywords are extracted from the information retrieval request and matched with the keywords of each text slice in the knowledge base to obtain multiple candidate text slices. Correspondingly, semantic vector matching is performed between the information retrieval request and the text slices in the knowledge base to obtain the text slice with the most relevant match, which is then used as the initial text slice. The information retrieval request is semantically matched with the candidate text slice to obtain the candidate text slice with the highest matching degree, and this candidate text slice is used as the initial text slice.

[0036] The method in this application employs a dual filtering approach—slice keyword matching and semantic vector matching—to obtain the most relevant candidate text slices from the knowledge base. These candidate text slices are then used as initial text slices, ensuring both the accuracy of the initial text slices and the accuracy of the final information retrieval results. Furthermore, since the semantic vector is a vectorized summary of the corresponding text slice, the retrieval of the initial text slices focuses on the core content of the text slices, reducing interference from redundant information and thus improving the matching between the initial text slices and the information retrieval request, thereby increasing the accuracy of the information retrieval results.

[0037] Seventhly, embodiments of this application also provide an information retrieval device, including: The acquisition module is used to acquire an initial text slice from a knowledge base when an information retrieval request is received. The knowledge base includes neighbor slice association indicators and slice content of multiple text slices. The initial text slice corresponds to a neighbor slice association indicator. The splicing module is used to determine whether the association index of the neighboring slices of the initial text slice meets the splicing conditions of the neighboring slices. If the association index of the neighboring slices of the initial text slice meets the splicing conditions of the neighboring slices, the neighboring slices of the initial text slice are obtained from the knowledge base based on the initial text slice. The neighboring slices are the text slices that are adjacent to the initial text slice among multiple text slices.

[0038] Eighthly, embodiments of this application also provide an electronic device, including: Processor; and, Memory for stored programs; The program includes instructions, which the processor executes to cause the computer device to perform the method according to the sixth aspect or any possible implementation thereof according to the embodiments of this application.

[0039] Ninthly, embodiments of this application also provide a computer storage medium storing computer instructions that, when executed on an electronic device, cause the processor of the electronic device to perform the method described in accordance with the sixth aspect or any possible implementation thereof according to embodiments of this application.

[0040] In a tenth aspect, embodiments of this application also provide a computer program product, including a computer program, wherein the computer program, when executed by a processor, implements the method described in the sixth aspect or any possible implementation thereof.

[0041] The beneficial effects of the technical solutions of the sixth to tenth aspects of the embodiments of this application can be referred to the beneficial effects of the methods described in the first aspect or any possible implementation of the first aspect of the embodiments of this application, which will not be repeated here. Attached Figure Description

[0042] Further details, features, and advantages of this application are claimed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which: Figure 1 A schematic diagram of the system framework of a knowledge question-answering system that can be applied to various methods in the embodiments of this application is shown. Figure 2 A flowchart illustrating the knowledge base construction method according to an embodiment of this application is shown; Figure 3 A schematic diagram illustrating the framework of the knowledge base construction principle in an embodiment of this application is shown; Figure 4 A flowchart illustrating the information retrieval method according to an embodiment of this application is shown; Figure 5 This invention illustrates a schematic diagram of the information retrieval principle according to an embodiment of the present application. Figure 6 A schematic diagram of the search configuration interface according to an embodiment of this application is shown; Figure 7 A schematic block diagram of a functional module of a knowledge base construction apparatus according to an exemplary embodiment of this application is shown; Figure 8 A schematic block diagram of a functional module of an information retrieval device according to an exemplary embodiment of this application is shown; Figure 9 A schematic block diagram of a chip according to an exemplary embodiment of this application is shown; Figure 10 A structural block diagram of an exemplary computer device that can be used to implement embodiments of this application is shown. Detailed Implementation

[0043] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While some embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this application. It should be understood that the drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0044] It should be understood that the steps described in the method embodiments of this application may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this application is not limited in this respect.

[0045] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc., mentioned in this application are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0046] It should be noted that the terms "a" and "a plurality of" used in this application are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more". Before introducing the embodiments of this application, the relevant terms involved in the embodiments of this application are first explained as follows: Pre-trained language models refer to models that learn general language representations by pre-training on large-scale text data, and then fine-tuning them to adapt them to specific tasks. This approach significantly improves the model's generalization ability and development efficiency.

[0047] Vectorization refers to the process of converting data into numerical vectors. Vectorization is very important in the field of data processing, especially in text processing, feature extraction, and data representation.

[0048] Keyword extraction involves extracting words from text that express the central content of a document. It is commonly used in document retrieval, automatic summarization, text clustering, and classification. Keyword extraction algorithms are mainly divided into two categories: unsupervised and supervised.

[0049] Reranking is a process that re-evaluates and sorts the candidate documents retrieved initially, prioritizing the presentation of the most relevant documents to improve the quality of the generative model's response.

[0050] Figure 1 A schematic diagram of the system framework of a knowledge question-answering system applicable to various methods in embodiments of this application is shown. For example... Figure 1 As shown, the knowledge question answering system 100 of this application includes a construction device 101 and a retrieval device 102. The construction device 101 may include a preprocessing module 1011 and a knowledge base 1012.

[0051] like Figure 1 As shown, the preprocessing module 1011 can acquire the information to be processed, convert the information to be processed into text to be processed, preprocess the text to be processed to obtain multiple text slices, and then store the multiple text slices in the knowledge base 1012, so that the multiple text slices stored in the knowledge base 1012 can be used by the retrieval device 102.

[0052] Optionally, the information to be processed can be a document or speech. When the information to be processed is a document, the document format can be text, Word, or Portable Document Format (PDF), but is not limited to these. In this case, a text extraction model (such as an optical character recognition model, TextBoxes model, etc.) can be used to extract the text to be processed from the document. When the information to be processed is speech, a speech recognition model (Moshi, Mini-Omni, Freeze-Omni, etc.) can be used to extract the text to be processed from the speech.

[0053] Optional, such as Figure 1 As shown, when storing each text slice to the knowledge base 1012, the preprocessing module 1011 can configure an identity identifier for each text slice (the identity identifier can be represented in the form of a slice ID), and bind all the detailed content included in the text slice to the identity identifier of the text slice.

[0054] Optional, such as Figure 1 As shown, the local terminal 103 can respond to the input operation of information retrieval and send an information retrieval request to the retrieval device 102. The retrieval device 102 can obtain the initial text slice from the knowledge base 1012 and combine it with the detailed data of the initial text slice to further determine whether it is necessary to splice the slice content of the neighboring slice into the slice content of the initial text slice.

[0055] like Figure 1 As shown, considering that when the neighbor slice key index is relatively large, the semantic correlation between two adjacent text slices is stronger, and when the neighbor slice key index is relatively small, the semantic correlation between two adjacent text slices is weaker, therefore, as Figure 1 As shown, the retrieval device 102 can determine whether to retrieve the neighboring slices of the initial text slice from the knowledge base based on the association index of the neighboring slices of the initial text slice, and then concatenate the slice content of the neighboring slices of the initial text slice with the slice content of the initial text slice.

[0056] Optionally, after concatenating the slice content of the neighbor slice with the slice content of the initial text slice, the neighbor slice of the initial text slice can be used as the new initial text slice, and the previous operation can be repeated until the retrieval device 102 detects that it is not necessary to concatenate the slice content of the neighbor slice with the slice content of the initial text slice. Information retrieval results can be obtained, and the information retrieval results can be further returned to the local terminal 103, or the information retrieval results can be sent to the large language model, and the results returned by the large language model can be returned to the local terminal 103.

[0057] Optionally, some text slices in the knowledge base may include slice content and neighbor slice association indicators, whereby the neighbor slice association indicators refer to the degree of association between two adjacent text slices. If a first text slice and a second text slice exist in the knowledge base, and the first and second text slices are adjacent in the text to be processed, then the slice content similarity between the first and second text slices can be shared.

[0058] When the first text slice is a preceding neighbor slice of the second text slice, the content similarity between the first and second text slices can be considered as an association index of the following neighbor slices of the first text slice. Similarly, the content similarity between the first and second text slices can be considered as an association index of the preceding neighbor slices of the first text slice.

[0059] It can be seen that, as Figure 1 As shown, the knowledge question answering system of this application embodiment can configure the detailed content included in the text slices stored in the knowledge base 1012 to ensure the completeness and coherence of the information retrieval results retrieved by the retrieval device 102 from the knowledge base 1012 in a recursive manner, thereby meeting the information retrieval requirements of user needs and professional scenarios.

[0060] In some optional embodiments, the knowledge base of this application may further include at least one slice summary and slice keywords. When the knowledge base includes slice keywords and slice summaries of the same text slice, the slice summary may also be stored in a form. Through slice keywords and slice summaries, multiple text slices in the knowledge base can be double-filtered to accurately retrieve the initial text slice and ensure that the final information retrieval results are accurate, preventing excessive text fragmentation in the information retrieval results.

[0061] From a hardware perspective, the knowledge base construction method of this application embodiment can be executed by an electronic device or a chip applied to an electronic device. The computer device can be a standalone computer device or a cluster of computer devices. Figure 2 A flowchart illustrating the knowledge base construction method according to an embodiment of this application is shown. Figure 2 As shown, the knowledge base construction method 200 of this application embodiment includes steps 201 to 203.

[0062] In step 201, the text information to be processed is processed to obtain the slice content of multiple text slices. Here, the slice content of each text slice has an independent and complete semantic unit.

[0063] Optionally, the text information to be processed can refer to various possible files that can be converted into text. The file format can include text format, doc format, PDF format, or audio format, but is not limited to these.

[0064] Optionally, the text content of the text information to be processed can be divided into multiple text slices. These text slices can then be sorted according to their positions relative to the text content to be processed. Here, the text content of the text information to be processed can be segmented according to segmentation rules to obtain multiple text slices.

[0065] In one example, the segmentation rules can be set so that the content of each text slice includes a preset number of paragraphs. For instance, each text slice may include the text information of one paragraph. The paragraph segmentation criteria here can refer to the paragraph segmentation criteria included in the text content of the text information to be processed.

[0066] In another example, the segmentation rules can be set so that each text slice includes a preset number of sentences of text information. For example, each text slice may include 50 to 200 sentences of text information. The sentences here can be determined by referring to the sentence segmentation criteria within the text content to be processed.

[0067] In another example, the segmentation rules can be set such that the content of each text slice includes a preset number of characters, the specific number of which can be set according to the actual situation.

[0068] In step 202, the neighbor slice association index of adjacent text slices is determined. The neighbor slice association index is used to indicate the degree of association between two adjacent text slices.

[0069] Optionally, when the neighbor slice association index is used to indicate the semantic similarity of the slice content of two adjacent text slices, the slice content of each text slice in the adjacent text slices can be determined. Then, based on the slice content of each text slice in the adjacent text slices, the semantic similarity of the slice content is determined. Finally, based on the semantic similarity of the slice content, the neighbor slice association index of the adjacent text slices is obtained. Considering that the semantic similarity of the slice content can reflect the degree of semantic association between the slice content of adjacent text slices, the neighbor slice association index of the adjacent text slices can be obtained based on the semantic similarity of the slice content.

[0070] Optionally, two adjacent text slices can refer to text slices whose text content is located in adjacent positions within the text information to be processed. These two positions can be adjacent in a narrow sense or in a broad sense. When two positions are broadly adjacent, it can mean that there is some text content between the positions of the two adjacent text slices in the text information to be processed, but this part of the text content is not sliced ​​due to information redundancy or insignificance.

[0071] In step 203, a target knowledge base is constructed based on neighbor slice association metrics and the slice content of multiple text slices. The target knowledge base can store neighbor slice association metrics for a single text slice or for multiple text slices.

[0072] When an information retrieval request is received, the neighbor slice association index of the initial text slice obtained from the knowledge base is used to indicate the degree of association between two adjacent text slices. Therefore, it can be determined whether the neighbor slice association index of the initial text slice meets the concatenation condition of the neighbor slice. When the neighbor slice association index of the initial text slice meets the concatenation condition of the neighbor slice, it means that the degree of association between the initial text slice and its neighbor slice is relatively high. Therefore, based on the order of the neighbor slice and the initial text slice, the slice content of the neighbor slice of the initial text slice can be concatenated with the slice content of the initial text slice.

[0073] As can be seen, the method in this application embodiment can determine whether it is necessary to splice the slice content of the neighboring slices into the slice content of the initial text slice based on the neighboring slice association index of the initial text slice. This can ensure the logical coherence of the final information retrieval results, thereby meeting the information retrieval requirements of user needs and professional scenarios.

[0074] It should be noted that the order of the multiple text slices in the embodiments of this application can be the sorting of the positions of the multiple text slices in the text to be processed. For two adjacent text slices, their positions in the text to be processed are adjacent.

[0075] In one possible implementation, determining the neighbor slice association index of adjacent text slices includes: determining the neighbor slice association index of each adjacent text slice; correspondingly, the target knowledge base includes: the neighbor slice association index of each adjacent text slice, and the slice content of multiple text slices.

[0076] When the neighbor slice association indicators of each adjacent text slice in multiple text slices are stored in the target knowledge base, a global search of text slices can be performed in the target knowledge base during the information retrieval process. This allows for the search of the maximum range of associated slices in the target knowledge base, with the initial text slice as the root node, thus ensuring the completeness of the final information retrieval results and avoiding information omissions.

[0077] Optionally, after concatenating the content of neighboring slices with the content of the initial text slice, the neighboring slices can be updated to the initial neighboring slices. The association index of the neighboring slices of the updated initial text slice is then determined to see if it meets the concatenation conditions of the neighboring slices. This further determines whether the content of the neighboring slices needs to be concatenated with the content of the updated initial text slices. By recursively searching for text slices associated with the initial text slices from the target knowledge base, the completeness and logical coherence of the information retrieval results can be guaranteed to be relatively high, thereby meeting the information retrieval requirements of user needs and professional scenarios.

[0078] In one possible implementation, embodiments of this application can input the slice content of a text slice and the slice content of its neighboring slices into a pre-trained language model to determine the semantic similarity of the slice content. Then, based on the semantic similarity of the slice content, a neighboring slice association index for adjacent text slices can be determined. Alternatively, a slice summary of the text slice can be determined based on its slice content, and a slice summary of the neighboring slices can be determined based on the slice content of the neighboring slices. Then, the slice summaries of the text slice and the neighboring slices are input into a pre-trained language model to obtain the semantic similarity of the slice content. Finally, based on the semantic similarity of the slice content, a neighboring slice association index for adjacent text slices can be determined.

[0079] Optionally, pre-trained language models, such as generative pre-trained Transformer series models, can be used to process the content of each text slice to obtain a text summary. This generative pre-trained Transformer analyzes the content of each text slice, extracting core information such as time, location, task, and time, and uses this core information to generate a concise text summary. This removes redundant information from the text slices, highlights the key information, and represents this key information in the form of semantic vectors.

[0080] When a pre-trained language model determines the neighbor slice association index of a text slice based on the slice summary of the text slice and the slice summary of the neighbor slice, the slice summary contains the core content of the text slice. Without the interference of redundant information, the neighbor slice association index of the text slice determined by the pre-trained language model can more accurately indicate the degree of association between the text slice and the neighbor slice. This can reduce the interference of redundant information in the text slice on information retrieval and improve the accuracy of information retrieval.

[0081] In one example, several correlation degree intervals can be set, and a mapping relationship can be established between the semantic similarity intervals of the slice content and the correlation indicators of the neighboring slices of the text slice. Then, the correlation indicators of the neighboring slices of the text slice can be determined with reference to the mapping relationship.

[0082] For example, 11 semantic similarity intervals for slice content can be set, and the neighbor slice association index of the text slice is 0~10. According to the size order of the semantic similarity intervals of slice content, the neighbor slice association index of the text slice corresponding to the 11 semantic similarity intervals of slice content can be configured as 0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9 and 1.0.

[0083] In one possible implementation, the neighboring slices of the text slice in this application embodiment may include the preceding text slice, the following text slice, or both the preceding and following text slices.

[0084] When the neighbor slice association index includes the preceding neighbor slice association index, in two adjacent text slices, the text content of one text slice is the preceding text content of the other text slice. In this case, the preceding neighbor slice association index can be used to determine whether the slice content of the initial text slice can be concatenated with the slice content of the preceding neighbor slice of the initial text slice, thereby ensuring that the preceding content starting from the slice content of the initial text slice in the final information retrieval result has a relatively high degree of coherence.

[0085] When the neighboring slice association index includes the following neighboring slice association index, it means that in two adjacent text slices, the text content of one text slice is the following text of the text content of the other text slice. In this case, the following neighboring slice association index can be used to determine whether the slice content of the initial text slice can be concatenated with the slice content of the following neighboring slice of the initial text slice, thereby ensuring the coherence between the initial text slice and the following content of the initial text slice included in the information retrieval results.

[0086] Optionally, when multiple text slices are arranged in order to form a text slice queue, if the neighboring slices of a certain text slice only include the preceding neighbor slice, the text slice can be the last text slice in the text slice queue.

[0087] When the concatenation conditions for neighboring slices include: the correlation index of the preceding neighboring slice of the initial text slice is greater than or equal to the first preset index, it can be confirmed that the initial text slice meets the concatenation conditions for neighboring slices. The slice content of the preceding neighboring slice of the initial text slice is then concatenated with the slice content of the initial text slice. Therefore, the embodiments of this application can use the relationship between the preceding neighboring slice correlation index and the first preset index to determine whether the initial text slice needs to be recursively concatenated upwards, thereby ensuring the continuity of the preceding content of the text content included in the information retrieval results.

[0088] Optionally, when multiple text slices are arranged in order to form a text slice queue, if the neighboring slices of a certain text slice only include the following neighboring slices, the text slice can be the first text slice in the text slice queue.

[0089] When the concatenation conditions for neighboring slices include: the correlation index of the following neighboring slices of the initial text slice is greater than or equal to the second preset index, it can be confirmed that the initial text slice meets the concatenation conditions for neighboring slices. The slice content of the following neighboring slices of the initial text slice is then concatenated with the slice content of the initial text slice. Therefore, the embodiments of this application can utilize the relationship between the correlation index of the following neighboring slices and the second preset index to achieve recursive concatenation of the initial text slices downwards, thereby ensuring the continuity of the following content of the text content included in the information retrieval results.

[0090] Optionally, when multiple text slices are arranged sequentially to form a text slice queue, if a text slice's neighboring slices include both preceding and following neighboring slices, the first text slice can be the text slice between the first and last text slices in the text slice queue. In this case, when the neighboring slice association index of the initial text slice satisfies the neighboring slice concatenation condition, it is necessary not only to determine whether the preceding neighboring slice association index of the initial text slice satisfies the neighboring slice concatenation condition, but also to determine whether the following neighboring slice association index of the initial text slice satisfies the neighboring slice concatenation condition. This allows for the recursive upward and downward concatenation of the initial text slice using both the following neighboring slice association index and the preceding neighboring slice key index, thereby ensuring the coherence of the text content included in the information retrieval results.

[0091] In one possible implementation, the target knowledge base of this application embodiment further includes semantic vectors of each text slice. These semantic vectors are used to achieve semantic similarity matching with information retrieval requests; correspondingly, the method further includes: extracting a slice summary of each of the multiple text slices based on the slice content of each of the multiple text slices; and vectorizing each slice summary to obtain a semantic vector of each of the multiple text slices.

[0092] Each slice summary has removed redundant information from the corresponding text slice summary, allowing the vector representation to focus more on the core content of the text slice. Therefore, each slice summary can be vectorized to obtain the semantic vector of the corresponding slice text, and stored in the target knowledge base. Based on this, when retrieving the initial text slice from the target knowledge base using a vectorized retrieval method, the influence of redundant information in the text slices stored in the target knowledge base can be reduced, ensuring that the retrieved initial text slice matches the information retrieval request. Thus, recursively retrieving from the target knowledge base using this initial text slice as the root node can guarantee the accuracy of the final information retrieval results.

[0093] Optionally, the method further includes: determining a corresponding slice keyword for each of the multiple text slices; correspondingly, the target knowledge base also includes: the slice keyword corresponding to each of the multiple text slices. For example, the slice keyword corresponding to one of the multiple text slices can be directly obtained based on the slice content of each of the multiple text slices, or a slice summary corresponding to one of the multiple text slices can be determined based on the slice content of each of the multiple text slices, and then the slice keyword corresponding to one of the multiple text slices can be obtained based on the slice summary of each of the multiple text slices.

[0094] When the target knowledge base includes multiple text slices, each with its corresponding slice keywords, multiple candidate text slices can be selected from the target knowledge base first through slice keyword matching, thus completing the initial filtering of text slices. Then, semantic vector matching is used to further filter the initial text slices from the multiple candidate text slices, thus completing the secondary filtering of text slices. This dual filtering method of slice keyword matching and semantic vector matching not only takes into account whether the text slice contains slice keywords that match the information retrieval request, but also whether the text slice contains core content that matches the information retrieval request. Therefore, the embodiments of this application can reduce the interference of redundant information on the retrieval of the initial text slices and improve the accuracy of information retrieval results through the dual filtering method of slice keyword matching and semantic vector matching.

[0095] Optionally, duplicate slice keywords can be deduplicated across multiple text slices. This reduces the storage pressure on the knowledge base. For example, if there are five text slices, and three of them share the same slice keyword, only the slice keyword of one of these three text slices can be retained to achieve the purpose of deduplicating slice keywords, thereby preventing the problem of invalid storage space usage caused by retaining duplicate slice keywords in the knowledge base.

[0096] In one example, the number of slice keywords for a text slice can be one or more, depending on the specific situation. To establish a mapping relationship between slice keywords and text slices, this mapping relationship can also be established. Thus, after retrieving a slice keyword, the candidate text slice corresponding to that slice keyword can be determined based on the association between the slice keyword and the text slice.

[0097] To clearly describe the knowledge base construction method of this application embodiment, the following is combined with... Figure 3 The following description is for illustrative purposes only and is not intended to be limiting.

[0098] First step, such as Figure 3 As shown, long documents are segmented: Segmentation module 301 segments long documents based on semantic paragraphs, sentence count, or character length, thereby obtaining a sequence of text slice content [C1, C2, ..., C...]. n ], where C1 represents the slice content of the first text slice, C2 represents the slice content of the second text slice, ..., C n This represents the content of the nth text slice. The second step, as Figure 3 As shown, slice summary generation 302A-1: Using a pre-trained language model, core elements such as time, location, and event can be extracted from the slice content of each text slice. Slice summaries are generated using these core elements, and the slice summaries of each text slice are vectorized (e.g., using SBERT or Doc2Vec techniques) 302A-2 to generate semantic vectors for the text slices.

[0099] The third step, as Figure 3 As shown, segment keyword extraction 302B: This involves using a pre-trained language model to extract segment keywords from the content of each text segment. The pre-trained language model can analyze the content of each text segment based on its semantics and grammatical structure, thereby extracting segment keywords that accurately summarize the content of each text segment.

[0100] Step four, as Figure 3 As shown, the contextual relevance score 302C is calculated using a pre-trained language model. For example, the C value for the i-th text slice can be calculated using a pre-trained language model.i The content of the slice and the text slice C of the (i-1)th generation i-1 The upper correlation index U of the slice content i And calculate the i-th text slice C i The content of the slice and the text slice C of the (i+1)th time are related. i+1 The sub-correlation index D of the slice content i .

[0101] Optionally, the pre-trained language model can measure the i-th text slice C through multiple dimensions (such as logical coherence, semantic similarity, topic consistency, etc.). i The content of the slice and the text slice C of the (i-1)th generation i-1 The upper correlation index of the slice content is used, and then a weighted sum is used to determine the final text slice C of the i-th text slice. i The content of the slice and the text slice C of the (i-1)th generation i-1 The upper correlation index U of the slice content i The value ranges from 0 to 1.

[0102] Similarly, pre-trained language models can measure the i-th text slice C through multiple dimensions (such as logical coherence, semantic similarity, topic consistency, etc.). i The content of the slice and the text slice C of the (i+1)th time are related. i+1 The sub-correlation index D of the slice content i Then, a weighted summation method is used to determine the i-th text slice C. i The content of the slice and the text slice C of the (i+1)th time are related. i+1 The sub-correlation index D of the slice content i The value ranges from 0 to 1.

[0103] The up-correlation index U of the i-th text slice is measured from multiple dimensions. i and related indicator D i This can guarantee the correlation index U. i and related indicator D i This ensures accuracy, thereby improving the precision of subsequent searches.

[0104] Step 5, Data Storage: Assign an identity to each text slice and store each text slice in the vector library of knowledge base 303. Taking the i-th text slice as an example, under the identity of the i-th text slice, the slice summary, slice keywords, and upper correlation index U of that text slice can be stored. i and related indicator D i Of course, it can also save the i-th text slice C. i The sliced ​​content.

[0105] Considering that different text slices may have the same slice keywords, the slice keywords of multiple text slices can be deduplicated, and all the deduplicated slice keywords can be bound to the corresponding text slices, thereby reducing the data storage pressure.

[0106] This application also provides an information retrieval method, which can be executed by a computer device or a chip applied to a computer device.

[0107] Figure 4 A flowchart illustrating an embodiment of the information retrieval method of this application is shown. Figure 4 As shown, the information retrieval method 400 of this application embodiment includes steps 401 to 405.

[0108] In step 401, when an information retrieval request is received, an initial text slice is obtained from the knowledge base. The knowledge base includes neighbor slice association indicators and slice content of multiple text slices. The initial text slice corresponds to a neighbor slice association indicator.

[0109] Optionally, the knowledge base also includes semantic vectors for each text slice, which are used to achieve semantic similarity matching with the information retrieval request. Accordingly, the information retrieval request can be semantically matched with the text slices in the knowledge base to obtain the text slice with the highest matching degree, and this text slice is used as the initial text slice.

[0110] When the initial text slice is the most relevant text slice among multiple text slices, and the text slice with the highest matching degree is obtained, the semantic matching degree of the initial text slice with the information retrieval request is the highest. This can effectively ensure the accuracy of information retrieval results, thereby meeting the information retrieval requirements of user needs and professional scenarios.

[0111] In one example, the semantic vector is a vectorized summary of the corresponding text slice. In this case, redundant information in the text slice has less interference with semantic vector matching, effectively improving the matching between the initial text slice and the information retrieval request, and ensuring the accuracy of the final information retrieval results.

[0112] Optionally, a reranking method can be used to evaluate the matching of the semantic vectors of multiple candidate text slices with the information retrieval request. Then, the text slices are reranked according to the evaluation results, and the semantic vector of the candidate text slice with the highest ranking is selected. The text slice corresponding to this semantic vector is the initial text slice.

[0113] Optionally, each of the multiple text slices may also include slice keywords. Accordingly, before performing semantic vector matching between the information retrieval request and the text slices in the knowledge base to obtain the most relevant text slice, the query keyword can be extracted from the information retrieval request and matched with the slice keywords included in each text slice in the knowledge base to obtain multiple candidate text slices. At this point, semantic vector matching can be performed between the information retrieval request and the candidate text slices to obtain the most relevant candidate text slice, which is then used as the initial text slice.

[0114] As can be seen, the embodiments of this application can filter multiple text slices in the knowledge base through both slice keyword matching and semantic vector matching to ensure that the obtained initial text slice has the highest matching degree with the information retrieval request, and use the candidate text slice as the initial text slice, which can ensure both the accuracy of the initial text slice and the accuracy of the final information retrieval result.

[0115] Furthermore, since the semantic vector is a vectorization of the slice summary of the corresponding text slice, the retrieval of the initial text slice focuses on the core content of the text slice, reducing the interference of redundant information on the retrieval of the initial text slice, thereby improving the matching between the initial text slice and the information retrieval request and increasing the accuracy of the information retrieval results.

[0116] In step 402, it is determined whether the neighbor slice association index of the initial text slice meets the splicing condition of the neighbor slice. The neighbor slice is the text slice that is adjacent to the initial text slice among multiple text slices.

[0117] If the neighboring slice association index of the initial text slice meets the neighboring slice splicing condition, it can be considered that the slice content of the initial text slice and the neighboring slice of the initial text slice have a relatively strong logical relationship, and step 403 can be executed; if the neighboring slice association index of the initial text slice meets the neighboring slice splicing condition, it can be considered that the slice content of the initial text slice and the neighboring slice of the initial text slice have a relatively poor logical relationship, and step 405 can be executed.

[0118] Optionally, the concatenation criteria for neighboring slices may include the neighboring slice correlation index of the initial text slice being greater than or equal to a preset index. That is, if the neighboring slice correlation index of the initial text slice is greater than or equal to the preset index, it can be considered that the neighboring slice correlation index of the initial text slice meets the concatenation criteria for neighboring slices; otherwise, the neighboring slice correlation index of the initial text slice does not meet the concatenation criteria for neighboring slices.

[0119] When a neighboring slice is a preceding neighboring slice of the initial text slice, the neighboring slice association index of the initial text slice includes the preceding neighboring slice association index. In this case, the concatenation condition for neighboring slices includes: the preceding neighboring slice association index of the initial text slice is greater than or equal to a first preset index. Therefore, when a preceding neighboring slice is detected in the initial text slice, if the preceding neighboring slice association index of the initial text slice is detected to be greater than or equal to the first preset index, it can be confirmed that the neighboring slice association index of the initial text slice satisfies the concatenation condition for neighboring slices.

[0120] Optionally, the neighboring slices include the following neighboring slices of the initial text slice, and the neighboring slice association index of the initial text slice includes: the following neighboring slice association index of the initial text slice. In this case, the concatenation condition for neighboring slices includes: the following neighboring slice association index is greater than or equal to a second preset index. Therefore, when a following neighboring slice is detected for the initial text slice, if the following neighboring slice association index of the initial text slice is detected to be greater than or equal to the second preset index, it can be confirmed that the neighboring slice association index of the initial text slice meets the concatenation condition for neighboring slices.

[0121] In other words, as long as the correlation index of the preceding neighbor slice of the initial text slice is greater than or equal to the first preset index, or the correlation index of the preceding neighbor slice of the initial text slice is greater than or equal to the correlation index of the following neighbor slice, it can be determined that the correlation index of the neighbor slice of the initial text slice satisfies the splicing condition of the neighbor slice.

[0122] In step 403, neighboring slices of the initial text slice are obtained from the knowledge base based on the initial text slice. Optionally, if the text slices stored in the knowledge base have identity information, the identity identifiers of the neighboring slices of the initial text slice can be retrieved from the knowledge base based on the identity identifier of the initial text slice, and then the relevant content of the neighboring slices can be obtained based on the identity identifiers of the neighboring slices.

[0123] In step 404, based on the order of the neighbor slices and the initial text slice, the slice content of the neighbor slices is concatenated into the slice content of the initial text slice. Since each text slice includes its own slice content and its neighbor slice association index, the neighbor slices obtained from the knowledge base actually include both the slice content and the neighbor slice association index. Because there is an order relationship between the identity identifiers of the neighbor slices and the identity identifiers of the initial text slices, the order of the neighbor slices and the initial text slices can be determined based on these two identifiers, and then the slice content of the neighbor slices is concatenated into the slice content of the initial text slice based on this order.

[0124] In step 405, the information retrieval result is determined based on the content of the initial text slice. In one example, if the neighbor slice association index of the initial text slice does not meet the neighbor slice concatenation conditions, the content of the initial text slice can be used as the information retrieval result. In another example, if the updated initial text slice does not meet the neighbor slice concatenation conditions, the information retrieval result determined in step 405 can be used as the final output information retrieval result.

[0125] As can be seen, in the method of this application embodiment, an initial text slice can be obtained from the knowledge base, and the neighbor slice association index of the initial text slice is used as a basis to determine whether it is necessary to splice the slice content of the neighbor slice into the slice content of the initial text slice. This can ensure the logical coherence of the final information retrieval results, thereby meeting the information retrieval requirements of user needs and professional scenarios.

[0126] In one alternative approach, after executing step 404, the initial text slice can be updated based on the neighbor slices, and the process can return to step 402, which involves determining whether the neighbor slice association index of the initial text slice meets the concatenation conditions of the neighbor slices, until the neighbor slice association index of the initial text slice no longer meets the concatenation conditions of the neighbor slices.

[0127] When the knowledge base includes neighbor slice association indicators for each adjacent text slice, if the neighbor slice association indicators of the initial text slice meet the concatenation conditions of neighbor slices, the slice content of the initial text slice can not only be concatenated with the slice content of neighbor slices, but the neighbor slices can also be updated to the initial text slice, and the step of determining whether the neighbor slice association indicators of the initial text slice meet the concatenation conditions of neighbor slices is returned. In this way, more text slices associated with the initial text slice can be obtained recursively, thereby ensuring the completeness of the content of the final information retrieval results.

[0128] When the neighbor slice's identifier is the (i-1)th text slice C i-1 The initial text slice is identified as the i-th text slice C. i The (i-1)th text slice C can be concatenated by placing the neighboring slices first and the initial text slice last. i-1 The slice content and the i-th text slice C i The sliced ​​content.

[0129] When the neighbor slice's identifier is the (i+1)th text slice C i+1 The initial text slice is identified as the i-th text slice C. i The (i+1)th text slice C can be concatenated by placing the neighboring slices last and the initial text slice first. i+1 The slice content and the i-th text slice C iThe sliced ​​content.

[0130] The identity of the neighboring slice is identified as the (i-1)th text slice C. i-1 The initial text slice is identified as the i-th text slice C. i For example, when splicing the (i-1)th text slice C i-1 The slice content and the i-th text slice C i After obtaining the sliced ​​content, use the (i-1)th text slice C i-1 Update the i-th text slice C i When performing step 402, it can be determined that the (i-1)th text slice C i-1 The preceding neighbor slice (i.e., the (i-2)th text slice C) i-2 If the correlation index meets the splicing conditions of neighbor slices, it means that the (i-2)th text slice C needs to be spliced. i-2 The content of the slice and the text slice C of the (i-1)th generation i-1 Therefore, the (i-2)th text slice can be obtained from the knowledge base, and the slice content of the (i-2)th text slice and the (i-1)th text slice C can be concatenated. i-1 The sliced ​​content.

[0131] Because the (i-1)th text slice C i-1 The content of the slice and the i-th text slice C i-1 The content of the slices has already been concatenated in the previous loop. Therefore, the content of the (i-2)th text slice and the (i-1)th text slice C are concatenated. i-1 After the content of the slice is obtained, it can be considered as the (i-2)th text slice C. i-2 The content of the slice is concatenated into the (i-1)th text slice C included in the information retrieval results. i-1 The content of the slice is used to update the information retrieval results, thereby making the i-th text slice C... i-1 The content of the slice, the (i-1)th text slice C i-1 The slice content and the (i-2)th text slice C i-2 The sliced ​​content was pieced together.

[0132] When a neighboring slice is the preceding neighboring slice of the initial text slice, and the neighboring slice is updated to the initial text slice, and the step of determining whether the association index of the neighboring slice of the initial text slice meets the splicing conditions of the neighboring slice is returned, more text slices associated with the initial text slice can be obtained by recursion upwards.

[0133] When a neighboring slice is a subsequent neighboring slice of the initial text slice, and the neighboring slice is updated to the initial text slice, and the step of determining whether the association index of the neighboring slice of the initial text slice meets the splicing conditions of the neighboring slice is returned, a downward recursive approach can be used to obtain more text slices associated with the initial text slice.

[0134] When the initial text slice is the first text slice among multiple text slices, its neighboring slices only include the following neighboring slices. In this case, all text slices associated with the initial text slice can be obtained by simply using a downward recursive approach.

[0135] When the initial text slice is the last text slice among multiple text slices, its neighboring slices only include the slices preceding it. In this case, all text slices associated with the initial text slice can be obtained by simply using an upward recursive approach.

[0136] When the initial text slice is the text slice between the first and last of multiple text slices, it has both preceding and following neighbor slices. Therefore, all text slices associated with the initial text slice can be obtained using upward and downward recursion. These upward and downward recursion methods can be used synchronously or asynchronously to retrieve the text slices associated with the initial text slice.

[0137] To clearly describe the principle of information retrieval using the recursive method in the embodiments of this application, Figure 5 A schematic diagram illustrating the information retrieval principle of an embodiment of this application is shown. For example... Figure 5 As shown, the information retrieval principle of this application embodiment may include the following steps.

[0138] First step, such as Figure 5 As shown, after obtaining an information retrieval request, the filtering unit 501 can filter multiple text fragments included in the knowledge base to obtain an initial text slice.

[0139] Optionally, the slice keywords of multiple text slices included in the knowledge base can be used to form a keyword set K. i Then, an inverted index filtering method is used to select at least one candidate text slice set C corresponding to the target slice keyword. The candidate text slice set C can include multiple candidate text slices, thus completing the initial filtering.

[0140] When completing the initial filtering, semantic vector matching can be performed on the semantic vectors of multiple candidate text slices included in the candidate text slice set C based on the information retrieval request, and the candidate text slice corresponding to the slice summary with the best matching is defined as the initial text slice.

[0141] Optionally, a reranking method can be used to evaluate the semantic vectors of multiple candidate text slices and match them with the vectorized information retrieval request. Then, the text slices are reranked according to the evaluation results, and the candidate text slice corresponding to the top-ranked semantic vector is selected. This candidate text slice is used as the slice summary of the initial text slice, thereby completing the secondary filtering of the text slices.

[0142] The second step involves recursively concatenating the content of the initial text slice upwards using the upward concatenation unit 502A, and recursively concatenating the content of the initial text slice downwards using the downward concatenation unit 502B. Using the initial text slice as the target text slice C_target, bidirectional recursive concatenation is performed on the target text slice C_target.

[0143] When performing upward recursive concatenation on the target text slice C_target, it can be determined whether the initial text slice C_target has a preceding neighbor slice C_prev. If the target text slice C_target has a preceding neighbor slice C_prev, it is determined whether the upper correlation index U_target of the preceding neighbor slice C_prev meets the standard (the upper correlation index U_target is greater than or equal to the first preset index). If the upper correlation index U_target of the preceding neighbor slice C_prev meets the standard, the slice content of the target text slice C_target and the slice content of the preceding neighbor slice C_prev can be concatenated, and the preceding neighbor slice C_prev is set as the target text slice. The upward recursive concatenation is then repeated.

[0144] If the target text slice C_target does not have a preceding neighbor slice C_prev, or if the preceding neighbor slice C_prev's upper correlation index U_target does not meet the standard, it can be determined whether the initial text slice C_target has a following neighbor slice C_next. If the target text slice C_target has a following neighbor slice C_next, it can be determined whether the target text slice C_target's lower correlation index D_target meets the standard (the lower correlation index D_target is greater than or equal to the second set index). If the lower correlation index D_target of the following neighbor slice C_next meets the standard, the slice content of the target text slice C_target and the following neighbor slice C_next can be concatenated, and the following neighbor slice C_next can be set as the target text slice. The recursive concatenation can be repeated.

[0145] If the target text slice C_target does not have a neighboring slice C_next, or if the relevance index D_target of the neighboring slice C_next is not met, the concatenated text can be output and used as the information retrieval result.

[0146] In an optional embodiment, the first and second preset indicators of this application can be parameters preset by the retrieval device, or parameters configured by the user before sending an information retrieval request through a local terminal. In this case, the information retrieval request includes not only the keywords to be queried, but also the first and second preset indicators. Therefore, after obtaining the information retrieval request, it can be parsed to obtain the key information to be queried, the first preset indicator, and the second preset indicator. The key information to be queried may include the keywords to be queried, and when there are multiple keywords to be queried, it may also include the logical relationships between different keywords.

[0147] Optionally, a search configuration interface can be displayed on the local terminal's screen, where relevant parameters for information retrieval can be configured. Figure 6 A schematic diagram of the search configuration interface according to an embodiment of this application is shown. For example... Figure 6 As shown, it is possible to Figure 6 The text box for configuring "Tavily API Key" allows you to enter an API key, which can be used as an identity credential to access the retrieval device. You can also select the type of knowledge base to search in the knowledge base selection box. Additionally, you can set the keyword similarity weight, slice association index threshold (i.e., the first preset index and the second preset index), and the TopN hit parameter for reranking by sliding the slider.

[0148] Optionally, by setting keyword similarity weights, the filtering threshold for keyword filtering in the filtering unit can be configured. By setting the slice association index threshold, the possibility of splicing the preceding and following neighbor slices of the initial text slice can be limited. When TopN=1, it can be assumed that the candidate text slice with the highest ranking in the rerank sort is selected as the initial text slice.

[0149] To clearly explain the knowledge base construction and information retrieval process of the question-answering system in this application embodiment, two examples are given below for illustrative purposes.

[0150] I. Contract document retrieval.

[0151] Knowledge Base Construction: The contract document can be divided into 50 text slices according to clause paragraphs, with each text slice containing up to 1500 words. Then, a clause summary is generated for each text slice. For example, for the third text slice: "Article 3: Liability for Breach of Contract - Party A shall pay a penalty of 0.1% per day for overdue payments," keywords such as "liability for breach of contract," "overdue payment," and "penalty" can be extracted using the TextRank model. Then, the upper and lower association indicators of each text slice are calculated. Finally, the slice content, the semantic vector of the text slice generated from the slice summary, the slice keywords, and the associations between upper and lower association indicators are saved to the vector library for information retrieval.

[0152] Taking the third text slice as an example, the upper correlation index between the third text slice and the second text slice (definition clause) is calculated to be 0.85 (high correlation), and the lower correlation score between the third text slice and the fourth text slice (dispute resolution) is calculated to be 0.62 (medium correlation).

[0153] When an information retrieval request includes "overdue payment penalty", the keyword filtering in filtering unit 501 can filter out 12 candidate text slices containing "overdue payment" or "penalty". Then, vector retrieval is used for slice summary filtering, which retrieves the third text slice with a similarity of 0.92. This third text slice is set as the initial text slice, and upward and downward recursive splicing are performed on it. The process of upward and downward recursive splicing is described below using an example where both the first and second preset indices are 0.7.

[0154] When performing upward recursive concatenation, it can be found that the uplink index of the third text slice to the second text slice is 0.85, which is greater than 0.7. Therefore, the third text slice and the second text slice can be concatenated upwards. Then, the second text slice is used as the new initial text slice. It is found that the uplink index of the second text slice to the first text slice is 0.5, which is less than 0.7. Therefore, it is not necessary to concatenate the first text slice upwards from the second text slice. The upward concatenation can be stopped, thus completing the upward recursive concatenation of the third text slice.

[0155] When performing downward recursive splicing, it can be found that the downward correlation index of the 3rd text slice to the 4th text slice is 0.62, which is less than 0.7. Therefore, it is not necessary to splice the 1st text slice downward from the 3rd text slice. The upward splicing can be stopped, thus completing the downward recursive splicing of the 3rd text slice.

[0156] As can be seen, by recursively concatenating the third text slice upwards and downwards, the information retrieval result of concatenating the second and third text slices can be output.

[0157] II. Journey to the West Knowledge Quiz System.

[0158] 1. Construction of the knowledge base for "Journey to the West".

[0159] The first step is text segmentation: The entire text of "Journey to the West" is divided into 300 text slices according to chapters and plot segments. Each text slice contains 1 to 2 plots and has 1200 to 1800 words. For example, the chapter containing "Three Battles with the White Bone Demon" is divided into 3 text slices.

[0160] The second step is summary generation: a pre-trained language model is used to generate a slice summary for each text slice. For example, the summary of the plot of the White Bone Demon turning into a village girl is: "The White Bone Demon of White Tiger Ridge transforms into a village girl to deceive the master and his disciples. Sun Wukong sees through her deception and kills her, which displeases Tang Sanzang."

[0161] The third step is to vectorize the summary: use an embedding model (such as bge-large-zh-v1.5) to vectorize the slice summary (such as the summary of the plot of the white-bone demon turning into a village girl), so that the semantic vector of the text slice generated by the slice summary focuses on the core content.

[0162] The fourth step is keyword extraction: using a pre-trained language model to extract slice keywords from the slice summary. For example, "Sun Wukong", "White Bone Demon", and "village girl" can be extracted from the slice summary of the plot of the White Bone Demon turning into a village girl.

[0163] The fifth step is to score the degree of association: use a pre-trained language model to analyze the association index of each text slice with its neighboring slices. For example, the text slice "beat the village girl to death" has an upward association index of 0.88 with the preceding text slice "planning changes" and a downward association index of 0.91 with the following text slice "beautify the old woman".

[0164] Step 6, Data storage: Save the content of each text slice, the semantic vector generated from the slice summary, the slice keywords, and the associations between the upper and lower correlation indicators to the vector library.

[0165] 2. Information retrieval related to excerpts from "Journey to the West".

[0166] Step 1: When creating the enhanced search on the local terminal, set both the first and second preset indicators to 0.8; enter "the plot of Sun Wukong's three battles with the White Bone Demon" in the search box on the local terminal, and transmit this information to the search device as an information retrieval request.

[0167] The second step involves two rounds of text filtering to obtain the initial text slices. For example, during keyword filtering, the retrieval device can parse the information retrieval request to obtain the keywords to be queried. Based on these keywords, it can select 20 candidate text slices containing words such as "Sun Wukong" and "White Bone Demon" from 300 text slices. During semantic vector retrieval of the text slices, the semantic vectors of the 20 candidate text slices are retrieved and sorted using reranking. The candidate text slice corresponding to the semantic vector of "beat the village girl to death..." is selected and set as the initial text slice. Then, upward and downward recursive splicing is performed on this initial text slice.

[0168] During the upward recursive splicing process, it can be observed that the correlation index between the text slice "Kill the village girl" (i.e., the candidate text slice) and the preceding text slice "Planning Changes" is 0.88, which is greater than 0.8. Therefore, the text slice "Kill the village girl" can be spliced ​​upwards with the text slice "Planning Changes". Then, the text slice "Planning Changes" is used as the new initial text slice. It is then observed that the correlation index between the text slice "Planning Changes" and its preceding neighbor slice is 0.75, which is less than 0.8. Therefore, it is not necessary to splice the preceding neighbor slice upwards from the text slice "Planning Changes". The upward splicing can be stopped, thus completing the upward recursive splicing of the text slice "Kill the village girl".

[0169] During the downward recursive splicing, it can be found that the lower correlation index of the text slice "beat the village girl" (i.e., the candidate text slice) to the following text slice "transform the old woman" is 0.91, which is greater than 0.8. Therefore, the text slice "transform the old woman" can be spliced ​​downwards from the text slice "beat the village girl". Then, the text slice "transform the old woman" is used as the new initial text slice. It is then found that the upper correlation index of the text slice "transform the old woman" to the following neighbor slice is 0.785, which is less than 0.8. Therefore, it is not necessary to splice the following neighbor slice downwards from the text slice "transform the old woman". The upward splicing can be stopped, thus completing the downward recursive splicing of the text slice "beat the village girl".

[0170] It is evident that by recursively piecing together the text slice "Killing the Village Girl" upwards and downwards, the complete plot of "Three Battles with the White Bone Demon" can be output. In sequence, the plot of "Three Battles with the White Bone Demon" can include three text slices: the text slice "Planning Changes", the text slice "Killing the Village Girl", and the text slice "Transforming the Old Woman".

[0171] In one or more technical solutions provided in this application, a slice summary can be generated through a pre-trained language model, and then the slice summary can be vectorized to obtain the semantic vector of the text slice. This not only removes redundant information in the slice content, but also makes the reading process more focused on the core text content and reduces noise interference. At the same time, by combining slice keyword filtering and reranking, information retrieval results can be further filtered and optimized, significantly improving the matching accuracy of vector retrieval and allowing users to obtain more accurate content that meets their needs.

[0172] Furthermore, by calculating the bidirectional association index between each text slice and its context neighbor slices, the system can determine whether to perform upward and downward text concatenation on the initial text slice's content during information retrieval by judging whether the bidirectional association index meets the standard. This ensures the completeness of the output information retrieval results. It is evident that this recursive concatenation strategy can integrate fragmented related content into a complete document based on the relevance (such as semantic relevance) between the initial text slice and its context neighbor slices, avoiding the problem of incomplete retrieval results in traditional methods and ensuring that the information retrieval results returned to the local terminal are complete and coherent.

[0173] In addition, when calculating the bidirectional association index between each text slice and its context neighbor slices, we can comprehensively consider dimensions such as semantic similarity, logical coherence, and topic consistency between each text slice and its context neighbor slices. This quantifies the contextual dependency between each text slice and its context neighbor slices, enabling the retrieval device to have a stronger context awareness capability. During the retrieval and splicing process, it can better understand the logical relationships between texts and present users with logically clear and semantically coherent information retrieval results.

[0174] The foregoing primarily describes the solutions provided in the embodiments of this application from the perspective of computer devices. It is understood that, in order to achieve the above functions, the computer device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments applied herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0175] This application embodiment can divide a computer device into functional units based on the above method example. For example, it can divide each function into separate functional modules, or it can integrate two or more functions into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; in actual implementation, there may be other division methods.

[0176] In the case of dividing each functional module according to its corresponding function, an exemplary embodiment of this application provides a knowledge base construction apparatus, which can be a computer device or a chip applied to a computer device. Figure 7 A schematic block diagram of a functional module of a knowledge base construction apparatus according to an exemplary embodiment of this application is shown. Figure 7 As shown, the knowledge base construction apparatus 700 includes: The acquisition module 701 is used to process the text information to be processed, obtain the slice content of multiple text slices, and determine the neighbor slice association index of adjacent text slices. The neighbor slice association index is used to indicate the degree of association between two adjacent text slices. Organization module 702 is used to construct a target knowledge base based on neighbor slice association indicators and slice content of multiple text slices, wherein the target knowledge base includes the neighbor slice association indicators and slice content of multiple text slices.

[0177] In one possible implementation, the acquisition module 701 is used to determine the neighbor slice association index of each adjacent text slice; accordingly, the target knowledge base includes: the neighbor slice association index of each adjacent text slice, and the slice content of multiple text slices.

[0178] In one possible implementation, the target knowledge base also includes semantic vectors of each text slice, which are used to achieve semantic similarity matching with the information retrieval request; the acquisition module is also used to extract a slice summary of each of the multiple text slices based on the slice content of each of the multiple text slices; and to vectorize each slice summary to obtain the semantic vector of each of the multiple text slices.

[0179] In one possible implementation, the acquisition module 701 is further configured to determine the corresponding slice keyword for each of the plurality of text slices; accordingly, the target knowledge base also includes the slice keyword corresponding to each of the plurality of text slices.

[0180] By dividing each functional module according to its corresponding function, an exemplary embodiment of this application provides an information retrieval device, which may be a computer device or a chip applied to a computer device. Figure 8 A schematic block diagram of a functional module of an information retrieval device according to an exemplary embodiment of this application is shown. Figure 8 As shown, the information retrieval device 800 includes: The acquisition module 801 is used to acquire an initial text slice from a knowledge base when receiving an information retrieval request. The knowledge base includes neighbor slice association indicators and slice content of multiple text slices. The initial text slice corresponds to a neighbor slice association indicator. The splicing module 802 is used to determine whether the association index of the neighboring slices of the initial text slice meets the splicing conditions of the neighboring slices. If the association index of the neighboring slices of the initial text slice meets the splicing conditions of the neighboring slices, the neighboring slices of the initial text slice are obtained from the knowledge base based on the initial text slice. The neighboring slices are the text slices that are adjacent to the initial text slice among multiple text slices.

[0181] In one possible implementation, the knowledge base contains: neighbor slice association indicators of each adjacent text slice; correspondingly, the device 800 also includes an update module 803.

[0182] After the splicing module 802 splices the content of the neighbor slice with the content of the initial text slice based on the order of the neighbor slice and the initial text slice, the update module 803 updates the initial text slice to the neighbor slice. The splicing module 802 then re-executes the step of judging whether the neighbor slice association index of the initial text slice meets the splicing conditions of the neighbor slice, until the neighbor slice association index of the initial text slice no longer meets the splicing conditions of the neighbor slice.

[0183] In one possible implementation, the knowledge base also includes semantic vectors of each text slice, which are used to achieve semantic similarity matching with the information retrieval request; accordingly, the acquisition module 801 is used to perform semantic vector matching between the information retrieval request and the text slices in the knowledge base to obtain the text slice with the most relevant matching degree, and use the text slice as the initial text slice.

[0184] In one possible implementation, the semantic vector is: a vectorization of the slice summary of the corresponding text slice.

[0185] In one possible implementation, each of the multiple text slices also includes a slice keyword of the text slice; accordingly, the acquisition module is also used to extract the query keyword from the information retrieval request and perform keyword matching with the slice keywords of each text slice in the knowledge base to obtain multiple candidate text slices; perform semantic vector matching between the information retrieval request and the candidate text slices to obtain the candidate text slice with the most relevant matching degree, and use the candidate text slice as the initial text slice.

[0186] Figure 9 A schematic block diagram of a chip according to an exemplary embodiment of this application is shown. Figure 9 As shown, the chip 900 includes one or more processors 901 and a communication interface 902.

[0187] In one alternative embodiment, the communication interface 902 can support the computer device in executing the data sending and receiving steps in the above-described knowledge base construction method, and the processor 901 can support the computer device in executing the data processing steps in the above-described knowledge base construction method.

[0188] In another alternative, the communication interface 902 can support the computer device to perform the data sending and receiving steps in the above information retrieval method, and the processor 901 can support the computer device to perform the data processing steps in the above information retrieval method.

[0189] Optional, such as Figure 9 As shown, the chip 900 also includes a memory 903, which may include read-only memory and random access memory, and provides operation instructions and data to the processor. A portion of the memory may also include non-volatile random access memory (NVRAM).

[0190] In some implementations, such as Figure 9 As shown, processor 901 executes corresponding operations by calling operation instructions stored in memory (which may be stored in the operating system). Processor 901 controls the processing operations of any terminal device; processor can also be called a central processing unit (CPU). Memory 903 may include read-only memory and random access memory, and provides instructions and data to processor 901. A portion of memory 903 may also include NVRAM. For example, in applications, memory, communication interfaces, and other components are coupled together via a bus system, which may include, in addition to a data bus, a power bus, a control bus, and a status signal bus, etc. However, for clarity, in... Figure 9 The general labeled all buses as Bus System 904.

[0191] The methods disclosed in the embodiments of this application can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams of the application in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods applied in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0192] An exemplary embodiment of this application also provides a computer device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a program executable by the at least one processor, the program including instructions, which the processor executes to cause the computer device to perform a method for constructing a knowledge base according to an embodiment of this application.

[0193] An exemplary embodiment of this application also provides a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a method for constructing a knowledge base according to an embodiment of this application.

[0194] An exemplary embodiment of this application also provides a computer program product, including a computer program, wherein, when executed by a computer's processor, the computer program is used to cause the computer to perform a method for constructing a knowledge base according to an embodiment of this application.

[0195] An exemplary embodiment of this application also provides a computer device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a program executable by the at least one processor, the program including instructions, which the processor executes to cause the computer device to perform an information retrieval method according to an embodiment of this application.

[0196] An exemplary embodiment of this application also provides a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform an information retrieval method according to an embodiment of this application.

[0197] An exemplary embodiment of this application also provides a computer program product, including a computer program, wherein, when executed by a computer's processor, the computer program is used to cause the computer to perform an information retrieval method according to an embodiment of this application.

[0198] refer to Figure 10 The following is a structural block diagram of a computer device 1000 that can serve as the present application, which is an example of a hardware device that can be applied to various aspects of the present application. The term "computer device" is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. A computer device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present application described and / or claimed herein.

[0199] like Figure 10 As shown, the computer device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. The RAM 1003 may also store various programs and data required for the operation of the computer device 1000. The computing unit 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0200] like Figure 10As shown, multiple components in computer device 1000 are connected to I / O interface 1005, including: input unit 1006, output unit 1007, storage unit 1008, and communication unit 1009. Input unit 1006 can be any type of device capable of inputting information to computer device 1000. Input unit 1006 can receive input numerical or character information and generate key signal inputs related to user settings and / or function control of the computer device. Output unit 1007 can be any type of device capable of presenting information and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 1008 may include, but is not limited to, hard disks and optical disks. Communication unit 1009 allows computer device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0201] like Figure 10 As shown, computing unit 1001 can be various general-purpose and / or dedicated processing components with processing and computing capabilities. Some examples of computing unit 1001 include, but are not limited to, central processing unit (CPU), graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Computing unit 1001 performs the various methods and processes described above. For example, in some embodiments, the methods of the embodiments of this application can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed on computer device 1000 via ROM 1002 and / or communication unit 1009. In some embodiments, computing unit 1001 can be configured to perform the methods of the embodiments of this application by any other suitable means (e.g., by means of firmware).

[0202] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0203] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0204] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0205] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0206] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0207] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this application are performed entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a terminal, a user equipment, or other programmable device. The computer program or instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; it can also be an optical medium, such as a digital video disc (DVD); or it can be a semiconductor medium, such as a solid-state drive (SSD).

[0208] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. A method of constructing a knowledge base, characterized by, The method comprises the following steps: processing the text information to be processed to obtain the slice content of a plurality of text slices; determining the neighbor slice correlation index of adjacent text slices, which is used to indicate the correlation degree of two adjacent text slices; based on the neighbor slice correlation index and the slice content of the plurality of text slices, constructing a target knowledge base, wherein the target knowledge base includes the neighbor slice correlation index and the slice content of the plurality of text slices.

2. The method of claim 1, wherein, The method further comprises the following steps: determining the neighbor slice correlation index of each adjacent text slice; Accordingly, the target knowledge base includes the neighbor slice correlation index of each adjacent text slice and the slice content of the plurality of text slices.

3. The method according to claim 1 or 2, characterized in that, The target knowledge base further includes the semantic vector of each text slice, which is used to realize semantic similarity matching with the information retrieval request; accordingly, the method further comprises the following steps: extracting the slice abstract of each of the plurality of text slices based on the slice content of each of the plurality of text slices; vectorizing each of the slice abstracts to obtain the semantic vector of each of the plurality of text slices.

4. The method according to any one of claims 1 to 3, characterized in that, The method further comprises the following steps: determining the slice keyword corresponding to each of the plurality of text slices; Accordingly, the target knowledge base further includes the slice keyword corresponding to each of the plurality of text slices.

5. An information retrieval method characterized by, The method comprises the following steps: Upon receiving an information retrieval request, obtaining an initial text slice from a knowledge base, wherein the knowledge base includes a neighbor slice correlation index and slice content of a plurality of text slices, and the initial text slice corresponds to the neighbor slice correlation index; determining whether the neighbor slice correlation index of the initial text slice meets the splicing condition of a neighbor slice, wherein the neighbor slice is a text slice adjacent to the initial text slice in the plurality of text slices; if the neighbor slice correlation index of the initial text slice meets the splicing condition of the neighbor slice, obtaining the neighbor slice of the initial text slice from the knowledge base based on the initial text slice; splicing the slice content of the neighbor slice with the slice content of the initial text slice based on the order of the neighbor slice and the initial text slice.

6. The method of claim 5, wherein, The knowledge base includes the neighbor slice correlation index of each adjacent text slice; Accordingly, after the step of splicing the slice content of the neighbor slice with the slice content of the initial text slice based on the order of the neighbor slice and the initial text slice, the method further comprises the following steps: updating the initial text slice to the neighbor slice, and returning to the step of determining whether the neighbor slice correlation index of the initial text slice meets the splicing condition of the neighbor slice, until the neighbor slice correlation index of the initial text slice does not meet the splicing condition of the neighbor slice.

7. The method according to claim 5 or 6, characterized in that, The knowledge base further includes the semantic vector of each text slice, which is used to realize semantic similarity matching with the information retrieval request; Accordingly, the step of obtaining the initial text slice from the knowledge base upon receiving the information retrieval request comprises the following steps: The information retrieval request is matched with the text slices in the knowledge base according to semantic vectors to obtain a text slice with the highest matching degree, and the text slice is taken as the initial text slice.

8. The method of claim 7, wherein, The semantic vector is a vectorization of a slice abstract of the corresponding text slice.

9. The method according to any one of claims 6 to 8, characterized in that, Each of the plurality of text slices further comprises a slice keyword of the text slice; accordingly, before the information retrieval request is matched with the text slices in the knowledge base according to semantic vectors to obtain a text slice with the highest matching degree, the following further comprises: A to-be-queried keyword is extracted from the information retrieval request, and is matched with slice keywords of the text slices in the knowledge base to obtain a plurality of candidate text slices; Accordingly, the information retrieval request is matched with the text slices in the knowledge base according to semantic vectors to obtain a text slice with the highest matching degree, and the text slice is taken as the initial text slice, and the matching comprises: The information retrieval request is matched with the candidate text slices according to semantic vectors to obtain a candidate text slice with the highest matching degree, and the candidate text slice is taken as the initial text slice.

10. A computer device, comprising: Comprise: A processor; And A memory for storing programs; Wherein the programs comprise instructions, and the processor runs the instructions to enable the computer device to perform the method according to any one of claims 1-4 or any one of claims 5-9.

Images