Text retrieval method, apparatus, and system

Abstract

The present disclosure provides a text retrieval method, device and system, comprising: obtaining a retrieval request, the retrieval request being used to indicate retrieving text corresponding to a retrieval word from a text to be retrieved; sequentially splitting the text to be retrieved based on a paragraph, a sentence and a preset first number of words to obtain each coarse-grained text; obtaining target coarse-grained text corresponding to the retrieval word from each coarse-grained text; splitting the target coarse-grained text based on a punctuation mark to obtain each fine-grained text; obtaining target fine-grained text corresponding to the retrieval word from each fine-grained text; extracting key character information corresponding to the retrieval word from the target fine-grained text; and determining text corresponding to the retrieval word according to the key character information, which can improve the accuracy and reliability of text retrieval.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology and can be applied to information retrieval and query, particularly to a text retrieval method, device, and system. Background Technology

[0002] As the amount of information and data increases, users have higher expectations and requirements for the ability to retrieve and extract information from text. For example, users expect to be able to obtain accurate related information corresponding to the search terms directly through retrieval or search.

[0003] In related technologies, text retrieval devices used to perform text retrieval can obtain text corresponding to search terms from the text to be retrieved through literal matching.

[0004] However, the search terms may differ from the corresponding text in the searched text, resulting in lower accuracy in text retrieval.

[0005] It should be noted that the above-mentioned related technologies are only information known to the inventor personally, and do not mean that the above information had entered the public domain before the date of this application, nor do they mean that it can be considered prior art in this disclosure. Summary of the Invention

[0006] This disclosure provides a text retrieval method, apparatus, and system to avoid the aforementioned technical problems.

[0007] In a first aspect, this disclosure provides a text retrieval method, the method comprising:

[0008] Obtain a search request, which instructs the retrieval of text corresponding to the search term from the text to be searched;

[0009] The text to be searched is split sequentially based on paragraphs, sentences, and a preset first character count to obtain coarse-grained texts, and the target coarse-grained text corresponding to the search term is obtained from each coarse-grained text.

[0010] The target coarse-grained text is split based on preset punctuation marks to obtain fine-grained text, and the target fine-grained text corresponding to the search term is obtained from each fine-grained text.

[0011] Extract key character information corresponding to the search term from the target fine-grained text, and determine the text corresponding to the search term based on the key character information.

[0012] Secondly, this disclosure provides a text retrieval device, the device comprising:

[0013] The first acquisition unit is used to acquire a retrieval request, which is used to instruct the retrieval of text corresponding to the retrieval term from the text to be retrieved.

[0014] The first splitting unit is used to split the text to be retrieved sequentially based on paragraphs, sentences, and a preset first number of characters to obtain various coarse-grained texts.

[0015] The second acquisition unit is used to acquire the target coarse-grained text corresponding to the search term from each of the coarse-grained texts;

[0016] The second splitting unit is used to split the target coarse-grained text based on punctuation marks to obtain fine-grained text.

[0017] The third acquisition unit is used to acquire the target fine-grained text corresponding to the search term from each fine-grained text;

[0018] The extraction unit is used to extract key character information corresponding to the search term from the target fine-grained text;

[0019] The determining unit is used to determine the text corresponding to the search term based on the key character information.

[0020] Thirdly, this disclosure provides a text retrieval system, including:

[0021] At least one memory, the memory including at least one set of instructions for pushing information;

[0022] At least one processor communicates with the at least one memory;

[0023] Wherein, when the at least one processor executes the at least one set of instructions, the method as described in any of the first aspects is implemented.

[0024] Fourthly, this disclosure provides a processor-readable storage medium storing a computer program for causing the processor to perform the method described in any of the first aspects.

[0025] Fifthly, this disclosure provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0026] The memory stores computer-executed instructions;

[0027] The processor executes computer execution instructions stored in the memory to implement the method as described in any of the first aspects.

[0028] In a sixth aspect, this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the first aspect.

[0029] This disclosure provides a text retrieval method, apparatus, and system, comprising: acquiring a retrieval request, the retrieval request being used to instruct the retrieval of text corresponding to a search term from a text to be retrieved; sequentially splitting the text to be retrieved based on paragraphs, sentences, and a preset first character count to obtain coarse-grained texts; retrieving target coarse-grained texts corresponding to the search term from each coarse-grained text; splitting the target coarse-grained texts based on punctuation marks to obtain fine-grained texts; retrieving target fine-grained texts corresponding to the search term from each fine-grained text; extracting key character information corresponding to the search term from the target fine-grained texts; and determining the text corresponding to the search term based on the key character information. The text retrieval apparatus first performs text splitting at the coarse-grained level to obtain coarse-grained texts, then filters out target coarse-grained texts corresponding to the search term from each coarse-grained text; then performs text splitting at the fine-grained level to obtain fine-grained texts, then filters out target fine-grained texts corresponding to the search term from each fine-grained text; and finally filters out characters corresponding to the search term from the target fine-grained texts based on character dimensions. The technical features used to determine the characters in the text corresponding to the search terms can, on the one hand, avoid the drawbacks of low accuracy caused by engineering methods in the above examples, thereby improving the accuracy and reliability of text retrieval; on the other hand, it can also avoid the drawbacks of narrow applicability caused by algorithm technology in the above examples, thereby increasing the scope of application of text retrieval and improving its diversity; furthermore, the text retrieval device, based on the above-analyzed "coarse-grained + fine-grained + even finer-grained" splitting and filtering method, can enhance the refinement of the text to be retrieved and achieve comprehensiveness in the splitting of the text to be retrieved, thus making the filtering more targeted and effective, and consequently making the text retrieval more reliable and accurate; still moreover, the text retrieval device, based on the above-analyzed "coarse-grained + fine-grained + even finer-grained" splitting and filtering method, not only ensures that the retrieved text corresponding to the search terms has linguistic relevance at the contextual semantic level, but also at the character level, thereby improving the accuracy and reliability of text retrieval. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a schematic diagram illustrating an application scenario of the text retrieval method according to an embodiment of this disclosure;

[0032] Figure 2This is a schematic diagram of a text retrieval method according to an embodiment of the present disclosure;

[0033] Figure 3 This is a schematic diagram of a text retrieval method according to another embodiment of the present disclosure;

[0034] Figure 4 This is a schematic diagram illustrating the principle of the text retrieval method disclosed herein;

[0035] Figure 5 This is a schematic diagram illustrating the principle of training a coarse-grained text relevance classifier according to an embodiment of the present disclosure;

[0036] Figure 6 This is a schematic diagram illustrating the principle of training the key text extraction model according to an embodiment of this disclosure;

[0037] Figure 7 This is a schematic diagram of a text retrieval device according to an embodiment of the present disclosure;

[0038] Figure 8 This is a hardware structure diagram of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0039] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0040] It should be understood that the terms “comprising” and “having” and any variations thereof in the embodiments of this disclosure are intended to cover but not exclude inclusion. For example, a product or device that includes a series of components is not necessarily limited to those components that are explicitly listed, but may include other components that are not explicitly listed or that are inherent to such product or device.

[0041] In this disclosure, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0042] In this disclosure, the term "multiple" refers to two or more, and other quantifiers are similar.

[0043] The terms “first,” “second,” “third,” etc., used in this disclosure are used to distinguish similar or related objects or entities and do not necessarily imply a specific order or sequence, unless otherwise indicated. It should be understood that such terms can be used interchangeably where appropriate, for example, in situations where implementation can proceed in an order other than those given in the illustrations or descriptions of embodiments of this disclosure.

[0044] As used in this disclosure, the term "unit / module" means any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code capable of performing the functions associated with that element.

[0045] To facilitate the reader's understanding of this disclosure, at least some of the terms used in this disclosure are explained below:

[0046] End-to-end refers to the process from input to output, where a prediction result is obtained, the prediction result is compared with the true result (such as the ground truth) to obtain the error (such as the loss function or loss value), the error is backpropagated to each layer of the network model, and the weights and parameters of the network model are adjusted until the model converges or achieves the expected effect. All the intermediate operations are contained within the network model and are no longer processed in multiple modules.

[0047] Correspondingly, an end-to-end model refers to a network model that is self-contained (or can be viewed as a black box) from the input of raw data to the output of the result.

[0048] Natural Language Processing (NLP) is a discipline that studies language problems in human-computer interaction.

[0049] A character is a glyph or symbol, including letters, numbers, operators, punctuation marks, and other symbols, as well as some functional symbols. A character is a collective term for letters, numbers, and symbols used in electronic computers or radio communications. It is the smallest unit of data access in a data structure, typically represented by 8 binary bits (one byte).

[0050] A list is a collection of objects whose elements are distinguished by their indexes. The type of each element can be any object, and different elements do not need to be of the same type. Elements themselves can be other complex data types; for example, an element of a list can also be a list.

[0051] Deep Neural Networks (DNNs) are multi-layer unsupervised neural networks that use the output features of the previous layer as the input of the next layer for feature learning. Through layer-by-layer feature mapping, the features of existing spatial samples are mapped to another feature space, thereby learning to have a better feature representation of the existing input.

[0052] Long Short-Term Memory (LSTM) is a type of time-recurrent neural network.

[0053] Search terms, also known as search terms, refer to a specific word or phrase that a user uses to search for information in the text that provides explanatory information for that search term.

[0054] Explanatory information refers to the definition or specific explanation of the search term, such as the definition of a proper noun (e.g., if the search term is "real number", the explanatory information to be extracted is "a type of number that can correspond one-to-one with points on a number line" in the text), or a specific example of an abstract word (e.g., if the search term is "location", the explanatory information to be extracted is "a certain city and a certain district" in the text).

[0055] Long text refers to text that is relatively long, such as text that reaches (or is greater than or equal to) a preset number of characters (e.g., thousands or even tens of thousands of characters). Long text is typically used to store large amounts of text data.

[0056] In other words, based on the number of words in a text, we can categorize it into long text and short text (or even short text). Relatively speaking, long text has more words than short text. We can understand text with a relatively large number of words as long text and text with a relatively small number of words as short text. Specifically, text with a preset word count can be called long text, while text with fewer words (i.e., not reaching) the preset word count can be called short text.

[0057] As the amount of information and data increases, and long texts continue to grow, users have higher and higher expectations and requirements for the ability to retrieve and extract information from texts, especially long texts. Compared to simply locating the position of search terms in the text, users expect to be able to directly obtain the exact related information corresponding to the search terms through retrieval or searching.

[0058] For example, a user might want to find specific locations mentioned in a text (which could be short or long) by searching for the keyword "location." Similarly, a user might want to find the definition and explanation of the word "real number" in a text by searching for it.

[0059] In related technologies, two main types of methods are used to determine explanatory information corresponding to the search term from the text to be searched. These methods include explanations of the search term or text related to the search term. For example, one type of method is an engineering approach based on fuzzy search (or conditional matching), and the other type is an algorithmic technique that uses entity recognition algorithm models to extract information from the text to be searched.

[0060] Taking the text retrieval device as the execution subject of the text retrieval method as an example, the engineering method based on fuzzy query (or condition matching) can be understood as follows: when the text retrieval device obtains the search term, it can filter the search term with the words in the text to be retrieved by literal matching, so as to filter the text in the text to be retrieved that has the same literal dimension as the search term, and determine the text that corresponds to the search term.

[0061] The advantages of this type of method are its versatility, low startup cost, and applicability to various types of retrieval scenarios. However, on the one hand, the search terms may differ from the text appearing in the text, making it difficult for text retrieval devices to accurately locate the text corresponding to the search terms; on the other hand, even if the text retrieval device correctly locates the search terms in the text to be retrieved, it cannot accurately determine whether the text before and after the located search terms is related to the search terms, and information extraction and selection based on preset rules may lead to serious misjudgments.

[0062] In other words, although this type of method has a relatively wide range of applicable scenarios and a relatively low cost, it relies entirely on literal matching and is not suitable for retrieving texts where there is no character correspondence or similarity between the text description and the search terms. It is also difficult to use rules to correctly select explanatory texts that are directly related to the search terms.

[0063] Taking the text retrieval device as the execution subject of the text retrieval method as an example, the algorithm technology of the text retrieval device to extract information from the text to be retrieved using the entity recognition algorithm model can be understood as follows: the text retrieval device pre-constructs each entity category, constructs each entity under each entity category, and trains an entity recognition algorithm model based on each entity to identify the entity category corresponding to the search term from the text to be retrieved through the entity recognition algorithm model, thereby obtaining the text corresponding to the search term.

[0064] This type of method overcomes the limitation of literal matching inherent in the first type of method, and can extract text belonging to a specific entity category more accurately. However, a significant drawback of this type of method is that the entity categories it can identify are fixed at the beginning of model construction, and it cannot accept new search terms or adapt to situations where users search freely. Furthermore, while current entity recognition algorithms perform well in short texts, the large amount of information and content in long texts makes it still difficult for this type of method to extract relevant information.

[0065] In other words, this type of method can extract text relatively accurately. However, on the one hand, this type of method is suitable for scenarios with short texts; on the other hand, this type of method pre-specifies the scope of entities to be analyzed and extracted (such as names of people, places, company names, etc.). Therefore, once the search terms used by the user exceed the pre-specified scope of entities, it is impossible to extract information from the text.

[0066] In some embodiments, based on this type of method, the text retrieval device can incorporate external knowledge such as knowledge graphs and relationship graphs to accurately identify texts that are relevant to the search terms, thereby improving the retrieval capability of the text retrieval device. However, building a knowledge graph is time-consuming and labor-intensive, and constructing a fully effective knowledge graph that can adapt to users' free text retrieval requires a considerable investment of resources.

[0067] It should be noted that the content in the above-mentioned related technologies is merely information known only to the inventor and does not mean that the above information had entered the public domain before the date of this application, nor does it mean that it can be considered prior art in this disclosure.

[0068] To avoid at least one of the aforementioned problems, this disclosure proposes a technical concept developed through inventive effort: the text retrieval device first performs coarse-grained segmentation of the text to be retrieved (e.g., segmenting the text to be retrieved from dimensions such as paragraphs and sentences) to obtain coarse-grained texts, and then determines the coarse-grained texts related to the search terms from the coarse-grained texts. Next, it performs fine-grained segmentation of the coarse-grained texts related to the search terms (e.g., segmenting the text to be retrieved from dimensions such as word count and punctuation marks) to obtain fine-grained texts, and then determines the fine-grained texts related to the search terms from the fine-grained texts. Then, it extracts key character information from the fine-grained texts related to the search terms to extract characters related to the search terms, and finally determines the text corresponding to the search terms based on the extracted characters.

[0069] In other words, the text retrieval device first filters longer texts associated with the search term from a relatively long range of texts in a coarse-grained dimension. Then, based on this filtering, it further filters shorter texts associated with the search term from the longer texts obtained in the previous filtering in a fine-grained dimension. Finally, based on this filtering, it further filters characters associated with the search term from the shorter texts obtained in the previous filtering in an even finer-grained dimension, and combines these characters to determine the text corresponding to the search term.

[0070] Before explaining the implementation principle of the text retrieval method disclosed herein, an exemplary description of the application scenarios of the text retrieval method disclosed herein will be provided to deepen the reader's understanding of the text retrieval method disclosed herein.

[0071] Based on the above examples, texts can be categorized into long texts and short texts based on their length. The text retrieval method disclosed herein can be applied to both long and short text retrieval scenarios. Furthermore, as can be seen from the technical concept of this disclosure, it employs a multi-granularity text splitting and multi-dimensional filtering approach to achieve text retrieval. Therefore, when the text retrieval method of this disclosure is applied to long text retrieval scenarios, its accuracy and effectiveness are more evident.

[0072] In terms of text content, the text retrieval method disclosed herein can be applied to retrieval scenarios for texts such as news, work reports, academic papers, and books.

[0073] Figure 1 This is a schematic diagram illustrating an application scenario of the text retrieval method according to an embodiment of the present disclosure. The text retrieval method of the present disclosure can be applied to, for example... Figure 1 The text retrieval system 100 shown is an example. Figure 1 As shown, the text retrieval system 100 may include a target user 101, a client 102, a server 103, and a network 104.

[0074] The target user 101 can be the user who triggers the text search for the search terms, and the target user 101 can perform the text search operation on the client 102.

[0075] Client 102 can be a device for performing text retrieval of search terms in response to a text retrieval operation by target user 102. That is, the text retrieval method can be executed on client 102. At this time, client 102 may store data or instructions for performing the text retrieval described in this specification, and may execute or be used to execute the data or instructions. In some embodiments, client 102 may include a hardware device with data information processing capabilities and the necessary programs required to drive the hardware device to work.

[0076] like Figure 1 As shown, client 102 can establish a communication connection with server 103. Server 103 can communicate with one client 102 or multiple clients 102. In some embodiments, client 102 can interact with server 103 via network 104 to receive or send messages, etc.

[0077] In some embodiments, client 102 may include a mobile device, tablet, laptop, built-in device in a motor vehicle, or similar content, or any combination thereof. In some embodiments, mobile devices may include smart home devices, smart mobile devices, virtual reality devices, augmented reality devices, or similar devices, or any combination thereof. In some embodiments, smart home devices may include smart TVs, desktop computers, etc., or any combination thereof. In some embodiments, smart mobile devices may include smartphones, personal digital assistants, gaming devices, navigation devices, etc., or any combination thereof. In some embodiments, built-in devices in a motor vehicle may include in-vehicle computers, in-vehicle televisions, etc. In some embodiments, client 102 may include a text collection device for collecting search terms.

[0078] In some embodiments, client 102 may have one or more applications (APPs) installed. The APP provides target user 101 with the ability and interface to interact with the outside world via network 104. APPs include, but are not limited to: web browser APPs, search APPs, chat APPs, shopping APPs, video APPs, financial management APPs, instant messaging tools, email clients, social media platform software, etc. In some embodiments, target APP may be installed on client 102. Target APP can collect search terms for client 102.

[0079] Server 103 can be a server that provides various services, such as a backend server that supports the collection of user data sets and account login information corresponding to multiple accounts collected on client 102, and provides support for text retrieval of multiple accounts.

[0080] In some embodiments, the text retrieval method can be executed on server 103. In this case, server 103 may store data or instructions for executing the text retrieval method described herein, and may execute or be used to execute the data or instructions.

[0081] In some embodiments, server 103 may include a hardware device with data processing capabilities and the necessary programs to drive the hardware device. Similarly, server 103 may communicate with one client 103 and receive data sent by that client 103, or it may communicate with multiple clients 103 and receive data sent by each client 103.

[0082] Network 104 is a medium used to provide a communication connection between client 102 and server 103. Network 104 can facilitate the exchange of information or data. Figure 1 As shown, client 102 and server 103 can connect to network 104 respectively and transmit information or data to each other through network 104.

[0083] In some embodiments, network 104 can be any type of wired or wireless network, or a combination thereof. For example, network 104 may include a cable network, a wired network, a fiber optic network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a public switched telephone network (PSTN), a Bluetooth network™, a ZigBee™ short-range wireless network, a near field communication (NFC) network, or a similar network.

[0084] In some embodiments, network 104 may include one or more network access points. For example, network 104 may include wired or wireless network access points, such as base stations or internet switching points, through which one or more components of client 102 and server 103 can connect to network 104 to exchange data or information.

[0085] It should be understood that, Figure 1 The number of clients 102, servers 103, and networks 104 shown is merely illustrative. Depending on implementation needs, there can be any number of clients 102, servers 103, and networks 104. Furthermore, the text retrieval method provided in this disclosure can be executed entirely on client 102, entirely on server 103, or partially on client 102 and partially on server 103.

[0086] In other words, Figure 1 and targeting Figure 1The above description is only used to illustrate the possible application scenarios for the text retrieval method disclosed herein, and should not be construed as limiting the application scenarios.

[0087] Please see Figure 2 , Figure 2 This is a schematic diagram of a text retrieval method according to an embodiment of this disclosure. Figure 2 As shown, the method includes:

[0088] S201: Obtain a search request. The search request is used to instruct the retrieval of text corresponding to the search terms from the text to be searched.

[0089] For example, the execution subject of this embodiment can be a text retrieval device, which can be a server, a terminal device, a processor, a chip, etc., and will not be listed here.

[0090] If the text retrieval device is a server, it can be a standalone server or a cluster server; it can be a cloud server or a local server, and this embodiment does not impose any limitations.

[0091] For example, in combination Figure 1 In the application scenarios shown, the text retrieval device can be a client, a server, or both.

[0092] This embodiment does not limit the method of obtaining the search request, for example:

[0093] In one example, the text retrieval device can connect to other devices and receive retrieval requests sent by those other devices.

[0094] For example, with Figure 1 Taking the application scenario shown as an example, a text retrieval device can be used for, for example... Figure 1 The server shown can be other devices such as Figure 1 As shown in the client, users can enter search terms on the client via an app or a non-app to trigger the client to send a search request to the server.

[0095] In another example, the text retrieval device can provide a tool for loading search terms, which users can use to transfer search terms to the text retrieval device.

[0096] The tool for loading search terms can be an interface for connecting to external devices, such as an interface for connecting to other storage devices, through which search terms transmitted by external devices are obtained; the tool for loading search terms can also be a display device, such as a text retrieval device, which can have an interface for inputting search terms on the display device, through which users can import search terms into the text retrieval device, and the text retrieval device obtains the imported search terms.

[0097] S202: Separate the text to be retrieved based on paragraphs, sentences, and a preset first character count to obtain each coarse-grained text, and extract the target coarse-grained text corresponding to the search terms from each coarse-grained text.

[0098] The preset first character count can be determined by the text retrieval device based on needs, historical records, and experiments, and this embodiment does not impose any limitations. For example, considering that the target coarse-grained text can be determined by a neural network model, the neural network model may include an encoder, and the maximum text length required by the encoder is consistent, such as a maximum text length of 127 characters or words. In order to avoid truncation due to the first preset character count exceeding 127 characters or words, the preset first character count can be 127 characters or words.

[0099] The splitting in this step can be understood as coarse-grained splitting, that is, the retrieval device splits the retrieval text from the dimension of a relatively large number of words.

[0100] In this step, "corresponding" can be understood as "related" or "associated." For example, the target coarse-grained text refers to the text in each coarse-grained text that is related to or associated with the search term. Alternatively, combining the above example, the target coarse-grained text can be understood as the explanatory information of the search term in each coarse-grained text.

[0101] Correspondingly, this step can be understood as follows: the text retrieval device first splits the text to be retrieved into coarse-grained text. For ease of distinction, we can call the text obtained from the splitting coarse-grained text. Then, the text retrieval device filters from each coarse-grained text to obtain the text of the explanation information of the search terms. Similarly, for ease of distinction, we can call the text of the explanation information of the search terms obtained from the filtering coarse-grained text the target coarse-grained text.

[0102] This embodiment does not limit the method by which the text retrieval device obtains the target coarse-grained text. For example, it can be obtained through text matching or through a network model.

[0103] S203: Segment the target coarse-grained text based on punctuation marks to obtain fine-grained text, and extract the target fine-grained text corresponding to the search term from each fine-grained text.

[0104] In contrast, text retrieval devices split text into coarse-grained segments based on a preset first number of characters, and text retrieval devices split text into fine-grained segments based on punctuation marks. That is, the fine-grained segmentation performed by the document retrieval device is based on the coarse-grained segmentation. For example, if the document retrieval device splits the text to be retrieved into coarse-grained texts at the sentence level, it can further split each coarse-grained text based on punctuation marks.

[0105] Similarly, the "correspondence" in this step can be understood as "related" or "associated." For example, the target fine-grained text is the text in each fine-grained text that is related to or associated with the search term. Alternatively, combining the above examples, the target fine-grained text can be understood as the explanatory information of the search term in each fine-grained text.

[0106] Correspondingly, this step can be understood as follows: Based on the coarse-grained target text obtained by the text retrieval device through screening, the text retrieval device first splits the target coarse-grained text into fine-grained text. For ease of distinction, we can call the text obtained from the splitting fine-grained text. Then, the text retrieval device filters from each fine-grained text to obtain the text of the explanation information of the search terms. Similarly, for ease of distinction, we can call the text of the explanation information of the search terms obtained from the screening fine-grained text the target fine-grained text.

[0107] Similarly, this embodiment does not limit the method by which the text retrieval device obtains the target fine-grained text. For example, it can be obtained through text matching or through a network model.

[0108] S204: Extract key character information corresponding to the search terms from the target fine-grained text, and determine the text corresponding to the search terms based on the key character information.

[0109] Similarly, the "correspondence" in this step can be understood as "related" or "associated." For example, key character information refers to text in the target fine-grained text that is related to or associated with the search term. Alternatively, combining the above examples, key character information can be understood as explanatory information about the search term in the target fine-grained text.

[0110] Correspondingly, this step can be understood as follows: the text retrieval device performs finer-grained filtering on the target fine-grained text to obtain information on the character dimension (or word dimension) of the target fine-grained text that is related to or associated with the search term (i.e., key character information), so as to determine the text corresponding to the search term based on the information on the character dimension.

[0111] Similarly, this embodiment does not limit the method by which the text retrieval device obtains key character information. For example, it can obtain it through text matching or through a network model.

[0112] This embodiment does not limit the method of determining the text corresponding to the search term based on the key character information. For example, it can be determined by combination, and the combination method can be concatenation or insertion.

[0113] Based on the above analysis, this disclosure provides a text retrieval method, including: obtaining a retrieval request, the retrieval request being used to instruct the retrieval of text corresponding to a search term from the text to be retrieved; sequentially splitting the text to be retrieved based on paragraphs, sentences, and a preset first character count to obtain coarse-grained texts; obtaining target coarse-grained texts corresponding to the search term from each coarse-grained text; splitting the target coarse-grained texts based on punctuation marks to obtain fine-grained texts; obtaining target fine-grained texts corresponding to the search term from each fine-grained text; extracting key character information corresponding to the search term from the target fine-grained text; and determining the text corresponding to the search term based on the key character information. In this embodiment, the text retrieval device first splits the text at the coarse-grained level to obtain coarse-grained texts, then filters out target coarse-grained texts corresponding to the search term from each coarse-grained text; then splits the text at the fine-grained level to obtain fine-grained texts, then filters out target fine-grained texts corresponding to the search term from each fine-grained text; and finally filters out characters corresponding to the search term from the target fine-grained text based on the character dimension. By using technical features to determine the text corresponding to search terms based on the selected characters, the following advantages can be achieved: First, it avoids the low accuracy drawbacks caused by engineering methods in the above examples, thus improving the accuracy and reliability of text retrieval. Second, it avoids the narrow applicability of algorithmic techniques in the above examples, thus increasing the scope of application and diversity of text retrieval. Third, the text retrieval device, based on the "coarse-grained + fine-grained + even finer-grained" splitting and filtering method analyzed above, can enhance the refinement of the text to be retrieved and achieve comprehensiveness in the splitting of the text to be retrieved, thereby making the filtering more targeted and effective, and thus making the text retrieval more reliable and accurate. Fourth, the text retrieval device, based on the "coarse-grained + fine-grained + even finer-grained" splitting and filtering method analyzed above, ensures that the retrieved text corresponding to the search terms has linguistic relevance not only at the contextual semantic level but also at the character level, thereby improving the accuracy and reliability of text retrieval.

[0114] To help readers gain a deeper understanding of the implementation principles of this disclosure, the following is combined with... Figure 3 The text retrieval method disclosed herein is described in detail. Among other things, Figure 3 This is a schematic diagram of a text retrieval method according to another embodiment of the present disclosure, such as... Figure 3 As shown, the method includes:

[0115] S301: Obtain a search request. The search request is used to instruct the retrieval of text corresponding to the search term from the text to be searched.

[0116] It should be understood that, in order to avoid tedious descriptions, the technical features that are the same as or similar to those in the above embodiments will not be repeated in this embodiment.

[0117] For example, the implementation principle of S201 can be referred to for a description of S301, which will not be repeated here.

[0118] S302: Split the text to be retrieved based on paragraphs to obtain paragraph text.

[0119] For example, if the text to be retrieved is an academic paper, the text retrieval device can segment the academic paper according to its paragraphs or line breaks to divide the academic paper into paragraph texts corresponding to each paragraph.

[0120] The text retrieval device can store paragraph text as a list, which can be called a paragraph text list, and the paragraph text list includes paragraph text.

[0121] S303: Split the paragraph text based on sentences to obtain the complete sentence text.

[0122] For example, a text retrieval device can segment paragraph text into whole sentences based on periods, exclamation marks, and question marks to obtain whole sentence text.

[0123] Similarly, a text retrieval device can store entire sentences of text as a list, and this list can be called a list of entire sentences of text, which includes entire sentences of text.

[0124] S304: Based on punctuation marks, split the text in the whole sentence where the number of characters is greater than the preset first number of characters to obtain each coarse-grained text.

[0125] For example, a text retrieval device can first determine texts with a word count greater than a preset first word count from a list of whole sentences, and then segment the determined texts according to the punctuation marks in the determined texts to obtain each coarse-grained text.

[0126] In some embodiments, for texts in the whole sentence text list whose word count is less than or equal to a preset first word count, the text retrieval device may not segment them and may identify them as coarse-grained text.

[0127] Similarly, a text retrieval device can store each coarse-grained text as a list, and this list can be called a coarse-grained text list, which includes each coarse-grained text.

[0128] In this embodiment, the text retrieval device first splits the text by paragraph, then by sentence, and finally by punctuation. Through this progressive splitting process, the effectiveness and reliability of the splitting can be achieved.

[0129] In some embodiments, S304 may include the following steps:

[0130] First step: Identify long sentences from the entire text that have more than the preset first character count.

[0131] For example, there are multiple whole sentences, with a preset first character count of 127 characters. For each whole sentence, the text retrieval device determines the character count of the whole sentence and compares the character count with 127 to determine the size of the two. The whole sentence with a character count greater than 127 is called a long sentence.

[0132] Similarly, a text retrieval device can store long sentence texts as a list, and this list can be called a long sentence text list, which includes long sentence texts.

[0133] The second step is to traverse from the end character of the long sentence text to the beginning character of the long sentence text. If the current character is a punctuation mark and the length of the traversed sentence text between the current character and the current starting character is less than the preset first number, then the traversed sentence text is determined as the text in each coarse-grained text.

[0134] The "start" and "end" in this step are determined based on the habit of reading long sentences.

[0135] For example, the text retrieval device fixes the starting character of a long sentence text, such as fixing the first character of the long sentence text, and traverses from the last character of the long sentence text to the beginning, such as traversing from the last character of the long sentence text to the first character of the long sentence text, one character at a time. If the current character encountered during traversal (i.e., the current character) is a punctuation mark, the text retrieval device calculates the number of characters between the current character and the starting character of the current traversal (i.e., the current starting character) (i.e., the length of the traversed sentence text). If the number of characters is less than 127 characters, the traversed sentence text is determined as the text in each coarse-grained text.

[0136] In other embodiments, the text can be traversed from the beginning character to the end character, with the same principle as the examples above, and will not be repeated here.

[0137] In this embodiment, the text retrieval device first selects long sentences with a large number of characters from the whole text, and then traverses the long sentences to split them into coarse-grained texts with fewer characters. This makes each coarse-grained text relatively short, i.e., with a relatively small number of characters, thereby achieving the reliability and effectiveness of coarse-grained splitting.

[0138] In some embodiments, S304 may further include the following steps:

[0139] The third step: If the length of the traversed sentence text reaches the preset first character count, then determine the character splitting interval based on the preset second character count, the number of characters in the traversed sentence text, and the preset first character count. The character splitting interval is less than the preset first character count.

[0140] The above example illustrates the concept by traversing a sentence whose length is less than a preset first character count. However, in a possible example, the length of the traversed sentence text may be greater than or equal to the first character count, such as when the traversed sentence text contains 300 characters. In this case, the text retrieval device can further split the traversed sentence text.

[0141] For example, in conjunction with the above example, if the length of the last sentence text obtained through traversal is greater than or equal to 127 characters, the text retrieval device will further split the sentence text so that the length of the split text is less than 127 characters.

[0142] Similarly, the second character count can be determined based on requirements, historical records, and experiments, and this embodiment does not impose any limitations. For example, the preset second character count is 250 characters.

[0143] In some embodiments, the word splitting interval L can be calculated based on Equation 1, Equation 1:

[0144] L=ceil(len(sentence) / (len(sentence) / 127))

[0145] Where len(sentence) is the length of the traversed sentence text, 127 is the preset first character count, and ceil is used to represent the rounding function.

[0146] Step 4: Split the traversed sentence text according to the word splitting interval and the preset second word count to obtain the coarse-grained text of the traversed sentence text in each coarse-grained text.

[0147] In this embodiment, when the number of characters in the traversed sentence text is greater than or equal to a preset first number of characters, the text retrieval device can first determine the character splitting interval, and then further split the traversed sentence text with a relatively large number of characters based on the character splitting interval, which can further improve the effectiveness and reliability of coarse-grained splitting.

[0148] In some embodiments, the fourth step may include the following two cases:

[0149] The first scenario: If the number of characters in the traversed sentence text reaches the preset second character count, then the traversed sentence text is split according to the character splitting interval to obtain the coarse-grained text of the traversed sentence text in each coarse-grained text.

[0150] For example, if the number of characters in the second sentence is 250, and the number of characters in the traversed sentence text is greater than or equal to 250, the text retrieval device will split the traversed sentence text with the number of characters split as the splitting granularity.

[0151] For example, a text retrieval device breaks the traversed sentence text every L characters to obtain the corresponding coarse-grained text, until the traversed sentence text is completely split.

[0152] The second scenario: If the number of characters in the traversed sentence text is less than the preset second number of characters, then the traversed sentence text is split based on the position of half the number of characters in the traversed sentence text, resulting in coarse-grained texts of the traversed sentence texts in each coarse-grained text.

[0153] Similarly, taking a preset second character count of 250 characters as an example, if the number of characters in the traversed sentence text is less than 250 characters, the text retrieval device will split the traversed sentence text into two coarse-grained texts with the same number of characters. If the number of characters in the traversed sentence text is odd, then the number of characters in any one of the two coarse-grained texts split by the text retrieval device can be one more than the number of characters in the other text.

[0154] In other embodiments, if the length of the traversed sentence text is greater than a preset first number of characters, the text retrieval device may first determine the length of the traversed sentence text. If the length of the traversed sentence text reaches a preset second number of characters, the text retrieval device may split the traversed sentence text into two coarse-grained texts with the same number of characters (if the number of characters is odd, refer to the processing method in the example above). If the length of the traversed sentence text is less than the preset second number of characters, the text retrieval device may calculate the split character interval and split the traversed sentence text based on the split character interval to obtain the corresponding coarse-grained text.

[0155] In this embodiment, the text retrieval device uses different splitting methods to split the traversed sentence text according to the number of characters, which can achieve flexibility and diversity in splitting. When the number of characters in the traversed sentence text is relatively small but greater than a preset second number of characters, the text retrieval device splits the traversed sentence text into two parts of text with approximately the same number of characters, which can save splitting resources. When the number of characters in the traversed sentence text is greater than the preset second number of characters and is relatively large, the text retrieval device splits the traversed sentence text based on the calculated splitting character interval, which can improve the effectiveness and reliability of splitting.

[0156] S305: Input each coarse-grained text and the search term into a pre-trained coarse-grained text relevance classifier, and output the first classification result corresponding to each coarse-grained text. The first classification result of each coarse-grained text indicates whether the coarse-grained text is related to the search term.

[0157] The coarse text relevance classifier is trained based on a sample dataset, which includes sample text, sample search terms, and ground truth values ​​of search answers corresponding to the sample search terms in the sample text.

[0158] For example, such as Figure 4 ( Figure 4 As shown in the schematic diagram of the text retrieval method disclosed herein, the text retrieval device performs coarse-grained segmentation of the text to be retrieved to obtain each coarse-grained text. Then, each coarse-grained text and the search term are input into the coarse-segmented text relevance classifier, so that the coarse-segmented text relevance classifier can classify each coarse-grained text in each coarse-grained text to predict whether each coarse-grained text is related to the search term, and obtain the first classification result corresponding to each coarse-grained text.

[0159] In this embodiment, the type (which can be a binary classifier), architecture, parameters, etc. of the coarse text relevance classifier are not limited, and can be determined by the text retrieval device based on requirements, historical records, and experiments.

[0160] Taking a binary classifier as an example, the entity that trains the coarse text relevance classifier can be a text retrieval device or other devices. If it is another device, the text retrieval device communicates with other devices. When the other device trains the coarse text relevance classifier, the text detection device obtains the coarse text relevance classifier from the other device based on the communication connection.

[0161] For example, combining Figure 1 As shown in the application scenarios, the text retrieval device can be used for, for example... Figure 1 The client shown can be other devices such as Figure 1The server shown allows for network connectivity between the client and server. Once the server has trained a coarse-grained text relevance classifier, it can be sent to the client via the network.

[0162] Accordingly, taking the server as the main body for training the coarse-grained text relevance classifier as an example, this embodiment does not limit the implementation method of the server training the coarse-grained text relevance classifier. For example, the server can split the sample text based on the coarse-grained splitting method described above. For ease of distinction, we can call the split text the sample coarse-grained text.

[0163] The server inputs coarse-grained sample text and sample search terms into a pre-defined network model (such as a binary classifier), outputs a predicted classification result, and the predicted classification result indicates whether there is text related to the sample search terms in the coarse-grained sample text. The server calculates a loss function based on the predicted classification result and the true value of the search answer, and iteratively optimizes the pre-defined network model based on the loss function to train the pre-defined network model to predict whether the sample coarse-grained text includes text related to the sample search terms, thereby obtaining a coarse text relevance classifier with the ability to determine text related to search terms from coarse-grained text.

[0164] In some embodiments, combined with Figure 5 ( Figure 5 As shown in the schematic diagram illustrating the principle of training a coarse-grained text relevance classifier according to an embodiment of this disclosure, the preset network model includes: two encoders, a self-attention layer, a query-attention layer, a concat layer, and a deep neural network (DNN) layer. For ease of distinction, we will refer to one of the encoders as the first encoder and the other as the second encoder.

[0165] like Figure 5 As shown, the input to the first encoder is the sample coarse-grained text, and the output of the first encoder is connected to the input of the self-attention mechanism layer and the input of the query text attention layer, respectively. The input to the second encoder is the sample search term, and the output of the second encoder is connected to the input of the query text attention layer. The outputs of the self-attention mechanism layer and the query text attention layer are connected to the input of the connection layer, respectively. The output of the connection layer is connected to the input of the deep neural network. The output of the deep neural network includes the predicted classification result, such as the binary classification label value related to the sample coarse-grained text and the sample search term, or the binary classification label value related to the sample coarse-grained text and the sample search term.

[0166] In some embodiments, the coarse-grained text relevance classifier includes a first encoder and a second encoder. The first classification result corresponding to each coarse-grained text is obtained by encoding each coarse-grained text by the first encoder and encoding the search terms by the second encoder.

[0167] For example, the coarse-grained text relevance classifier includes two encoders. For ease of distinction, we can refer to one encoder as the first encoder and the other as the second encoder. The text retrieval device can input each coarse-grained text into the first encoder for encoding, and input the search terms into the second encoder for encoding. The specific encoding logic can be found in related technologies, and this embodiment is not limited thereto.

[0168] For example, combining Figure 5 As shown, various coarse-grained text inputs can be submitted to a program like this. Figure 5 The first encoder shown inputs the search terms into, as follows: Figure 5 The second encoder is shown.

[0169] Correspondingly, the coarse-grained text relevance classifier makes predictions based on the encoded features obtained by the two encoders, and obtains the first classification result corresponding to each coarse-grained text.

[0170] In this embodiment, the text retrieval device combines two encoders in the coarse-grained text relevance classifier to encode each coarse-grained text and search term respectively, so as to determine each first classification result based on their respective encoding results. Parallel encoding can avoid encoding interference, thereby improving the effectiveness and reliability of each first classification result.

[0171] S306: Determine the target coarse-grained text based on the first classification result.

[0172] For example, if the first classification result of a coarse-grained text indicates that the coarse-grained text is related to the search term, the text retrieval device can identify the coarse-grained text as the target coarse-grained text; conversely, if the first classification result of a coarse-grained text indicates that the coarse-grained text is not related to the search term, the text retrieval device may not identify the coarse-grained text as the target coarse-grained text.

[0173] Similarly, a text retrieval device can store the target coarse-grained text as a list, and this list can be called the target coarse-grained text list, which includes the target coarse-grained text.

[0174] In this embodiment, the first classification result is determined by combining a network model, so that the target coarse-grained text can be determined based on the first classification result, which can improve the efficiency and reliability of determining the target coarse-grained text.

[0175] S307: Segment the target coarse-grained text based on punctuation marks to obtain fine-grained text.

[0176] For example, regarding the description of S307, please refer to part of the implementation principle in S203, which will not be repeated here.

[0177] In some embodiments, such as Figure 4 As shown, when the text retrieval device has determined the target coarse-grained text, it can further perform fine-grained segmentation of the target coarse-grained text based on a preset third character count and punctuation marks, thereby obtaining each fine-grained text.

[0178] Similarly, a text retrieval device can store each fine-grained text as a list, and this list can be called a fine-grained text list, which includes each fine-grained text.

[0179] In some embodiments, each fine-grained text includes: target coarse-grained text with a word count less than a preset third word count, and text obtained by splitting target coarse-grained text with a word count reaching a preset third word count based on punctuation marks.

[0180] For example, taking a third character of 3 characters as an example, the text retrieval device uses 3 characters as the dividing standard to divide the target coarse-grained text into target coarse-grained text with more than 3 characters and target coarse-grained text with less than 3 characters.

[0181] For target coarse-grained text with fewer than 3 characters, the text retrieval device can classify it as fine-grained text. For target coarse-grained text with 3 or more characters, the text retrieval device can further break it down to obtain fine-grained text. This further breakdown can be performed based on punctuation marks.

[0182] For example, for a target coarse-grained text with three or more characters, the text retrieval device can traverse from the beginning to the end of the sentence. If the current character being traversed is a punctuation mark such as a comma, semicolon, colon, parentheses, question mark, exclamation mark, period, or quotation mark, the text retrieval device will segment the text between the current character and the starting character of the current traversal into fine-grained text, until the target fine-grained text has been segmented in the same way. Consecutive punctuation marks can be considered as a single punctuation mark.

[0183] In this embodiment, for target granular text with a relatively small number of words, the text retrieval device identifies it as fine-grained text. For target granular text with a relatively large number of words, the text retrieval device performs fine-grained segmentation to obtain the corresponding fine-grained text, which can improve the diversity and flexibility of segmentation.

[0184] S308: Input each fine-grained text and the search term into a pre-trained fine-grained text relevance classifier, and output the second classification result corresponding to each fine-grained text. The second classification result of each fine-grained text represents whether the fine-grained text is related to the search term.

[0185] Among them, the fine-grained text relevance classifier is trained based on a sample dataset, which includes sample text, sample search terms, and the ground truth values ​​of the search answers corresponding to the sample search terms in the sample text.

[0186] For example, such as Figure 4 As shown, the text retrieval device performs coarse-grained splitting of the target coarse-grained text to obtain fine-grained texts. Then, each fine-grained text and the search term are input into the fine-grained text relevance classifier so that the fine-grained text relevance classifier can classify each fine-grained text in each fine-grained text to predict whether each fine-grained text is related to the search term, and obtain the second classification result corresponding to each fine-grained and coarse-grained text.

[0187] Similarly, this embodiment does not limit the type (which can be a binary classifier), architecture, parameters, etc. of the fine-grained text relevance classifier, and can determine them based on requirements, historical records, and experiments.

[0188] Taking a binary classifier as an example, the entity that trains the fine-segment text relevance classifier can be a text retrieval device or other devices. If it is another device, the text retrieval device communicates with other devices. When the fine-segment text relevance classifier is trained by other devices, the text detection device obtains the fine-segment text relevance classifier from other devices based on the communication connection.

[0189] For example, combining Figure 1 As shown in the application scenarios, the text retrieval device can be used for, for example... Figure 1 The client shown can be other devices such as Figure 1 The server shown allows for network connectivity between the client and server. Once the server has trained a fine-grained text relevance classifier, this classifier can be sent to the client via the network.

[0190] Accordingly, taking the server as the execution entity for training the fine-grained text relevance classifier as an example, this embodiment does not limit the implementation method of the server training the fine-grained text relevance classifier. For example, the server can split the target sample coarse-grained text that is related to the sample search terms in the sample coarse-grained text based on the fine-grained splitting method described above. For easy distinction, we can call the split text the sample fine-grained text.

[0191] The server inputs fine-grained sample text and sample search terms into a pre-defined network model (such as a binary classifier), outputs a predicted classification result, and the predicted classification result indicates whether there is text in the fine-grained sample text that is related to the sample search terms. The server calculates a loss function based on the predicted classification result and the true value of the search answer, and iteratively optimizes the pre-defined network model based on the loss function to train the pre-defined network model to predict whether the fine-grained sample text includes text related to the sample search terms. This results in a fine-grained text relevance classifier that has the ability to determine text related to search terms from fine-grained text.

[0192] For example, the structure of a fine-grained text relevance classifier can be seen as follows: Figure 5 The coarse text relevance classifier shown here will not be described in detail here.

[0193] Accordingly, in some embodiments, the fine-grained text relevance classifier includes a third encoder and a fourth encoder, and the second classification result corresponding to each fine-grained text is obtained by encoding the fine-grained text by the third encoder and encoding the search term by the fourth encoder.

[0194] For example, the fine-grained text relevance classifier includes two encoders. For ease of distinction, we can refer to one encoder as the third encoder and the other as the fourth encoder. The text retrieval device can input each fine-grained text into the third encoder for encoding, and input the search terms into the fourth encoder for encoding. The specific encoding logic can be found in related technologies, and this embodiment is not limited thereto.

[0195] Similarly, in this embodiment, the text retrieval device combines two encoders in the fine-grained text relevance classifier to encode each fine-grained text and search term respectively, so as to determine each second classification result based on their respective encoding results. Parallel encoding can avoid encoding interference, thereby improving the effectiveness and reliability of each second classification result.

[0196] S309: Determine the target fine-grained text based on the second classification result.

[0197] For example, if the second classification result of a fine-grained text indicates that the fine-grained text is related to the search term, the text retrieval device can identify the fine-grained text as the target fine-grained text; conversely, if the second classification result of a fine-grained text indicates that the fine-grained text is not related to the search term, the text retrieval device may not identify the fine-grained text as the target fine-grained text.

[0198] Similarly, a text retrieval device can store target fine-grained text as a list, and this list can be called a target fine-grained text list, which includes the target fine-grained text.

[0199] In this embodiment, the first classification result is determined by combining a network model, so that the target coarse-grained text can be determined based on the second classification result, which can improve the efficiency and reliability of determining the target fine-grained text.

[0200] S310: Input the target fine-grained text and the search term into the pre-trained key text extraction model, and output the third classification result corresponding to each character in the target fine-grained text. The third classification result corresponding to each character in the target fine-grained text indicates whether the character is related to the search term.

[0201] The key text extraction model is trained on a sample dataset, which includes sample text, sample search terms, and the ground truth values ​​of the search answers corresponding to the sample search terms in the sample text.

[0202] For example, such as Figure 4 As shown, the text retrieval device inputs the target fine-grained text and search terms into the key text extraction model, so that the key text extraction model can extract the characters (i.e., key characters) related to the search terms in the target fine-grained text, thereby obtaining the third classification result.

[0203] Similarly, this embodiment does not limit the type (which can be a binary classifier), architecture, parameters, etc. of the key text extraction model, and can determine them based on requirements, historical records, and experiments. The entity that performs the training of the key text extraction model can be a text retrieval device or other devices. If it is another device, the text retrieval device communicates with other devices. When the other device trains the key text extraction model, the text detection device obtains the key text extraction model from the other device based on the communication connection.

[0204] For example, combining Figure 1 As shown in the application scenarios, the text retrieval device can be used for, for example... Figure 1 The client shown can be other devices such as Figure 1 The server shown allows for network connectivity between the client and server. Once the server has trained a key text extraction model, this model can be sent to the client over the network.

[0205] Accordingly, taking the server as the execution entity for training the key text extraction model as an example, this embodiment does not limit the implementation method of the server training the key text extraction model. For example, the server can determine the target sample fine-grained text in the sample fine-grained text based on the above-described method.

[0206] The server inputs the fine-grained text of the target sample and the sample search terms into a pre-defined network model, and outputs a prediction classification result. The prediction classification result indicates whether there is text related to the sample search terms in the fine-grained text of the target sample. The server calculates a loss function based on the prediction classification result and the true value of the search answer, and iteratively optimizes the pre-defined network model based on the loss function to train the pre-defined network model to predict whether the fine-grained text of the target sample includes text related to the sample search terms. This results in a key text extraction model with the ability to determine text related to search terms from fine-grained text.

[0207] In some embodiments, combined with Figure 6 ( Figure 6 As shown in the schematic diagram illustrating the principle of training the key text extraction model in this embodiment, the preset network model includes: two encoders, three self-attention mechanism layers, two trainable weight layers, three bidirectional long short-term memory artificial neural network (LSTM) layers, three multiply fusion network layers, a difference layer, and a dense fully connected neural network layer. For ease of distinction, as... Figure 6 As shown, the two encoders include a fifth encoder and a sixth encoder; the three self-attention mechanism layers include a first self-attention mechanism layer, a second self-attention mechanism layer, and a third self-attention mechanism layer; the two trainable weight layers include a first trainable weight layer and a second trainable weight layer; the three bidirectional long short-term memory artificial neural network layers include a first bidirectional long short-term memory artificial neural network layer, a second bidirectional long short-term memory artificial neural network layer, and a third bidirectional long short-term memory artificial neural network layer; and the three fusion network layers include a first fusion network layer, a second fusion network layer, and a third fusion network layer.

[0208] like Figure 6As shown, the input to the fifth encoder is the fine-grained text of the sample target, and the output of the fifth encoder is connected to the input of the first self-attention mechanism layer; the output of the first self-attention mechanism layer is connected to the first fusion network layer; the input to the sixth encoder is the sample search term, and the output of the sixth encoder is connected to the input of the first trainable weight layer; the output of the first trainable weight layer is connected to the input of the first bidirectional long short-term memory artificial neural network layer; the output of the first bidirectional long short-term memory artificial neural network layer is connected to the input of the second trainable weight layer; the output of the second trainable weight layer is connected to the inputs of the first fusion network layer, the second fusion network layer, the third fusion network layer, and the difference layer, respectively; the output of the first fusion network layer is connected to the input of the second self-attention mechanism layer; the second self-attention mechanism... The output of the first layer is connected to the input of the second fusion network; the output of the second fusion network is connected to the input of the third self-attention mechanism layer; the output of the third self-attention mechanism layer is connected to the input of the third fusion network; the output of the third fusion network is connected to the input of the second bidirectional long short-term memory artificial neural network layer; the output of the second bidirectional long short-term memory artificial neural network layer is connected to the input of the difference layer; the output of the difference layer is connected to the input of the third bidirectional long short-term memory artificial neural network layer; the output of the third bidirectional long short-term memory artificial neural network layer is connected to the input of the fully connected neural network layer; the output of the fully connected neural network includes the predicted classification results, such as the text sequence labeling results of the fine-grained text of the sample target, which are used to characterize whether each word or character in the fine-grained text of the target is related to the sample search term.

[0209] For example, the input to the trainable weight layer is an input matrix. If the input matrix is ​​X, then the output Y of the trainable weight layer can be represented by Equation 2: Equation 2: Y = k*X + b. Where k and b are preset sequential data matrices.

[0210] The input to the interpolation layer consists of two distinct input matrices. For example, the output of the second bidirectional long short-term memory artificial neural network layer can be called the first input matrix X1 of the interpolation layer, and the output of the second trainable weight layer can be called the second input matrix X2 of the interpolation layer. The operation performed by the interpolation layer can then be represented by Equation 3: Z = X1 - |X1 - X2|. Where Z is the output of the interpolation layer.

[0211] In some embodiments, the key text extraction model includes a fifth encoder and a sixth encoder. The third classification result corresponding to each character in the target fine-grained text is obtained by encoding the target fine-grained text based on the fifth encoder and encoding the search terms based on the sixth encoder.

[0212] For example, the key text extraction model includes two encoders. For ease of distinction, we can refer to one encoder as the fifth encoder and the other as the sixth encoder. The text retrieval device can input the target fine-grained text into the fifth encoder for encoding, and input the search terms into the sixth encoder for encoding. The specific encoding logic can be found in related technologies, and this embodiment is not limited thereto.

[0213] For example, combining Figure 6 As shown, fine-grained text can be input into a target system such as... Figure 6 The fifth encoder shown takes the search terms as input into... Figure 6 The sixth encoder shown.

[0214] Correspondingly, the key text extraction model makes predictions based on the encoded features obtained by the two encoders, and obtains a third classification result as to whether each character in the target fine-grained text is related to the search term.

[0215] In this embodiment, the text retrieval device combines two encoders in the key text extraction model to encode the target fine-grained text and search terms respectively, so as to determine each third classification result based on their respective encoding results. Parallel encoding can avoid encoding interference, thereby improving the effectiveness and reliability of each third classification result.

[0216] S311: Determine key character information based on the results of the third classification.

[0217] For example, if the third classification result of a certain character or word in the target fine-grained text indicates that the character or word is related to the search term, the text retrieval device can identify the character or word as a key character or keyword in the key character information; conversely, if the third classification result of a certain character or word in the target fine-grained text indicates that the character or word is not related to the search term, the text retrieval device may not identify the character or word as a key character or keyword in the key character information.

[0218] In this embodiment, the first classification result is determined by combining a network model, so that the target coarse-grained text can be determined based on the first classification result, which can improve the efficiency and reliability of determining the target coarse-grained text.

[0219] S312: Determine the text corresponding to the search term based on the key character information.

[0220] For example, regarding the description of S312, please refer to part of the implementation principle in S204, which will not be repeated here.

[0221] In some embodiments, if the key character information includes multiple key characters, then the text corresponding to the search term is the text obtained by concatenating multiple key characters.

[0222] For example, if the key character information includes multiple key characters (or multiple keywords), the text retrieval device can concatenate the multiple key characters (or multiple keywords) to determine the concatenated text as the text corresponding to the search term.

[0223] Combination Figure 4 As can be seen from the above analysis, in this embodiment, other devices or text retrieval devices can pre-construct a coarse-grained text relevance classifier, a fine-grained text relevance classifier, and a key text extraction model. This allows the text retrieval device to achieve end-to-end text retrieval when it receives a retrieval request, by combining these three models. This improves the efficiency and reliability of text retrieval. Especially when the text retrieval method in this embodiment is applied to long text retrieval scenarios, the text retrieval device, through "multi-level splitting (such as the coarse-grained and fine-grained splitting mentioned above) + layer-by-layer classification (such as the process of obtaining the first, second, and third classification results mentioned above)," overcomes the difficulty of NLP technology in handling long texts. It can perform information retrieval and extraction on long texts of any length, without being limited by length. Furthermore, compared to a single model that directly extracts from an entire text segment, it significantly improves the accuracy of information extraction.

[0224] The apparatus used to construct the coarse text relevance classifier, the fine text relevance classifier, and the key text extraction model can be the same apparatus or different apparatuses; the sample data used to construct the coarse text relevance classifier, the fine text relevance classifier, and the key text extraction model can be the same sample data as described in the above examples or different sample data, and this embodiment does not impose any limitations.

[0225] It is worth noting that the above examples are merely illustrative of possible implementations of the text retrieval method of this disclosure, and should not be construed as limiting the implementation of the text retrieval method of this disclosure. For example, based on the above technical concept, some of the technical features described above can be combined to obtain new embodiments; new technical features can be added to the above examples to obtain new embodiments; some technical features can be removed from the above examples to obtain new embodiments; some technical features in the above examples can be replaced with other technical features; some technical features and their order in the above examples can be adjusted to obtain new embodiments, and so on, which will not be listed here.

[0226] For example, consider replacing some of the technical features in the above example with other technical features:

[0227] Text retrieval devices can replace encoders such as the first encoder and the second encoder with bidirectional encoder representations from a transformer (BERT); they can also replace encoders such as the first encoder and the second encoder with a Roberta model, and so on.

[0228] Text retrieval devices can replace one or more bidirectional long short-term memory artificial neural network layers in a key text extraction model with a gated recurrent unit (GRU) structure.

[0229] When training a key text extraction model, a text retrieval device can use a binary classification loss function as the loss function; it can also use a binary cross-entropy loss function; or it can use the sum of the binary cross-entropy, the binary cross-entropy of the predicted results of relevant characters, the binary cross-entropy of the predicted value after rounding to the true value with a preset value (such as 0.5) as the boundary, and the difference between the total mean of the true value and the predicted value as the final loss value, etc.

[0230] For example, by adjusting some of the technical features and their order in the above example, a new embodiment can be obtained:

[0231] The text retrieval device can perform coarse-grained segmentation of the text to be retrieved, obtaining coarse-grained texts, and input each coarse-grained text and the search term into a coarse-segmented text relevance classifier, outputting a first classification result; the text retrieval device can perform fine-grained segmentation of each coarse-grained text, obtaining fine-grained texts, and input each fine-grained text and the search term into a fine-segmented text relevance classifier, outputting a second classification result; the text retrieval device can input each fine-grained text and the search term into a key text extraction model, outputting a third classification result; the text retrieval device determines the text corresponding to the search term based on the union of the first classification result, the second classification result, and the third classification result.

[0232] Alternatively, in other embodiments, the text retrieval device may combine this embodiment with some of the technical features in the above embodiments to obtain new embodiments.

[0233] Based on the above technical concept, this disclosure also provides a text retrieval device.

[0234] Please see Figure 7 , Figure 7 This is a schematic diagram of a text retrieval device according to an embodiment of the present disclosure, as shown below. Figure 7 As shown, the text retrieval device 700 includes:

[0235] The first acquisition unit 701 is used to acquire a retrieval request, which is used to instruct the retrieval of text corresponding to the search term from the text to be retrieved.

[0236] The first splitting unit 702 is used to split the text to be retrieved sequentially based on paragraphs, sentences, and a preset first number of characters to obtain various coarse-grained texts.

[0237] The second acquisition unit 703 is used to acquire the target coarse-grained text corresponding to the search term from each of the coarse-grained texts.

[0238] The second splitting unit 704 is used to split the target coarse-grained text based on preset punctuation marks to obtain fine-grained text.

[0239] The third acquisition unit 705 is used to acquire the target fine-grained text corresponding to the search term from each fine-grained text.

[0240] Extraction unit 706 is used to extract key character information corresponding to the search term from the target fine-grained text.

[0241] The determining unit 707 is used to determine the text corresponding to the search term based on the key character information.

[0242] In some embodiments, combined with Figure 7 It can be seen that the first splitting unit 702 includes:

[0243] The first splitting subunit 7021 is used to split the text to be retrieved based on paragraphs to obtain paragraph text.

[0244] The second splitting subunit 7022 is used to split the paragraph text based on sentences to obtain complete sentence text.

[0245] The third splitting subunit 7023 is used to split the text in the whole sentence with a number of characters greater than the preset first number of characters based on punctuation marks, so as to obtain the coarse-grained text.

[0246] In some embodiments, the third splitting subunit 7023 includes:

[0247] The first determining module is used to determine long sentences from the whole sentence text that have a word count greater than the preset first word count.

[0248] The traversal module is used to traverse from the end character of the long sentence text to the beginning character of the long sentence text. If the current character is a punctuation mark and the length of the traversed sentence text between the current character and the current starting character is less than the preset first number, then the traversed sentence text is determined as the text in each coarse-grained text.

[0249] In some embodiments, the third splitting subunit 7023 further includes:

[0250] The second determining module is used to determine the character splitting interval based on the preset second character count, the number of characters in the traversed sentence text, and the preset first character count if the length of the traversed sentence text reaches the preset first character count, wherein the character splitting interval is less than the preset first character count.

[0251] The splitting module is used to split the traversed sentence text according to the splitting character interval and the preset second character number to obtain the coarse-grained text of the traversed sentence text in each coarse-grained text.

[0252] In some embodiments, the splitting module is used to split the traversed sentence text according to the splitting character interval if the number of characters in the traversed sentence text reaches the preset second number of characters, so as to obtain the coarse-grained text of the traversed sentence text in each coarse-grained text.

[0253] The splitting module is used to split the traversed sentence text based on the position of half the number of characters in the traversed sentence text if the number of characters in the traversed sentence text is less than the preset second number of characters, so as to obtain the coarse-grained text of the traversed sentence text in each coarse-grained text.

[0254] In some embodiments, combined with Figure 7 It can be seen that the second acquisition unit 703 includes:

[0255] The first input subunit 7031 is used to input the coarse-grained texts and the search terms into a pre-trained coarse-grained text relevance classifier, and output the first classification result corresponding to each coarse-grained text. The first classification result of each coarse-grained text indicates whether the coarse-grained text is related to the search terms.

[0256] The first determining subunit 7032 is used to determine the target coarse-grained text based on the first classification result.

[0257] The coarse text relevance classifier is trained based on a sample dataset, which includes sample text, sample search terms, and ground truth values ​​of search answers in the sample text corresponding to the sample search terms.

[0258] In some embodiments, the coarse-grained text relevance classifier includes a first encoder and a second encoder, and the first classification result corresponding to each coarse-grained text is obtained by encoding the coarse-grained text by the first encoder and encoding the search term by the second encoder.

[0259] In some embodiments, each fine-grained text includes: the target coarse-grained text with a word count less than a preset third word count, and text obtained by splitting the target coarse-grained text with a word count reaching the preset third word count based on punctuation marks;

[0260] The preset first character count is greater than the preset third character count.

[0261] In some embodiments, combined with Figure 7 It can be seen that the third acquisition unit 705 includes:

[0262] The second input subunit 7051 is used to input the fine-grained texts and the search terms into a pre-trained fine-grained text relevance classifier, and output the second classification result corresponding to each fine-grained text. The second classification result of each fine-grained text represents whether the fine-grained text is related to the search terms.

[0263] The second determining subunit 7052 is used to determine the target fine-grained text based on the second classification result.

[0264] The fine-grained text relevance classifier is trained based on a sample dataset, which includes sample text, sample search terms, and ground truth values ​​of search answers corresponding to the sample search terms in the sample text.

[0265] In some embodiments, the fine-grained text relevance classifier includes a third encoder and a fourth encoder, and the second classification result corresponding to each fine-grained text is obtained by encoding the fine-grained text by the third encoder and encoding the search term by the fourth encoder.

[0266] In some embodiments, combined with Figure 7 It is understood that the extraction unit 706 includes:

[0267] The third input subunit 7061 is used to input the target fine-grained text and the search term into a pre-trained key text extraction model, and output the third classification result corresponding to each character in the target fine-grained text. The third classification result corresponding to each character in the target fine-grained text indicates whether the character is related to the search term.

[0268] The third determining subunit 7062 is used to determine the key character information based on the third classification result.

[0269] The key text extraction model is trained based on a sample dataset, which includes sample text, sample search terms, and the ground truth values ​​of the search answers corresponding to the sample search terms in the sample text.

[0270] In some embodiments, the key text extraction model includes a fifth encoder and a sixth encoder, and the third classification result corresponding to each character in the target fine-grained text is obtained by encoding the target fine-grained text based on the fifth encoder and encoding the search term based on the sixth encoder.

[0271] In some embodiments, if the key character information includes multiple key characters, then the text corresponding to the search term is the text obtained by concatenating the multiple key characters.

[0272] Based on the above technical concept, this disclosure also provides a processor-readable storage medium storing a computer program for causing the processor to execute the text retrieval method as described in any of the above embodiments.

[0273] Based on the above technical concept, this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the text retrieval method as described in any of the above embodiments.

[0274] Based on the above technical concept, this disclosure also provides a text retrieval system, including:

[0275] At least one memory, the memory including at least one set of instructions for pushing information;

[0276] At least one processor communicates with the at least one memory;

[0277] When the at least one processor executes the at least one set of instructions, the text retrieval method as described in any of the above embodiments is implemented.

[0278] Based on the above technical concept, this disclosure also provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0279] The memory stores computer-executed instructions;

[0280] The processor executes computer execution instructions stored in the memory to implement the text retrieval method as described in any of the above embodiments.

[0281] in, Figure 8This is a hardware structure diagram of an electronic device 800 according to an embodiment of this disclosure. The electronic device 800 can perform the text retrieval method as described in any of the above embodiments.

[0282] The text retrieval method of this disclosure embodiment is applied to, for example, Figure 1 Taking the application scenario shown as an example, the electronic device 800 is explained as follows:

[0283] When the text retrieval method described in any of the above embodiments is executed on client 102, electronic device 800 can be client 102. When the text retrieval method described in any of the above embodiments is executed on server 103, electronic device 800 can be server 103. When the method described in any of the above embodiments is partially executed on client 102 and partially executed on server 103, electronic device 800 can be both client 102 and server 103.

[0284] like Figure 8 As shown, the electronic device 800 may include at least one storage medium 801 and at least one processor 802. In some embodiments, the electronic device 800 may also include a communication port 803 and an internal communication bus 804. Additionally, the electronic device 800 may also include an input / output (I / O) component 805.

[0285] The internal communication bus 804 can connect different system components, including storage medium 801, processor 802, and communication port 803. I / O component 805 supports input / output between electronic device 800 and other components. Communication port 803 is used for data communication between electronic device 800 and the outside world; for example, communication port 803 can be used for data communication between electronic device 800 and network 104. Communication port 803 can be a wired communication port or a wireless communication port.

[0286] Storage medium 801 may include a data storage device. The data storage device may be a non-transitory storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 8011, a read-only memory (ROM) 8012, or a random access memory (RAM) 8013. Storage medium 801 also includes at least one instruction set stored in the data storage device. The instructions are computer program code, which may include programs, routines, objects, components, data structures, procedures, modules, etc., that execute the text retrieval methods provided in this specification.

[0287] At least one processor 802 can be communicatively connected to at least one storage medium 801 and a communication port 803 via an internal communication bus 804. The at least one processor 802 is used to execute at least one instruction set. When the electronic device 800 is running, the at least one processor 802 reads the at least one instruction set and, according to the instructions of the at least one instruction set, executes the text retrieval method provided in this specification. The processor 802 can execute all the steps included in the text retrieval method. Processor 802 may be in the form of one or more processors. In some embodiments, processor 802 may include one or more hardware processors, such as microcontrollers, microprocessors, reduced instruction set computers (RISC), application-specific integrated circuits (ASICs), application-specific instruction processors (ASIPs), central processing units (CPUs), graphics processing units (GPUs), physical processing units (PPUs), microcontroller units, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), advanced RISC machines (ARMs), programmable logic devices (PLDs), any circuitry or processor capable of performing one or more functions, or any combination thereof. For illustrative purposes only, only one processor 802 is described in this specification for electronic device 800. However, it should be noted that the electronic device 800 described in this specification may also include multiple processors. Therefore, the operation and / or method steps disclosed in this specification may be executed by one processor as described in this specification, or they may be executed jointly by multiple processors. For example, if the processor 802 of the electronic device 800 in this specification executes steps A and B, it should be understood that steps A and B may also be executed jointly or separately by two different processors 802 (e.g., the first processor executes step A, the second processor executes step B, or the first and second processors jointly execute steps A and B).

[0288] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, apparatus, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0289] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-executable instructions. These computer-executable instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0290] These processor-executable instructions may also be stored in a processor-readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the processor-readable memory produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0291] These processors can execute instructions that can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0292] Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. A text retrieval method, characterized in that, The method includes: Obtain a search request, which instructs the retrieval of text corresponding to the search term from the text to be searched; The text to be retrieved is split sequentially based on paragraphs, sentences, and a preset first character count to obtain coarse-grained texts. The target coarse-grained text corresponding to the search term is then obtained from each coarse-grained text. The preset first character count is determined by the input length constraint of the object to which the text to be retrieved is split. The target coarse-grained text is split based on punctuation marks to obtain fine-grained texts, and the target fine-grained text corresponding to the search term is obtained from each fine-grained text. Extract key character information corresponding to the search term from the target fine-grained text, and determine the text corresponding to the search term based on the key character information, wherein the key character information is character information related to the search term in the target fine-grained text.

2. The method according to claim 1, characterized in that, The text to be retrieved is sequentially segmented based on paragraphs, sentences, and a preset first character count to obtain various coarse-grained texts, including: The text to be retrieved is split into paragraphs to obtain the paragraph text. The paragraph text is split into whole sentences based on the sentences; Using punctuation marks as a reference, the text with a word count greater than the preset first word count in the whole sentence is split into each coarse-grained text.

3. The method according to claim 2, characterized in that, Based on punctuation marks, the text in the whole sentence with a word count greater than the preset first word count is split into various coarse-grained texts, including: From the entire sentence text, identify long sentences with a word count greater than the preset first word count; Traverse from the last character of the long sentence text to the first character of the long sentence text. If the current character is a punctuation mark and the length of the traversed sentence text between the current character and the current starting character is less than the preset first number of characters, then the traversed sentence text is determined as the text in each coarse-grained text.

4. The method according to claim 3, characterized in that, The method further includes: If the length of the traversed sentence text reaches the preset first number of characters, then the character splitting interval is determined based on the preset second number of characters, the number of characters in the traversed sentence text, and the preset first number of characters, and the character splitting interval is less than the preset first number of characters; The traversed sentence text is split according to the split character number interval and the preset second character number to obtain the coarse-grained text of the traversed sentence text in each coarse-grained text.

5. The method according to claim 4, characterized in that, The traversed sentence text is split according to the split character number interval and the preset second character number to obtain the coarse-grained text of the traversed sentence text in each coarse-grained text, including: If the number of characters in the traversed sentence text reaches the preset second number of characters, then the traversed sentence text is split according to the split character interval to obtain the coarse-grained text of the traversed sentence text in each coarse-grained text; If the number of characters in the traversed sentence text is less than the preset second number of characters, then the traversed sentence text is split based on the position of half the number of characters in the traversed sentence text to obtain the coarse-grained text of the traversed sentence text in each coarse-grained text.

6. The method according to any one of claims 1-5, characterized in that, Obtaining the target coarse-grained text corresponding to the search term from each of the coarse-grained texts includes: The coarse-grained texts and the search terms are input into a pre-trained coarse-grained text relevance classifier, which outputs the first classification result corresponding to each coarse-grained text. The first classification result of each coarse-grained text indicates whether the coarse-grained text is related to the search terms. The target coarse-grained text is determined based on the first classification result; The coarse text relevance classifier is trained based on a sample dataset, which includes sample text, sample search terms, and ground truth values ​​of search answers in the sample text corresponding to the sample search terms.

7. The method according to claim 6, characterized in that, The coarse-grained text relevance classifier includes a first encoder and a second encoder. The first classification result corresponding to each coarse-grained text is obtained by encoding the coarse-grained text by the first encoder and encoding the search term by the second encoder.

8. The method according to any one of claims 1-7, characterized in that, Each fine-grained text includes: the target coarse-grained text with fewer than a preset third number of characters, and text obtained by splitting the target coarse-grained text with more than the preset third number of characters based on punctuation marks; The preset first character count is greater than the preset third character count.

9. The method according to any one of claims 1-8, characterized in that, Obtaining the target fine-grained text corresponding to the search term from each of the fine-grained texts includes: The fine-grained texts and the search terms are input into a pre-trained fine-grained text relevance classifier, which outputs the second classification result corresponding to each fine-grained text. The second classification result of each fine-grained text indicates whether the fine-grained text is related to the search terms. The target fine-grained text is determined based on the second classification result; The fine-grained text relevance classifier is trained based on a sample dataset, which includes sample text, sample search terms, and ground truth values ​​of search answers corresponding to the sample search terms in the sample text.

10. The method according to claim 9, characterized in that, The fine-grained text relevance classifier includes a third encoder and a fourth encoder. The second classification result corresponding to each fine-grained text is obtained by encoding the fine-grained text by the third encoder and encoding the search term by the fourth encoder.

11. The method according to any one of claims 1-10, characterized in that, Extracting key character information corresponding to the search term from the target fine-grained text includes: The target fine-grained text and the search term are input into a pre-trained key text extraction model, which outputs the third classification result corresponding to each character in the target fine-grained text. The third classification result corresponding to each character in the target fine-grained text indicates whether the character is related to the search term. The key character information is determined based on the third classification result; The key text extraction model is trained based on a sample dataset, which includes sample text, sample search terms, and the ground truth values ​​of the search answers corresponding to the sample search terms in the sample text.

12. The method according to claim 11, characterized in that, The key text extraction model includes a fifth encoder and a sixth encoder. The third classification result corresponding to each character in the target fine-grained text is obtained by encoding the target fine-grained text based on the fifth encoder and encoding the search term based on the sixth encoder.

13. The method according to any one of claims 1-12, characterized in that, If the key character information includes multiple key characters, then the text corresponding to the search term is the text obtained by concatenating the multiple key characters.

14. A text retrieval device, characterized in that, The device includes: The first acquisition unit is used to acquire a retrieval request, which is used to instruct the retrieval of text corresponding to the retrieval term from the text to be retrieved. The first splitting unit is used to split the text to be retrieved sequentially based on paragraphs, sentences, and a preset first character count to obtain each coarse-grained text, wherein the preset first character count is determined by the input length constraint of the object to which the text to be retrieved is split; The second acquisition unit is used to acquire the target coarse-grained text corresponding to the search term from each of the coarse-grained texts; The second splitting unit is used to split the target coarse-grained text based on punctuation marks to obtain fine-grained text. The third acquisition unit is used to acquire the target fine-grained text corresponding to the search term from each fine-grained text; An extraction unit is used to extract key character information corresponding to the search term from the target fine-grained text, wherein the key character information is character information related to the search term in the target fine-grained text; The determining unit is used to determine the text corresponding to the search term based on the key character information.

15. A text retrieval system, characterized in that, include: At least one memory, the memory including at least one set of instructions for pushing information; At least one processor communicates with the at least one memory; Wherein, when the at least one processor executes the at least one set of instructions, the method as described in any one of claims 1 to 13 is implemented.

Images