Text processing method, text processing device, electronic device, and computer program

Abstract

The present application provides a text processing method, a text processing device, an electronic device, and a computer-readable storage medium, the method including the steps of obtaining a query text; invoking a search engine interface based on the query text to obtain a plurality of text search results corresponding to the query text; obtaining a plurality of answer text passages matching the query text from the plurality of text search results; determining a relevance between the query text and each of the plurality of answer text passages, and setting the answer text passage corresponding to the maximum relevance as a reference text of the query text; and invoking a first language model based on the query text and the reference text to obtain an answer text for the query text.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application is based on a Chinese patent application bearing application number 202310525850.2, filed with the China Patent Office on May 11, 2023, and claims priority to that Chinese patent application, the entire contents of which are incorporated herein by reference.

[0002] The present application relates to artificial intelligence technology, and in particular to a text processing method, a text processing device, an electronic device, and a computer-readable storage medium. [Background technology]

[0003] Natural Language Processing (NLP) is an important field in the fields of computer science and artificial intelligence, and it studies various logics and methods that enable humans and computers to communicate effectively using natural language. NLP deals with natural language, the language that people use every day, and is closely related to the study of linguistics, as well as being an important technique for model training in computer science, mathematics, and artificial intelligence.

[0004] In related technology, large language models (LLMs) are typically used for natural language learning and understanding, automatically generating corresponding text based on a given context in downstream tasks. Generally, large language models are typically based on a Transformer architecture, which requires a large number of model parameters and makes deployment and training relatively difficult. Furthermore, large language models lack the ability to acquire information or self-update. If the model is updated infrequently, text generation can only be performed within a fixed range of existing natural languages, resulting in limitations on the generated content and difficulty in control. Large language models are unable to incorporate and learn new natural languages ​​in a timely manner, which can lead to information discrepancies between the generated text and the given text, resulting in significant differences in the nature of the generated text at different times and reducing the timeliness of the generated text. Summary of the Invention [Problem to be solved by the invention]

[0005] Embodiments of the present application provide a text processing method, a text processing device, an electronic device, a computer-readable storage medium, and a computer program product that can improve the timeliness of text generated by a language model. [Means for solving the problem]

[0006] The technical solution of the embodiments of the present application is realized as follows:

[0007] An embodiment of the present application provides a text processing method executed by an electronic device, the method comprising: obtaining a query text; invoking a search engine interface based on the query text to obtain a plurality of text search results corresponding to the query text; obtaining a plurality of answer text passages from the plurality of text search results that match the query text; determining a relevance between the query text and each of the plurality of answer text passages, and determining the answer text passage corresponding to the maximum relevance as a reference text of the query text; and invoking a first language model based on the query text and the reference text to obtain an answer text to the query text.

[0008] An embodiment of the present application provides a text processing device, the device comprising: an acquisition module; a determination module; and a calling module; the acquisition module is configured to acquire query text; the retrieval module is further configured to invoke a search engine interface based on the query text to obtain a plurality of text search results corresponding to the query text; the retrieval module is further configured to retrieve a plurality of answer text passages matching the query text from the plurality of text search results; the determining module is configured to determine a relevance between the query text and each of the plurality of answer text passages, and determine the answer text passage corresponding to the maximum relevance as a reference text of the query text; The invocation module is configured to invoke a first language model based on the query text and the reference text to obtain an answer text to the query text.

[0009] An embodiment of the present application provides an electronic device, the electronic device comprising: a memory for storing computer-executable instructions or computer programs; a processor that executes computer-executable instructions or computer programs stored in the memory to perform a text processing method according to an embodiment of the present application.

[0010] An embodiment of the present application provides a computer-readable storage medium having stored thereon computer-executable instructions or a computer program that, when executed by a processor, causes the processor to perform a text processing method according to an embodiment of the present application.

[0011] An embodiment of the present application provides a computer program product comprising computer executable instructions or a computer program that, when executed by a processor, causes the processor to perform a text processing method according to an embodiment of the present application. [Effects of the Invention]

[0012] The embodiments of the present application have the following beneficial effects:

[0013] The query text is obtained and a search engine interface is invoked to obtain a plurality of text search results. Answer text passages that match the query text are screened from the text search results. Reference texts are screened from the answer text passages based on their relevance to the query text. The query text and the screened reference texts are related to each other. The language model uses the query text and the matching reference texts to generate an answer text corresponding to the query text, thereby avoiding information discrepancies between the query text and the answer text and increasing the controllability of the content generated by the language model. The searched text is used as the reference text to improve the accuracy of the generated question and answer text. The query text is used to invoke a search engine interface for a search, and the searched answer text related to the query text is used as the reference text for the language model, allowing the language model to timely obtain abundant and timely information through the search engine interface. The real-time searched reference text is used to generate text, thereby avoiding the difference in properties between the query text and the answer text due to the difference in generation time, overcoming the constraints on timeliness during text generation, and also improving the timeliness of the content generated by the language model. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is an exemplary structural diagram of a text processing system architecture according to an embodiment of the present application; [Figure 2] 1 is an exemplary structural diagram of an electronic device according to an embodiment of the present application; [Figures 3A-3L] 1 is an exemplary flowchart of a text processing method according to an embodiment of the present application; [Figure 4] 1 is a flowchart of a process of a text processing method according to an embodiment of the present application; [Figure 5] FIG. 1 is a process diagram of a process for extracting answer text passages according to an embodiment of the present application. [Figure 6] FIG. 1 is a process diagram for determining a relevance score between a query text and an answer text passage according to an embodiment of the present application. [Figure 7] 1 is a flowchart of a process for performing citation alignment on generated text according to an embodiment of the present application. [Figure 8] FIG. 1 is a diagram illustrating a prediction process of a semantic similarity matching model according to an embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0015] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be described in more detail hereinafter with reference to the drawings, and the described embodiments should not be understood as limitations on the present application, and all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present application.

[0016] In the following description, the term "some embodiments" refers to a subset of all possible embodiments, and it will be understood that the terms "some embodiments" may refer to the same or different subsets of all possible embodiments, which may be combined with each other without conflict.

[0017] The terms "first / second / third" referred to in the following description are merely for distinguishing between similar objects and do not represent a particular order of the objects; it is to be understood that "first / second / third" may, in some cases, be interchangeable with a particular order or precedence order, whereby the embodiments of the present application described herein may be performed in an order other than that illustrated or described.

[0018] It is to be understood that the embodiments of the present application refer to relevant data such as user information (e.g., query text entered by a user using a terminal device), but when the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data should comply with the relevant laws, regulations and standards of the relevant countries or regions.

[0019] Unless otherwise defined, all technical and scientific terms used in the examples of this application have the same semantics as commonly understood by those skilled in the art. The terms used in the examples of this application are only for the purpose of describing the examples of this application and are not intended to limit the scope of this application.

[0020] Before describing the embodiments of the present application in more detail, the nouns and terms referred to in the embodiments of the present application will be explained, and the nouns and terms referred to in the embodiments of the present application are applicable to the following description.

[0021] 1) Large Language Models (LLMs), abbreviated as language models, can process and generate machine learning models of natural language, such as the Bidirectional Encoder Representation from Transformers (BERT) model and the Generative Pre-Training Model (GPT) model. Prediction tasks of language models can include text classification, blank fill-in, question answering, etc.

[0022] 2) Timeliness refers to the fact that the same thing may have significant differences in its properties at different times. When a large-scale language model generates text content, the lack of timeliness refers to the model's inability to integrate new language knowledge in real time, its inability to update the scope of its acquired language knowledge, and the resulting in a limited and insufficient scope of content in the generated text.

[0023] 3) The Bidirectional Encoder Representation from Transformers (BERT) model is a pre-trained text processing model that can perform encoding processing on text and can be used in various application scenarios such as text generation, text similarity determination, and text identification.

[0024] The embodiments of the present application provide a text processing method and apparatus, an electronic device, a computer-readable storage medium, and a computer program product that can improve the timeliness of text generated by a language model.

[0025] Referring to FIG. 1, FIG. 1 is a schematic diagram of the architecture of a text processing system 100 according to an embodiment of the present application, including a terminal 500, a network 300, and a server 200, where the terminal 500 is connected to the server 200 via the network 300, and the network 300 may be a wide area network or a local area network, or a combination of both.

[0026] In the first application scenario, the terminal 500 is running various text editing and text processing application scenario applications (APPs). When a user inputs a question text requiring an answer in the text editing interface of the APP, the question text is received and transmitted to the server 200 via the network 300. After receiving the question text transmitted from the terminal, the server 200 invokes a search engine interface using the question text as a query text to obtain multiple text search results corresponding to the query text. Then, from the multiple text search results, the server 200 obtains multiple answer text passages matching the query text and screens the answer text passages based on relevance to obtain reference text. Finally, the server 200 invokes a language model based on the query text to obtain answer text corresponding to the query text from the reference text. The obtained answer text is returned to the terminal 500 via the network 300 and displayed in the text editing interface of the corresponding APP on the terminal 500.

[0027] In the second application scenario, when a user inputs a waiting question text that requires an answer into the text editing interface of a terminal application, the terminal can directly process the question text. Specifically, the terminal invokes a search engine interface using the question text as a query text to obtain multiple text search results corresponding to the query text. Then, from the multiple text search results, multiple answer text passages that match the query text are obtained, and the answer text passages are screened based on relevance to obtain reference text. Finally, a language model is invoked based on the query text to obtain answer text corresponding to the query text from the reference text, and the answer text is directly displayed in the text editing interface.

[0028] In some embodiments, the server 200 shown in FIG. 1 may be an independent physical server, a server cluster or a distributed system configured by multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), big data, and artificial intelligence platforms. The terminal 500 shown in FIG. 1 may be, but is not limited to, a smartphone, tablet computer, notebook computer, desktop computer, smart speaker, smart watch, smart TV, in-car terminal, etc. The terminal and server may be directly or indirectly connected via wired or wireless communication, and the embodiments of the present application are not limited thereto.

[0029] Embodiments of the present application may be realized by relying on artificial intelligence (AI), which refers to theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate and extend human intelligence, sense the environment, acquire knowledge, and use the knowledge to achieve optimal results. In other words, artificial intelligence is a comprehensive technology of computer science that aims to understand the essence of intelligence and create new intelligent machines that can respond in a manner similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have the capabilities of sensing, reasoning, and decision-making.

[0030] For example, a server according to an embodiment of the present application may be a server cluster deployed in the cloud, thereby providing users or developers with an artificial intelligence cloud service (AIaaS: AI as a Service). The AIaaS platform divides several common AI services and provides them as independent or packaged services on the cloud. This service model is similar to an AI theme model, where all users or developers can access and use one or more artificial intelligence services provided by the AIaaS platform through an application programming interface.

[0031] For example, a cloud server is packaged with a text processing program according to an embodiment of the present application. A user invokes a text processing service on the cloud service through a terminal device (running a text editing application on the terminal device), which then invokes the packaged text processing program on the server deployed in the cloud, receives a question text awaiting an answer entered by the user, and invokes a search engine interface using the question text as a query text to obtain multiple text search results corresponding to the query text. Then, multiple answer text passages matching the query text are obtained from the multiple text search results, and the answer text passages are screened based on relevance to obtain reference text. Finally, a language model is invoked based on the query text, and answer text corresponding to the query text is obtained from the reference text, and the answer text is directly displayed on the terminal device.

[0032] Referring to FIG. 2, FIG. 2 is an exemplary structural diagram of an electronic device 400 according to an embodiment of the present application. The electronic device 400 may be implemented as a server in the first application scenario described above or as a terminal in the second application scenario described above. The electronic device 400 shown in FIG. 2 includes at least one processor 410, a memory 450, and at least one network interface 420. Components within the electronic device 400 are coupled together by a bus system 440. It is understood that the bus system 440 is used to realize connection and communication between these components. In addition to a data bus, the bus system 440 further includes a power bus, a control bus, and a status signal bus. However, for clarity, various buses are referred to as the bus system 440 in FIG. 2.

[0033] The processor 410 may be an integrated circuit chip having signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc., where the general-purpose processor may be a microprocessor or any conventional processor, etc.

[0034] Memory 450 may be removable, non-removable, or a combination thereof, and example hardware devices thereof include solid-state memory, hard disk drives, optical disk drives, etc. Memory 450 optionally includes one or more storage devices that are physically located remote from processor 410.

[0035] The memory 450 may include volatile memory, non-volatile memory, or both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in the embodiments herein is intended to include any suitable type of memory.

[0036] In some embodiments, memory 450 may store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as illustratively described below.

[0037] The operating system 451 includes system programs for processing various basic system services and executing hardware-related tasks, such as a framework layer, a core library layer, and a driver layer, which are used to realize various basic services and process hardware-based tasks.

[0038] The network communication module 452 is configured to reach other electronic devices via one or more (wired or wireless) network interfaces 420, illustratively including Bluetooth® technology, Wireless Fidelity (WiFi), and Universal Serial Bus (USB), among others.

[0039] In some embodiments, the apparatus according to the embodiments of the present application can be realized in a software manner, and FIG. 2 shows a text processing device 453 stored in a memory 450. The text processing device 453 may be software in the form of a program or plug-in, and includes software modules, such as an acquisition module 4531, a determination module 4532, and a calling module 4533. These modules are logical and can be arbitrarily combined or further divided based on the functions to be realized. The functions of each module are described below.

[0040] In some embodiments, a terminal or server may execute various computer-executable instructions or computer programs to implement a text processing method according to an embodiment of the present application. For example, the computer-executable instructions may be microprogram-level instructions, machine instructions, or software instructions. For example, the computer program may be a native program or software module in an operating system, a native application (APP), i.e., a program that needs to be installed in an operating system for execution, or a mini-program that can be embedded in any APP, i.e., a program that can be executed simply by downloading it to a browser environment. In short, the computer-executable instructions may be any type of instruction, and the computer program may be any type of application, module, or plug-in.

[0041] The text processing method according to the embodiment of the present application will be described with reference to exemplary applications and implementations of electronic devices according to the embodiment of the present application.

[0042] Referring to FIG. 3A, FIG. 3A is an exemplary flowchart of a text processing method according to an embodiment of the present application, where the execution entity may be a server in the first application scenario above, or a terminal device in the second application scenario above. In the following, the execution entity will not be described in detail, but will be described with reference to the steps shown in FIG. 3A.

[0043] In step 101, the query text is obtained.

[0044] In some embodiments, the query text is generally related to a corresponding application scenario, such as a context and prompt given in the application scenario, or a question text that needs to be answered, or a search keyword or a search term consisting of multiple search keywords corresponding to a search system scenario, etc. The query text is mainly used to obtain the latest search information, extract answer text passages that match the query text, and provide corresponding reference texts to the first language model.

[0045] In step 102, a search engine interface is invoked based on the query text to obtain a plurality of text search results corresponding to the query text.

[0046] Exemplarily, a search engine interface is invoked based on the query text to perform a search operation from a network or a specific database. The source of the search results may be a web page, a social platform, etc. In the search result list obtained by the search engine interface, the search results are sorted in ascending order of time difference based on the difference between the generation time and the current time, with the search result corresponding to the latest generation time being displayed first.

[0047] In some embodiments, referring to FIG. 3B, step 102 shown in FIG. 3A can be realized through the following steps 1021 and 1022, which will be described in detail below.

[0048] In step 1021, a search engine interface is invoked based on the query text, and the search engine interface is caused to search a plurality of text search results related to the query text in a creation time sorted manner.

[0049] After obtaining the query text, a related search engine interface is invoked to obtain multiple text search results based on the query text. Here, the search engine interface may be a search server interface, a search interface invoked from a related terminal browser, or any other engine or functional interface with a search function. The search method of the search engine interface is sorted by creation time, i.e., the most recently created search information is sorted and displayed at the top. The text search results may typically be in plain text format or rich media format, and in the latter case, the text therein can be extracted, typically including a paragraph of main text and a corresponding text title.

[0050] In step 1022, a number of text search results related to the query text are obtained from the search engine interface.

[0051] After invoking a search engine interface to search for the query text, a plurality of text search results are obtained, and then a plurality of text search results related to the query text are sequentially obtained from the search engine interface in a manner sorted in order of the generation time of the text search results, that is, a plurality of text search results related to the query text and having the latest generation time are obtained.

[0052] In the embodiment of the present application, the query text is used, and the latest search information related to the query text is obtained from the search engine interface according to the principle of prioritizing timeliness, and then the reference text is screened from the latest search information, thereby ensuring the timeliness of the reference text information.

[0053] Still referring to FIG. 3A, in step 103, a plurality of answer text passages that match the query text are obtained from the plurality of text search results.

[0054] In the body text contained in the text search results, only a portion of the text can be used as an answer to the query text, and the rest is text information unrelated to the query text, so it is necessary to extract these text segments that can be used as answers from the body text contained in the text search results.

[0055] In some embodiments, referring to FIG. 3C, FIG. 3C is an exemplary flowchart of a text processing method according to an embodiment of the present application, and step 103 shown in FIG. 3A can be realized through steps 1031 to 1035 in FIG. 3C, which will be described in detail below.

[0056] In step 1031, the text search results are divided into multiple candidate citation text segments of fixed length.

[0057] Since only a portion of the body text included in the text search results can be used as an answer matching the query text, the body text in each text search result is divided into fixed lengths to obtain multiple candidate quoted text segments. Here, the fixed length of the candidate quoted text segments can be set according to the text size or number of segments of the actual text search results, and can be, for example, 500. The fixed length is directly proportional to the text size or number of segments of the actual text search results.

[0058] In step 1032, a matching score between the query text and each of the plurality of candidate cited text segments is obtained, as well as the start probability and end probability of each element in the candidate cited text segment being the start and end position of the answer text passage.

[0059] After the body text is segmented, the query text can be combined with each candidate cited text segment to determine candidate cited text segments that match the query text, and the combined query text can be input to an associated model for processing to predict a matching score between the query text and each of the candidate cited text segments. Considering that an answer text passage to the query text is screened from the candidate cited text segments, the start and end positions of the answer text passage in the candidate cited text segment are determined by determining whether each element in the candidate cited text segment can be used as the start and end element of the answer text passage.

[0060] In some embodiments, referring to FIG. 3D, FIG. 3D is an exemplary flowchart of a text processing method according to an embodiment of the present application, and step 1032 shown in FIG. 3C can be realized via steps 10321 to 10322 in FIG. 3D, which will be described in detail below.

[0061] In step 10321, each of the plurality of candidate quoted text segments is combined with the query text as a text pair.

[0062] In some embodiments, in order to predict a matching score between the query text and each of the plurality of candidate cited text segments, it is necessary to determine the similarity between the query text and the candidate cited text segments. In embodiments of the present application, each of the plurality of candidate cited text segments and the query text are combined into a text pair, and a subsequent second language model is used to predict the similarity between the query text and the candidate cited text segments in the text pair.

[0063] In step 10322, a second language model is invoked based on each text pair to obtain a matching score between the query text and each of the plurality of candidate cited text segments, as well as a start probability and an end probability that each element in the candidate cited text segment will be the start and end location of the answer text passage.

[0064] Following the above example, the query text is combined with each of the candidate cited text segments to form text pairs, and then a second language model is invoked to process the text pairs and obtain a matching score between the query text and each of the plurality of candidate cited text segments, where the matching score represents the similarity between the semantics of the candidate cited text and the semantics of the query text, with a higher matching score indicating a closer semantic fit.

[0065] Obtaining whether each element in the candidate cited text segment in the text pair is suitable as the start element and the end element of the answer text passage can be achieved in the following manner: determine the start probability that each element in the candidate cited text segment will be the start position of the answer text passage and the end probability that it will be the end position, and select the element with the highest start probability as the start position, and select the element with the highest end probability as the end position.

[0066] In some embodiments, referring to FIG. 3E, FIG. 3E is an exemplary flowchart of a text processing method according to an embodiment of the present application, and step 10322 shown in FIG. 3D can be realized via steps 103221 to 103225 of FIG. 3E, which will be described in detail below.

[0067] In step 103221, each element in the query text and the candidate quoted text segment is treated as one ordinary token, and the ordinary tokens are concatenated as a first token sequence.

[0068] Before calling the second language model to process the text pair, we obtain an embedded feature vector for the text pair by treating each element (token) in the query text and the candidate text segment as a regular token, where a regular token is used to represent the text token of a text element, as opposed to a start token and a delimiter token; we then string together all the regular tokens into a first token sequence, where an element can be a character, word, or short phrase in the text, because forcing some text to be distinguished by characters or words may fail to express specific semantic information; and finally, we insert a start token (CLS) at the beginning of the first token sequence.

[0069] In step 103222, an embedding process is performed on the first token sequence to obtain an embedded feature vector of the first token sequence.

[0070] Following the above example, after obtaining a first token sequence into which a start token is inserted, an embedding process is performed on the first token sequence, and the first token sequence is mapped to an embedded feature vector, which is then input to a second language model for processing. Here, the embedding process on the first token sequence can be realized by invoking a single-layer neural network or text embedding model, and the embedding process converts discretized text into a dense continuous vector, i.e., maps the text to a point in vector space. Finally, the semantic information of the text in semantic space is converted into a vector, which is then used for subsequent encoding of the vector by the second language model.

[0071] In step 103223, a second language model is invoked based on the embedding feature vector of the first token sequence to perform encoding processing, thereby obtaining a semantic feature vector of the first token sequence.

[0072] Following the above example, an embedding feature vector of the first token sequence is obtained by embedding, and then a second language model is invoked to perform encoding on the embedded feature vector, thereby obtaining a semantic feature vector for each token (including the start token and normal tokens) in the first token sequence. Here, the second language model may be a BERT model with a 12-layer structure (BERT-12-Layer), or may be another text encoder or text encoding model with a multi-layer structure.

[0073] In step 103224, a mapping process is performed on the semantic feature vector of the starting token in the semantic feature vector of the first token sequence to obtain a matching score between the query text and the candidate quoted text segment.

[0074] The semantic feature vector of the starting token more fairly fuses the semantic information of each element of the text pair compared to other elements in the text pair, and therefore can be considered to contain the semantic information of the entire text pair.

[0075] In the present embodiment, the semantic feature vector corresponding to the starting token is input to a fully connected layer, and a mapping process is performed to obtain a matching score between the query text and the candidate citation text segment. Here, the mapping process can be regarded as a predictive classification process, and the activation function of the fully connected layer may be a Softmax function, which is used to map the corresponding semantic feature vector to a predicted classification probability, and this classification probability becomes the matching score.

[0076] In step 103225, a mapping process is performed on the semantic feature vectors corresponding to each normal token in the semantic feature vector of the first token sequence, and the start probability and end probability of each element in the candidate quoted text segment being the start position and end position of the answer text passage are obtained.

[0077] While determining the matching score between the query text and the candidate cited text segments, the system also determines which text segments among the candidate cited text segments in the text pair can be used as answer text passages. That is, the system predicts the start probability and end probability of each element in the candidate cited text segment as a suitable start position and end position for the answer text passage. Specifically, the system inputs semantic feature vectors corresponding to each normal token in the candidate cited text segment into a fully connected layer for mapping. The fully connected layer is preset with two position vectors, a start position vector and an end position vector. The system calculates the dot product of the semantic feature vector corresponding to each input normal token with the start position vector and the end position vector. The activation function of the fully connected layer is then used to map the two dot product results to two classification probabilities. The activation function of the fully connected layer may be a Softmax function, and these two classification probabilities become the corresponding start probability and end probability.

[0078] In an embodiment of the present application, the start probability of an element in a candidate cited text segment as the start position of an answer text passage and the end probability of the element as the end position of the answer text passage are analyzed, and the start position and end position of the answer text passage corresponding to the candidate cited text segment are determined based on the probability, thereby reducing the redundancy of the answer text passage, increasing the relevance of the answer text passage and the query text, and improving the accuracy of determining the answer text passage.

[0079] Continuing to refer to FIG. 3C, in step 1033, the best matching segment is determined.

[0080] By determining a matching score between the query text and each candidate cited text segment, the candidate cited text segment corresponding to the highest matching score based on the matching scores can be determined and used as the best matching segment.

[0081] In step 1034, the start and end elements in the best matching segment are determined.

[0082] After determining the best matching segment, it is determined which part of the text segment in the best matching segment belongs to the answer text passage. Specifically, it determines the start probability of each element in the best matching segment as the start position of the answer text passage, and the element in the best matching segment corresponding to the maximum start probability is determined as the start element of the answer text passage. Similarly, it further determines the end probability of each element in the best matching segment as the end position of the answer text passage, and the element in the best matching segment corresponding to the maximum end probability is determined as the end element of the answer text passage.

[0083] In step 1035, the portion of the best matching segment located between the start element and the end element is taken as the answer text passage that matches the query text.

[0084] After determining the start element and end element of the answer text passage from the best matching segment, the text portion between the start element and end element corresponding to the best matching segment is determined as the answer text passage that matches the query text.

[0085] Exemplarily, when the query text is "What is the temperature today?", and the optimal matching segment with the highest matching score determined from the candidate citation text segments is "The weather is good, the temperature is 20 degrees, cloudy". At this time, it is necessary to determine which part of the text segment from the optimal matching segment can be used as the answer text passage. When predicting the matching score, the start probability that each element in the candidate citation text segment is appropriate as the start position of the answer text passage and the end probability that it is appropriate as the end position are also predicted. Therefore, based on the start probability predicted corresponding to each element in the optimal matching segment, it is determined that the start probability predicted corresponding to the element "qi" in it is the highest, and the element "qi" is used as the start element of the answer text passage. Similarly, based on the end probability predicted corresponding to each element in the optimal matching segment, it is determined that the end probability predicted corresponding to the element "du" in it is the highest, and the element "du" is used as the end element of the answer text passage. Thereby, the start element and the end element of the answer text passage are determined from the optimal matching segment, and by using the text part between the start element "qi" and the end element "du" in the optimal matching segment as the answer text passage, it is determined that the answer text passage is "The temperature is 20 degrees". Through this process, finally, one answer text passage is screened for each text search result.

[0086] Continuing to refer to FIG. 3A, in step 104, the relevance between the query text and each of the plurality of answer text passages is determined, and the answer text passage corresponding to the maximum relevance is used as the reference text of the query text.

[0087] For each text search result, one answer text passage is finally screened, and multiple answer text passages are screened. The specific screening process is to determine the relevance between the query text and each of the multiple answer text passages, and determine the answer text passage corresponding to the maximum relevance as the reference text of the query text.

[0088] The relevance is an index for expressing the relevance between the contents of the query text and the answer text passage, and is expressed by the probability that the features of the query text and the features of the answer text passage belong to the same type.

[0089] In some embodiments, referring to FIG. 3F, FIG. 3F is an exemplary flowchart of a text processing method according to an embodiment of the present application, and “determining the relevance between the query text and each of the plurality of answer text passages” in step 104 shown in FIG. 3A can be achieved through steps 1041 to 1044 in FIG. 3F, which will be described in detail below.

[0090] In step 1041, each element of the query text, the answer text passage, and the title text is treated as one normal token, and the normal tokens of each element are connected as a second token sequence.

[0091] For each answer text passage, each element in the query text and the answer text passage is treated as one normal token, and then the normal tokens of each element are strung together as a second token sequence, where the normal tokens between the query text and the answer text passage are connected by a delimiter token, and a start token is inserted at the beginning of the second token sequence.

[0092] In some embodiments, the title text of the answer text passage can be extracted and used as part of the second token sequence. Thus, each element of the query text, the answer text passage, and the title text can be treated as a regular token, and the regular tokens of each element can be spliced ​​together to form the second token sequence. The title text generally expresses strong semantic features, which can be useful in determining subsequent relevance and improve the accuracy of the relevance calculation.

[0093] In step 1042, an embedding process is performed on the second token sequence to obtain an embedded feature vector of the second token sequence.

[0094] Following the above example, after obtaining the second token sequence, an embedding process is performed on the second token sequence to obtain an embedded feature vector of the second token sequence, which is used for subsequent encoding by a second language model. Here, the embedding process on the second token sequence can be realized by invoking a single-layer neural network (fully connected layer), a multi-layer perceptron (MLP), or other text embedding models.

[0095] In step 1043, a third language model is invoked to perform an encoding process on the embedded feature vector of the second token sequence to obtain a semantic feature vector of the second token sequence.

[0096] Following the above example, a third language model is called to perform encoding processing on the embedded feature vector of the second token sequence to obtain a semantic feature vector of the second token sequence, where the semantic feature vector of the second token sequence corresponds to the start token of the second token sequence, because the semantic encoding result corresponding to the start token position can represent the semantic encoding result of the entire second token sequence. The third language model here may be a cross-encoder, a BERT model, or another non-cross-encoder or text encoder with no restrictions on the number of network layers or network structure.

[0097] In step 1044, a mapping process is performed on the semantic feature vector of the second token sequence to obtain the relevance between the query text and the answer text passage.

[0098] Following the above example, the semantic feature vector corresponding to the start token of the second token sequence is taken as the semantic feature vector of the entire second token sequence, and then a multi-layer perceptron is called to perform a mapping process on the semantic feature vector. Specifically, the semantic feature vector corresponding to the start token position is mapped to a probability of one predicted classification using the multi-layer perceptron. The probability of this predicted classification is the relevance between the query text and the answer text passage, and S q,t+p It is written as follows.

[0099] Through the above example, the relevance (S q,t+p ) is determined, and the maximum value of S q,t+p The answer text passage corresponding to is determined as the reference text corresponding to the query text.

[0100] Still referring to FIG. 3A, in step 105, a first language model is invoked based on the query text and the reference text to obtain an answer text to the query text.

[0101] Through the above embodiments, a reference text corresponding to the query text is finally obtained, and at this time, the first language model can be invoked to process the query text and the reference text, and an answer text corresponding to the query text can be predicted from the reference text.

[0102] In some embodiments, referring to FIG. 3G, FIG. 3G is an exemplary flowchart of a text processing method according to an embodiment of the present application, and step 105 shown in FIG. 3A can be realized via steps 1051 to 1052 in FIG. 3G, which will be described in detail below.

[0103] In step 1051, a first language model is invoked based on the query text and the reference text to perform a prediction process on the query text, and the start and end elements of the answer text in the reference text are determined.

[0104] After determining the reference text from the answer text passage, the reference text is used as a reference material to guide the first language model to predict the input answer text and generate the corresponding answer text. Specifically, the query text and the reference text are input into the first language model to perform a prediction process, and the start element and end element of the reference text of the answer text are determined, thereby obtaining the start position and end position of the answer text in the reference text.

[0105] In some embodiments, referring to FIG. 3H, FIG. 3H is an exemplary flowchart of a text processing method according to an embodiment of the present application, and step 1051 shown in FIG. 3G can be realized via steps 10511 to 10515 of FIG. 3H, which will be described in detail below.

[0106] In step 10511, each element of the query text and the reference text is treated as one normal token, and the normal tokens are concatenated as a third token sequence.

[0107] In some embodiments, since the language model cannot be directly predicted based on the original text, it is still necessary to determine the semantic feature vector of the text. First, obtain the embedded feature vector of the text, and treat each element in the query text and the reference text as one regular token, and then splice the regular tokens of each element into a third token sequence, where the regular tokens between the query text and the reference text are connected by a delimiter token, and insert a start token (CLS) at the beginning of the third token sequence.

[0108] In step 10512, an embedding process is performed on the third token sequence to obtain an embedded feature vector of the third token sequence.

[0109] Following the above example, an embedding process is performed on the third token sequence obtained by splicing, and an embedded feature vector of the third token sequence is obtained, which is used for subsequent encoding by the first language model. Similarly, the embedding process on the third token sequence can be realized by calling a single-layer neural network (fully connected layer), a multi-layer perceptron, or by using other text embedding models.

[0110] In step 10513, the first language model is invoked to perform encoding processing based on the embedded feature vector of the third token sequence, and a semantic feature vector of each normal token in the third token sequence is obtained.

[0111] Following the above example, the encoder of the first language model is called to perform an encoding process on the embedded feature vector of the third token sequence, thereby obtaining a semantic feature vector corresponding to each normal token in the third token sequence. Here, the first language model may be any large-scale language model, which can fully learn and understand the semantic features of a given text, and generate a text that matches the given text based on the semantic features of a given text context.

[0112] In step 10514, a mapping process is performed on the semantic feature vectors corresponding to each normal token to obtain the start probability and end probability of each element of the reference text becoming the start element and end element of the answer text, respectively.

[0113] Following the above embodiment, after obtaining the semantic feature vector of each normal token in the third token sequence, in this embodiment, considering that the existing reference text is used as the reference material for the query text, it is only necessary to use the first language model to predict an answer text matching the query phrase from the reference text. Specifically, the semantic feature vector of each normal token in the third token sequence is mapped to two predicted classification probabilities, and the start position vector and end position vector preset in the fully connected layer of the first language model are used to calculate the dot product of the semantic feature vector corresponding to each normal token with the start position vector and the end position vector. Then, the activation function of the fully connected layer is used to map the two dot product results to two classification probabilities, respectively, and these two classification probabilities are the start probability that the element corresponding to the normal token in the reference text will be the start element of the answer text and the end probability that it will be the end element, respectively, where the activation function of the fully connected layer may be a normalization function (Softmax function).

[0114] In step 10515, the element corresponding to the maximum value of the start probability is taken as the start element of the answer text, and the element corresponding to the maximum value of the end probability is taken as the end element of the answer text.

[0115] After determining the two classification probabilities of the corresponding elements of each normal token in the third token sequence, the element in the third token sequence corresponding to the maximum value of the start probability is taken as the start element of the answer text, and the element corresponding to the maximum value of the end probability is taken as the end element of the answer text, thereby determining the specific position of the answer text in the reference text.

[0116] Continuing to refer to FIG. 3G, in step 1052, the text between the start and end elements of the reference text is taken as the answer text to the query text.

[0117] After determining the specific position of the answer text in the reference text, the text segment between the start element and the end element in the reference text is determined as the answer text, which is the final predicted text of the first language model, i.e., the generated text by the first language model.

[0118] Exemplarily, assume that the query text is "What is the temperature today?" and the corresponding reference text is "It is sunny today and the temperature is 20 degrees." Each character of the query text and the reference text is regarded as one normal token, and they are concatenated with a delimiter token to obtain a third token sequence. An start token is inserted at the beginning of the sequence to obtain "[CLS]What is the temperature today[SEP]It is sunny today and the temperature is 20 degrees", and then embedding processing is performed on the third token sequence to obtain the corresponding embedding feature vector. The embedding feature vector is input into the first language model for encoding processing to obtain the semantic feature vector corresponding to each normal token. Thereafter, the semantic feature vectors of each normal token are mapped to two classification probabilities, namely, the start probability that becomes the start position of the answer text and the end probability that becomes the end position of the answer text. As a result, it is determined that the start probability mapped by the semantic feature vector corresponding to the element "qi" in the reference text is the highest, and it can be used as the start position of the answer text. Correspondingly, the end probability mapped by the semantic feature vector corresponding to the element "du" in the reference text is the highest, and it can be used as the end position of the answer text. Therefore, the text between the element "qi" and the element "du" in the reference text is used as the answer text, that is, "The temperature is 20 degrees" is used as the answer text for the query text "What is the temperature today?".

[0119] According to an embodiment of the present application, a query text is acquired, a search engine interface is invoked to generate a plurality of recent text search results with time priority, relevant answer text passages matching the query text are extracted from the text search results, and a screening is performed to determine a text segment from the plurality of answer text passages that is most relevant to the query text as a reference text, which is used as a reference material for a first language model to predict a generated text, and the first language model is then invoked to predict a corresponding answer text matching the query text from the reference text, so that the first language model can answer the query text based on the latest reference material and generate a corresponding answer text, and since the first language model predicts the answer text from a fixed generated reference text, the controllability of the content generated by the first language model is improved and the timeliness of the generated content is improved.

[0120] In some embodiments, the first language model, the second language model, and the third language model may be integrated into one large-scale language model, and the first language model, the second language model, and the third language model may be trained separately.

[0121] In some embodiments, referring to FIG. 3I, FIG. 3I is an exemplary flowchart of a text processing method according to an embodiment of the present application, and after step 105 shown in FIG. 3A, steps 106 to 108 of FIG. 3I can be further performed, which will be described in detail below.

[0122] In step 106, a number of candidate citation texts are obtained.

[0123] In the related art, after generating an answer text based on a query text, the language model considers that the answer text may have low authenticity and may be fabricated, and adds related citation text after the corresponding answer text to use as the text source of the answer text, thereby improving the authenticity of the answer text generated by the first language model and solving the problem of possible fabrication. Corresponding candidate citation texts are obtained through the answer text, where the candidate citation texts include materials cited in the answer text. The candidate citation texts can be obtained by invoking a search engine interface based on the answer text to search for text search content related to the answer text, or by invoking a text generation model to randomly generate multiple texts based on the semantics of the answer text.

[0124] In step 107, the answer text is divided into a plurality of answer text passages, and each candidate quoted text is divided into a plurality of quoted text segments.

[0125] Following the above example, after obtaining the candidate citation text, it is determined whether the candidate citation text matches the answer text, and then the candidate citation text can be used as citation material. The method for determining whether the candidate citation text matches the answer text is to determine the text similarity between the two.

[0126] In the embodiment of the present application, considering that the text scale of the answer text and the candidate citation text may be large, directly determining the text similarity would require too much calculation. Therefore, the answer text is divided into multiple answer text passages longer than a fixed length, and each candidate citation text is divided into multiple citation text segments longer than a fixed length, thereby achieving the method of determining the similarity of the text segments. Citation material can also be added as citation text to each answer text passage of the answer text.

[0127] In step 108, a quoted text segment that matches the at least one answer text passage is determined, and the quoted text segment that matches the at least one answer text passage is inserted into the answer text.

[0128] In some embodiments, referring to FIG. 3J, FIG. 3J is an exemplary flowchart of a text processing method according to an embodiment of the present application, and the step of "determining quoted text segments that match at least one answer text passage" in step 108 shown in FIG. 3I can be realized through the following steps 1081A to 1082A, which will be described in detail below.

[0129] In step 1081A, the answer text passages and the quoted text segments are paired together to form candidate text pairs.

[0130] After the answer text and the candidate quoted text are respectively divided, a plurality of text segments are obtained, respectively. Considering that citation content needs to be added to each answer text passage in the answer text, each answer text passage is respectively combined with each quoted text segment to form a plurality of candidate text pairs, i.e., a plurality of answer text passages and a plurality of quoted text segments are combined two by two, so that the similarity between the answer text passage and the quoted text segment in the candidate text pair can be subsequently determined.

[0131] For example, if an answer text is divided into 20 answer text passages, and for 10 candidate quoted texts, each candidate quoted text is divided into five quoted text segments, after combining each answer text passage with two quoted text segments, the final number of candidate text pairs obtained will be 1000 (20 x 10 x 5).

[0132] In step 1082A, at least one matching text pair is identified from the plurality of candidate text pairs, and a quoted text segment in the matching text pair is determined to match an answer text passage in the matching text pair.

[0133] After combining the answer text passage and the quoted text segment into a plurality of candidate text pairs, at least one matching text pair can be identified from the plurality of candidate text pairs by determining the similarity between the two text segments in the candidate text pair to identify whether the answer text passage matches the quoted text segment, thereby determining that the quoted text segment in the matching text pair matches the answer text passage in the matching text pair.

[0134] In some embodiments, considering that the number of candidate text pairs formed is too large due to the large number of answer text passages and quoted text segments, in order to improve the efficiency of text processing and reduce the computational complexity of subsequent steps, before identifying matching text pairs from the multiple candidate text pairs, initial filtering can be performed on the multiple candidate text pairs to filter out candidate text pairs that are obviously unmatched or unimportant.

[0135] In some embodiments, in an answer text passage, citations are usually added only to text parts containing keywords, so a method for initial filtering of candidate text pairs may be text keyword identification. Keywords express the most semantic information, and other non-keyword text is unimportant text in the text, so there is no need to add citations to these unimportant texts. Therefore, initial filtering can filter out candidate text pairs that do not contain keywords.

[0136] The specific filtering process based on text keyword identification is as follows: For each candidate text pair, a keyword identification model is invoked to identify keywords in the answer text passage and the quoted text segment in the candidate text pair, respectively. If no keywords are identified in at least one of the answer text passage and the quoted text segment in the candidate text pair, it indicates that both the answer text passage and the quoted text segment are unimportant text with no key semantic information, or if no keywords are identified in one of the answer text passage and the quoted text segment, it indicates that the semantics of the two do not clearly match. These unimportant texts and candidate text pairs with clearly unmatched semantics are directly filtered out. Here, the keyword identification model may be a text ranking algorithm (Text Rank), a Term Frequency-Inverse Document Frequency (TF-IDF), a Rapid Automatic Keyword Extraction (RAKE), or a combination of these three methods.

[0137] In some embodiments, the method for performing initial filtering on candidate text pairs is a similarity matching method based on sentence embedding. The specific filtering process is as follows: For each candidate text pair, a text encoding model is invoked to perform encoding processes on the answer text passage and the quoted text segment in the candidate text pair, respectively, to obtain embedding feature vectors for the answer text passage and the quoted text segment, where the embedding feature vector of the answer text passage can be the embedding feature vector of the entire segment or the embedding feature vector of a subsegment (further dividing the answer text passage), and the embedding feature vector of the quoted text segment can be the embedding feature vector of the entire segment or the embedding feature vector of a subsegment (further dividing the quoted text segment). Then, the similarity between the embedding feature vector of the answer text passage and the embedding feature vector of the quoted text segment is determined, where the similarity between the embedding feature vectors can be the similarity between the embedding feature vectors of the entire segment or the similarity between the embedding feature vectors of the subsegments. This allows us to determine the similarity of each candidate text pair, and in the present embodiment, we set a similarity threshold (e.g., if the similarity of a candidate text pair is smaller than the similarity threshold), and remove the corresponding candidate text pair.

[0138] Illustratively, in a candidate text pair, the embedding feature vector of the answer text passage may be the embedding feature vector of the entire segment (e.g., embedding feature vector A), and the embedding feature vector of the quoted text segment in the corresponding candidate text pair may be the embedding feature vector of the entire segment (e.g., embedding feature vector B). The similarity between the embedding feature vector A and the embedding feature vector B can be determined, and if the similarity is less than a similarity threshold, the corresponding candidate text pair is deleted.

[0139] In some other examples, the answer text passage in the candidate text pair is further divided into multiple answer text subsegments, and then a text coding model is invoked to perform an encoding process on the multiple answer text subsegments, thereby obtaining an embedding feature vector for each answer text subsegment, denoted as A1, A2, ..., An. Correspondingly, the quoted text segment in the candidate text pair is further divided into multiple quoted text subsegments, and then a text coding model is invoked to perform an encoding process on the multiple quoted text subsegments, thereby obtaining an embedding feature vector for each quoted text subsegment, denoted as B1, B2, ..., Bn. Then, the similarity between A1 and B1, the similarity between A2 and B2, and the similarity between An and Bn are determined, respectively. If one of the similarities is smaller than the similarity threshold, it indicates that the answer text passage does not match the quoted text segment, and the corresponding candidate text pair is deleted.

[0140] In embodiments of the present application, before identifying matching text pairs from the plurality of candidate text pairs, initial filtering can be performed on the plurality of candidate text pairs, so as to first filter out obviously unmatched or unimportant candidate text pairs, thereby reducing the computational complexity of subsequent steps and improving the efficiency of text processing.

[0141] In some embodiments, referring to FIG. 3K, FIG. 3K is an exemplary flowchart of a text processing method according to an embodiment of the present application, and the step of "identifying at least one matching text pair from a plurality of candidate text pairs" in step 1082A shown in FIG. 3J can be realized through steps 10821A to 10825A in FIG. 3J, which will be described in detail below.

[0142] In step 10821A, a fourth language model is invoked to perform encoding processes on the answer text passage and the candidate quoted text segment in the candidate text pair, respectively, to obtain an embedded feature vector of the answer text passage and an embedded feature vector of the quoted text segment.

[0143] In some embodiments, determining the similarity between two text segments in a candidate text pair can be achieved by invoking a language model. For each candidate text pair, a fourth language model can be invoked to perform an encoding process on the answer text passage and the quoted text segment in the candidate text pair to obtain an embedding feature vector of the answer text passage and the embedding feature vector of the quoted text segment, where the fourth language model can be a text-based BERT model (Sentence-BERT) or other text encoding model.

[0144] In step 10822A, a pooling process is performed on the embedded feature vector of the answer text passage and the embedded feature vector of the quoted text segment, respectively, to obtain a semantic feature vector of the answer text passage and a semantic feature vector of the quoted text segment.

[0145] Because the text length or text scale of the answer text passage and the quoted text segment may be significantly different, a pooling layer performs pooling processing on the embedded feature vectors of the answer text passage and the quoted text segment, respectively, so that the dimension of the embedded feature vector of the answer text passage is the same as that of the embedded feature vector of the quoted text segment, which facilitates the determination of the similarity of the subsequent vectors and reduces the amount of calculation. After the pooling processing is performed by the pooling layer, the semantic feature vector of the answer text passage and the semantic feature vector of the quoted text segment are obtained.

[0146] In step 10823A, a splicing process is performed on the semantic feature vector of the quoted text segment and the semantic feature vector of the answer text passage to obtain a fused feature vector of the candidate text pair.

[0147] Following the above example, after determining the semantic feature vector of the answer text passage and the semantic feature vector of the quoted text segment, considering that the dimensions of the two semantic feature vectors are the same, we can directly perform a splicing process on the two semantic feature vectors to obtain a fusion feature vector of the text pair.

[0148] In step 10824A, a mapping process is performed on the fused feature vector to obtain the semantic similarity between the answer text passage and the quoted text segment in the candidate text pair.

[0149] Following the above example, after obtaining the fusion feature vector of each candidate text pair, a single layer neural network (fully connected layer) or a multi-layer perceptron can be invoked to perform a mapping process on the fusion feature vector, and the fusion feature vector can be mapped into a classification probability, which is the semantic similarity between the answer text passage and the quoted text segment in the candidate text pair, where the activation function of the fully connected layer can be a Softmax function.

[0150] In some embodiments, there is no need to stitch together the semantic feature vector of the answer text passage and the semantic feature vector of the quoted text segment, i.e., there is no need to construct a fusion feature vector, but the two semantic feature vectors can be processed directly, for example, by calculating cosine-similarity or edit distance and using it as the semantic similarity.

[0151] In step 10825A, candidate text pairs whose semantic similarity is greater than a semantic similarity threshold are determined as matching text pairs.

[0152] For each candidate text pair, the semantic similarity of the answer text passage and the quoted text segment in the candidate text pair is determined, and then the candidate text pairs are screened using a predetermined semantic similarity threshold. Text pairs with semantic similarity greater than the semantic similarity threshold are determined as matching text pairs. The semantic similarity threshold can be set based on the number of candidate text pairs, and the set semantic similarity threshold is positively correlated with the number of candidate text pairs. That is, the larger the number of candidate text pairs, the higher the semantic similarity threshold can be set, and the fewer the number of candidate text pairs, the lower the semantic similarity threshold can be set. For example, if there are too many candidate text pairs, such as 1,000 candidate text pairs, the semantic similarity threshold can be set relatively high (e.g., 0.9). If there are only a small number of text pairs, such as only 500 candidate text pairs, the semantic similarity threshold can be set relatively low (e.g., 0.7).

[0153] In some embodiments, referring to FIG. 3L, FIG. 3L is an exemplary flowchart of a text processing method according to an embodiment of the present application, and the step of "inserting a quoted text segment that matches at least one answer text passage into the answer text" in step 108 shown in FIG. 3I can be realized through the following steps 1081B to 1082B, which will be described in detail below.

[0154] In step 1081B, in response to the number of the at least one answer text passage being less than or equal to the first quantity threshold, quoted text segments that match the at least one answer text passage are inserted at positions following the at least one answer text passage, respectively.

[0155] In some embodiments, through the determined matching text pairs, the quoted text segments in the matching text pairs can be used as citation material for the corresponding answer text passages. Specifically, based on the matching text pairs, answer text passages that match the quoted text segments can be determined from the answer text, and thus the citation position of each quoted text segment in the answer text can be determined.

[0156] Considering that the number of answer text passages in the answer text may be large, adding citation material to each answer text passage would be too computationally expensive.In an embodiment of the present application, a first quantity threshold for an answer text passage is set, and if the number of answer text passages in the answer text is equal to or less than the first quantity threshold, the citation text segment in the answer text can be spliced ​​into the corresponding answer text passage based on the matching text pair and then used as citation material, where the expression form of the citation material can be a specific symbol identifier (for example, a text superscript or a text subscript), or can use some quotation marks (for example, "[]" or "||"), which can be used to quickly identify and find the corresponding citation material.

[0157] In step 1082B, in response to the number of quoted text segments that match the at least one answer text passage being greater than a first quantity threshold, the quoted text segments that each match the at least one answer text passage are inserted together at the end of the answer text.

[0158] Following the above embodiment, if the number of answer text passages in the answer text is greater than a first quantity threshold, quoted text segments that respectively match at least one answer text passage are inserted sequentially at the end of the entire answer text to serve as quoted material for the answer text, and the insertion manner of the quoted text segments does not need to be in order.

[0159] In some embodiments, each answer text passage is paired with two quoted text segments, meaning that one answer text passage may have multiple quoted text segments that can be combined with it, i.e., multiple quoted material in the answer text passage. However, if the number of quoted material is too large, it may be detrimental to the presentation of the answer text and increase the size and reading volume of the text. Therefore, in some embodiments, before inserting matching quoted text segments into the answer text passage, a second quantity threshold is set to limit the number of quoted text segments in the answer text passage. If the number of quoted text segments matching at least one answer text passage is greater than the second quantity threshold, the answer text passages are sorted in descending order according to the similarity between the at least one answer text passage and the matching quoted text segments. From the header of the descending sorted results, quoted text segments that match a predetermined number or a predetermined percentage of the answer text passages are selected as text segments to be inserted. Finally, these text segments to be inserted are inserted at positions after the corresponding answer text passage.

[0160] For example, suppose there are six quoted text segments that match one answer text passage. The six quoted text segments can be inserted sequentially after the answer text passage in descending order of semantic similarity between the quoted text segments and the answer text passage, so that the number of quoted texts in the answer text passage is six. In this case, the second quantity threshold is set to 2. If the number of quoted text segments is greater than the second quantity threshold, a predetermined number (e.g., 2) or a predetermined proportion (e.g., half, 50%) of quoted text segments are selected as text segments to be inserted, and finally inserted at a position after the answer text passage.

[0161] In some embodiments, step 1081B and step 1082B shown in FIG. 3L can be performed in parallel, with no distinction made between the order of execution.

[0162] According to an embodiment of the present application, texts that have a high similarity to the text generated by the first language model (i.e., the answer text) are screened from the provided candidate citation texts and used as citation material to add the citation content to the relevant answer text passage in the generated text. On the one hand, this solves the problem that the content generated by the first language model lacks citation information, making it difficult to determine its authenticity or gain a deeper understanding. On the other hand, by using the citation content as reference material for the text generated by the first language model, this improves the authenticity and reliability of the text generated by the first language model, and solves the problem that the factuality and reliability of conventional generated texts are low and there is a possibility of fabrication.

[0163] The following describes an exemplary application of the present embodiment in one practical application scenario.

[0164] In application scenarios such as some search systems and dialogue systems, related technologies typically use large-scale language models to learn and understand natural language and automatically generate high-quality text based on a given context or question. This can realize various functions, such as code creation, essay generation, and role-playing, significantly improving search information retrieval efficiency and dialogue quality. However, large-scale language models are typically based on a Transformer-encoder architecture, where multi-head attention layers are stacked on very deep neural networks, potentially reaching hundreds of millions of parameters, making model deployment and training relatively difficult. Furthermore, large-scale language models lack the ability to acquire information or self-update. If the model is updated infrequently, it can only generate text within a fixed range of existing natural language, resulting in low controllability. Because the model cannot incorporate and learn new natural language knowledge in a timely manner, there may be an information gap between the generated text and the given text, resulting in poor timeliness of the final generated text content. In addition to the low timeliness, the generated text lacks factuality, making it less reliable and leading to problems such as fabrication, which affects the performance of large-scale language models.

[0165] In view of this scenario, an embodiment of the present application provides a text processing method that collects time-sensitive search information, screens corresponding answer texts as reference texts for a model, and then guides a large-scale language model to generate controllable text content based on the given text, allowing the model to timely learn new search knowledge through the reference text and improve the timeliness of the generated text. Based on this, texts with high similarity to the generated text are further screened from the provided candidate texts as citation texts, and the citation content is added to the generated text content, thereby improving the authenticity and reliability of the generated text and solving the problems of conventional generated texts that are low in factuality and reliability and may contain fabrications.

[0166] Referring to FIG. 4, FIG. 4 is a processing flowchart of a text processing method according to an embodiment of the present application, which specifically includes six step processes, including a step of obtaining a query text, a step of obtaining a query result, a step of extracting an answer text passage, a step of determining a relevance score between the query text and the answer text passage, a step of invoking a large-scale language model to generate text, and a step of performing citation alignment on the generated text. The specific implementation of each step process will be described below with reference to FIG. 4.

[0167] Referring to FIG. 4, in step 401, the query text is obtained.

[0168] The query text (query) is usually associated with a corresponding application scenario. For example, in the case of a dialogue scenario, it may be a search keyword related to the topic of the dialogue scenario, or a search term consisting of multiple search keywords. The latest search information is obtained to extract corresponding answer text passages, which are then used to provide corresponding reference text to the large-scale language model. Here, a processing framework diagram for obtaining the latest search information and extracting answer text passages based on the query text can be seen in FIG. 5, which is a process diagram of the process of extracting answer text passages according to an embodiment of the present application.

[0169] Continuing to refer to FIG. 4, in step 402, a search engine interface is invoked to obtain search results.

[0170] As shown in Figure 5, after obtaining the query text, a related search engine interface can be called to obtain search results, where the search engine interface can be a related terminal browser, a search server, or other engine or function interface with search function. The search results are usually one segment of plain text, including one segment of body text and the corresponding text title.

[0171] In some embodiments, the search results of the search engine interface are sorted in order of creation time, i.e., the search information with the most recent creation time is sorted at the top, and then the search engine interface retrieves the 10 most recent related text search results, i.e., the "top-10 body text" shown in FIG. 5, in descending order of creation time.

[0172] Still referring to FIG. 4, in step 403, the answer text passage is extracted.

[0173] In the text search results, only a portion of the text is the answer to the query text, and the rest is all worthless text information unrelated to the query text. Therefore, first, answer text passages are extracted from the text search results.

[0174] As shown in Figure 5, the 10 related text search results (top-10 main texts) are used as the main text. Each input main text is first divided into 500 text segments, as shown in Figure 5, into text segment 1, text segment 2, ..., text segment n. Then, each text segment is input to a 12-layer BERT model (i.e., BERT-12-layer, equivalent to the second language model described above) along with the query text, and the BERT model is used to calculate the matching score (QS score) between each text segment and the query text. Because text cannot be directly input and processed into the BERT model, first, each text element (token) in the query text and each text segment is treated as a normal token, and all normal tokens are spliced ​​together using a start token (CLS) and a delimiter token (SEP). Then, an embedding process is performed to obtain an embedded feature vector for the spliced ​​text. The embedding feature vector is input to a 12-layer BERT model for encoding, resulting in a semantic feature vector. The semantic feature vector corresponding to the start token (CLS) is input to a fully connected layer to map to a matching score (QS Score) between the text segment and the query text, where the activation function of the fully connected layer may be a Softmax function. When the BERT model is invoked to encode the embedding feature vector of the spliced ​​text, it predicts the start probability (Start Score) and end probability (End Score) of each text element (Token) of the text segment in the spliced ​​text as a suitable start element of the answer text passage and a suitable end element of the answer text passage, respectively. Specifically, after encoding the embedding feature vector (Token Represent) of each text element of the text segment, the semantic feature vector obtained by encoding is mapped to the probability of it being the start element and the end element of the answer text passage using a fully connected layer.

[0175] As shown in Figure 5, after connecting each text segment and the query text, the text segment with the highest matching score (Max QS Score) between the text segment and the query text is determined. From this text segment, the element with the highest starting probability (Max Start Score) suitable as the starting element of the answer text passage is screened to determine it as the starting element of the answer text passage. At the same time, the element with the highest ending probability (Max End Score) suitable as the ending element of the answer text passage is screened to determine it as the ending element of the answer text. Finally, the text between the starting element and the ending element in the text segment is determined as the final answer text passage. This allows us to extract 10 text search results (top-10 body texts), that is, 10 answer text passages corresponding to the 10 body texts.

[0176] Continuing with reference to FIG. 4, in step 404, a relevance score between the query text and the answer text passage is determined.

[0177] After obtaining the 10 answer text passages, the answer text passages related to the query text are screened as reference texts for the large-scale language model, specifically, the relevance scores between the query text and the answer text passages are determined respectively.

[0178] The process of determining the relevance score between a query text and an answer text passage can be seen in Figure 6, which is a process diagram of determining the relevance score between a query text and an answer text passage according to an embodiment of the present application. First, for each answer text passage, the elements of the query text (Query shown in Figure 6), the title of the answer text passage (Title shown in Figure 6), and the answer text passage (Passage shown in Figure 6) are treated as normal tokens, and the normal tokens between these three are joined by delimiter tokens (SEPs) to obtain a joined text token sequence. A start token (CLS) is inserted at the beginning of the joined text token sequence, and an embedding process is performed on the joined text token sequence to obtain a joined text embedding feature vector. This is then input to a cross-encoder 601 for encoding, and a corresponding semantic feature vector is obtained. The cross-encoder here corresponds to the third language model mentioned above. Finally, the semantic feature vector corresponding to the start token (CLS) is input to the multi-layer perceptron 602, which maps it to a relevance score between the query text and the answer text passage, S q,t+p where the relevance score is the relevance score fitted by the multi-layer perceptron.

[0179] In some embodiments, the cross-encoder is not limited to a specific deep neural network model type or topology structure, and can be replaced with various other model structures. For example, a non-cross-encoder can be used, or the number of layers in the network model can be increased or decreased, or the structure of the deep network model can be adjusted according to the actual needs of the model. The cross-encoder is trained using text pairs consisting of 50 million query texts and answer text passages, and the training data is obtained by data mining or manual precise annotation.

[0180] After determining the relevance scores with the query text for each of the 10 answer text passages, a relevance score threshold is used to screen the answer text passages, and those answer text passages whose relevance scores are lower than the relevance score threshold are screened out, and the remaining answer text passages can be used as reference texts for the large-scale language model.

[0181] Continuing with reference to FIG. 4, in step 405, a large-scale language model is invoked to generate text.

[0182] After the reference text is determined, the reference text and the query text are input into a large-scale language model for prediction, where the large-scale language model corresponds to the first language model mentioned above. The specific prediction process is as follows: First, each element of the reference text and the query text is treated as an ordinary token, and the ordinary tokens between the reference text and the query text are spliced ​​into a sequence connected by a separator token (SEP), and a start token (CLS) is inserted at the beginning of the connected sequence. Then, an embedding process is performed on the connected sequence to obtain an embedded feature vector of the connected sequence. Next, the embedded feature vector is input into a large-scale language model for encoding processing, and finally a semantic feature vector of the concatenated sequence is obtained. The semantic feature vector of each element is mapped to the start probability that it will be the start element of the answer text for the query text and the end probability that it will be the end element. The element corresponding to the maximum start probability is determined as the start element of the answer text, and the element corresponding to the maximum end probability is determined as the end element of the answer text. This makes it possible to predict the start position and end position in the reference text of the answer text for the query text, and the text between the start position and end position of the reference text is determined as the answer text, i.e., it is output as the generated text for the query text by the large-scale language model.

[0183] According to an embodiment of the present application, a query text is obtained, and a search engine interface is invoked to generate a plurality of recent text search results with timeliness prioritized, and then relevant answer text passages matching the query text are extracted from the text search results, and screening is performed, and from the plurality of answer text passages, a text segment having the highest relevance to the query text is determined as a reference text, which is used as the reference text for the large-scale language model, and then the large-scale language model is invoked to predict an answer text corresponding to the query text from the reference text, and generate the corresponding text, thereby improving the controllability of the content generated by the large-scale language model and the timeliness of the generated content.

[0184] Continuing with reference to FIG. 4, in step 406, citation alignment is performed on the generated text.

[0185] Considering that the text generated by a large-scale language model in the related art lacks veracity, problems such as low reliability and fabrication arise, which affect the performance effect of the large-scale language model. The embodiment of the present application screens provided candidate texts for texts that have a high similarity to the generated text to identify them as citation texts, and adds the citation content to the generated text content, thereby achieving citation alignment and improving the veracity and reliability of the generated text, thereby solving the problems of the conventional generated texts having low veracity and reliability and the possibility of fabrication.

[0186] Referring to FIG. 7, FIG. 7 is a flowchart of the process of performing citation alignment on generated text according to an embodiment of the present application, and the specific process of citation alignment will be introduced with reference to FIG.

[0187] In step 701, the answer text and candidate text are obtained, and the process proceeds to steps 702 and 705.

[0188] After the large-scale language model predicts the corresponding answer text based on the query text, it can search for search texts related to the query text, or invoke a text mining model based on the query text to mine the search texts, and these search texts can be used as candidate texts, where the candidate texts are used as quoted texts for the answer text.

[0189] In step 702, the sub-sentence segments are split and combined in pairs.

[0190] After obtaining the answer text and candidate text, the answer text and candidate text are each divided into multiple subsentences, i.e., the text is divided into multiple match-ready segments. The match-ready segments must contain the basic semantic information of the text, and the segment length should not be short (e.g., no longer than 10). This is because if a segment is too short, there will be too little reference information used by subsequent modules, which will disable the subsequent modules and prevent effective semantic extraction. The specific process is as follows: The text is divided by punctuation marks. If the length of a segment after division is less than 10, adjacent segments are combined into longer segments. The above process is repeated until the length of all match-ready segments exceeds 10 or the entire input text is contained in the match-ready segments.

[0191] After the segmentation process is completed, two pairs of segments waiting to be matched corresponding to the answer text and the candidate text are obtained. The two pairs of segmented subsentences (segments waiting to be matched) are combined two by two to obtain multiple candidate subsentence pairs. Specifically, each segmented subsentence in the answer text is combined with each segmented subsentence in the candidate text one by one to obtain multiple candidate subsentence pairs.

[0192] For example, if an answer text is divided into 20 subsentences, containing a total of 10 candidate texts, and each candidate text is divided into five subsentences, combining two sets of divided subsentences one-to-one results in a total of 1000 (20 x 10 x 5) candidate subsentence pairs.

[0193] In step 703, the candidate sub-sentence pairs are filtered.

[0194] The purpose of filtering matched sub-sentence pairs is to perform an initial screening of the obtained multiple candidate sub-sentence pairs to filter out phrases that are clearly not a match or are not important, thereby reducing the computational complexity of subsequent processing steps.

[0195] In some embodiments, the filtering method may be keyword-based key segment identification. Specifically, an unsupervised identification algorithm is invoked to identify the keywords of the subsentences corresponding to the answer text and the candidate text in the candidate subsentence pairs, respectively, and to determine the subsentences containing the keywords as key text segments. All candidate subsentence pairs are filtered if one of them is not a key text segment, indicating that the keywords of the two subsentences in the candidate subsentence pairs are different, and the candidate subsentence pairs are filtered. Here, the unsupervised identification algorithm may be a graph-based text processing sorting algorithm, such as a text ranking algorithm (TextRank), a term frequency-inverse document frequency (TF-IDF), or a rapid automatic keyword extraction algorithm (RAKE). These three methods may also be used in combination to more accurately extract keywords in the text.

[0196] In some embodiments, the filtering method may be a similarity matching method based on sentence embedding. Specifically, the filtering method may determine the edit distance between the two subsentences in a subsentence pair, determine the longest common subsequence between the two subsentences, or use both methods simultaneously. If the edit distance is less than a distance threshold or the longest common subsequence is greater than a certain number of sequences, it is determined that there is subsentence similarity. Because the similarity effect of text segment embedding for long sentences is low and it is difficult to specify length criteria for long sentences, the two subsentences in a candidate subsentence pair may be further divided to obtain multiple short sentences. Then, it is determined whether there is short sentence similarity. If there is one short sentence similarity, the corresponding candidate subsentence pair is retained; if there is no short sentence similarity, the candidate subsentence pair is filtered.

[0197] In step 704, the similarity of the candidate sub-sentence pairs is determined.

[0198] It is determined whether the subsentence of the candidate text can constitute a quotation of the subsentence of the answer text. More specifically, it is determined whether the semantics of the subsentence of the candidate text in the candidate subsentence pair are similar to or contain the semantics of the subsentence of the answer text. If they are similar or contain the semantics of the subsentence of the answer text, the candidate subsentence pair can be determined as a matching subsentence pair. Specifically, determining matching subsentence pairs from multiple candidate subsentence pairs involves using a semantic similarity matching model to calculate the semantic similarity of the two subsentences. If the calculated semantic similarity exceeds a similarity threshold, it indicates that the semantic of the subsentence of the answer text is similar to the semantic of the subsentence of the candidate text, that is, it indicates that the subsentence of the candidate text in the candidate subsentence pair can be used as a quotation of the corresponding subsentence of the answer text, and the candidate subsentence pair can be determined as a matching subsentence pair. The similarity threshold can be set according to actual needs, for example, 0.8.

[0199] In some embodiments, the semantic similarity matching model (corresponding to the above-mentioned fourth language model) may specifically be a Sentence-BERT model. The model prediction process can be seen in FIG. 8, which is a prediction process diagram of the semantic similarity matching model according to an embodiment of the present application. As shown in FIG. 8, first, two phrases A and B to be detected are input to the model, and then encoded in the hidden layer of the BERT model. Then, they are input to the pooling layer and pooled. Phrase A is mapped to an embedding vector u, and phrase B is mapped to an embedding vector v, where u and v have the same vector dimension. Then, vector splicing is performed on the embedding vector u and the embedding vector v, and a fused vector is output by the hidden layer, denoted as "|uv|." Here, a specific splicing method may be to add the elements in the embedding vector u to the elements at the corresponding positions in the embedding vector v, respectively. Then, the embedding vector u, the embedding vector v, and the fusion vector are directly spliced ​​together, and the spliced ​​features are input into a fully connected layer for predictive classification (Softmax classifier), thereby obtaining the semantic similarity between phrase A and phrase B through prediction, where the activation function of the fully connected layer is Softmax function, and the semantic similarity can be the minimum edit distance similarity between vectors, or the Euclidean distance similarity, or the cosine-similarity similarity.

[0200] In step 705, matching is performed based on citation flags in the answer text.

[0201] After determining that a subsentence of the candidate text in a matching subsentence pair can be used as quoted text for a subsentence of the corresponding answer text, the matching subsentence pair can be used to insert the subsentence of the candidate text as quoted text at the end of the subsentence of the corresponding answer text based on the citation flag of each subsentence in the answer text.

[0202] Steps 702 to 704 can be executed in parallel or in series with step 705, and the embodiment of the present application is not limited thereto.

[0203] In step 706, post-processing is performed on the matching sub-sentence pairs.

[0204] After determining all matching subsentence pairs that satisfy a similarity threshold from the candidate subsentence pairs, the subsentences of the candidate texts in the matching subsentence pairs can be used as citation texts of the subsentences of the corresponding answer texts, and at the same time, the position and quantity of the numbers in the citation texts are adjusted based on the answer texts.

[0205] Considering that matching subsentence pairs are obtained by combining two subsentences of the candidate text with two subsentences of the answer text, there may be multiple subsentences of the candidate text as quoted texts in one subsentence of the answer text, that is, when there are multiple quoted texts in one subsentence of the answer text, the corresponding matching subsentence pairs are sorted based on the semantic similarity, and the subsentence of the candidate text that has the highest semantic similarity with the subsentence of the answer text is directly added to the end of the subsentence of the answer text, and the remaining quoted texts (subsentences of the candidate text) are sorted in order of the magnitude of semantic similarity, thereby completing the post-processing of the matching subsentence pairs.

[0206] For example, if subsentence A in the answer text has multiple subsentences of the candidate text (e.g., subsentence B, subsentence C) as quoted texts, and the semantic similarity between subsentence A and subsentence C is 0.9 and the semantic similarity between subsentence A and subsentence B is 0.8, then subsentence C is added directly to the end of subsentence A as quoted text, and subsentence B is also sorted after subsentence C as quoted text. At the same time, subsentence C in the candidate text can be adjusted as citation 1 and subsentence B in the candidate text as citation 2, and the number of citations for subsentence A in the answer text can be set to 2.

[0207] In step 707, an answer text is generated with the quoted text.

[0208] After adding citation text to the subsentences of all answer texts, the large-scale language model generates answer texts based on the query text. Each subsentence in the answer text has a corresponding citation text, and the content of the corresponding citation text can be quickly queried based on the citation text number.

[0209] In accordance with an embodiment of the present application, the provided candidate texts are screened for texts that have a high similarity to the text generated by a given large-scale language model (i.e., the answer text), and the quoted texts are selected. The quoted texts are added to each sub-sentence in the generated text, thereby improving the authenticity and reliability of the text generated by the large-scale language model.

[0210] In summary, the embodiments of the present application obtain a query text and invoke a search engine interface to generate multiple time-first recent text search results, then extract relevant answer text passages that match the query text from the text search results, perform screening, determine from the multiple answer text passages the text segment that is most relevant to the query text as a reference text, and use it as the reference text for the large-scale language model, and then invoke the large-scale language model to predict an answer text corresponding to the query text from the reference text and generate the corresponding text, thereby improving the controllability and timeliness of the content generated by the large-scale language model. The provided candidate texts are then screened for texts that have a high similarity to the text generated by the large-scale language model (i.e., the answer text), and these texts are set as citations. The citations are added to each subsentence in the generated text, which solves the problem of the lack of citation information in the content generated by the large-scale language model, making it difficult to determine its authenticity or gain a deeper understanding. On the other hand, by using the citations as reference material for the text generated by the large-scale language model, the authenticity and reliability of the text generated by the large-scale language model are improved, solving the problem of the low factuality and reliability of conventional generated texts, and the possibility of fabrication. Furthermore, the text processing method according to the embodiments of the present application can be applied to a search system to quickly screen reference material from search results through query text, greatly improving the efficiency of retrieving search information and improving the search system experience.

[0211] The following will now describe an exemplary structure of the text processing device 453 implemented as a software module according to an embodiment of the present application. In some embodiments, the software modules in the text processing device 453 stored in the memory 450 as shown in FIG. 2 include: The system may include an acquisition module 4531 configured to acquire a query text, invoke a search engine interface based on the query text, obtain a plurality of text search results corresponding to the query text, and acquire a plurality of answer text passages matching the query text from the plurality of text search results; a determination module 4532 configured to determine a relevance between the query text and each of the plurality of answer text passages, and set the answer text passage corresponding to the maximum relevance as a reference text for the query text; and a call module configured to invoke a first language model based on the query text and the reference text, and obtain an answer text for the query text.

[0212] In some embodiments, the obtaining module 4531 is further configured to invoke a search engine interface based on the query text, cause the search engine interface to retrieve a plurality of text search results related to the query text in a creation time sorted manner, and obtain a plurality of text search results related to the query text from the search engine interface.

[0213] In some embodiments, the obtaining module 4531 is further configured to divide the text search result into a plurality of candidate cited text segments of fixed length, obtain a matching score between the query text and each of the plurality of candidate cited text segments, and a start probability and an end probability that each element in the candidate cited text segment will be the start position and end position of an answer text passage, determine a best matching segment, where the best matching segment is the candidate cited text segment corresponding to the largest matching score, determine a start element and an end element in the best matching segment, where the start element is the element corresponding to the largest start probability and the end element is the element corresponding to the largest end probability, and determine a portion between the start element and the end element in the best matching segment as the answer text passage that matches the query text.

[0214] In some embodiments, the acquisition module 4531 is further configured to combine each of the plurality of candidate cited text segments with the query text to obtain a text pair, invoke a second language model based on each text pair, and obtain a matching score between the query text and each of the plurality of candidate cited text segments, and a start probability and an end probability that each element in the candidate cited text segment will be the start position and end position of the answer text passage.

[0215] In some embodiments, the obtaining module 4531 is further configured to treat each element in the query text and the candidate cited text segment as a regular token, splice the regular tokens of each element into a first token sequence, where a start token is inserted at the beginning of the first token sequence, perform an embedding process on the first token sequence to obtain an embedded feature vector of the first token sequence, invoke a second language model based on the embedded feature vector of the first token sequence to perform an encoding process to obtain a semantic feature vector of the first token sequence, perform a mapping process on the semantic feature vector of the start token in the semantic feature vector of the first token sequence to obtain a matching score between the query text and the candidate cited text segment, perform a mapping process on the semantic feature vector corresponding to each regular token in the semantic feature vector of the first token sequence, and obtain a start probability that each element in the candidate cited text segment will be the start position and the end position of the answer text passage.

[0216] In some embodiments, the determination module 4532 is further configured to treat each element of the query text, the answer text passage, and the title text as one normal token, splice the normal tokens of each element into a second token sequence, in which a start token is inserted at the beginning of the second token sequence, and the normal tokens between the query text and the answer text passage, and between the answer text passage and the title text are connected by delimiter tokens; perform an embedding process on the second token sequence to obtain an embedded feature vector of the second token sequence; invoke a third language model to perform an encoding process on the embedded feature vector of the second token sequence to obtain a semantic feature vector of the second token sequence; and perform a mapping process on the semantic feature vector of the second token sequence to obtain a relevance between the query text and the answer text passage.

[0217] In some embodiments, the invocation module 4533 is configured to invoke a first language model based on the query text and the reference text to perform a prediction process on the query text, determine a start element and an end element in the reference text of the answer text, and determine the text between the start element and the end element of the reference text as the answer text for the query text.

[0218] In some embodiments, the calling module 4533 further treats each element of the query text and the reference text as one normal token, and splices the normal tokens into a third token sequence, where a start token is inserted at the beginning of the third token sequence, and the normal tokens between the query text and the reference text are connected by a delimiter token; An embedding process is performed on the third token sequence to obtain an embedded feature vector of the third token sequence, a first language model is called based on the embedded feature vector of the third token sequence to perform encoding process, a semantic feature vector of each normal token in the third token sequence is obtained, a mapping process is performed on the semantic feature vector corresponding to each normal token, and a start probability and an end probability that each element of the reference text will be the start element and the end element of the answer text are obtained, and the element corresponding to the maximum start probability is set as the start element of the answer text, and the element corresponding to the maximum end probability is set as the end element of the answer text.

[0219] In some embodiments, the determination module 4532 is further configured to obtain a plurality of candidate quoted texts, where the candidate quoted texts include material cited in the answer text, divide the answer text into a plurality of answer text passages, divide each candidate quoted text into a plurality of quoted text segments, determine a quoted text segment that matches at least one answer text passage, and insert the quoted text segment that matches the at least one answer text passage into the answer text.

[0220] In some embodiments, the determination module 4532 further combines the plurality of answer text passages and the plurality of quoted text segments in pairs to form a plurality of candidate text pairs, identifies at least one matching text pair from the plurality of candidate text pairs, and determines that the quoted text segment in the matching text pair matches the answer text passage in the matching text pair.

[0221] In some embodiments, the determination module 4532 is further configured to identify keywords for the answer text passage and the quoted text segment, respectively, in the candidate text pair, and remove the candidate text pair in response to no keywords being identified from at least one of the answer text passage and the quoted text segment.

[0222] In some embodiments, the determination module 4532 is further configured to perform an encoding process on the answer text passage and the quoted text segment in the candidate text pair, respectively, to obtain an embedding feature vector of the answer text passage and an embedding feature vector of the quoted text segment, determine a similarity between the embedding feature vector of the answer text passage and the embedding feature vector of the quoted text segment, and remove the candidate text pair in response to the similarity being less than a similarity threshold.

[0223] In some embodiments, the determination module 4532 is further configured to invoke a fourth language model to perform an encoding operation on the answer text passage and the candidate quoted text segment in the candidate text pair, to obtain an embedding feature vector of the answer text passage and an embedding feature vector of the quoted text segment, to perform a pooling operation on the embedding feature vector of the answer text passage and the embedding feature vector of the quoted text segment, respectively, to obtain a semantic feature vector of the answer text passage and a semantic feature vector of the quoted text segment, to perform a splicing operation on the semantic feature vector of the quoted text segment and the semantic feature vector of the answer text passage, to obtain a fused feature vector of the candidate text pair, to perform a mapping operation on the fused feature vector, to obtain a semantic similarity between the answer text passage and the quoted text segment in the candidate text pair, and to determine the candidate text pairs whose semantic similarity is greater than a semantic similarity threshold as matching text pairs.

[0224] In some embodiments, the determination module 4532 is further configured to, in response to the number of the at least one answer text passage being less than or equal to the first quantity threshold, insert the quoted text segments that match the at least one answer text passage at a position following the at least one answer text passage, and in response to the number of the quoted text segments that match the at least one answer text passage being greater than the first quantity threshold, insert the quoted text segments that match the at least one answer text passage together at the end of the answer text.

[0225] In some embodiments, the determination module 4532 is further configured, in response to the number of the at least one answer text passage being greater than a second quantity threshold, to sort the at least one answer text passage and the matching quoted text segments in descending order according to their similarity, and from the headers of the descending sorted results, select quoted text segments that match a predetermined number or a predetermined percentage of the answer text passages as text segments to be inserted.

[0226] An embodiment of the present application provides a computer program product including a computer program or computer-executable instructions stored in a computer-readable storage medium, wherein a processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions to cause the electronic device to perform the above-described text processing method of the embodiment of the present application.

[0227] An embodiment of the present application provides a computer-readable storage medium having computer-executable instructions or a computer program stored thereon, the computer-readable storage medium storing the computer-executable instructions or the computer program, which, when executed by a processor, causes the processor to perform a text processing method according to an embodiment of the present application, for example, the text processing method shown in Figures 3A to 3L.

[0228] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM, or may be any of a variety of devices including one or any combination of the above memories.

[0229] In some embodiments, the computer-executable instructions may take the form of a program, software, software module, script, or code, and may be written in any form of programming language (including compiled or interpreted, or declarative or procedural languages), and deployed in any form, such as as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0230] As an example, the computer-executable instructions may correspond to a file in a file system, but not necessarily, and may be stored in part of a file that stores other programs or data, such as in one or more scripts within a HyperText Markup Language (HTML) document, in a single file of the program discussed, or in multiple joint files (e.g., files that store one or more modules, subprograms, or code portions).

[0231] As an example, the computer-executable instructions may be arranged to be executed on one electronic device, or on multiple electronic devices at the same location, or on multiple electronic devices distributed across multiple locations and interconnected by a communications network.

[0232] In summary, the embodiments of the present application obtain a query text and invoke a search engine interface to generate a plurality of time-sensitive text search results, then extract relevant answer text passages matching the query text from the text search results, perform screening, determine a text segment from the plurality of answer text passages that is most relevant to the query text as a reference text, and use it as the reference text for a large-scale language model, and then invoke the large-scale language model to predict an answer text corresponding to the query text from the reference text and generate the corresponding text, thereby improving the controllability and timeliness of the content generated by the large-scale language model. The provided candidate citation texts are screened for those that have a high similarity to the text generated by the large-scale language model (i.e., the answer text), and the citations are added to each subsentence in the generated text. On the one hand, the problem of the lack of citation information in the content generated by the large-scale language model making it difficult to determine its authenticity or for a deeper understanding is solved. On the other hand, by using the citations as citations in the text generated by the large-scale language model, the authenticity and reliability of the text generated by the large-scale language model are improved, and the problem of the low factuality and reliability of the text generated by the conventional text and the possibility of fabrication is solved. Furthermore, the text processing method according to the embodiments of the present application can be applied to a search system to quickly screen reference materials from search results through query text, greatly improving the efficiency of retrieving search information and improving the search system experience.

[0233] The above description is merely an example of the present application and does not limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and scope of the present application shall all be included in the scope of protection of the present application.

Claims

1. A text processing method executed by an electronic device, comprising: obtaining a query text; invoking a search engine interface based on the query text to obtain a plurality of text search results corresponding to the query text; obtaining a plurality of answer text passages from the plurality of text search results that match the query text; determining a relevance between the query text and each of the plurality of answer text passages, and determining the answer text passage corresponding to the maximum relevance as a reference text of the query text; and invoking a first language model based on the query text and the reference text to obtain an answer text to the query text.

2. Invoking a search engine interface based on the query text and obtaining a plurality of text search results corresponding to the query text, Invoking a search engine interface based on the query text, and causing the search engine interface to retrieve a plurality of text search results related to the query text in a creation time sorted manner; and obtaining from the search engine interface a plurality of text search results related to the query text.

3. obtaining a plurality of answer text passages matching the query text from the plurality of text search results, For each of said text search results, dividing the text search results into a plurality of candidate citation text segments of fixed length; obtaining a matching score between the query text and each of the plurality of candidate cited text segments, and a start probability and an end probability of each element in the candidate cited text segment being the start location and the end location of an answer text passage; determining a best matching segment, the best matching segment being the candidate cited text segment corresponding to the highest matching score; determining a start element and an end element in the best matching segment, the start element being the element corresponding to the start probability of a maximum value and the end element being the element corresponding to the end probability of a maximum value; and determining a portion of the best-matching segment located between the start element and the end element as an answer text passage that matches the query text.

4. The step of obtaining a matching score between the query text and each of the plurality of candidate cited text segments and a start probability and an end probability of each element in the candidate cited text segment being a start position and an end position of an answer text passage, includes: combining each of the plurality of candidate quoted text segments with the query text as a text pair; and invoking a second language model based on each of the text pairs to obtain a matching score between the query text and each of the plurality of candidate cited text segments, and a start probability and an end probability of each element in the candidate cited text segment being the start and end location of an answer text passage.

5. Invoking a second language model based on each of the text pairs to obtain a matching score between the query text and each of the plurality of candidate cited text segments, and a start probability and an end probability of each element in the candidate cited text segment being a start position and an end position of an answer text passage, For each of said candidate quoted text segments: treating each element in the query text and each element in the candidate quoted text segment as a regular token and concatenating the regular tokens of each element into a first token sequence, wherein a start token is inserted at the beginning of the first token sequence; performing an embedding process on the first token sequence to obtain an embedded feature vector of the first token sequence; Invoking the second language model based on the embedding feature vector of the first token sequence to perform an encoding process to obtain a semantic feature vector of the first token sequence; performing a mapping process on the semantic feature vector of the starting token in the semantic feature vector of the first token sequence to obtain a matching score between the query text and the candidate quoted text segment; and performing a mapping process on the semantic feature vectors of the first token sequence corresponding to each of the normal tokens to obtain a start probability and an end probability that each element in the candidate cited text segment will be the start position and end position of an answer text passage, respectively.

6. The answer text passages include title text, and determining a relevance between the query text and each of the plurality of answer text passages includes: For each answer text passage, a step of treating each element of the query text, the answer text passage, and the title text as one normal token, and connecting the normal tokens of each element as a second token sequence, wherein a start token is inserted at the beginning of the second token sequence, and normal tokens between the query text and the answer text passage, and between the answer text passage and the title text are connected by delimiter tokens; performing an embedding process on the second token sequence to obtain an embedded feature vector of the second token sequence; calling a third language model to perform an encoding process on the embedded feature vector of the second token sequence to obtain a semantic feature vector of the second token sequence; and performing a mapping process on the semantic feature vector of the second token sequence to obtain a relevance between the query text and the answer text passage.

7. Invoking a first language model based on the query text and the reference text to obtain an answer text to the query text, Invoking a first language model based on the query text and the reference text to perform a prediction process on the query text and determine a start element and an end element of an answer text in the reference text; The method of claim 1 , further comprising the step of: determining text between a start element and an end element of the reference text as the answer text to the query text.

8. The step of calling the first language model to perform a prediction process on the query text and the reference text and determining a start element and an end element of an answer text in the reference text includes: a step of treating each element of the query text and the reference text as one normal token and connecting the normal tokens as a third token sequence, wherein a start token is inserted at the beginning of the third token sequence and normal tokens between the query text and the reference text are connected by a delimiter token; performing an embedding process on the third token sequence to obtain an embedded feature vector of the third token sequence; Invoking the first language model to perform an encoding process based on the embedding feature vector of the third token sequence, and obtaining a semantic feature vector of each normal token in the third token sequence; performing a mapping process on the semantic feature vectors corresponding to each of the normal tokens to obtain a start probability that each element of the reference text will be a start element of the answer text and a stop probability that each element of the reference text will be an end element of the answer text; 8. The text processing method of claim 7, further comprising the step of: determining the element corresponding to the maximum value of the starting probability as the starting element of the answer text; and determining the element corresponding to the maximum value of the ending probability as the ending element of the answer text.

9. After invoking a first language model based on the query text and the reference text to obtain an answer text to the query text, the text processing method includes: obtaining a plurality of candidate citation texts, the candidate citation texts comprising material for citing in the answer text; Segmenting the answer text into a plurality of answer text passages and segmenting each of the candidate quoted texts into a plurality of quoted text segments; 9. The method of claim 1, further comprising the steps of: determining a quoted text segment that matches at least one answer text passage; and inserting the quoted text segment that matches the at least one answer text passage into the answer text.

10. determining a quoted text segment matching the at least one answer text passage; combining the plurality of answer text passages and the plurality of quoted text segments pairwise to form a plurality of candidate text pairs; identifying at least one matching text pair from the plurality of candidate text pairs and determining that the quoted text segment in the matching text pair matches the answer text passage in the matching text pair.

11. identifying at least one matching text pair from the plurality of candidate text pairs; calling a fourth language model to perform encoding processes on the answer text passage and the candidate quoted text segment in the candidate text pair, respectively, to obtain an embedded feature vector for the answer text passage and an embedded feature vector for the quoted text segment; performing a pooling process on the embedding feature vector of the answer text passage and the embedding feature vector of the quoted text segment, respectively, to obtain a semantic feature vector of the answer text passage and a semantic feature vector of the quoted text segment; performing a splicing process on the semantic feature vector of the quoted text segment and the semantic feature vector of the answer text passage to obtain a fused feature vector of the candidate text pair; performing a mapping process on the fused feature vector to obtain a semantic similarity between the answer text passage and the quoted text segment in the candidate text pair; and determining the candidate text pairs whose semantic similarity is greater than a semantic similarity threshold as matching text pairs.

12. Before identifying a matching text pair from the plurality of candidate text pairs, the text processing method comprises: For each of the candidate text pairs, performing the following process: performing keyword identification for each answer text passage and quoted text segment in the candidate text pair; and removing the candidate text pair in response to no keywords being identified from at least one of the answer text passage and the quoted text segment.

13. Before identifying a matching text pair from the plurality of candidate text pairs, the text processing method comprises: For each of the candidate text pairs, performing the following process: performing encoding processes on the answer text passage and the quoted text segment in the candidate text pair, respectively, to obtain an embedded feature vector of the answer text passage and an embedded feature vector of the quoted text segment; determining a similarity between the embedded feature vector of the answer text passage and the embedded feature vector of the quoted text segment; and removing the candidate text pair in response to the similarity being less than a similarity threshold.

14. inserting a quoted text segment matching the at least one answer text passage into the answer text, the step comprising: responsive to the number of the at least one answer text passage being less than or equal to a first quantity threshold, inserting quoted text segments that match the at least one answer text passage at positions following the at least one answer text passage; and inserting quoted text segments that respectively match the at least one answer text passage together at the end of the answer text in response to the quantity of the at least one answer text passage being greater than the first quantity threshold.

15. Prior to inserting a quoted text segment matching the at least one answer text passage into the answer text, the text processing method comprises:

14. The text processing method of claim 9, further comprising: in response to a number of quoted text segments matching the at least one answer text passage being greater than a second quantity threshold, sorting the at least one answer text passage and the matching quoted text segments in descending order according to similarity, and selecting, from the headers of the descending sorted result, quoted text segments that match a predetermined number or a predetermined percentage of the answer text passages as text segments to be inserted.

16. A text processing apparatus comprising: an acquisition module; a determination module; and a calling module; the acquisition module is configured to acquire query text; the retrieval module is further configured to invoke a search engine interface based on the query text to obtain a plurality of text search results corresponding to the query text; the retrieval module is further configured to retrieve a plurality of answer text passages matching the query text from the plurality of text search results; the determining module is configured to determine a relevance between the query text and each of the plurality of answer text passages, and determine the answer text passage corresponding to the maximum relevance as a reference text of the query text; The text processing device, wherein the invocation module is configured to invoke a first language model based on the query text and the reference text to obtain an answer text to the query text.

17. An electronic device, a memory for storing computer-executable instructions or computer programs; A processor configured to execute computer-executable instructions or computer programs stored in said memory to perform the text processing method of any one of claims 1 to 15.

18. A computer-readable storage medium having stored thereon computer-executable instructions or a computer program that, when executed by a processor, causes the processor to perform the text processing method of any one of claims 1 to 15.

19. A computer program product comprising computer executable instructions or a computer program which, when executed by a processor, causes the processor to perform a text processing method according to any one of claims 1 to 15.

Images