A text information processing method, apparatus, electronic device, and storage medium

Abstract

This invention provides a text information processing method, apparatus, electronic device, and storage medium. The method includes: acquiring text information to be processed; performing word segmentation on the text information to be processed to determine at least one candidate keyword vector; dynamically adjusting the candidate keyword vectors to form a candidate keyword vector set; using a deep reinforcement learning network, determining a keyword extraction strategy that matches the text information to be processed based on the candidate keyword vector set and a reward value parameter; and extracting from the candidate keyword vector set based on the keyword extraction strategy to obtain at least one keyword vector as a keyword of the text information to be processed. This reduces the dependence on word granularity in keyword extraction, making the extracted keywords of the text information to be processed applicable to different usage scenarios and improving the user experience.

Description

Technical Field

[0001] This invention relates to information processing technology, and more particularly to text information processing methods, apparatus, electronic devices, and storage media. Background Technology

[0002] With the development of machine learning technology, machine learning can answer user questions through artificial intelligence mechanisms. Among these, understanding text queries is a core technology in NLP fields such as question-answering engines and dialogue systems. A more accurate understanding of user requests allows the system to provide better answers. Queries are typically text or sentences obtained through speech recognition. However, for common contextual types in multi-turn question answering, such as "context-independent questions," "repeated questions," "default follow-ups," and "reference questions," related technologies cannot accurately identify these context types, affecting the AI's processing of text information and resulting in poor accuracy of the processing results. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a text information processing method, apparatus, electronic device, and storage medium, which can determine the context type of associated text information by associating text information, thereby achieving more accurate processing results for the text information to be processed, making the processing results of the text information to be processed applicable to the current usage scenario, reducing the impact of missing associated text information on the generated response statement, and improving the user experience.

[0004] The technical solution of this invention is implemented as follows:

[0005] This invention provides a text information processing method, including:

[0006] Obtain the text information to be processed in the text processing context and the associated text information that matches the text to be processed;

[0007] Based on the text information to be processed and the associated text information, determine the context type of the associated text information that matches the associated text information;

[0008] Obtain candidate similar text information that matches the text information to be processed, and determine the similarity between the text information to be processed and the candidate similar text information based on the text information to be processed, the associated text information, and the context type of the associated text information;

[0009] Based on the similarity between the text information to be processed and the candidate similar text information, a response statement that matches the text information to be processed is determined.

[0010] This invention also provides a text information processing device, comprising:

[0011] The information transmission module is used to acquire the text information to be processed in the text processing context and the associated text information that matches the text to be processed;

[0012] The information processing module is used to determine the context type of the associated text information that matches the associated text information based on the text information to be processed and the associated text information;

[0013] The information processing module is used to acquire candidate similar text information that matches the text information to be processed, and to determine the similarity between the text information to be processed and the candidate similar text information based on the text information to be processed, the associated text information, and the context type of the associated text information.

[0014] The information processing module is used to determine a response statement that matches the text information to be processed based on the similarity between the text information to be processed and the candidate similar text information.

[0015] In the above scheme,

[0016] The information processing module is used to process the text information to be processed and the associated text information through the associated text information context type discriminator network of the text information matching model, and determine the associated text information context type that matches the associated text information; or,

[0017] The text information processing network of the text information matching model is used to determine the text information embedding vector corresponding to the text information to be processed.

[0018] The text information processing network of the text information matching model is used to determine the associated text information embedding vector corresponding to the associated text information;

[0019] The text information matching model uses a context type discriminator network to process the text information embedding vector and the associated text information embedding vector to determine the context type of the associated text information that matches the associated text information.

[0020] In the above scheme,

[0021] The information processing module is used to trigger the corresponding word segmentation library according to the text information parameters carried by the text information to be processed.

[0022] The information processing module is used to perform word segmentation on the text information to be processed based on the triggered word segmentation dictionary through the text information processing network of the text information matching model, forming different word-level text information embedding vectors to be processed.

[0023] The information processing module is used to perform noise reduction processing on the different word-level text information embedding vectors to be processed, forming a set of text information embedding vectors corresponding to the text information to be processed.

[0024] In the above scheme,

[0025] The information processing module is used to determine the name of the word segmentation library used when performing word segmentation processing on the text information to be processed;

[0026] The information processing module is used to determine the parameters of a word segmentation library that matches the word-level embedding vector of the text information to be processed, based on the name of the word segmentation library. The parameters of the word segmentation library include:

[0027] The types of word segmentation libraries, the names of word segmentation libraries, and the versions of word segmentation libraries.

[0028] In the above scheme,

[0029] The information processing module is used to determine a dynamic noise threshold that matches the text processing context.

[0030] The information processing module is used to perform noise reduction processing on the text information to be processed according to the dynamic noise threshold, and trigger a dynamic word segmentation strategy that matches the dynamic noise threshold.

[0031] The information processing module is used to perform word segmentation on the text information to be processed according to a dynamic word segmentation strategy that matches the dynamic noise threshold, so as to form a text information embedding vector corresponding to the text information to be processed.

[0032] In the above scheme,

[0033] The information processing module is used to determine a fixed noise threshold that matches the text processing context;

[0034] The information processing module is used to perform noise reduction processing on the text information to be processed according to the fixed noise threshold, and trigger a fixed word segmentation strategy that matches the fixed noise threshold.

[0035] The information processing module is used to perform word segmentation on the text information to be processed according to a fixed word segmentation strategy that matches the fixed noise threshold, so as to form a text information embedding vector corresponding to the text information to be processed.

[0036] In the above scheme,

[0037] The information processing module is used to determine the candidate similar text information embedding vector corresponding to the candidate similar text information through the text information processing network of the text information matching model;

[0038] The information processing module is used to process the text information embedding vector to be processed and the associated text information embedding vector through the multi-round matching network of the text information matching model to determine the first concatenation feature vector;

[0039] The information processing module is used to determine the similarity between the embedded vector of the text information to be processed and the embedded vector of the candidate similar text information based on the context type of the associated text information and the first concatenated feature vector through the multi-round matching network of the text information matching model.

[0040] In the above scheme,

[0041] The information processing module is used to determine the candidate similar text information embedding vector corresponding to the candidate similar text information through the text information processing network of the text information matching model;

[0042] The information processing module is used to process the text information embedding vector to be processed and the associated text information embedding vector through the multi-round matching network of the text information matching model to determine the first concatenation feature vector;

[0043] The information processing module is used to determine the similarity between the text information embedding vector to be processed and the candidate similar text information embedding vector based on the first concatenated feature vector through the multi-round matching network of the text information matching model.

[0044] The information processing module is used to sort the similarity between the embedding vector of the text information to be processed and the embedding vector of the candidate similar text information based on the context type of the associated text information.

[0045] The information processing module is used to determine the similarity between the text information to be processed and the candidate similar text information based on the sorting results.

[0046] In the above scheme, the device further includes:

[0047] The training module is used to obtain a training sample set, wherein the training sample set includes different text information samples that have been labeled with entities;

[0048] The training module is used to process the training sample set through the text information matching model to determine the initial parameters of the associated text information context type discriminator network and the initial parameters of the multi-round matching network in the text information matching model.

[0049] The training module is used to process the training sample set through the text information matching model based on the initial parameters of the associated text information context type discriminator network and the initial parameters of the multi-round matching network, and to determine the update parameters corresponding to the associated text information context type discriminator network and the update parameters corresponding to the multi-round matching network, respectively.

[0050] The training module is used to iteratively update the context type discriminator network and the multi-round matching network respectively using the training sample set, based on the update parameters corresponding to the context type discriminator network and the update parameters corresponding to the multi-round matching network.

[0051] In the above scheme,

[0052] The training module is used to determine the first loss function that matches the context type discriminator network of the associated text information;

[0053] The training module is used to determine the second loss function that matches the multi-round matching network;

[0054] The training module is used to determine the fusion loss function corresponding to the text information matching model based on the first loss function and the second loss function.

[0055] In the above scheme,

[0056] The training module is used to obtain a set of training samples that match the context type of the text information to be processed from the data source based on the context type of the associated text information.

[0057] The training module is used to determine the model parameters of a text information matching model that is adapted to the context type of the text information to be processed, based on the training sample set that matches the context type of the text information to be processed.

[0058] This invention also provides an electronic device, the electronic device comprising:

[0059] Memory, used to store executable instructions;

[0060] A processor, when executing executable instructions stored in the memory, implements a preceding text information processing method.

[0061] This invention also provides a computer-readable storage medium storing executable instructions, characterized in that the executable instructions, when executed by a processor, implement a preceding text information processing method.

[0062] The embodiments of the present invention have the following beneficial effects:

[0063] This invention acquires the text information to be processed within a text processing context, along with related text information matching the text to be processed. Based on the text information to be processed and the related text information, it determines the context type of the related text information matching the related text information. It then acquires candidate similar text information matching the text information to be processed, and based on the text information to be processed, the related text information, and the context type of the related text information, it determines the similarity between the text information to be processed and the candidate similar text information. Finally, based on the similarity between the text information to be processed and the candidate similar text information, it determines a response statement matching the text information to be processed. Therefore, by determining the context type of related text information through related text information, the processing effect on the text information to be processed can be more accurate, making the processing result applicable to the current usage scenario. It reduces the impact of missing related text information on the generated response statement, reduces generic replies, and thus improves the matching and forward-looking nature of the response, enhancing the user experience. Attached Figure Description

[0064] Figure 1 This is a schematic diagram illustrating a usage scenario of the text information processing method provided in an embodiment of the present invention;

[0065] Figure 2 This is a schematic diagram of the composition structure of the text information processing device provided in an embodiment of the present invention;

[0066] Figure 3 A schematic diagram illustrating the generation of response statements for a basic text information matching model in related technologies;

[0067] Figure 4 This is a schematic diagram of the data structure of the text information processing method provided in the embodiments of the present invention;

[0068] Figure 5 A schematic flowchart of an optional text information processing method provided in an embodiment of the present invention;

[0069] Figure 6 This is a schematic diagram illustrating text information processing via a mini-program in an embodiment of the present invention;

[0070] Figure 7 This is a schematic diagram of the data structure of the text information processing method provided in the embodiments of the present invention;

[0071] Figure 8 This is a schematic diagram of the data structure of the text information processing method provided in the embodiments of the present invention;

[0072] Figure 9 This is a schematic diagram of the text matching process in an embodiment of the present invention;

[0073] Figure 10A schematic flowchart of an optional text information processing method provided in an embodiment of the present invention;

[0074] Figure 11 This is a schematic diagram illustrating the application environment of the text information matching model in this embodiment of the invention;

[0075] Figure 12 This is a schematic diagram illustrating the working process of the text information matching model provided in an embodiment of the present invention. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0077] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0078] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0079] Before providing a further detailed description of the embodiments of the present invention, the nouns and terms involved in the embodiments of the present invention will be explained, and the nouns and terms involved in the embodiments of the present invention shall be interpreted as follows.

[0080] 1) In response to, used to indicate the conditions or states on which the operation performed depends. When the conditions or states on which it depends are met, one or more operations performed may be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order in which the multiple operations are performed.

[0081] 2) Word segmentation: Also known as word cutting, it functions to divide the text information of a complete sentence into multiple words. For example: XX is a Chinese singer. The result after word segmentation is: XX, China, singer.

[0082] 3) Word segmentation library: also known as word segmentation library, refers to a specific word segmentation method. Different word segmentation libraries have their own corresponding word dictionaries, and can perform word segmentation processing on the corresponding text information according to their respective word dictionaries.

[0083] 4) Token: Before any actual processing is performed on the input text, it needs to be divided into language units such as words, punctuation marks, numbers, or alphanumeric characters. These units are called tokens.

[0084] 5) Siamese Network: In a pseudo-Siamese network, the two neural networks corresponding to the input data can be different neural networks, or they can be the same type of neural network. Pseudo-Siamese networks are suitable for handling situations where the two inputs are "somewhat different". The text processing method in this paper is implemented in a pseudo-Siamese network.

[0085] 6) Model Training: Multi-class classification learning on the image dataset. This model can be built using deep learning frameworks such as TensorFlow and Torch, employing multiple layers of neural networks like CNNs to form a multi-class classification model. The model input is a three-channel or original-channel matrix generated from images read using tools like OpenCV. The model output is the multi-class probability, ultimately outputting the webpage category through algorithms such as softmax. During training, the model approximates the correct trend using objective functions such as cross-entropy.

[0086] 7) Neural Network (NN): Artificial Neural Network (ANN), also known as neural network or neural network-like network, is a mathematical or computational model in the fields of machine learning and cognitive science that imitates the structure and function of biological neural networks (the central nervous system of animals, especially the brain) and is used to estimate or approximate functions.

[0087] 8) Bidirectional Attention Neural Network Model (BERT Bidirectional Encoder Representations from Transformers): A bidirectional attention neural network model proposed by Google. Transformers: A new network structure that uses an attention mechanism instead of the traditional encoder-decoder model, which must rely on other neural networks. This method is widely used in various natural language processing tasks, such as text classification, text matching, and machine reading comprehension.

[0088] 9) A Mini Program is a program developed using a front-end-oriented language (such as JavaScript) and implemented within a Hyper Text Markup Language (HTML) page. It is downloaded by a client (such as a browser or any client with an embedded browser engine) via a network (such as the Internet) and interpreted and executed within the client's browser environment, saving the step of installation on the client side. For example, a Mini Program can be activated via voice command on a terminal, enabling the download and execution of various services such as airline ticket purchase, task processing and creation, and data display on a social network client.

[0089] Figure 1 This is a schematic diagram illustrating a usage scenario of the text information processing method provided in this embodiment of the invention. (See attached diagram.) Figure 1 The terminals (including terminals 10-1 and 10-2) are equipped with corresponding clients capable of performing different functions. These clients access various articles from the server 200 via network 300 for browsing or to retrieve mini-programs or public accounts stored on the server. When the terminal runs an instant messaging client process, it can obtain corresponding responses based on the input questions using the provided question-and-answer function. The responses can be candidate responses from various sources such as Moments, mini-programs, articles, and public accounts, or responses adapted to the medical application scenario obtained by the medical robot. The terminals connect to the server 200 via network 300, which can be a wide area network (WAN), a local area network (LAN), or a combination of both, using a wireless link for data transmission. The terminals (including terminals 10-1 and 10-2) can also have a chat software client. Users can input questions through the chat client, and the chat client can receive and display the responses to the user.

[0090] In some embodiments of the present invention, the different types of mini-programs stored in the server 200 can be written in software code environments of different programming languages, and the code objects can be different types of code entities. For example, in C language software code, a code object can be a function. In JAVA language software code, a code object can be a class, and in iOS OC language, it can be a piece of object code. In C++ language software code, a code object can be a class or a function to execute text information from different terminals. This application does not distinguish the source of the retrieval instructions. The mini-program in the instant messaging client process can trigger a question-and-answer engine. A mini-program is a program developed based on a front-end language (such as JavaScript), implementing services in a Hyper Text Markup Language (HTML) page. It is downloaded by the client (such as a browser or any client with an embedded browser core) via a network (such as the Internet) and interpreted and executed in the client's browser environment, saving the step of installation on the client. For example, by waking up a mini-program on the terminal via voice commands, users can download and run mini-programs in social network clients to provide various services such as ticket purchase, task processing and creation, and data display.

[0091] Server 200 sends a corresponding text message response statement to the terminal (terminal 10-1 and / or terminal 10-2) via network 300 based on the keywords identified by the terminal. As an example, server 200 can be used to obtain text information to be processed in a text processing context and related text information matching the text to be processed; based on the text information to be processed and related text information, determine the context type of related text information matching the related text information; obtain candidate similar text information matching the text information to be processed; based on the text information to be processed, related text information, and the context type of related text information, determine the similarity between the text information to be processed and the candidate similar text information; based on the similarity between the text information to be processed and the candidate similar text information, determine a response statement matching the text information to be processed.

[0092] The text information processing method provided in this application is based on artificial intelligence (AI). AI is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, AI is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making functions.

[0093] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0094] In the embodiments of this application, the main artificial intelligence software technologies involved include the aforementioned speech processing technologies and machine learning. For example, it may involve Automatic Speech Recognition (ASR) technology in speech technology, including speech signal preprocessing, speech signal frequency analyzing, speech signal feature extraction, speech signal feature matching / recognition, and speech training.

[0095] For example, this could involve machine learning (ML), a multidisciplinary field encompassing probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning typically includes techniques such as deep learning, which includes artificial neural networks, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and deep neural networks (DNNs).

[0096] It is understood that the text information processing method and voice processing provided in this application can be applied to intelligent devices. Intelligent devices can be any device with voice command recognition function, such as intelligent terminals, smart home devices (such as smart speakers, smart washing machines, etc.), smart wearable devices (such as smartwatches), in-vehicle intelligent central control systems (which wake up small programs in the terminal to perform different tasks through voice commands), or AI intelligent medical devices (which are woken up and triggered by voice commands), etc.

[0097] The structure of the text information processing device according to an embodiment of the present invention will be described in detail below. The text information processing device can be implemented in various forms, such as a dedicated terminal with text information processing function, or a server equipped with text information processing function, for example, the preceding... Figure 1 Server 200. Figure 2 This is a schematic diagram of the composition structure of the text information processing device provided in the embodiments of the present invention. It can be understood that... Figure 2 Only an exemplary structure of the text information processing device is shown, not the entire structure; it can be implemented as needed. Figure 2 The structure shown may be part or all of the structure.

[0098] The text information processing apparatus provided in this embodiment of the invention includes: at least one processor 201, a memory 202, a user interface 203, and at least one network interface 204. The various components in the text information processing apparatus are coupled together via a bus system 205. It can be understood that the bus system 205 is used to realize the connection and communication between these components. In addition to a data bus, the bus system 205 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 2 The general labeled all buses as Bus System 205.

[0099] The user interface 203 may include a monitor, keyboard, mouse, trackball, click wheel, buttons, touchpad, or touch screen.

[0100] It is understood that memory 202 can be volatile memory or non-volatile memory, or both. In this embodiment of the invention, memory 202 is capable of storing data to support the operation of a terminal (such as 10-1). Examples of this data include any computer programs used to operate on the terminal (such as 10-1), such as operating systems and applications. The operating system includes various system programs, such as the framework layer, core library layer, driver layer, etc., used to implement various basic services and handle hardware-based tasks. Applications can include various applications.

[0101] In some embodiments, the text information processing device provided in this invention can be implemented using a combination of hardware and software. For example, the text information processing device provided in this invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the text information processing method provided in this invention. For instance, the processor in the form of a hardware decoding processor can employ one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0102] As an example of the text information processing device provided in this embodiment of the invention, which is implemented by combining software and hardware, the text information processing device provided in this embodiment of the invention can be directly embodied as a combination of software modules executed by processor 201. The software modules can be located in a storage medium, which is located in memory 202. Processor 201 reads the executable instructions included in the software modules in memory 202 and combines them with necessary hardware (e.g., including processor 201 and other components connected to bus 205) to complete the text information processing method provided in this embodiment of the invention.

[0103] As an example, processor 201 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., wherein the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0104] As an example of the hardware implementation of the text information processing device provided in the embodiments of the present invention, the device provided in the embodiments of the present invention can be directly executed by a processor 201 in the form of a hardware decoding processor. For example, it can be executed by one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components to implement the text information processing method provided in the embodiments of the present invention.

[0105] In this embodiment of the invention, the memory 202 is used to store various types of data to support the operation of the text information processing device. Examples of such data include: any executable instructions for operation on the text information processing device, such as executable instructions that can be included in a program implementing the text information processing method of this embodiment of the invention.

[0106] In other embodiments, the text information processing apparatus provided in this invention can be implemented in software. Figure 2A text information processing device stored in memory 202 is shown. This device can be software in the form of programs and plugins, and includes a series of modules. As an example of a program stored in memory 202, it may include the text information processing device. The text information processing device includes the following software modules: an information transmission module 2081 and an information processing module 2082. When the software modules in the text information processing device are read into RAM and executed by processor 201, the text information processing method provided in this embodiment of the invention will be implemented. The functions of each software module in the text information processing device in this embodiment of the invention are described below.

[0107] The information transmission module 2081 is used to acquire the text information to be processed in the text processing context and the associated text information that matches the text to be processed.

[0108] The information processing module 2082 is used to determine the context type of the associated text information that matches the associated text information based on the text information to be processed and the associated text information.

[0109] The information processing module 2082 is used to acquire candidate similar text information that matches the text information to be processed, and to determine the similarity between the text information to be processed and the candidate similar text information based on the text information to be processed, the associated text information, and the context type of the associated text information.

[0110] The information processing module 2082 is used to determine a response statement that matches the text information to be processed based on the similarity between the text information to be processed and the candidate similar text information.

[0111] according to Figure 2 The electronic device shown, in one aspect of this application, also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform various embodiments and combinations of embodiments provided in the various optional implementations of the above-described text information processing method.

[0112] Combination Figure 2 The illustrated text information processing apparatus describes the text information processing apparatus method provided in the embodiments of the present invention. Before introducing the text information processing apparatus method provided in the embodiments of the present invention, we first introduce the process in related technologies where a traditional text information matching model is used to generate a corresponding response statement based on the question text. Figure 3This diagram illustrates how a text-based information matching model generates response statements in related technologies. Traditional end-to-end text retrieval methods based on contextual information primarily improve the matching effect of multi-turn information by optimizing the model's matching process. For example... Figure 3 As shown, multi-turn question-and-answer statements are merged into one column, separated by _SOS_, and the entire dialogue history is treated as a complete vector statement to match the next response statement. Then, a multi-view matching mechanism is introduced during the matching process to enhance the matching effect. Specifically, the entire dialogue history is merged into one column, and word-level features are extracted using a GRU module after character embedding vectors. These features are then matched with candidate response statements. This process uses only data-driven methods to enhance the model's matching effect. Therefore, this method has high data requirements, increases the computational burden on terminal devices, and is not conducive to the processing of text information on mobile terminals.

[0113] refer to Figure 4 , Figure 4 This is a data structure diagram of the text information processing method provided in this embodiment of the invention. The method of generating response statements based on contextual query rewriting mainly involves extracting or generating the current user query using contextual information, and replacing the original query with the rewritten query. This method also transforms multi-round matching into single-round matching. Specifically, user input A1 and system response B1 are first concatenated, and then used together with user input A2 (the statement to be rewritten) as input to the rewriting model, with the output being the rewritten query. However, in related technologies, the effect of the rewriting model is uncontrollable. During the rewriting process, a large amount of rewriting bias is introduced, leading to new noise in multi-round dialogues, and it is also impossible to adapt to the contextual information and language type of the related text.

[0114] To overcome the above-mentioned shortcomings, see Figure 5 , Figure 5 This is an optional flowchart illustrating a text information processing method provided in an embodiment of the present invention. It can be understood that... Figure 5 The steps shown can be performed by various electronic devices running text messaging processing devices, such as dedicated terminals with text message reply capabilities, mobile phones, or communication devices running search function applets. The following addresses... Figure 5 The steps shown are explained.

[0115] Step 501: Obtain the text information to be processed in the text processing context and the associated text information that matches the text to be processed.

[0116] The source of the text information to be processed can vary depending on the text processing context. For example, in a smart medical terminal, the text information to be processed could be diagnostic questions entered by the patient. In a media asset usage environment on a mobile terminal, the text information to be processed could be voice-based questions entered by the user and received by a smart speaker or in-vehicle intelligent system.

[0117] In some embodiments of the present invention, a corresponding word segmentation library can be triggered based on the text information parameters carried by the text information to be processed; the text information processing network of the text information matching model is used to segment the text information to be processed based on the triggered word segmentation library dictionary to form different word-level text information embedding vectors; the different word-level text information embedding vectors are subjected to noise reduction processing to form a set of text information embedding vectors corresponding to the text information to be processed. Specifically, the name of the word segmentation library used for segmenting the text information to be processed is determined; based on the name of the word segmentation library, the parameters of the word segmentation library that match the word-level text information embedding vectors corresponding to the text information to be processed are determined, wherein the parameters of the word segmentation library include: the type of word segmentation library, the name of the word segmentation library, and the version of the word segmentation library.

[0118] refer to Figure 6 , Figure 6 This is a schematic diagram illustrating text information processing via a mini-program in an embodiment of the present invention. As described in the preceding embodiments, different terminal devices (e.g., those in the preceding embodiments) Figure 1 The terminals 10-1 and / or 10-2 shown can provide a question-and-answer bar for inputting keywords on their respective question-and-answer interfaces (e.g., web pages, information question-and-answer apps, and WeChat question-and-answer mini-programs), as well as a question-and-answer button for data-based question-and-answering based on those keywords. Users input text information in the question-and-answer bar via voice commands. When the terminal device detects a click on the question-and-answer button, it triggers the server to initiate a corresponding word segmentation instruction. This word segmentation instruction carries the keywords from the question-and-answer bar, and the server receives the instruction. Alternatively, the terminal device can display popular question-and-answer keywords on the question-and-answer interface. When a click on a popular question-and-answer keyword is detected, the terminal device sends the word segmentation instruction to the server, carrying the popular question-and-answer keyword, and the server receives the instruction. It should be noted that this embodiment of the invention does not limit the triggering method of the word segmentation instruction.

[0119] Since the word-level feature vectors formed when processing the same text information using different word segmentation libraries are not completely identical, the parameters of the word segmentation library that match the word-level feature vectors corresponding to the text information to be processed are determined based on the name of the word segmentation library. This determines the parameters of the word segmentation library used for word segmentation of the text information to be processed. For example, if the text information to be processed is "What to do when your credit card expires", after processing with word segmentation library A, a set of word-level feature vectors A (credit card; expired; what to do) corresponding to the text information to be processed is formed; after processing with word segmentation library B, a set of word-level feature vectors B (credit card expired; what to do) corresponding to the text information to be processed is formed; and after processing with word segmentation library A1, a set of word-level feature vectors A1 (credit card, what to do when expired) corresponding to the text information to be processed is formed.

[0120] In some embodiments of the present invention, a dynamic noise threshold matching the text processing context can be determined; the text information to be processed is denoised according to the dynamic noise threshold, and a dynamic word segmentation strategy matching the dynamic noise threshold is triggered; the text information to be processed is segmented according to the dynamic word segmentation strategy matching the dynamic noise threshold to form a text information embedding vector corresponding to the text information to be processed. Alternatively, a fixed noise threshold matching the text processing context can be determined; the text information to be processed is denoised according to the fixed noise threshold, and a fixed word segmentation strategy matching the fixed noise threshold is triggered; the text information to be processed is segmented according to the fixed word segmentation strategy matching the fixed noise threshold to form a text information embedding vector corresponding to the text information to be processed. When the word vector processing network is embedded in a corresponding hardware system, such as an intelligent triage system in a hospital, and the usage environment is to process spoken instructions related to medical information and extract keywords, the noise is relatively simple. By fixing the corresponding fixed noise threshold of the word vector processing network, the training speed of the word vector processing network can be effectively improved, the user's waiting time can be reduced, and the word vector processing network can be more adapted to the intelligent triage environment of the hospital.

[0121] Step 502: Based on the text information to be processed and the associated text information, determine the context type of the associated text information that matches the associated text information.

[0122] In some embodiments of the present invention, the context type of the associated text information that matches the associated text information is determined based on the text information to be processed and the associated text information. This can be achieved in the following ways:

[0123] Based on the text information to be processed and the associated text information, a matching text information matching model is triggered; the text information to be processed and the associated text information are processed through the associated text information context type discriminator network of the text information matching model to determine the associated text information context type that matches the associated text information; or, the text information processing network of the text information matching model is used to determine the text information embedding vector corresponding to the text information to be processed; the text information processing network of the text information matching model is used to determine the associated text information embedding vector corresponding to the associated text information; the associated text information context type discriminator network of the text information matching model is used to process the text information embedding vector to be processed and the associated text information embedding vector to determine the associated text information context type that matches the associated text information.

[0124] Among them, reference Figure 7 , Figure 7 This is a data structure diagram of the text information processing method provided in this embodiment of the invention. The associated text information context type discriminator network and the multi-round matching network can present a serial structure. When serially combining contextual context type information, the contextual context type discrimination and multi-round matching processes can be executed serially. First, the type is determined through the contextual context, and then the determination result, the query context information, and the current user query are used as part of the input of the matching model for a multi-round retrieval matching process. The advantage of processing the text information to be processed and the associated text information through the associated text information context type discriminator network of the text information matching model is that the model can be trained in stages offline, and can be used serially after deployment, which facilitates flexible online adaptation and reduces the adjustment time after model application. Furthermore, when using the serial structure, candidate similar text information that matches the text information to be processed is obtained. Based on the text information to be processed, the associated text information, and the context type of the associated text information, the similarity between the text information to be processed and the candidate similar text information is determined. This can be achieved in the following way:

[0125] The text information processing network of the text information matching model determines the candidate similar text information embedding vector corresponding to the candidate similar text information; the multi-round matching network of the text information matching model processes the text information embedding vector to be processed and the associated text information embedding vector to determine the first concatenation feature vector; the multi-round matching network of the text information matching model determines the similarity between the text information embedding vector to be processed and the candidate similar text information embedding vector based on the context type of the associated text information and the first concatenation feature vector.

[0126] Among them, reference Figure 8 , Figure 8 The diagram shows the data structure of the text information processing method provided in this embodiment of the invention. The context type discriminator network and the multi-round matching network can also be presented in parallel. The multi-round matching network can use the Siamese-BERT structure, in which the left and right BERT models do not share parameters and keep their calculations independent.

[0127] Step 503: Obtain candidate similar text information that matches the text information to be processed. Based on the text information to be processed, related text information, and the context type of the related text information, determine the similarity between the text information to be processed and the candidate similar text information. (Reference) Figure 9 , Figure 9 This is a schematic diagram of the text matching process in an embodiment of the present invention. When the context type discriminator network and the multi-round matching network can also present a parallel structure, the text information processing network of the text information matching model determines the candidate similar text information embedding vector corresponding to the candidate similar text information; the multi-round matching network of the text information matching model processes the text information embedding vector to be processed and the context type discriminator network to determine the first concatenated feature vector; the multi-round matching network of the text information matching model determines the similarity between the text information embedding vector to be processed and the candidate similar text information embedding vector based on the first concatenated feature vector; the similarity between the text information embedding vector to be processed and the candidate similar text information embedding vector is sorted according to the context type of the context information; and the similarity between the text information to be processed and the candidate similar text information is determined based on the sorting result.

[0128] Step 504: Based on the similarity between the text information to be processed and the candidate similar text information, determine the response statement that matches the text information to be processed.

[0129] Of course, the text information matching model needs to be trained before it can be used. (Referencing...) Figure 10 , Figure 10 This is an optional flowchart illustrating a text information processing method provided in an embodiment of the present invention. It can be understood that... Figure 10 The steps shown can be performed by various electronic devices running text information processing devices, such as dedicated terminals with text message reply functions, mobile phones, or communication devices running search function applets, to train a text information matching model and deploy the trained text information matching model on a server. The following section addresses... Figure 10 The steps shown are explained.

[0130] Step 1001: Obtain a training sample set, wherein the training sample set includes different text information samples that have been labeled with entities.

[0131] Specifically, a set of training samples matching the context type of the text information to be processed can be obtained from the data source based on the context type of the associated text information. The model parameters of the text information matching model that is adapted to the context type of the text information to be processed can be determined according to the set of training samples matching the context type of the text information to be processed.

[0132] Step 1002: Process the training sample set using the text information matching model to determine the initial parameters of the associated text information context type discriminator network and the initial parameters of the multi-round matching network in the text information matching model.

[0133] Step 1003: Based on the initial parameters of the associated text information context type discriminator network and the initial parameters of the multi-round matching network, the training sample set is processed by the text information matching model to determine the update parameters corresponding to the associated text information context type discriminator network and the update parameters corresponding to the multi-round matching network, respectively.

[0134] Step 1004: Based on the update parameters corresponding to the context type discriminator network and the multi-round matching network, iteratively update the context type discriminator network and the multi-round matching network respectively using the training sample set.

[0135] Specifically, during the iterative updates of the context type discriminator network and the multi-round matching network using the training sample set, a first loss function matching the context type discriminator network can be determined; a second loss function matching the multi-round matching network can be determined; and based on the first and second loss functions, a fusion loss function corresponding to the text information matching model can be determined. When the context type discriminator network and the multi-round matching network can also present a sequential structure, the model parameters during training refer to Formula 1.

[0136] Formula 1

[0137] The context-type discriminator network and the multi-round matching network can also be presented in parallel structure. The model parameters during training can be referenced from Formula 2:

[0138] Formula 2

[0139] During the testing phase, only the context, the current query, and candidates need to be input; no stage prediction is required. This saves user waiting time and provides a better user experience for the text matching model. Specifically, when the text matching model is used in a non-Chinese environment (which can be a single English or other language environment, or an environment with at least two language sources for the problem statement), the corresponding bilingual evaluation values ​​determined based on the negative example sample set can be used as supervisory parameters to evaluate the semantic understanding results of the text matching model.

[0140] In some embodiments of the present invention, the decoder in the text information matching model can be a bidirectional network model. For example, a Bi-GRU bidirectional GRU model can be used as both the encoder and decoder. The Bi-GRU bidirectional GRU model is a model capable of recognizing inverted sentence structures. Since users may input inverted sentence structures when asking questions, such as "Are you married, Xia Mei?", while the normal sentence structure is "Is Xia Mei married?", the Bi-GRU bidirectional GRU model can recognize inverted sentence structures, thereby enriching the functionality of the trained model and improving the robustness of the final trained text information matching model.

[0141] The following describes the text information matching model provided in this embodiment of the invention using a text information matching model encapsulated in a corresponding APP. Figure 11 This is a schematic diagram illustrating the application environment of the text information matching model in an embodiment of the present invention, such as... Figure 11 As shown, with the development of human-computer interaction (HCI) technology, more and more intelligent products based on HCI technology have emerged, such as automatic reply robots in instant messaging clients. These intelligent products can chat and communicate with users and generate corresponding answers based on user questions. However, the generated answers are usually high-frequency generic replies or incorrect replies, contrary to what users expect. Figure 11 The response statement shown on the right, but Figure 11 The responses generated on the left are often meaningless and do not match the context of the text information matching model (for example, different text information matching models need to generate the most appropriate response based on the context of the usage environment). At the same time, the related technology can only generate one corresponding response based on a question, but cannot generate multiple corresponding response based on a question and the corresponding virtual robot's persona information, which affects the user experience.

[0142] Figure 12This is a schematic diagram illustrating the working process of the text information matching model provided in this embodiment of the invention, specifically including the following steps:

[0143] Step 1201: Obtain the text information to be processed and the associated text information, and trigger the text information matching model in the instant messaging APP.

[0144] Step 1202: Based on the dialogue information in the instant messaging APP, determine the context type of the associated text information that matches the associated text information.

[0145] Step 1203: Determine the similarity between the text information to be processed and the candidate similar text information.

[0146] Step 1204: Based on the similarity between the text information to be processed and the candidate similar text information, and the matching similarity threshold, determine whether the candidate similar text information is compliant.

[0147] Step 1205: When it is determined that the candidate similar text information is compliant, add the candidate similar text information to the text recommendation process.

[0148] Step 1206: Sort the recall order of all candidate similar text information in the text recommendation process;

[0149] Step 1207: Recommend text information to the target user based on the ranking result of the recall order of the candidate similar text information.

[0150] Therefore, by associating text information, the context type of the associated text information can be determined, resulting in more accurate processing of the text information to be processed, so as to adapt to the current usage scenario and enable users to have a better automatic dialogue experience in instant messaging apps.

[0151] The present invention has the following beneficial technical effects:

[0152] This invention acquires the text information to be processed within a text processing context, along with related text information matching the text to be processed. Based on the text information to be processed and the related text information, it determines the context type of the related text information matching the related text information. It then acquires candidate similar text information matching the text information to be processed, and based on the text information to be processed, the related text information, and the context type of the related text information, it determines the similarity between the text information to be processed and the candidate similar text information. Finally, based on the similarity between the text information to be processed and the candidate similar text information, it determines a response statement matching the text information to be processed. Therefore, by determining the context type of related text information through related text information, the processing effect on the text information to be processed can be more accurate, making the processing result applicable to the current usage scenario. It reduces the impact of missing related text information on the generated response statement, reduces generic replies, and thus improves the matching and forward-looking nature of the response, enhancing the user experience.

[0153] The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method of processing text information, characterized by, The method comprises: acquiring text information to be processed in a text processing context and associated text information matched with the text information to be processed; based on the text information to be processed and the associated text information, triggering a matched text information matching model; acquiring candidate similar text information matched with the text information to be processed, determining a text information to be processed embedding vector corresponding to the text information to be processed, an associated text information embedding vector corresponding to the associated text information, and a candidate similar text information embedding vector corresponding to the candidate similar text information through a text information processing network of the text information matching model; processing the text information to be processed and the associated text information through an associated text information context type discriminator network of the text information matching model to determine an associated text information context type matched with the associated text information; or, processing the text information to be processed embedding vector and the associated text information embedding vector through the associated text information context type discriminator network of the text information matching model to determine an associated text information context type matched with the associated text information; based on the text information to be processed embedding vector, the associated text information embedding vector, and the associated text information context type, determining a similarity between the text information to be processed embedding vector and the candidate similar text information embedding vector through a multi-round matching network of the text information matching model; based on the similarity between the text information to be processed embedding vector and the candidate similar text information embedding vector, determining a reply statement matched with the text information to be processed.

2. The method of claim 1, wherein, The method further comprises: triggering a corresponding word segmentation library according to a text information parameter carried by the text information to be processed; based on a word dictionary of the triggered word segmentation library, performing word segmentation processing on the text information to be processed through a text information processing network of the text information matching model to form different word-level text information to be processed embedding vectors; performing noise removal processing on the different word-level text information to be processed embedding vectors to form a text information to be processed embedding vector set corresponding to the text information to be processed.

3. The method of claim 2, wherein, The method further comprises: determining a name of a word segmentation library used when performing word segmentation processing on the text information to be processed; determining a parameter of a word segmentation library matched with a word-level text information to be processed embedding vector corresponding to the text information to be processed according to the name of the word segmentation library, wherein the parameter of the word segmentation library comprises: a type of the word segmentation library, the name of the word segmentation library, and a version of the word segmentation library.

4. The method of claim 2, wherein, The noise removal processing on the different word-level text information to form a word-level feature vector set corresponding to the text information comprises: determining a dynamic noise threshold matched with the text processing context; performing noise removal processing on the text information to be processed according to the dynamic noise threshold and triggering a dynamic word segmentation strategy matched with the dynamic noise threshold; According to the dynamic word segmentation strategy matched with the dynamic noise threshold, the to-be-processed text information is segmented to form a to-be-processed text information embedding vector corresponding to the to-be-processed text information.

5. The method of claim 2, wherein, The denoising processing of the different word-level text information to form a word-level feature vector set corresponding to the text information comprises: determining a fixed noise threshold matched with the text processing context; According to the fixed noise threshold, the to-be-processed text information is denoised, and a fixed word segmentation strategy matched with the fixed noise threshold is triggered; According to the fixed word segmentation strategy matched with the fixed noise threshold, the to-be-processed text information is segmented to form a to-be-processed text information embedding vector corresponding to the to-be-processed text information.

6. The method of claim 1, wherein, The multi-round matching network of the text information matching model determines the similarity between the to-be-processed text information embedding vector and the candidate similar text information embedding vector based on the to-be-processed text information embedding vector, the associated text information embedding vector, and the associated text information context type, comprising: The multi-round matching network of the text information matching model processes the to-be-processed text information embedding vector and the associated text information embedding vector to determine a first splicing feature vector. The multi-round matching network of the text information matching model determines the similarity between the to-be-processed text information embedding vector and the candidate similar text information embedding vector based on the first splicing feature vector.

7. The method of claim 1, wherein, The multi-round matching network of the text information matching model determines the similarity between the to-be-processed text information embedding vector and the candidate similar text information embedding vector based on the to-be-processed text information embedding vector, the associated text information embedding vector, and the associated text information context type, comprising: The multi-round matching network of the text information matching model processes the to-be-processed text information embedding vector and the associated text information embedding vector to determine a first splicing feature vector. The multi-round matching network of the text information matching model determines the similarity between the to-be-processed text information embedding vector and the candidate similar text information embedding vector based on the first splicing feature vector. The similarity between the to-be-processed text information embedding vector and the candidate similar text information embedding vector is sorted based on the associated text information context type. Based on the sorting result, the similarity between the to-be-processed text information and the candidate similar text information is determined.

8. The method of claim 7, wherein, The method further comprises: Based on the similarity between the to-be-processed text information and the candidate similar text information and the matched similarity threshold, it is determined whether the candidate similar text information is compliant; When it is determined that the candidate similar text information is compliant, the candidate similar text information is added to the text recommendation process; The recall order of all candidate similar text information in the text recommendation process is sorted; Based on the sorting result of the recall order of the candidate similar text information, the text information is recommended to the target user.

9. The method of claim 1, wherein, The method further comprises: Obtaining a training sample set, wherein the training sample set comprises different text information samples marked with entities; Processing the training sample set by the text information matching model to determine initial parameters of an associated text information context type discriminator network and initial parameters of a multi-round matching network in the text information matching model; Processing the training sample set by the text information matching model based on the initial parameters of the associated text information context type discriminator network and the initial parameters of the multi-round matching network to respectively determine updated parameters corresponding to the associated text information context type discriminator network and updated parameters corresponding to the multi-round matching network; According to the updated parameters corresponding to the associated text information context type discriminator network and the updated parameters corresponding to the multi-round matching network, iteratively updating the associated text information context type discriminator network and the multi-round matching network respectively by the training sample set.

10. The method of claim 9, wherein, The method further comprises: determining a first loss function matched with the associated text information context type discriminator network; determining a second loss function matched with the multi-round matching network; determining a fusion loss function corresponding to the text information matching model based on the first loss function and the second loss function.

11. The method of claim 1, wherein, The method further comprises: Based on the associated text information context type matched with the associated text information, obtaining a training sample set matched with the context type of the text information to be processed in a data source; According to the training sample set matched with the context type of the text information to be processed, determining model parameters of a text information matching model adapted to the context type of the text information to be processed.

12. A text information processing apparatus characterized by comprising: The device comprises: An information transmission module for obtaining text information to be processed in a text processing context and associated text information matched with the text information to be processed; An information processing module for triggering a matched text information matching model based on the text information to be processed and the associated text information, obtaining candidate similar text information matched with the text information to be processed, and determining a text information to be processed embedding vector corresponding to the text information to be processed, an associated text information embedding vector corresponding to the associated text information, and a candidate similar text information embedding vector corresponding to the candidate similar text information through a text information processing network of the text information matching model; The information processing module is configured to process the text information to be processed and the associated text information through an associated text information context type discriminator network of the text information matching model to determine an associated text information context type matched with the associated text information, or to process the text information to be processed embedding vector and the associated text information embedding vector through the associated text information context type discriminator network of the text information matching model to determine an associated text information context type matched with the associated text information. The information processing module is configured to determine a similarity between the to-be-processed text information embedding vector and the candidate similar text information embedding vector based on the to-be-processed text information embedding vector, the associated text information embedding vector, and the associated text information context type by using a multi-round matching network of the text information matching model; and determine a reply statement matched with the to-be-processed text information based on the similarity between the to-be-processed text information embedding vector and the candidate similar text information embedding vector.

13. The apparatus of claim 12, wherein, The information processing module is further configured to trigger a corresponding word segmentation library according to a text information parameter carried by the to-be-processed text information. The information processing module is further configured to perform word segmentation processing on the to-be-processed text information based on a word dictionary of the triggered word segmentation library by using a text information processing network of the text information matching model, to form different word-level to-be-processed text information embedding vectors. The information processing module is further configured to perform noise removal processing on the different word-level to-be-processed text information embedding vectors, to form a to-be-processed text information embedding vector set corresponding to the to-be-processed text information.

14. An electronic device, comprising: The electronic device comprises: a memory configured to store executable instructions; a processor configured to execute the executable instructions stored in the memory, to implement the text information processing method according to any one of claims 1 to 11.

15. A computer-readable storage medium storing executable instructions, the instructions being executable by a processor to cause the processor to perform operations comprising: The executable instructions, when executed by the processor, implement the text information processing method according to any one of claims 1 to 11.

16. A computer program product comprising computer programs or computer executable instructions, characterized in that, The computer program or computer executable instructions, when executed by the processor, implement the text information processing method according to any one of claims 1 to 11.

Images