Text processing method, device, equipment and storage medium based on artificial intelligence

Abstract

This application provides a text processing method, apparatus, electronic device, computer-readable storage medium, and computer program product based on artificial intelligence; relating to fields such as cloud technology, artificial intelligence, and intelligent transportation. The method includes: performing multi-scale encoding processing on a first word vector sequence corresponding to a first text to obtain multi-scale semantic features of the first text; performing multi-scale encoding processing on a second word vector sequence corresponding to a second text to obtain multi-scale semantic features of the second text; constructing a similar semantic matrix based on the multi-scale semantic features of the first text and the second text; performing feature extraction processing on the similar semantic matrix to obtain similar semantic features; and determining the matching degree between the first text and the second text based on the similar semantic features. This application can accurately express the semantics of text, thereby effectively improving the accuracy of text matching.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a text processing method, apparatus, electronic device, computer-readable storage medium, and computer program product based on artificial intelligence. Background Technology

[0002] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0003] In text matching methods provided by related technologies, convolutional neural networks (CNNs) or recurrent neural networks (RNNs) are typically used to extract semantic features from the text. The output of the last layer of the network is taken as the semantic feature of the text, and the semantic similarity between the two texts is calculated based on this semantic feature to determine whether the two texts match. However, this method, which only uses the output of the last layer of the network as the semantic feature of the text, cannot accurately represent the semantic information of the entire text, thus affecting the accuracy of subsequent text matching. Summary of the Invention

[0004] This application provides an artificial intelligence-based text processing method, apparatus, electronic device, computer-readable storage medium, and computer program product that can accurately express the semantics of text, thereby effectively improving the accuracy of text matching.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides an artificial intelligence-based text processing method, including:

[0007] Multi-scale encoding is performed on the first word vector sequence corresponding to the first text to obtain the multi-scale semantic features of the first text.

[0008] Multi-scale encoding is performed on the second word vector sequence corresponding to the second text to obtain the multi-scale semantic features of the second text.

[0009] Based on the multi-scale semantic features of the first text and the multi-scale semantic features of the second text, a similarity semantic matrix is ​​constructed;

[0010] The similar semantic matrix is ​​subjected to feature extraction processing to obtain similar semantic features;

[0011] Based on the similar semantic features, the matching degree between the first text and the second text is determined.

[0012] This application provides an artificial intelligence-based text processing device, comprising:

[0013] The encoding module is used to perform multi-scale encoding processing on the first word vector sequence corresponding to the first text to obtain the multi-scale semantic features of the first text.

[0014] The encoding module is also used to perform multi-scale encoding processing on the second word vector sequence corresponding to the second text to obtain the multi-scale semantic features of the second text;

[0015] The construction module is used to construct a similar semantic matrix based on the multi-scale semantic features of the first text and the multi-scale semantic features of the second text;

[0016] The extraction module is used to perform feature extraction processing on the similar semantic matrix to obtain similar semantic features;

[0017] The determination module is used to determine the matching degree between the first text and the second text based on the similar semantic features.

[0018] In the above scheme, the encoding module is further used to perform cascaded encoding processing at L scales based on the first word vector sequence corresponding to the first text, thereby obtaining the semantic features of the first text at L scales in sequence; wherein, L is an integer greater than or equal to 2, and the sliding window length used for encoding processing at different scales is different.

[0019] In the above scheme, the encoding module is further configured to iteratively perform the following processing on i: slide sampling the output of the (i-1)th encoding network according to a set step size through the sliding window corresponding to the i-th encoding network, and encode the sampling result to obtain the encoding result output by the i-th encoding network; wherein, when the value of i is 1, the input of the i-th encoding network is the first word vector sequence corresponding to the first text; when the value of i is 2≤i≤L, the input of the i-th encoding network is the encoding result output by the (i-1)th encoding network; the determining module is further configured to determine the encoding results output by each of the L encoding networks as the semantic features of the first text at L scales.

[0020] In the above scheme, the encoding module is further used to concatenate the output of the (i-1)th encoding network and the state corresponding to the (i-1)th encoding network to obtain a concatenation result; to slide the concatenation result through the sliding window corresponding to the i-th encoding network according to a set step size, and to encode the sampling result to obtain the encoding result output by the i-th encoding network.

[0021] In the above scheme, the device further includes a word segmentation module, which is used to perform word segmentation processing on the first text and combine the words obtained from the word segmentation processing to obtain a first word sequence corresponding to the first text; the device further includes a word embedding module, which is used to perform word embedding processing on the first word sequence to obtain a word vector corresponding to each word in the first word sequence, and combine multiple word vectors to obtain a first word vector sequence corresponding to the first text.

[0022] In the above scheme, the device further includes a part-of-speech tagging module, which is used to perform part-of-speech tagging on the words obtained by word segmentation of the first text; and to combine the words in the first text that meet the part-of-speech conditions to obtain the first word sequence corresponding to the first text.

[0023] In the above scheme, the encoding module is further used to perform cascaded encoding processing at L scales based on the second word vector sequence corresponding to the second text, thereby obtaining the semantic features of the second text at L scales in sequence; wherein, L is an integer greater than or equal to 2, and the sliding window length used for encoding processing at different scales is different.

[0024] In the above scheme, the encoding module is further configured to iteratively perform the following processing on i: slide sampling the output of the (i-1)th encoding network according to a set step size through the sliding window corresponding to the i-th encoding network, and encode the sampling result to obtain the encoding result output by the i-th encoding network; wherein, when i is 1, the input of the i-th encoding network is the second word vector sequence corresponding to the second text; when i is 2≤i≤L, the input of the i-th encoding network is the encoding result output by the (i-1)th encoding network; the determining module is further configured to determine the encoding results output by each of the L encoding networks as the semantic features of the second text at L scales.

[0025] In the above scheme, the encoding module is further used to concatenate the output of the (i-1)th encoding network and the state corresponding to the (i-1)th encoding network to obtain a concatenation result; to slide the concatenation result through the sliding window corresponding to the i-th encoding network according to a set step size, and to encode the sampling result to obtain the encoding result output by the i-th encoding network.

[0026] In the above scheme, the construction module is further configured to multiply the semantic features of the same scale in the first text and the second text; combine the multiplication results corresponding to multiple scales respectively to obtain a multi-scale similar semantic matrix; and concatenate the similar semantic matrix of each scale in the multi-scale similar semantic matrix according to the channel to obtain a similar semantic matrix.

[0027] In the above scheme, the determining module is further used to perform fully connected processing on the similar semantic features to obtain the fully connected processing result; and to call the trained classifier to perform matching degree prediction processing on the fully connected processing result to obtain the matching degree between the first text and the second text.

[0028] This application provides an electronic device, including:

[0029] Memory, used to store executable instructions;

[0030] The processor, when executing executable instructions stored in the memory, implements the AI-based text processing method provided in the embodiments of this application.

[0031] This application provides a computer-readable storage medium storing executable instructions for inducing a processor to execute and implement the artificial intelligence-based text processing method provided in this application.

[0032] This application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the artificial intelligence-based text processing method provided in this application.

[0033] The embodiments of this application have the following beneficial effects:

[0034] By performing multi-scale encoding on the word vector sequences corresponding to the text to be matched, richer semantic information of the text can be accurately extracted. Then, a similar semantic matrix is ​​constructed based on the multi-scale semantic features of the text, and feature extraction processing is performed on the similar semantic matrix to obtain similar semantic features between texts. Finally, the similar semantic features are used to determine whether two texts match. In this way, the richer semantic information contained in the multi-scale semantic features can better perform text matching, thereby effectively improving the accuracy of text matching. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the architecture of the AI-based text processing system 100 provided in this application embodiment;

[0036] Figure 2 This is a schematic diagram of the structure of the server 200 provided in the embodiments of this application;

[0037] Figure 3 This is a flowchart illustrating the artificial intelligence-based text processing method provided in an embodiment of this application;

[0038] Figure 4 This is a flowchart illustrating the artificial intelligence-based text processing method provided in an embodiment of this application;

[0039] Figure 5 This is a flowchart illustrating the artificial intelligence-based text processing method provided in an embodiment of this application;

[0040] Figure 6 This is a schematic diagram of the formal structure of short text data provided in the embodiments of this application;

[0041] Figure 7 This is a flowchart illustrating the artificial intelligence-based text processing method provided in an embodiment of this application;

[0042] Figure 8 This is a schematic diagram of the principle of the fully connected layer provided in the embodiments of this application. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0044] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0045] In the following description, the terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0047] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0048] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0049] 1) Scale: This refers to the precision of the encoding, which is related to the length of the sliding window used in the encoding process. Different scales of encoding use different sliding window lengths. For example, the first scale of encoding uses a sliding window length of 2, while the second scale uses a sliding window length of 4. This means that different scales of encoding need to consider different lengths of contextual information.

[0050] 2) Sliding window: also known as a sliding window, is used to perform sliding sampling on word vector sequences or the encoding results output by the encoding network to obtain multiple sampling results.

[0051] 3) Cascading: In the coding process, cascading refers to the dependency relationship between the coding process of the previous scale and the coding process of the next scale, that is, the coding process of the previous scale depends on the coding process of the next scale.

[0052] 4) Word Embedding: A general term for a class of models that vectorize words. Its core idea is to map each word to a dense vector in a low-dimensional space, and words with similar meanings have similar vector representations.

[0053] 5) Convolutional Neural Networks (CNN): A type of feedforward neural network that includes convolutional computation and has a deep structure. It is one of the representative algorithms of deep learning. CNN has representation learning capabilities and can be used to extract features from the similar semantic matrix between texts to obtain similar semantic features.

[0054] 6) Recurrent Neural Network (RNN): A recurrent neural network that takes sequence data as input, recurses in the direction of sequence evolution, and all nodes (i.e., recurrent units) are connected in a chain. It is suitable for processing time-related problems such as video, speech, and text. In a recurrent neural network, neurons can not only receive information from other neurons, but also receive information from themselves, forming a network structure with loops.

[0055] 7) Long Short-Term Memory (LSTM): A special type of RNN, mainly designed to solve the gradient vanishing and gradient explosion problems during long sequence training. Compared to RNN (which has only one transmission state), LSTM has two transmission states, namely the previous state h(t-1) and the current input x(t).

[0056] This application provides an artificial intelligence-based text processing method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can accurately express the semantics of text and thus effectively improve the accuracy of text matching. The following describes exemplary applications of the electronic devices provided in this application. These electronic devices can be implemented as various types of terminal devices such as laptops, tablets, desktop computers, set-top boxes, mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), and in-vehicle terminals. They can also be implemented as servers, or implemented in coordination between terminal devices and servers.

[0057] The following description uses an example of an AI-based text processing method provided in this application, implemented in coordination between a terminal device and a server. For example, see [link to example]. Figure 1 , Figure 1 This is a schematic diagram of the architecture of the AI-based text processing system 100 provided in this application embodiment. In order to support an application that effectively improves the deduplication effect of WeChat official account articles, the terminal device 400 connects to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0058] A client 410 runs on the terminal device 400. The client 410 can be of various types, such as an instant messaging client, a text editing client, a text reading client, a browser, etc. In some embodiments, the client 410 can subscribe to multiple public accounts from the server 200. Among them, some articles published by the multiple public accounts subscribed to by the client 410 may be duplicated. Therefore, before pushing the article list composed of multiple public account articles to the client 410, the server 200 can use the artificial intelligence-based text processing method provided in this application embodiment to perform deduplication (for example, for any two public account articles among multiple public account articles, the artificial intelligence-based text processing method provided in this application embodiment is used to determine the matching degree between the two public account articles. When the matching degree is greater than the matching degree threshold, it means that the two public account articles belong to the same content, and only one of the public account articles can be kept). The deduplicated list of public account articles is then pushed to the client 410. In this way, duplicate public account articles can be avoided from being pushed to the user, saving the resources consumed by information transmission, and also improving the user experience.

[0059] In some embodiments, the AI-based text processing method provided in this application can also be applied to information recommendation systems (e.g., article recommendation systems, news recommendation systems, etc.). Before using an information recommendation system to recommend information, similarity mining can be performed using the AI-based text processing method provided in this application. That is, the association between two pieces of individual information (e.g., two articles from public accounts) can be pre-mined, such as similarity mining between text content (also known as text matching). When the matching degree between two texts is greater than the matching degree threshold (i.e., the two texts are similar texts), only one of them can be retained in the recommended information (e.g., when it is determined that the matching degree between the titles of two public account articles is greater than the matching degree threshold, it means that the two public account articles are similar articles, and only one of the public account articles can be pushed to the user's terminal device). In this way, duplicate information can be recommended to the user, improving the user experience. It should be noted that the AI-based text processing method provided in this application is not limited to article and news recommendation systems, but can also be applied to other types of recommendation systems, such as video recommendation systems (e.g., judging whether two videos are similar videos by comparing whether the resume information of two videos matches), hot topic recall systems, etc.

[0060] In some embodiments, the AI-based text processing method provided in this application can also be implemented independently by the terminal device, for example, using... Figure 1Taking the terminal device 400 shown as an example, in response to the user's selection operation, the client 410 selects the first text and the second text to be matched from the text set stored locally on the terminal device 400. Then, the terminal device 400 performs word segmentation and word embedding processing on the first text and the second text respectively to obtain the first word vector sequence corresponding to the first text and the second word vector sequence corresponding to the second text. Subsequently, the terminal device 400 performs multi-scale encoding processing on the first word vector sequence corresponding to the first text to obtain the multi-scale semantic features of the first text, and performs multi-scale encoding processing on the second word vector sequence corresponding to the second text to obtain the multi-scale semantic features of the second text. After obtaining the multi-scale semantic features of the first text and the multi-scale semantic features of the second text, the terminal device 400 performs multi-scale encoding processing on the first word vector sequence corresponding to the first text to obtain the multi-scale semantic features of the second text. After obtaining the multi-scale semantic features, the terminal device 400 can construct a similar semantic matrix based on the multi-scale semantic features of the first text and the second text, and perform feature extraction processing on the constructed similar semantic matrix to obtain similar semantic features (i.e., similar semantic features between the first text and the second text). Finally, the terminal device 400 can determine the matching degree between the first text and the second text based on the extracted similar semantic features. When the matching degree between the first text and the second text is greater than the matching degree threshold (i.e., the first text and the second text are similar texts), only one of the texts can be retained (e.g., deleting the text selected by the user, or automatically deleting one of the texts) to save the storage space of the terminal device 400.

[0061] In some embodiments, the embodiments of this application can also be implemented with the aid of cloud technology, which refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or local area network to realize the computation, storage, processing, and sharing of data.

[0062] Cloud technology is a general term encompassing network technology, information technology, integration technology, management platform technology, and application technology based on the cloud computing business model. It can form resource pools, allowing for on-demand use with flexibility and convenience. Cloud computing technology will become a crucial support. The backend services of cloud computing systems require substantial computing and storage resources.

[0063] Example, Figure 1The server 200 shown can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal device 400 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal device 400 and the server 200 can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0064] In other embodiments, the AI-based text processing method provided in this application can also be implemented in conjunction with blockchain technology, for example... Figure 1 The terminal device 400 and server 200 shown can join the blockchain network and become a node in it. Different nodes in the blockchain network can have information connections, and nodes can also transmit information through the above information connections. For example, the data related to the artificial intelligence-based text processing method provided in this application embodiment (such as the first text and the second text to be matched, and the matching degree between the first text and the second text) can be stored in the blockchain network, thereby ensuring the traceability and security of the data.

[0065] In some embodiments, the AI-based text processing method provided in this application can also be applied to the field of intelligent transportation, such as to intelligent transportation systems (ITS). An intelligent transportation system (ITS), also known as an intelligent transportation system, effectively integrates advanced technologies (such as information technology, computer technology, data communication technology, sensor technology, electronic control technology, automatic control theory, operations research, and artificial intelligence) into transportation, service control, and vehicle manufacturing. This strengthens the connection between vehicles, roads, and users, thereby forming a comprehensive transportation system that ensures safety, improves efficiency, enhances the environment, and saves energy. For example, when a user subscribes to multiple traffic information items through an in-vehicle terminal, the intelligent transportation system can call the AI-based text processing method provided in this application to perform similarity matching and deduplication on the multiple traffic information items, thereby avoiding sending duplicate traffic information to the user's in-vehicle terminal.

[0066] In some embodiments, the terminal device or server can also implement the AI-based text processing method provided in this application by running a computer program. For example, the computer program can be a native program in the operating system (e.g., a dedicated text matching program) or a software module, such as a text matching module that can be embedded in any program (e.g., an instant messaging client, a text reading client, etc.); it can also be a native application (APP), i.e., a program that needs to be installed in the operating system to run, such as... Figure 1 The client 410 is shown in the figure. In summary, the above-described computer program can be any form of application, module, or plug-in.

[0067] The following continues... Figure 1 The structure of server 200 shown in the diagram will be explained. See also... Figure 2 , Figure 2 This is a schematic diagram of the structure of the server 200 provided in the embodiments of this application. Figure 2 The server 200 shown includes at least one processor 210, memory 240, and at least one network interface 220. The various components of server 200 are coupled together via a bus system 230. Understandably, the bus system 230 is used to implement communication between these components. In addition to a data bus, the bus system 230 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 230.

[0068] Processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0069] The memory 240 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 240 may optionally include one or more storage devices physically located away from the processor 210.

[0070] The memory 240 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 240 described in this application embodiment is intended to include any suitable type of memory.

[0071] In some embodiments, memory 240 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0072] Operating system 241 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0073] The network communication module 242 is used to reach other computing devices via one or more (wired or wireless) network interfaces 220, such as Bluetooth, WiFi, and Universal Serial Bus (USB).

[0074] In some embodiments, the AI-based text processing apparatus provided in this application can be implemented in software. Figure 2 An AI-based text processing device 243, stored in memory 240, is shown. This device can be software in the form of programs and plugins, and includes the following software modules: encoding module 2431, construction module 2432, extraction module 2433, determination module 2434, word segmentation module 2435, word embedding module 2436, and part-of-speech tagging module 2437. These modules are logically connected and can therefore be arbitrarily combined or further split according to the functions implemented. It should be noted that... Figure 2 For ease of explanation, all the above modules are shown at once, but this should not be interpreted as excluding the implementation of the AI-based text processing device 243, which may only include the encoding module 2431, the construction module 2432, the extraction module 2433, and the determination module 2434. The functions of each module will be described below.

[0075] In other embodiments, the AI-based text processing device provided in this application can be implemented in hardware. As an example, the AI-based text processing device provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the AI-based text processing method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0076] The following will describe the artificial intelligence-based text processing method provided in this application embodiment, with reference to the exemplary application and implementation of the server provided in the embodiments of this application.

[0077] See Figure 3 , Figure 3 This is a flowchart illustrating an artificial intelligence-based text processing method provided in an embodiment of this application, which will be combined with... Figure 3 The steps shown are explained.

[0078] In step 101, the first word vector sequence corresponding to the first text is subjected to multi-scale encoding processing to obtain the multi-scale semantic features of the first text.

[0079] Here, the first text can be text entered by the user in real time (e.g., the client responds to the user's input and sends the user's input text to the server), or text retrieved from a database or local storage based on the user's selection, or text retrieved by the server from a database. Furthermore, this application embodiment does not limit the type of the first text. For example, the first text can be long text such as articles or blogs (i.e., text with more than a certain number of characters), or short text such as comments or product descriptions (i.e., text with fewer than a certain number of characters).

[0080] In some embodiments, before performing multi-scale encoding on the first word vector sequence corresponding to the first text, the following processing may also be performed: performing word segmentation on the first text, and combining the words obtained from the word segmentation to obtain the first word sequence corresponding to the first text; performing word embedding on the first word sequence to obtain the word vector corresponding to each word in the first word sequence, and combining multiple word vectors to obtain the first word vector sequence corresponding to the first text.

[0081] For example, the word segmentation process described above can employ a statistical approach. This involves statistically analyzing the frequency of character combinations formed by adjacent characters in the first text, calculating the frequency of each combination, and determining that a combination constitutes a word when its frequency exceeds a set threshold. This achieves word segmentation of the first text. Alternatively, the word segmentation process can also employ a string matching approach. For instance, the first text can be matched against each word in a pre-set machine dictionary on the server, and then segmented based on the matched words. The string matching principle can be forward maximum matching, backward maximum matching, segmentation tagging, word-by-word traversal matching, forward optimal matching, or backward optimal matching, etc.

[0082] It should be noted that in practical applications, Hidden Markov Models (HMMs) can also be used to segment the first text. The specific implementation method of segmentation is not limited in the embodiments of this application.

[0083] In other embodiments, after the first text is segmented, the following processing can also be performed: the words obtained by the segmentation of the first text are subjected to part-of-speech tagging, and the words in the first text that meet the part-of-speech conditions are combined to obtain the first word sequence corresponding to the first text.

[0084] For example, the part-of-speech tagging mentioned above is essentially a classification problem. It involves classifying the multiple words obtained from word segmentation of the first text according to their parts of speech. A word's part of speech is determined by its meaning, morphology, and grammatical function in its language. Taking Chinese as an example, the Chinese word class system has 18 subclasses, including 7 categories of nouns, 4 categories of predicates, 5 categories of function words, pronouns, and interjections. After tagging the multiple words obtained from word segmentation of the first text, meaningless or unimportant words such as conjunctions, auxiliary words, adverbs, prepositions, and stop words can be deleted. The remaining words are then combined to obtain the first word sequence corresponding to the first text. By filtering out meaningless or unimportant words, the system resources consumed by the server during subsequent word embedding processing can be saved, and the accuracy of subsequent text matching can be further improved.

[0085] In some embodiments, the word embedding process described above can be implemented by calling a word embedding model, which may include a skip-gram model and a continuous bag of words (CBOW) model. The basic idea of ​​the skip-gram model is to predict the window functions that are used in the order of each central function and to correct the vector of the central function based on the prediction results. The basic idea of ​​the continuous bag of words model is to predict the vector of the central function by using the vector of the window functions that are used in the order of each function.

[0086] It should be noted that in practical applications, other word vector generation methods can also be used to determine the word vector (i.e., a vector representing the features of a word, where each dimension of the word vector represents a feature with certain semantic and grammatical interpretations, and the features can be various information used to characterize the basic elements of a word) for each word in the first word sequence. For example, a large number of correspondence tables between words and word vectors can be pre-stored on the server, and each word vector can have the same dimension. For each word in the first word sequence, the corresponding word vector can be found from the correspondence table stored on the server. Furthermore, the large number of word vectors pre-stored on the server can be obtained by pre-training a machine learning model or by pre-setting based on a large amount of statistical data. Of course, various open-source word vector calculation tools (such as Word2Vec) can also be used to determine the word vector corresponding to each word in the first word sequence.

[0087] In some embodiments, the multi-scale semantic features of the first text may include semantic features at L scales. The multi-scale encoding processing of the first word vector sequence corresponding to the first text can be achieved in the following way: performing cascaded encoding processing at L scales based on the first word vector sequence corresponding to the first text to obtain the semantic features at L scales of the first text in sequence; where L is an integer greater than or equal to 2, and the sliding window length used for encoding processing at different scales is different (for example, when L=3, the sliding window length used for encoding processing at the first scale can be 2, the sliding window length used for encoding processing at the second scale can be 4, and the sliding window length used for encoding processing at the third scale can be 8).

[0088] In other embodiments, when the multi-scale semantic features of the first text include L scale semantic features, then it can be achieved through... Figure 4 Steps 1011 to 1012 shown are implemented Figure 3 The step 101 shown will combine Figure 4 The steps shown are explained.

[0089] In step 1011, iteration i performs the following processing: the output of the (i-1)th encoding network is sampled by sliding the window corresponding to the i-th encoding network according to the set step size, and the sampling result is encoded to obtain the encoding result of the output of the i-th encoding network.

[0090] Here, the lengths of the sliding windows corresponding to multiple encoding networks can be progressively increased or decreased. For example, assuming the lengths of the sliding windows corresponding to multiple encoding networks are progressively increased, if the length of the sliding window corresponding to the first encoding network is 3, then the length of the sliding window corresponding to the second encoding network can be 6, and further, the length of the sliding window corresponding to the third encoding network can be 9. That is, the encoding precision is progressively increased. In this way, by combining the encoding results with different precisions, the semantic information of the text can be accurately represented, thereby effectively improving the accuracy of text matching in subsequent text matching. In addition, it should be noted that when i is 1, the input of the i-th encoding network is the first word vector sequence corresponding to the first text (that is, the first encoding network encodes the first word vector sequence corresponding to the first text); when i is 2≤i≤L, the input of the i-th encoding network is the encoding result output by the (i-1)-th encoding network (for example, the third encoding network encodes the encoding result output by the second encoding network).

[0091] For example, taking the third coding network as an example, assuming the sliding window length corresponding to the third coding network is 9, the coding result output by the second coding network (assuming the coding result output by the second coding network is a vector of length 90) can be sampled by sliding window of length 9 according to a set step size (assuming the step size is 3) to obtain 30 sampling results. Then, these 30 sampling results are encoded and the coding results corresponding to the 30 sampling results are combined to finally obtain the coding result output by the third coding network.

[0092] It should be noted that the aforementioned encoding network can be a recurrent neural network (RNN), such as a bidirectional recurrent neural network (Bi-RNN), or a long short-term memory network (LSTM). When the encoding network is an LSTM network, the above-mentioned method of sliding sampling the output of the (i-1)th encoding network with a set step size through the sliding window corresponding to the i-th encoding network and encoding the sampling result can be achieved as follows: concatenating the output of the (i-1)th encoding network and the state corresponding to the (i-1)th encoding network to obtain the concatenated result; sliding sampling the concatenated result with a set step size through the sliding window corresponding to the i-th encoding network and encoding the sampling result to obtain the encoded result of the output of the i-th encoding network.

[0093] For example, taking the third LSTM as an example, firstly, the output of the second LSTM and its corresponding state are concatenated to obtain the concatenated result (for example, assuming the output of the second LSTM is a 28-bit vector and the corresponding state is a 128-bit vector, then the concatenated result is a 156-bit vector). Then, the concatenated result is sampled using a sliding window corresponding to the third LSTM according to a set step size, and multiple sampling results are encoded separately. Finally, the encoded results corresponding to multiple sampling results are combined to obtain the encoded result of the third LSTM output. In this way, since the current state and the historical state are considered simultaneously when using LSTM for encoding, the gradient vanishing and gradient exploding problems in the long sequence training process can be solved, thereby better representing the semantic information of the text and improving the accuracy of subsequent text matching.

[0094] It should be noted that in practical applications, the set step size of the sliding windows corresponding to different coding networks can be the same. For example, taking L=3 as an example, the set step size of the sliding windows corresponding to the 1st to 3rd coding networks is 2, that is, the movement range of the sliding windows corresponding to the 1st to 3rd coding networks is the same each time they move. Of course, the set step size of the sliding windows corresponding to different coding networks can also be different. For example, still taking L=3 as an example, assuming that the set step size of the sliding window corresponding to the 1st coding network is 2, the set step size of the sliding window corresponding to the 2nd coding network is 4, and the set step size of the sliding window corresponding to the 3rd coding network is 6, that is, the movement range of the sliding windows corresponding to different coding networks can be different. In addition, they can also be partially the same, for example, the set step size of the sliding window corresponding to the 1st coding network and the set step size of the sliding window corresponding to the 3rd coding network are both 2. This application embodiment does not limit this.

[0095] In step 1012, the encoding results output by each of the L encoding networks are determined as the semantic features of the first text at L scales.

[0096] In some embodiments, after obtaining the encoding results output by each of the L encoding networks, the encoding result output by each encoding network can be determined as a semantic feature of a scale corresponding to the first text. For example, taking L=3 as an example, the encoding result output by the first encoding network can be determined as the semantic feature of the first scale of the first text, the encoding result output by the second encoding network can be determined as the semantic feature of the second scale of the first text, and the encoding result output by the third encoding network can be determined as the semantic feature of the third scale of the first text. Then, these three scales of semantic features are combined to obtain the three scales of semantic features of the first text.

[0097] In step 102, the second word vector sequence corresponding to the second text is subjected to multi-scale encoding processing to obtain the multi-scale semantic features of the second text.

[0098] Here, the second text can be text entered by the user in real time (e.g., the client responds to the user's input and sends the user-inputted text to the server), text retrieved from a database or local storage based on the user's selection, or text automatically retrieved from the database by the server. Furthermore, this application embodiment does not limit the type of the second text. For example, the second text can be long text such as articles or blogs (i.e., text with more than a certain number of characters), or short text such as comments or product descriptions (i.e., text with fewer than a certain number of characters).

[0099] In some embodiments, before performing multi-scale encoding on the second word vector sequence corresponding to the second text, the following processing may also be performed: performing word segmentation on the second text, and combining the words obtained from the word segmentation to obtain the second word sequence corresponding to the second text; performing word embedding on the second word sequence to obtain the word vector corresponding to each word in the second word sequence, and combining multiple word vectors to obtain the second word vector sequence corresponding to the second text.

[0100] It should be noted that the specific implementation process of step 102 is similar to that of step 101. Step 102 can be implemented with reference to the description of step 101. The embodiments of this application will not be described again here.

[0101] In some embodiments, the scale semantic features of the second text may include semantic features at L scales. This can be achieved by performing multi-scale encoding on the second word vector sequence corresponding to the second text, as described above, to obtain the multi-scale semantic features of the second text: performing concatenated encoding on the second word vector sequence corresponding to the second text to sequentially obtain the semantic features at L scales of the second text; where L is an integer greater than or equal to 2, and the window lengths used for encoding at different scales are different (for example, the window length used for the first scale encoding is 2, the window length used for the second scale encoding is 4, and the window length used for the third scale encoding is 8).

[0102] In other embodiments, when the multi-scale semantic features of the second text include L scale semantic features, then it can be achieved through... Figure 4 Steps 1021 to 1022 shown are implemented Figure 3 Step 102 shown will combine Figure 4 The steps shown are explained.

[0103] In step 1021, iteration i performs the following processing: the output of the (i-1)th encoding network is sampled by sliding the window corresponding to the i-th encoding network according to the set step size, and the sampling result is encoded to obtain the encoding result of the output of the i-th encoding network.

[0104] In step 1022, the encoding results output by each of the L encoding networks are determined as the semantic features of the second text at L scales.

[0105] It should be noted that the specific implementation process of steps 1021 to 1022 is similar to that of steps 1011 to 1012. Steps 1021 and 1022 can be executed with reference to the description of steps 1011 to 1012. The embodiments of this application will not be described again here.

[0106] In step 103, a similar semantic matrix is ​​constructed based on the multi-scale semantic features of the first text and the multi-scale semantic features of the second text.

[0107] In some embodiments, Figure 3 Step 103 shown can be achieved through Figure 5 Steps 1031 to 1033 shown are implemented, and will be combined with Figure 5 The steps shown are explained.

[0108] In step 1031, semantic features of the same scale in the first and second texts are multiplied together.

[0109] In some embodiments, after obtaining the multi-scale semantic features of the first text and the second text, the semantic features of the same scale in the first text and the second text can be multiplied. For example, assuming that the multi-scale semantic features of the first text and the second text both include semantic features of 3 scales, then the semantic features of the first text at the first scale are multiplied by the semantic features of the second text at the first scale, the semantic features of the first text at the second scale are multiplied by the semantic features of the second text at the second scale, and the semantic features of the first text at the third scale are multiplied by the semantic features of the second text at the third scale.

[0110] In step 1032, the multiplication results corresponding to multiple scales are combined to obtain a multi-scale similarity semantic matrix.

[0111] In some embodiments, after multiplying semantic features of the same scale in the first text and the second text, the multiplication results corresponding to multiple scales can be combined to obtain multi-scale similar semantic features. For example, if the multi-scale semantic features of the first text and the second text both include semantic features of 3 scales, after performing matrix multiplication on the semantic features of the same scale in the first text and the second text, the matrix multiplication results corresponding to the 3 scales can be combined to obtain a 3-scale similar semantic matrix between the first text and the second text.

[0112] In step 1033, the similar semantic matrix of each scale in the multi-scale similar semantic matrix is ​​spliced ​​according to the channel to obtain the similar semantic matrix.

[0113] In some embodiments, after constructing the multi-scale similarity semantic matrix, the similarity semantic matrices of each scale in the constructed multi-scale semantic matrix can be concatenated according to channels to obtain the final similarity semantic matrix. This similarity semantic matrix can be a three-dimensional matrix; for example, assuming that the dimension of the similarity semantic matrix at each scale is 1. Then, by concatenating the similarity semantic matrices of L scales according to channels, we can obtain a matrix with dimension L. The three-dimensional matrix is ​​used to determine the semantic similarity between the first and second texts in the final construction.

[0114] In step 104, feature extraction is performed on the similar semantic matrix to obtain similar semantic features.

[0115] In some embodiments, a convolutional neural network can be invoked to perform feature extraction processing on the similarity semantic matrix to obtain similar semantic features, for example, using the similarity semantic matrix as... Taking a three-dimensional matrix as an example, feature extraction can be performed on the three-dimensional matrix to obtain similar semantic features in the following way: For each layer in the three-dimensional matrix (for example, the three-dimensional matrix can be split into L layers, where the dimension of the similar semantic matrix corresponding to each layer is 1), For the similar semantic matrix corresponding to the layer, perform the following processing: input the similar semantic matrix corresponding to the layer into the convolutional neural network and obtain the feature information output by the convolutional neural network. After obtaining the feature information corresponding to the L layers respectively (i.e., L feature information, each feature information is obtained by the convolutional neural network to extract features from the similar semantic matrix of the corresponding layer), the feature information corresponding to the L layers respectively can be concatenated (i.e., the L feature information is concatenated) to obtain the final similar semantic features.

[0116] It should be noted that in practical applications, the convolutional neural network described above can be a convolutional neural network containing multi-scale convolutional layers. Each scale of the convolutional layer has a different window length (i.e., a different number of vectors are input to the convolution kernel each time; the convolution kernel is the parameter matrix, also known as a filter). There can be one or more convolutional layers at each scale, and each scale's convolutional layer can be used to perform convolution calculations on the input vector sequence based on the window length corresponding to that scale. Each convolutional layer can contain any number of convolutional kernels. For each convolutional layer, after performing convolution calculations on the input vector sequence, it can output another vector sequence. Since each multi-scale convolutional layer consists of multiple scales, each multi-scale convolutional layer can output multiple vector sequences. Subsequently, pooling can be performed on each of these multiple vector sequences, and the pooling results can be concatenated to obtain the final feature information output by the convolutional neural network. In this way, the correlation information between two texts can be deeply mined, further improving the accuracy of text matching.

[0117] In step 105, the matching degree between the first text and the second text is determined based on similar semantic features.

[0118] In some embodiments, the above-mentioned matching degree between the first text and the second text based on similar semantic features can be achieved by performing fully connected processing on the similar semantic features to obtain the fully connected processing result; calling the trained classifier to perform matching degree prediction processing on the fully connected processing result to obtain the matching degree between the first text and the second text.

[0119] For example, the matching degree mentioned above can refer to the similarity between the first word sequence corresponding to the first text and the second word sequence corresponding to the second text in terms of word vector features such as word meaning, part of speech, and word frequency. The matching degree can be a numerical range, such as 0-1, 0-10, etc., or it can be a percentage range, such as 0-100%.

[0120] In other embodiments, after determining the matching degree between the first text and the second text, a text matching identifier can be further generated based on the determined matching degree. The text matching identifier is used to represent the matching degree between the first text and the second text. For example, when the matching degree between the first text and the second text is greater than the matching degree threshold, the value of the text matching identifier can be 1, which means that the first text and the second text are similar texts; when the matching degree between the first text and the second text is less than the matching degree threshold, the value of the text matching identifier can be 0, which means that the first text and the second text are different texts.

[0121] The AI-based text processing method provided in this application can accurately extract richer semantic information from the text by performing multi-scale encoding on the word vector sequence corresponding to the text to be matched. Then, a similar semantic matrix is ​​constructed based on the multi-scale semantic features of the text, and feature extraction processing is performed on the similar semantic matrix to obtain similar semantic features between the texts. Finally, the similar semantic features are used to determine whether the two texts match. In this way, the richer semantic information contained in the multi-scale semantic features can better perform text matching, thereby effectively improving the accuracy of text matching.

[0122] The following example, using a WeChat official account subscription scenario, illustrates an exemplary application of this application's embodiments in a real-world application scenario.

[0123] In daily life, users can subscribe to multiple public accounts that they are interested in through instant messaging clients. However, some articles published by the multiple public accounts that a user has subscribed to may be duplicated. Therefore, the server needs to perform deduplication before pushing a list of articles published by multiple public accounts to the client.

[0124] However, most text matching methods provided by related technologies use CNN or RNN networks to extract semantic features from short texts (such as the titles of WeChat official account articles). Generally, the output of the last layer of the network (i.e., single-scale semantic features) is taken as the semantic features of the article title, and semantic similarity (e.g., Euclidean distance, cosine distance, etc.) is calculated based on these features to determine whether two articles match. However, such methods, using only the output of the last layer of the network as the semantic features of the article title, cannot accurately represent the information of the entire article title, thus affecting the subsequent deduplication effect.

[0125] In view of this, the AI-based text processing method provided in this application introduces the multi-scale concept from the visual domain into the classification of short texts (such as the titles of WeChat official account articles). Simultaneously, it mines the multi-scale semantic features of the WeChat official account article titles, constructs a multi-scale similarity semantic matrix based on the mined multi-scale semantic features, and uses a convolutional neural network to perform similarity semantic mining on the constructed multi-scale similarity semantic matrix. Subsequently, it determines whether two WeChat official account article titles match based on the mined similarity semantic features. Thus, by mining the multi-scale semantic features of WeChat official account article titles, which contain richer textual information than the single-scale semantic features in related technologies, the multi-scale semantic features are more conducive to matching WeChat official account articles, thereby effectively improving the accuracy of WeChat official account article title matching and enhancing the subsequent deduplication effect.

[0126] The AI-based text processing method provided in this application can also be applied to short text datasets such as comments. Each comment is a short text dataset. The task is to determine whether there is a matching short text dataset given a given short text dataset (i.e., given a comment, determine whether there is a matching comment). The short text dataset has the following format and structure: Figure 6 As shown, for example, for the comment "The cake looks pretty, it was delivered on time, it wasn't particularly good or bad, the experience was just average!", the matching comment is "I just received the birthday cake I ordered, the delivery guy delivered it on time, but the taste was just average."

[0127] The AI-based text processing method provided in this application can more fully represent the semantic information of short texts by extracting multi-scale semantic features. During the short text matching process, a short text similarity semantic matrix with stronger matching signals can be constructed. Subsequently, the short text similarity semantic feature information is extracted through the similarity semantic matrix, thereby improving the short text matching accuracy.

[0128] The following is a detailed description of the artificial intelligence-based text processing method provided in the embodiments of this application.

[0129] For example, see Figure 7 , Figure 7 This is a flowchart illustrating the artificial intelligence-based text processing method provided in this application embodiment, such as... Figure 7 As shown, the AI-based text processing method provided in this application can be divided into three steps: short text multi-scale semantic feature extraction, short text multi-scale similarity semantic matrix construction, and short text similarity semantic feature mining. In the short text multi-scale semantic feature extraction step, an LSTM network can be used to extract multi-scale semantic features from two short texts to be matched. These multi-scale semantic features contain richer semantic information about the short texts. In the short text multi-scale similarity semantic matrix construction step, a multi-scale similarity semantic matrix is ​​constructed from the multi-scale semantic features extracted from the previous two short texts. In the short text similarity semantic feature mining step, a CNN network is used to extract the similar semantic features contained in the short text multi-scale similarity semantic matrix, and then the short text matching is determined based on these similar semantic features. The following is a detailed explanation of the above three steps.

[0130] See also Figure 7 In the short text multi-scale semantic feature extraction step, firstly, short text A and short text B are obtained from the dataset. Then, short text A is segmented into words, and the multiple words (e.g., n words) obtained from the segmentation are combined to obtain the corresponding word sequence W_a={wa_1, wa_2, ..., wa_n}. Subsequently, each word in the word sequence W_a is fed into the Google pre-trained word vector model Word2Vec for quantization, and the multiple quantized word vectors are combined to obtain the word vector sequence W_av corresponding to short text A. The word vector sequence W_av corresponding to short text A is then fed into an L-layer LSTM network (including LSTM-1, LSTM-2, ..., LSTM-L). The encoding result lstm_a_i output by the i-th layer LSTM network is used as the semantic feature of short text A at the i-th scale. Finally, the semantic features Fa_L = {lstm_a_1, lstm_a_2, ..., lstm_a_L} at the L scales of short text A can be obtained. Similarly, short text B is processed in the same way as short text A, and the semantic features Fb_L = {lstm_b_1, lstm_b_2, ..., lstm_b_L} at the L scales of short text B can be obtained.

[0131] In the step of constructing a multi-scale similarity semantic matrix for short texts, for the semantic features Fa_L at L scales of short text A, the dimension of the semantic feature lstm_a_i at each scale is... (Where n is the number of sequences and D is the feature dimension), similarly, for the L-scale semantic features Fb_L of short text B, the dimension of the semantic feature lstm_b_i at each scale is also... Then, the semantic features of the corresponding scales of the two short texts can be multiplied by matrix to obtain a similar semantic matrix S_ab={s_1, s_2, ..., s_L} with L scales, where s_i=lstm_a_i*lstm_b_i. This completes the construction of the multi-scale similar semantic matrix of the short texts.

[0132] In the short text similarity semantic feature mining step, the similarity semantic matrices at each scale in the previously constructed multi-scale similarity semantic matrix S_ab are first concatenated along the channel dimension. The concatenated similarity semantic matrix is ​​a single... The specific process of splicing the three-dimensional matrix is ​​as follows: After obtaining L similar semantic matrices of different scales, where each similar semantic matrix has a dimension of 1... Therefore, we can directly set the L dimensions as The similar semantic matrices are stacked together to form a The three-dimensional matrix is ​​obtained, and then a multi-scale CNN network is used to extract features from the similar semantic matrix to obtain similar semantic features (mvec). Finally, the similar semantic features are fed into a fully connected layer (FC) and a classification layer (e.g., a softmax layer) for short text matching and recognition.

[0133] For example, see Figure 8 , Figure 8 This is a schematic diagram of the fully connected layer provided in the embodiments of this application, such as... Figure 8 As shown, the obtained similar semantic features (mvec) are input into the fully connected layer. Then, the output (fc_out) of the fully connected layer is non-linearly transformed by a Softmax layer into Y=f(Wx+b) and output. That is, the Softmax layer transforms the output of the fully connected layer into the probability of whether the short text matches or not, where Wx+b are the parameters in the Softmax layer, W is the weight matrix, b is a bias constant, and f is... Figure 8 The nodes in the equation represent activation functions, as shown in the following formula:

[0134]

[0135] in, Z j For the output of the fully connected layer, This represents the probability of whether a short text matches or not. When the probability value is greater than the probability threshold, short text A and short text B can be considered to match, that is, short text A and short text B are similar texts.

[0136] The AI-based text processing method provided in this application introduces the multi-scale concept from the visual domain into short text matching, thereby obtaining richer semantic information from the short text. Based on this richer multi-scale semantic information, short text matching can be performed better, improving the accuracy of short text matching. This effectively improves the similarity accuracy of WeChat official account article titles, and further enhances the subsequent deduplication effect.

[0137] The following continues to describe the exemplary structure of the artificial intelligence-based text processing device 243 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software modules stored in the AI-based text processing device 243 in the memory 240 may include: an encoding module 2431, a construction module 2432, an extraction module 2433, and a determination module 2434.

[0138] Encoding module 2431 is used to perform multi-scale encoding processing on the first word vector sequence corresponding to the first text to obtain the multi-scale semantic features of the first text; encoding module 2431 is also used to perform multi-scale encoding processing on the second word vector sequence corresponding to the second text to obtain the multi-scale semantic features of the second text; construction module 2432 is used to construct a similar semantic matrix based on the multi-scale semantic features of the first text and the multi-scale semantic features of the second text; extraction module 2433 is used to perform feature extraction processing on the similar semantic matrix to obtain similar semantic features; determination module 2434 is used to determine the matching degree between the first text and the second text based on the similar semantic features.

[0139] In some embodiments, the encoding module 2431 is further configured to perform cascaded encoding processing at L scales based on the first word vector sequence corresponding to the first text, thereby obtaining the semantic features of the first text at L scales in sequence; wherein, L is an integer greater than or equal to 2, and the length of the sliding window used for encoding processing at different scales is different.

[0140] In some embodiments, the encoding module 2431 is further configured to iteratively perform the following processing on i: slide sampling the output of the (i-1)th encoding network according to a set step size through the sliding window corresponding to the i-th encoding network, and encode the sampling result to obtain the encoding result output by the i-th encoding network; wherein, when the value of i is 1, the input of the i-th encoding network is the first word vector sequence corresponding to the first text; when the value of i is 2≤i≤L, the input of the i-th encoding network is the encoding result output by the (i-1)th encoding network; the determining module is further configured to determine the encoding results output by each of the L encoding networks as the semantic features of the first text at L scales.

[0141] In some embodiments, the encoding module 2431 is further configured to concatenate the output of the (i-1)th encoding network and the state corresponding to the (i-1)th encoding network to obtain a concatenation result; to slide sample the concatenation result according to a set step size through the sliding window corresponding to the i-th encoding network, and to encode the sampled result to obtain the encoding result output by the i-th encoding network.

[0142] In some embodiments, the AI-based text processing device 243 further includes a word segmentation module 2435, which is used to perform word segmentation processing on the first text and combine the words obtained from the word segmentation processing to obtain a first word sequence corresponding to the first text; the AI-based text processing device 243 further includes a word embedding module 2436, which is used to perform word embedding processing on the first word sequence to obtain word vectors corresponding to each word in the first word sequence and combine multiple word vectors to obtain a first word vector sequence corresponding to the first text.

[0143] In some embodiments, the AI-based text processing device 243 further includes a part-of-speech tagging module 2437, which is used to perform part-of-speech tagging on the words obtained by word segmentation of the first text; and to combine the words in the first text that meet the part-of-speech conditions to obtain the first word sequence corresponding to the first text.

[0144] In some embodiments, the encoding module 2431 is further configured to perform cascaded encoding processing at L scales based on the second word vector sequence corresponding to the second text, thereby obtaining the semantic features of the second text at L scales in sequence; wherein, L is an integer greater than or equal to 2, and the length of the sliding window used for encoding processing at different scales is different.

[0145] In some embodiments, the encoding module 2431 is further configured to iteratively perform the following processing on i: slide sampling the output of the (i-1)th encoding network according to a set step size through the sliding window corresponding to the i-th encoding network, and encode the sampling result to obtain the encoding result output by the i-th encoding network; wherein, when the value of i is 1, the input of the i-th encoding network is the second word vector sequence corresponding to the second text; when the value of i is 2≤i≤L, the input of the i-th encoding network is the encoding result output by the (i-1)th encoding network; the determining module is further configured to determine the encoding results output by each of the L encoding networks as the semantic features of the second text at L scales.

[0146] In some embodiments, the encoding module 2431 is further configured to concatenate the output of the (i-1)th encoding network and the state corresponding to the (i-1)th encoding network to obtain a concatenation result; to slide sample the concatenation result according to a set step size through the sliding window corresponding to the i-th encoding network, and to encode the sampled result to obtain the encoding result output by the i-th encoding network.

[0147] In some embodiments, the construction module 2432 is further configured to multiply semantic features of the same scale in the first text and the second text; combine the multiplication results corresponding to multiple scales respectively to obtain a multi-scale similar semantic matrix; and concatenate the similar semantic matrix of each scale in the multi-scale similar semantic matrix according to the channel to obtain a similar semantic matrix.

[0148] In some embodiments, the determining module 2434 is further configured to perform fully connected processing on similar semantic features to obtain a fully connected processing result; and to call a trained classifier to perform matching degree prediction processing on the fully connected processing result to obtain the matching degree between the first text and the second text.

[0149] It should be noted that the description of the device in the embodiments of this application is similar to the implementation of the AI-based text processing method described above, and has similar beneficial effects, therefore it will not be repeated. For any technical details not covered in the AI-based text processing device provided in the embodiments of this application, please refer to... Figures 3-5 ,or Figure 7 The meaning is understood in accordance with the description of any of the accompanying drawings.

[0150] This application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the artificial intelligence-based text processing method described above in this application.

[0151] This application provides a computer-readable storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to execute the artificial intelligence-based text processing method provided in this application. For example, ... Figures 3-5 ,or Figure 7 The text processing method based on artificial intelligence is shown.

[0152] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0153] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0154] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0155] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0156] In summary, this application embodiment, by performing multi-scale encoding on the word vector sequence corresponding to the text to be matched, can accurately extract richer semantic information from the text. Then, a similar semantic matrix is ​​constructed based on the multi-scale semantic features of the text, and feature extraction processing is performed on the similar semantic matrix to obtain similar semantic features between texts. Finally, a judgment on whether two texts match is made based on the similar semantic features. In this way, the richer semantic information contained in the multi-scale semantic features can better perform text matching, thereby effectively improving the accuracy of text matching.

[0157] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. An artificial intelligence-based text processing method, characterized by, The method comprises: based on the first text corresponding to the first word vector sequence, through the sliding window corresponding to the i th coding network in the L coding networks, the output of the i-1 th coding network is slidingly sampled, and the sampling result is subjected to L scale coding processing of concatenation to obtain the coding result of the output of the i th coding network, and the lengths of the sliding windows adopted by the coding processing of different scales are different; wherein L is an integer greater than or equal to 2, the value of i is 2≤i≤L, and determining the coding results output by the L coding networks respectively as L scale semantic features of the first text; based on the second text corresponding to the second word vector sequence, through the sliding window corresponding to the i th coding network in the L coding networks, the output of the i-1 th coding network is slidingly sampled, and the sampling result is subjected to L scale coding processing of concatenation to obtain the coding result of the output of the i th coding network, and the lengths of the sliding windows adopted by the coding processing of different scales are different, and determining the coding results output by the L coding networks respectively as L scale semantic features of the second text; multiplying the semantic features of the same scale in the first text and the second text, combining the multiplication results corresponding to multiple scales respectively, and obtaining a multi-scale similar semantic matrix; pasting each scale of the similar semantic matrix in the multi-scale similar semantic matrix according to the channel to obtain a similar semantic matrix; performing feature extraction processing on the similar semantic matrix to obtain a similar semantic feature; based on the similar semantic feature, determining the matching degree between the first text and the second text.

2. The method of claim 1, wherein, The coding result output by the i th coding network in the L coding networks through the sliding window corresponding to the i th coding network, the output of the i-1 th coding network is slidingly sampled, and the sampling result is subjected to L scale coding processing of concatenation, comprising: iterating i to perform the following processing: the output of the i-1 th coding network is slidingly sampled according to a set step length through the sliding window corresponding to the i th coding network, and the sampling result is subjected to coding processing to obtain the coding result output by the i th coding network; wherein when the value of i is 1, the input of the i th coding network is the first word vector sequence corresponding to the first text; when the value of i is 2≤i≤L, the input of the i th coding network is the coding result output by the i-1 th coding network.

3. The method of claim 2, wherein, The coding result output by the i th coding network in the L coding networks through the sliding window corresponding to the i th coding network, the output of the i-1 th coding network is slidingly sampled, and the sampling result is subjected to L scale coding processing of concatenation, comprising: pasting the output of the i-1 th coding network and the state corresponding to the i-1 th coding network to obtain a pasting result; slidingly sampling the pasting result according to a set step length through the sliding window corresponding to the i th coding network, and coding processing the sampling result to obtain the coding result output by the i th coding network.

4. The method of claim 1, wherein, Before the sliding sampling of the output of the i-1th encoding network by the sliding window corresponding to the i th encoding network in the L encoding networks, the method further comprises: performing part-of-speech tagging processing on the words obtained by the part-of-speech processing of the first text, and combining the words to obtain a first word sequence corresponding to the first text; performing word embedding processing on the first word sequence to obtain a word vector corresponding to each word in the first word sequence, and combining a plurality of word vectors to obtain a first word vector sequence corresponding to the first text.

5. The method of claim 4, wherein, The combining of the words obtained by the part-of-speech processing of the first text to obtain a first word sequence corresponding to the first text comprises: performing part-of-speech tagging processing on the words obtained by the part-of-speech processing of the first text, and combining the words to obtain a first word sequence corresponding to the first text; The combining of the words obtained by the part-of-speech processing of the first text to obtain a first word sequence corresponding to the first text comprises:

6. The method of claim 1, wherein, performing part-of-speech tagging processing on the words obtained by the part-of-speech processing of the first text, and combining the words to obtain a first word sequence corresponding to the first text; The sliding sampling of the output of the i-1th encoding network by the sliding window corresponding to the i th encoding network and the encoding processing of the sampling result to obtain the encoding result output by the i th encoding network comprises:

7. The method of claim 6, wherein, iteratively performing the following processing: sliding sampling of the output of the i-1th encoding network by the sliding window corresponding to the i th encoding network according to a set step size, and encoding processing of the sampling result to obtain the encoding result output by the i th encoding network; wherein, when the value of i is 1, the input of the i th encoding network is a second word vector sequence corresponding to the second text; when the value of i is 2≤i≤L, the input of the i th encoding network is the encoding result output by the i-1th encoding network. The sliding sampling of the output of the i-1th encoding network by the sliding window corresponding to the i th encoding network and the encoding processing of the sampling result to obtain the encoding result output by the i th encoding network comprises: concatenating the output of the i-1th encoding network and the state corresponding to the i-1th encoding network to obtain a concatenation result; 8. The method of claim 1, wherein, sliding sampling of the concatenation result by the sliding window corresponding to the i th encoding network according to a set step size, and encoding processing of the sampling result to obtain the encoding result output by the i th encoding network. The determination of the matching degree between the first text and the second text based on the similar semantic features comprises: fully connecting the similar semantic features to obtain a fully connected processing result; 9. An artificial intelligence-based text processing apparatus, characterized by comprising: calling a trained classifier to perform matching degree prediction processing on the fully connected processing result to obtain the matching degree between the first text and the second text. The device comprises: The encoding module is configured to: based on a first word vector sequence corresponding to the first text, perform sliding sampling on an output of an (i-1)th encoding network in the L encoding networks by using a sliding window corresponding to an ith encoding network, perform L-scale encoding processing in cascade on a sampling result, and obtain an encoding result output by the ith encoding network, wherein lengths of the sliding windows used in the encoding processing of different scales are different; L is an integer greater than or equal to 2, i is 2≤i≤L, and the encoding results output by the L encoding networks are determined as L-scale semantic features of the first text. The encoding module is further configured to: based on a second word vector sequence corresponding to the second text, perform sliding sampling on an output of an (i-1)th encoding network in the L encoding networks by using a sliding window corresponding to an ith encoding network, perform L-scale encoding processing in cascade on a sampling result, and obtain an encoding result output by the ith encoding network, wherein lengths of the sliding windows used in the encoding processing of different scales are different, and the encoding results output by the L encoding networks are determined as L-scale semantic features of the second text. The construction module is configured to: multiply the same-scale semantic features in the first text and the second text, combine multiplication results corresponding to multiple scales respectively, and obtain a multi-scale similar semantic matrix; and perform channel splicing processing on each-scale similar semantic matrix in the multi-scale similar semantic matrix, and obtain a similar semantic matrix. The extraction module is configured to: perform feature extraction processing on the similar semantic matrix, and obtain a similar semantic feature. The determination module is configured to: determine a matching degree between the first text and the second text based on the similar semantic feature.

10. The apparatus of claim 9, wherein, The encoding module is further configured to: perform the following processing by iteration i: perform sliding sampling on an output of an (i-1)th encoding network by using a sliding window corresponding to an ith encoding network according to a set step length, and perform encoding processing on a sampling result, to obtain an encoding result output by the ith encoding network; when i is 1, an input of the ith encoding network is a first word vector sequence corresponding to the first text; when i is 2≤i≤L, the input of the ith encoding network is the encoding result output by the (i-1)th encoding network.

11. The apparatus of claim 10, wherein, The encoding module is further configured to: perform splicing processing on the output of the (i-1)th encoding network and a state corresponding to the (i-1)th encoding network, to obtain a splicing result; perform sliding sampling on the splicing result by using a sliding window corresponding to the ith encoding network according to a set step length, and perform encoding processing on a sampling result, to obtain the encoding result output by the ith encoding network.

12. The apparatus of claim 9, wherein, The segmentation module is further configured to: perform segmentation processing on the first text, and combine words obtained by the segmentation processing, to obtain a first word sequence corresponding to the first text. The embedding module is configured to: perform word embedding processing on the first word sequence, to obtain a word vector corresponding to each word in the first word sequence, and combine multiple word vectors, to obtain a first word vector sequence corresponding to the first text.

13. The apparatus of claim 12, wherein, The word nature marking module is further configured to: perform word nature marking processing on the words obtained by performing the word segmentation processing on the first text.

14. The apparatus of claim 9, wherein, The encoding module is further configured to: perform the following processing in the i-th iteration: slidingly sampling the output of the (i-1)-th encoding network according to a set step length by using a sliding window corresponding to the i-th encoding network, and performing encoding processing on the sampling result to obtain an encoding result output by the i-th encoding network; when the value of i is 1, the input of the i-th encoding network is the second word vector sequence corresponding to the second text; when the value of i is 2≤i≤L, the input of the i-th encoding network is the encoding result output by the (i-1)-th encoding network.

15. The apparatus of claim 14, wherein, The encoding module is further configured to: perform concatenation processing on the output of the (i-1)-th encoding network and a state corresponding to the (i-1)-th encoding network to obtain a concatenation result; and slidingly sample the concatenation result according to a set step length by using a sliding window corresponding to the i-th encoding network, and perform encoding processing on the sampling result to obtain an encoding result output by the i-th encoding network.

16. The apparatus of claim 9, wherein, The determination module is further configured to: perform full connection processing on the similar semantic features to obtain a full connection processing result; and call a trained classifier to perform matching degree prediction processing on the full connection processing result to obtain a matching degree between the first text and the second text.

17. An electronic device, comprising: The electronic device comprises: a memory configured to store executable instructions; a processor configured to execute the executable instructions stored in the memory to implement the artificial intelligence-based text processing method in any one of claims 1 to 8.

18. A computer-readable storage medium storing executable instructions, wherein the instructions, when executed by a processor, cause the processor to perform operations comprising: The executable instructions, when executed by the processor, implement the artificial intelligence-based text processing method in any one of claims 1 to 8.

19. A computer program product comprising computer programs or instructions, characterized in that, The computer program or instructions, when executed by the processor, implement the artificial intelligence-based text processing method in any one of claims 1 to 8.

Images