Text processing method and device, equipment and medium

Abstract

The invention provides a text processing method and device, equipment and a medium, and relates to the technical field of artificial intelligence, and can be applied to scenes such as cloud technology, artificial intelligence, smart traffic and aided driving, the method comprises the following steps: obtaining a text word sequence of a to-be-processed text, the text word sequence comprising a plurality of text segmented words corresponding to the to-be-processed text; semantic feature extraction is conducted on the text word sequence, deep semantic features and at least one middle semantic feature are obtained, and the depth of a network layer outputting the deep semantic features is larger than that of a network layer outputting the middle semantic features; based on the text word segmentation granularity, fusing the deep semantic feature and the at least one intermediate semantic feature to obtain a target text feature, the target text feature comprising word fusion features corresponding to a plurality of text word segments; and screening out target segmented words of the to-be-processed text from the plurality of text segmented words according to the target text features. The accuracy of target word screening can be remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a text processing method, device, equipment and medium. Background Art

[0002] Natural language processing is an important direction in the fields of computer science and artificial intelligence. It combines algorithms to complete the analysis of text, thereby strengthening the understanding of the semantic information of the article. It is one of the core tasks in various scenarios such as search and recommendation. Extracting entity information from text based on target words is an important basis for understanding the relevant text content information and is used to represent the main core content of the article. In related technologies, text is usually characterized by semantic extraction models (refer to Figure 3 ), and then screen words based on the feature vector output by the model output layer to identify the target word. However, this method can only focus on the semantic vector of the text, and the accuracy and comprehensiveness of the text meaning expression are low, which has an adverse effect on the target word recognition effect. Summary of the invention

[0003] The present application provides a text processing method, apparatus, device and medium, which can significantly improve the accuracy of target word screening of text.

[0004] In one aspect, the present application provides a text processing method, the method comprising:

[0005] Acquire a text word sequence of a text to be processed, wherein the text word sequence includes a plurality of text segmentations corresponding to the text to be processed;

[0006] Extracting semantic features from the text word sequence to obtain a deep semantic feature and at least one intermediate semantic feature, wherein the depth of a network layer that outputs the deep semantic feature is greater than the depth of a network layer that outputs the intermediate semantic feature;

[0007] Based on the text segmentation granularity, the deep semantic feature and the at least one intermediate semantic feature are fused to obtain a target text feature, wherein the target text feature includes word fusion features corresponding to each of the multiple text segmentations;

[0008] The target word segment of the text to be processed is selected from the multiple text word segments according to the target text feature.

[0009] Another aspect provides a text processing device, the device comprising:

[0010] An acquisition module: used for acquiring a text word sequence of a text to be processed, wherein the text word sequence includes a plurality of text segmentations corresponding to the text to be processed;

[0011] Feature extraction module: used for extracting semantic features from the text word sequence to obtain deep semantic features and at least one intermediate semantic feature, wherein the depth of the network layer outputting the deep semantic features is greater than the depth of the network layer outputting the intermediate semantic features;

[0012] Feature fusion module: used for fusing the deep semantic feature and the at least one intermediate semantic feature based on the text segmentation granularity to obtain a target text feature, wherein the target text feature includes word fusion features corresponding to each of the multiple text segmentations;

[0013] The word segmentation screening module is used to screen out the target word segmentation of the text to be processed from the multiple text segmentations according to the target text features.

[0014] On the other hand, a computer device is provided, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the text processing method as described above.

[0015] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored in the storage medium, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the text processing method as described above.

[0016] On the other hand, a server is provided, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the text processing method as described above.

[0017] On the other hand, a terminal is provided, which includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the text processing method as described above.

[0018] On the other hand, a computer program product or a computer program is provided, which includes computer instructions, and when the computer instructions are executed by a processor, the text processing method as described above is implemented.

[0019] The text processing method, apparatus, device, storage medium, server, terminal, computer program and computer program product provided by this application have the following technical effects:

[0020] The technical solution of the present application obtains a text word sequence of a text to be processed, and the text word sequence includes multiple text segmentations corresponding to the text to be processed; performs semantic feature extraction on the text word sequence to obtain deep semantic features and at least one intermediate semantic feature, and the depth of the network layer that outputs the deep semantic features is greater than the depth of the network layer that outputs the intermediate semantic features, so as to construct a modeling framework for feature expression by combining the deep abstract semantic vector of the model and the shallow semantic vector, and simultaneously obtains information such as the semantic vector, grammatical features, and part-of-speech structure of the text, so as to improve the comprehensiveness and accuracy of text representation, thereby improving the accuracy of subsequent word classification. Then, based on the granularity of the text segmentation, the deep semantic features and at least one intermediate semantic feature are fused to obtain the target text feature, and the target text feature includes the word fusion features corresponding to each of the multiple text segmentations; the target segmentation of the text to be processed is selected from the multiple text segmentations according to the target text feature; in this way, the information interaction between features at different levels is strengthened through feature fusion, and the information expression and classification accuracy of the text feature are further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 is a schematic diagram of an application environment provided by an embodiment of the present application;

[0023] Figure 2 It is a flowchart of a text processing method provided in an embodiment of the present application;

[0024] Figure 3 It is a principle framework diagram of a text processing method provided by the prior art;

[0025] Figure 4 It is a flowchart of another text processing method provided in an embodiment of the present application;

[0026] Figure 5 It is a flowchart of another text processing method provided in an embodiment of the present application;

[0027] Figure 6 It is a flowchart of another text processing method provided in an embodiment of the present application;

[0028] Figure 7 It is a principle framework diagram of a text processing method provided in an embodiment of the present application;

[0029] Figure 8is a schematic diagram of a framework of a text processing device provided in an embodiment of the present application;

[0030] Fig. 9 It is a hardware structure block diagram of an electronic device for executing a text processing method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server comprising a series of steps or submodules is not necessarily limited to those steps or submodules clearly listed, but may include other steps or submodules that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] In related technologies, parsing text based on text content combined with basic algorithms such as natural language processing to enhance the understanding of article semantic information is one of the core tasks of search systems and other task scenarios. For example, extracting entity information of articles based on titles is an important basis for understanding relevant text content information. Among them, the entity recognition task is to extract entity information in sentences and characterize the main components of the core information of the subject in the title. In recent years, deep pre-training models have become the mainstream method of deep modeling. For example, semantics are extracted based on pre-trained BERT and entity lists are obtained based on classification networks. The modeling method is generally as follows: Figure 3As shown, after the model inputs the text, the feature vector corresponding to the token of the deepest layer - the output layer of BERT is taken to represent the semantic vector of each character in the text, and the vector is output to the classifier to make the final classification judgment. The above classification process is a typical application of the large-scale semantic model BERT in various tasks, but the architecture basically only focuses on deep abstract semantic vectors and cannot express other basic information of the sentence itself, such as sentence expression, part of speech structure, etc., resulting in poor accuracy of the classification results. In view of this, in order to solve at least one of the above problems, the present application provides a text processing method, which performs classification prediction based at least on multi-layer feature extraction and feature fusion.

[0034] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0035] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0036] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0037] Natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language people use in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question answering, knowledge graph and other technologies.

[0038] In recent years, with the research and progress of artificial intelligence technology, artificial intelligence technology has been widely used in many fields. The solution provided in the embodiment of the present application involves artificial intelligence machine learning / deep learning, computer vision technology, natural language processing and other technologies, which are specifically explained through the following embodiments.

[0039] See also Figure 1 , Figure 1 is a schematic diagram of an application environment provided by an embodiment of the present application, such as Figure 1 As shown, the application environment may include at least a terminal 01 and a server 02. In practical applications, the terminal 01 and the server 02 may be directly or indirectly connected via wired or wireless communication, which is not limited in this application.

[0040] The server 02 in the embodiment of the present application can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.

[0041] Specifically, cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or a local area network to achieve data computing, storage, processing, and sharing. Cloud technology can be applied to various fields, such as medical cloud, cloud Internet of Things, cloud security, cloud education, cloud conferencing, artificial intelligence cloud services, cloud applications, cloud calls, and cloud social networking. Cloud technology is based on the cloud computing business model application. It distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space, and information services as needed. The network that provides resources is called a "cloud". The resources in the "cloud" are infinitely expandable in the eyes of users, and can be obtained at any time, used on demand, expanded at any time, and paid for by use. As a basic capability provider for cloud computing, a cloud computing resource pool (referred to as a cloud platform, generally referred to as an IaaS (Infrastructure as a Service)) platform will be established, and various types of virtual resources will be deployed in the resource pool for external customers to choose to use. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices.

[0042] According to the logical function division, the PaaS (Platform as a Service) layer can be deployed on the IaaS layer, and the SaaS (Software as a Service) layer can be deployed on the PaaS layer, or SaaS can be directly deployed on IaaS. PaaS is a platform for software operation, such as databases, web containers, etc. SaaS is a variety of business software, such as web portals, SMS mass senders, etc. Generally speaking, SaaS and PaaS are upper layers relative to IaaS.

[0043] Specifically, the server 02 mentioned above may include a physical device, which may specifically include a network communication submodule, a processor, a memory, etc., and may also include software running in the physical device, which may specifically include an application program, etc.

[0044] Specifically, terminal 01 may include physical devices such as smart phones, desktop computers, tablet computers, laptops, digital assistants, augmented reality (AR) / virtual reality (VR) devices, intelligent voice interaction devices, smart home appliances, smart wearable devices, and vehicle-mounted terminal devices, and may also include software running in physical devices, such as applications.

[0045] In the embodiment of the present application, the terminal 01 can be used to receive text input or text submission to obtain the text to be processed, and send it to the server 02, so that the server 02 performs operations such as text word sequence generation, semantic feature extraction, feature fusion and target word screening on the text to be processed. The server 02 can also provide a model training service for the text processing model. Alternatively, the terminal 01 itself can deploy a text processing model to directly perform related target word screening and other processing.

[0046] Specifically, the text processing method of the present application can be used to extract text entity information, that is, to express text entity information through target word segmentation screening to understand the core content of the sentence text. It can be applied to all scenarios that require understanding the core words and themes of the text, such as general scenarios such as title understanding and paragraph sentence understanding.

[0047] Furthermore, it is understandable that Figure 1 What is shown is merely an application environment of a text processing method, and the application environment may include more or fewer nodes, and this application does not impose any limitation thereto.

[0048] The application environment involved in the embodiment of the present application, or the terminal 01 and the server 02 in the application environment, etc., can be a distributed system formed by connecting a client and multiple nodes (any form of computing devices in the access network, such as a server, a user terminal) through network communication. The distributed system can be a blockchain system, which can provide the above-mentioned text processing services, model training services and related data storage services.

[0049] The following is an introduction to the technical solution of this application based on the above application environment. The embodiments of this application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc. Please refer to Figure 2 , Figure 2 It is a flowchart of a text processing method provided by an embodiment of the present application. This specification provides method operation steps such as the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or server product is executed, it can be executed in the order of the method shown in the embodiment or the accompanying drawings or in parallel (for example, in a parallel processor or multi-threaded processing environment). Specifically, as Figure 2 As shown, the method may include the following steps S201-S207:

[0050] S201: Obtain a text word sequence of a text to be processed.

[0051] Specifically, the text to be processed can be a text obtained after preprocessing the text to be screened for target word segmentation; the preprocessing is used to process the text into a format that can be input into a text processing model, such as sentence segmentation, etc., to convert a long text into a short sentence, and the sentence segmentation can be performed with the punctuation position in the text as the sentence segmentation position to obtain one or more text sentences. Exemplarily, "This Lu Ban is hopeless, the economy is suppressed, and it can't get up at all, and the mobile phone is for you to play with!" After sentence segmentation, four phrases are obtained, "This Lu Ban is hopeless", "The economy is suppressed", "It can't get up at all", "The mobile phone is for you to play with", and a separator (such as "\n") is added between each phrase to indicate a line break, as an input for a subsequent text processing model. In this way, the input text length is reduced by segmenting the sentence and the semantic integrity is ensured, the text understanding effect of the model is improved, the text processing accuracy is improved, and the original sentence is connected with a separator and input as a whole, the sentence interaction length is increased, the semantic context extraction is realized, and the feature expression accuracy is improved.

[0052] Specifically, the text word sequence is obtained by performing word segmentation processing on the text to be processed, and the text word sequence includes multiple text word segments corresponding to the text to be processed. In some embodiments, each text word segment is a text character in the text to be processed, and accordingly, S201 includes S301: performing word segmentation processing on the text to be processed based on the word granularity, taking each text character as a single text word segmentation, and obtaining a text word sequence. Exemplarily, after the word segmentation processing of the text to be processed "This Lu Ban is hopeless\nThe economy is suppressed\nIt can't get up at all\nThe mobile phone is for you to play with" is obtained, that is, each word is a text word segmentation, so as to reduce the input complexity of the subsequent semantic extraction network and the word classification network, and reduce the network parameter amount.

[0053] In some other embodiments, the number of text characters in a single text segmentation may exceed one, and accordingly, S201 may include S401-S405:

[0054] S401: Perform word segmentation processing on the text to be processed based on the character granularity, taking each text character as a single first text segmentation to obtain a first word sequence.

[0055] Specifically, the first word sequence and S401 are similar to the aforementioned S301, and are not elaborated here.

[0056] S403: Perform word segmentation processing on each text sentence of the to-be-processed text based on semantics to obtain a second text segmentation corresponding to each text sentence.

[0057] S405: splicing the second text word segments into a second word sequence based on the position sequence of the text to be processed.

[0058] Specifically, at least part of the second text segmentation includes two or more text characters. The text to be processed is segmented at the character granularity and the semantic word granularity respectively to obtain two word sequences, the first word sequence is used to represent the text meaning information of the minimum granularity of the text to be processed, and the second word sequence is used to represent the text meaning information with continuous semantic expression. The text to be processed can be matched through a pre-stored segmentation dictionary to use the matched segmentation as the second text segmentation. Further, the word segmentation dictionary can be classified according to part of speech, such as distinguishing between content words and function words, or further refine the content word category into nouns, pronouns, etc., and refine the function word category into adverbs, prepositions, conjunctions, auxiliary words, etc. In the word segmentation process, the text words matched as function words in the to-be-processed text are segmented into word granularity to obtain a second text segmentation including only a single text word, and the text words matched as content words retain their granularity in the word segmentation dictionary to obtain a second text segmentation including two or more text words, and then the second text segmentations are spliced ​​into a second word sequence according to the position order of each second text segmentation in the to-be-processed text. Exemplarily, the second word sequence corresponding to "this Lu Ban is hopeless\nThe economy is suppressed\nIt can't get up at all\nThe mobile phone is for you to play" is "this\Lu Ban\is hopeless\nThe economy\is\suppressed\nIt can't get up at all\nThe mobile phone\is\for\you\to play". In this way, a second word sequence is formed through the second text segmentation of two or more text characters to serve as a semantic supplement to the first word sequence, thereby improving the semantic completeness and accuracy of subsequent feature expressions and further optimizing word recognition results.

[0059] S203: Extract semantic features from the text word sequence to obtain a deep semantic feature and at least one intermediate semantic feature.

[0060] Specifically, the depth of the network layer that outputs deep semantic features is greater than the depth of the network layer that outputs intermediate semantic features; deep semantic features can be features output by the output layer of the semantic extraction network, that is, features output by the deepest network layer, and intermediate semantic features are intermediate features generated during the process of interactive feature extraction of each text segment in the text word sequence. The number of intermediate semantic features is preferably two or more, so as to further obtain intermediate features at different levels, realize the expression of multiple shallow text structures, part-of-speech structures, etc., and enhance the vertical dimension of text feature information expression.

[0061] In some embodiments, reference Figure 4 , S203 may include S501-S505:

[0062] S501: Inputting a text word sequence into a semantic extraction network of a text processing model to extract semantic features, and obtaining semantic features corresponding to each of multiple network layers of the semantic extraction network;

[0063] S503: Determine the semantic features corresponding to the output network layer in the multiple network layers as the deep semantic features;

[0064] S505: Extracting semantic features corresponding to each network layer except the output network layer in the multiple network layers to obtain at least one intermediate semantic feature.

[0065] Specifically, a text processing model is used to perform target word segmentation screening on the text to be processed. The text processing model can specifically include a semantic extraction network, a feature fusion module and a word classification network. The output layer of the text processing model takes the text to be processed as input for word segmentation processing and outputs a text word sequence as the input of the semantic extraction network. The semantic extraction network includes multiple network layers connected in sequence. The multiple network layers include an output network layer (i.e., the last network layer), an input network layer (the first network layer) and an intermediate network layer, which are divided into a shallow network layer, a middle network layer and a deep network layer, and can respectively extract features of different depths, including shallow semantic features, middle semantic features and deep semantic features. Each network layer can output the corresponding semantic features after interactive extraction and input them into another network layer. The semantic features output by the output network layer are determined as deep semantic features, and the outputs of other intermediate network layers are determined as intermediate semantic features. In this way, the shallow network semantics of the model pays more attention to the sentence structure expression of the sentence, and the deep network pays more attention to the abstract semantics of the sentence. By combining the deep semantic features of the output network layer and the semantic features of other shallow networks, different layers of vectors are used to represent the grammatical and semantic information, thereby modeling the feature framework of subsequent classification tasks, and obtaining the semantic vectors of the text sentences and the grammatical information of the word segmentation. The model that combines grammatical and semantic information can better represent the core information in the sentence in the experiment, thereby improving the extraction accuracy and better expressing the semantic integrity, especially effectively improving the semantic expression accuracy of complex sentence structures.

[0066] The semantic extraction network can be constructed based on a large-scale semantic vector model. In some embodiments, the semantic extraction network can be constructed based on a BERT (Bidirectional Encoder Representations from Transformers) model. The BERT model can be a pre-trained model obtained by pre-training using text predictions similar to the target text, or a pre-trained model obtained by pre-training based on other Chinese / English corpora. The network layer of the semantic extraction network is constructed using Transformer, which performs bidirectional interaction and understanding on the input text word sequence, more accurately understands the association between word segments, and improves the degree of semantic understanding. The text word sequence is first input into the Embedding network of the semantic extraction network, and word embedding, position embedding and type embedding processing are performed. The word embedding processing is used to represent the text segmentation as an initial feature vector, the position embedding is used to vectorize the position sequence of the text segmentation in the text to be processed, and the type embedding processing can be used to vectorize the type of the text segmentation. In one embodiment, the type embedding process is only used to characterize the text segmentation as text or label, and in another embodiment, the type embedding process is also used to vectorize the part of speech (such as content word, function word, or noun, verb, auxiliary word, etc.) of the second text segmentation matched in the second word sequence. In this way, the former method can simplify the model parameters and training process, and the latter method, although increasing the model parameters, can further reduce the interference of non-key words and improve the accuracy of word screening by combining the part of speech information.

[0067] In one embodiment, starting from the input network layer, the semantic features output by each intermediate network layer can be extracted every preset number of layers to obtain the intermediate semantic features. Exemplarily, the preset number of layers can be three layers, that is, the intermediate semantic features are extracted every three layers.

[0068] S205: Based on the text segmentation granularity, the deep semantic feature and at least one intermediate semantic feature are integrated to obtain the target text feature.

[0069] Specifically, the target text feature includes word fusion features corresponding to multiple text segmentations. Fusion here refers to spatial mapping of deep semantic features and at least one intermediate semantic feature to map them to the same semantic space and output a feature. Granular fusion based on text segmentation means that for each text segmentation, the corresponding feature vectors in the deep semantic features and intermediate semantic features are fused to obtain the word fusion features of each text segmentation, and then the target text feature is generated based on the word fusion features of each text segmentation.

[0070] Specifically, the deep semantic features include the deep word features corresponding to each of the multiple text segmentations, and the intermediate semantic features include the intermediate word features corresponding to each of the multiple text segmentations; accordingly, reference Figure 5 , S205 includes S601-S603:

[0071] S601: for each of the multiple text segmentations, based on the tensor fusion method, the deep word feature corresponding to the single text segmentation and the intermediate word features are fused to obtain a word fusion feature;

[0072] S603: Generate target text features based on word fusion features corresponding to each of the multiple text segmentations.

[0073] Specifically, the deep semantic features can be formed by concatenating the deep word features corresponding to multiple text segmentations, and the intermediate semantic features can be formed by concatenating the intermediate word features corresponding to multiple text segmentations. The deep word features and intermediate word features belonging to the same text segmentation are first fused into word fusion features through the tensor fusion method, and then the word fusion features are concatenated into target text features based on the position order of multiple text segmentations in the text to be processed. Tensor fusion is implemented based on vector bilinear multiplication, which promotes the multiplication between vectors to second-order interaction. In this way, text segmentation is used as the fusion granularity and the mechanism of multi-vector tensor fusion is introduced to strengthen the feature interaction of segmentation features in different layers, further improving the accuracy of feature expression of each segmentation.

[0074] It can be understood that if there is an intermediate semantic feature, a single text segmentation involves the fusion of deep word features and an intermediate word feature, and a tensor fusion can be directly performed. The expression used for tensor fusion is as follows, where T token,k Characterizes the word fusion feature of the kth text segmentation among multiple text segmentations (n ​​in total), T 1k represents the intermediate semantic features of the kth text segmentation, T 2k represents the deep semantic features of the kth text segmentation, W [1:m] is a tensor matrix with dimensions of a*c*m, where m is the number of neurons in the feature fusion module and also the output dimension of the module. The parameters of the tensor matrix constitute the feature fusion space, mapping the two features to the fusion space. 1k The dimension is d1*1, T 2k The dimension of is d2*1, the dimension of the fused word fusion feature is m, b is the bias information, and the dimension is also m, which is used to make the model more robust. The matrix parameters of the tensor matrix and the bias information b are adjusted and fixed through model training.

[0075] T token,k =T 1k *W [1:m] *T 2k +b

[0076] In other cases, two or more intermediate semantic features may be included. Accordingly, S601 may include S6011-S6013:

[0077] S6011: performing feature combinations of deep word features and intermediate word features corresponding to a single text segmentation word in pairs to obtain at least one feature group;

[0078] S6012: Perform feature fusion on each feature group based on the tensor fusion method to obtain the intermediate fusion features corresponding to each feature group;

[0079] S6013: Concatenate the intermediate fusion features corresponding to the single text word segmentation to obtain the word fusion feature.

[0080] Specifically, the deep semantic features and the intermediate semantic features can be combined in pairs, and the combination result can be used as the word feature combination result of each text segmentation to obtain at least one feature group to simplify the operation of word feature combination. The above-mentioned two-by-two combination specifically refers to arbitrarily extracting two different features from the deep semantic features and the intermediate semantic features to form a pair of features as a feature group. Exemplarily, when there are two intermediate semantic features and one deep semantic feature, there are three feature combination situations, and each text segmentation corresponds to three feature groups.

[0081] Specifically, tensor fusion is performed on the word feature pairs in each feature group to obtain an intermediate fusion feature corresponding to the text segmentation, and then the intermediate fusion features of the text segmentation are spliced ​​to form a word fusion feature. In this way, the fusion of different layers of features is performed by combining them in pairs, further strengthening the feature interaction between different layers, and simplifying the neuron complexity of the tensor layer in the feature fusion module. Multiple word features can be fused by simple neuron stacking, thereby improving training efficiency.

[0082] In one embodiment, the expression of the tensor fusion method used for each feature group is shown in the following formula, where T ik is the i-th word feature of the k-th text segmentation in multiple text segmentations (which can be a deep word feature or an intermediate word feature), T jk is the jth word feature of the kth text segmentation in multiple text segmentations. i and j represent the index of the network layer extracted in the semantic extraction network. For example, if there are 12 network layers in total, the 4th, 8th and 12th network layers are extracted, and the index numbers are 1, 2 and 3 respectively. If i is 1, T ik It is the intermediate word feature of the kth text segmentation output by the 3rd network layer.

[0083] T token,k =T ik *W [1:m] *T jk+b

[0084] In some embodiments, S6012 includes: for each feature group, using a tensor matrix to perform semantic space mapping processing on two intermediate word features in the feature group, or to perform semantic space mapping processing on the deep word features and the intermediate word features in the feature group, so as to map the two intermediate word features to the same semantic space, or to map the deep word features and the intermediate word features to the same semantic space, and obtain the intermediate fusion features corresponding to the feature group. The same semantic space mapped to above can be a tensor matrix (such as W [1:m] ) is based on the parameter composition of the tensor matrix. The fusion of two feature vectors through the matrix can flexibly adjust the feature output dimension and fusion space parameters, improve the scene adaptability and generalization of feature fusion, and thus improve the practicability of the text processing method in different scenarios.

[0085] S207: Filter out target word segments of the text to be processed from multiple text word segments according to the target text features.

[0086] In text analysis tasks such as entity extraction, simply relying on the deepest sentence semantic vector modeling often cannot better capture the core components of the sentence. The above technical solution combines the model's deep abstract semantic vector and shallow semantic vector to build a modeling framework for feature expression. It not only uses the model's deep abstract semantic vector but also integrates the shallow semantic vector to simultaneously obtain the text's semantic vector, grammatical features, part-of-speech structure and other information, improves the comprehensiveness and accuracy of text representation, and thus improves the accuracy of subsequent word classification. In addition, feature fusion strengthens the information interaction between features at different levels, further improving the information expression and classification accuracy of text features.

[0087] Specifically, the target text feature can express the semantics, grammatical structure and word structure information of each text segment in the text to be processed, and using it as the input of the word classification network can realize the category recognition of each text segment, and then determine the target segment according to the classification recognition result. Accordingly, S207 can include S701-S703:

[0088] S701: Performing word segmentation category recognition at a text word segmentation granularity according to target text features to obtain classification recognition results corresponding to each of a plurality of text word segments.

[0089] As mentioned above, the target text features include the word fusion features corresponding to each text segmentation, the target text features are input into the word classification network of the text processing model, and the category recognition is performed on each of the multiple text segmentations based on the word fusion features, so as to obtain the classification recognition results corresponding to each of the multiple text segmentations. In this way, the target text features fused with multi-layer feature vectors are used as the input of the classification network to realize the category recognition of each text segmentation and realize the efficient screening of the target segmentation. In one embodiment, the word classification network can be constructed based on the CRF (Conditional Random Field) model.

[0090] S703: Determine a target segmented word from multiple text segmented words according to the classification and recognition results.

[0091] Specifically, the classification recognition result can be used to indicate the word category of the text segmentation, and the word category can be set based on the application scenario, such as being used to indicate that the text segmentation is a keyword or a non-keyword, etc. In some embodiments, the classification recognition result is used to characterize the entity word category of the text segmentation, and the entity word category can correspond to two categories, including an entity word category or a non-entity word category, or correspond to multiple categories, such as indicating that the text segmentation is a non-entity word (i.e., a meaningless segmentation), an entity word starting word, an entity word middle word, or an entity word ending word, etc. Correspondingly, according to the classification recognition result corresponding to each text segmentation, the text segmentation whose entity word category is indicated as belonging to an entity word can be determined as the target segmentation, and the target segmentation can be spliced ​​according to the start, middle, and end order in combination with the aforementioned classification results to obtain the entity information of the text to be processed.

[0092] In some embodiments, each text segmentation in the text word sequence is a single text word, and accordingly, each word fusion feature corresponds to a single text word, and the classification recognition result represents the word category of the text word, such as indicating that the text segmentation is a non-entity word, an entity word starting word, an entity word middle word, or an entity word ending word, etc. The classification recognition result of each text segmentation is determined based on the prediction index data generated by the word classification network, and the prediction index data includes the confidence that the text word belongs to each preset category, and the preset category corresponding to the highest confidence is the category to which the text word belongs.

[0093] In another embodiment, the text word sequence includes a first word sequence and a second word sequence, and the first word sequence and the second word sequence are respectively input into the semantic extraction network for feature extraction to obtain a first deep semantic feature and at least one first intermediate semantic feature of the first word sequence, and a second deep semantic feature and at least one second intermediate semantic feature of the second word sequence; and then input into the feature fusion module to fuse the first deep semantic feature and at least one first intermediate semantic feature to obtain a first text feature corresponding to the first word sequence, and fuse the second deep semantic feature and at least one second intermediate semantic feature to obtain a second text feature corresponding to the second word sequence; and respectively input the first text feature and the second text feature into the word classification network for category recognition of the text word segmentation granularity to obtain the first prediction index data of each first text word segmentation and the second prediction index data of each second text word segmentation. It can be understood that if the second text word segmentation in the second word sequence is the same as the first text word segmentation at the same position in the first word sequence, or the second text word segmentation is composed of two or more adjacent first text word segments, the first prediction index data and the second prediction index data are fused at the text word granularity to obtain the fusion index data of each first text word segmentation, and then the classification recognition result of each first text word segmentation is generated based on the fusion index data.

[0094] Specifically, the indicator data fusion here can be an indicator summing and averaging process, such as simple summing and averaging or weighted summing and averaging. In the weighted summing and averaging, for the same text word, the weight of the corresponding first prediction indicator data is greater than the second prediction indicator data. For example, in a binary classification scenario, the confidence of the first prediction indicator data of the first text segmentation word "Lu" and "Ban" belonging to entity words is 76% and 46% respectively, while the confidence of the second prediction indicator data of the second text segmentation word "Lu Ban" belonging to entity words is 86%. After simple summing and averaging fusion, the confidence of "Lu" and "Ban" belonging to entity words is 81% and 66% respectively, and then both text words are identified as entity words. Combined with the second word sequence for semantically continuous word segmentation processing, and further used as auxiliary information in feature extraction, feature fusion and classification recognition, the text semantic understanding and expression accuracy are improved, thereby improving the accuracy of the classification results, and at the same time improving the robustness of the model.

[0095] In one embodiment, a BERT encoding model is used as a semantic extraction network. After the text word sequence of the text to be processed (taking the title as an example) is input and feature extracted and encoded, the deep semantic features are obtained by the output layer, and the intermediate semantic features are obtained by the intermediate network layer. The semantic features of the title can be expressed as BERT (title) = CLS, T0, T1...Tn. It can be understood that the word segmentation label information needs to be input during the model training process. Therefore, the semantic features also include the word feature CLS corresponding to the token (segmentation) of the label information. In addition, T0, T1...Tn represent the word features of each of the n+1 text tokens, which can be intermediate word features or deep word features. Figure 7 The BERT encoding model of this embodiment includes 12 network layers, and a semantic vector is extracted every 4 layers to obtain the first intermediate semantic feature (T 1,emb ), the second intermediate semantic feature (T 2,emb ) and deep semantic features (T 3,emb ) three semantic vectors, representing the semantic vectors expressed by the shallow network, the middle network, and the deep network respectively. The first intermediate semantic feature is output by the fourth network layer L4 (T 1,emb =Bert-L4(Sententce)=CLS1,T 1,0 ,T 1,1 ,…T 1,n ), the second intermediate semantic feature is output by the fourth network layer L8 (T 1,emb =Bert-L8(Sententce)=CLS2,T 2,0 ,T 2,1 ,…T 2,n ), the deep semantic features are output by the fourth network layer L12 (T 1,emb =Bert-L12(Sententce)=CLS3,T 3,0 ,T 3,1 ,…T 3,n ), where Bert-Li(Sentence) is the semantic feature corresponding to the title output by the i-th network layer. It can be understood that semantic extraction can also be performed every three layers or other number of layers, not limited to the above example.

[0096] Furthermore, the first intermediate semantic feature, the second intermediate semantic feature and the deep semantic feature are input into the feature fusion module for fusion. The example in the figure adopts three fusion submodules based on the tensor fusion mechanism, wherein each fusion submodule is responsible for the fusion of word feature vectors at corresponding positions between any two layers of semantic features, so as to realize the pairwise fusion of the three semantic features. Specifically, the two intermediate word features corresponding to each text segmentation in the first intermediate semantic feature and the second intermediate semantic feature are respectively tensor fused to form the first intermediate fusion feature (Ttoken1,k ), similarly, the first intermediate semantic feature and the deep semantic feature are fused into the second intermediate fusion feature (T token2,k ), the second intermediate semantic features and deep semantic features are fused into the third intermediate fusion features of each text segmentation (T token3,k ), the first intermediate fusion feature, the second intermediate fusion feature and the third intermediate fusion feature of the same text segmentation are concatenated into the word fusion feature of the text segmentation (T tokenfusion,k ), and then concatenate the word fusion features of each text segmentation into the target text features.

[0097] Specifically, the fusion expression of each fusion submodule can be T token,k =T ik *W [1:m] *T jk +b. More specifically, T token1,k =T 1k *W [1:m] *T 2k +b,T token2,k =T 1k *W [1:m] *T 3k +b,T token3,k =T 2k *W [1:m] *T 3k +b,T tokenfusion,k =[T token1,k :T token2,k :T token3,k ], where “:” represents splicing.

[0098] Furthermore, the above target text features (CLS, T tokenfusion,0 ,T tokenfusion,1 …, T tokenfusion,n ) Input the word classification network to obtain the classification recognition result of each text token. In this embodiment, multiple classifications are adopted, and the preset categories are other (O, representing non-entity words), begin (B, representing entity start words), middle (M, representing entity middle words), and end (E, representing entity end words). Exemplarily, the input text is "This novel written by the famous writer A was eventually moved to the screen by director X", and the classification recognition result is "O\O\O\O\O\O\O\B\O\O\B\E\nO\O\B\O\O\O\B\M\E\O\B\E\O", and the entity information composed of the target token is "A novel was moved to the screen by B".

[0099] The architecture of the deep model shows that the shallow network is more concerned with extracting the basic structure of the text, such as basic features such as part of speech and expression structure, which are reflected by intermediate semantic features, while the deep network is more concerned with extracting abstract features of the text, such as semantic expression, which is reflected by deep semantic features. Basic features are very important factors in text expression, especially when the structure is relatively complex. The above embodiment proposes an entity extraction model that integrates Bert multi-layer semantic vectors, strengthens the basic structural information and combines it with a tensor-based mechanism to strengthen the interaction between features, enriches the information dimension and enables information reinforcement to be more effective, significantly improving the accuracy of classification.

[0100] Based on some or all of the above implementations, the present application also provides a text processing model training method, which may specifically include:

[0101] S801: Acquire sample word sequences of multiple sample texts and label information of the sample texts, where the sample word sequences include multiple sample word segments, and the label information includes word labels corresponding to the multiple sample word segments.

[0102] Specifically, the sample text can be obtained based on an existing text corpus, similar to the aforementioned text to be processed, and the sample word segmentation is similar to the aforementioned text word segmentation, which will not be elaborated here. Each sample text corresponds to a label information, and the word label contained therein is used to mark the preset category of the sample word segmentation, such as any one of the aforementioned O, B, M, and E. In some cases, each sample word segmentation is a text word in a single sample text, and each text word corresponds to a word label. In other cases, in addition to the first sample word sequence composed of a single text word, the sample text also corresponds to a second sample word sequence. Similar to the aforementioned second text word sequence, at least part of the second sample word segmentations included in the second sample word sequence may include two or more text words, and the label information includes the word label of each first sample word segmentation in the first sample word sequence and the word label of each second sample word segmentation in the second sample word sequence. Exemplarily, the first sample word sequence of “This Lu Ban is hopeless” is “This / Lu / Ban / is / hopeless”, and the label information is “O / B / E / O / O / O”, and the second sample word sequence is “This / Lu Ban / is / hopeless”, and the label information is “O / B / O / O”.

[0103] S803: Input the sample word sequence into the semantic extraction network of the initial text model to extract semantic features, and obtain deep sample semantic features and at least one intermediate sample semantic feature, wherein the depth of the network layer that outputs the deep sample semantic features is greater than the depth of the network layer that outputs the intermediate sample semantic features.

[0104] S805: Through the feature fusion module of the initial text model, the deep sample semantic feature and at least one intermediate sample semantic feature are fused based on the text segmentation granularity to obtain a sample text feature, where the sample text feature includes sample word fusion features corresponding to each of the multiple sample segmentations.

[0105] It can be understood that S803 is similar to the aforementioned S203, S805 is similar to the aforementioned S205, and S807 is similar to the aforementioned S208.

[0106] The deep sample semantic features, intermediate sample semantic features, sample text features and sample word fusion features correspond to the aforementioned deep semantic features, intermediate semantic features, target text features and word fusion features, respectively, and will not be elaborated here.

[0107] S807: Input the sample text features into the word classification network of the initial text model to perform word segmentation category recognition at the text word segmentation granularity, and obtain category indicator data corresponding to each of the multiple sample word segments.

[0108] Specifically, the category indicator data is used to characterize the confidence level that the sample segmentation belongs to the word category, including the confidence level that the sample segmentation belongs to each preset category. For example, the confidence level that the sample segmentation "this" belongs to O, B, M, and E is 90%, 4%, 5%, and 1%, respectively, indicating that the preset category to which it belongs is "O-non-entity word".

[0109] S809: Determine the model loss according to the category index data corresponding to each of the multiple sample word segments and the word labels corresponding to each of the multiple sample word segments.

[0110] Specifically, the model loss of a single iteration is the sum of the losses corresponding to each sample text input in the current iteration, and the loss corresponding to each sample text is the sum of the loss values ​​of each sample word in each sample text. The loss value of a single sample word can be obtained by performing a cross entropy algorithm on the category indicator data of the sample word and its word label value. In one embodiment, the loss (Loss) of a single sample text can be calculated using the following formula, where n+1 is the number of sample word segments, k is the index of the sample word segment, m is the index of the word label, r is the number of preset categories, and y is the value of the word tag. km Indicates whether the kth sample word in the sample text belongs to the mth preset category, if yes, it is 1, otherwise 0; a km The confidence (probability) that the kth sample word in the sample text is predicted by the network as the mth preset category, such as the probability of predicting it as the starting word of an entity.

[0111]

[0112] S811: Train an initial text model based on the model loss to obtain a text processing model.

[0113] Specifically, the network parameters of the semantic extraction network, feature fusion module and word classification network, such as tensor matrix parameters, bias information b, etc., are adjusted based on the model loss of a single iteration to obtain an updated initial text model, and S803-S809 are repeatedly executed to perform sample text input and text processing for the next iteration, and then the network parameters are continuously updated in combination with the model loss of the next iteration, and this is repeated until the training end condition is met to obtain a text processing model. The training end condition can be met when the number of iterations reaches a preset number, or when the loss difference between the model losses of two adjacent iterations is less than or equal to the preset loss.

[0114] The present application embodiment also provides a text processing device 800, such as Figure 8 As shown, Figure 8 A structural schematic diagram of a text processing device provided in an embodiment of the present application is shown, and the device may include the following modules.

[0115] Acquisition module 10: used to acquire a text word sequence of a text to be processed, wherein the text word sequence includes a plurality of text segmentations corresponding to the text to be processed;

[0116] Feature extraction module 20: used to extract semantic features from a text word sequence to obtain a deep semantic feature and at least one intermediate semantic feature, wherein the depth of the network layer outputting the deep semantic feature is greater than the depth of the network layer outputting the intermediate semantic feature;

[0117] Feature fusion module 30: used to fuse the deep semantic feature and at least one intermediate semantic feature based on the text segmentation granularity to obtain a target text feature, where the target text feature includes word fusion features corresponding to each of the multiple text segmentations;

[0118] The word segmentation screening module 40 is used to screen out target word segments of the text to be processed from multiple text segmentations according to the target text features.

[0119] In some embodiments, the feature extraction module 20 includes:

[0120] Semantic extraction submodule: used to input the text word sequence into the semantic extraction network of the text processing model to extract semantic features, and obtain the semantic features corresponding to each of the multiple network layers of the semantic extraction network;

[0121] The first feature extraction submodule is used to determine the semantic features corresponding to the output network layer in the multiple network layers as the deep semantic features;

[0122] The second feature extraction submodule is used to extract the semantic features corresponding to each network layer in the multiple network layers except the output network layer to obtain at least one intermediate semantic feature.

[0123] In some embodiments, the deep semantic features include deep word features corresponding to each of the plurality of text segmentations, and the intermediate semantic features include intermediate word features corresponding to each of the plurality of text segmentations; and the second feature extraction submodule may include:

[0124] Fusion unit: used for fusing the deep word features and intermediate word features corresponding to each text segmentation in multiple text segmentations based on the tensor fusion method to obtain word fusion features;

[0125] Target feature generation unit: used to generate target text features based on word fusion features corresponding to multiple text segmentations.

[0126] In some embodiments, the fusion unit comprises:

[0127] Feature combination subunit: used to perform feature combination of deep word features and intermediate word features corresponding to a single text segmentation word in pairs to obtain at least one feature group;

[0128] Feature fusion subunit: used to fuse each feature group based on the tensor fusion method to obtain the intermediate fusion features corresponding to each feature group;

[0129] Splicing subunit: used to splice the intermediate fusion features corresponding to a single text word segmentation to obtain the word fusion feature.

[0130] In some embodiments, the feature fusion subunit can be specifically used for: for each feature group, using a tensor matrix to perform semantic space mapping processing on the two intermediate word features in the feature group, or to perform semantic space mapping processing on the deep word features and the intermediate word features in the feature group, so as to map the two intermediate word features to the same semantic space, or to map the deep word features and the intermediate word features to the same semantic space, and obtain the intermediate fusion features corresponding to the feature group.

[0131] In some embodiments, the acquisition module includes a first word segmentation submodule: configured to perform word segmentation processing on the text to be processed based on word granularity, taking each text word as a single text segmentation to obtain a text word sequence.

[0132] In some other embodiments, the text to be processed includes at least one text sentence, the text word sequence includes a first word sequence and a second word sequence, and the acquisition module includes:

[0133] The first word segmentation submodule is used to perform word segmentation processing on the text to be processed based on the word granularity, taking each text word as a single first text segmentation to obtain a first word sequence;

[0134] The second word segmentation submodule is used to perform word segmentation processing on each text sentence of the to-be-processed text based on semantics, and obtain a second text segmentation corresponding to each text sentence, wherein at least part of the second text segmentation includes two or more text characters;

[0135] Sequence splicing submodule: used for splicing the second text word segments into a second word sequence based on the position sequence of the text to be processed.

[0136] In some embodiments, the word segmentation and screening module includes:

[0137] Category recognition submodule: used to identify the word segmentation categories at the text segmentation granularity according to the target text features, and obtain the classification recognition results corresponding to multiple text segmentations;

[0138] Word segmentation determination submodule: used to determine the target word segmentation from multiple text word segmentations based on the classification recognition results.

[0139] In some embodiments, the category recognition submodule may be specifically used to: input the target text features into the word classification network of the text processing model, perform category recognition on each of the multiple text segmentations based on the fusion features of each word, obtain the classification recognition results corresponding to each of the multiple text segmentations, and use the classification recognition results to characterize the entity word category of the text segmentation;

[0140] Accordingly, the word segmentation determination submodule may be specifically used to determine a text word whose entity word category is indicated as belonging to an entity word as a target word segmentation.

[0141] In some embodiments, the apparatus further comprises:

[0142] Sample acquisition module: used to obtain sample word sequences of multiple sample texts and label information of the sample texts, where the sample word sequences include multiple sample word segments, and the label information includes word labels corresponding to each of the multiple sample word segments;

[0143] Sample feature extraction module: used for inputting the sample word sequence into the semantic extraction network of the initial text model to extract semantic features, and obtaining deep sample semantic features and at least one intermediate sample semantic feature, wherein the depth of the network layer outputting the deep sample semantic features is greater than the depth of the network layer outputting the intermediate sample semantic features;

[0144] Sample feature fusion module: used to obtain sample text features by fusing deep sample semantic features and at least one intermediate sample semantic feature based on the text segmentation granularity through the feature fusion module of the initial text model. The sample text features include sample word fusion features corresponding to each of the multiple sample segmentations.

[0145] Sample classification module: used to input sample text features into the word classification network of the initial text model to perform word segmentation category recognition at the text word segmentation granularity, and obtain category index data corresponding to each of multiple sample word segments. The category index data is used to characterize the confidence level that the sample word segment belongs to the word category;

[0146] Model loss determination module: used to determine the model loss according to the category index data corresponding to each of the multiple sample segmentations and the word labels corresponding to each of the multiple sample segmentations;

[0147] Training module: used to train the initial text model based on the model loss to obtain the text processing model.

[0148] It should be noted that the above device embodiment and method embodiment are based on the same implementation method.

[0149] An embodiment of the present application provides a device, which may be a terminal or a server, including a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement a text processing method or a neural network training method as provided in the above method embodiment.

[0150] The memory can be used to store software programs and modules. The processor executes various functional applications and abnormality detection by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, application programs required for functions, etc.; the data storage area can store data created according to the use of the device, etc. In addition, the memory can include a high-speed random access memory and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory can also include a memory controller to provide the processor with access to the memory.

[0151] The method embodiments provided in the embodiments of the present application can be executed in electronic devices such as mobile terminals, computer terminals, servers or similar computing devices. Fig. 9 1 is a hardware structure block diagram of an electronic device of a text processing method provided in an embodiment of the present application. Fig. 9As shown, the electronic device 900 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 910 (the processor 910 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 930 for storing data, and one or more storage media 920 (such as one or more mass storage devices) for storing application programs 923 or data 922. Among them, the memory 930 and the storage medium 920 can be short-term storage or permanent storage. The program stored in the storage medium 920 may include one or more modules, each of which may include a series of instruction operations in the electronic device. Furthermore, the central processing unit 910 can be configured to communicate with the storage medium 920 and execute a series of instruction operations in the storage medium 920 on the electronic device 900. The electronic device 900 may also include one or more power supplies 960, one or more wired or wireless network interfaces 950, one or more input and output interfaces 940, and / or, one or more operating systems 921, such as Windows Server TM , Mac OS X TM , Unix TM , LinuxTM, FreeBSDTM, etc.

[0152] The input / output interface 940 may be used to receive or send data via a network. The specific example of the network may include a wireless network provided by a communication provider of the electronic device 900. In one example, the input / output interface 940 includes a network adapter (Network Interface Controller, NIC), which may be connected to other network devices via a base station so as to communicate with the Internet. In one example, the input / output interface 940 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0153] It can be understood by those skilled in the art that Fig. 9 The structure shown is only for illustration and does not limit the structure of the above electronic device. Fig. 9 More or fewer components as shown, or with Fig. 9 Different configurations are shown.

[0154] An embodiment of the present application also provides a computer-readable storage medium, which can be set in an electronic device to store at least one instruction or at least one program related to an anomaly detection method in a method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the anomaly detection method provided by the above method embodiment.

[0155] Optionally, in this embodiment, the storage medium may be located in at least one of the multiple network servers of the computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0156] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above-mentioned various optional implementations.

[0157] The text processing method, device, equipment, storage medium, server, terminal and program product provided by the above-mentioned present application, the technical solution of the present application obtains the text word sequence of the text to be processed, the text word sequence includes multiple text segmentations corresponding to the text to be processed; the text word sequence is subjected to semantic feature extraction to obtain deep semantic features and at least one intermediate semantic feature, the depth of the network layer outputting the deep semantic feature is greater than the depth of the network layer outputting the intermediate semantic feature, so as to combine the deep abstract semantic vector of the model and the shallow semantic vector to construct a modeling framework for feature expression, and at the same time obtain the semantic vector, grammatical features, part of speech structure and other information of the text, so as to improve the comprehensiveness and accuracy of the text representation, thereby improving the accuracy of subsequent word classification. Then, based on the text segmentation granularity, the deep semantic features and at least one intermediate semantic feature are fused to obtain the target text feature, the target text feature includes the word fusion features corresponding to each of the multiple text segmentations; the target segmentation of the text to be processed is selected from the multiple text segmentations according to the target text feature; in this way, the information interaction between features at different levels is strengthened through feature fusion, and the information expression and classification accuracy of the text feature are further improved.

[0158] It should be noted that the above-mentioned sequence of the embodiments of the present application is for description only and does not represent the advantages and disadvantages of the embodiments. The above-mentioned specific embodiments of the present application are described. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0159] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0160] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by instructing the relevant hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0161] The above are only preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A text processing method, characterized in that: The method comprises: Acquire a text word sequence of a text to be processed, wherein the text word sequence includes a plurality of text segmentations corresponding to the text to be processed; Extracting semantic features from the text word sequence to obtain a deep semantic feature and at least one intermediate semantic feature, wherein the depth of a network layer that outputs the deep semantic feature is greater than the depth of a network layer that outputs the intermediate semantic feature; Based on the text segmentation granularity, the deep semantic feature and the at least one intermediate semantic feature are fused to obtain a target text feature, wherein the target text feature includes word fusion features corresponding to each of the multiple text segmentations; The target word segment of the text to be processed is selected from the multiple text word segments according to the target text feature.

2. The method according to claim 1, characterized in that The extracting of semantic features from the text word sequence to obtain a deep semantic feature and at least one intermediate semantic feature comprises: Inputting the text word sequence into the semantic extraction network of the text processing model to extract semantic features, and obtaining semantic features corresponding to each of the multiple network layers of the semantic extraction network; Determine the semantic feature corresponding to the output network layer among the multiple network layers as the deep semantic feature; The semantic features corresponding to each network layer in the multiple network layers except the output network layer are extracted to obtain the at least one intermediate semantic feature.

3. The method according to claim 1, characterized in that The deep semantic features include deep word features corresponding to each of the multiple text segmentations, and the intermediate semantic features include intermediate word features corresponding to each of the multiple text segmentations; The step of fusing the deep semantic feature and the at least one intermediate semantic feature based on the text segmentation granularity to obtain the target text feature comprises: For each of the multiple text segmentations, based on a tensor fusion method, a deep word feature corresponding to the single text segmentation and each intermediate word feature are fused to obtain the word fusion feature; The target text feature is generated based on the word fusion features corresponding to each of the multiple text segmentations.

4. The method according to claim 3, characterized in that The tensor fusion method is used to fuse the deep word features and the intermediate word features corresponding to the single text segmentation to obtain the word fusion features, including: Combining the deep word features and the intermediate word features corresponding to the single text segmentation words in pairs to obtain at least one feature group; Based on the tensor fusion method, feature fusion is performed on each of the feature groups to obtain intermediate fusion features corresponding to each of the feature groups; The intermediate fusion features corresponding to the single text word segmentation are concatenated to obtain the word fusion feature.

5. The method according to claim 4, characterized in that The tensor fusion method is used to fuse the features of each feature group to obtain the intermediate fusion features corresponding to each feature group, including: For each feature group, a tensor matrix is ​​used to perform semantic space mapping processing on the two intermediate word features in the feature group, or a semantic space mapping processing is performed on the deep word features and the intermediate word features in the feature group, so as to map the two intermediate word features to the same semantic space, or map the deep word features and the intermediate word features to the same semantic space, and obtain the intermediate fusion features corresponding to the feature group.

6. The method according to claim 1, characterized in that The text word sequence of the text to be processed is obtained by: The text to be processed is segmented based on the character granularity, and each text character is used as a single text segmentation to obtain the text word sequence.

7. The method according to claim 1, characterized in that The text to be processed includes at least one text sentence, the text word sequence includes a first word sequence and a second word sequence, and obtaining the text word sequence of the text to be processed includes: Perform word segmentation on the text to be processed based on word granularity, taking each text word as a single first text segmentation word, to obtain the first word sequence; Performing word segmentation processing on each of the text sentences of the to-be-processed text based on semantics to obtain second text segmentations corresponding to each of the text sentences, wherein at least part of the second text segmentations includes two or more text characters; Based on the position sequence of the text to be processed, each of the second text segmentations is spliced ​​into the second word sequence.

8. The method according to any one of claims 1 to 7, characterized in that The step of selecting a target word segment of the text to be processed from the plurality of text segment segments according to the target text feature comprises: Performing word segmentation category recognition at a text word segmentation granularity according to the target text features to obtain classification recognition results corresponding to each of the multiple text word segments; The target word segment is determined from the multiple text word segments according to the classification and recognition result.

9. The method according to claim 8, characterized in that The word segmentation category recognition of the text word segmentation granularity is performed according to the target text feature to obtain the classification recognition results corresponding to each of the multiple text word segments, including: Inputting the target text features into a word classification network of a text processing model, performing category recognition on each of the multiple text segmentations based on the word fusion features, and obtaining classification recognition results corresponding to each of the multiple text segmentations, wherein the classification recognition results are used to characterize the entity word category of the text segmentation; Determining the target segmented word from the multiple text segmented words according to the classification recognition result includes: The text segmentation that indicates the entity word category as belonging to the entity word is determined as the target segmentation.

10. The method according to any one of claims 1 to 7, characterized in that The method further comprises: Acquire sample word sequences of multiple sample texts and label information of the sample texts, wherein the sample word sequences include multiple sample word segments, and the label information includes word labels corresponding to each of the multiple sample word segments; Inputting the sample word sequence into the semantic extraction network of the initial text model to extract semantic features, obtaining a deep sample semantic feature and at least one intermediate sample semantic feature, wherein the depth of the network layer outputting the deep sample semantic feature is greater than the depth of the network layer outputting the intermediate sample semantic feature; By means of the feature fusion module of the initial text model, the deep sample semantic feature and the at least one intermediate sample semantic feature are fused based on the text segmentation granularity to obtain a sample text feature, wherein the sample text feature includes sample word fusion features corresponding to each of the multiple sample segmentations; Inputting the sample text features into the word classification network of the initial text model to perform word segmentation category recognition at the text word segmentation granularity, and obtaining category index data corresponding to each of the multiple sample word segments, wherein the category index data is used to characterize the confidence level that the sample word segment belongs to the word category; Determine the model loss according to the category index data corresponding to each of the multiple sample segmentations and the word labels corresponding to each of the multiple sample segmentations; The initial text model is trained based on the model loss to obtain a text processing model.

11. A text processing device, characterized in that: The device comprises: An acquisition module: used for acquiring a text word sequence of a text to be processed, wherein the text word sequence includes a plurality of text segmentations corresponding to the text to be processed; Feature extraction module: used for extracting semantic features from the text word sequence to obtain deep semantic features and at least one intermediate semantic feature, wherein the depth of the network layer outputting the deep semantic features is greater than the depth of the network layer outputting the intermediate semantic features; Feature fusion module: used for fusing the deep semantic feature and the at least one intermediate semantic feature based on the text segmentation granularity to obtain a target text feature, wherein the target text feature includes word fusion features corresponding to each of the multiple text segmentations; The word segmentation screening module is used to screen out the target word segmentation of the text to be processed from the multiple text segmentations according to the target text features.

12. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the text processing method according to any one of claims 1 to 10.

13. A computer device, characterized in that: The device includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the text processing method according to any one of claims 1 to 10.

Images